- Meta developing AI puzzles : Meta sees Llama and its family of generative artificial intelligence software as a Linux-like open source operating system that could compete with Microsoft Windows.
Mark Zuckerberg has focused the company’s attention on artificial intelligence, while maintaining a huge investment in Metaverse’s technology.
- The company’s bad reputation in Washington, DC, has prevented some agencies, such as the Government Accountability Office, from using its AI language model.
At last month’s annual Connect conference, virtual reality enthusiasts gathered to hear Mark Zuckerberg’s multibillion-dollar bet on the metaverse, a technology that will define the future of the enterprise.
But at this year’s event, VR developers were inundated with panels on the topic of artificial intelligence, which quickly became less about tomorrow and more about now. “Don’t tell Mark, but it seems to be less mixed reality and more artificial intelligence,” joked Joseph Spisak, who joined the company two months ago as director of product development for generative artificial intelligence, at the Connect conference. “It felt like an artificial intelligence conference that was kind of in my wheelhouse.”
Between panels on Meta’s latest Quest 3 VR headset and augmented reality development software, there were several sessions dedicated to Llama, Meta’s Large Language Model (LLM) that has been in use since OpenAI’s ChatGPT chatbot. Its popularity has grown since the outbreak in November, prompting a rush by leading technology companies to bring competitive products to market. Zuckerberg, who renamed Facebook Meta in late 2021 to show his commitment to the Metaverse, reminded Connect attendees that Llama is the latest digital assistant powerhouse the company unveiled at the conference.
While Zuckerberg still sees the development of new virtual worlds as critical to his company’s success, artificial intelligence has become the market he’s trying to conquer today. Meta sees Llama and its family of generative AI software as an open source alternative to GPT (Microsoft LLM)
- Support OpenAI and Google
PaLM 2 powered by search company Bard’s AI technology.
Industry experts compare Lama’s position in the field of generative artificial intelligence to Linux, an open source competitor to Microsoft Windows, in the PC operating system market. Just as Linux software entered enterprise servers around the world and became an important part of the modern Internet, Meta sees Llama as a potential digital scaffolding to power the next generation of artificial intelligence applications.
Camels are difficult to value on Wall Street and difficult to understand for many investors. Because AI researchers are expensive and the infrastructure needed to build and run models is expensive, Meta is investing heavily in building Llama (an updated version of Llama 2 launched in July) and related generative AI software.
After the announcement in July, Jan Lekun, an artificial intelligence researcher who Zuckerberg hired in 2013 to lead Facebook’s new AI research group, wrote on Twitter: “This will change the landscape of the LLM market.”
But open source means that Meta makes software free for developers, a radical departure from the traditional software licensing and subscription model and away from the highly profitable digital advertising business that made Facebook an online giant. When Meta announced Llama 2, Meta said the new version would have a commercial license that would allow companies to integrate it into their products. The company said it is not focused on directly monetizing Llama 2, but it did receive an undisclosed amount of funding from cloud computing companies such as Microsoft and Amazon.
, which provides access to Llama 2 as part of their own AI enterprise services. Zuckerberg said on the company’s second-quarter earnings call that he doesn’t expect Llama 2 to “generate a lot of revenue in the short term, but hopefully in the long term.”
Attract the best talent
The meta tries to take advantage of llamas in other ways.
Zuckerberg told analysts in July that improvements to Llama by third-party developers could lead to “efficiency gains” that would make it cheaper for Meta to run the artificial intelligence software. Meta said capital spending will be between $27 billion and $30 billion in 2023, down from $32 billion last year. Treasurer Susan Li said that number is likely to grow by 2024, thanks in part to investments related to data centers and artificial intelligence. Influence has its benefits. With the world’s leading AI researchers using Llama, Meta can more easily hire skilled engineers who understand the company’s development approach. Facebook has long used open-source projects, such as the PyTorch coding framework for machine learning applications, as recruiting tools to attract technologists who want to work on cutting-edge software projects.
At Meta from 2018 to January 2023, Spisak helped oversee PyTorch and other open source artificial intelligence projects. He left the company to work briefly at Google and returned to Meta in July.
The meta is also betting that third-party developers will continually improve Llama 2 and its associated AI software to make it more efficient, a way to outsource development to an army of volunteers. Cai GoGwilt, chief architect at legal tech startup Ironclad, said the open source community is working on the first version of Llama to “make it faster and work on mobile phones.” GoGwilt said his company is looking forward to seeing how enthusiastic developers will support Llama 2.
“Part of the reason we didn’t use it right away was because we were more interested in what the open source community was going to do with it,” GoGwilt said. Meta launched the original Llama LLM in February, offering several different variants ranging from 7 billion parameters to 65 billion parameters, which are essentially variables that affect the size of the model and the amount of data it can handle. In general, more parameters mean a more powerful model, but there is a trade-off in the cost of running and training the AI software. Like the OpenAI GPT and other LLMs, Llama is an example of Transformer Neural Networks, an artificial intelligence software developed by Google’s research team, which has become the basis for generative artificial intelligence that generates data based on simple text prompts. Smart responsive and smart. figurative language.
To help with the computationally intensive process of training giant AI models like Llama, Meta has used its Research SuperCluster supercomputer, which has up to 16,000 Nvidia processors.
A100 GPU, the “powerhouse” computer chip in the artificial intelligence industry. Although initially incubated in Meta’s Fundamental Artificial Intelligence Research (FAIR) group, Llama was later moved to the company’s generative AI organization, led by Ahmad Al-Dahle, who previously spent more than 16 years at Apple.
. Zuckerberg announced the organization in late February. Meta said Llama 2 was trained for six months, starting in January and ending in July, using “publicly available online data” that does not contain any Facebook user information. It is unclear if Meta plans to include user profiles in the upcoming Llama 3. In an effort to become more efficient, Zuckerberg has turned his attention to Nvidia, which generates billions of dollars in quarterly profits from its artificial intelligence chips. Meta is one of the biggest customers. Jim Fan, Nvidia’s senior AI researcher, said in a post about
The mainstream adoption of Llama 2 could affect Nvidia’s efforts to ensure that its graphics processing units (GPUs) work well with Meta-approved software, reducing the company’s AI training and computing costs. Meta, meanwhile, has its own in-house AI chip project, making it a potential replacement for Nvidia processors.
“It gives them some leverage to negotiate a price,” said Arjun Bansal, CEO of startup Log10 and a former head of artificial intelligence. “Nvidia wants to charge a lot, and they could say, ‘Hey, we’ve got our thing.’
Nathan Lambert remembers the energy that radiated from his colleagues at artificial intelligence startup Hugging Face when Meta launched its long-awaited Llama 2 over the weekend.
Lambert and his teammates worked overtime to ensure the company’s infrastructure was ready to handle the influx of programmers who wanted to try Llama 2. Like cloud computing engines Microsoft Azure and Amazon Web Services, Hugging Face is one of Meta’s chosen launch partners for Llama 2, but arguably the most important. Developers, AI researchers and thousands of companies use the Hugging Face platform to share code, datasets and models, making it one of the largest communities in the industry. Lambert said that while there are many open source LLMs to choose from, Llama 2 is the most popular.
“This is the model most people use, and this is the model most startups use,” Lampert said on Oct. 17. 4 He leaves Hugging Face, although he hasn’t revealed where.
Like everything Zuckerberg does, the project is not without controversy. Some in the industry believe that Meta’s license agreement for the use of Llama 2 is restrictive and against the spirit of collaborative development and innovation.
For example, if a third-party developer integrates Llama 2 into a product or service that had “more than 700 million monthly active users” in the month before its launch in July, it must request Meta’s approval to use Llama 2. Critics say the rule is a way to keep out competitors like Snap
Or TikTok, which uses Llama 2 for its service.
“It’s pretty strict,” said Umesh Padwal, a venture partner at Thomvest Ventures and an investor in artificial intelligence startup Cohere, which has created a proprietary LL.M. “Meta seems to want to bring all the benefits of open source to their company while staying away from the competition.”
Lambert said Meta can help the open source community and release details about the specific base dataset used to train Llama 2 so developers can better understand the training process. Open source advocates and privacy experts have pushed for more transparency about the types of materials used to train LL.M.s, but so far the company has released few details.
“We believe in open innovation and don’t want to place unnecessary restrictions on how others can use our models,” a Met spokesperson said in a statement. “But we want people to use it responsibly. This is a tailored commercial license that balances open access to the model with responsibilities and protections to help prevent potential abuse.”
Despite some critics, the Meta model has seen massive early adoption. Connect revealed that “Lama-based models using Hugging Face have been downloaded more than 30 million times, including more than 10 million in the last 30 days alone.”
An Nvidia fan noted in his X post that Llama 2’s new commercial license could encourage more companies to try the language model than the original Llama.
“Major AI researchers were wary of Llama-1 due to licensing issues, but now I think many of them will get on board and lend their firepower,” Fan wrote. According to a recent TC Cowen survey of 680 cloud computing companies, companies investing in artificial intelligence so far prefer to use commercial LLMs. The survey found that 32% of respondents already use or plan to use commercially packaged LLMs such as OpenAI’s GPT-4 software, while 28% focus on open source LLMs such as Llama and Falcon developed in the United Arab Emirates. Only 12% of respondents planned to use the internal LL.M.
The Meta Reputation Challenge
The U.S. Government Accountability Office is investigating Taka Ariga, how advanced technologies such as LL.M. can help the agency better conduct audits and investigations through the Innovation Lab.
By the end of this year, Ariga’s team plans to conduct the first experiment, studying how the LL.M. can be used to compile a large number of GAO reports and materials on a particular topic and then combine those documents with several other potentially relevant documents from other agencies. “The public or members of Congress may ask, ‘What is GAO doing in nuclear safety?'” about the LL.M. program, Arriaga said. “Obviously we do a lot of work, but that’s just the basis of each report; you can’t do this kind of topic search.”
GAO is currently using AWS Bedrock’s generative AI service to help the agency experiment with several popular LLMs, including proprietary models from startups like Cohere and Anthropic. Although AWS recently announced that Bedrock would soon support Llama 2, Ariga said GAO is first examining Anthropic’s Claude LLM and may decline to use Llama 2 because of the Met’s poor reputation in Washington.
Arriaga noted that Meta has drawn the ire of lawmakers over a range of issues over the years, including data privacy scandals, antitrust investigations and allegations that Facebook censored conservative voices, and compared Zuckerberg to Tesla CEO Elon Musk.
and the owner of X. “Mark Zuckerberg, like Elon, is a bit of a lightning rod in terms of political technology,” Arriaga said.
A Met spokesman said: “We know that while artificial intelligence brings huge advances to society, it also comes with risks.” “Meta is committed to building responsibly, and we provide a number of resources, such as our Responsible Use Guide, to help those using Llama 2.”
Even among potential customers who are not concerned about reputational issues, Meta has to prove that it has superior LLM technology.
Nur Hamdan, product manager at AI startup aiXplain, said OpenAI’s GPT-4 is better than Llama 2 for understanding the context of long, extended conversations. Hamadan said this means GPT-4 can potentially generate conversations in a more realistic way.
Tests comparing GPT-4, Llama 2 and other LL.M.s are becoming commonplace. In one such test, the researchers found that GPT-4 produced better software code than Llama 2. Since then, Meta has released a version of Llama 2 specifically designed for code generation. In today’s battle for market, Meta is competing with Amazon, Google and well-funded startups like OpenAI and Cohere. Each is designed to be a cornerstone of the next generation of applications. Meta sees open source as an important advantage over other companies that sell technology and bundle it with other services.
“Companies like Google or Microsoft can be a bit controversial,” said Guido Appenzeler, a longtime chief infrastructure technology officer who held a senior position at VMware.
. “Facebook isn’t, and that’s the way they’re going forward and democratizing and giving broad access to open source. I think that’s a very powerful thing.”
A Microsoft spokesman said by email that the company will let customers choose the model they want, whether it’s “proprietary, open source, or both.”
“Each base model has unique benefits, and we want to make it easy for customers to select, fine-tune, and deploy responsibly to maximize the results of these tools,” Microsoft said. A Google spokesperson said the company “has a long history of publicly contributing to open source projects” and is perhaps the biggest supporter of the broader artificial intelligence ecosystem.
Amazon representatives did not respond to requests for comment.
Experts say Lam’s impact on the tech industry is comparable to that of Kubernetes, Google’s 2014 open-source software release for data center infrastructure. By abandoning Kubernetes, Google has significantly affected the business models of once-hot startups like Docker and CoreOS (acquired by Red Hat in 2018).
Meta is pursuing a Kubernetes-like strategy with Llama 2, but the market is expected to grow larger. “I’m a Facebook fan and I understand what Mark is doing,” said Thomvest’s Padwal. “They are reinventing the business.”
However, open source doesn’t always win, and Padwal admits, “In this case, I don’t know how it’s going to end.”