It’s raining chatbots! After OpenAI’s ChatGPT brought about a revolution, Google unveiled its BARD, and several others followed suit. Now it seems, Meta Platforms, Inc. is gearing up to have an edge over its peers. The California-based tech behemoth has introduced a new research tool that will soon aid in building AI-based chatbots.
The company has publicly released its Large Language Model Meta AI (LLaMA). According to the official release, LLaMA is a state-of-the-art foundational language model developed to assist researchers in their work in the subfield of AI. Interestingly, this would be Meta’s third LLM after Glactica and Blender Bot 3 that were shut down immediately following incorrect results.
What is LLaMA?
LLaMA is not essentially a chatbot; it is a research tool that, according to Meta, will likely solve issues concerning AI language models. “Smaller, more performant models such as LLaMA enable others in the research community who don’t have access to large amounts of infrastructure to study these models, further democratizing access in this important, fast-changing field,” said Meta in its official blog.
LLaMA is a collection of language models that range from 7B to 65B parameters. The company has said that it trains its models on trillions of tokens claiming that it is possible to train state-of-the-art models using public datasets and not relying on proprietary and inaccessible data sets.
How is LLaMA different?
According to Meta, training smaller foundational models such as LLaMA is ideal as they require significantly low computing power and resources to test, validate and explore new use cases. Foundational language models are known to train on large chunks of data that are unlabeled and this makes them ideal for customising according to various tasks. Meta has said that it will offer LLaMA in sizes such as 7B, 13B, 33B, and 65B parameters.
In its research paper, Meta noted that LLaMA-13B outperformed OpenAI’s GPT-3 (175B) on most benchmarks and LLaMA-65B is competitive with the best models, DeepMind’s Chinchilla70B and Google’s PaLM-540B. Once fully trained, LLaMA-13B can be a boon for small businesses that are looking forward to running tests on these systems, however, it may still be far from researchers working isolation.
LLaMA is currently not in use on any of Meta’s products, however, the company has plans to make it available to researchers. The company had earlier launched its LLM OPT-175B but LLaMA is its more advanced system. Meta has also made the LLaMA model source code available for outsiders to see how the system works. This will let them customise and collaborate on related projects.
Decoding Large Language Models
Large language models or LLMs are AI systems that consume massive volumes of digital text from internet sources such as articles, news reports, and social media posts. These digital texts are used to train software that predicts and produces content from scratch based on prompts and queries. These models can help in tasks such as writing essays, composing social media posts, suggesting programming code, and generating chatbot conversations.
The latest release from Meta comes at a time when the company was largely absent from the chatter surrounding the revolutionary AI chatbots. It had been one of the first to release its own chatbots. However, owing to the incorrect results and lacklustre response, its plans went awry. With LLaMa, Meta seems to have hurled itself back into the game.
