Meta Releases Llama 4 AI Models with Improved Capabilities and Balance
Introduction
Meta has released a new collection of AI models, Llama 4, as part of its Llama family. The new models, including Llama 4 Scout, Llama 4 Maverick, and Llama 4 Behemoth, were trained on large amounts of unlabeled text, image, and video data to provide a broad visual understanding.
New Features and Architecture
Llama 4 is the first cohort of models to use a mixture of experts (MoE) architecture, which is more computationally efficient for training and answering queries. MoE architectures break down data processing tasks into subtasks and delegate them to smaller, specialized "expert" models. This architecture allows for more efficient processing and better performance.
Llama 4 Models
- Scout: Scout has 17 billion active parameters, 16 experts, and 109 billion total parameters. It has a very large context window of 10 million tokens, allowing it to process and work with extremely lengthy documents. Scout is best for tasks like document summarization and reasoning over large codebases.
- Maverick: Maverick has 400 billion total parameters, but only 17 billion active parameters across 128 "experts." It is best for "general assistant and chat" use cases like creative writing. According to Meta’s internal testing, Maverick exceeds models like OpenAI’s GPT-4o and Google’s Gemini 2.0 on certain coding, reasoning, multilingual, long-context, and image benchmarks.
- Behemoth: Behemoth has 288 billion active parameters, 16 experts, and nearly two trillion total parameters. It will require even beefier hardware to run. Meta’s internal benchmarking has Behemoth outperforming GPT-4.5, Claude 3.7 Sonnet, and Gemini 2.0 Pro on several evaluations measuring STEM skills like math problem solving.
Tuning and Balance
Meta has tuned all of its Llama 4 models to refuse to answer "contentious" questions less often. According to the company, Llama 4 responds to "debated" political and social topics that the previous crop of Llama models wouldn’t. Llama 4 is also "dramatically more balanced" with which prompts it flat-out won’t entertain.
Concerns about Bias
Some have raised concerns about the balance of AI chatbots, accusing them of being too politically "woke." However, bias in AI is an intractable technical problem. Meta’s Llama 4 models aim to provide helpful, factual responses without judgment, and the company is continuing to make Llama more responsive to answer more questions and respond to a variety of different viewpoints.
Availability and Licensing
Scout and Maverick are openly available on Llama.com and from Meta’s partners, including the AI dev platform Hugging Face. Behemoth is still in training. Users and companies "domiciled" or with a "principal place of business" in the EU are prohibited from using or distributing the models due to governance requirements imposed by the region’s AI and data privacy laws.
Conclusion
Meta’s Llama 4 models mark the beginning of a new era for the Llama ecosystem. With their improved capabilities and balance, these models are set to revolutionize the way we interact with AI. However, the debate about bias and balance in AI chatbots continues to rage on.
FAQs
Q: What is the Mixture of Experts (MoE) architecture?
A: The MoE architecture is a more computationally efficient way of training and answering queries by breaking down data processing tasks into subtasks and delegating them to smaller, specialized "expert" models.
Q: What are the key features of Llama 4 Scout?
A: Scout has 17 billion active parameters, 16 experts, and 109 billion total parameters. It has a very large context window of 10 million tokens, allowing it to process and work with extremely lengthy documents.
Q: What is the difference between Llama 4 Scout and Llama 4 Maverick?
A: Scout is best for tasks like document summarization and reasoning over large codebases, while Maverick is best for "general assistant and chat" use cases like creative writing.
Q: Are Llama 4 models biased?
A: Meta has tuned its Llama 4 models to refuse to answer "contentious" questions less often and to respond to a variety of different viewpoints.
Q: Are Llama 4 models available for use in the EU?
A: No, users and companies "domiciled" or with a "principal place of business" in the EU are prohibited from using or distributing the models due to governance requirements imposed by the region’s AI and data privacy laws.
