Mistral AI Launches Small 3: A High-Performance, Open-Source AI Model
Efficient and Accurate
On Thursday, French lab Mistral AI launched Small 3, which the company calls "the most efficient model of its category" and says is optimized for latency. Small 3 can compete with Llama 3.3 70B and Qwen 32B, among other large models, and is "an excellent open replacement for opaque proprietary models like GPT4o-mini."
Open-Source and Released Under Apache 2.0 License
Like Mistral’s other models, the 24B-parameter Small 3 is open-source, released under the Apache 2.0 license. This allows developers to use, modify, and distribute the model as they see fit.
Designed for Local Use
Designed for local use, Small 3 provides a base for building reasoning abilities, Mistral says. "Small 3 excels in scenarios where quick, accurate responses are critical," the release continues, noting that the model has fewer layers than comparable models, which helps its speed.
Performance Benchmarks
The model achieved better than 81% accuracy on the MMLU benchmark test, and was not trained with reinforcement learning (RL) or synthetic data, which Mistral says makes it "earlier in the model production pipeline" than DeepSeek R1.
Comparison to Other Models
"Our instruction-tuned model performs competitively with open weight models three times its size and with proprietary GPT4o-mini model across Code, Math, General knowledge and Instruction following benchmarks," the announcement notes.
Human Evaluation
Using a third-party vendor, Mistral had human evaluators test Small 3 with more than 1,000 coding and generalist prompts. A majority of testers preferred Small 3 to Gemma-2 27B and Qwen-2.5 32B, but numbers were more evenly split when Small 3 went up against Llama-3.3 70B and GPT-4o mini. Mistral acknowledged the discrepancies in human judgment that make this test differ from standardized public benchmarks.
Recommended Use Cases
Mistral recommends Small 3 for building customer-facing virtual assistants, especially for quick-turnaround needs like fraud detection in financial services, legal advice, and healthcare, because it can be fine-tuned to create "highly accurate subject matter experts," according to the release. Small 3 can also be used for robotics and manufacturing and may be ideal for "hobbyists and organizations handling sensitive or proprietary information," since it can be run on a MacBook with a minimum of 32GB RAM.
Future Developments
Mistral teased that we can expect more models of varying sizes "with boosted reasoning capabilities in the coming weeks." You can access Small 3 on HuggingFace here.
FAQs
Q: What is the size of Small 3?
A: Small 3 is a 24B-parameter model.
Q: What is the purpose of Small 3?
A: Small 3 is designed for local use, providing a base for building reasoning abilities and exceling in scenarios where quick, accurate responses are critical.
Q: How does Small 3 compare to other models?
A: Small 3 can compete with Llama 3.3 70B and Qwen 32B, among other large models, and is an excellent open replacement for opaque proprietary models like GPT4o-mini.
Q: Is Small 3 open-source?
A: Yes, Small 3 is open-source, released under the Apache 2.0 license.
