Mistral releases NeMo: A high-performance 12B Open LLM
Mistral is at it again with a NeMo!
Created on July 18|Last edited on July 18
Comment
The Mistral AI team has announced the release of Mistral NeMo, a cutting-edge 12-billion parameter model developed in partnership with NVIDIA. Notable for its large 128,000-token context window, Mistral NeMo excels in reasoning, world knowledge, and coding accuracy within its size category. The model's architecture ensures ease of integration, serving as a direct replacement for the Mistral 7B model.
Long Context
Mistral NeMo supports up to 128,000 tokens, providing an expansive context for complex tasks. It demonstrates superior accuracy in reasoning and coding compared to other models in its class. The model utilizes a standard architecture, making it easy to adopt for existing systems using Mistral 7B. Released under the Apache 2.0 license, it facilitates use by researchers and enterprises.
The performance of Mistral NeMo's base model is benchmarked against Gemma 2 (9B) and Llama 3 (8B), highlighting its superior accuracy and efficiency.
Cultured
Mistral NeMo is designed for global applications, supporting languages such as English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Arabic, and Hindi. It is optimized for multilingual benchmarks, ensuring robust performance across diverse languages.
Cutting Edge Tokenization
The new Tekken tokenizer, based on Tiktoken, enhances compression efficiency across over 100 languages. It is notably more efficient in compressing source code and several major languages compared to the previous SentencePiece tokenizer. Tekken is also significantly more effective than the Llama 3 tokenizer for most languages.
Open-Source
Mistral NeMo's weights are available on HuggingFace for both the base and instruction-tuned models. Users can implement the model via mistral-inference and customize it with mistral-finetune. Additionally, the model is accessible through NVIDIA's NIM inference microservice on ai.nvidia.com.
This release marks a significant advancement in making high-performance AI models accessible and efficient for a wide range of applications globally!
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.