Skip to main content

Microsoft Releases Phi-3

ChatGPT level performance on an iPhone?
Created on April 23|Last edited on April 23
Microsoft recently introduced a new series of compact AI models, the phi-3 series, designed to deliver powerful computational abilities directly on smartphones. The lineup includes the phi-3-mini, phi-3-small, and phi-3-medium, each balancing size and performance to extend the capabilities of compact models in practical applications.

Performance

The phi-3-mini, the smallest of the series with 3.8 billion parameters, is optimized for mobile devices and rivals larger models like GPT-3.5 with a 69% score on MMLU and 8.38 on MT-bench. It uses a curated mix of filtered web data and synthetic data to efficiently handle complex language tasks. The phi-3-small, which has 7 billion parameters, builds on the mini’s capabilities, particularly in multilingual contexts, thanks to a specialized tokenizer. It scores 75% on MMLU and 8.7 on MT-bench, showing enhanced proficiency for detailed queries. The phi-3-medium is the largest with 14 billion parameters, designed for more demanding tasks and scores 78% on MMLU and 8.9 on MT-bench, demonstrating its ability to manage extensive reasoning and long-context interactions.


Training Regime

Moving away from the "compute optimal regime" or "over-train regime," Microsoft's phi-3 models are developed under a "data optimal regime." This method fine-tunes the training data to ensure it is of the highest relevance and quality for the model's scale, removing less critical information to boost the model's reasoning capabilities. For instance, while data about a specific Premier League game might be relevant for larger models, it is omitted in the training of phi-3-mini to conserve model capacity for more crucial reasoning tasks.

Deployment on Mobile Devices

A standout feature of the phi-3-mini is its ability to operate directly on consumer hardware such as the iPhone 14. This capability is made possible by the model's size and efficiency, which allow it to be deployed on a smartphone while maintaining impressive performance. The ability to run such a sophisticated AI model on a mobile device opens up numerous possibilities for real-time, on-the-go AI applications, providing users with powerful AI tools directly in their pockets.

Challenges

The phi-3 models do face challenges. The smaller models, like the phi-3-mini, struggle with tasks requiring extensive factual knowledge. While currently optimized primarily for English, expanding their multilingual capabilities is a work in progress. Additionally, ensuring factual accuracy and maintaining safety without biases are ongoing challenges. To address these issues, Microsoft has implemented rigorous safety measures, including red-teaming and safety alignment in post-training.
Overall, the phi-3 series underscores Microsoft's commitment to advancing AI technology by developing solutions that balance performance with practical usability. As these models evolve, they are expected to overcome current limitations, paving the way for more reliable and universally accessible AI applications.
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.