OpenAI Unveils GPT-4o Mini
Open AI has a new successor to GPT-3.5!
Created on July 19|Last edited on July 19
Comment
OpenAI has announced the release of GPT-4o mini, its most cost-effective small AI model to date. Priced at 15 cents per million input tokens and 60 cents per million output tokens, GPT-4o mini is significantly more affordable than previous models, including GPT-3.5 Turbo.
Performance and Applications
GPT-4o mini achieves an 82% score on the MMLU benchmark and outperforms GPT-4 on chat preferences in the LMSYS leaderboard. The model supports a broad range of tasks due to its low cost and latency. It is particularly effective for applications that involve multiple API calls, handling extensive context (like entire code bases or conversation histories), and providing real-time text responses for customer support.
Currently, GPT-4o mini supports text and vision inputs in the API, with future updates expected to include image, video, and audio capabilities. With a context window of 128K tokens and support for up to 16K output tokens per request, GPT-4o mini is equipped for extensive and complex interactions.
Benchmark Results
GPT-4o mini has been evaluated against several benchmarks:
Reasoning Tasks: Scoring 82.0% on MMLU, outperforming Gemini Flash (77.9%) and Claude Haiku (73.8%).
Math and Coding Proficiency: Achieving 87.0% on MGSM for math reasoning and 87.2% on HumanEval for coding performance, surpassing other small models.
Multimodal Reasoning: Scoring 59.4% on MMMU, higher than Gemini Flash (56.1%) and Claude Haiku (50.2%).
Safety Measures
GPT-4o mini includes built-in safety measures, similar to GPT-4, such as filtering inappropriate content during pre-training and employing reinforcement learning with human feedback (RLHF) to align the model's behavior with OpenAI's policies. Additionally, the new instruction hierarchy method improves resistance to jailbreaks and prompt injections.
Availability
Developers can access GPT-4o mini through the Assistants API, Chat Completions API, and Batch API. It will be available to Free, Plus, and Team ChatGPT users immediately, with Enterprise users gaining access next week. Fine-tuning capabilities for GPT-4o mini will also be introduced shortly.
OpenAI continues to reduce costs while enhancing AI capabilities, making models like GPT-4o mini accessible for a wide range of applications, from customer support to complex data processing.
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.