GPT 4.5: The end of scaling laws?
Created on February 28|Last edited on February 28
Comment
OpenAI has introduced GPT-4.5, a research preview of its most advanced language model to date. As an improvement over GPT-4, this iteration enhances unsupervised learning, leading to better accuracy, more fluid conversations, and an improved understanding of user intent. While GPT-4.5 is currently accessible to ChatGPT Pro users and developers, OpenAI has indicated that it may not remain in the API long-term due to its high computational demands.
Benchmarks: Accuracy Gains and Fewer Hallucinations
GPT-4.5 demonstrates notable improvements in factual reliability and response quality. In the SimpleQA benchmark, which evaluates knowledge-based question answering, GPT-4.5 achieved an accuracy score of 62.5 percent, outperforming GPT-4o at 38.2 percent, OpenAI o1 at 47 percent, and OpenAI o3-mini at 15 percent. It also exhibited a reduced hallucination rate of 37.1 percent, a significant improvement over GPT-4o at 61.8 percent, OpenAI o1 at 44 percent, and OpenAI o3-mini at 80.3 percent.
In academic and technical benchmarks, GPT-4.5 further solidifies its improvements. In GPQA, which assesses science-related knowledge, it scored 71.4 percent, significantly higher than GPT-4o’s 53.6 percent, though slightly below OpenAI o3-mini at 79.7 percent. In coding evaluations like SWE-Bench Verified, GPT-4.5 outperformed GPT-4o at 38.0 percent versus 30.7 percent, reinforcing its capabilities in software development.
Pricing: A Sharp Increase Over Previous Models
GPT-4.5 enters the market with a steep price tag, reflecting both its enhanced performance and computational intensity. API pricing is set at 75 dollars per million input tokens and 150 dollars per million output tokens, with cached input tokens available at 37.50 dollars per million. This is a 2.5x increase over GPT-4’s launch pricing of 30 dollars per million input tokens and 60 dollars per million output tokens.
In stark contrast, OpenAI’s o3-mini model is priced at just 1.10 dollars per million input tokens and 4.40 dollars per million output tokens, making it over 100x cheaper than GPT-4.5’s output pricing. The significant pricing gap suggests OpenAI is targeting different segments, with GPT-4.5 aimed at high-end applications while o3-mini serves as a more accessible reasoning model.
How Big is GPT-4.5?
OpenAI has not disclosed the exact parameter count for GPT-4.5, but based on past scaling trends, it is likely 5-10x larger than GPT-4, given that this is a .5 update rather than a full-number jump. Previous model sizes offer a rough estimate:
- GPT-1 (2018) had 117 million parameters.
- GPT-2 (2019) had 1.5 billion parameters, representing about a 10x increase.
- GPT-3 (2020) scaled up to 175 billion parameters, roughly a 100x increase.
- GPT-4 (2023) was estimated to have over 1 trillion parameters, with a 5-10x increase over GPT-3.
If GPT-4.5 follows this pattern, it could be in the 5-10 trillion parameter range. However, OpenAI has emphasized architectural optimizations rather than just raw scaling, meaning GPT-4.5 may achieve its performance gains through more efficient training techniques rather than a massive increase in size.
Future Availability and Adoption
At present, GPT-4.5 is available in ChatGPT Pro and has limited API access. OpenAI has suggested that it may not be a permanent API offering, potentially due to its high computational cost. While it represents a clear step forward in factual accuracy, reasoning, and coding, its steep pricing may limit its widespread adoption.
As AI models continue to evolve, OpenAI appears to be experimenting with different pricing tiers, offering premium models like GPT-4.5 alongside more affordable options like o3-mini. If the trend of efficiency improvements continues, models with GPT-4.5’s capabilities could become far more accessible in the near future, reshaping AI availability across different industries.
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.