OpenAI Launches First Second-Generation Embedding Model Available In Their API
OpenAI introduces text-embedding-ada-002, a text embedding model that outperforms existing models and it significantly faster, cheaper, and simpler.
Created on December 16|Last edited on December 16
Comment
OpenAI has just announced a new model available through their embedding model API. This model, called text-embedding-ada-002 (the -002 indicating it as a second-generation model), beats all existing embedding models available in the API on nearly all tasks at a fraction of the cost.
OpenAI's embedding models embed textual content into numerical vectors that machines can more easily read. There are a few different model families available in the embedding API, and a few different target tasks they are tuned for (such as text search queries or text similarity).
The new embedding model from OpenAI
Being the first second-generation embedding model in the API, text-embedding-ada-002 improves over its predecessors in many ways. While Ada models were considered the fastest and cheapest of the API's first-generation model lineup, they also had the weakest performance.
Now, Ada stands at the top of the pack, beating all other models on nearly all tasks while staying the fastest and by far the cheapest option. The new Ada model is also just a single model which performs all tasks rather than being split into many individual models fine-tuned for specific tasks and sub-tasks like the others.
In benchmarking comparison tests, text-embedding-ada-002:
- Leads text search task performance by 0.5% and beats Ada 1 by 4.3%.
- Leads code search task performance by 0.2% and beats Ada 1 by 1.3%.
- Leads sentence similarity task performance by 1.2% and beats Ada 1 by 1.7%.
- Remains the lowest performing on text classification tasks (by 2.1% compared to the leader and 1% compared to the second-worst), but beats Ada 1 by 0.8%.
The second-generation Ada model introduces a couple of new architectural improvements over the first-generation models:
- A 4x increase in input context length from 2048 to 8096 tokens, allowing for better large document handling or other long-form text content.
- Output embeddings have 1536 dimensions, which is up from Ada 1's 1024 dimensions but far smaller than the 12288 dimensions of Davinci (the leading first-generation model set). This greatly reduces computation and storage requirements down the line.
The second-generation Ada model is also available at a significantly lower cost compared to the first-generation models. In rough estimates, Davinci models can process 6 pages per dollar, Ada 1 models can process 300 pages per dollar, and the second-generation Ada model can process 3000 pages per dollar.
OpenAI currently recommends the new model's use over all other models in all cases aside from text classification, where it should instead first be compared with the other models to decide which is best.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.