How Cohere Trains Business-Critical LLMs with the Help of W&B
"Reports have been great for us. They allow us to seamlessly communicate nuanced technical information in a way that’s digestible for non-technical teams."
Ellie Evans
Product Manager
About Cohere
Cohere is a language AI company focused on bringing the power of large language models (LLMs) to developers of all stripes, and businesses of all sizes. Whether used for semantic search, content moderation, content creation, or another business-critical use case, Cohere is building and tuning models that make a real difference for its growing customer base.
Training the Best Model for Each Customer
Chances are, you’ve heard of LLMs. After all, they’ve gained widespread acceptance and use by vastly outperforming the natural language models of just a few years ago.
Although Cohere’s baseline models are incredibly powerful, customization capabilities are available through the Cohere platform. Customers can upload a unique dataset and begin training a custom language model with a few simple clicks, resulting in some of the most powerful models in the world for domain or vertical-specific tasks.
For enterprise customers that require additional support, Cohere’s internal team can help them build a custom solution behind the scenes. This often involves lots of experimentation to build the best possible model for a customer’s unique use case, such as powering a better toxicity classifier for a social app, generating better ad copy for a marketing campaign. That’s where Weights & Biases (W&B) comes in.
The Scope of the Challenge
Before we dig in further, it’s important to level-set on just how large large language models are. Cohere’s models, for instance, contain billions of parameters. And because of the nature of LLMs, smaller scale, bite-sized experimenting simply doesn’t work — –it doesn’t accurately reflect how these models will behave when they’re scaled up to the size required.
The Cohere product team uses Weights & Biases for experimentation when building a custom model for a customer. The W&B platform logs and tracks which unique datasets, hyperparameters, or other model configurations produce the best language model for enterprise customers. W&B makes it easy to pull up a centralized dashboard and quickly understand which candidate model outperforms the others, unlocking insights about the particular architectures and data recipes that work best.
We spoke with Ellie Evans, a Product Manager at Cohere, who put it succinctly: “W&B lets us examine all of our candidate models at once. We can identify which model produces state of the art results on our robust test suite. This helps ensure we give enterprise customers a state-of-the-art solution for their specific application. This is vital for understanding which model will work best for each customer.”
Having a centralized workplace where all this information is easily accessible across Cohere’s growing product team is vital to their productivity and speed. Different machine learning engineers can easily compare their newest breakthrough to past successes. Everyone has visibility into each other’s best ideas and experiments, as well as the experiments that didn’t quite work. And that last part is particularly important. Training LLMs is expensive and time-consuming, and avoiding wasting modeling time saves on everything from compute costs to turn-around time for customers.
Some of this information is also captured in W&B Reports. Reports make it easier to share knowledge within the team and communicate with non-technical stakeholders, as well as preserve tacit knowledge within the company for future reference..
“Reports have been great for us,” Ellie said. “They allow us to seamlessly communicate nuanced technical information in a way that’s digestible for non-technical teams.”
Conclusion
As LLMs continue to gain widespread use across every industry, companies that can both train custom models and understand their nuances are in an ideal position. Cohere is a leader in this space with baseline models that can solve a huge array of NLP tasks, and customization capabilities that offer even more flexibility and power to customers. With Cohere, every customer can build and leverage the right model for their specific, immediate need.
As model size and complexity continues to increase, smaller companies without big machine learning teams simply won’t be able to train their own models in-house. LLMs cost millions of dollars and thousands of hours to train. Cohere makes the power of these models accessible to the vast majority of companies who simply don’t have those resources or that expertise. And that means more businesses have access to cutting-edge models today — and tomorrow.