New Weave features: REST API, trace filtering, polished docs, and cookbooks
From our REST API to an exciting new LLM judgement day event, here's what's going on with Weave over the past two weeks
Created on September 12|Last edited on September 12
Comment
It’s time again for the W&B Weave newsletter. This week’s new Weave features span a new REST API, powerful filters, and expanded export options, plus we’ve published all the details in our new Weave release notes. But, as usual, let’s start with the tip of the week:
LLM tip of the week ✅
When using an LLM provider’s structured output modes (for example, JSON mode), check that it isn’t reducing performance on your task. In this paper, from Tam et. al, they explain that in smaller LLMs, the mode can hamper reasoning. Instead try a two step process: first generate without formatting constraints, then make a second call to a small, fast LLM to correctly format the output.
Product news 🚀
Service API
Interested in logging to W&B Weave from languages other than Python? We've exposed our REST API so you can see how to use Weave from clients built with other languages.
Powerful filters
To find W&B Weave traces of interest, you can now easily filter the calls table based on the inputs, outputs, date, or custom attributes. Simply select which W&B Weave op—a versioned function that automatically logs calls—is displayed and start filtering.
Pro tip: You can optionally click on a cell to automatically add a filter based on the cell’s content.
💡
Export calls
In addition to CSV export, you can now export all calls from W&B Weave into TSV, JSONL, and JSON.
Cookbooks and how-tos 🥣
We’ve got everything you need to cook up a great LLM app including new W&B Weave cookbooks for everything from tracking the input and output of functions to using third party integrations such as OpenAI, Anthropic, and Mistral, to completing evaluations.
We’ve also revamped the documentation by adding tutorials to the getting started guide for common tasks such as evaluating RAG apps, and we reorganized the product-specific documentation so its easier and faster to find what you need.
Finally, the new auto-generated reference documentation describes the Weave open-source library, everything from key classes and functions to fields, attributes, and methods so you can see all ingredients used to make Weave.
Popular blogs 📑
A frog dressed as a knight at night
Prompt upsampling using an LLM for prompt creation can result in more detailed and aligned images. Learn how prompt optimization libraries like DSPy can help automate a path to more nuanced, detailed image generations.
Is Hirschsprung disease a Mendelian or multifactorial disorder?
How to use Snowflake’s Arctic LLM for RAG over medical paper repositories such as PubMed, and evaluate it using RAG-specific evaluation criteria.
Events 🏢
Flexible RAG: development and evaluation strategies
Join the Weights & Biases ML team and Cohere to learn about how to build and evaluate agentic RAG systems to handle complex user queries on August 27th, 5pm CET / 8am PT.
Judgment day: Building LLM judges
Evaluating LLMs quickly is vital for improving their performance. Since human annotation is slow and expensive, in our hackathon next month we’ll build LLMs to judge LLMs. This is a two day event in San Francisco, starting on September 21st and we hope you can join us. (And yes, there will be prizes.)
Community💡
BigDataBass used Weave to evaluate 4 methods of structured output generation from GPT-4o. Interestingly it looks like Marvin consumes 10-15% more tokens than the other methods.
Need help getting started with W&B Weave?
Add a comment
Iterate on AI agents and models faster. Try Weights & Biases today.