Skip to main content

Product newsletter: Updates and new features from October

Here's a quick rundown of what we've been working on in September
Created on November 5|Last edited on November 5
It’s been a busy month for both Weave and Models. In October, we launched audio support, new scorers, custom naming options, and a chat tab for Weave. For Models, we focused on new metrics, performance improvements, better panel management, and a new integration with the Cohere fine-tuning API.
Here’s a look at what we’ve been building:

W&B Weave

📦 Out-of-the-box evaluation metrics

Not all metrics are unique snowflakes, but most important metrics often are are. Use the new W&B Weave scoring functions as references to get started tracking what’s important so you can iterate faster. Get started running evals quickly with these new first-class scorers.

🌐 Typescript & JavaScript support

Dear web devs: come join the fun. W&B Weave now supports JS/TS.

🔊 Audio format support

We’re continuing to add modalities to help teams build with Weave. Weave now supports audio file formats as inputs and outputs, as well as within dataset entries.
These audio formats—alongside support for code, images, and text—gives you the ability to understand multimodal models far better. To get started fast, try the cookbook. It uses the OpenAI chat completions API with GPT-4o Audio Preview to generate audio responses to text prompts and tracks it all in Weave.


🏷️ Custom evaluations and calls naming

We’ve made it easier to organize your GenAI application evaluations with custom naming in W&B Weave. Now you can organize and compare your evaluations by setting a display name in the code. And that’s not all. You can also to keep your work better organized by setting the display name of calls based on inputs, outputs, or any other custom logic.

💬 Chat tab

Check out the new chat tab in W&B Weave to see the entire conversation history quickly. Browse to find past messages, tool use, and context efficiently. Chat details are automatically displayed when selecting calls that conform to popular LLM chat message formats.



W&B Models

📈 Six new NVIDIA GPU system metrics

We've added six more out-of-the-box metrics in W&B Models to help you monitor and optimize the performance of NVIDIA GPUs when training and fine-tuning deep learning and large language models (LLMs).
New metrics include GPU Streaming Multiprocessor Clock Speed (MHz), GPU Memory Clock Speed (MHz), GPU Graphics Clock Speed (MHz), GPU Corrected Memory Errors, GPU Uncorrected Memory Errors, and GPU Encoder Utilization (%).
This addition makes automatic GPU performance tracking in W&B Models some of the most extensive in the industry. To get started, you need to be on SDK version v0.17.8 or later.


🛂 New project-level access controls

Now W&B Models Team admins and Project owners have the flexibility to assign specific roles to users at the project-level. This means there are now three identity and access management (IAM) scopes within Models: Organizations, Teams, and Projects.

📈 Additional TPU metrics

We also added three more metrics to automatically track Google TPU resources when running experiments with W&B Models. Specifically, you can now analyze TPU memory usage (in bytes and percentage of total memory) and duty cycle (the percentage of time the TPU is actively processing) in W&B Models. These metrics compliment the other TPU utilization metrics you're used to seeing in Weights & Biases. And, since the metrics are streamed into the workspace in real time, you can spot trends such as growing memory usage and take preventive actions to avoid system crashes.
To get started, you need to be on SDK version v0.18.3 or later. You can read this blog post for more information.


🎛️ Panel management

Now you can easily declutter your workspace by removing unnecessary panels and adding back only the metrics and keys that matter most to you.
Why you'll love it:
  • Enhanced usability: Quickly remove automatically added panels while keeping your custom ones intact.
  • Simplified focus: Tailor your workspace to display only the data that's relevant to you.
  • Improved performance: Reduce data load for a faster, smoother experience—especially beneficial for large workspaces.
Pro Tip: We highly recommend using Panel Management to maintain optimal workspace performance for large workspaces.
💡


Role-level Automations

W&B Models now allows Automations to be applied at the registry level and not just the collections level.
Prior to this enhancement, creating identical Automations for multiple collections within a registry required building a separate Automation for each individual collection. But you can now create a single Automation that will execute when the selected triggering event, either adding a new model version or adding an alias to a model version, occurs for any collection in the registry.


Integration with Cohere fine-tuning API

The integration between W&B Models and the Cohere fine-tuning API, makes it easier than ever to fine-tune Cohere's Command R and R+ models. With just one line of code, you can start tracking experiments, automatically log key metrics, and streamline the fine-tuning process. This integration provides real-time insights, a centralized system of record, and powerful visualization tools to help you achieve better model accuracy, all while saving time and boosting collaboration. To get started, follow along with the Cohere fine-tuning cookbook.
For more information on the integration, see the Cohere API documentation and the W&B Models developer guide.

Filter artifacts by version tag using the SDK

W&B Models introduces support for retrieving artifacts by version tag. Tags added to artifact versions either manually or programmatically can now be used to filter and access artifacts from Registry or ML projects using the SDK. Filtering directly by artifact version tags offers a more efficient way to retrieve only the artifacts you need instead of grabbing every artifact from a collection and parsing them after retrieval using Python code.


That's all for this month's product newsletter. If you'd like to browse updates from the past few months, click any of the links below:
September product updates: For W&B Weave, custom cost tracking, image support, and new export option to Python code. For W&B Models, wandb-core GA to provide fast, reliable logging tailored for large-scale, long-running experiments, security updates, and full fidelity charts on by default.
August product updates: For W&B Weave, new service API, powerful filters, and export calls. For W&B Models, registry version tags, experiment name truncation, and billing dashboard updates.
July product updates: For Weave, visual evaluation comparisons and human feedback. For Models, track and organize your models and datasets with Registry, rewind feature, artifact discovery, and security updates.
June product updates: Weave and Anthropic integration, token and cost calculations in Weave, W&B SDK performance enhancements, and customized workspaces
May product updates: Mistral integration, SDK improvements, full fidelity line plots, and team-level BYOB
April product updates: Weave release, restricted projects, and governance enhancements
March product updates: Saved views, Artifact lineage clustering, and Fully Connected conference update
February product updates: User and role management, org admin controls, and download performance improvements
January product updates: Time-to-live (TTYL) policies, webhooks updates, and panel search improvements

Iterate on AI agents and models faster. Try Weights & Biases today.