Skip to main content
W&B Fully Connected
>
Articles

Integrating W&B Weave With NVIDIA AI Blueprint for AI Virtual Assistants to Enhance AI Observability

Announcing our latest integration with NVIDIA, W&B Weave with NVIDIA AI Blueprints
Abraham Leal
Created on December 31|Last edited on January 9
This post is co-written with Ruchika Kharwar from NVIDIA

Generative AI is transforming how enterprises build intelligent systems, offering personalized and efficient solutions for complex challenges. W&B Weave from Weights & Biases and NVIDIA AI Blueprints, part of NVIDIA AI Enterprise, help accelerate this transformation. Weave provides an end-to-end AI platform including evaluations, tracing, observability tools for production monitoring, easy ways to incorporate human feedback, and secure collaboration. NVIDIA AI Blueprints provide a robust foundation for developing advanced AI agents.
In this blog post, we’ll explore how integrating Weave and AI Blueprints empowers developers to create scalable, high-performing AI-powered applications that they can deliver to production with confidence. We will use the AI Blueprint for AI virtual assistants as an example.

What is an AI Blueprint?


AI Blueprints provide customizable generative AI reference architectures designed to equip developers with essential assets such as reference code, detailed documentation, Helm charts for deployment, and the relevant NVIDIA NIM microservices. This allows developers to quickly get a baseline RAG-powered application with personalization, summarization, and sentiment analysis.

What Is W&B Weave?


Weave provides an end-to-end AI developer platform to evaluate, monitor, and iterate on your LLM-powered application.
Weave simplifies the task of understanding the interaction between various AI agents and RAG components, making it an ideal companion for NVIDIA AI Blueprints. Through the combination, developers get:
  1. Enhanced observability: Gain detailed insights into how the agent processes queries, handles nested functions, and generates responses.
  2. Improved feedback loops: Combine NVIDIA NIM with Weave’s tracking to continuously fine-tune models via LLM-as-a-judge or human-in-the-loop feedback for reinforcement learning.
  3. Efficient debugging: Use Weave’s trace capabilities to identify bottlenecks or inefficiencies in the pipeline.
  4. Customizable metrics: Track system attributes such as user IDs or environmental contexts during production.
  5. Seamless scaling: Deploy robust AI assistants with consistent monitoring, even in high-demand environments

Get Started

Weave is simple to incorporate into existing code — it can be as simple as 3 lines of code:
import weave
weave.init('wandb-project-name')
@weave.op()
def my_assistant_call():
  pass
Weave’s tracing is driven by its decorator @weave.op(). Tracing allows developers to track all the inputs and outputs of the specific function, and connect the entire function tree together for the user in the UI for complex applications.


Sample pseudo-code from the AI Blueprint Integration: Tracking behavior with Weave

import weave
import os

client = weave.init(os.environ.get("WANDB_PROJECT"))

@weave.op()
def route_primary_assistant(
    state: State,
) -> Literal[
    "enter_product_qa",
    "enter_order_status",
    "enter_return_processing",
    "other_talk",
    "__end__",
]:
    route = tools_condition(state)
    if route == END:
        return END
    tool_calls = state["messages"][-1].tool_calls
    if tool_calls:
        if tool_calls[0]["name"] == ToProductQAAssistant.__name__:
            return "enter_product_qa"
        elif tool_calls[0]["name"] == ToOrderStatusAssistant.__name__:
            return "enter_order_status"
        elif tool_calls[0]["name"] == ToReturnProcessing.__name__:
            return "enter_return_processing"
        elif tool_calls[0]["name"] == HandleOtherTalk.__name__:
            return "other_talk"
    raise ValueError("Invalid route")

As you can see, creating this integration was really easy.If you’d like to try out the NVIDIA AI Blueprint with Weave, the sample repo can be found here.

How does this combination work together?

With Weave, you can trace and iterate on the AI Blueprint. For example, you can take a look at all the LangGraph transversals to serve queries that are happening quite nicely:


We can then inspect conversations more deeply, and even replay them and prompt engineer our agent to ensure we get better outcomes with the Weave Playground:


We can also create datasets with the live interaction data that is being traced. These data sets can further power the data flywheel, leading to better model tuning over time.


With this rich data of real interactions, we can evaluate our changes over time, ensuring we are making the right changes to make a better GenAI application.


Conclusion

Integrating W&B Weave with AI Blueprints creates a powerful synergy for building and managing GenAI-powered agents. This integration not only simplifies the development process but also delivers continuous improvement through traceability and observability. By leveraging these tools together, enterprises are equipped to unlock the full potential of AI while maintaining control and accountability over their applications. If you’d like to try this out yourself, the github repo can be found here or access the blueprint on NVIDIA here.
Tags: Articles, Weave, GenAI

Created with ❤️ on Weights & Biases.

https://wandb.ai/wandb_fc/jan-annoucement/reports/Integrating-W-B-Weave-With-NVIDIA-AI-Blueprint-for-AI-Virtual-Assistants-to-Enhance-AI-Observability--VmlldzoxMDc3NzQ2Mw
Made with Weights & Biases. Sign up or log in to create reports like this one.
Iterate on AI agents and models faster. Try Weights & Biases today.
Sign upTry W&B now