A guide to LLM debugging, tracing, and monitoring

Understanding LLM observability

What is LLM observability and its importance?

Key features of LLM observability solutions

End-to-end tracing of LLM calls: The tool should capture each span of work – e.g. a single model call or tool invocation – and link these spans into complete traces representing a full request/response cycle. This means if your application involves multiple steps (like an agent calling multiple tools or models), all those steps are recorded as part of one trace. Having complete traces makes it possible to replay and analyze the sequence of events leading to an output, which is invaluable for debugging.
Logging inputs and outputs (and intermediate data): All inputs fed to the LLM (prompts, user queries, etc.) and outputs generated should be logged. Additionally, any intermediate results (like an agent’s thought process or retrieved context in a RAG system) should be captured. This comprehensive logging ensures that when something goes wrong, you can inspect exactly what the model saw and produced at each step. W&B Weave, for instance, automatically tracks inputs, outputs, and even the code of functions if you instrument them, giving full visibility into each operation.
Performance and health metrics: Observability isn’t just about content of responses—it’s also about metrics. Key metrics include latency (response time for each call), throughput (requests handled), error rates (failed calls or exceptions), and resource usage such as token counts and costs. Good monitoring tools will track token usage and estimated API costs per call, alerting you if usage spikes unexpectedly. W&B Weave, for example, automatically calculates token usage and associated costs for each LLM call, helping you keep an eye on latency and expenses. Monitoring these metrics ensures your LLM app stays within performance and budget expectations.
Debugging and error analysis: When an LLM call fails (throws an error, times out, or returns a malformed response), the observability tool should surface this clearly. Features like stack trace logging for exceptions, or highlighting of calls that returned incomplete data, are very helpful. The ability to dive into a particular failed span and inspect variables or error messages accelerates troubleshooting. Tools might allow tagging or commenting on problematic traces for team members to investigate.
Bias and safety evaluation: Modern LLM deployments need to be ethical and safe. Observability solutions should aid in evaluating the content of outputs for biases, toxicity, or policy violations. This could mean integration with scorers or evaluators that rate an output for certain criteria (e.g. a toxicity score, or a bias metric). For instance, W&B Weave includes pre-built LLM-based scorers for common tasks like hallucination detection, summarization quality, and relevance. These can be used to automatically evaluate each output. By examining these scores in your monitoring dashboard, you can catch if your model’s outputs are drifting into unsafe territory or becoming biased. Some tools even support guardrails, which are active checks that can block or modify unsafe outputs before they reach the user (more on this later).
Interpretability and metadata: It’s helpful if the observability platform can enrich traces with additional context. This might include metadata like which model version was used, which prompt template, or environment info. It could also involve storing embeddings or other representations for each call to analyze patterns. Advanced platforms offer visualization of conversation flows or allow searching through past calls by content. The goal is to make it easier to interpret why the model responded as it did. Features like W&B Weave’s custom attributes (via weave.attributes context manager) let you tag certain calls with domain-specific metadata for easier filtering and analysis.

How LLM observability tools enhance monitoring

Monitoring model performance and health

Evaluating bias and safety in LLM outputs

Tutorial: using W&B Weave for debugging, tracing, and monitoring

pip install crewai weave openai openai-agents

import os
import weave
from openai import OpenAI


weave.init("weave_demo")
client = OpenAI()

@weave.op()
def gpt5_chat(system_prompt: str, user_prompt: str, temperature: float = 0.2) -> str:
    resp = client.chat.completions.create(
        model="gpt-5",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )
    return resp.choices[0].message.content.strip()


# since Weave is integrated with OpenAI, we can use the same function without the @weave.op decorator, and result will still be traced
def gpt5_chat_no_weave_op(system_prompt: str, user_prompt: str, temperature: float = 0.2) -> str:
    resp = client.chat.completions.create(
        model="gpt-5",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )
    return resp.choices[0].message.content.strip()

if __name__ == "__main__":
    out = gpt5_chat(
        "You are concise and factual.",
        "Explain what Weights & Biases Weave is in one sentence."
    )
    print(out)

    out = gpt5_chat_no_weave_op(
        "You are concise and factual.",
        "Explain what Weights & Biases Weave is in one sentence."
    )
    print(out)    





Integrating Weave with OpenAI Agents

import asyncio
import os
from agents import Agent, Runner, WebSearchTool
from agents import set_trace_processors
from weave.integrations.openai_agents.openai_agents import WeaveTracingProcessor
import weave


# weave tracing on
weave.init("openai_agent_websearch")

set_trace_processors([WeaveTracingProcessor()])

async def main():
    agent = Agent(
        name="Web searcher",
        instructions="Answer briefly. Use the tool if it helps.",
        tools=[WebSearchTool(user_location={"type": "approximate", "city": "New York"})],
    )

    q = "search the web for 'local sports news' and give me 1 interesting update in a sentence."
    result = await Runner.run(agent, q)
    print(result.final_output)

if __name__ == "__main__":
    asyncio.run(main())



Utilizing Weave with CrewAI Agents

import os
from typing import Type, List, Dict, Any
from pydantic import BaseModel, Field
from crewai import Agent, Task, Crew, Process
from crewai.tools import BaseTool
from langchain_openai import ChatOpenAI
import mimetypes
import weave


weave.init("crewai_dir_agent")

ROOT = os.getenv("DIR_ROOT", ".")
MAX_LIST = int(os.getenv("DIR_MAX_LIST", "800"))
MAX_READ = int(os.getenv("DIR_MAX_READ", "5"))
MAX_BYTES = int(os.getenv("DIR_MAX_BYTES", "20000"))

def _is_text(path: str) -> bool:
    mtype, _ = mimetypes.guess_type(path)
    if mtype and mtype.startswith("text/"):
        return True
    text_exts = {".py", ".md", ".txt", ".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".csv", ".ts", ".js", ".java", ".go", ".rs", ".rb"}
    return os.path.splitext(path)[1].lower() in text_exts

# ── Tools ───────────────────────────────────────────────────────────────────────

class RepoQueryInput(BaseModel):
    query: str = Field(..., description="Describe what you are looking for in the codebase.")
    k: int = Field(default=MAX_READ, description="How many files to return.")

class RepoExplorerTool(BaseTool):
    name: str = "Repo Explorer"
    description: str = "Given a query, return up to k relevant FILE PATHS from the current directory tree. Output one absolute path per line."
    args_schema: Type[BaseModel] = RepoQueryInput

    def _run(self, query: str, k: int = MAX_READ) -> str:
        files: List[str] = []
        for root, _, fs in os.walk(ROOT):
            for f in fs:
                p = os.path.join(root, f)
                if _is_text(p):
                    files.append(os.path.abspath(p))
                if len(files) >= MAX_LIST:
                    break
            if len(files) >= MAX_LIST:
                break
        llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0)
        prompt = f"""You are a codebase navigator.
Pick up to {k} files that best help answer the user question.
Return ONLY absolute paths, one per line. Do not add commentary.

User question:
{query}

Candidate files:
{os.linesep.join(files)}
"""
        resp = llm.invoke(prompt)
        return resp.content.strip()

class ViewFileInput(BaseModel):
    filepath: str = Field(..., description="Absolute path of the file to read.")

class ViewFileTool(BaseTool):
    name: str = "View File Content"
    description: str = f"Read a text file. Truncates to {MAX_BYTES} bytes."
    args_schema: Type[BaseModel] = ViewFileInput

    def _run(self, filepath: str) -> str:
        path = filepath.strip()
        try:
            with open(path, "rb") as f:
                data = f.read(MAX_BYTES)
            return data.decode("utf-8", errors="replace")
        except Exception as e:
            return f"[read error] {e}"

class SummarizeManyInput(BaseModel):
    question: str = Field(..., description="The user question to answer from files.")
    files: str = Field(..., description="Absolute file paths separated by newlines.")

class SummarizeFilesTool(BaseTool):
    name: str = "Summarize Files For Answer"
    description: str = "Read the listed files, then answer the question using only their contents. Include filenames you used."
    args_schema: Type[BaseModel] = SummarizeManyInput

    def _run(self, question: str, files: str) -> str:
        paths = [p.strip() for p in files.splitlines() if p.strip()][:MAX_READ]
        chunks = []
        for p in paths:
            try:
                with open(p, "rb") as f:
                    data = f.read(MAX_BYTES)
                txt = data.decode("utf-8", errors="replace")
            except Exception as e:
                txt = f"[read error] {e}"
            chunks.append(f"FILE: {p}\n---\n{txt}")
        context = "\n\n".join(chunks)
        llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.2)
        prompt = f"""Answer strictly from the files below. If you cannot find the answer, say so.
Include the filenames you relied on.

Question:
{question}

Files:
{context}
"""
        resp = llm.invoke(prompt)
        return resp.content.strip()
    

repo_explorer_tool = RepoExplorerTool()
view_file_tool = ViewFileTool()
summarize_files_tool = SummarizeFilesTool()

# ── Agent ──────────────────────────────────────────────────────────────────────

navigator = Agent(
    role="Directory QA",
    goal="Given a natural language question about this directory, find relevant files and answer from their contents.",
    backstory="Knows how to search the tree, open files, and synthesize answers.",
    tools=[repo_explorer_tool, view_file_tool, summarize_files_tool],
    allow_delegation=False,
    verbose=True,
    llm=ChatOpenAI(model_name="gpt-4o-mini", temperature=0.2),
)

# ── Task ───────────────────────────────────────────────────────────────────────

QUESTION = os.getenv("DIR_AGENT_QUESTION", "What does this repo do?")

task = Task(
    description=(
        f"Step 1: Use Repo Explorer with the question: {QUESTION}. "
        "Step 2: Inspect one or more of the returned paths with View File Content. "
        "Step 3: Call Summarize Files For Answer with the question and the newline list of selected paths to produce the final answer."
    ),
    expected_output="A concise answer derived from the selected files, including the filenames used.",
    agent=navigator,
)

crew = Crew(
    agents=[navigator],
    tasks=[task],
    process=Process.sequential,
    verbose=True,
)

if __name__ == "__main__":
    result = crew.kickoff()
    print("\nAnswer:\n")
    print(result)



Alternative use cases for W&B Weave

Multi-turn conversation tracking: If your application involves multi-turn dialogues (for example, a chat between a user and an AI assistant), Weave’s Threads feature lets you group related calls into a conversation thread. You can assign a thread_id to a series of calls representing one session. In the Weave UI, you’ll then see a threaded view showing the conversation flow turn by turn. This is extremely useful for analyzing how an agent’s responses evolve over a conversation and where it might go off track. Instead of viewing each call in isolation, you can see the whole session’s context.
Comparing prompts and model versions: Weave makes it easy to experiment with changes and compare outcomes. For instance, you can run your application with two different prompt versions or even two different LLM models, each logging to Weave (perhaps under separate runs or projects). The Weave dashboard allows side-by-side comparison of traces. This helps answer questions like “Did the new prompt reduce the hallucination rate?” or “Is model B faster than model A in this task?” You can literally click between runs to see differences in outputs or metrics. Weave also supports version control for prompts and models in a lightweight way, so you can track which version of a prompt was used in each run.
Custom dashboards and monitoring: Beyond the default trace view, W&B Weave is part of the W&B ecosystem, which means you can leverage W&B’s dashboarding features. You can log custom metrics (like any counters or performance measures your code computes) and visualize them. For example, if you use a custom scorer that rates the correctness of each answer on a scale of 1-10, you could plot this score over time to ensure it’s improving. Weave’s integration with W&B also means you can set up alerts through W&B’s alerting system (like an email or Slack notification if a metric goes out of bounds). While at the time of writing, fine-grained alerting for Weave’s own monitors is a developing feature, you can always export Weave data and use W&B Core for robust alerting.
Integration with existing MLops stacks: Weave plays nicely with other tools. If your organization uses MLflow or other experiment trackers, you might still use Weave specifically for the LLM tracing part and link it back to your other system via IDs or metadata. Weave also supports exporting traces (for example, via OpenTelemetry format) if needed, although it primarily shines as a self-contained solution. Additionally, because Weave has both Python and TypeScript SDKs, you can use it in various environments – be it backend services or web applications that call LLMs.
Guardrails and feedback collection: We touched on guardrails in the context of safety. Weave also helps in collecting user feedback or ratings on LLM outputs (if your application allows users to thumbs-up/down an answer or provide corrections). By logging this feedback alongside the trace (e.g., as a scorer or as metadata), you gather a valuable dataset for improving the model. Over time, you could even fine-tune your LLM on examples where it performed poorly vs. where it succeeded, using the Weave-collected data as your evaluation set.
Enterprise and self-hosting: For companies with strict data policies, W&B offers Weave in a self-managed deployment as well. So if needed, you can run the Weave backend on your own infrastructure to keep all prompts and outputs in-house. This addresses concerns around data privacy while still benefiting from the powerful debugging and monitoring capabilities.

Conclusion

Recap and future directions

Frequently asked questions (FAQ)

Latency per request: How long each model call or agent action takes. This helps track response times.
Throughput or QPS: How many requests are being handled per minute/hour. Useful for capacity planning and spotting traffic spikes.
Error rate: The frequency of errors/exceptions or the rate of failed requests. This could include timeouts from the model API or your own application errors.
Token usage: How many tokens each request uses (prompt + completion) and cumulative tokens over time. This correlates with cost if using a paid API.
Cost: If applicable, an estimate of the API cost or computational cost per request.
Quality metrics: Any domain-specific metric of output quality. For example, accuracy if you have a way to measure it, or user rating if your users provide feedback. Even proxy metrics like length of response or number of facts can be tracked.
Safety metrics: Counts of how often the model produces flagged content (e.g., how many outputs were caught by a toxicity filter or needed to be blocked).
Drift metrics: If you have a reference distribution (like average response length or sentiment), track these over time to see if the model’s behavior drifts.

W&B Weave: A comprehensive solution (as discussed) that tracks LLM calls, agent steps, and also provides evaluation and guardrails. It’s cloud-based with on-prem options and integrates with many LLM frameworks.
LangSmith: Offered by the LangChain team, it’s tailored for applications built with LangChain, logging chains and agents and providing an interface to monitor them.
OpenTelemetry-based setups: Some teams integrate LLM monitoring into their existing telemetry by instrumenting code to emit spans and traces to systems like Jaeger or Datadog. This can be done via OpenTelemetry SDKs. It’s more DIY, but it’s an option if you need everything in one place.
Other LLMOps platforms: There are startups and tools (such as Langfuse, Future AGI’s TraceAI, etc.) which focus on LLM monitoring. These often provide similar features like trace visualization, cost tracking, and alerting, each with their own twist (e.g., FutureAGI emphasizes open-source and OTel compatibility).
Custom Logging + Dashboards: In absence of a dedicated tool, some developers roll out their own lightweight monitoring – for example, logging each request’s info to a database or logging service and building a simple dashboard. This is less feature-rich but can be a quick solution. During deployment, ideally you want a solution that requires minimal changes to your model serving code, and that can scale with your load. Many find that adopting a purpose-built LLM monitoring tool is worth it for the advanced insight it provides. Among the options, W&B Weave is popular for its ease of use and integration with the development workflow (experiment tracking and model registry integration), while others might choose a different tool if it aligns better with their stack. It’s also not uncommon to use a combination (for instance, use Weave for deep traces and an APM tool for system metrics). The good news is that the ecosystem is maturing, so you have plenty of choices to ensure your LLM is well-tracked during deployment.