Introducing Prompts: LLM Monitoring

W&B Prompts – LLM Monitoring provides large language model usage monitoring and diagnostics. Start simply, then customize and evolve your monitoring analytics over time.

LLM Monitoring dashboard

Out of the box usage insights, customizable analytics

  • Organizations can track and manage prompts usage statistics like cost per prompt request, number of prompt tokens, and prompt latency
  • Individuals can filter and drill down to analyze prompt performance, observe all inputs sent to the prompt, and track experiments over time
  • Analyze trade offs between performance, latency, and costs to optimize project ROI

A single system of record for LLMs with one centralized API gateway

  • Shared and secure data artifacts and API keys with role based access controls and TTL policies
  • Manage and monitor access to LLMs using our enterprise API gateway
  • Immediately compare and contrast LLM performance with exploratory plots, graphs, and fine-grain activity details
  • Adaptive charts to group by users, model,  or any custom LLM attribute

Prompt engineering productivity, streaming LLM analysis

  • Bring your own data with stream logging with a few lines of code
  • Quickly create panels, add controls, and stay in sync
  • Generate custom evaluation in Python or through the visual interface

Example use cases

  • Build document extraction pipelines with LLMs
  • Create Q&A chatbots for internal or product documentation
  • Structured data extraction from text, image, and video files
  • Generate customer success or sales outreach emails


W&B LLM Monitoring works with the tools you’re using already. This includes everything from OpenAI Evals and Anyscale Endpoints to Azure OpenAI and HuggingFace, plus thousands of popular open source repositories.