Plot ROC Curves With Weights & Biases
This article provides usage and examples for plotting ROC curves with Weights & Biases using wandb.plot.roc_curve().
Created on October 7|Last edited on November 8
Comment
wandb.log({"roc" : wandb.plot.roc_curve( ground_truth, predictions,labels=None, classes_to_plot=None)})
You can log this whenever your code has access to:
- a model's predicted scores (predictions) on a set of examples
- the corresponding ground truth labels (ground_truth) for those examples
- (optionally) a list of the labels/ class names (labels=["cat", "dog", "bird"...] if label index 0 means cat, 1 = dog, 2 = bird, etc.)
- (optionally) a subset (still in list format) of these labels to visualize on the plot
Basic Usage
I finetune a CNN to predict 10 classes of living things: plants, birds, insects, etc. In my validation step, I call
wandb.log({"my_custom_plot_id" : wandb.plot.roc_curve(ground_truth,predictions,labels=["Amphibia", "Animalia", ... ,"Reptilia"])})
to produce the following curve for each run of my model (where each run logs to the same plot key, my_custom_plot_id). Scroll over the chart area to zoom in, click+drag to pan, and hover to see more detail about a line.
Toy CNNs
3
Customized Usage
To make this chart more legible, I can simply edit the built-in W&B chart definition (or Vega spec), following the Vega visualization grammar and produce this new chart, where the differences between a single class are much easier to spot across runs.
Now each line's color represents one of my 10 classes, and each line's stroke type—solid, dash, dot—represents one of my three experiments with different numbers of epochs/training examples. You can hover over the top right corner of the chart and click on the "eye" icon to see the full Vega spec.
Toy CNNs
3
Add a comment
Tags: Beginner, Computer Vision, Object Detection, Experiment, W&B Meta, CNN, Custom Charts, Panels, Plots, iNaturalist
Iterate on AI agents and models faster. Try Weights & Biases today.