Base case: Log metric Y vs time

In the simple scenario, we want to log a metric like loss or accuracy over the course of training a model. The metric of interest, say “train_loss”, is logged to wandb every timestep (e.g. at the end of each epoch of training). Use the native W&B “Line plot” to visualize the full list of “train_loss” values logged to wandb over time. Click on "Add visualization" > "Line plot" > set Y = “train_loss”. This plot extends easily to multiple runs in a Workspace.

Modify X-axis: Log metric Y vs arbitrary X

What if we need units other than time on the x-axis? There are several ways to log a metric y versus an arbitrary metric x.

Single (x, y) data point per run

Use the native W&B “Scatter plot” and select the desired metrics for x and y. By default these will be pulled from the wandb run summary field, which stores only the last value logged to a given key (i.e. every new value logged overwrites the previous one). Thus each run will become a single data point of the form (summary.x, summary.y).

Multiple (x, y) data points per run (across time)

What if we need to log a data point multiple times over the course of each run, say at the end of every epoch? In this case, the run summary data is insufficient. We need the run history data, which we can access via a "Custom Chart" > "history" builtin. For this chart, we assume that the two metrics are joined by time—each point (x, y) is logged at a particular time step. Select the logged metric corresponding to each axis from the dropdown, and set the chart colors to represent the run names. You can modify the Custom Chart spec using Vega, for example to modify the vertical scale and zoom in on a region of interest as in the highlighted line. You can see the query editor panel and Vega spec for these charts at the bottom of this section.

Add a dimension: Log metrics Y, Z vs arbitrary X

What if the typical dimensionality of wandb (2 dimenisons: y vs x) is not enough? Add extra lines or colors to accommodate an extra dimension of data.

Multiple (x, y, z...) data points, once per run

To log multi-dimensional data at each timestep, use a wandb.Table(), which accepts a 2D array of data. Each row is a data point, and each column is a dimension of that data point. This code snippet will make the dimensions/fields "x", "y", and "z" available under the table key "custom_data_table":

# Logging a custom table of data
my_custom_data = [[x0, y0, z0], [x1, y1, z1]]
wandb.log({“custom_data_table”: wandb.Table(data=my_custom_data,
                                columns = ["x", "y", "z"])})

To generate these PR and ROC curves, I currently use sklearn.metrics.precision_recall_curve and roc_curve then log the arrays of (recall, precision, class name) and (false positive rate, true positive rate, class name) to a wandb.Table(). This table is then accessible by key from the run's summaryTable and rendered with a query mapping & Vega spec very similar to the previous one (see end of this section).

Note: since I created this example, we've added PR and ROC curve functionality so you can log these with one line from Python. Read more details and see examples here: wandb.plot.pr_curve() and wandb.plot.roc_curve().

Multiple (x, y, z...) data points, multiple times per run

These plots are similar to the previous section, except each set of curves is plotted once at the end of each of 10 training epochs. If you click on the gear settings icon in the top right of each chart, you will see a menu to "Select epoch". You can use this to focus each chart on a particular epoch and compare across them. Here we use the run historyTable instead of just summaryTable to get the 2D array of values logged for each epoch. An extra key for the training epoch can condition the opacity to filter the points to one epoch at a time. You can see the query mapping and the full Vega below the charts. Note the selection key used to create the dropdown menu and the opacity key used to condition the opacity to render one epoch at a time.

Add two dimensions: x, y, time, and multiple runs

Using opacity to indicate time and color to indicate runs can give you four dimensions in one chart! Here I log microaveraged precision for each model at the end of every epoch. Darker lines tend towards the top right of the chart, with substantial noise since the training dataset for these experiments is very small (for fast iteration).

What's your dream visualization?

Between wandb and Vega, you can log virtually anything. Try making your own custom chart and let us know how it goes!