Custom Charts
Use Custom Charts to generate interactive custom visualizations using Vega. In this report I'll show you an example of a custom visualization and how I made it.
Let's consider that you're training a neural machine translation model, translating from German to English, using some form of sequence to sequence network with an attention mechanism. In these models, there's an attention layer called a decoder which attends to each of the different input states. Your model seems to be working well, but you'd like to understand how attention is being used to make these translations. In this case, you may be interested in seeing where the decoder was attending to when it output a certain word. This is what the visualization below shows.
In order to create this visualization, I trained a model as above using the nmt.py script in this github repo. During training, I have the model produce some translation, after each epoch. I construct a wandb.Table object where each row has 5 entries:
- the source word index
- the translated word index
- the source word
- the translated word
- the attention from the translated word to the source word
An example of the code for logging this information is shown below:
# src = ['array' , 'of', 'source', 'words']
# len(src) == n
# translation = ['array', 'of', 'translated', 'words']
# len(translation) = m
# attn = np.array shape (m,n)
# Construct a 2-D array of information
attn_data = []
for m in range(attn.shape[0]):
for n in range(attn.shape[1]):
attn_data.append([n, m, src[n], translation[m], attn[m, n]])
wandb.log({"attn_table": wandb.Table(data=attn_data, columns=["s_ind", "t_ind", "s_word", "t_word", "attn"])})
Logging a wandb.Table object makes it accesible to the Custom Charts UI. In the project workspace, click on the add panel button. From the list of options available select the Custom Chart option.
This will open a window that will let you create your custom visualization.
From this window, you can construct a query to use data you logged during your run. In this case, I'd like to select the attn_table key from the SummaryTable.
Creating a New Custom Chart
From here, I'd like to create a new Custom Chart, this can be done by clicking the Edit modal, next to the drop down menu in the top left of the window. This opens the Vega Editor:
From here you can start writing your Vega Spec. I'm using Vega, but both Vega and Vega-lite are compatible. If it's your first time using Vega the documentation is great, and there are a ton of examples that you can learn from.
To bring your query into the table, you should create a data source named wandb
To access data from specific columns of the table, you use a template string to construct a field reference.
Where the field reference here is ${field:value}.
Each field reference creates a new entry in the chart fields options on the right side of the screen, you can use these chart fields to select the column of data you logged in your wandb.Table object.
In order to keep this smooth, I'm going to share the full Vega spec for my visualization at the bottom of the report. However, it has 5 custom fields, one for each of the logged columns. The data is referred to throughout the entire spec, and always through the same field reference notation shown above. (${field:column_name}).
The resulting field column list looks like so:
While editing, the data structures can be accessed using the debug tab of the editor. This tab will show the data you've created, including any marks you've made on the chart, and any signals(Vegas method for making charts interactive) you've made on the chart. This is very useful for debugging your visualization:
A sample of your current visualization is shown in the upper right. You can save your spec by clickin the save button and choosing a name. This will make it accessible in all your projects
And that's all there is to it.
The Vega Spec
I wanted to put this at the end because it's long, so here it is:
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"data": [
{
"name": "wandb",
"transform": [
{
"type": "formula",
"expr": "datum.${field:attn} > 0.00001 ? 10*(pow(datum.${field:attn},1.6)): 0",
"as": "swidth"
}
]
},
{
"name": "attns_lists",
"source": "wandb",
"transform": [
{
"type": "aggregate",
"fields": ["${field:attn}"],
"ops": [
"argmax"
],
"groupby": [
"${field:s_index}"
],
"as": [
"allvals"
]
}
]
},
{
"name": "attnt_lists",
"source": "wandb",
"transform": [
{
"type": "aggregate",
"fields": [
"${field:attn}"
],
"ops": [
"argmax"
],
"groupby": [
"${field:t_index}"
],
"as": [
"allvalt"
]
}
]
}
],
"title": {
"text": "Seq2Seq Decoder Attention Visualization"
},
"signals": [
{
"name": "active",
"value": null,
"on": [
{
"events": "rect:mouseover",
"update": "datum.${field:t_index}"
},
{
"events": "rect:mouseout",
"update": "null"
}
]
},
{
"name": "active2",
"value": null,
"on": [
{
"events": "rect:mouseover",
"update": "datum.${field:s_index}"
},
{
"events": "rect:mouseout",
"update": "null"
}
]
},
{
"name": "colorOut",
"value": "firebrick"
},
{
"name": "cornerRadius",
"update": "height/50"
},
{
"name": "yspacing",
"update": "8"
},
{
"name": "center",
"update": "width/2"
},
{
"name": "bottom",
"update": "0"
},
{
"name": "top",
"update": "yspacing+0.5"
}
],
"scales": [
{
"name": "xscale",
"type": "band",
"domain": {
"data": "wandb",
"field": "${field:s_index}"
},
"range": "width"
},
{
"name": "txscale",
"type": "band",
"domain": {
"data": "wandb",
"field": "${field:t_index}"
},
"range": "width"
},
{
"name": "yscale",
"domain": [
0,
{
"signal": "yspacing"
}
],
"nice": true,
"range": "height"
}
],
"marks": [
{
"type": "text",
"encode": {
"enter": {
"x": {
"signal": "center"
},
"text": {
"value": ""
}
}
}
},
{
"name": "sbaseMarks",
"type": "rect",
"from": {
"data": "attns_lists"
},
"encode": {
"enter": {
"x": {
"scale": "xscale",
"field": "${field:s_index}",
"offset": 1
},
"width": {
"scale": "xscale",
"band": 0.95,
"offset": -1
},
"y": {
"scale": "yscale",
"value": 1
},
"y2": {
"scale": "yscale",
"value": "0"
},
"tooltip": {
"signal": "{title: 'Maximum Attention From:', 'Word': datum.allvals.${field:tword}, 'Value': datum.allvals.${field:attn}}"
}
},
"update": {
"fill": [
{
"test": "datum.${field:s_index} === active2 || datum.${field:s_index} === active2",
"signal": "colorOut"
},
{
"value": "steelblue"
}
],
"cornerRadius": {
"signal": "cornerRadius"
},
"opacity": {
"value": 0.7
}
}
},
"transform": [
{
"type": "formula",
"expr": "datum.x + datum.width/2",
"as": "act_posx"
},
{
"type": "formula",
"expr": "datum.datum.${field:sword}",
"as": "text"
},
{
"type": "formula",
"expr": "datum.y + datum.height/2 + 4",
"as": "act_posy"
},
{
"type": "formula",
"expr": "datum.width-5",
"as": "max_width"
}
]
},
{
"name": "sourcewords",
"type": "text",
"from": {
"data": "sbaseMarks"
},
"encode": {
"update": {
"x": {
"field": "act_posx"
},
"y": {
"field": "act_posy"
},
"text": {
"field": "datum.allvals.${field:sword}"
},
"align": [
{
"value": "center"
}
],
"fontSize": {
"value": 12
},
"limit": {
"field": "max_width"
},
"tooltip": {
"signal": "{title: 'Maximum Attention From:', 'Word': datum.datum.allvals.${field:tword}, 'Value': datum.datum.allvals.${field:attn}}"
}
}
}
},
{
"name": "tbaseMarks",
"type": "rect",
"from": {
"data": "attnt_lists"
},
"encode": {
"enter": {
"x": {
"scale": "txscale",
"field": "${field:t_index}",
"offset": 1
},
"width": {
"scale": "txscale",
"band": 0.95,
"offset": -1
},
"y": {
"scale": "yscale",
"signal": "yspacing-1"
},
"y2": {
"scale": "yscale",
"signal": "yspacing-2"
},
"tooltip": {
"signal": "{title: 'Maximum Attention To:', 'Word': datum.allvalt.${field:sword}, 'Value': datum.allvalt.${field:attn}}"
}
},
"update": {
"fill": [
{
"test": "datum.${field:t_index} === active || datum.${field:t_index} === active",
"signal": "colorOut"
},
{
"value": "steelblue"
}
],
"cornerRadius": {
"signal": "cornerRadius"
},
"opacity": {
"value": 0.7
}
}
},
"transform": [
{
"type": "formula",
"expr": "datum.x + datum.width/2",
"as": "act_posx"
},
{
"type": "formula",
"expr": "datum.y + datum.height/2 + 4",
"as": "act_posy"
},
{
"type": "formula",
"expr": "datum.width-5",
"as": "max_width"
}
]
},
{
"name": "trans_words",
"type": "text",
"from": {
"data": "tbaseMarks"
},
"encode": {
"update": {
"x": {
"field": "act_posx"
},
"y": {
"field": "act_posy"
},
"text": {
"field": "datum.allvalt.${field:tword}"
},
"align": [
{
"value": "center"
}
],
"fontSize": {
"value": 12
},
"limit": {
"field": "max_width"
},
"note": {
"field": "datum.${field:t_index}"
},
"tooltip": {
"signal": "{title: 'Maximum Attention To:', 'Word': datum.datum.allvalt.${field:sword}, 'Value': datum.datum.allvalt.${field:attn}}"
}
}
},
"transform": [
{
"type": "lookup",
"from": "tbaseMarks",
"key": "act_posx",
"fields": [
"x"
],
"values": [
"datum.${field:tword}"
],
"as": [
"texty"
]
}
]
},
{
"name": "paths",
"type": "path",
"from": {
"data": "wandb"
},
"encode": {
"update": {
"stroke": [
{
"test": "datum.${field:t_index} === active || datum.${field:s_index} === active2 || datum.${field:t_index} === active || datum.${field:s_index} === active2",
"signal": "colorOut"
},
{
"value": "#000"
}
],
"strokeOpacity": [
{
"test": "datum.${field:t_index} === active || datum.${field:s_index} === active2 || datum.${field:t_index} === active || datum.${field:s_index} === active2",
"value": 1
},
{
"value": 0.3
}
],
"strokeWidth": {
"field": "swidth"
}
}
},
"transform": [
{
"type": "lookup",
"from": "tbaseMarks",
"key": "datum.${field:t_index}",
"fields": [
"datum.${field:t_index}"
],
"as": [
"sourceNode1"
]
},
{
"type": "lookup",
"from": "sbaseMarks",
"key": "datum.${field:s_index}",
"fields": [
"datum.${field:s_index}"
],
"as": [
"targetNode1"
]
},
{
"type": "linkpath",
"sourceX": {
"expr": "datum.sourceNode1.x + datum.sourceNode1.width/2"
},
"targetX": {
"expr": "datum.targetNode1.x + datum.targetNode1.width/2"
},
"sourceY": {
"expr": "datum.sourceNode1.y+height/yspacing"
},
"targetY": {
"expr": "datum.targetNode1.y"
},
"shape": "diagonal"
}
]
}
]
}