Attention Maps

Adds support Heatmaps that can be used to make attention maps, confusion matrices et all.

# ExplainText

'''
Arguments:
         matrix_values (arr): 2D dataset of shape x_labels * y_labels, containing
                            heatmap values that can be coerced into an ndarray.
         x_labels  (list): Named labels for rows (x_axis).
         y_labels  (list): Named labels for columns (y_axis).
         show_text (bool): Show text values in heatmap cells.
'''
wandb.log({'heatmap_with_text': wandb.plots.HeatMap(x_labels, y_labels, matrix_values, show_text=False)})

Example - Neural Machine Translation from english → french.

ExplainText

ExplainText adds support for eli5's LIME based TextExplainer.

tl;dr: helps debug model predictions by showing what features (words/phrases) were important to make this classification prediction.

# ExplainText

'''
Arguments:
text (str): Text to explain
probas (probabilities from a black-box classifier): A function which
                 takes a list of strings (documents) and returns a matrix
                 of shape (n_samples, n_classes) with probability values,
                 i.e. a row per document and a column per output label.
'''
wandb.log({'explain_nlp': wandb.plots.ExplainText(text=doc, probas=pipe.predict_proba, target_names=twenty_train.target_names)})

Example - Classify newsgroups by topic.

Below we train some scikit models to classify 18000 newsgroups posts into 20 topics using the 20 newsgroups dataset.

Behind The Scenes

Implements LIME's algorithm:

• generate distorted versions of the text;
• predict probabilities for these distorted texts using the black-box classifier;
• train another classifier (one of those eli5 supports) which tries to predict output of a black-box classifier on these texts.

Named Entity Recognition

Adds support for spaCy's entity visualizer, which highlights named entities and their labels in a text.

# ExplainText

'''
Arguments:
         docs (list, Doc, Span): Document(s) to visualize.
'''
wandb.log({'NER': wandb.plots.NER(docs=doc)})

Part of Speech Tagging

Adds support for spaCy's dependency visualizer which shows part-of-speech tags and syntactic dependencies based on both definition and context.

# ExplainText

'''
Arguments:
         docs (list, Doc, Span): Document(s) to visualize.
'''
wandb.log({'part_of_speech': wandb.plots.POS(docs=doc)})

ROC and PR curves in wandb.log()

# ROC
wandb.log({'roc': wandb.plots.ROC(y_test, y_probas, nb.classes_)})

# Precision Recall
wandb.log({'pr': wandb.plots.precision_recall(y_test, y_probas, nb.classes_)})

# Confusion Matrices
wandb.sklearn.plot_confusion_matrix(y_test, y_pred, nb.classes_)```