Select runs that logged avg_loss
to visualize data in this line chart.
to visualize data in this line chart.
Paper Figures
Starts with the Key-Recall task. Then, perhaps the associative retrieval task from Ba & Hinton, repeated sequences, or even the baby names or short stories datasets. Might consider hyperparameter sweeps or a search for the ideal effective update magnitude.
Created on January 18|Last edited on January 19
Comment
Key-Recall
A synthetic dataset is constructed for simple, single character retrieval. The sequence contains a character, '?', indicating that the next input contains a value to be stored. Later in the sequence, another character, '!', requests the stored value. The gaps are filled with 0s. Some examples:
0000?10!100000?200!200?,00!,00?.0!.
Select runs that logged avg_loss
to visualize data in this line chart.
to visualize data in this line chart.
Repeated Sequences
Simple next-character prediction on a repeated sequence of four random characters.
Add a comment