Skip to main content
stacey
Projects
pytorch_intro
Reports
RNN Exploration

RNN Exploration

Exploring hyperparameters in a toy RNN and showing the editing and configuration settings of a Parallel Coordinates chart
Stacey Svetlichnaya
Created on December 26|Last edited on December 26

Balance Generalization and Overfitting




01234epoch9092949698

What hyperparameters matter in a small RNN?

Choose exactly one of the 3 tabs below to see the effect of each hyperparmeter on accuracy.

Following this RNN example with W&B, I tried a few combinations for my independent variables:

  1. number of layers: 2, 3, 4
  2. batch size: 32, 64, 128
  3. hidden layer size: 32, 64, 128
1 Layer Count
27
2 Batch Size
27
3 Hidden Size
27
Name
27 visualized
acc
9
97.81444
9
97.8
9
95.63444
loss
hidden_size
num_layers
batch_size
State
Notes
User
Created
Runtime
0.086841
64
3
74.66667
Finished
-
stacey
1mo 25d 1h 22m 15s
0.059991
32
3
74.66667
Finished
-
stacey
1mo 26d 16h 11m 49s
0.15904
128
3
74.66667
Finished
v2, duplicate settings
stacey
1mo 24d 23h 41m 45s
1-3
of 3


Layer count: Decrease




1 Layer Count
27


Increase batch size and keep tuning the hidden layer




1 Layer Count
27
2 Batch Size
27
3 Hidden Size
27


All RNN Experiments




All RNN Experiments
44


Created with ❤️ on Weights & Biases.

https://wandb.ai/stacey/pytorch_intro/reports/RNN-Exploration--Vmlldzo0MTkxNg
Made with Weights & Biases. Sign up or log in to create reports like this one.