Skip to main content

scale the bench

Created on August 23|Last edited on September 9

base metrics

plast_proportion


Parameter importance with respect to
avg_loss

Config parameter
Importance
Correlation
Loading...
grad clip sweep
288
comprehensive sweep
1313
3_palindrome_dataset_vary_length
606
2_palindrome_dataset_vary_length
491
4_palindrome_dataset_vary_length
102

n_hidden

3_palindrome_dataset_vary_length
606
2_palindrome_dataset_vary_length
491
4_palindrome_dataset_vary_length
102

clip_weights

3_palindrome_dataset_vary_length
534
2_palindrome_dataset_vary_length
419
4_palindrome_dataset_vary_length
216

effective_lr,

3_palindrome_dataset_vary_length
606
2_palindrome_dataset_vary_length
491
4_palindrome_dataset_vary_length
102

plotting effective_lr

3_palindrome_dataset_vary_length
344
2_palindrome_dataset_vary_length
397
4_palindrome_dataset_vary_length
102



3_palindrome_dataset_vary_length
154
2_palindrome_dataset_vary_length
491
4_palindrome_dataset_vary_length
102

only the best. For this visualization, I see which hyperparameters the best runs had.

3_palindrome_dataset_vary_length
344
2_palindrome_dataset_vary_length
491
4_palindrome_dataset_vary_length
102


specific questions

What are the parameters of the actual top performers?

3_palindrome_dataset_vary_length
123
2_palindrome_dataset_vary_length
141

What does plast_clip do? Does it make things explode? If so, when? Is there a min. value for which it can use memory at all? For simplicity, we fix clip_weights. We also fix plast_proportion and n_hidden.

3_palindrome_dataset_vary_length
33

ok, now, what's the effect of clip_weights

3_palindrome_dataset_vary_length
23



3_palindrome_dataset_vary_length
123