Comment
val_mse_main
val_mse_main
run_mode: grow-zero Small teacher (20 : 10 : 10)
run_mode: grow-random Small teacher (20 : 10 : 10)
run_mode: baseline-large Small teacher (20 : 10 : 10)
run_mode: baseline-small Small teacher (20 : 10 : 10)
run_mode: grow-zero Large teacher (100 : 50 : 10)
run_mode: grow-random Large teacher (100 : 50 : 10)
run_mode: baseline-large Large teacher (100 : 50 : 10)
run_mode: baseline-small Large teacher (100 : 50 : 10)
train_mse_main
train_mse_main
run_mode: grow-zero Small teacher (20 : 10 : 10)
run_mode: grow-random Small teacher (20 : 10 : 10)
run_mode: baseline-large Small teacher (20 : 10 : 10)
run_mode: baseline-small Small teacher (20 : 10 : 10)
run_mode: grow-zero Large teacher (100 : 50 : 10)
run_mode: grow-random Large teacher (100 : 50 : 10)
run_mode: baseline-large Large teacher (100 : 50 : 10)
run_mode: baseline-small Large teacher (100 : 50 : 10)
Small teacher (20 : 10 : 10)
20
Large teacher (100 : 50 : 10)
20
Name
20 visualized
run_mode: grow-zero
run_mode: grow-zero
5
run_mode: grow-random
run_mode: grow-random
5
run_mode: baseline-large
run_mode: baseline-large
5
run_mode: baseline-small
run_mode: baseline-small
5
State
Notes
User
Tags
Created
Runtime
Sweep
U1
U1_freeze
U2
U2_freeze
W1
W1_freeze
W2
W2_freeze
a_weight_1
a_weight_2
activation
b_weight_1
b_weight_2
batch_size
coefficients
convergence_epsilon
epochs
expand_epochs
expand_seed
expand_seeds
expansion_seed
expansion_seeds
expansion_steps
final_additional_steps
gpu
hidden_size
lr
mode
neurons_to_add
no_expand
no_preserve_parent_norm
noise_magnitude
noise_std
num_expansions
override_expand
override_original
rescale_to_magnitude
run_mode
run_type
save_to_dir
seed
step_size
student_seed
student_seeds
Finished
ltorroba
10m 11s
-
-
-
-
-
-
-
-
-
-
-
relu
-
-
-
-
0.1
-
-
-
-
-
-
500
1000
false
25
0.001
zero
5
-
false
-
-
5
-
-
-
grow-zero
zero
-
-
-
-
-
Finished
ltorroba
10m 11s
-
-
-
-
-
-
-
-
-
-
-
relu
-
-
-
-
0.1
-
-
-
-
-
-
500
1000
false
25
0.001
random
5
-
false
-
-
5
-
-
-
grow-random
random
-
-
-
-
-
Finished
ltorroba
10m 9s
-
-
-
-
-
-
-
-
-
-
-
relu
-
-
-
-
0.1
-
-
-
-
-
-
2500
1000
false
50
0.001
random
5
-
false
-
-
1
-
-
-
baseline-large
random
-
-
-
-
-
Finished
ltorroba
10m 4s
-
-
-
-
-
-
-
-
-
-
-
relu
-
-
-
-
0.1
-
-
-
-
-
-
2500
1000
false
25
0.001
random
5
-
false
-
-
1
-
-
-
baseline-small
random
-
-
-
-
-
1-4
of 4
Add a comment
Created with ❤️ on Weights & Biases.
https://wandb.ai/ltorroba/exploring-inverse-kd/reports/Replicates--VmlldzoxNDg5OTgz