Skip to main content
ltorroba
Projects
exploring-inverse-kd
Reports
ReLU 4-->8; 8-->16 Runs

ReLU 4-->8; 8-->16 Runs

Lucas F Torroba Hennigen
Created on October 31|Last edited on October 31

Overall results

Results obtained over 10 runs for all possible combinations. Using ReLU.
Experimental settings: Adam with default settings; minibatches of size 32; training for 100 + 100 epochs.
Expansion: Start 4-->8 or 8-->16

Showing first 10 groups
050100150Epoch0.20.40.60.8Validation accuracy
run_type: U[copy, copy; FR:True] W[copy, copy; FR:True] val_accuracy_pretrain
run_type: U[copy, copy; FR:False] W[copy, copy; FR:False] val_accuracy_pretrain
run_type: U[copy, random; FR:False] W[copy, copy; FR:False] val_accuracy_pretrain
run_type: U[copy, random; FR:True] W[copy, random; FR:True] val_accuracy_pretrain
run_type: U[copy, random; FR:False] W[copy, random; FR:False] val_accuracy_pretrain
run_type: U[random, random; FR:False] W[copy, copy; FR:False] val_accuracy_pretrain
run_type: U[random, random; FR:False] W[copy, random; FR:False] val_accuracy_pretrain
run_type: U[random, random; FR:False] W[random, random; FR:False] val_accuracy_pretrain
run_type: U[copy, copy; FR:True] W[copy, copy; FR:True] val_accuracy_expand
run_type: U[copy, copy; FR:False] W[copy, copy; FR:False] val_accuracy_expand
run_type: U[copy, random; FR:False] W[copy, copy; FR:False] val_accuracy_expand
run_type: U[copy, random; FR:True] W[copy, random; FR:True] val_accuracy_expand
run_type: U[copy, random; FR:False] W[copy, random; FR:False] val_accuracy_expand
run_type: U[random, random; FR:False] W[copy, copy; FR:False] val_accuracy_expand
run_type: U[random, random; FR:False] W[copy, random; FR:False] val_accuracy_expand
run_type: U[random, random; FR:False] W[random, random; FR:False] val_accuracy_expand
4 --> 8
80
8 --> 16
80



Created with ❤️ on Weights & Biases.

https://wandb.ai/ltorroba/exploring-inverse-kd/reports/ReLU-4-8-8-16-Runs--VmlldzoxMTc0MDA5
Made with Weights & Biases. Sign up or log in to create reports like this one.