Skip to main content
ltorroba
Projects
exploring-inverse-kd
Reports
3-->4

3-->4

Lucas F Torroba Hennigen
Created on November 11|Last edited on November 11

Overall results

Results obtained over 10 runs for 4 most interesting combinations of run.
Experimental settings: Adam with default settings; minibatches of size 32; training for 100 + 100 epochs. Initial parameters are all sampled from N(0, 1).
Types of initialization: We add a tiny bit of noise to all the matrices to break ties
  • Random: Sample new matrix from N(0, 1)
  • Random adjusted: Sample from N(\mu, \sigma) where \mu is mean of parent matrix, \sigma is std of parent matrix
  • Permuted: Randomly shuffle entries in matrix
  • Copy: Copy matrix
Expansion: Start 4-->8 or 8-->16

Computing group metrics from first 50 groups
100120140160180Epoch0.20.30.40.50.60.70.8Validation accuracy
coefficients: [1,0,0] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.9,0.1,-2.7755575615628914e-17] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.9,0,0.09999999999999998] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.8,0.2,-5.551115123125783e-17] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.8,0.1,0.09999999999999996] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.8,0,0.19999999999999996] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.7000000000000001,0.30000000000000004,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.7000000000000001,0.2,0.09999999999999992] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.7000000000000001,0.1,0.19999999999999993] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.7000000000000001,0,0.29999999999999993] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.6000000000000001,0.4,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.6000000000000001,0.30000000000000004,0.09999999999999988] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.6000000000000001,0.2,0.1999999999999999] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.6000000000000001,0.1,0.29999999999999993] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.6000000000000001,0,0.3999999999999999] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0.5,0] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0.4,0.09999999999999998] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0.30000000000000004,0.19999999999999996] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0.2,0.3] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0.1,0.4] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.5,0,0.5] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.6000000000000001,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.5,0.09999999999999998] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.4,0.19999999999999996] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.30000000000000004,0.29999999999999993] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.2,0.4] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0.1,0.5] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.4,0,0.6] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.7000000000000001,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.6000000000000001,0.09999999999999988] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.5,0.19999999999999996] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.4,0.29999999999999993] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.30000000000000004,0.3999999999999999] 3 --> 4 (linear) val_accuracy_expand
coefficients: [0.30000000000000004,0.2,0.49999999999999994] 3 --> 4 (linear) val_accuracy_expand
coefficients: [1,0,0] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.9,0.1,-2.7755575615628914e-17] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.9,0,0.09999999999999998] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.8,0.2,-5.551115123125783e-17] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.8,0.1,0.09999999999999996] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.8,0,0.19999999999999996] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.7000000000000001,0.30000000000000004,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.7000000000000001,0.2,0.09999999999999992] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.7000000000000001,0.1,0.19999999999999993] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.7000000000000001,0,0.29999999999999993] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.6000000000000001,0.4,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.6000000000000001,0.30000000000000004,0.09999999999999988] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.6000000000000001,0.2,0.1999999999999999] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.6000000000000001,0.1,0.29999999999999993] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.6000000000000001,0,0.3999999999999999] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0.5,0] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0.4,0.09999999999999998] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0.30000000000000004,0.19999999999999996] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0.2,0.3] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0.1,0.4] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.5,0,0.5] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.6000000000000001,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.5,0.09999999999999998] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.4,0.19999999999999996] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.30000000000000004,0.29999999999999993] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.2,0.4] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0.1,0.5] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.4,0,0.6] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.7000000000000001,-1.1102230246251563e-16] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.6000000000000001,0.09999999999999988] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.5,0.19999999999999996] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.4,0.29999999999999993] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.30000000000000004,0.3999999999999999] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [0.30000000000000004,0.2,0.49999999999999994] 3 --> 4 (linear) val_accuracy_pretrain
coefficients: [1,0,0] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.9,0.1,-2.7755575615628914e-17] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.9,0,0.09999999999999998] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.8,0.2,-5.551115123125783e-17] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.8,0.1,0.09999999999999996] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.8,0,0.19999999999999996] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.7000000000000001,0.30000000000000004,-1.1102230246251563e-16] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.7000000000000001,0.2,0.09999999999999992] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.7000000000000001,0.1,0.19999999999999993] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.7000000000000001,0,0.29999999999999993] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.6000000000000001,0.4,-1.1102230246251563e-16] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.6000000000000001,0.30000000000000004,0.09999999999999988] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.6000000000000001,0.2,0.1999999999999999] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.6000000000000001,0.1,0.29999999999999993] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.6000000000000001,0,0.3999999999999999] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0.5,0] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0.4,0.09999999999999998] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0.30000000000000004,0.19999999999999996] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0.2,0.3] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0.1,0.4] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.5,0,0.5] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.6000000000000001,-1.1102230246251563e-16] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.5,0.09999999999999998] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.4,0.19999999999999996] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.30000000000000004,0.29999999999999993] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.2,0.4] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0.1,0.5] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.4,0,0.6] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.30000000000000004,0.7000000000000001,-1.1102230246251563e-16] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.30000000000000004,0.6000000000000001,0.09999999999999988] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.30000000000000004,0.5,0.19999999999999996] 3 --> 4 (relu) val_accuracy_expand
coefficients: [0.30000000000000004,0.4,0.29999999999999993] 3 --> 4 (relu) val_accuracy_expand
3 --> 4 (linear)
199
3 --> 4 (relu)
199




Created with ❤️ on Weights & Biases.

https://wandb.ai/ltorroba/exploring-inverse-kd/reports/3-4--VmlldzoxMjE0NDk2
Made with Weights & Biases. Sign up or log in to create reports like this one.