Comment
Detailed description is in progress...
Run set
2335
Name
2,335 visualized
State
Notes
User
Tags
Created
Runtime
Sweep
loss_metrics/global_avg_loss
loss_metrics/global_max_loss
lr/opt_0/group_0
lr/opt_0/group_1
lr/opt_0/group_2
memory/max_active(%)
memory/max_active(GiB)
memory/max_reserved(%)
memory/max_reserved(GiB)
memory/num_alloc_retries
memory/num_ooms
mfu(%)
tflops
throughput(tps)
time_metrics/data_loading(%)
time_metrics/data_loading(s)
time_metrics/end_to_end(s)
track_param_condition_number/model_part_0/layers.0.attention.wk
track_param_condition_number/model_part_0/layers.0.attention.wo
track_param_condition_number/model_part_0/layers.0.attention.wq
track_param_condition_number/model_part_0/layers.0.attention.wv
track_param_condition_number/model_part_0/layers.0.feed_forward.w1
track_param_condition_number/model_part_0/layers.0.feed_forward.w2
track_param_condition_number/model_part_0/layers.0.feed_forward.w3
track_param_condition_number/model_part_0/layers.1.attention.wk
track_param_condition_number/model_part_0/layers.1.attention.wo
track_param_condition_number/model_part_0/layers.1.attention.wq
track_param_condition_number/model_part_0/layers.1.attention.wv
track_param_condition_number/model_part_0/layers.1.feed_forward.w1
track_param_condition_number/model_part_0/layers.1.feed_forward.w2
track_param_condition_number/model_part_0/layers.1.feed_forward.w3
track_param_condition_number/model_part_0/layers.10.attention.wk
track_param_condition_number/model_part_0/layers.10.attention.wo
track_param_condition_number/model_part_0/layers.10.attention.wq
track_param_condition_number/model_part_0/layers.10.attention.wv
track_param_condition_number/model_part_0/layers.10.feed_forward.w1
track_param_condition_number/model_part_0/layers.10.feed_forward.w2
track_param_condition_number/model_part_0/layers.10.feed_forward.w3
track_param_condition_number/model_part_0/layers.100.attention.wk
track_param_condition_number/model_part_0/layers.100.attention.wo
track_param_condition_number/model_part_0/layers.100.attention.wq
track_param_condition_number/model_part_0/layers.100.attention.wv
track_param_condition_number/model_part_0/layers.100.feed_forward.w1
track_param_condition_number/model_part_0/layers.100.feed_forward.w2
Finished
-
ofivite
19s
-
4.12969
5.03049
0.70711
0.70711
0.70711
72.00524
28.43811
73.18561
28.9043
0
0
15.73344
49.08834
20249.48956
3.4818
0.028171
0.80911
21563.88281
4351.05322
85980.78906
581.68311
753.26068
747.01184
761.17084
2674.08936
1166.73425
1436.33203
18115.44336
646.90619
690.73535
637.26819
2651.9978
3271.31006
4184.54297
1283.18921
649.633
678.86121
696.03613
3168.36621
1070.45898
1524.93628
13163.39355
650.57227
1878.32056
Finished
-
ofivite
17s
-
4.29059
4.76435
0.70711
0.70711
0.70711
72.00524
28.43811
73.07682
28.86133
0
0
12.42898
38.77843
15996.53453
3.2466
0.033252
4.09689
15124.49902
40221.61328
12382.71191
10011.37988
328.84348
465.88019
315.82184
13821.28418
62086.66797
13638.45313
7618.0166
269.44272
379.73825
294.83069
25685.60156
33490.77734
32384.71289
19565.09961
228.81108
405.28619
247.40347
60832.03125
32860.34766
129340.5625
33113.96484
414.64615
1100.41565
Finished
-
ofivite
20s
-
4.2787
4.54174
0.70711
0.70711
0.70711
72.10453
28.47733
73.45266
29.00977
0
0
17.30984
54.0067
22278.36814
3.63497
0.026732
0.73542
15969.4209
646.60754
15819.94727
2322.72363
927.03833
1103.62866
883.16412
1322.90149
1761.02234
9282.63379
3190.22119
880.43512
1026.55688
863.10559
1159.96167
513.81177
877.21051
1998.04089
667.20593
818.67578
758.35052
9021.44629
3263.34888
7327.73096
1932.08728
789.37299
1967.67151
Finished
-
ofivite
17s
-
4.08447
5.20354
0.70711
0.70711
0.70711
72.00524
28.43811
73.07682
28.86133
0
0
12.03476
37.54846
15489.15966
3.13218
0.033131
2.11554
23230.36133
31579.0957
98251.92188
6259.46533
428.2319
545.84595
457.33456
5759.04492
14218.79883
2227.43848
18885.2793
400.97186
588.39679
428.2099
4124.44336
3796.64453
5614.00977
2519.97168
344.20108
506.43063
381.21408
282001.6875
23059.39648
52535.33984
79165.11719
412.39203
1306.09302
Finished
-
ofivite
16s
-
4.1219
5.83213
0.70711
0.70711
0.70711
71.95559
28.41851
73.07682
28.86133
0
0
11.4749
35.8017
14768.60119
2.91352
0.032322
1.10938
3853.09546
731.4563
15303.96387
6901.0708
528.80487
618.23035
550.0495
2627.64331
674.70258
3370.13135
1805.77332
448.16205
578.71326
460.76184
2271.74268
27287.22266
1692.58875
788.31958
478.70383
513.17212
487.76974
62422.60547
16965.1875
22134.38086
121273.67969
433.73566
1521.37659
Finished
-
ofivite
17s
-
4.94326
5.49311
0.70711
0.70711
0.70711
72.00524
28.43812
73.07682
28.86133
0
0
12.71547
39.67227
16365.2574
3.37603
0.033799
8.00916
29365.92969
6691.03271
32999.5625
30980.64453
226.07637
290.60153
233.69481
4132.5376
7864.13916
33502.53125
5386.71289
196.01967
331.70215
210.8885
19058.9707
32190.78906
48834.95703
3262.46436
146.95076
320.37665
164.15259
69378.65625
47088.81641
34583.85547
575552.0625
231.87151
823.78802
Finished
-
ofivite
17s
-
3.99687
4.87884
0.17678
0.17678
0.17678
72.00524
28.43811
73.18561
28.9043
0
0
15.94664
49.75353
20523.88638
3.58604
0.028627
0.79829
8871.36719
271.85889
306419.875
1075.76367
152.03525
378.93307
144.09776
7770.27734
847.34918
3204.41431
9994.79297
150.60434
441.06625
169.47031
2690.70752
1859.45532
52856.28906
997.72308
143.52956
448.83401
160.46092
51660.6875
2271.72192
10361.89844
10867.60352
196.73453
993.54175
Finished
-
ofivite
17s
-
4.20825
4.67946
0.17678
0.17678
0.17678
72.00524
28.43811
73.07682
28.86133
0
0
12.43482
38.79664
16004.05023
3.2097
0.032859
4.09496
159353.45313
719.25037
30695.93164
2584.92651
67.61385
226.22697
73.28381
9620.56445
2273.57129
6860.51758
19658.70117
68.34104
286.21063
68.72919
1119.82031
919.06763
8386.22168
9272.99707
60.63507
364.02106
65.54527
10358.43066
77302.49219
14057.8877
6098566.5
113.41306
1040.33362
Finished
-
ofivite
20s
-
4.1109
4.36574
0.17678
0.17678
0.17678
72.10453
28.47733
73.45266
29.00977
0
0
17.65809
55.09323
22726.5728
3.70761
0.026729
0.72092
11974.41602
3484.10889
5519.72754
985.64368
181.62047
503.54364
178.35719
3928.28638
589.69379
1942.48547
523.23688
191.26884
467.43188
172.58302
2169.86841
2853.32227
2526.15088
1278.89929
154.10367
464.97818
180.22304
6627.11914
1045.79114
1877.91919
2351.84595
227.7491
1019.43964
Finished
-
ofivite
17s
-
4.03215
5.06616
0.17678
0.17678
0.17678
72.00524
28.43811
73.07682
28.86133
0
0
12.14474
37.89159
15630.70599
3.12828
0.03279
2.09639
83914.35156
753.51117
5624.58203
1467.79785
77.62476
242.85716
92.39297
1268.06177
644.51099
2212.00757
835.5
92.66469
363.07565
99.90571
2049.31494
572.57471
2298.27637
843.11725
96.46036
493.70297
97.51347
91589.0625
26562.11523
81702
226035.45313
146.35036
1215.94556
1-10
of 2,335
Add a comment
Created with ❤️ on Weights & Biases.
https://wandb.ai/sdlaml-llm/norm-transfer/reports/Norm-Transfer--VmlldzoxNDYwNjE2Mw