homebrewnlp

Clashluke's group workspace

Group: ops-longctx-bigfeatures

443

435

1-2

of 2

Timestamps visible

2023-05-17 11:28:11

349 | /optimizer:0//body_ctx:0/reversible:7/write:3/scale_norm_act_linear:26/weight1/ema:0/momentum_buffer | (8, 4096, 1024) | 134,217,728

2023-05-17 11:28:11

350 | /optimizer:0//body_ctx:0/reversible:7/write:3/scale_norm_act_linear:26/weight1/ema:1/momentum_buffer | (8, 4096, 1024) | 134,217,728

2023-05-17 11:28:11

Parameters: 1,569,361,920

2023-05-17 11:28:11

Buffers: 3,138,723,840

2023-05-17 11:33:01

[ 129/8388608] Loss: 4.683 - Accuracy: 0.095 | LearningRate: 0.00001 | StepTime: 291.181218s - Rate: 114.3 Tokens/s

2023-05-17 11:33:41

[ 257/8388608] Loss: 4.661 - Accuracy: 0.101 | LearningRate: 0.00003 | StepTime: 31.630414s - Rate: 200.4 Tokens/s

2023-05-17 11:34:15

[ 385/8388608] Loss: 4.469 - Accuracy: 0.090 | LearningRate: 0.00004 | StepTime: 31.632321s - Rate: 270.9 Tokens/s

2023-05-17 11:34:51

[ 513/8388608] Loss: 4.237 - Accuracy: 0.101 | LearningRate: 0.00005 | StepTime: 31.639036s - Rate: 328.3 Tokens/s

2023-05-17 11:35:27

[ 641/8388608] Loss: 3.999 - Accuracy: 0.099 | LearningRate: 0.00006 | StepTime: 31.634830s - Rate: 377.2 Tokens/s

2023-05-17 11:36:01

[ 769/8388608] Loss: 3.621 - Accuracy: 0.111 | LearningRate: 0.00007 | StepTime: 31.633724s - Rate: 418.9 Tokens/s

2023-05-17 11:36:35

[ 897/8388608] Loss: 3.517 - Accuracy: 0.124 | LearningRate: 0.00009 | StepTime: 31.642241s - Rate: 454.6 Tokens/s

2023-05-17 11:37:11

[ 1025/8388608] Loss: 3.473 - Accuracy: 0.145 | LearningRate: 0.00010 | StepTime: 31.647683s - Rate: 485.2 Tokens/s

2023-05-17 11:37:49

[ 1153/8388608] Loss: 3.447 - Accuracy: 0.154 | LearningRate: 0.00010 | StepTime: 31.643772s - Rate: 511.3 Tokens/s