homebrewnlp

Clashluke's group workspace

Group: ops-swapped-states-longctx

437

1-1

of 1

Timestamps visible

2023-05-17 09:21:40

343 | /optimizer:0//body_ctx:0/reversible:7/write:3/_output:7/scale_norm_act_linear:27/weight1/ema:0/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

344 | /optimizer:0//body_ctx:0/reversible:7/write:3/_output:7/scale_norm_act_linear:27/weight1/ema:1/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

345 | /optimizer:0//body_ctx:0/reversible:7/write:3/input_fn:7/scale_norm_act_linear:24/weight0/ema:0/momentum_buffer | (8, 512, 2048) | 33,554,432

2023-05-17 09:21:40

346 | /optimizer:0//body_ctx:0/reversible:7/write:3/input_fn:7/scale_norm_act_linear:24/weight0/ema:1/momentum_buffer | (8, 512, 2048) | 33,554,432

2023-05-17 09:21:40

347 | /optimizer:0//body_ctx:0/reversible:7/write:3/input_fn:7/scale_norm_act_linear:25/weight2/ema:0/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

348 | /optimizer:0//body_ctx:0/reversible:7/write:3/input_fn:7/scale_norm_act_linear:25/weight2/ema:1/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

349 | /optimizer:0//body_ctx:0/reversible:7/write:3/scale_norm_act_linear:26/weight1/ema:0/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

350 | /optimizer:0//body_ctx:0/reversible:7/write:3/scale_norm_act_linear:26/weight1/ema:1/momentum_buffer | (8, 2048, 512) | 33,554,432

2023-05-17 09:21:40

Parameters: 398,807,040

2023-05-17 09:21:40

Buffers: 797,614,080

2023-05-17 09:54:02

[ 16385/1073741824] Loss: 3.621 - Accuracy: 0.153 | LearningRate: 0.00000 | StepTime: 1944.092932s - Rate: 2,157.7 Tokens/s

2023-05-17 10:30:48

[ 32769/1073741824] Loss: 3.540 - Accuracy: 0.155 | LearningRate: 0.00000 | StepTime: 1686.831043s - Rate: 2,022.0 Tokens/s

2023-05-17 11:07:43

[ 49153/1073741824] Loss: 3.484 - Accuracy: 0.167 | LearningRate: 0.00000 | StepTime: 1686.855287s - Rate: 1,977.3 Tokens/s