Skip to main content
eleutherai
Projects
distilling
Run groups
9wxUSDTZNPCc7CqsQS2AWz_mnyoq8ma
Logs
Overview
Workspace
Runs
Logs
Files
Artifacts

Preetham-gali's group workspace

Timestamps visible
2021-08-22 21:38:32
%comms: 1.9424236218649988
2021-08-22 21:38:32
 %optimizer_step 0.666521968700206
2021-08-22 21:38:32
 %forward: 38.781747361068994
2021-08-22 21:38:32
 %backward: 45.95481704513912
2021-08-22 21:38:32
[2021-08-22 21:38:30,601] [INFO] [logging.py:60:log_dist] [Rank 0] rank=0 time (ms) | train_batch: 0.00 | batch_input: 183.93 | forward: 37249.37 | backward_microstep: 44143.09 | backward: 44139.01 | backward_inner_microstep: 44127.00 | backward_inner: 44122.45 | backward_allreduce_microstep: 5.72 | backward_allreduce: 1.98 | reduce_tied_grads: 0.35 | comms: 1865.67 | reduce_grads: 1536.40 | step: 640.19 | _step_clipping: 0.11 | _step_step: 637.71 | _step_zero_grad: 0.88 | _step_check_overflow: 0.62
2021-08-22 21:38:32
 samples/sec: 59.923 | iteration    17000/   25000 | elapsed time per iteration (ms): 9612.4 | learning rate: 7.423E-05 | approx flops per GPU: 120.9TFLOPS | loss: 4.209839E+00 | lm_loss: 3.311871E+00 | kld_loss: 8.979670E-01 | mse_loss: 0.000000E+00 | loss scale: 262144.0 | number of skipped iterations:   2 | number of nan iterations:   0 |
2021-08-22 21:38:32
time (ms)
2021-08-22 21:39:19
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2021-08-22 21:39:19
 validation loss at iteration 17000 | loss value: 4.207137E+00 | loss PPL: 6.716395E+01 | lm_loss value: 3.306602E+00 | lm_loss PPL: 2.729224E+01 | kld_loss value: 9.005342E-01 | kld_loss PPL: 2.460917E+00 | mse_loss value: 0.000000E+00 | mse_loss PPL: 1.000000E+00 |
2021-08-22 21:39:19
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------