Skip to main content
bgiddwani
Projects
Hi_Pretraining
Reports
Hindi Lllama2 7B Pretraining
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Hindi Lllama2 7B Pretraining
Bharat Giddwani
Created on November 21
|
Last edited on November 21
Comment
H100 - FP8
validation_step_timing in s
validation_step_timing in s
200
400
600
800
Step
0.1
0.2
0.3
0.4
val_loss
val_loss
200
400
600
800
Step
3.5
4
4.5
5
5.5
6
6.5
reduced_train_loss
reduced_train_loss
0
200
400
600
800
Step
4
6
8
10
12
train_step_timing in s
train_step_timing in s
0
200
400
600
800
Step
0.8
0.9
1
consumed_samples
consumed_samples
0
200
400
600
800
Step
5000
10000
15000
20000
25000
30000
35000
grad_norm
grad_norm
0
200
400
600
800
Step
20
40
60
80
Run: hi-hgxh100-fp8-llama2-7b-event
1
Add a comment