Skip to main content
bgiddwani
Projects
Hi_Pretraining
Reports
Hindi Lllama2 7B Pretraining
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Hindi Lllama2 7B Pretraining
Bharat Giddwani
Created on November 21
|
Last edited on November 21
Comment
H100 - BF16
validation_step_timing in s
validation_step_timing in s
200
400
600
800
1k
1.2k
1.4k
Step
0.05
0.1
0.15
val_loss
val_loss
200
400
600
800
1k
1.2k
1.4k
Step
4
5
6
7
reduced_train_loss
reduced_train_loss
0
200
400
600
800
1k
1.2k
1.4k
Step
4
6
8
10
12
14
train_step_timing in s
train_step_timing in s
0
200
400
600
800
1k
1.2k
1.4k
Step
0.9
1
1.1
1.2
consumed_samples
consumed_samples
0
200
400
600
800
1k
1.2k
1.4k
Step
10000
20000
30000
40000
50000
grad_norm
grad_norm
0
200
400
600
800
1k
1.2k
1.4k
Step
50
100
150
200
Run: hi-hgxh100-bf16-llama2-7b-event
1
Add a comment