Skip to main content
bgiddwani
Projects
Hi_Pretraining
Reports
Hindi Lllama2 7B Pretraining
Log in
Sign up
Share
Comment
Star
Share
Comment
Star
Hindi Lllama2 7B Pretraining
Bharat Giddwani
Created on November 21
|
Last edited on November 21
Comment
A100 - BF16
validation_step_timing in s
validation_step_timing in s
500
1k
1.5k
2k
Step
0.1
0.2
0.3
0.4
val_loss
val_loss
500
1k
1.5k
2k
Step
3
4
5
6
7
reduced_train_loss
reduced_train_loss
0
500
1k
1.5k
2k
Step
4
6
8
10
12
train_step_timing in s
train_step_timing in s
0
500
1k
1.5k
2k
Step
2
2.05
2.1
2.15
2.2
2.25
2.3
consumed_samples
consumed_samples
0
500
1k
1.5k
2k
Step
20000
40000
60000
grad_norm
grad_norm
0
500
1k
1.5k
2k
Step
20
40
60
80
Run: hi-hgxa100-bf16-llama2-7b-event
1
Run set 2
0
Run set 3
0
Add a comment