OLMoE 8x1B vs OLMo 7B

Created on July 8|Last edited on August 29
Comment
﻿
﻿
eval/c4_en-validation/CrossEntropyLoss
eval/c4_en-validation/CrossEntropyLoss
200k400k600k800k1MStep22.22.42.62.83
olmoe-8x1b-newhp-newds-final   Run set
olmoe-8x1b-newhp-newds-final   Run set
olmoe-8x1b-newhp-newds-final   Run set
OLMo-1.7-7B-run-025   Run set 3
OLMo-1.7-7B-run-024   Run set 3
OLMo-1.7-7B-run-023   Run set 3
OLMo-1.7-7B-run-022   Run set 3
OLMo-1.7-7B-run-021   Run set 3
OLMo-1.7-7B-run-020   Run set 3
OLMo-1.7-7B-run-019   Run set 3
OLMo-1.7-7B-run-018   Run set 3
OLMo-1.7-7B-run-017   Run set 3
OLMo-1.7-7B-run-016   Run set 3
OLMo-1.7-7B-run-013   Run set 3
OLMo-1.7-7B-run-010   Run set 3
OLMo-1.7-7B-run-008   Run set 3
OLMo-1.7-7B-run-007   Run set 3
OLMo-1.7-7B-run-006   Run set 3
OLMo-1.7-7B-run-005   Run set 3
OLMo-1.7-7B-run-004   Run set 3
OLMo-1.7-7B-run-003   Run set 3
OLMo-1.7-7B-run-001   Run set 3
train/CrossEntropyLoss
train/CrossEntropyLoss
0200k400k600k800k1MStep22.22.42.62.83
OLMo-1.7-7B-run-025   Run set 3
OLMo-1.7-7B-run-024   Run set 3
OLMo-1.7-7B-run-023   Run set 3
OLMo-1.7-7B-run-022   Run set 3
OLMo-1.7-7B-run-021   Run set 3
OLMo-1.7-7B-run-020   Run set 3
OLMo-1.7-7B-run-019   Run set 3
OLMo-1.7-7B-run-018   Run set 3
OLMo-1.7-7B-run-017   Run set 3
OLMo-1.7-7B-run-016   Run set 3
OLMo-1.7-7B-run-013   Run set 3
OLMo-1.7-7B-run-012   Run set 3
OLMo-1.7-7B-run-011   Run set 3
OLMo-1.7-7B-run-010   Run set 3
OLMo-1.7-7B-run-009   Run set 3
OLMo-1.7-7B-run-008   Run set 3
OLMo-1.7-7B-run-007   Run set 3
OLMo-1.7-7B-run-006   Run set 3
OLMo-1.7-7B-run-005   Run set 3
OLMo-1.7-7B-run-004   Run set 3
OLMo-1.7-7B-run-003   Run set 3
OLMo-1.7-7B-run-002   Run set 3
OLMo-1.7-7B-run-001   Run set 3
olmoe-8x1b-newhp-newds-final   Run set
olmoe-8x1b-newhp-newds-final   Run set
olmoe-8x1b-newhp-newds-final   Run set
Run set3
Run set 2
Run set 325
﻿
﻿
Add a comment