Skip to main content
zhehuaic
Projects
canary-v0_speechllm
Reports
lora distributed fused adam

lora distributed fused adam

Zhehuai Chen
Created on October 1|Last edited on October 1

Section 1


050100150200250300Step024681012
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
01k2k3k4ktrainer/global_step0246
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
01k2k3k4ktrainer/global_step0123456
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
01k2k3k4ktrainer/global_step01234
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
050100150200250300Step0246
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
050100150200250300Step02468
prompt_lorac_lhmerge6_p2h_d5_oci_FC-GPT_gemma-2b-it-v1.1_canaryset_b6s4kf-sunolong_noCC_langtemp0.5_dsettemp0.5_lr1e-4wd1e-3_CosineAnnealing_warmup2500_minlr1e-6_gbs128_mbs16_ep200
Run set
462



Run set
462


Created with ❤️ on Weights & Biases.

https://wandb.ai/zhehuaic/canary-v0_speechllm/reports/lora-distributed-fused-adam--Vmlldzo5NTY5ODI0
Made with Weights & Biases. Sign up or log in to create reports like this one.