fla Workspace – Weights & Biases

Skip to main content

Zaydzuhri's workspace

Runs

325

1-10

of 10

mtp_transformer-mtp.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_loss

mtp_transformer-mtp.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_ntp_loss

mtp_transformer-mtp.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_myopic_loss

mtp_transformer-mtp.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_top_loss

top_transformer-top.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_loss

top_transformer-top.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_ntp_loss

top_transformer-top.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_myopic_loss

top_transformer-top.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_top_loss

transformer-vanilla.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_loss

transformer-vanilla.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_ntp_loss

transformer-vanilla.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_myopic_loss

transformer-vanilla.code.1B.batch16.seqlen4096.context4096.warmup400.update1.steps40000.lr5e-5.cosine loss_metrics/global_avg_top_loss