Reward Ablation
Created on October 6|Last edited on October 7
Comment
Insight:
- Freeze 70 percent of layers seem to improve performance
- Sigmoid and cross entropy loss makes no difference (there is probably something about the formula itself)
no-deepspeed
3
freeze70percent
3
deepspeed
3
sigmoid freeze70percent
3
Add a comment