**Editorial Brief**
I taught my 1B model to follow instructions, and it got worse at doing so. This finding is noteworthy because previous research has suggested that smaller models can benefit from instruction-following techniques like Self-Training (SFT), which improves their ability to adhere to given tasks. However, this study reveals an unexpected outcome: a 1 billion parameter model actually performed worse after SFT.
The results are summarized in the table above, showing that all three models-trained from scratch at 1B, 2B, and 3B parameters-saw their Instruction-Following Evaluation (IFEval) scores decrease. The 1B model saw a drop of -5.75 points, while the 2B model decreased by -4.91 points. Interestingly, the 3B parameter model actually improved slightly, increasing its IFEval score by +2.04 points.
This finding is significant because it challenges current understandings about how instruction-following techniques affect different sized models. It raises questions about whether smaller models are indeed better candidates for SFT and if there might be a threshold below which this technique fails to improve performance. Further investigation with the 2B model at a more aggressive learning rate, as well as broader experimentation across various models and tasks, will help clarify these observations.
**Takeaways:**
– Smaller models may not benefit from instruction-following techniques like SFT.
– There is likely an optimal size for models to see improvements through such methods.
– Further research is needed to understand the mechanisms behind this phenomenon.
