**Editorial Brief**
I taught my 1B to follow instructions. It got worse at following instructions…
A recent post on Reddit by a user named GPUburnout describes an experiment where they trained three models from scratch—each with progressively larger parameters (1B, 2B, and 3B) using the same simple instruction-following method. The results were surprising: the smallest model (1B) actually performed worse after training compared to before.
This finding is noteworthy because it challenges a common assumption that increasing model size improves their ability to follow instructions. The post raises questions about why this might be happening and invites discussion on potential mechanisms behind such observations, especially at smaller scales where instruction-following performance seems diminished.
**Takeaways:**
– **Instruction-Following Performance Drops**: Smaller models may struggle with instruction following even after training.
– **Complex Mechanisms Involved**: Understanding what causes these shifts requires further investigation into the interaction between model size and instruction handling.
– **Need for More Research**: The Reddit post suggests that more research is needed to understand such phenomena, particularly in smaller language models.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
