Is there any <3B model with usable 200k+ context window?

“`html

• A user on Reddit is looking for a small, 3B model that can handle a context window of at least 200k tokens. This is needed for processing conversation transcripts from larger models in their interpretability project.
• The user emphasizes the importance of having a low hallucination rate and not being overly verbose, as their work involves prefill without actual token output. They are particularly interested in whether Qwen 3.5-2B can meet these requirements.

“`

Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.

Please enable JavaScript in your browser to complete this form.

Email Name

Name *

First

Last

Email *

Submit