Is there any <3B model with usable 200k+ context window?

“`html

• A user posted a query on Reddit asking for any 3B model with a usable context window of at least 200,000 tokens. They specifically wanted a low hallucination rate and did not require verbose outputs.
• The post highlighted the need for such a model in their interpretability project where they would only use pre-filled inputs without actually outputting any tokens from the model. This requirement was set based on the context window size being optimal for both speed and intelligence of the models they had tested so far.

“`

Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.

Please enable JavaScript in your browser to complete this form.

Name *

First

Last

Name Email

Email *

Submit