Tencent has released Hy3, a new large language model licensed under Apache 2.0. This 295 billion parameter Mixture-of-Experts system activates 21 billion parameters and includes 3.8 billion parameters in its MTP layers. The team developed the model after gathering feedback from over 50 products during a preview launch in late April. They then scaled up post-training using higher quality data. The final version outperforms similar sized models and rivals flagship open source models with two to five times the parameters. It demonstrates significant gains in utility across various products and productivity tasks.
The full sized model occupies 598GB on Hugging Face, while the FP8 quantized version takes 300GB. The system supports a context length of 256K tokens. Tencent made the model available for free on OpenRouter until July 21st. A test prompt asking for an SVG of a pelican riding a bicycle produced a flat style cartoon illustration with a white pelican and a large orange beak. The image shows the bird on a red bicycle against a pale blue background with gray motion lines suggesting speed.
* The model is available on OpenRouter until 21 July
* FP8 quantization reduces the file size to 300GB
* Context window capacity reaches 256K tokens
