Tencent's Hy4 Preview: 770B Params, No Vision, 1M Context
Hy4 Preview is Tencent's new open-weight, text-only large language model with 770B total parameters, 49B active parameters, and a 1M token context window.
What it is
Hy4 Preview is an open weight LLM from Chinese company Tencent, released as a preview announcement with weights available on Hugging Face at 1.56TB.
What it does
It accepts text input only, with no vision capability, using a mixture-of-experts architecture with 770B total parameters but only 49B active per forward pass, and a 1M token context window, a large jump in size from Tencent's previous Hy3 model (295B total, 21B active, 256,000 context) released in July.
Why it matters
The size jump shows Tencent scaling up mixture-of-experts sparsity rather than adding multimodal capability, a different bet than most competing labs chasing vision and omni-modal support in the same release cycle.
How to use it
Download the weights from Hugging Face if you need a large-context, text-only open model and have the hardware to run a 1.56TB checkpoint, likely via quantization for local use.