Tencent Hybrid Hy4 Flash released to support multiple modes and 1M context length, greatly improving task speed

Zhitongcaijing · 3d ago

The Zhitong Finance App learned that on November 16, Tencent Hybrid officially launched Hy4 Flash. This is the most cost-effective Hy model so far. The total number of parameters is 224B (including 64B Engram parameters), 12B activation parameters, and supports multiple modes and 1M context length. The API price is greatly reduced compared to Hy4.

pictures

Through a series of tasks, costs are optimized to the extreme while balancing intelligence and practicality:

In the pre-training phase, Hy4 Flash added architectural features such as blended attention, LatentMoe, and Engram. The KV cache size was only 1/24 of Hy3 and 1/6 of Hy4, and extensive scaling ladder experiments ensured that the effect met expectations;

In the post-training stage, a strong level of comprehensive intelligence was obtained through self-distillation of Hy4, and the length of the end-to-end task output was reduced by 50% through reinforcement learning;

In the evaluation phase, an internal evaluation set WorkIndex is constructed based on WorkBuddy's typical tasks, and combined with human reviews to focus on the product results of daily work, single-round output length, and end-to-end usage time, rather than a public list score far from daily work;

At the inference stage, combined with megakernel, 4-bit quantization, DFly and other methods, the Hy4 flash decoding speed exceeds 150 tps, and can be deployed and operated on a large scale at low cost.

In the end, Hy4 flash is slightly superior to Hy4 preview in terms of overall results; the cost of a single task on WorkIndex is only 1/13 of Hy4 preview, 1/20 of Hy4, and 1/8 of DeepSeek-v4.1-Flash-1017 (max range, busy time price).