DeepInfra raises $107M Series B to scale the inference cloud — read the announcement
tencent/
$0.834
in
$2.501
out
$0.042
cached
/ 1M tokens
Hy4 is a 770B-parameter Mixture-of-Experts flagship model from the Tencent Hy Team, with 49B parameters activated per token, a 1M-token context window and image understanding. It uses Gated DeepSeek Sparse Attention with IndexCache, and improves on Hy4 preview in instruction following, long-context reasoning, long-horizon agentic tasks, computer use, coding and deep research.

Ask me anything
Settings
Hy4 is a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team, supporting text and image understanding. The language backbone comprises 770B total parameters, of which 49B are activated per token. It consists of 78 layers, with a standard dense FFN in the first layer and MoE FFNs in the remaining 77 layers. Each MoE layer contains 256 routed experts and 1 shared expert; every token activates the top-8 routed experts along with the shared expert. In addition to the backbone, a 5-layer DFly2 drafter (3.3B parameters) is built in for speculative decoding.
On the architecture side, inspired by DeepSeek and GLM, the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse. The residual pathway uses iHC (identity Hyper-Connections) to expand inter-layer information flow. A ViT supports dynamic image resolutions, with an adapter converting image features into visual tokens that the language model processes alongside text tokens. DFly2, a parallel drafter, reuses Hy4's KV cache and sparse indices to support long-context speculative decoding at a low, constant cost.
| Property | Value |
|---|---|
| Architecture | Mixture-of-Experts (MoE) |
| Total Parameters (excluding ViT, adapter and DFly2) | 770B |
| Activated Parameters per Token | Text 49B + Vision 1.23B |
| DFly2 Drafter Parameters | 3.3B |
| Layers | 78 (1 dense, 77 MoE) |
| Hidden Size | 6144 |
| Attention Type | Gated DSA with IndexCache (MLA) |
| Attention Heads | 64 |
| Query Compression Dimension | 2048 |
| Key-Value Compression Dimension | 512 |
| Indexer Heads / Head Dimension | 32 / 128 |
| Indexer top-k | 2048 |
| Residual Streams | 4 (iHC) |
| Routed Experts | 256 |
| Shared Experts | 1 |
| Activated Routed Experts per Token | 8 |
| MoE Intermediate Size | 2048 |
| FFN Intermediate Size | 18432 |
| Vision Encoder | 40 layers, hidden size 1536, 16 × 16 patches |
| Context Length | 1M |
| Vocabulary Size | 120832 |
Compared with Hy4 preview, Hy4 substantially scales up both pre-training and post-training. It adds vision capabilities and delivers major gains in instruction following, long-context understanding, long-horizon tasks, and deep research.
Vision and computer use: By combining perception, reasoning, and tool use, Hy4 understands fine-grained image details and interprets complex charts. It supports computer-use workflows across command-line and graphical interfaces, with targeted training on specialized software such as Blender and CAD and on long-horizon workflows spanning multiple applications.
Frontend and game development: Optimized for complex UI interactions, business logic, state management, and full-stack integration, so generated interfaces are functionally complete as well as visually polished. Built with professional game developers at Tencent IEG and adapted to Unreal Engine, Unity, and Godot, Hy4 improves on Hy4 preview by 19.1% on GameDev and 9.8% on GameLogic.
Finance and office workflows: Trained on thousands of realistic workspaces spanning fund research, equity analysis, quantitative strategies, asset allocation, corporate finance, and tax accounting. Across 579 real-world financial tasks rated by 213 finance professionals, Hy4 ranks first among open-weight models (3.04 out of 4, ahead of DeepSeek-V4.1-Flash-1017 at 2.97 and Kimi K3.1 Pro and GLM-5.3 at 2.68). Search upgrades for time-sensitive retrieval, multi-source synthesis, and fact checking cut the hallucination rate from 15.76% on Hy4 preview to 5.45%, and speculative errors from 23.1% to 5.6%.
Scientific research and AI R&D: Combines deep reasoning, scientific computing, and long-horizon execution. On TenureTrail, an internal benchmark of 123 research tasks across seven disciplines, Hy4 ranks first among open-weight models, with strong results in mathematical construction, physical modeling, earth sciences, systems engineering, and socioeconomic decision-making.
Long-horizon personal assistant: Given work context, Hy4 synthesizes chat history, emails, documents, calendars, and meeting notes to track project updates, surface action items, prepare meeting briefs, flag deadlines, and help prioritize tasks.
reasoning_effort to none for direct responses without reasoning; any other value (low, medium, high, xhigh) keeps reasoning on.temperature=0.9, top_p=1.0.
Although Hy4 is one of the strongest open-weight models available today, a notable gap remains between it and top proprietary models, particularly on challenging benchmarks for research-level mathematics, long-horizon coding, model self-improvement, and advanced visual question answering.
Hy4 is released under the Tencent Hy License 1.0, derived from the Apache License 2.0 with additional commercial terms for large Model-as-a-Service businesses. See LICENSE for details.
© 2026 DeepInfra. All rights reserved.