DeepInfra raises $107M Series B to scale the inference cloud — read the announcement

tencent/

Hy4

$0.834

in

$2.501

out

$0.042

cached

/ 1M tokens

Hy4 is a 770B-parameter Mixture-of-Experts flagship model from the Tencent Hy Team, with 49B parameters activated per token, a 1M-token context window and image understanding. It uses Gated DeepSeek Sparse Attention with IndexCache, and improves on Hy4 preview in instruction following, long-context reasoning, long-horizon agentic tasks, computer use, coding and deep research.

Public
Zero retention
fp4
1,048,576
JSON
Function
Multimodal
ProjectLicense
tencent/Hy4 cover image
api
tencent/Hy4 cover image
Hy4

Ask me anything

0.00s

Settings

Model Information

Hy4

Model Introduction

Hy4 is a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team, supporting text and image understanding. The language backbone comprises 770B total parameters, of which 49B are activated per token. It consists of 78 layers, with a standard dense FFN in the first layer and MoE FFNs in the remaining 77 layers. Each MoE layer contains 256 routed experts and 1 shared expert; every token activates the top-8 routed experts along with the shared expert. In addition to the backbone, a 5-layer DFly2 drafter (3.3B parameters) is built in for speculative decoding.

On the architecture side, inspired by DeepSeek and GLM, the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse. The residual pathway uses iHC (identity Hyper-Connections) to expand inter-layer information flow. A ViT supports dynamic image resolutions, with an adapter converting image features into visual tokens that the language model processes alongside text tokens. DFly2, a parallel drafter, reuses Hy4's KV cache and sparse indices to support long-context speculative decoding at a low, constant cost.

Model Specifications

PropertyValue
ArchitectureMixture-of-Experts (MoE)
Total Parameters (excluding ViT, adapter and DFly2)770B
Activated Parameters per TokenText 49B + Vision 1.23B
DFly2 Drafter Parameters3.3B
Layers78 (1 dense, 77 MoE)
Hidden Size6144
Attention TypeGated DSA with IndexCache (MLA)
Attention Heads64
Query Compression Dimension2048
Key-Value Compression Dimension512
Indexer Heads / Head Dimension32 / 128
Indexer top-k2048
Residual Streams4 (iHC)
Routed Experts256
Shared Experts1
Activated Routed Experts per Token8
MoE Intermediate Size2048
FFN Intermediate Size18432
Vision Encoder40 layers, hidden size 1536, 16 × 16 patches
Context Length1M
Vocabulary Size120832

Key Improvements

Compared with Hy4 preview, Hy4 substantially scales up both pre-training and post-training. It adds vision capabilities and delivers major gains in instruction following, long-context understanding, long-horizon tasks, and deep research.

Capabilities and Use Cases

Vision and computer use: By combining perception, reasoning, and tool use, Hy4 understands fine-grained image details and interprets complex charts. It supports computer-use workflows across command-line and graphical interfaces, with targeted training on specialized software such as Blender and CAD and on long-horizon workflows spanning multiple applications.

Frontend and game development: Optimized for complex UI interactions, business logic, state management, and full-stack integration, so generated interfaces are functionally complete as well as visually polished. Built with professional game developers at Tencent IEG and adapted to Unreal Engine, Unity, and Godot, Hy4 improves on Hy4 preview by 19.1% on GameDev and 9.8% on GameLogic.

Finance and office workflows: Trained on thousands of realistic workspaces spanning fund research, equity analysis, quantitative strategies, asset allocation, corporate finance, and tax accounting. Across 579 real-world financial tasks rated by 213 finance professionals, Hy4 ranks first among open-weight models (3.04 out of 4, ahead of DeepSeek-V4.1-Flash-1017 at 2.97 and Kimi K3.1 Pro and GLM-5.3 at 2.68). Search upgrades for time-sensitive retrieval, multi-source synthesis, and fact checking cut the hallucination rate from 15.76% on Hy4 preview to 5.45%, and speculative errors from 23.1% to 5.6%.

Scientific research and AI R&D: Combines deep reasoning, scientific computing, and long-horizon execution. On TenureTrail, an internal benchmark of 123 research tasks across seven disciplines, Hy4 ranks first among open-weight models, with strong results in mathematical construction, physical modeling, earth sciences, systems engineering, and socioeconomic decision-making.

Long-horizon personal assistant: Given work context, Hy4 synthesizes chat history, emails, documents, calendars, and meeting notes to track project updates, surface action items, prepare meeting briefs, flag deadlines, and help prioritize tasks.

Usage Notes

  • Reasoning mode: Hy4 thinks by default (deep chain-of-thought for math, coding and complex tasks). Set reasoning_effort to none for direct responses without reasoning; any other value (low, medium, high, xhigh) keeps reasoning on.
  • Recommended sampling: temperature=0.9, top_p=1.0.
  • Images: one image per request.

Benchmark Appendix

Known Limitations

Although Hy4 is one of the strongest open-weight models available today, a notable gap remains between it and top proprietary models, particularly on challenging benchmarks for research-level mathematics, long-horizon coding, model self-improvement, and advanced visual question answering.

License

Hy4 is released under the Tencent Hy License 1.0, derived from the Apache License 2.0 with additional commercial terms for large Model-as-a-Service businesses. See LICENSE for details.