Moonshot AI Releases Kimi K3: A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context

Short summary

Moonshot AI released Kimi K3, a 2.8-trillion-parameter open mixture-of-experts model activating 16 of 896 experts. It introduces Kimi Delta Attention and Attention Residuals as architectural innovations and supports a 1-million-token context window. The article is a brief announcement with no benchmarks or deployment details.

• •Moonshot AI launches Kimi K3, a 2.8T-parameter open MoE model
• •Activates 16 of 896 experts with Kimi Delta Attention and 1M context
• •No benchmarks or technical deep-dive provided in the article

Generated with AI, which can make mistakes.