Moonshot AI’s Kimi K3 has spent the past two months quietly working its way from a GitHub release notice into mainstream cloud infrastructure, and the past three weeks have turned that slow build into the biggest open-weight AI story of the fall. The Beijing-based lab published Kimi K3 as an open-weight model in July 2026, but it is only now, with Amazon Bedrock listing the model on September 18 and Unsloth shipping an updated deployment guide on September 8, that Kimi K3 is becoming something enterprise teams outside China can actually run in production. TechJuice was among the first outlets to break down the model’s architecture, and the specs it described are still the numbers everyone in the open-weight community is arguing about.
At the center of the story is a single, striking figure: 1.56 terabytes. That is the size of the Kimi K3 checkpoint sitting on Hugging Face, split across 96 safetensors shards. It is one of the largest open-weight model releases to date, and it is forcing a fresh round of questions about what “open weights” even means when a single download is bigger than most people’s home internet connections can handle in a day.
What Moonshot AI Actually Released
Kimi K3, identified in Moonshot AI’s own repository as moonshotai/Kimi-K3, is a Mixture-of-Experts (MoE) language and vision model. According to the official GitHub repository and Hugging Face model card, Moonshot AI describes it as “our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.” That single sentence packs in most of the headline specs journalists have been repeating since the release: 2.8 trillion total parameters, native multimodal input, and a context window that stretches past one million tokens.
Crucially, only a fraction of those 2.8 trillion parameters activate for any given token. Moonshot AI’s documentation puts the activated parameter count at 104 billion per token, routed through 896 total experts of which 16 are selected per forward pass, plus two shared experts that process every token regardless of routing. That sparse-activation design is what makes a 2.8-trillion-parameter model computationally closer to a 100-billion-parameter dense model in practice, at least for inference cost, even though the full weight set still has to be stored and, in most deployment scenarios, loaded into memory.
TechJuice’s architecture breakdown and the model’s own technical documentation describe a 93-layer network, including 69 layers built on a new attention mechanism called Kimi Delta Attention (KDA) and 24 layers using Gated Multi-head Latent Attention (MLA). Vision understanding comes from a MoonViT-V2 encoder, reported at roughly 401 million parameters, bolted onto the language backbone rather than trained as a fully separate pipeline. The vocabulary sits at 160,000 tokens, on the larger end for a model this size, which typically helps with multilingual and code-heavy text.
Inside Stable LatentMoE: How the Architecture Actually Works
Moonshot AI calls the overall design “Stable LatentMoE,” and the name is a hint about what problem the lab was trying to solve. Mixture-of-Experts models are notoriously touchy to train at scale: routing collapse, where the network learns to funnel almost all traffic through a handful of favored experts, has sunk more than one ambitious MoE project. Moonshot’s own documentation, quoted on the Hugging Face model page, states plainly that Kimi K3 “is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), and scales up MoE sparsity with a Stable LatentMoE framework that activates 16 out of 896 experts.”
The mechanics of that routing are unusually granular for a public model description. A summary of the model’s technical paper, hosted on alphaXiv, notes that “the model contains 896 specialized sub-networks called experts, but each token only activates 16 of them.” Each of those sub-networks is thought to specialize in different types of patterns, whether that’s a particular language, a coding syntax, or a category of reasoning task, and the routing layer’s job is to send each token to the experts best suited to handle it without collapsing into a small clique of overused pathways.
Attention Residuals, the second architectural addition Moonshot highlights, appears aimed at a different failure mode: signal degradation across very deep networks. With 93 layers to pass information through, and a context window extending past a million tokens, preserving relevant signal from early layers and early tokens becomes harder the deeper and longer the sequence gets. Residual connections that specifically target attention outputs, rather than just the standard residual stream, are Moonshot’s stated fix. Independent researchers have not yet published a rigorous ablation study confirming how much each piece contributes, so for now these should be read as the lab’s own architectural claims rather than externally validated results.
The 1.56TB Question: Why Kimi K3’s Footprint Is So Large
The 1.56 terabyte figure has become the number most people fixate on, and it deserves some unpacking. That size reflects the full released checkpoint repository, not the number of parameters that activate during a single inference pass. A 2.8-trillion-parameter model, even stored in a compact format, is going to be enormous simply because there are 2.8 trillion numbers to write down.
What keeps that footprint from being even larger is Moonshot’s decision to natively train and release the model using MXFP4 weights, a 4-bit microscaling floating-point format, alongside MXFP8 activations. This is different from the more common practice of training a model in a higher-precision format like BF16 and then quantizing it down afterward for distribution. Training and shipping natively in a low-precision format is a bet that the model’s weights can tolerate that level of compression without the accuracy loss that post-hoc quantization sometimes introduces. If Moonshot’s approach holds up under independent testing, it would mean Kimi K3 was designed from day one to be more deployable than a same-sized model trained in full precision and quantized as an afterthought.
Practically, though, 1.56TB is still 1.56TB. Downloading the full repository requires either a very fast connection and patience, or reliance on a hosted API rather than local deployment. That single fact is arguably doing more to shape how Kimi K3 actually gets used in the wild than any of its architectural innovations: most teams outside a handful of well-resourced labs and cloud providers are going to interact with Kimi K3 through a hosted endpoint, not a local download, which is exactly the gap Amazon Bedrock’s September listing appears aimed at closing.
Kimi K3 Architecture Specs at a Glance
| Spec | Kimi K3 Value |
|---|---|
| Architecture | Mixture-of-Experts (Stable LatentMoE) |
| Total parameters | 2.8 trillion |
| Activated parameters per token | 104 billion |
| Total experts / selected per token | 896 experts, 16 selected, plus 2 shared |
| Layers | 93 total (69 KDA, 24 Gated MLA) |
| Context window | 1,048,576 tokens |
| Vocabulary | 160,000 tokens |
| Vision encoder | MoonViT-V2, ~401 million parameters |
| Native precision | MXFP4 weights, MXFP8 activations |
| Hugging Face repository size | Approximately 1.56 TB across 96 safetensors shards |
From GitHub to Amazon Bedrock: How Kimi K3 Reached the Cloud
The timeline matters here because it explains why a July release is generating fresh headlines in late September. Moonshot AI’s GitHub and Hugging Face repositories for Kimi K3 went live in July 2026, and TechJuice’s architecture report followed within days, on July 28. For the next month, coverage of Kimi K3 was largely confined to AI-focused outlets and the open-source ML community poring over the model card.
That changed as tooling caught up. Unsloth, a popular library for fine-tuning and running open-weight models efficiently on consumer and prosumer hardware, published an updated Kimi K3 usage guide on September 8, giving developers without access to a multi-GPU cluster a more realistic path to experimenting with the model. Ten days later, on September 18, Unite.AI reported that Kimi K3 had arrived on Amazon Bedrock, Amazon Web Services’ managed platform for foundation models. That listing matters because it moves Kimi K3 from “download a 1.56TB repository and manage your own inference stack” to “call an API,” which is the difference between a model living in research circles and a model showing up in production roadmaps at ordinary enterprises.
It is worth being precise about what remains unconfirmed even after the Bedrock listing. Public reporting has not yet established an official per-token price for Kimi K3 on Bedrock or through any other hosted channel, and no standardized benchmark table, covering evaluations like MMLU, GPQA, AIME, SWE-bench, or LiveCodeBench, has surfaced with independently verified scores. Readers comparing Kimi K3 against rivals on cost or raw benchmark performance should treat any specific numbers circulating online with caution until an official pricing page or a peer-reviewed evaluation appears.
Kimi K3 Rollout Timeline
| Date | Milestone | Source |
|---|---|---|
| July 2026 | Kimi K3 open weights published on GitHub and Hugging Face | Moonshot AI official repositories |
| July 28, 2026 | Architecture and specification breakdown published | TechJuice |
| September 8, 2026 | Updated local deployment and fine-tuning guide released | Unsloth |
| September 18, 2026 | Kimi K3 listed as available on Amazon Bedrock | Unite.AI |
From Kimi K2 to Kimi K3: What Actually Changed
Moonshot AI’s own repository frames Kimi K3 as a direct successor to Kimi K2, its previous flagship open-weight release, and claims that the Stable LatentMoE framework delivers roughly a 2.5x improvement in overall scaling efficiency over Kimi K2. That figure comes from Moonshot’s own documentation rather than an independently reproduced benchmark, so it should be read as the lab’s stated design goal rather than a confirmed, third-party-verified result. Still, the direction of travel is consistent with what the rest of the open-weight field has been doing over the past year: rather than simply scaling total parameter counts further, labs are spending more engineering effort on making sparse activation and long-context retrieval more stable and efficient per unit of compute.
A technical analysis from Embedl frames Kimi K3 as a departure from a pure scaling playbook, describing it as “a 2.8-trillion-parameter open model with a 1-million-token context window, native vision, and two architectural bets that go beyond scaling up the usual recipe.” That framing, from a technical analyst rather than Moonshot itself, is one of the more useful independent read-outs available so far, precisely because it treats the KDA and AttnRes additions as genuine research bets rather than marketing copy.
How Kimi K3 Fits Into the Open-Weight Race
Kimi K3 does not exist in a vacuum. It lands in a field crowded with other large, sparsely-activated open-weight models from Chinese labs racing to match, and in some benchmarks claim to beat, closed frontier models from US labs. Readers who want detailed price and benchmark comparisons between Kimi K3 and rivals like Qwen3.8-Max and GLM-5.2 can find that breakdown in our earlier coverage of Kimi K3 vs Qwen3.8-Max vs GLM-5.2 and GLM-5.2 vs Qwen3.8-Max vs Kimi K3, both of which focus on the commercial side of that competition. This piece is concerned with what changed architecturally and how the model is actually reaching users, which is a different, and in some ways more consequential, story than a price table.
What is notable is the pattern across that broader field: models like DeepSeek’s V4 family, covered in our DeepSeek V4.1 Flash vs V4 Pro vs V4-Flash comparison, and Western open-weight entries such as those detailed in our Granite vs Mistral Large 3 vs Falcon H1 roundup, are all converging on the same broad architecture family: sparse MoE, long context windows, and native low-precision training. Kimi K3’s contribution to that trend is less about being bigger and more about the specific mechanisms, Kimi Delta Attention and Attention Residuals, it introduces to keep a model of this scale trainable and stable.
On the proprietary side, the pressure is coming from a different direction. Enterprises evaluating whether to build on open weights or a hosted frontier model are increasingly comparing total cost and control, a tension our ChatGPT vs Claude vs Gemini vs Grok pricing analysis lays out for the consumer and prosumer tier. Kimi K3’s open-weight status, and now its Bedrock availability, gives procurement teams a genuine middle path: the flexibility of self-hosting with the operational convenience of a managed API, at least for the subset of workloads where a 104-billion-active-parameter model is sufficient.
Market Impact: Why This Matters Beyond the Spec Sheet
The Amazon Bedrock listing is the part of this story with the clearest near-term market impact. AWS does not add every open-weight model that gets released to Bedrock, and a listing signals that AWS’s own infrastructure team judged Kimi K3 worth the engineering investment to support at scale, including whatever quantization and serving optimizations are needed to make a 2.8-trillion-parameter MoE model run efficiently in a shared cloud environment. For enterprise buyers who were unwilling to stand up their own inference infrastructure for a model this large, that listing is the practical trigger that turns “interesting open-weight release” into “model we can actually evaluate for a project.”
There is a second, quieter effect worth watching: what Kimi K3’s release does to the broader conversation about how “open” an open-weight release really is when the checkpoint is 1.56TB. A model that large is, in practice, only fully reproducible and fine-tunable by organizations with serious storage and compute budgets, even though the weights themselves carry a permissive license. That dynamic has already shaped tooling investment, evident in Unsloth’s decision to prioritize a Kimi K3 guide, and it is likely to shape how the next generation of open-weight releases balances scale against genuine accessibility.
What’s Still Unverified
It is worth being explicit about the gaps in public reporting, because a lot of the noise around Kimi K3 online fills them in with numbers that have not actually been confirmed. As of this writing, there is no independently verified, standardized benchmark table for Kimi K3 covering common evaluations such as MMLU, GPQA, AIME, SWE-bench, or LiveCodeBench. There is also no confirmed official price per million tokens, either through Amazon Bedrock or any other hosted channel. Moonshot’s claimed 2.5x scaling-efficiency improvement over Kimi K2 comes from the lab’s own repository rather than a third-party reproduction. Any comparison chart claiming to rank Kimi K3 against DeepSeek, Qwen, or GLM on head-to-head benchmark scores should be treated skeptically until a primary source publishes those figures.
Historical Context: Moonshot AI’s Rapid Ascent
Moonshot AI has moved fast even by the standards of an industry defined by fast movement. Kimi K2 established the lab as a serious open-weight competitor earlier in the current model generation, and Kimi K3 arrived as a direct architectural successor rather than an incremental patch. The lab’s pattern, releasing large, research-forward models under open licenses rather than keeping architecture details proprietary, has made it a frequent subject of comparison articles on this site, from Kimi K2.6 vs Llama 4 vs Qwen3-Coder to the newer Kimi K3 comparisons linked above. That publishing cadence, and the willingness to detail architectural mechanisms like Kimi Delta Attention in public documentation, has become one of Moonshot’s clearest points of differentiation from labs that publish benchmark claims without comparable architectural transparency.
Predictions: Where Kimi K3 Goes From Here
More hosted availability follows Bedrock. Once one major cloud provider lists a model this size, competing platforms typically move to avoid ceding developer mindshare, so additional hosted endpoints for Kimi K3 beyond Amazon Bedrock are a reasonable near-term expectation.
Independent benchmark tables will start to circulate. With Unsloth’s guide lowering the barrier to running Kimi K3 locally, community-run evaluations on standard benchmark suites are likely to appear over the coming weeks, even before Moonshot or a major research group publishes an official scorecard.
Quantized, smaller derivatives are likely. A 1.56TB checkpoint all but guarantees that the fine-tuning community will produce more aggressively compressed or distilled versions aimed at single-GPU or consumer-hardware deployment, following the same pattern seen with other outsized open-weight releases.
Kimi Delta Attention gets tested outside Moonshot’s own models. If KDA and Attention Residuals hold up under outside scrutiny, expect other labs to experiment with similar attention variants in their own architectures within the next model generation, the way earlier attention innovations have propagated across the field.
Pricing transparency will become a competitive pressure point. As rival open-weight labs publish clearer per-token pricing for their hosted offerings, the absence of a confirmed official price for Kimi K3 is likely to become a point of friction that pushes Moonshot or its cloud partners toward publishing a formal rate card.
Frequently Asked Questions
What is Kimi K3?
Kimi K3 is an open-weight Mixture-of-Experts language and vision model released by Moonshot AI, with 2.8 trillion total parameters and 104 billion activated per token, according to the model’s official GitHub and Hugging Face documentation.
When was Kimi K3 released?
Moonshot AI published Kimi K3’s open weights in July 2026, with TechJuice publishing a detailed architecture report on July 28, 2026.
Why is the Kimi K3 file size 1.56TB?
The Hugging Face repository stores the full 2.8-trillion-parameter checkpoint across 96 safetensors shards using native MXFP4 weights, resulting in a total repository size of approximately 1.56 terabytes.
Is Kimi K3 available on Amazon Bedrock?
Yes. Unite.AI reported that Kimi K3 became available on Amazon Bedrock on September 18, 2026, giving developers a hosted API option instead of managing the full checkpoint themselves.
How many experts does Kimi K3 use?
Kimi K3 uses 896 total experts, activating 16 per token plus two shared experts that process every token, according to Moonshot AI’s official documentation and a technical paper summary hosted on alphaXiv.
What is Kimi Delta Attention?
Kimi Delta Attention (KDA) is one of two architectural additions, alongside Attention Residuals, that Moonshot AI built into Kimi K3’s Stable LatentMoE framework to help stabilize training and inference at this scale.
Does Kimi K3 have official benchmark scores?
No independently verified, standardized benchmark table for evaluations like MMLU, GPQA, AIME, SWE-bench, or LiveCodeBench had been published as of this report. Moonshot’s claimed 2.5x scaling-efficiency gain over Kimi K2 is the lab’s own figure, not a third-party-confirmed result.
How does Kimi K3 compare to Kimi K2?
Kimi K3 is Moonshot AI’s direct successor to Kimi K2, introducing the Stable LatentMoE framework, Kimi Delta Attention, and Attention Residuals, which Moonshot’s own documentation credits with a roughly 2.5x improvement in scaling efficiency over its predecessor.