AI self-hosting quantization llama.cpp Mistral open weights Kimi inference economics

Mistral Large 4 vs Kimi K3.1 Pro Self-Hosting: What 1T vs 3T Open Weights Actually Cost

Posted by Reda Fornera on 2026-11-09
Estimated Reading Time 22 Minutes
Words 3.6k In Total

October and November 2026 dropped two trillion-class open-weight models within roughly a week of each other. If you’re weighing Mistral Large 4 vs Kimi K3.1 Pro for self-hosting, the deciding factors aren’t benchmark scores — they’re the memory bill, the quant ladder, engine maturity, the license toll, and the break-even math. A previous showdown post covered the capability axis of this matchup; this one is the second axis: what it actually costs in memory, tooling maturity, licensing toll, and break-even math to run these things on hardware you control.

Data center aisle lined with illuminated server racks — generic stock photo representing the GPU-node scale at stake in the Mistral Large 4 vs Kimi K3.1 Pro self-hosting comparison, not the actual hardware of either vendor

Moonshot’s Kimi K3.1 Pro hit the API on October 23 (the vendor’s announcement post is dated Oct 23), and its weights landed on Hugging Face on November 2 — 96 ungated MXFP4 shards, 1.56 TB. Mistral’s Large 4, the “Le Chonk” checkpoint, went open weight on October 29 per the date on Mistral’s announcement post: 54 FP8 shards totaling 1.06 TB on Hugging Face, plus an official NVFP4 checkpoint.

Both models speak a 1-million-token context window. Both reason on every request. And both are “free” (downloads and licenses permitting) — which is exactly where the comparison gets interesting. One recap line before we dive in:

Mistral Large 4 Kimi K3.1 Pro
Total / active params ~1.05T total / 52B active (49B excluding embeddings) 2.8T total / 104B activated
Architecture Granular MoE: 128 routed experts, 4 active/token + 1 shared MoE: 896 experts, 16 selected + 2 shared, 93 layers (69 KDA + 24 Gated DSA)
Context window 1M 1,048,576
Vision Pixtral-style encoder (1.6B per Mistral’s docs page) MoonViT-V2, 401M
Native quant Block-scaled FP8 official checkpoint + official NVFP4 MXFP4 weights / MXFP8 activations, trained quantization-aware
Reasoning effort low / medium / high (default high) low / high / max (default max)

If you only read one thing here, make it the license section — the asymmetry there is the most overlooked part of this story.

The native-quant divergence: FP8 + NVFP4 + community GGUF vs MXFP4-only

This is the single most consequential difference for anyone self-hosting Mistral Large 4 or Kimi K3.1 Pro, and it cuts against intuition: the smaller-memory model is the harder one to shrink.

Mistral shipped a ladder. The FP8 master checkpoint runs 1.061 TB across 54 shards (sizes summed from the Hugging Face Hub API). Mistral also published an official NVFP4 checkpoint targeting Blackwell GPUs that the model card describes as same-performance, and the open-source ecosystem did the rest within days: ggml-org maintains an official GGUF repo (ggml-org/Mistral-Large-4-1T-A52B-GGUF) with a five-step ladder from near-lossless down to 298 GB, and Unsloth published Dynamic GGUFs on what their docs call “day zero.” (For the full weights-and-shards timeline, see our Mistral Large 4 open weights on Hugging Face walkthrough.)

Kimi shipped a floor — as a feature and a constraint. K3.1 Pro exists only as MXFP4 weights (with MXFP8 activations), a product of quantization-aware training. There is no BF16 or FP8 build to fall back on, and the model card doesn’t hint at one. As ComputingForGeeks’ checkpoint forensics put it (their checks are dated early November 2026): “The 4-bit checkpoint is the native model.” A practical consequence both ways: your memory bill is capped at 1.56 TB because no 16-bit version exists to inflate it, but if you wanted higher fidelity than 4-bit — the purist argument that QAT at 4 bits might have traded away something — there’s nothing to switch to. As of this writing, no independent evaluation exists on whether 4-bit quantization-aware training degraded K3.1 Pro’s mid- or long-context quality. Don’t assume either direction.

The GGUF ecosystem tells the clearest story. Mistral gets a full community ladder; K3.1 Pro, as of November 9, has only experimental uploads (a preliminary GGUF from danielhanchen, a few scattered community repos), no GGUF from Unsloth or ggml-org, and per ComputingForGeeks’ HF search on November 5, the GGUF filter returned nothing.

Mistral Large 4 vs Kimi K3.1 Pro self-hosting: VRAM floors from a 512 GB Mac Studio to an 8×288 GB node

Here is the ladder, tier by tier. Numbers are from the quant repos and the vendor cards; anything measured by hand is labeled.

Close-up of a red printed circuit board with dense circuitry — generic stock photo representing the VRAM-floor hardware math in this section, not an actual GPU node or memory configuration

Smallest runnable quant. For Mistral: Unsloth’s Dynamic 1-bit (UD-IQ1_S) at ~233 GB, which their guide says fits on a 256 GB-RAM device “including a Mac Studio or 2x NVIDIA DGX Sparks.” Unsloth’s docs claim this 1-bit “retains 91% of top-1 accuracy across coding, agentic tasks, math and other areas whilst being 80% smaller (223GB vs 1.1TB FP8)” — note that this retention figure is self-published by Unsloth, not independently verified. It’s also a claim about top-1 accuracy on their eval suite, not about serving quality, tool-calling reliability, or what happens at long context. “Runs” ≠ “serves well” ≠ “matches full precision.”

For Kimi, there is no GGUF ladder to compare against — that’s the point. The smallest verified path to K3.1 Pro on consumer-class hardware is community MLX builds for Apple Silicon: pipenetwork’s REAP80-code builds are expert-pruned (to 20% of experts, code-weighted calibration) MLX conversions sized “to load on a 512 GB Apple Silicon machine” at 313 GiB. A community video demo of streaming the full, unpruned model on a 512 GB Mac Studio reports roughly 1.1 tokens/second. [UNVERIFIED: footprint and published speed for a clean, unpruned MLX build of K3.1 Pro — treat any “fits comfortably on a 512 GB Mac at usable speed” claim as unconfirmed.]

The 512 GB Mac Studio lane, both directions. For Mistral, ggml-org’s official ladder gives you two quant names that literally say it: Q2_K at 381 GB and IQ2_XXS at 298 GB, both annotated “Fits in 512 GB of unified memory” in the official README. For Kimi, the equivalent lane on a Mac is either a heavily pruned community MLX build or full-model streaming off SSD, and nothing has been formally benchmarked. If your machine is a 512 GB Mac Studio, Mistral has an official, named quant for you; Kimi has experiments.

Realistic full-speed serving. This is where Kimi’s 3T-class reality bites:

  • K3.1 Pro weights floor: 1,561 GB. The KV cache is far smaller than you’d guess, because only 24 of the 93 layers keep a per-token cache (the other 69 use KDA linear attention with a fixed-size state) — a community arithmetic script (ComputingForGeeks, reproducible from config.json) puts the cache at 33,792 bytes/token plus 0.43 GB per sequence, so a full 1M-token conversation adds only ~36 GB, and weights + KV for one such sequence is ~1,597 GB. For sizing, don’t confuse the two published floors: 1,597 GB is the weights-plus-KV arithmetic figure, while vLLM’s official K3.1 Pro recipe states a minimum of 1,874 GB (1,561 GB safetensors × 1.2 headroom) — the first is what the files need, the second is what the engine actually asks for.
  • The serving nodes that fit: per the same write-up, 8×H200 (1,128 GB total) comes up more than 400 GB short even before KV; a DGX B200 at 1,440 GB is short of the weights alone; 8×288 GB GPUs (B300 or MI355X class, 2,304 GB) is the realistic node — which matches AMD’s own day-0 K3.1 Pro deployment note for 8× MI355X.
  • Mistral, by contrast: the official FP8 checkpoint fits weights+overheads on a single 8-GPU node — the vendor card’s reference deployment is vLLM with --tensor-parallel-size 8 and an official Docker image, and the NVFP4 build targets four-GPU Blackwell-class boxes per the technical-report coverage.

Who is this actually for, in three lanes:

  1. 512 GB Mac Studio homelab: Mistral IQ2_XXS/Q2_K, officially named, reasonably fast, vision included. Kimi only via pruned MLX or streaming, un-benchmarked, low tok/s — a curiosity, not a daily driver.
  2. Prosumer multi-GPU rig (say 4–8× 80/96 GB cards): Mistral’s 1-bit (233 GB) and 2-bit (302–378 GB) quants genuinely live here. K3.1 Pro does not — no ggml-org ladder exists yet for it (danielhanchen’s preliminary UD-Q4_K_XL at ~1.509 TB is the sole early GGUF, and it still doesn’t clear a 512 GB machine), and there’s no pruned vLLM build; everything else sub-1.6 TB is Apple-Silicon MLX (pipenetwork’s REAP80 builds at 300–326 GiB), which don’t help an NVIDIA rig.
  3. Small company deciding weights-vs-API: read the license section below first; it flips the default answer for Mistral.

Engine support timelines: llama.cpp v0.9.0 vs b12180 — and the gaps that remain

Two weeks after the weights, the tooling picture is already a proxy war for maturity:

Engine Mistral Large 4 Kimi K3.1 Pro
llama.cpp v0.9.0, released Nov 4 — adds Mistral Large 4 among four new architectures, plus OCP FP8 E4M3 quantization support (the format family the checkpoint needs) PR #31779 merged Nov 6 (in release b12180, published Nov 6); replaces K3’s full-attention MLA layers with DSA — 24 of the 93 layers carry the sparse indexer
Official GGUF ggml-org + Unsloth repos live since late Oct/early Nov None (experimental community uploads only)
vLLM Supported in the official recipe; dedicated Docker image vllm/vllm-openai:mistral-large-4 in the vendor card Model PR #65685 still open (5-stack split); per third-party checks Nov 5, Docker images only — PyPI vLLM 0.33.0 (uploaded Nov 2) can’t run it
SGLang Supported; official Docker image in the vendor card Merged Nov 4 (PR #48415), but PyPI still ships 0.5.23 from Oct 30 — it predates the merge
NVIDIA Served via NVIDIA-hosted endpoints, per Mistral’s announcement —

Kimi’s llama.cpp PR is a genuinely good sign — the merge confirms the DSA integration (the PR body states “K3.1 reuses most of K3, but replaces the full-attention MLA layers with DSA,” and the Kimi model card identifies 24 of the 93 layers as the Gated DSA layers that carry the sparse indexer) and reports successful q2/q3 test quantization. But note the sequencing: K3.1 Pro got llama.cpp support two days after Mistral got it, Kimi still has no GGUF converter artifact, and vLLM’s wheel story is stuck in review. For a self-hoster, that’s the difference between “clone the repo and go” and “wait, or build from source on day N.”

Worth pausing on one architectural irony: both models independently converged on DeepSeek Sparse Attention. Kimi made Gated DSA its native long-context mechanism. Mistral adopted it — but only after a mid-training detour the technical report admits openly: the team pushed KV-cache reduction (a 16× reduction via ragged MLA, cross-layer attention sharing, and YOCO) and later found it “underperformed on downstream agentic tasks,” fixing it via model surgery to full attention and then DSA. For self-hosters the detail matters because it explains why Mistral’s final checkpoint needs a fresh llama.cpp quant path (OCP FP8) and why Kimi’s DSA layers compress KV so cheaply.

The license diff that flips the hosting asymmetry

Here’s the sharpest finding of this exercise. Mistral’s new Ecosystem Open-Weight License copies, word-for-word, the 47-word hosting definition from Moonshot’s K3.1 Pro license — but quietly deletes Moonshot’s $20 million revenue floor for Model-as-a-Service businesses.

The identical 47 words in both LICENSE files, verbatim:

“…in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data. This does not include (a) end-user products with model capabilities solely embedded within specific features or harnesses, or (b) mere relaying of requests to models hosted by others.”

Side by side, the condition attached to those words:

Kimi K3.1-Pro License, §2: “If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.”

Mistral Ecosystem Open-Weight License, §2: “If the Licensee or any of its affiliates operates a Model as a Service business, the Licensee must enter into a separate commercial license with Mistral before using the Model or its derivative works for any commercial purpose.”

Below $20M of annual revenue, Moonshot explicitly doesn’t require a deal from you. Mistral’s clause requires a deal from every commercial MaaS operator, from a two-person GPU shop on up — the floor was removed, not lowered — Plain AI’s license comparison documents the same word-for-word diff. (There’s also a second Kimi-only rule: products past 100M monthly active users or $20M monthly revenue must prominently display “Kimi K3.1-Pro” in the UI — the same disclosure posture we covered in our write-up on K3.1 Pro’s open-weights cyber disclosure.)

And the market data on OpenRouter, which I pulled from the endpoints API on November 9: kimi-k3.1-pro is served by 24 endpoints from 17 providers (Fireworks, DeepInfra, Nebius, Together, Parasail, and others, plus Moonshot itself). mistral-large-4-0 shows 2 endpoints, both Mistral-operated; a mapped third-party inference-provider list for it did not appear.

Now the mandatory caveat, because it’s the most honest-reading beat in this whole comparison: Mistral Large 3 shipped under Apache 2.0 (verified on HF for Mistral-Large-3-675B-Instruct-2512) and its OpenRouter endpoints were also Mistral-run only (2 endpoints, one provider — same API check). Third parties were seemingly not falling over each other to host a 1T-model class even when nobody’s permission was needed. So the Ecosystem License’s floor removal changes what an empty endpoints list means (for K3.1’s license class: potential hosts are welcome; for Mistral’s: they’d need contracts) — it does not prove third parties would host Large 4 if allowed. Read it both ways.

Internal use is exempt on both licenses, and notably Kimi’s exemption extends to access “through Moonshot AI’s official products or certified inference partners.”

API pricing and the self-hosting break-even

Verified rates as of this writing (November 9, 2026), vendor pages checked directly (Mistral’s pricing docs, Moonshot’s K3.1 Pro pricing page):

Mistral Large 4 Kimi K3.1 Pro
Input $0.70 / MTok $2.00 / MTok
Output $2.10 / MTok $10.00 / MTok
Cached input $0.07 / MTok $0.20 / MTok
Cache write — $2.00 (5m TTL) / $4.00 (1h) per Moonshot’s pricing page
Highspeed tier — $4.00 / $20.00 (cached input stays $0.20 — only input, output, and cache-write fees double)
Batch — 60% of standard (so $1.20 / $6.00)

(The confusingly similar $3/$15 figure floating around in third-party articles is Kimi K3, the previous model — Moonshot’s own page confirms K3 at $3.00/$15.00/$0.30. Don’t transplant it onto K3.1 Pro.)

The reasoning-cost trap. Both models bill thinking as output (per the hands-on documentation, reasoning tokens draw from the output line and output caps can be swallowed by them). Two measured examples, both from single third-party runs — reproduce before you budget on them:

  • ComputingForGeeks’ hosted Kimi K3 runs hit the classic pattern: at a 1,200-token output cap, a question returned “completely empty content, with all 1200 tokens consumed by hidden reasoning”; at a 4,000 cap, the run “returned empty again after 4088 reasoning tokens, billed past the cap rather than stopped at it.” Only lowering the effort level produced an answer (1,116 tokens).
  • OmniaKey’s K3.1 Pro measurements (nine real requests, November 1): with max_tokens at 4,000 and low effort, the first catalog request “stopped at the cap. The API reported 4,096 output tokens, 4,093 of them reasoning, and an empty answer” — costing $0.05564 for no content. In their runs, output was 84–97% of the bill in most requests, and the K3.1 Pro default of max reasoning effort is the aggressive setting the vendor documents.

The practical rule both runs converge on: budget ≥5,000 output tokens per call, or explicitly set the effort level down. For K3.1 Pro, reasoning_effort defaults to max; the high setting cost 43–82% more than low in the measured runs.

Break-even, honestly. Any “self-hosting pays off at N tokens/month” conclusion needs two adjustments before it’s true: Kimi’s batch API runs at 60% of list, and both vendors offer steep cached-input rates (Mistral $0.07, Kimi $0.20 — a 90% discount, and OmniaKey measured repeated-prefix questions at an 88% input-cost cut in practice). If your workload is batchable or cacheable, the API-side effective price is often a third to a tenth of list, which pushes break-even token volumes far higher than a naive list-rate comparison suggests. When the Mistral Large 4 vs Kimi K3.1 Pro self-hosting question comes down to token economics, batch and cache discounts move the finish line more than the sticker rates do. On top of that, the capex is real: a full-speed K3.1 Pro node is 8× 288 GB GPUs; renting is the honest comparison — ComputingForGeeks’ rental experiment on the older K3 (4×A100, 0.10 tok/s at 1-bit) totaled $8.29 and produced a model “too slow to hold a conversation with” — a data point that says: small GPU boxes don’t just underperform here, they under- run.

Verdict: Mistral Large 4 vs Kimi K3.1 Pro self-hosting, by reader profile

A MacBook sitting on a wooden desk in a home workspace — generic stock photo representing the 512 GB Mac Studio homelab lane in the verdict, not an actual Mac Studio or K3.1 Pro deployment

  • 512 GB Mac Studio: Mistral Large 4 at IQ2_XXS (298 GB) or Q2_K (381 GB) — official, named, fits, vision included. K3.1 Pro’s Mac story is still experimental pruned-MLX builds and slow streaming.
  • Prosumer multi-GPU rig: Mistral, running away with it. A 233–378 GB quant on your rig versus Kimi’s sub-1.6 TB CUDA-class artifacts — an experimental ~1.51 TB community GGUF aside, its smaller artifacts are Apple-Silicon MLX only — is not a contest; llama.cpp supports both as of Nov 4/6, but only Mistral has the ladder beneath the architecture support.
  • Small company, weights-vs-API: if you must serve as a service, Mistral’s license means even a tiny operator needs a commercial deal — which usually pushes you to the $0.70/$2.10 API or back to Apache-2.0 options. If you have a compliance reason to keep weights in-house (internal use only), both licenses explicitly exempt internal use — Kimi’s MaaS gate at $20M/12mo only bites large hosts, and at 1.6 TB of GPU class, its API at $2/$10 (or 60%-of-list in batch) is near unbeatable on price.
  • If you only care about throughput per dollar on an API: K3.1 Pro costs ~2.9× Mistral’s input rate and ~4.8× its output rate, so Mistral wins the rate card — unless its reasoning-effort (high default) or capability profile doesn’t fit, which is the capability-axis post’s question.

The clean summary: Kimi optimizes the API bill; Mistral optimizes the machine. If you’re running the weights anywhere smaller than a data-center node, this month the answer is Mistral. If you’re buying tokens by the million, this month the answer is K3.1 Pro — with the effort level set to something other than max.

References and further reading

Please let us know if you enjoyed this blog post. Share it with others to spread the knowledge! If you believe any images in this post infringe your copyright, please contact us promptly so we can remove them.

FEATURED TAGS
AI self-hosting quantization llama.cpp Mistral open weights Kimi inference economics
// adding consent banner
ESC