Mistral Large 4's open weights: when running a trillion-parameter model yourself makes sense, and when it does not

What happened

Mistral released the weights of Mistral Large 4 on 29 October. It is a mixture-of-experts model with about one trillion parameters in total, of which roughly 50 billion are active for any given token, and Mistral is positioning it for cybersecurity defenders who want to run it themselves¹². The vLLM project had support ready on day one³. The benchmark results quoted at launch are Mistral's own, and some commentators have questioned whether the licence terms meet common definitions of open weight³⁴.

The same day, Canada announced an equity investment of up to C$1.6 billion in Cohere, tied to commitments to build compute capacity in Canada⁵. Two different routes, one theme: governments and security teams want frontier-class models they control.

Why this matters

Until recently, "run it yourself" meant accepting a clear capability gap against the best hosted models. Large, capable open-weight models narrow that gap for some tasks, particularly in security, where sending logs, code and vulnerability data to a third party is often unacceptable. That brings self-hosting into serious consideration for regulated and sovereign buyers in Europe, the GCC, India and beyond.

It does not make self-hosting cheap. A mixture-of-experts model only activates part of its weights per token, which helps compute, but all trillion parameters still have to sit in accelerator memory. In a year when memory is the scarce part of AI hardware, as covered in the AI memory squeeze, that is the cost that decides most business cases.

Three ways to use it

Most organisations should start with option A to test quality on their own tasks, behind a gateway, and move to B or C only when a requirement forces it. The model routing and inference gateway notes cover how to keep that switch cheap.

If you do self-host: what it takes

• Multi-GPU, often multi-node serving. A model this size needs several high-memory accelerators even with quantisation, and possibly more than one server. Serving engines such as vLLM handle the parallelism; your platform has to schedule and heal it.
• Gang scheduling. Multi-node work has to start together or not at all. Kubernetes projects such as JobSet represent groups of pods as one unit for distributed jobs like evaluation runs and fine-tuning⁶; see gang scheduling for AI.
• Your own safety layer. A hosted API brings abuse monitoring and guardrails. Self-hosted, those are yours, which matters for a model marketed for offensive-security-adjacent work.
• Licence review. Read the terms for commercial use, redistribution and any field-of-use limits before building on it.
• A team. Someone has to patch, monitor, upgrade and be on call for it.

How to decide

For on-premises economics in detail, see private LLMs on-premises; for the security use case Mistral is targeting, defender-first AI models.

CTO take

Large 4 is good news for buyers because it adds a credible, self-hostable option at the top end and puts pressure on hosted pricing. It is not a reason to start buying GPUs. Test it through an API on your real tasks, behind a gateway you control. If it wins on quality and you have a hard requirement to keep data in your environment, plan dedicated hosting with the memory, scheduling and team it needs. Either way, the winning move is the same: keep your stack portable so you can take advantage of the next release too. Today's tech news brief has the wider context.

How I can help

I help teams decide between hosted and self-hosted models, and design portable inference platforms that keep that choice open. I am based in India, the United States and the UAE, working globally. See AI architecture or analyse your challenge and bring the result to a call.