Open-Weight Models: What They Are and When They Pay Off for Your Business

Almost every company now uses AI, usually through an API from OpenAI, Anthropic (Claude) or Google (Gemini). That is convenient and powerful. But it comes with three catches: cost per token rises with volume, your data leaves your building with every request, and the rules (guardrails) the model follows are set by the vendor, not by you.

This is exactly where open-weight models come in. Over the past year and a half they have gone from "nice toy for hobbyists" to a serious alternative for businesses. Since DeepSeek showed in early 2025 that a freely downloadable model could keep up with the most expensive closed models, the topic has reached the boardroom.

This article answers three questions: What is an open-weight model in the first place, and why should you use one? Which models exist, and how do they compare to Claude, Gemini and OpenAI? And how do you actually use one, meaning where do you get it, and what hardware does it need?

What is an open-weight model?

At its core, a language model consists of its "weights": the many billions of numbers learned during training that make up the model's actual "knowledge". Whether you can get your hands on those weights decides everything else. Broadly, there are three categories.

Proprietary / closed

The weights are never released. You can only use the model through the vendor's API or products. Examples: OpenAI's GPT-4o and GPT-5 series, Anthropic's Claude models, Google's Gemini Pro and Flash. You cannot run the model yourself, cannot fine-tune it on your own data, and cannot inspect its internals.

Open weight

The trained weights are available for public download, usually on the Hugging Face platform or in an official release. This lets you self-host the model, run it offline, and fine-tune it on your own data. The training code, training data and internal system prompts, however, are typically not included.

Open source (in the true sense)

True open source would mean that, in addition to the weights, the training code and datasets are disclosed and reproducible. In practice, almost no model from a major lab meets this bar. That is why the frequently heard term "open-source AI" is usually misleading: what people mean is almost always open-weight models.

Why should you use an open-weight model?

The appeal is not ideological but very practical. There are five concrete reasons that matter especially for small and mid-sized companies.

1. Rising token prices and cost control

With the big closed vendors you pay per token, and those prices are the vendor's call, not yours. At high volume (document processing, summaries, internal tools) this quickly gets expensive. Open-weight models are considerably cheaper here, whether self-hosted or via specialised providers.

To give a sense of scale: hosted open-weight models (as of mid-2026, e.g. on Together AI) run at roughly 0.15 to 1.80 US dollars per million input tokens, often just a fraction of what a top closed model of the same quality class costs. When you run it yourself, after the hardware purchase you only pay for electricity.

2. Keep data local (GDPR)

With every request to a closed API, your prompts, customer data and outputs leave your infrastructure and are processed on someone else's servers. For GDPR-regulated companies this creates transfer and processing obligations. A self-hosted open-weight model processes everything locally, so no data leaves the building. Particularly relevant for legal, HR, healthcare and financial data.

3. Your own guardrails and fine-tuning

With open weights you decide how the model behaves: you can fine-tune it on your domain vocabulary and processes, and tailor the content filters precisely to your use case. With closed models this is either impossible or only available at significant extra cost through the vendor.

4. No vendor lock-in

Switching from GPT to Claude to Gemini means prompt rework and API changes every time. Open-weight models run on standardised inference servers with OpenAI-compatible interfaces. You can swap models without rebuilding your whole application, and you are not dependent on a single vendor's pricing and product decisions.

5. Offline and air-gapped

In manufacturing, defence, healthcare or highly sensitive financial processes, internet-connected inference is sometimes simply prohibited. An open-weight model can be run completely disconnected from the internet.

Which models exist, and how do they compare to Claude, Gemini and OpenAI?

The short answer: surprisingly well. As of mid-2026, several open-weight models match the best closed models at coding, maths and reasoning, and beat them on some tasks. Here is an overview of the most important families.

The major families

• Meta Llama (Llama 4 Scout & Maverick): long context windows, multimodal (text and image), very widely used. Custom community licence.
• Alibaba Qwen (Qwen3 / Qwen3.5): toggleable "thinking mode", extremely broad language support, mostly under the very permissive Apache 2.0 licence.
• DeepSeek (V3 / R1 / V4 Pro): the model that shook up the market in 2025. Strong at code and maths, MIT licence.
• Mistral (7B, Mixtral, Small 3.1): from France, very efficient, the open variants under Apache 2.0.
• Google Gemma (Gemma 3 / Gemma 4): compact and strong, ideal for local deployment.
• Microsoft Phi (Phi-4 / Phi-4-mini): small but surprisingly capable; good for laptops and edge devices. MIT licence.
• OpenAI gpt-oss-120B: OpenAI's own open model (since around August 2025), Apache 2.0 and very cheap per token.
• Other heavyweights: NVIDIA Nemotron, Moonshot Kimi K2, MiniMax M3, all frontier-class models, mostly very large.

Frontier-comparable

These models play in the top league and go head-to-head in benchmarks with Claude Opus, Gemini Pro and OpenAI's o-models, especially on code and reasoning: DeepSeek V4 Pro, Moonshot Kimi K2.6, NVIDIA Nemotron 3 Ultra, Qwen3.5-397B and MiniMax M3. The catch: these are huge models that you realistically won't run on your own hardware but access through a hosting provider.

Strong for their size and locally usable

For most companies these are the more interesting models: small enough to run yourself, but with excellent quality for typical business tasks.

• Google Gemma 4 (31B): reaches reasoning scores close to far larger models.
• Mistral Small 3.1 (24B): surpasses similarly sized proprietary models and runs on a single high-end graphics card.
• Qwen3-32B and the smaller Qwen3 variants: versatile and multilingual.
• Microsoft Phi-4-reasoning (14B): strong logic with a small footprint.
• Phi-4-mini (3.8B) and Llama-3.x-8B: run even on a well-equipped laptop; ideal for summaries, classification and coding assistance.

How good are they really?

On the common benchmarks (e.g. coding tasks like SWE-Bench and LiveCodeBench, or knowledge tests like GPQA), the best open models are now on par with or ahead of the top closed models. In practice this means: for most of your tasks (text, analysis, code, customer communication) you'll barely notice a quality difference anymore.

How do you use an open-weight model?

There are two basic paths: run it yourself (on your hardware or server) or use it through a provider that hosts the open model for you. Both have their place.

Where do you get the models?

• Hugging Face: the central library. Practically every open model is available here for download (you have to accept each model's licence terms).
• Official sources from the makers: e.g. the GitHub repos and websites of Meta, DeepSeek, Qwen or Mistral.
• Hosted open-weight APIs: providers like Together AI, Groq, Fireworks or OpenRouter run the open models for you; you only pay per token. The best compromise if you don't want your own hardware.
• Cloud platforms: AWS Bedrock, Azure AI Foundry and Google Vertex AI host many open models enterprise-ready, including access control and compliance tooling.

What do you run them with?

• Ollama: the easiest start. One command (e.g. "ollama run qwen3") downloads and runs the model, including a local OpenAI-compatible interface. Perfect for first tests.
• LM Studio: a desktop app with a UI to download, try out and serve models. Good for anyone who doesn't want the command line.
• llama.cpp: the lean engine underneath Ollama and LM Studio. Runs from a Raspberry Pi to a server.
• vLLM (or SGLang): the production server for serious operation: fast, multi-GPU, efficient memory management. The right choice when many users access it at once.

What hardware do you need?

For local use, Apple Silicon Macs are especially attractive. Thanks to unified memory, the processor and graphics unit share the same memory, so practically all of your RAM is available to the model. A MacBook with plenty of memory easily replaces an expensive graphics card, stays quiet and even runs on battery. As a rough rule of thumb for the requirement: at full precision (FP16) a model needs about 2 gigabytes per billion parameters. "Quantisation", meaning compressing the numbers to fewer bits, lowers this dramatically: Q8 roughly halves the requirement, Q4 quarters it.

• MacBook Air (M-chip, 16 to 24 GB): small models like Phi-4-mini (3.8B) or Llama-3.x-8B in Q4, good for summaries, classification and coding help. Fanless and on battery all day.
• MacBook Pro (M-Pro or M-Max, 32 to 48 GB): models in the 24 to 32B range in Q4, e.g. Mistral Small 3.1 or Gemma 4 31B, close to frontier quality for most business tasks.
• MacBook Pro / Mac Studio with plenty of memory (64 to 128 GB): even a 70B model in Q4 (around 35 to 40 GB) runs smoothly; with 128 GB even larger MoE models fit.
• Mac Studio (M-Ultra, 192 GB and up): the very large models. Here you move towards data-centre territory, just locally on your desk.

A quick word on non-Mac hardware: on Windows or Linux with a dedicated graphics card the same principle applies, except here the card's VRAM counts rather than unified memory. An RTX 4090 (24 GB VRAM) covers models up to roughly 32B in Q4; the very large frontier models like Qwen3-235B or DeepSeek V4 Pro need multiple server GPUs (A100/H100) and you realistically use them through a hosting provider.

Edge cases, pitfalls and the latest developments

Licensing pitfalls

Not every "open" model is equally free to use. The most straightforward are Apache 2.0 and MIT models (e.g. DeepSeek, Qwen3, Mistral Small 3.1, Phi-4, Gemma 4, gpt-oss): commercial use and modification are explicitly allowed. Meta's Llama licence is workable for the vast majority of companies too, but contains a special clause: above 700 million monthly active users you need a separate licence from Meta. For brand-new or specialised models (such as NVIDIA Nemotron with its own licence, or freshly released ones like Kimi K2 and MiniMax M3) it's worth checking the specific terms before you build commercially on them.

Self-hosting means self-responsibility

With a closed API, a large safety team works in the background on filters and abuse detection. If you run a model yourself, that responsibility is yours: output filters, protection against manipulation (jailbreaks) and prompt injection all need to be handled by you. There are ready-made building blocks: Meta, for example, provides "Llama Guard" and "Prompt Guard" openly. But you have to deploy them deliberately.

The "free inference" trap

After buying the hardware, inference is "free", that's the promise. But the total cost of ownership includes acquisition (an RTX 4090 ~€1,600, an H100 server six figures), electricity, and above all ongoing operations: setup, monitoring, updates, scaling. Self-hosting pays off at high, steady volume, strict data-protection requirements or offline operation. For everything else, the hosted open-weight API is usually cheaper and less hassle.

EU AI Act: it applies to you as a deployer too

Important for European companies: hosting a model yourself does not exempt you from the EU AI Act. Obligations for providers of general-purpose AI have applied since August 2025; transparency obligations for deployers (e.g. that a chatbot must be recognisable as AI) take effect from August 2026. If you build a customer-facing AI product in the EU, these rules apply regardless of whether the model is open or closed. High-risk applications (e.g. hiring, credit scoring) carry additional assessment obligations.

What is new in 2026

• Agentic capabilities: the new open models are no longer just chat but built for autonomous agents: tool use, terminal control, multi-step tasks.
• Very long contexts: context windows of up to a million tokens are becoming standard: entire codebases or mountains of documents can be processed in one go.
• Native multimodality: many open models now understand text, images and partly video directly.
• Toggleable thinking modes: "fast and cheap" or "think hard": you decide when you pay for expensive reasoning.
• Beyond language: open speech synthesis (text-to-speech) and other modalities are being added too: the open ecosystem is growing beyond pure text LLMs.

Conclusion: when open-weight, when closed?

In 2026, open-weight models are no longer a compromise but the better choice for many use cases. They give you cost control, data sovereignty and independence, and the quality gap to the closed models has largely closed for everyday work.

Open weight pays off especially when:

• you process sensitive or personal data that has to stay in-house (GDPR),
• you have high, steady volume and want to push down token costs,
• you want to fine-tune the model to your domain or set your own guardrails,
• you need vendor independence or offline operation.

A closed API often remains the better fit when:

• you have low or irregular volumes and don't want to run infrastructure,
• you always need the absolute top model for the hardest tasks, without operational overhead,
• ready-made safety and compliance features from the vendor matter more to you than full control.