Choosing a language model for your business: the criteria that matter

Start from the use case before looking at leaderboards

Every month brings a new model, a new leaderboard and a new promise. For a company starting a project, the temptation is to pick the model at the top of the latest comparison and build around it. That is the most expensive shortcut: leaderboards measure generic tasks, and your project is not one.

The right starting point is a precise description of what the model will have to do. Read purchase orders and extract lines. Answer employee questions from a document base. Draft a first version of meeting minutes from notes. Classify incoming requests and route them to the right team. Each of these tasks calls on different qualities, and a model that excels at one may be oversized, or insufficient, for another.

Anthropic's documentation on choosing a model sums up the trade-off in three terms: the capabilities you need, the response speed you expect, and the budget, for development and for production. We add four criteria that weigh just as much in a business project: context size, how you access the model, data location and the contract that governs it, and continuity over time.

This article is about the initial choice. If you already have an agent in production and wonder whether to move to a newer model, our article on what Claude Opus 5.5 changes covers the switching method.

Quality is measured on your cases, not someone else's

A model is good or bad at a given task: yours, with your documents, your business vocabulary, your in-house abbreviations and your awkward cases. The only way to know is to try it on a sample of those cases.

Anthropic says it plainly in its own documentation: having a good evaluation set is the most important step in choosing. That set contains real cases, with the expected answer validated by someone from the business, including hard cases and out of scope requests. You run the candidate models through it, with the same instructions, and compare.

A few precautions keep the comparison honest:

• Adapt instructions to each model. A prompt written and refined for one model favours that model. Give each candidate a fair chance, with a minimum of tuning.
• Compare at equal cost or equal quality. A more capable model that costs far more is only a good choice if the quality gap matters for the use case.
• Look at errors, not just the score. Two models with the same score can fail on very different cases. An error on a sensitive case weighs more than an error on a trivial one.

This work lasts: the same set will serve every later change of prompt, model or data. It is the foundation for everything else.

Cost: reading beyond the list price

Vendors display a price per million tokens, for input and output. That figure is useful to compare models with each other. It says almost nothing about your bill, which depends on the length of instructions sent with each call, the amount of documents recalled, the number of exchanges per user, the share of context that is cached, and the invisible calls the system makes to check an answer or split a task.

Two consequences for the choice:

• A model that is cheaper per unit can end up costing more if it needs more steps, more retries or longer instructions to reach the same quality.
• Cost is measured during the prototype, on a realistic sample, never from a spreadsheet estimate. Our article on the cost of an AI agent in production lists the lines to track.

Anthropic's documentation offers two starting points. The first favours efficiency: start with a fast, economical model, test, and only move up if a capability gap requires it. The second favours capability: start with the strongest model for the task, then work on reducing cost once the system works. The first suits simple, high volume tasks. The second suits complex tasks where accuracy comes first. Either way, the evaluation set decides what happens next.

A third route exists: combining models. The same documentation describes two patterns, an economical model doing most of the work and escalating hard decisions to a stronger one, or a strong model orchestrating and delegating volume work to cheaper models. That is often where the best balance lies. Managing these trade-offs over time is the subject of our CADRE framework for AI costs.

Latency: who is waiting for the answer?

A model's speed matters differently depending on who is waiting. A visitor in front of a chat leaves if the answer is slow. A salesperson preparing a meeting tolerates a few seconds if the result is good. An overnight batch job waits for nobody.

Ask the question for each journey in the project, not for the project as a whole. One system can use a fast model for what appears on screen and a slower, more capable model for what runs in the background. Latency also depends on construction: prompt length, amount of context, number of chained steps. A fast model buried under too much context can answer more slowly than a heavier model fed well.

Some vendors also offer an effort setting that trades reasoning quality against speed and cost within a single model. Anthropic states that tuning this setting is often a better lever than switching models. One more reason to test before deciding.

Context size: a capacity to use with judgement

Context is the amount of text the model can take into account at once: instructions, conversation history, documents provided. Recent models accept a lot. Anthropic's documentation, for example, lists a one million token window for several of its current models, and two hundred thousand for the lightest one.

A large window opens up uses: analysing a complete file, comparing several long contracts, keeping the full history of a deal. It does not excuse you from thinking about what you send. Every token sent is paid for, and a context cluttered with barely relevant documents can degrade the answer as much as a context that is too thin. For an agent answering from a document base, retrieving the right passages is usually more effective than sending the whole corpus.

The criterion to keep: the model's window must cover the largest realistic case in your use, with a margin. Beyond that, a larger window brings nothing measurable to your project.

API model or open weight model hosted by you

This is often the most structural choice, and it comes up early.

The API model

You call a vendor's model, directly or through a cloud platform. Nothing to install or maintain. You get the most capable models on the market and their updates. In return, your data passes through the provider's infrastructure under its contract terms, and you depend on its calendar: a model can be retired, a price can change.

The open weight model

Some vendors publish their model weights under an open licence. Mistral AI, for example, distributes several of its models under the Apache 2.0 licence. You can then run them on your own infrastructure or with a hosting provider of your choice. Data does not leave the environment you control, and the model does not disappear overnight.

The price of that control is real. You need servers with graphics processors, sized for peak load, a team to run, secure and update them, and you have to accept that the open models easiest to host are not always at the level of the best closed models on complex tasks. Check each model's licence too: they are not all equal, and some restrict commercial use.

How to decide

The useful question is: which constraint makes self-hosting necessary? A contractual or regulatory requirement on data location, a very high and steady volume that justifies dedicated infrastructure, a need to work offline. Without one of these, an API model hosted in Europe is generally simpler and more capable. With one of them, the open model becomes a serious option, to be evaluated on the same test set as the others.

The two options are not exclusive: one project can process the most sensitive data with a self-hosted model and the rest with an API model. Designing that architecture is the work of an AI engineer, together with your infrastructure teams.

Data location and contract

Where is the data sent to the model processed? How long is it kept? Is it used to train the provider's models? Who can access it, and under which jurisdiction? These questions are answered in the provider's contract and documentation, not in the model's spec sheet.

The same model can also be reached through several routes, with different guarantees. Anthropic states, for example, that its models are available directly and through several major cloud platforms, some of which offer regional endpoints that guarantee where data is processed. For a French company, the choice of access route therefore matters as much as the choice of model. Our article on hosting an AI project in France compares the options.

Points to check before signing:

• the processing and storage region, and whether you can enforce it;
• how long requests and responses are retained;
• whether your data may be used for training;
• the provider's subprocessors;
• availability commitments and incident support;
• the information and transparency obligations that fall on you depending on the use, covered in our article on the AI Act for SMBs.

Continuity: a model has an end date

An API model has a limited lifespan. Vendors regularly retire older models to make room for new ones. Anthropic publishes the lifecycle of each, from active to retired, with a minimum retirement date for current models, and commits to notifying affected customers at least sixty days before a publicly released model is retired. After that date, calls to the retired model fail. The vendor also notes that partner cloud platforms set their own schedules.

For a business project, this has three consequences:

• The model is a parameter. The system must be able to change model through configuration, without a rewrite. That is an architecture decision to make from the start.
• The evaluation set is permanent. It will validate the replacement the day the current model is retired. Anthropic recommends testing the new model well before the retirement date.
• Dependency is measured. If your system relies on a capability specific to one provider, write it down and identify a fallback, even a weaker one.

Open weight models escape this imposed calendar, since you keep the weights. They replace it with other work: tracking versions, fixes and security of what you host yourself.

A decision grid in practice

For a first project, this is the order in which we ask the questions with our clients:

• Eliminating constraints. Is there a requirement on data location, a regulated sector, an offline need? They narrow the list before any test.
• Task and volume. Which task, how often, with what accuracy requirement? This points to an efficiency-first or capability-first start.
• Acceptable wait. For each journey, who is waiting and how long can they wait?
• Size of the largest case. Does the required context fit in the candidates' windows, with a margin?
• Evaluation on your cases. Two or three candidates, the same cases, the same criteria, real cost measured.
• Contract and continuity. Data processing terms, retirement calendar, fallback.

This grid runs during the prototype, and its answers are part of the production prerequisites. We detailed those prerequisites in our article on moving a business POC to production.

Common mistakes

• Choosing on reputation. The model everyone talks about is not necessarily the best for your task, nor the most cost-effective.
• Hard-coding the choice. A model name written all over the code turns every change into a project.
• Underestimating self-hosting. Running an open model is easy in a demo. Operating it reliably, securely and sized for the load is a profession.
• Forgetting the contract. The data question is settled before production, not after the first question from a customer or an auditor.
• Testing without the business. An engineer judges the form of an answer. Only the business judges whether it is right.

If you are starting a project and want to frame this choice on your own cases, our AI diagnostic sets out the constraints, scope and evaluation protocol before any development starts.

Sources

• Claude documentation, Choosing the right model
• Claude documentation, Models overview
• Claude documentation, Model deprecations
• Claude documentation, Define success criteria and build evaluations
• Mistral AI, Models overview

Frequently asked questions

Should we pick a single model provider?

Not necessarily. A well designed system treats the model as a parameter and can use several depending on the task. Keeping two providers evaluated on your cases also reduces dependency, at the cost of a little more testing.

Is a French or European model required for compliance?

The question is first about the data processing conditions: region, retention, subprocessors, use for training. A foreign model accessed from a European region under a suitable contract can be fine, while a poorly governed European model can be a problem.

How many models should we test at the start?

Two or three candidates are usually enough, chosen after ruling out those that do not meet your constraints. Testing more costs time without much extra information, especially if the evaluation set is well built.

Can we change model during the project?

Yes, and you should plan for it. If the model is a configuration parameter and the evaluation set exists, a change is tested and deployed like any other update. Without those two conditions, it becomes a risky operation.

Is an open weight model free?

The licence may be, operating it never is. You need suitable servers, monitoring, security and updates. The calculation is on total cost, compared with an API model for the same volume.