Build, buy, or fine-tune: a simple way to choose the right AI approach

Every time a team adds AI to a product, they face the same choice, even if they don't realise it. They can use a ready-made model through an API, like calling up OpenAI or Anthropic and sending a request. They can fine-tune an existing model, which means training it further on their own data. Or they can build something of their own from the ground up.

None of these choices is wrong by itself.

The problem is when a team picks one without really thinking about whether it fits the job. Someone uses an API because that's what the last project used. Someone pushes for fine-tuning because a vendor demo made it look easy. Someone wants to build from scratch because it feels like the more serious option. The choice ends up based on habit, not on what the feature actually needs.

McKinsey found that 70% of enterprise AI use cases are handled well enough by off-the-shelf tools. In other words, a lot of custom builds are solving problems a simple API call would have solved too. Forrester found that companies who tested a ready-made solution first, before building something custom, saw 3.2 times higher returns than those who built first. And Gartner found that companies who built custom AI without first testing a vendor's off-the-shelf option lost 14 months on average, time they didn't need to lose. Skipping this decision has a real cost. It shows up as a wasted year, and a feature that could have shipped in month two instead.

When a ready-made model is enough

Start here, because for most tasks, this is as far as you need to go. A general AI model, used through an API, can already handle a lot: writing drafts, summarising documents, sorting information into categories, pulling out key details, running a chat feature, or answering questions based on information you give it directly.

The best reason to start here is easy to miss. These models keep getting better, and you don't pay anything extra for it. The AI model you used at the start of 2024 is noticeably weaker than the one available today, at a similar price. Some teams that built their own model back then have spent more money to end up with something worse than what they could now get from a simple subscription.

Here's a simple way to know if this option is right for you. If the model already has, or can be given, all the information it needs to answer the question, you're fine. You don't have a technology problem. You have a "give the model better instructions and information" problem, and that's a much easier and cheaper thing to fix.

When you need retrieval, not fine-tuning

Before jumping to fine-tuning, it's worth trying something simpler first: giving the model the right information at the moment it needs it. This is called retrieval, and it solves a different problem than people usually expect fine-tuning to solve.

Fine-tuning changes how a model responds. It can teach a model to write in a certain style, or follow a certain format. What it doesn't do well is teach a model new facts, especially facts it hasn't seen very often. One well-known study found that fine-tuned models gave a wrong or made-up answer 38% of the time when asked about specific, less common topics. When the same models were instead given the right facts at the time of the question, that error rate dropped to 8%. So if your feature needs to be accurate about specific or up-to-date information, that's a job for retrieval, not fine-tuning. No amount of extra training data fixes that.

Retrieval is also cheaper to get started with. There's no training process, no need to build a big dataset, and no lengthy testing phase before you can launch. You pay for it a little each time it runs, rather than paying a large amount up front. That makes it a sensible place to start while you're still learning exactly what the feature needs to get right.

When fine-tuning is worth it

Fine-tuning does pay off, just in fewer situations than people expect. It's worth it when you need the model to answer in a very consistent style or format, when the task is narrow and happens constantly, or when the ongoing cost of retrieval starts to add up to more than a one-off training cost would.

There's a rough point where this tips in favour of fine-tuning. Teams running AI at scale have found that fine-tuning tends to become worthwhile somewhere around 500 million to 1 billion "tokens" a month, tokens being the small chunks of text a model processes. Below that, the cost of training and running your own model usually isn't worth it. Above it, a smaller, fine-tuned model can end up cheaper and more consistent, because it doesn't need to be given the same background information over and over on every request.

There's also a quality reason to fine-tune, not just a cost one, especially in fields with their own specific language and edge cases. In legal work, models fine-tuned on legal documents have cut contract review time by 70 to 85% compared with doing it by hand, while also making fewer mistakes on individual clauses. That happens because the model has learned the specific patterns of that kind of document, rather than working them out fresh each time.

To put real numbers on it: one comparison found that using a general model cost around $11 per thousand queries. A fine-tuned model on its own cost around $20. A general model combined with retrieval cost around $41. A fine-tuned model combined with retrieval cost around $49. So fine-tuning by itself isn't automatically the cheaper option. It only becomes cheaper once you're running enough volume that you stop paying for long background information on every single request.

When it makes sense to build your own model

Building a model of your own, one trained just for a single, specific task rather than adjusted from a general one, is the option teams need the least often. It's also the one whose value is most often underestimated, once the volume and the task are specific enough to justify it. Small, purpose-built models have matched or beaten general AI models on narrow, well-defined tasks, such as matching job candidates to roles or pulling specific clauses out of contracts, and they cost far less to run.

The catch is that you take on everything involved in owning that model. You have to gather and prepare the training data, retrain it as your needs change, and keep an eye on whether its answers start drifting off track. None of that goes away once it's built. This route pays off when the task is stable, well understood, and repeated often enough that owning it beats paying for someone else's general model every time.

A framework, not one single answer

The best way to think about this is layer by layer, not as one big decision. The right answer for the AI model itself might be completely different from the right answer for the rest of your system, like how it's connected, or how it's monitored and controlled. A retailer, for example, might use a general model for customer chat, fine-tune a smaller model to sort products into categories at high volume, and build custom retrieval on top of their own internal documents, all within the same feature. Treating the whole thing as one single choice usually gets at least one part of it wrong.

A few simple questions help with most of these decisions. Does the task depend on facts the model wouldn't already know, or on how it should respond? The first points towards retrieval, the second towards fine-tuning, and sometimes you need both. What's the real monthly volume you expect, not the number from a small pilot, and is it high enough that owning your own setup would pay off? Does the value of this feature come from something unique to your business, in which case it's worth protecting by owning that layer, or is it something any competitor could get from the same vendor tomorrow, in which case buying isn't a compromise. It's simply the right choice.

Where to start

You don't need to settle this once for your whole AI plan. It's worth working through once per feature, honestly, before spending engineering time on a path you haven't tested against the cheaper option first. A quick trial using a general model, even for something you expect will eventually need fine-tuning, is usually the fastest way to find out how much of the problem you actually need to solve yourself.

We've spent over a decade helping clients make this call, on real projects with real budgets at stake, and the same lesson keeps showing up. The businesses that get the most value aren't running the most advanced model. They're the ones who worked through this decision carefully, one feature at a time, instead of defaulting to whatever seemed most impressive to announce. Let's talk about how this framework applies to your product.