Most teams we speak with have already experimented with AI.
Someone has a ChatGPT subscription, a few developers use a coding assistant, and there is usually a pilot somewhere that never made it past the demo stage. The question that keeps coming back is different now. It is no longer whether to use large language models, but where the data goes when you do.
For companies in Europe and Asia, that question has real weight. A Dutch insurer cannot send claims data to a US API without a serious conversation with its compliance team. An Indonesian bank operates under local data protection law and OJK expectations around where customer data lives. Singapore's PDPA, the EU AI Act, and GDPR all push in the same direction: know where your data is processed, and be able to prove it.
This is where private LLMs come in. Running a model on infrastructure you control, whether that is your own servers or a private cloud in your region, means prompts, documents, and customer data never leave your perimeter. Two years ago that meant accepting a large drop in quality. Today it does not. Open weight models such as Qwen, Mistral, Gemma, and DeepSeek have narrowed the gap with proprietary systems considerably, and the tooling around them has matured to the point where deployment is a project, not a research programme.
The harder question is how to start without overcommitting. Here is the approach we use with clients.
A private LLM is worth the effort when at least one of three conditions applies. Your data cannot leave your jurisdiction or organisation for legal or contractual reasons. Your API costs are high and predictable enough that fixed infrastructure becomes cheaper. Or you need behaviour that public APIs cannot give you, such as fine-tuning on proprietary data or guaranteed availability offline.
If none of these apply, a hosted API with a solid data processing agreement may serve you perfectly well. We say this openly because the worst outcome is a GPU cluster purchased for a use case that never needed one. The businesses that succeed with private deployments are the ones that can name the constraint they are solving.
The temptation is to start with something ambitious: a company-wide assistant that knows everything.
Resist it. The best first projects are narrow, measurable, and internal
Good candidates include summarising internal documents, answering questions over a defined knowledge base (contracts, policies, technical manuals), classifying and routing incoming emails or tickets, or drafting first versions of routine reports.
Each of these has a clear input, a clear output, and a human who checks the result. That last part is important early on, because it gives you an honest read on quality before anything customer-facing depends on it.
We worked with our previous client on exactly this pattern: moving from data overload to automated, insight-driven reporting, while keeping accuracy and compliance intact. The scope was defined, the output was checkable, and the system earned trust before it earned more responsibility.
Model selection causes more delay than it should.
The practical answer in 2026 is that mid-sized open models are good enough for most business tasks.
A model in the 7B to 30B parameter range, quantised to run efficiently, handles summarisation, question answering, and drafting well. It runs on a single decent GPU server or a modest private cloud instance.
The frontier open models, the ones topping benchmarks with hundreds of billions of parameters, need multi-GPU infrastructure and are rarely the right starting point. You can always scale up once the use case proves itself. Scaling down a purchase is harder.
Licensing deserves ten minutes of attention. Many open models are released under Apache 2.0 or MIT licences with no commercial restrictions, but some carry conditions. Check before you build, not after.
For Asian markets specifically, multilingual capability is worth testing early. Several open model families now handle Bahasa Indonesia, Thai, Vietnamese, and Chinese well, but performance varies by model and task, so run your own evaluation on real examples rather than trusting a benchmark table.
A minimal private LLM stack has three parts: an inference server that runs the model (tools such as Ollama for prototyping, or vLLM for production workloads), a retrieval layer if the model needs to answer from your documents, and a thin application layer with authentication and logging in front of it.
That is the whole thing.
You do not need an agent framework, a vector database with twelve integrations, or a platform team on day one.
A prototype on a single server, tested with real users on real documents, tells you more in three weeks than an architecture diagram tells you in three months.
There is a middle path worth knowing about as well. Regional cloud providers in the EU and across Southeast Asia offer GPU instances inside your jurisdiction. Running an open model there gives you data residency and control without buying hardware, which suits teams that want to validate before committing to on-premise infrastructure.
Before the pilot starts, agree on what success looks like. Useful measures are concrete: time saved per document processed, percentage of answers rated correct by the reviewing team, tickets routed without human correction. Vague goals such as "improved productivity" make it impossible to decide whether to expand or stop.
Also budget for the unglamorous work.
Someone owns model updates, someone monitors output quality over time, and someone decides what happens when the model gives a wrong answer with commercial consequences. Private deployment means you own the stack, and ownership includes maintenance.
For a contained use case, a sensible plan looks like this.
Two weeks to define the use case, select a model, and set up the environment. Three to four weeks to build the retrieval layer and a basic interface, then test with a small user group. Two weeks to evaluate against the agreed measures and decide whether to expand, adjust, or stop.
Roughly two months from decision to evidence. Compare that with the eighteen-month transformation programmes this space tends to attract, and the appeal of starting small becomes obvious.
We have been building AI-driven systems for years, and much of our current work involves exactly this: helping European businesses move from AI experiments to systems that run inside their own infrastructure and their own compliance boundaries. Our teams handle the engineering, from model deployment and retrieval pipelines to the application layer around them, while you keep control of the data and the direction.
If you are weighing up a private LLM and want a second opinion on the use case, the model, or the infrastructure, we are happy to talk it through. Feel free to reach out via itsavirus.com/contact-us
.jpeg)
For most business tasks such as summarisation, document Q&A, and drafting, mid-sized open models are now good enough. Frontier proprietary models still lead on complex reasoning and multimodal work, which is why many organisations run both: hosted APIs for experimentation, private models for sensitive or high-volume workloads.
For a pilot, a single server with a capable GPU, or a GPU instance from a regional cloud provider, is usually sufficient. Models in the 7B to 30B parameter range run comfortably on this class of hardware when quantised.
It depends on volume. APIs are cheaper for low or unpredictable usage. Private deployment becomes economical when workloads are high and stable, because you shift from per-token pricing to fixed infrastructure costs.
