AI: Local models strengthen SMEs and independence
4 min read
Language models running on their own hardware can give companies more control over their AI applications. Google’s Gemma 4 model family is designed for a range of devices and can run locally. Locally executed requests incur no usage-based fees for an external model API. Only a test using a company’s own tasks can show whether running the model is cost-effective and whether its quality is sufficient.
Key points at a glance
- Local operation can keep data within the company. This requires connected applications and services to be configured accordingly.
- Costs shift. Instead of paying for an external model API, companies incur expenses for hardware, electricity and operations, among other things.
- Quality and data protection require separate assessment. A pilot using representative tasks shows whether the model and infrastructure suit the intended use.
What local language models enable
What is a local language model? A language model runs locally when its computations take place on infrastructure owned or managed by the organisation itself. Inputs do not need to be sent to an external model API for this computation. Whether data leaves the company elsewhere depends on the application as a whole and its connections.
In its Gemma 4 model card, Google describes variants for different environments, from high-performance mobile devices to workstations and servers. These include smaller models such as E2B and E4B as well as larger variants. The hardware sufficient for an application depends on factors including the model selected, processing speed and the number of simultaneous requests.
Running a model locally can reduce dependence on an external model API. Choosing a model also creates a relationship with its developer: Gemma 4 comes from Google. Hardware, software and subsequent updates may also depend on other providers.
Putting costs and data protection in context
Companies that run a model themselves pay no usage-based fees to an external model API for those requests. They must, however, provide and operate the necessary infrastructure. A cost assessment should include purchases or existing capacity, electricity, setup, maintenance and integration. Whether this is worthwhile compared with a cloud service depends on actual use.
Keeping personal data within the company’s own network does not remove the obligation to process it lawfully and securely. Article 32 of the GDPR requires appropriate technical and organisational measures based on the relevant risks. As part of the project, this includes checking who can access inputs, outputs and logs.
According to Google, Gemma 4 is offered under the Apache 2.0 licence. It generally permits commercial use. Distribution is subject to the terms of the licence, including requirements concerning licence copies and notices of changes. If additional components are used, their own terms must also be checked.
Start with a limited use case
A suitable pilot begins with a manageable task, such as summarising internal instructions or categorising incoming documents. The team decides in advance which data may be used and what standard of quality an output must meet. Approved or specially prepared documents are suitable for the initial test.
A qualified person assesses the responses against typical cases and checks for errors. Language models can produce incorrect or incomplete results. For tasks with legal, financial or other significant consequences, a responsible person should therefore review the output before it is used.
Only then can the company decide whether local operation is suitable for the chosen process. Alongside response quality, the assessment includes speed, access rights, maintenance effort and how the model fits into existing workflows.
Frequently asked questions
Does local AI need an internet connection?
A model that has already been set up can run locally without an external model API. An internet connection may be needed for downloads, updates or connected services. The specific setup determines what is required.
Does local AI automatically comply with data protection law?
No. Even when processing personal data locally, companies must meet GDPR requirements and implement appropriate security measures.
How can the quality of a local model be assessed?
The best approach is to use representative tasks from the company and criteria defined in advance. Experts should also examine what errors occur and what consequences they would have in the intended process.
Image source: AI-generated (November 2026)
Translated from the German original using artificial intelligence. The German version is authoritative.
