Switzerland has unveiled its first national large language model (LLM) for use in research, education and commercial applications, amid a wider tech sovereignty push across Europe.
Governments across Europe have accelerated investment in so-called “national LLMs”, including the UK and Ukraine. Unlike commercially-available models offered by OpenAI or Google, national AI models would be designed to comply with local standards from their inception, and not be subject to foreign regulatory or financial pressure.
The Swiss LLM — built in collaboration with researchers at Swiss universities EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) — is trained on public data sources from over 1,000 languages and is run on infrastructure from the CSCS, Europe’s second most powerful supercomputer.
The model named Apertus — Latin for “open” — comes amid growing concern in Europe over the transparency of leading AI models from Big Tech that often do not disclose how their AI systems are designed, or which data is used to train them.
Regulatory frameworks like the incoming EU AI Act aim to address these issues by requiring clear communication and labelling of AI systems, as well as openness regarding training data.
The creators of the Swiss LLM says its model architecture, weights and training data is openly accessible and fully documented. Anyone can build upon the model and adapt it to their specific needs, as well as inspect any part of the training process.
They say the model has also been designed with ethical principles in mind and will refuse certain requests that could harm the user.
The model
In terms of performance, the Swiss LLM — featuring 8bn and 70bn parameters — is smaller than the best models from Big Tech and has been trained on less data.
While the models are not comparable to much larger frontier models, the Swiss AI initiative says using the LLMs has certain advantages for European companies, particularly when it comes to compliance.
Imanol Schlag, a research scientist at the ETH AI Centre, says industry researchers and developers may soon be legally required to disclose the training data used when developing their AI product or service.
“If such a service is built upon an opaque open-weight model from a big US or Chinese technology company, then it is very unlikely that such transparency will be possible,” he says.
The largest and best-performing models are typically expensive to run, he says, and not all tasks require “the latest and greatest model”.
The idea is to continue to develop and improve the LLM over time, incorporating feedback from stakeholders using it, and explore domain-specific adaptations in fields like law, climate, health and education.
The Swiss LLM has funding for four years, including €20m CHF from the Swiss government and over 100m GPU hours on Swiss computing infrastructure, ‘Alps’.
The LLM is being released under the Apache 2.0 license — an open-source software license allowing users to modify and distribute the software for any purpose.
It will be available via the Swisscom Swiss AI platform or users can download it from Hugging Face.
