Challenges of Artificial Intelligence Through the Slovene Large Language Model GaMS
Prof Dr Marko Robnik Šikonja delivered a keynote speech at the 50th Conference of the Slovenian World Congress, held on 21 October 2026 at the Jožef Stefan Institute. The talk, titled Challenges of Artificial Intelligence Through the Slovene Large Language Model GaMS, addressed public perceptions of AI, the stages of LLM training, AI agents, and the presentation of the Slovene large language model GaMS.
The talk began by examining public perceptions of AI, emphasising how fear is a powerful emotion that is currently being weaponised by major AI companies for profit. Understanding and knowledge are among the best ways to counter this fear. So, how do large language models (LLMs) actually work?
LLMs are based on a neural network architecture called the transformer. Their training takes place in several stages. In the first stage, researchers pre-train the model using vast amounts of data. In the second stage, the model is trained to perform specific tasks and engage in conversations. This process is considerably faster than the initial pre-training stage. Finally, in the third stage, the model is fine-tuned to align with human preferences, including learning which types of responses are acceptable and which should be avoided.
From LLMs to Agents
What are the latest trends in AI? One of the most significant developments has been the shift from large language models to AI agents. When using commercial AI tools such as Claude or ChatGPT today, we are increasingly interacting not just with LLMs, but with agentic systems. While traditional LLMs generate text in response to human prompts, leaving humans to interpret the output and decide what to do next, AI agents go a step further. They can plan and execute a series of actions to achieve a given goal, using various tools along the way, such as web search, data analysis and code execution. In short, in recent years, we have moved from AI as an adviser to AI as an agent.
Large Language Model for Slovene: GaMS
The second part of the talk focused on the development of GaMS, an open-source large language model for Slovene. Slovene is one of the few European languages with its own open-source LLM. GaMS represents a key part of Slovenia’s AI infrastructure and an important foundation for Slovenia’s digital sovereignty.
GaMS was developed using Google’s Gemma 2 and Gemma 3 models. Its training relied on extensive language resources, including corpora developed by the Centre for Language Resources and Technologies over more than a decade. Additional training data came from Wikipedia, the Slovenian Press Agency, the National and University Library, and a citizen science campaign in which members of the public contributed their texts. GaMS is freely available as a chatbot and on Hugging Face.
What does the future hold for AI? Key developments include multimodal models, large-scale multi-agent systems and smaller language models tailored to business applications. At the same time, increasing attention will be paid to AI explainability and ethical challenges, particularly issues of bias, fairness and transparency.
