Racka-4B is a large language model (LLM) developed with Hungarian expertise by researchers at the Department of Artificial Intelligence, Faculty of Informatics, Eötvös Loránd University (ELTE) in collaboration with the Digitális Örökség Nemzeti Laboratórium. The team also cooperates with Mynds.ai and is preparing a follow-up project to build a substantially bigger model.
What is Racka capable of?
Racka-4B is a relatively small model with 4 billion parameters. Its notable feature is reasoning: it belongs to the class of so-called "chain-of-thought" or "reasoning" models that produce step‑by‑step, traceable reasoning before delivering an answer. Such models became widely recognized following recent developments in the field, including models from OpenAI.
The developers emphasize that Racka is not intended to compete directly with large commercial services like ChatGPT. Its primary purpose is to serve as an experimental platform to capture know‑how, explore Hungarian optimization and support specialised use cases where language and cultural context matter.
How does it differ from earlier Hungarian attempts?
- Racka is the first Hungarian‑focused reasoning model that intentionally incorporated a high share of cultural heritage materials in its training corpus — books, dissertations and other value‑bearing texts — rather than relying solely on web crawl data.
- Starting from the open‑source Qwen3-4B model (developed by Alibaba Cloud), the team removed 32,000 tokens from the original ~150,000 token vocabulary that were irrelevant for the project and replaced them with tokens optimized for Hungarian. The final training corpus contained 44% Hungarian text to avoid "catastrophic forgetting" that can occur if a previously general model is trained only in one language.
- Racka was trained on a high‑performance computing environment, specifically the Hungarian Komondor supercomputer, enabling scalability to larger models in subsequent projects.
The corpus also included English and German data and program code to retain translation and coding capabilities.
What can Racka do now, and what are its limits?
The developers acknowledge that Racka's size sets limits: for highly complex, multilingual or broad knowledge tasks, larger commercial models typically perform better. Nevertheless, Racka is already useful for certain specialised tasks. The reporting notes an example where a forensic expert used an experimental Racka‑based tool to sift through large volumes of documents while the system ran on a small external storage device.
Because the reasoning mechanism derives from the base Qwen3-4B model and internally operates in English, the team points out that outcomes can differ from reasoning done directly in Hungarian, especially for culture‑ or language‑specific problems.
Why build a domestic model?
The developers put forward several arguments for a homegrown model:
- Digital sovereignty: reliance on foreign (mainly US or Chinese) models means sending valuable data offsite and increasing dependence on external providers.
- Language coverage: for smaller languages like Hungarian, global models typically perform worse because they are trained on much less native‑language data. This gap is exacerbated for training data that strengthen reasoning abilities, which are often not available in Hungarian unless created manually or synthetically.
- Capacity building: developing smaller research LLMs helps build local expertise that is necessary not only for future LLM development but also for services built on top of them.
Mynds.ai CEO Radványi Miklós emphasized that there are domains — legal texts, healthcare, government or national security data — where organisations are unwilling to send data to foreign servers and therefore need locally deployable, auditable models.
Next steps and scaling up
The research group concurs that increasing model size is the natural next step: even a 30‑billion‑parameter model would open up many more complex applications. Racka's architecture is designed to scale, and the team is preparing a joint project with Mynds.ai to create models that can be integrated into agent systems and perform complex tasks autonomously.
Larger training runs require more compute; the team has considered using international supercomputers such as MareNostrum 5 in Barcelona and earlier applied for access to LUMI. Domestic infrastructure like Komondor — and the announced but not yet completed Levente supercomputer — are also part of long‑term planning.
Funding, government role and sustainability
Developers stress that sustained public funding or government contracts are crucial. The refreshed Hungarian AI strategy mentioned state‑backed development of Hungarian LLMs and named the Digitális Örökség Nemzeti Laboratórium, which has worked on projects like a Hungarian handwriting recognition model for archival documents. However, the researchers say that national laboratory programmes and stable budgets remain uncertain.
Palkó Gábor warns that without predictable funding, efforts tend to become ad hoc or "garage" projects, with researchers often participating without pay. Both Palkó and Radványi expect that state support and procurement will be necessary to build larger, production‑grade models suitable for public administration and industry.
Regional and commercial ambitions
The team aims to extend the models to cover neighbouring Central‑Eastern European languages so the resulting systems could be used not only in Hungary but also in countries like Slovakia or Romania. Achieving that will depend on winning larger grants, forming international partnerships, and accessing significant compute resources.
In sum, Racka demonstrates that Hungarian‑optimized, reasoning‑capable LLMs can be built and used for specialised tasks, and it lays groundwork for scaling to 30+ billion parameter models. The developers argue that technical scalability must be matched by stable funding and infrastructure if such systems are to be deployed broadly for government and industry use while preserving digital sovereignty.
