Researchers led by Eötvös Loránd Tudományegyetem (ELTE) have developed the Racka-4B model on the Komondor HPC infrastructure available to the Hungarian academic community. According to the team, Racka-4B achieves performance on Hungarian language tasks comparable to some models twice its size (8 billion parameters) while operating substantially faster. The developers highlight the model's logical and reasoning capabilities.
Why this matters for Hungary
ELTE's statement emphasizes that large global language models often struggle with the Hungarian language and cultural content. One reason is Hungarian's rich morphology: single word forms can contain many affixes and derivational elements, concentrating a lot of information in one token. Furthermore, there is no major world language closely related to Hungarian whose digital presence would indirectly benefit Hungarian. If language systems do not adequately understand and represent Hungarian legal, cultural or technical terminology, this can create disadvantages; training on domestic data is therefore important for preserving digital sovereignty.
How the model was adapted to Hungarian
Racka-4B is based on open-source foundations, using a Qwen3-4B backbone and parameter-efficient LoRA fine-tuning. The Hungarian adaptation involved several parallel engineering steps, the most significant being optimization of the model's "vocabulary" (tokenization).
International models trained mostly on world languages typically split Hungarian words into many tiny, semantically unhelpful pieces. The research team removed roughly 32,000 rare tokens (for example, certain Far Eastern characters) from the original vocabulary of about 150,000 entries and replaced them with tokens specifically optimized for Hungarian. They also retuned the system to prefer those new tokens.
Effects of token optimization and training
As a result of these changes, Racka can represent the same Hungarian text with 47 percent fewer tokens. In practice this yields much faster text generation and a notable reduction in grammatical and inflectional errors. The model was further trained on a dataset totaling 160 billion tokens: 44 percent of that data is Hungarian, with the remainder comprising English (24 percent), German (21 percent) and program code (11 percent).
Availability and future plans
The research group aims to support the domestic scientific sector: Racka is made openly and freely available for research and development purposes. According to the developers, the project’s Hugging Face page recorded more than 600 downloads in the past month, indicating community interest.
The team envisions a whole Racka model family rather than a single model and plans to expand to the Central and Eastern European regional market. Preparation and development of a larger model that integrates broader regional knowledge is already underway in close cooperation with Mynds.ai, with implementation planned on the emerging European AI infrastructure and the MareNostrum 5 supercomputer in Barcelona.
Brief summary
Racka-4B is a parameter-efficient, Hungarian-optimized reasoning language model developed by ELTE and partners. Through vocabulary optimization and continued training on 160 billion tokens, it produces Hungarian text with fewer tokens, faster generation and fewer inflectional errors than larger, non-Hungarian-optimized counterparts, and it is released openly for research use.
