Researchers at the Department of Artificial Intelligence of Eötvös Loránd University (ELTE), in collaboration with the National Laboratory for Digital Heritage, have developed Racka‑4B — the first large reasoning language model specifically tuned for Hungarian. The development was carried out by a nine‑person team of AI engineers and language technologists.
Data volumes and performance
The team processed data equivalent to roughly 200 million pages on the Komondor HPC infrastructure, Hungary’s largest supercomputer. Racka‑4B’s performance on Hungarian tasks improved to the point where it is comparable to some 8‑billion‑parameter models, while its runtime speed is substantially faster. The model was further trained on a total of 160 billion tokens.
Why a Hungarian model matters
Global AI providers typically optimize their models for English, which creates difficulties for languages like Hungarian that are morphologically rich: single word forms can contain many suffixes and derivational elements. If systems do not handle the language, local culture, legal terminology or domain knowledge well, the country risks losing advantages in areas that depend on accurate language understanding. Building technology trained on local data is therefore important for digital sovereignty.
Technical approach: vocabulary optimization and LoRA tuning
Racka is based on the open‑source Qwen3‑4B model and was adapted using a parameter‑efficient LoRA procedure. A central technical step was redesigning the model’s vocabulary: from the original roughly 150,000‑entry vocabulary the researchers removed about 32,000 rare tokens (for example certain East Asian characters) that were irrelevant to the project. They replaced those with tokens optimized for Hungarian and tuned the system to prefer using them.
As a result, Racka requires 47% fewer tokens to represent the same Hungarian text. In practice this yields much faster text generation and a marked reduction in grammatical and inflectional errors.
Training data composition and multilinguality
Although the focus was Hungarian, only 44% of the training data was domestic text. The remainder was English (24%), German (21%), and program code (11%). Including foreign language data prevented catastrophic forgetting — the loss of previously learned general capabilities — and helped preserve wide‑ranging knowledge. Program code was included specifically to help retain the base model’s logical and reasoning abilities.
Open release, adoption and recognition
Racka is freely available for research and development use. The project has seen rapid uptake: in the last month the model was downloaded more than 600 times from its Hugging Face page. The team’s paper and demonstration received the best publication award at the XXII. Magyar Számítógépes Nyelvészeti Konferencia (MSZNY).
Roadmap: a family of models and regional ambition
The researchers are planning a Racka model family rather than a single instance, aiming to address needs from compact models that run securely on local servers or mobile devices to much larger systems capable of interpreting long documents and making complex inferences. Work on a larger, regionally integrated model is already underway in close cooperation with Mynds.ai. That development is planned to use emerging European AI infrastructure and the MareNostrum 5 supercomputer in Barcelona.
Racka‑4B illustrates how resource‑intensive academic development can be aligned with practical, regional needs while preserving language‑specific and cultural details.
