A public, low-cost recipe demonstrates how to make a small language model substantially better at producing schema-compliant structured outputs. The authors fine-tune LiquidAI/LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library and evaluate gains on the IFStruct benchmark. The full pipeline uses roughly 500 training samples and 100 training steps, so it can run on free-tier Colab or Kaggle GPUs, and the runnable notebook is available on GitHub.
Why this matters
Structured output — returning parseable JSON or YAML that matches a requested schema and shape — is a central real-world LLM task. Whether a model reliably produces valid output in the expected format often determines if it can be integrated into downstream systems. Many benchmarks mix structure compliance into broader scores; IFStruct focuses specifically on schema adherence and output validity.
Baseline evaluation of the base model
Before fine-tuning, the authors evaluated LiquidAI/LFM2.5-350M locally via llama.cpp using the BF16 GGUF checkpoint. Running the full IFStruct suite on 2000 examples produced:
- Overall: 452/2000 passed (22.6%)
- JSON: 180/1000 passed (18.0%)
- YAML: 272/1000 passed (27.2%)
- Average latency: 1453 ms
The run also lists pass rates by top-level structure and entity type and reports common errors (e.g., required field missing, wrong item count, type mismatch). This local measurement is close to the IFStruct blog's 21.1% figure and is used as the baseline for the identical serving-stack comparison.
GRPO fine-tuning for structured outputs — procedure
The full runnable pipeline is provided in the accompanying notebook; below are the relevant components.
Training data
The training data is nvidia/Nemotron-RL-instruction_following-structured_outputs, which pairs each prompt with a target JSON Schema and an expected top-level field count. About 500 samples were used for training.
To reduce distributional gaps between Nemotron and IFStruct, prompts were augmented:
- 40% of prompts receive an appended instruction to place the output inside a fenced code block, teaching the model to follow explicit format instructions.
- A disjoint 20% are converted into top-level-array tasks (the schema wrapped in an array with a required item count) to train bare-list output and item-count compliance.
Model and LoRA
LiquidAI/LFM2.5-350M was loaded and a LoRA adapter attached. Because LFM2.5 uses a hybrid attention/convolution architecture, the LoRA targeted LFM-specific modules (q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3). The LoRA configuration (r=16, lora_alpha=32, bias="none") corresponds to roughly 6M trained parameters, about 1.66% of the model.
Reward functions
Three reward functions on a [0,1] scale were defined to score completions for structural correctness:
- json_format_reward: is the output parseable and in the requested form (1.0 for requested form, 0.2 for wrong-but-parseable form, 0.0 for unparseable output).
- field_count_reward: does the object have the expected number of top-level fields (1.0 for exact match, linearly decaying with miss).
- schema_validation_reward: does the output validate against the JSON Schema (counts constraint violations and gates partial credit on required-key coverage).
These were combined as a weighted sum with reward_weights=[1.0, 0.5, 2.0] (json_format, field_count, schema_validation).
Training run
The GRPO training used 100 steps with 8 generations per prompt group and settings sized for a free-tier 16 GB GPU: learning_rate=5e-5, warmup_steps=10, per_device_train_batch_size=4, gradient_accumulation_steps=8, steps_per_generation=2, max_completion_length=1024, temperature=1.1, beta=0.01, and logging/save intervals as shown in the notebook. During the run all three reward components increased, the KL penalty away from the reference model rose after warmup, and truncated completions remained low.
Merging and saving
After training, the LoRA adapter was merged back into the base model weights and saved as a single checkpoint. The merged checkpoint was then converted to a BF16 GGUF for serving with llama.cpp.
Evaluation of the GRPO-tuned model
The merged, converted model (served as lfm25-350m-grpo-structured-output) was evaluated with the same IFStruct test set. Results on 2000 samples:
- Overall: 594/2000 passed (29.7%)
- JSON: 319/1000 passed (31.9%)
- YAML: 275/1000 passed (27.5%)
- Average latency: 1518 ms
Several entity groups showed notable improvements (for example, clinical_trial 19.2% → 29.8%, log_parser_examples 29.2% → 45.8%, support_ticket_batch 37.0% → 49.3%, event_ticket_booking 45.8% → 57.9%), while some error categories persisted (e.g., required field missing remained the most frequent error).
Direct comparison (same serving stack)
- Overall: 22.6% → 29.7% (+7.1)
- JSON: 18.0% → 31.9% (+13.9)
- YAML: 27.2% → 27.5% (+0.3)
- Wrapper key: 28.5% → 29.7% (+1.2)
- Bare list: 16.6% → 29.7% (+13.1)
The largest gains align with the training objectives: JSON pass rate improves substantially while YAML remains nearly unchanged. Although the tuned 350M model still falls short of Qwen3.5-2B's 33.15% score, the result shows a light, task-specific fine-tuning procedure can bring a small model significantly closer to larger-model performance on structure compliance.
Conclusion
A short GRPO run using about 500 samples and 100 steps can raise a 350M-parameter model from 22.6% to 29.7% on IFStruct. The key takeaway is that a cheap, task-specific reward signal focused on format and schema validation can materially improve a small model's reliability for structured outputs, narrowing the gap to larger models. The notebook, IFStruct v1.0 blog, Liquid4All/ifstruct benchmark repo, and the LiquidAI/ifstruct-v1.0 dataset are provided for reproduction and extension.
Implementation notes (commands summary)
The notebook includes example commands for serving models with llama-server (BF16 GGUF), converting merged checkpoints to GGUF with the llama.cpp converter, and running IFStruct evaluation via the uv tool.
