Tools

AI-generated text

Building Small, Cheap Image and Text Models Using Hugging Face's ML‑Intern

A user leveraged HuggingChat's ML‑intern agent to design, train and publish several compact vision and text models on Hugging Face with very low compute costs.

Building Small, Cheap Image and Text Models Using Hugging Face's ML‑Intern

A user employed the ML‑intern agent in HuggingChat to design, train and publish several compact image and text models on the Hugging Face Hub with very low compute expense. By supplying detailed prompts that include datasets, base models, budgets, baseline checks and smoke tests, the agent automated dataset preparation, training runs, evaluation and publishing. The reported total compute cost across seven projects is about USD 103.

How ML‑intern was used

Work began with a single, carefully written message to ML‑intern. Each prompt typically contained a one‑line project idea and motivation, explicit dataset and base model names, the training script, and a section titled "Verified facts, do not re‑derive" so the agent would not waste budget re‑establishing known details.

Two prompt elements the author found critical were:

  • A baseline measurement: asking the agent to report the base model's zero‑shot score on the same metric before training so the improvement is measurable.
  • A smoke test: a short, cheap trial run (for example 50 training steps) followed by a check (for instance that saved weights actually changed) before approving the full run.

At the prompt's end the author specified expected deliverables and set a spending cap (e.g. "Cap total spend at USD 12 and ask me before exceeding it"). ML‑intern starts every task with a zero dollar budget and asks for permission before running paid jobs, which enforces cost limits. All original prompts are published on GitHub at yvrjsharma/ml‑intern‑prompts.

Six projects built in a few days

Below are the main details — datasets, base models, performance and compute costs — for several projects the author created.

1) Citrus disease classifier (citrus‑disease‑vlm‑instruct)

  • Data: a merged dataset from three Project‑AgML sources on the Hub, 3,017 annotated images covering 21 pests, diseases, nutrient deficiencies and treatments.
  • Base model: Qwen3.5‑2B. ML‑intern measured the base model first.
  • Result: on 335 test photos, the base model named the correct problem 14.9% of the time. After two epochs on a single A10G, the fine‑tuned model reached 52.8%.
  • Compute cost: about USD 1.90.

2) Huggy character LoRA

  • Data: 84 captioned drawings from the Chunte/huggy_for_training dataset.
  • Base model: FLUX.2 klein base 4B.
  • Method: the agent saved checkpoints every 100 steps and rendered the same prompts to compare outputs. Step 200 was the first point where Huggy was fully "on‑model"; by step 500 the style started appearing in unrelated prompts. The LoRA also worked on the distilled klein model after 4 steps.
  • Compute cost: about USD 7.60.

3) Camera‑angle LoRA — Viewpoint Orbit

  • Data: from Google Scanned Objects, 1,030 scanned household objects rendered at 24 angles (24,722 transparent images); 461 objects were finalized for training and 40 held out for testing; 1,844 before/after training pairs across 23 camera instructions.
  • Base model: Qwen‑Image 2.1.
  • Training: 2,000 steps in about 90 minutes on one A100.
  • Project duration: about half a day, 48 jobs including retries for failed runs.
  • Compute cost: about USD 16.

Doodle‑in LoRA (related project)

  • Goal: replace a magenta scribble in a photo with a named object while preserving lighting and composition.
  • Data construction: start from Open Images, remove an object using LaMa inpainting, add a scribble where the object was; the untouched photo is the target. ML‑intern wrote and tested the pairing scripts in a CPU sandbox and executed them on GPUs, recording source authors and licenses.
  • Size: 6,042 training pairs and a 160‑pair test set (40 of those from 23 object classes withheld from training).
  • Training: 2,000 steps in 1 hour 38 minutes on one A100; step 500 was chosen after comparing checkpoints on 48 test pairs.
  • Performance: paired with the Viggle turbo LoRA at 6 steps, 67.5% of objects appeared where the scribble was drawn, at 4.7 seconds per edit. Unseen classes performed comparably (65.0% vs 64.2%).
  • Project time: a little over a day, 59 jobs.
  • Compute cost: about USD 24.

4) Pocket Rewriter — CPU‑friendly student models (0.8B and 2B)

  • Start: the author generated 8,797 short image requests using a small instruct model through Inference Providers. The 9B teacher model then rewrote all requests on one A100 in 2 hours 37 minutes (~USD 6.50). After quality filtering, 1,840 examples were selected for training.
  • Training: the 0.8B and 2B students trained in 12 and 18 minutes respectively on an A10G (USD 0.75 total). The 0.8B student is distributed as an 812 MB GGUF file runnable on CPU.
  • Project time and cost: about 11 hours, 24 jobs; compute cost about USD 16.

Agate Preview 002 — 4‑step distillation

  • Base: Logolabs' Agate Preview 002, a 260M‑parameter TTI model that typically needs 50 steps with guidance (100 network passes per image).
  • Goal: distill it to a 4‑pass (4‑step) model for browser usage.
  • First run: cache 155,000 training images as latents, bake guidance into the model, cut steps in stages from 16 to 8 to 4 on A100s. The 4‑step student outperformed the teacher's 4‑step run on GenEval and FID. Exported to ONNX. This run took ~13 hours and ~USD 22.
  • Second run: generate 24,000 more training pairs from the teacher at 16 steps and fine‑tune the 4‑step student for ~1 hour. GenEval improved from 0.509 to 0.536 (teacher at 50 steps: 0.563). Second run ~8 hours. Total for both runs: about USD 37.

Reported compute cost breakdown (as reported by the author)

  • Citrus Doctor (Qwen3.5‑2B): USD 1.90
  • Huggy LoRA (FLUX.2 klein base 4B): USD 7.60
  • Pocket rewriter (0.8B and 2B): USD 16.05
  • Viewpoint Orbit LoRA (Qwen‑Image 2.1): USD 16
  • Doodle‑in LoRA (Qwen‑Image 2.1): USD 24.30
  • Agate 4‑step (two runs, Agate Preview 002): USD 37
  • Total: about USD 103

These figures are the GPU and CPU job charges reported for each session by the author.

How to reproduce

ML‑intern is available inside HuggingChat. Turn on ML‑intern mode and paste a prompt. The author's example prompts are free to copy as a starting point. Practical advice:

  • Start from a model you want and a dataset you have or can describe.
  • Give a small budget, require a baseline and a smoke test.
  • Read the agent's plan and outputs before increasing spend.

If you publish a model made this way, the author suggests sharing it on X and tagging @Gradio and @HuggingFace so the community can see novel models others create.

Closing note

The examples illustrate that with careful prompt engineering and an automated agent, it is possible to rapidly produce useful, small models for tasks ranging from disease identification and character styling to viewpoint editing and model distillation, while keeping costs low. The author has published full prompts and artifacts for others to reuse or adapt.