Hugging Face Jobs allows you to launch a private, OpenAI-compatible LLM endpoint using vLLM with a single command on HF infrastructure — no server provisioning or Kubernetes required, and billing is per second. This is a fast way to stand up a model for tests, evaluations, or batch generation. If you need a managed, production-ready service instead, Hugging Face Inference Endpoints are the alternative (see the guidance below on when to choose which).
Prerequisites
- A payment method or positive prepaid credit balance (Jobs charges by hardware usage). The post cites an a10g-large flavor at about $1.50/hour.
- huggingface_hub >= 1.20.0 (install with: pip install -U "huggingface_hub>=1.20.0").
- Logged in locally with: hf auth login
Launch the server
hf jobs run is HF’s equivalent of docker run. The example uses the official vllm/vllm-openai image, requests a GPU with --flavor, and exposes vLLM’s port with --expose:
hf jobs run --flavor a10g-large --expose 8000 --timeout 2h
vllm/vllm-openai:latest
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
The command prints the job ID and URL. Example output from the post:
✓ Job started id: 6a381ca1953ed90bfb947332 url: https://huggingface.co/jobs/qgallouedec/6a381ca1953ed90bfb947332
Exposed ports are reachable through the jobs proxy, e.g.:
https://6a381ca1953ed90bfb947332--8000.hf.jobs
Allow a few minutes for weights to download and the model to initialize. When logs show "Application startup complete", the server is ready.
Query it from anywhere
vLLM speaks the OpenAI API, and requests must include your HF token as a bearer token. Quick curl example:
curl https://<job_id>--8000.hf.jobs/v1/chat/completions
-H "Authorization: Bearer $(hf auth token)"
-H "Content-Type: application/json"
-d '{
"model": "Qwen/Qwen3-4B",
"messages": [{"role": "user", "content": "Hello!"}],
"chat_template_kwargs": {"enable_thinking": false}
}'
The response is OpenAI-style JSON; choices[0].message.content contains the assistant reply (example: "Hello! How can I assist you today? 😊").
From Python, set the OpenAI client’s base_url to the job’s exposed URL and use get_token() from huggingface_hub as the api_key. The post includes sample code that creates a chat completion and prints resp.choices[0].message.content.
A quick health check before use:
curl https://<job_id>--8000.hf.jobs/v1/models -H "Authorization: Bearer $(hf auth token)"
Security and access
The endpoint is gated: every request must carry an HF token with read access to the job’s namespace. A plain browser visit will be rejected. The jobs proxy acts as your API gate, scoping access to your user or organization. Don’t share the URL expecting public access and avoid pasting your token into untrusted places.
Clean up and cost
Stop the job when finished:
hf jobs cancel <job_id>
The --timeout you set provides an automatic stop, but cancelling explicitly is cheaper. The post notes that an a10g-large costs roughly $1.50/hour; run hf jobs hardware to view the full price list and choose the smallest flavor that fits your model.
Going further: bigger models
The same command works for larger models by selecting a larger --flavor and telling vLLM to shard across GPUs with --tensor-parallel-size. Example for the 122B Qwen3.5 mixture-of-experts model on 2× H200:
hf jobs run --flavor h200x2 --expose 8000 --timeout 2h
vllm/vllm-openai:latest
vllm serve Qwen/Qwen3.5-122B-A10B
--host 0.0.0.0 --port 8000 --tensor-parallel-size 2
--max-model-len 32768 --max-num-seqs 256
--tensor-parallel-size should match the number of GPUs in the flavor (h200x2 → 2, h200x8 → 8). Large models take longer to download and load, so give a longer --timeout. For big models, H200 flavors are usually the best value.
The example includes --max-model-len and --max-num-seqs because Qwen3.5-122B’s hybrid architecture and default large context can exhaust GPU memory with vLLM’s default batching; capping context length and concurrent sequences can avoid out-of-memory or cache-block errors.
Going further: chat UI, SSH, and agent backend
-
Chat UI: A small Gradio app can point at the same endpoint. Adding --reasoning-parser deepseek_r1 to vllm serve lets Qwen3 stream reasoning as a separate field; the post gives a local Gradio ChatInterface example that shows a collapsible thinking panel and the answer beneath.
-
SSH into the running server: For debugging, monitoring GPU memory or tailing logs interactively, start the job with --ssh and ensure your public key is registered at huggingface.co/settings/keys. Then connect with hf jobs ssh <job_id>. SSH support requires huggingface_hub >= 1.20.0.
-
Use it as a coding-agent backend with Pi: The same endpoint can back a terminal coding agent (Pi). vLLM must be launched with tool calling enabled (--enable-auto-tool-choice and a matching --tool-call-parser, e.g. hermes for Qwen3). The post shows how to add the job as a custom provider in ~/.pi/agent/models.json and then launch the agent.
HF Jobs or Inference Endpoints?
HF Jobs offers maximum flexibility and control — you pick the image, vllm flags and hardware, and you pay per second while the job runs, which makes it suitable for experiments, one-off evals and batch jobs. Inference Endpoints are the managed product that provide operational features for long-lived services: finer-grained access control (public/protected/private endpoints) and scale-to-zero so you aren’t billed during idle periods. Choose Jobs for short-lived or highly customizable runs, and Inference Endpoints for durable, production-ready deployments.
Further reading
This post focuses on vLLM, but the same port-expose pattern works with any OpenAI-compatible server. The Serve Models on Jobs guide covers other backends, such as serving GGUFs with llama.cpp or running SGLang.
