The Hugging Face WebAI team aims to make browser inference both fast and easy to use. Achieving this requires work across multiple layers: model representations for browsers, runtimes that produce efficient execution plans, and low-level GPU operations that exploit diverse devices and browser implementations.
Today they released the first part of that effort:
- A collection of 207 WebGPU kernels published as individual repositories under the webgpu-kernels organization on the Hugging Face Hub, licensed under Apache-2.0.
- @huggingface/kernels, a minimal JavaScript loader that downloads, prepares, and runs kernels directly from the Hub.
- Fleet, an in-browser benchmarking and testing suite that runs kernels on local hardware and, with user consent, contributes private correctness and performance evidence to the community.
Why separate kernels?
A model executed in the browser ultimately becomes a sequence of GPU operations—matrix multiplies, normalizations, convolutions, attention primitives, quantization, data-layout transforms, and more. WebGPU and WGSL provide a cross-browser API and shader language for these operations, but portability does not guarantee performance: two shaders that compute the same result can behave very differently across accelerators. Workgroup sizes, memory accesses, vectorization, data types, and fusion strategies all affect performance, and the best choice can vary with input shapes, device, browser, and available WebGPU features.
Kernels form a foundational layer: higher-level runtimes can only be as efficient as the operations they dispatch. Publishing operations as discoverable, testable, benchmarkable, and versioned artifacts lets the low-level implementations improve independently while preserving a stable contract for higher layers.
What each kernel repository contains
Every kernel has its own repository and kernel card on the Hub documenting semantics, inputs, outputs, attributes, supported data types, source files, and a ready-to-run @huggingface/kernels example. The repository bundles the artifacts needed to evaluate and reproduce the implementation:
- manifest.json: the operation contract (inputs, outputs, attributes, type constraints, shape derivation rules).
- metadata.json: kernel identifier, digests, and provenance.
- test.json: correctness cases to validate expected behavior.
- bench.json: benchmark and tuning cases representing workloads used to evaluate the kernel.
- *.wgsl.jinja: parameterized WGSL templates used to generate shaders for a particular request and device.
This structure turns a shader into a reusable software artifact: the interface is inspectable without reading WGSL, correctness and performance cases travel with the implementation, and published versions can be loaded explicitly rather than depending on unversioned file URLs.
Loading a kernel from the Hub
Install the package:
npm install @huggingface/kernels@preview
Running kernels requires a browser with WebGPU support (availability depends on browser, OS, GPU, and driver). The loader bridges a kernel repository and the application: call getKernel with a Hub repository ID and a contract version, then invoke the returned function with typed inputs and tensor shapes. The loader derives output shapes and logical data types from the manifest and allocates outputs automatically.
A small ai.onnx.Add bias-add example is provided to demonstrate the call pattern; the same pattern applies to heavier kernels such as ai.onnx.MatMul. Kernels include variants for different cases (equal-shape vectorized paths, broadcasted inputs, scalar paths, general broadcasting) so that a runtime can select the best implementation for the current call and device without changing the application-facing API.
The version option (for example, version: 1) selects a version of the kernel contract; this is separate from ONNX opsets or an operator's since_version, allowing a stable JavaScript-facing contract while kernel implementations evolve.
Performance: how much faster are the kernels?
Hugging Face compared their collection against ONNX Runtime Web (ORT WebGPU) on an Apple M4 GPU using ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a. From 1,756 initial test cases across all 207 operations they kept the 809 cases where both sides produced matching outputs and reliable timings.
Across those 809 cases, the Hugging Face WebGPU kernels were 2.57x faster by geometric mean and 1.90x faster at the median, with 629 wins, 176 losses, and 4 ties. Selected operation results:
- Add (5 compared cases): 0.064 ms vs 0.227 ms, 3.52x speedup.
- MatMul (29 cases): 0.115 ms vs 0.131 ms, 1.14x speedup.
- Softmax (12 cases): 0.114 ms vs 0.240 ms, 2.11x speedup.
- LayerNormalization (6 cases): 0.061 ms vs 0.135 ms, 2.22x speedup.
Some individual wins were dramatic: a bilinear Einsum case (i,ij,j with size 4096) ran in 0.136 ms with the Hugging Face kernel versus 1,396 ms with ORT WebGPU—over 10,000x faster—and a row-wise CumSum over [256, 4096] ran 301x faster (0.016 ms vs 4.784 ms). These outliers illustrate how specialized kernels can avoid slow paths that affect general implementations.
Timings measured only the GPU work itself and excluded setup overheads such as loading kernels, creating sessions, uploading inputs, compiling shaders, and reading outputs. Very short workloads are harder to measure and small cases can benefit from GPU caches, so these numbers are best treated as comparative evidence rather than guarantees for every application. They are per-operation results, not full-model benchmarks. Hugging Face is also working with the ONNX Runtime team to upstream these improvements for the broader ONNX Runtime Web ecosystem.
Fleet: crowdsourced measurements from real devices
Because WebGPU performance varies across GPUs, browsers, and drivers, Fleet lets anyone run correctness and performance checks in the browser and see how kernels behave on their hardware. With consent, each run contributes private evidence that helps detect device-specific failures, compare kernel variants, and improve selection rules. The aim is to use broad, real-world coverage to make kernels faster and more reliable for everyone.
Building a shared foundation for WebAI
The initial set of 207 kernels is a starting point. Publishing kernels independently on the Hub provides a shared place to inspect contracts, compare implementations, reproduce correctness checks, and improve performance without embedding every shader in every runtime. On the Hub’s Kernels page the WebGPU kernels appear alongside kernels for CUDA, ROCm, Metal, and other platforms, and can be filtered and explored like other artifacts.
The pieces reinforce one another: repository-defined contracts, @huggingface/kernels for loading and running from JavaScript, and Fleet for crowdsourcing real-world evidence across many devices. Each contributed run can reveal failures, guide tuning, improve variant selection, and validate future kernel versions.
This low-level foundation supports the next steps in browser inference: connecting kernels to higher-level model tooling, expanding operation coverage, and making fast local inference easier to use across the WebAI ecosystem.
Explore the WebGPU kernel collection, try @huggingface/kernels, and join Fleet to contribute evidence from your device and help improve the kernels for everyone.
