The Hugging Face PyTorch team published the main changes in PyTorch 2.15 with a strong focus on the Apple MPS (Metal Performance Shaders) backend and running models on Macs. Key additions include FP8 tensor support on MPS, faster decoding for both dense and Mixture‑of‑Experts (MoE) language models, and many migrations from MPSGraph to dedicated Metal kernels that speed up linear algebra, convolution/padding paths and other costly audio, video and training operations.
Who is behind this and why it matters
The post is authored by the Hugging Face team that contributes to PyTorch, with a particular emphasis on improving the MPS backend so Macs become first-class citizens for running and training PyTorch models. In early 2026 they observed unsupported ops, performance cliffs and memory growth when running even small models. They therefore tested real models (language, image, video) from the Hugging Face Hub, found MPS shortcomings and fixed many of them upstream in PyTorch—often by replacing MPSGraph with dedicated Metal kernels.
TL;DR
- FP8 arrives on MPS: torch.float8_e4m3fn tensors use one byte per value (half the storage of BF16/FP16) and F.scaled_mm supports scaled matrix multiplication on the GPU.
- MoE models decode 5–6.5× faster than in 2.14: for example, Qwen3.6-35B-A3B on an M3 Ultra with torch.compile goes from 9.4 to 61.3 tokens/s.
- Less time per token for every LLM: single-row linear layers use specialized matrix–vector kernels and softmax over the vocabulary is about 8× faster for large rows.
- More linear algebra on GPU: complex LU, LU solve, triangular solves, faster small-matrix inverses, faster QR.
- Audio, video and training get faster Metal paths for convs, padding, dropout, interpolation backward and reductions; Wan2.2 video VAE decode driver memory peak reduced from about 270 GiB to ~8 GiB.
- More Python control: public MPS streams and more reliable event timing.
- Requirements updated: MPS requires macOS 15 or later; PyTorch 2.15 requires Python 3.11 or later.
What this means in practice
If you run transformers, diffusers, or interfaces like ComfyUI on a Mac:
- Much faster local LLMs. MoE models decode 5–6.5× faster than in 2.14. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s. Dense models gain from faster linear layers and an ~8× faster softmax for sampling.
- Video generation that fits in memory. Decoding Wan2.2 tiled VAE previously pushed driver memory to 270 GiB; it now peaks at about 8 GiB.
- Correct image editing with Qwen-Image-2.1. A padding fix brings encode–decode PSNR on MPS up to 29.95 dB (previously 10.06 dB), matching CPU behavior.
- FP8 weights on your Mac. torch.float8_e4m3fn takes half the memory of BF16 and F.scaled_mm multiplies them directly on the GPU.
Getting started
Requirements: Apple silicon Mac, macOS 15 or later, Python 3.11 or later (see 2.15 compatibility matrix). If the OS is too old or no Metal GPU is available, the MPS availability check now reports why. To try the release candidate in a separate environment: pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu
FP8 on MPS
PyTorch 2.15 introduces torch.float8_e4m3fn tensors on MPS, occupying one byte per value. These FP8 tensors can be multiplied with F.scaled_mm supporting tensorwise and rowwise scaling. The announcement included a short example that shows quantization to FP8 and a scaled matrix multiply with bfloat16 output.
Faster language-model decoding
Decoding in LLMs takes single-row inputs per new token, which turns large matrix multiplies into matrix–vector products. PyTorch 2.15 speeds both dense linear layers and MoE layers that dominate this stage.
Linear layers
MPS had GEMV support for torch.mm since 2.14, but F.linear still went through MPSGraph. In 2.15 F.linear is connected to the GEMV kernels so the common [1,1,hidden_size] decoding input uses them directly and eligible bias additions are fused. Across 96 BF16 layer shapes from real models this produced a 1.23× geometric-mean speedup; the largest gains are on small projection layers (e.g. MoE gates).
Mixture-of-Experts layers
A grouped matmul handles uneven groups of tokens in one operation; 2.15 implements a native grouped matmul on MPS. Previously grouped matmul copied group offsets to CPU and looped over groups; the new Metal kernels read offsets on GPU, use Metal Performance Primitives when available and provide separate Metal paths otherwise. Small groups get their own kernel (the common case in decoding). On an M3 Ultra with torch.compile, MoE models generate roughly 1.3× more tokens per second, and the announcement cites concrete gains such as Qwen3.6-35B-A3B moving from 9.4 to 61.3 tokens/s.
Linear algebra improvements
- Complex-valued computations: complex64 LU factorization, LU solve and triangular solve run on MPS, unlocking operations and gradients built on them (e.g., complex determinants).
- Small matrices: FP32 matrices up to 4×4 are inverted in a single Metal kernel that combines factorization and solving (useful for camera geometry and image transforms).
- Larger problems: QR uses blocked updates and triangular solves convert much work into efficient matmuls. Gated DeltaNet gets dedicated fast paths for batches of 16×16, 32×32 and 64×64 FP32 triangular systems with at least eight right-hand sides.
- SVD robustness improved for very small singular values, extreme input magnitudes and full_matrices=True cases, making pseudoinverse and least-squares more reliable.
Convolutions, padding and training
Several expensive audio/video/training operations were given new Metal paths:
- conv1d forward: specialized kernels replace MPSGraph and an im2col+matmul path helps short outputs (important for audio/streaming shapes).
- Constant padding moved to Metal kernels, fixing runaway driver-memory growth in tiled VAE decoding.
- Reflection and replication padding: new Metal implementations that are faster and correct for backward.
- Out-of-place dropout: nn.Dropout and F.dropout use the existing fused kernel.
- Upsampling backward: a Metal gather kernel replaces the MPSGraph resize-gradient path.
- Channels-last conv2d backward remains on an efficient channels-last path rather than falling back to an NCHW graph with costly strided access.
Softmax and reductions
softmax and log_softmax now run on dedicated Metal kernels. A single FP32 softmax over a [1,128256] logits row dropped from 57 µs to 7 µs (≈8× faster) compared to the MPSGraph version; similar speedups apply for other common vocabulary sizes, improving token sampling performance.
Much smaller memory footprint for video decoding
The constant-padding change produced one of the most striking results: the Wan-AI/Wan2.2-TI2V-5B-Diffusers tiled VAE decode makes 8,764 padding calls over a full 28-tile decode. With padding on Metal, total driver-memory peak fell from ~270.135 GiB to ~8.108 GiB. Current tensor-allocation and allocator-reserved peaks remained modest (~4.4 GiB and ~8.07 GiB respectively).
More control with MPS streams
You can now choose which MPS stream runs work from Python: torch.mps.Stream and the torch.mps.stream context manager provide the interface, and Tensor.record_stream lets the allocator track a tensor's use on another stream. The announcement included a short example that runs work on a separate stream and synchronizes before reading results.
More operations run on MPS
Additional ops now supported on MPS include torch.frexp, torch.logspace, torch.take and Tensor.put_, torch.heaviside, torch.special.laguerre_polynomial_l and unsigned-integer comparisons. Note: logspace does not yet support integer or boolean outputs with more than one element. The MPS math implementation of scaled dot-product attention also received a correctness fix to handle batch and head broadcasting properly.
A look ahead to PyTorch 2.16
Work for 2.16 is underway (some changes already landed on main/nightlies, others still in review). Highlights:
- Faster LLM decoding by committing the command buffer every 16 kernels so the GPU can start work earlier while the CPU continues encoding. Early BF16 compiled decoding numbers showed 4–19% improvements for some models.
- Attention completely on Metal: scaled dot-product attention will drop MPSGraph entirely, with dedicated vector kernels for decoding and MPP for half-precision prefill on newer macOS versions. Extremely large prefill and grouped-query attention shapes see major latency reductions in early tests.
- Continued MPSGraph→Metal migrations: batch norm forward/backward ongoing; MSE loss, std_mean/var_mean, and bincount already moved with dramatic speedups in some cases.
- Smaller install: bundled shaders will build for Metal 3.1 by default, shrinking libtorch_cpu.dylib by about 25% (384 MiB → 287 MiB) and enabling future Metal 4.1 shaders with faster matmuls and native FP8 matrix multiplication.
Wrap-up
PyTorch 2.15 brings FP8 to Macs, accelerates LLM decoding for dense and MoE models, moves much linear algebra onto the GPU and migrates many operations from MPSGraph to dedicated Metal kernels. The Hugging Face PyTorch team encourages trying the release candidate and filing issues on GitHub for slow, missing or broken functionality. They thank contributors including Nikita Shulga, Kurt Mohler and Aaron Gokaslan for reviews and the wider contributor community for MPS work in this release.
