MPS in PyTorch 2.15

The PyTorch team has released version 2.15, bringing native Metal support to Apple Silicon Macs. This update replaces the older MPSGraph abstraction with dedicated kernels, fixing memory issues and speeding up inference for local AI models.

What changed

Running small models on Mac hardware previously required workarounds for unsupported operations, performance drops, and unexpected memory growth. The new release addresses these by fixing upstream issues in PyTorch and swapping MPSGraph for custom Metal kernels.

FP8 support

Version 2.15 introduces torch.float8_e4m3fn tensors. These store data in half the space of BF16. You can perform scaled matrix multiplication and other supported CUDA and CPU operations directly on the GPU using these tensors.

Language model decoding

Decoding tokens for large language models is now significantly faster. Mixture-of-experts models see a 5 to 6.5 times speed increase compared to version 2.14. On an M3 Ultra running torch.compile, the Qwen3.6-35B-A3B model jumps from 9.4 to 61.3 tokens per second. The LFM2.5-8B-A1B model improves from 23.6 to 123.6 tokens per second.

Dense models also benefit from faster linear layers and an eightfold speedup in softmax sampling operations.

Linear layers

While GEMV kernels existed for torch.mm since 2.14, the F.linear function previously relied on MPSGraph. The new version links them directly. This allows common single-row decoding inputs to use specialized kernels with fused bias additions. Across 96 BF16 layer shapes, the geometric mean speedup is 1.23 times. Small projection layers, such as gates in MoE models, see the largest gains because the old path wasted memory bandwidth. Large layers and language model heads remain near 1x speedup as they were already bandwidth-bound.

Mixture-of-experts layers

Mixture-of-experts layers distribute tokens unevenly across experts. The new implementation handles these uneven groups in a single operation using native Metal kernels. Previously, the system copied group offsets to the CPU and looped over them. Now, the Metal kernels read offsets directly on the GPU, using Metal Performance Primitives where available. Small groups, which are common during decoding, have their own dedicated kernel.

Linear algebra improvements

Complex-valued computations are now easier. complex64 LU factorization, LU solve, and triangular solve run on MPS. This unlocks operations and gradients built on them, including complex determinants. Users can solve small systems directly on the GPU.

Small matrices receive fast kernels too. Batches of FP32 matrices up to 4×4 are inverted in a single Metal kernel that combines factorization and solving. Larger problems also see attention through blocked updates in QR decomposition and efficient matrix multiplication in triangular solves. Gated DeltaNet workloads get a dedicated fast path for batches of 16×16, 32×32, and 64×64 FP32 triangular systems with at least eight right-hand sides.

Singular Value Decomposition is more robust at numerical extremes. It now correctly handles very small singular values and very large or small input magnitudes. Basis completion with full_matrices=True makes pseudoinverse and least-squares results more reliable.

Convolutions and training

Audio models, video decoders, and training loops have new paths through Metal:

• conv1d forward: Specialized kernels replace MPSGraph, plus an im2col-plus-matmul path for short outputs. Useful for audio and streaming-model shapes.
• Constant padding: Moves to Metal kernels, fixing runaway driver-memory growth in tiled video VAE decoding.
• Reflection and replication padding: New Metal implementations are faster and fix backward correctness.
• Out-of-place dropout: nn.Dropout and F.dropout now use the existing fused kernel.
• Upsampling backward: A Metal gather kernel replaces the MPSGraph resize-gradient path.
• Channels-last conv2d backward: Stays on an efficient channels-last path instead of an NCHW graph with costly strided access.

Softmax and reductions

Token sampling gets faster as well. softmax and log_softmax now run on dedicated Metal kernels. Large vocabularies benefit the most: an FP32 softmax over a [1, 128256] row of logits drops from 57 microseconds to 7 microseconds. This is about eight times faster than the MPSGraph version. The speedup holds across other common vocabulary sizes.

Video decoding memory footprint

One of the most striking results comes from the constant-padding change. The VAE decode in Wan-AI/Wan2.2-TI2V-5B-Diffusers makes 8,764 padding calls over a full 28-tile decode. With padding on Metal, peak driver memory drops from about 270 GiB to 8 GiB.

MPS streams

You can now pick which MPS stream runs your work, directly from Python. torch.mps.Stream and the torch.mps.stream context manager provide the interface. Tensor.record_stream lets the allocator track a tensor’s use on another stream.

Here is a small example that sends work to a separate stream and synchronizes before reading the result:

import torch

x = torch.randn(1024, device="mps")
torch.mps.synchronize()
worker = torch.mps.Stream()

with torch.mps.stream(worker):
    y = x.square()
    x.record_stream(worker)

del x
worker.synchronize()
print(y.cpu().shape)

More operations

Several more operations now run on MPS: torch.frexp, torch.logspace, torch.take, and Tensor.put_, torch.heaviside.

What it means for you

If you run models on a Mac with transformers, diffusers, or tools like ComfyUI, here is the practical impact:

• Local LLMs are faster. Mixture-of-experts models generate 5 to 6.5 times more tokens per second than in 2.14. Dense models benefit from faster linear layers and an eightfold faster softmax for sampling.
• Video generation fits in memory. Decoding a video with Wan2.2’s tiled VAE used to push driver memory to 270 GiB; it now peaks at about 8 GiB.
• Image editing works correctly. On 2.14, the Qwen-Image-2.1 VAE encoder silently corrupted images on MPS, breaking image-edit workflows. The same padding change fixes it: an encode-decode round trip now matches CPU at 29.95 dB PSNR, up from 10.06 dB.
• FP8 weights are available. FP8 tensors take half the memory of BF16, and F.scaled_mm multiplies them directly on the GPU.

Requirements and compatibility

MPS now requires macOS 15 or later. PyTorch 2.15 requires Python 3.11 or later. If your OS is too old or no Metal GPU is available, the MPS availability check tells you why.

Next steps

To try the release candidate in a separate environment:

uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu

A sneak peek at PyTorch 2.16 suggests attention will move fully to Metal, with more MPSGraph-to-Metal migrations and faster LLM decoding.