Rigsy · benchmarks

Attention kernels on a home RTX 5080.

Attention is the most expensive step in video and image models. Here is how fast the kernels people actually install run on one consumer GPU — and what they cost in accuracy.

These are results of official, third-party kernels. PyTorch's built-in SDPA and SageAttention 2 and 3 by their original authors — Rigsy did not write them and ships no kernel of its own yet. A custom attention kernel tuned for this card is on the roadmap; when it exists, it will be measured here the same way.

Test machine

GPU	NVIDIA GeForce RTX 5080, 16 GB (Blackwell, sm_120)
Driver / CUDA	617.14 / CUDA 13.2
PyTorch	2.14.0+cu132
SageAttention	2.2.0 · SageAttention3 1.0.0 (built from source for Windows, sm_120a)
ComfyUI	0.42.0
OS	Windows 11
Measured	29 October 2026

1 · The kernel alone

One attention call the size Wan 2.2 14B uses for an 81-frame 832×480 video: 1 × 40 heads × 32 760 tokens × 128, bf16. One warm-up run, then the mean of five; nothing else on the GPU but an idle ComfyUI, with enough free VRAM (see the note below).

Kernel	Time	Speed-up	Error vs FP32
PyTorch SDPA	511 ms	1.0×	0 %
SageAttention 2 INT8 QK · FP8 PV	76 ms	6.7×	3.9 %
SageAttention 3 FP4 block-scaled	56 ms	9.2×	19 %

The error is the relative difference of the output from an FP32 reference, measured on random inputs — the worst case for low-precision kernels. Real model activations usually do better; the video test below is the honest check.
SageAttention 3's FP4 kernel itself took 42 ms (≈ 520 TOPS); the remaining ~14 ms is its preprocessing.
SDPA here did not take PyTorch's flash path on this build, so it is slower than it could be on other setups.
With ComfyUI holding ~9 GB, a first run that overflowed VRAM made SageAttention 3 look 15× slower — Windows quietly spilled memory to system RAM. Always measure with headroom.

2 · A real video

Wan 2.2 image-to-video 14B (NVFP4, lightx2v 4 steps), 81 frames at 1104×752, RIFE ×2. Same image, prompt and seed; only the attention kernel changes.

Kernel	Whole video	What we saw
SageAttention 2	146 s	an extra object appeared on the subject in the last frames
SageAttention 3	151 s	clean

First, middle and last frame of the same Wan video: top row SageAttention 2, bottom row SageAttention 3 — First, middle and last frame. Top: SageAttention 2. Bottom: SageAttention 3.

With only 4 sampling steps, attention is a small part of the total, so the 27 % faster kernel barely moves the whole video — the difference is within run-to-run noise.
One seed is an anecdote, not a verdict: the artefact says nothing general about SageAttention 2. More seeds and longer step counts are next.

Reproduce it Soon

The benchmark script will be published on GitHub, so anyone can run the same test on their own card and compare.

Roadmap · Rigsy Kernel Planned

A custom attention kernel written for this one card — no general-purpose compromises:

rotary position embedding (RoPE, with NTK / YaRN scaling) fused into the kernel instead of a separate pass;
mixed precision per layer and per denoising step — FP4/FP8 where it doesn't show, FP16 where it does;
tile sizes and launch shape tuned for the RTX 5080's 84 SMs and 64 MB L2 cache;
the same accuracy checks first, speed second — and every result published on this page, wins and losses alike.

Status: research. No dates promised.