Which AMD GPUs actually run local AI well?
Hi, and thanks for signing up for FitGPU!
The question we hear most is simple: "I don't want to pay NVIDIA prices. Will an AMD card actually run local AI well?"
Short answer: yes. Here's what the numbers say.
Our own test: RX 6950 XT vs. a mini PC
We ran the same models through Ollama on two AMD machines we own:
- Desktop: Ryzen 9 7900X + Radeon RX 6950 XT (16 GB)
- Mini PC: MinisForum AI X1 Pro-470 (Ryzen AI 9 HX 470, Radeon 890M integrated graphics, 64 GB RAM)
Generation speed in tokens per second (higher is better; anything above about 10–15 feels comfortable to read along):
| Model | RX 6950 XT | Radeon 890M mini PC |
|---|---|---|
| gpt-oss 20B | 115.83 | 18.05 |
| Qwen3.5 9B | – | 13.93 |
| Qwen3 30B-A3B | – | 29.7 |
| Llama 3.1 8B | 99.36 | – |
| Qwen2.5-Coder 7B | 106.02 | – |
| Phi-4 14B | 55.61 | – |
What this tells us:
- A 16 GB AMD card is fast. On the 6950 XT, gpt-oss 20B ran at about 116 tokens/s, more than 6x the mini PC on the same model.
- The mini PC wins on memory, not speed. With 64 GB of shared RAM it can load models that don't fit in 16 GB of VRAM. "Mixture-of-experts" models, like Qwen3 30B-A3B, still run at a usable 29.7 tokens/s on it.
- Running out of VRAM hurts. Qwen3.5 27B didn't fully fit on the 6950 XT (only 81% ran on the GPU) and dropped to 9.9 tokens/s.
What the community is seeing
We also collect published results from the community. These are community-reported and unverified (we haven't re-run them), but they're a useful guide. All of these use the same small test model (Llama 2 7B, Q4_0) in llama.cpp, so the cards compare fairly:
- RX 7900 XTX: 182.63 tok/s on Vulkan and 167.11 on ROCm. Source: Vulkan scoreboard and ROCm scoreboard (ROCm result by @Diablo-D3)
- RX 7900 XT: 123.18 tok/s (Vulkan). Source
- RX 7800 XT: 118.27 tok/s (Vulkan). Source
- RX 6800: 101.47 tok/s (Vulkan). Source
- RX 6700 XT (12 GB): 83.88 tok/s (Vulkan). Source
- Ryzen AI Max+ 395 ("Strix Halo"): 50.01 tok/s (ROCm), by @firefox42. Source
A tip that surprised us: on many AMD cards, Vulkan is as fast or faster than ROCm for generating text. @alainnothere measured an RX 7900 XT at 83.82 tok/s on Vulkan vs. 68.53 on ROCm with Qwen3.5 9B (source). If you have an AMD card, try both.
So what should you buy?
- For speed on models up to ~14B: a 16 GB card like the RX 6800 / 6900 XT / 6950 XT or the 7800 XT is a great value spot.
- For the biggest models: VRAM is king (7900 XTX at 24 GB), or go big on shared memory with an APU mini PC and accept lower speed.
All results, with sources and filters, are here: fitgpu.com/benchmarks
Your turn
What AMD hardware are you running, and what would you like us to test next? Just hit reply. I read every email, and good suggestions become next week's tests.
Thanks for reading, Lorne at FitGPU