What happened

According to a PyTorch blog post, AMD's Primus-Turbo optimization library has been upstreamed into TorchTitan and TorchAO, enabling native FP8 training on AMD Instinct GPUs with competitive performance out of the box. This follows a demonstration at PyTorch Conference 2025 of linear scaling beyond 1,000 GPUs on AMD Instinct clusters.

The blog reports that on dense models like Llama3-8B, FP8 training delivers a 13.4% throughput gain over BF16, while on MoE models like DeepSeek-V3 671B, fused Triton quantization kernels recovered 89% of the FP8 quantization overhead. Additionally, the correct handling of AMD's e4m3fnuz FP8 format is highlighted as a correctness requirement, with automatic hardware detection added to prevent silent gradient corruption.

Evidence

• AMD's Primus-Turbo optimizations have been upstreamed into TorchTitan and TorchAO, enabling native FP8 training on AMD Instinct GPUs.

  PyTorch · attributed

  We have since upstreamed those AMD optimizations so TorchTitan supports AMD Instinct GPUs directly, with competitive FP8 performance out of the box.

• FP8 training delivers a 13.4% throughput gain over BF16 on Llama3-8B dense models.

  PyTorch · attributed

  On dense models, FP8 training delivers a 13.4% throughput gain over BF16 on Llama3-8B.

• Fused Triton quantization kernels recovered 89% of FP8 quantization overhead on DeepSeek-V3 671B MoE shapes.

  PyTorch · attributed

  Through fused Triton quantization kernels, we recovered 89% of the FP8 quantization overhead on DeepSeek-V3 671B MoE shapes.

• AMD's e4m3fnuz FP8 format has a max of 240, and initial mismatches caused silent gradient corruption.

  PyTorch · attributed

  On AMD Instinct GPU, where e4m3fnuz has a max of 240, this produced silently wrong results: tensors were scaled into a range that exceeded the hardware’s representable values, clipping activations and corrupting gradients.

• Linear scaling beyond 1,000 GPUs was demonstrated using Primus-Turbo on AMD Instinct clusters.

  PyTorch · attributed

  At the PyTorch Conference 2025, we demonstrated linear scaling beyond 1,000 GPUs on AMD Instinct clusters using Primus-Turbo, an AMD optimization library for training frameworks such as TorchTitan.

Why it matters

This reduces engineering overhead for teams adopting AMD GPUs, allowing them to leverage proven large-scale scaling behaviors without maintaining proprietary forks, and lowers the barrier to high-performance FP8 training.

Limits and uncertainties

The reported performance numbers are specific to the tested models (Llama3-8B and DeepSeek-V3 671B) and hardware configuration (8xMI300X); results may vary on other workloads.

The blog post does not provide independent benchmarks or comparisons against NVIDIA GPUs, so relative performance on alternative hardware is not established.

Practical implications

Teams using AMD Instinct GPUs should ensure they use the latest upstream TorchTitan and TorchAO to benefit from native FP8 support and avoid custom forks.

When enabling FP8 training on AMD, verify that hardware auto-detection is active to select the correct e4m3fnuz format and prevent silent model corruption.

For dense and MoE model training, FP8 can yield tangible throughput gains, but engineers should benchmark on their specific models and cluster sizes.

What to watch

Future upstream contributions from AMD expanding FP8 support to additional model architectures and larger cluster sizes.

Adoption of the upstreamed TorchTitan FP8 path in production training pipelines and reported real-world scaling results.

Performance benchmarks comparing AMD Instinct FP8 training against NVIDIA H100/H200 to assess competitive positioning.