# Nvidia H800 A version of Nvidia's H100 (Hopper architecture) made for the Chinese market to comply with the October 2022 [[AI Chip Export Controls]]. ## What was "crippled" The H800 kept the H100's compute but had its chip-to-chip link cut. [[NVLink]] bandwidth dropped from roughly 900 GB/s to about 400 GB/s. For a single GPU this barely matters. For training a large model across thousands of GPUs it matters a lot, because the GPUs spend a large share of their time sending gradients and activations to each other. A slower interconnect means GPUs sit idle waiting for data. Mixture-of-experts models ([[Mixture of Experts (MoE)]]) are especially sensitive, because each token is routed to experts that may live on different GPUs, which creates heavy all-to-all traffic. The October 2023 update to the controls closed the loophole and the H800 could no longer be exported, but labs that had already bought them kept using them. ## Why DeepSeek V3 is the reference case DeepSeek's V3 technical report says the model was trained on a cluster of 2,048 H800s. To make an MoE model train efficiently over a nerfed interconnect, the team wrote custom communication kernels, overlapped computation with communication so GPUs were not left waiting, and trained in 8-bit (FP8) precision to cut memory and bandwidth. See [[Below CUDA - GPU Kernels, PTX and Streaming Multiprocessors]] and [[DeepSeek Technical Differentiators]]. Related: [[InfiniBand]], [[Systems and Compiler Depth]]