NVIDIA H200 Announced at SC23
Narrative
First GPU with HBM3e memory. 141GB capacity at 4.8 TB/s bandwidth. 76% more memory than H100. Nearly doubles Llama 2 70B inference speed.
Reality
Shipped Q2 2024 as promised. Memory capacity gains verified. Inference speed improvements real but workload-dependent. Full H100 compatibility confirmed. Cloud providers deployed rapidly.
Implication
Mid-generation refresh validated memory-first optimization for AI inference. HBM3e became new standard. H200 bridged Hopper to Blackwell transition. Memory bandwidth competition intensified.