As large foundation models scale to trillions of parameters, the general-purpose graphics processing unit (GPU) has evolved into a hyper-specialized tensor compute engine. Concurrently, cloud hyperscalers have invested billions designing custom in-house application-specific integrated circuits (ASICs) to break their dependence on commercial merchant silicon.
In this deep architectural teardown, we benchmark NVIDIA’s Blackwell B200, Google’s TPU v6 (Trillium), and Amazon’s Trainium2.
Architecture Specs & FLOPS Comparison
| Specification | NVIDIA B200 (Blackwell) | Google TPU v6 (Trillium) | AWS Trainium2 |
|---|---|---|---|
| Process Node | Custom TSMC 4NP (Dual-Die) | TSMC 3nm | TSMC 3nm |
| FP8 / FP4 Compute | 20 PFLOPS (FP4) / 9 PFLOPS (FP8) | 4.7 PFLOPS (BF16/FP8) | 6.2 PFLOPS (FP8) |
| Memory Bandwidth | 8.0 TB/s (192GB HBM3e) | 4.8 TB/s (HBM3) | 4.0 TB/s (96GB HBM3) |
| Interconnect Bandwidth | 1.8 TB/s bi-directional NVLink 5 | Optical Circuit Switch (OCS) | 3.2 Tbps NeuronLink-v2 |
