NVIDIA has recently provided an extensive overview of its latest innovation in artificial intelligence technology—the Blackwell Ultra GB300 chip. This groundbreaking chip boasts a remarkable performance increase of 50% over the previous GB200 model and features a staggering 288 GB of memory, positioning it as a true leader in the AI chip market.
As of now, the Blackwell Ultra GB300 is in full production and has already been distributed to key customers. This chip is an evolution of NVIDIA's Blackwell solution, presenting a significant upgrade in both performance and features. Similar to how the NVIDIA Super series improved upon the original RTX gaming cards, the Ultra series enhances the AI chips previously introduced.
While NVIDIA did not offer Ultra versions in earlier lineups such as Hopper and Volta, those generations did include enhanced configurations. The Ultra chips not only excel in hardware capabilities but are also complemented by software updates and optimizations that bring substantial improvements to non-Ultra or standard chips.
The Blackwell Ultra GB300 is engineered as an enhanced version of its predecessors, utilizing two reticle-sized dies connected via NVIDIA's NV-HBI high-bandwidth interface. This setup presents the two dies as a single GPU, resulting in a highly efficient design. Built on the TSMC 4NP process node, which is an optimized 5nm technology for NVIDIA, the chip houses an impressive 208 billion transistors.
The NV-HBI interface offers a bandwidth of 10 TB/s for the two GPU dies, operating seamlessly as a single chip. The Blackwell Ultra GB300 features a total of 160 Streaming Multiprocessors (SMs), each with 128 CUDA cores, four 5th Gen Tensor Cores supporting FP8, FP6, and NVFP4 precision compute, and 256 KB of Tensor memory (TMEM). This configuration results in a total of 20,480 CUDA cores and 640 Tensor cores, along with 40 MB of TMEM.
The advancements made in the 5th Gen Tensor Cores are pivotal to the GPU's capabilities, as they handle all AI compute operations. NVIDIA has consistently introduced groundbreaking innovations with each generation of Tensor Cores:
NVIDIA Volta: Introduced 8-thread MMA units, FP16 with FP32 accumulation for training.NVIDIA Ampere: Featured full warp-wide MMA, BF16, and TensorFloat-32 formats.NVIDIA Hopper: Implemented warp-group MMA across 128 threads and a Transformer Engine with FP8 support.NVIDIA Blackwell: Brought forth a 2nd Gen Transformer Engine with FP8, FP6, NVFP4 compute, and TMEM Memory.The Blackwell Ultra GB300 also significantly enhances memory capabilities, offering 288 GB of HBM3e compared to the maximum 192 GB found in the previous Blackwell GB200 models. This substantial memory upgrade is crucial for supporting multi-trillion-parameter AI models. The memory is organized into 8 stacks with a 16 512-bit controller, providing an 8192-bit wide interface and operating at 8 TB/s per GPU.
This memory architecture enables several enhancements, including:
Complete model residence for 300B+ parameter models without memory offloading.Extended context lengths, allowing for larger key-value (KV) cache capacities in transformer models.Improved compute efficiency, resulting in higher compute-to-memory ratios for diverse workloads.The interconnect on the Blackwell Ultra GB300 utilizes NVIDIA's NVLINK technology, which includes NVLINK Switch, NVLINK-C2C, and offers PCIe Gen6 x16 interface for seamless connection to host GPUs. Key specifications include:
Per-GPU Bandwidth: 1.8 TB/s bidirectional (18 links x 100 GB/s)Performance Scaling: 2x improvement over NVLink 4 (Hopper GPU)Maximum Topology: Supports up to 576 GPUs in a non-blocking compute fabricRack-Scale Integration: Includes 72-GPU NVL72 configurations with 130 TB/s aggregate bandwidthPCIe Interface: Gen6 × 16 lanes (256 GB/s bidirectional)NVLink-C2C: Facilitates Grace CPU-GPU communication with memory coherency (900 GB/s)The NVIDIA Blackwell Ultra GB300 platform achieves a 50% increase in Dense Low Precision Compute output using the new NVFP4 standard. The new model provides near FP8 accuracy, with differences often less than 1%, while also reducing the memory footprint by 1.8x compared to FP8 and 3.5x against FP16.
Furthermore, Blackwell Ultra introduces advanced scheduling management and new enterprise-grade security features, including:
Enhanced GigaThread Engine: A next-generation work scheduler that optimizes workload distribution across all 160 SMs.Multi-Instance GPU (MIG): Allows partitioning of GPUs into various-sized instances, enabling secure multi-tenancy with predictable performance isolation.Confidential Computing and Secure AI: Provides robust protection for sensitive AI models and data, incorporating industry-first TEE-I/O capabilities.Advanced NVIDIA Remote Attestation Service (RAS) engine: An AI-powered reliability system that monitors thousands of parameters to predict failures and optimize system uptime.With innovations like the Blackwell Ultra GB300, NVIDIA solidifies its position at the forefront of the AI industry. The combination of advanced hardware, extensive software support, and ongoing research and development ensures that NVIDIA will continue to lead the way in AI technology for years to come.
