NVIDIA Proposes AI Memory Collaborative Design Solution, Achieving Up to 30 Times Improvement in Inference Throughput

NVIDIA announced that as the scale of AI models expands and the application of intelligent agents becomes more widespread, GPU memory pressure is increasing, with HBM supply tightening and prices rising. NVIDIA proposes to improve memory usage efficiency through collaborative design of GPUs, CPUs, networks, storage, and software, rather than simply increasing GPU memory capacity. Specifically, NVIDIA compresses model weights and KV Cache using the NVFP4 low-precision format, transferring temporarily inactive contexts to lower-cost storage layers, and utilizes multi-GPU distributed deployment to free up HBM space. In the SemiAnalysis AgentX test, the GB300 NVL72 achieved a unit cost inference throughput up to 10 times that of the H200 platform under specific workloads; the Vera Rubin NVL72 achieved a maximum improvement of 30 times in unit cost inference throughput compared to the GB300 NVL72, while the single GPU HBM capacity increased by only about 3%.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.