\n
\n
![\"NVIDIA](\"https:\/\/image.blockchain.news:443\/features\/D8E08E86F8EDBDDCD68414CF49BDD8B1401B11A69515DFF98E6B2B03EE9CF9D7.jpg\")
\n
\n <\/a>
\n <\/figure>\n
NVIDIA has set new performance and scale records in the generative AI domain, according to a recent submission to MLPerf Training v4.0. This achievement underscores the company’s ongoing dominance in AI training benchmarks, particularly in the realm of large language models (LLMs) and generative AI.<\/p>\n
MLPerf Training v4.0 Updates<\/h2>\n
MLPerf Training, developed by the MLCommons consortium, is the industry-standard benchmark for evaluating end-to-end AI training performance. The latest version, v4.0, introduced two new tests to reflect popular industry workloads. The first test measures the fine-tuning speed of Llama 2 70B using the low-rank adaptation (LoRA) technique. The second test focuses on graph neural network (GNN) training, based on an implementation of the relational graph attention network (RGAT).<\/p>\n
The updated test suite includes a variety of workloads such as LLM pre-training (GPT-3 175B), LLM fine-tuning (Llama 2 70B with LoRA), text-to-image (Stable Diffusion v2), and several others, covering a wide range of AI applications.<\/p>\n
NVIDIA’s Record-Breaking Performance<\/h2>\n
In the latest MLPerf Training round, NVIDIA achieved remarkable performance using a full stack of its hardware and software solutions:<\/p>\n
- \n
- NVIDIA Hopper GPUs<\/li>\n
- Fourth-generation NVLink interconnect with third-generation NVSwitch chip<\/li>\n
- NVIDIA Quantum-2 InfiniBand networking<\/li>\n
- An optimized NVIDIA software stack<\/li>\n<\/ul>\n
These components have been further optimized since the last round, enabling NVIDIA to break previous records. For instance, NVIDIA improved its GPT-3 175B training time from 10.9 minutes using 3,584 H100 GPUs to just 3.4 minutes using 11,616 H100 GPUs, demonstrating near-linear performance scaling.<\/p>\n
Generative AI and LLM Fine-Tuning<\/h2>\n
NVIDIA also set new records in LLM fine-tuning, particularly with the Llama 2 70B model developed by Meta. Utilizing the LoRA technique, a single DGX H100 with eight H100 GPUs completed the fine-tuning in just over 28 minutes. The NVIDIA H200 Tensor Core GPU further reduced this time to 24.7 minutes. NVIDIA’s submissions also showcased scalability, achieving a fine-tuning time of just 1.5 minutes using 1,024 H100 GPUs.<\/p>\n
The company leveraged the context parallelism capability available in the NVIDIA NeMo framework to achieve these results. Additionally, the use of FP8 implementation of self-attention in cuDNN improved performance by 15% at the 8-GPU scale.<\/p>\n
Advancements in Visual Generative AI<\/h2>\n
MLPerf Training v4.0 also includes a benchmark for text-to-image generative AI based on Stable Diffusion v2. NVIDIA’s submissions delivered up to 80% more performance at the same scales through extensive software enhancements, such as the use of full-iteration CUDA Graphs and an optimized distributed optimizer for Stable Diffusion.<\/p>\n
Graph Neural Network Training<\/h2>\n
NVIDIA set new records in GNN training as well. Using 8, 64, and 512 H100 GPUs, the company achieved a record time of just 1.1 minutes in the largest-scale configuration. The use of eight H200 Tensor Core GPUs provided a 47% boost compared to the H100 submission at the same scale.<\/p>\n
Key Takeaways<\/h2>\n
NVIDIA continues to lead in AI training performance, showcasing the highest versatility and efficiency across a range of AI workloads. The company’s ongoing optimization of its software stack ensures more performance per GPU, reducing training costs and enabling the training of more demanding models.<\/p>\n
Looking ahead, the NVIDIA Blackwell platform, announced at GTC 2024, promises to democratize trillion-parameter AI, delivering up to 30x faster real-time trillion-parameter inference and up to 4x faster trillion-parameter training compared to NVIDIA Hopper GPUs.<\/p>\n
For more detailed information, visit the NVIDIA Technical Blog<\/a>.<\/p>\n