Nvidia Blackwell ruled the MLPERF training benchmark

The latest Mlperf benchmark results will be disappointed for those who enjoy routing for the underdog: NVIDIA’s GPU has dominated the competition As yetagainThis includes chart-topping performance on the latest and most demanded benchmark, with a show of the Lama 3.1 403B large language model. He said, the latest AMD GPU, computers built around MI325X, match the performance of H200 of NVIDIA, Blackwell East, the most popular LLM on Fine-Tuning Benchmark. This shows that AMD is a generation behind Nvidia.

Mlperf The training machine is one of the learning competitions Mlcommons Consortium. “AI performance can sometimes be like a wild waste. Mlperf wants to bring orders for that chaos,” Dave salveterDirector of quick computing products in NVIDIA. “This is not an easy task.”

The competition consists of six benchmarks, each of which checks a separate industry-relevant machine learning task. There are graph node classification for applications such as benchmark material recommendation, large language model pretering, large language model fine-tuning, machine vision applications for machine vision applications, image generations, and fraud detections and applications such as fraud detection and drug discovery.

Large language model pretering task is the most resourceful, and this round was updated to be even more. The word “pretering” is somewhat misleading – it can give an impression that it is after a phase called “training”. it. Pratrening is the place where most number of crunching occurs, and thus usually fine-stagnant, which refines the model for specific tasks.

In the previous recurrence, the GPT3 model was pretted. This recurrence, it was replaced by the Lama 3.1 403B of the meta, which is more than double the size of the GPT3 and uses four times larger reference window. The reference window is how much input text the model can process at once. This larger benchmark once represents industry trend for large models, as well as some architectural updates.

Blackwell tops the chart, AMD on your tail

For all six benchmarks, the fastest training time was on the Blackwell JPU of Nvidia. NVIDIA itself presented each benchmark (other companies also presented using various computers manufactured around NVIDIA GPU). Nvidia’s salveter emphasized that this is the first deployment of the Blackwell GPU on the scale and this performance is only likely to improve. “We are still quite early in Blackwell Development Life Bicycle,” he says.

This is the first time AMD has presented the training benchmark, although in previous years other companies have presented using computers which included AMD GPU. In the most popular benchmark, LLM Fine-Tuning, AMD displayed that its latest instinct Mi325x GPU performed at a parliamentary with NVIDIA’s H200s. Additionally, Instinct Mi325X showed a 30 percent improvement on its predecessor, Instinct Mi300X. (The main difference between the two is that 30 percent more over the MI325x Mi300x comes with high-bandwidth memory.)

For its part, Google presented a single benchmark, image-generation work, with it Trillium tPU,

Importance of networking

Among all submissions of the LLM Fine-Tuning benchmark, the system with the largest number of GPU was introduced by Nvidia, which was a computer connecting the 512B 200. On this scale, networking between GPU begins to play an important role. Ideally, adding more than one GPU will divide the time to be trained by the number of GPU. In fact, it is always less efficient, because some time is lost for communication. Reducing that loss is important to train the largest model efficiently.

It becomes even more important on the preterening benchmark, where 512 GPUs used the smallest submission, and the largest is used 8,192. For this new benchmark, the performance scaling with more GPU was particularly close to linear, obtained 90 percent of the ideal performance.

NVIDIA salveter credited NVL72, a skilled package that connects 36 Grace CPU and 72 Blackwell GPU NvlinkTo create a system that “serves as a single, large -scale GPU,” Data sheet Claims. Many NVL72S were then associated with Infiniband network technology.

In particular, the biggest submission for this era of Mlperf – 8192 in GPUS – despite the increasing demands of the pretering benchmark, is not the largest. In the previous round, submissions were observed with more than 10,000 GPU. Kenneth LeachPrincipal AI and machine learning engineers in Hewlet Pacord Enterprise, along with decrease in improvement in GPU, also do networking among them. “Earlier, we needed 16 server nodes (to show LLM), but today we are able to do it with 4. I think this is one of the reasons why we are not seeing such a huge system, because we are receiving too much skilled scaling.”

One way to avoid damage related to networking is to place many AI actions on the same huge wafer as it has recently claimed by cerebras Beat Nvidia’s Blackwell GPUS exceeds one factor at more than two factor. However, that result was measured Artificial analysisThe charge that is executed, it questions separate providers without controlling. So it does not compare an apple-to-Apple, the way the Mlperf benchmark ensures the benchmark.

A lack of power

The Mlperf benchmark also includes a power test, measuring how much power is consumed to achieve each training work. This round, only one submitter -Lenovo – made it impossible to compare among the artists, including a power measurement in its submission. The energy that was imposed to fix an LLM on two Blackwell GPUs was 6.11 Gigazoule, or 1,698 kW-hour, or broadly would carry the energy to heat a small house for winter. With increasing concerns about the energy use of AI, the power efficiency of training is important, and this authors probably expect more companies expect to present these results in future era.