The big picture: The AI industry is entering uncharted territory, even as questions remain about the practical limits of scalability and the return on these massive investments. Yet companies like Meta, OpenAI, Microsoft, xAI, and Google continue to push the boundaries of what's possible in AI computing.
A new benchmark for AI prowess has emerged: the ability to amass the most Nvidia chips in a single location. This competition among tech giants is reshaping the AI industry, driving unprecedented investments in computing infrastructure and pushing the boundaries of machine learning.
At the forefront of this technological arms race are companies like Elon Musk's xAI and Mark Zuckerberg's Meta. These firms are building massive super clusters of computer servers, each housing an astounding number of Nvidia's specialized AI processors. The scale of these projects is staggering, with costs running into billions of dollars and chip counts reaching into the hundreds of thousands.
xAI's entry into this high-stakes game is particularly noteworthy. In a remarkably short span of time, the company has built a supercomputer dubbed "Colossus" in Memphis. It boasts 100,000 Nvidia Hopper AI chips, a number that was considered extraordinary just a year ago when clusters of tens of thousands of chips were seen as very large.
Meanwhile, Zuckerberg recently announced that Meta is already training its most advanced AI models on a chip conglomeration that he claims surpasses anything reported by competitors.
The motivation behind these massive investments is clear: larger clusters of interconnected chips have thus far translated into more capable AI models developed at faster rates, with some industry leaders already envisioning clusters containing millions of GPUs.
Nvidia, the company at the center of this technological race, stands to benefit enormously from this trend, and CEO Jensen Huang sees no end in sight for this growth trajectory. He envisions future clusters starting at around 100,000 Blackwell chips.
However, this race towards ever-larger chip clusters is not without its challenges and uncertainties. As the size of these super clusters grows, so do the engineering hurdles. Keeping tens of thousands of power-hungry chips cool is a major concern, leading to innovations in cooling technology. Liquid cooling, where refrigerant is piped directly to the chips, is becoming increasingly common in these massive setups.
Reliability is another significant challenge. Meta researchers have found that a cluster of more than 16,000 Nvidia GPUs experienced routine failures of chips and other components during a 54-day training period for an advanced version of their Llama model.
Despite these challenges, the push towards larger and more powerful AI clusters shows no signs of slowing. Elon Musk has already announced plans to expand xAI's Colossus from 100,000 chips to 200,000 in a single building, with ambitions to reach 300,000 of Nvidia's newest chips by next summer.
The race for AI supremacy is also driving demand for Nvidia's networking equipment, which is rapidly becoming a significant business in its own right. The company's networking revenue reached $3.13 billion in 2024, a 51.8 percent increase from the previous year. Nvidia's networking offerings, including Accelerated Ethernet Switching for AI and the Cloud, Quantum InfiniBand for AI and Scientific Computing, and Bluefield Network Accelerators, are crucial in connecting and managing these massive chip clusters.
Despite these huge expenditures, the question of scalability remains unresolved. Dylan Patel, chief analyst at SemiAnalysis, told the Wall Street Journal that while there's no evidence that these systems will scale effectively to a million chips or a $100 billion system, they have demonstrated impressive scalability from dozens of chips to 100,000.
Tech companies race to build AI superclusters with 100,000+ GPUs in high-stakes competition
