WHY THIS MATTERS IN BRIEF
Data centers are getting bigger, AI needs more data faster, and now some of the hardware to support these needs is becoming incredibly high performant.
Love the Exponential Future? Join our XPotential Community, future proof yourself with courses from XPotential University, read about exponential tech and trends, connect, watch a keynote, or browse my blog.
Tech CEOs proclaiming unfeasibly large numbers at the launch of a new product is par for the course in the industry, and its common practice to take them with a pinch of salt until someone’s tested them. Even so, Nvidia’s Jen-Hsun Huang was perhaps hoping for a bigger reaction at Computex when he claimed that one spine of its new NVLink Fusion system “moves more traffic than the entire Internet.”
NVLink is Nvidia’s processor interconnect technology which plays an increasingly critical role in making sure that data centers – which are increasingly so massive they need dedicated nuclear power stations to power them – can move all the data they need around as fast as possible. Its latest version, NVLink Fusion, was announced at this year’s Computex event and during the presentation, Huang hauled an NVLink spine onto the table. Think of this as a tower of cables and controllers that allows 72 massive GPUs to communicate and share data.
That single spine apparently can transfer data at a rate of up to 130 TB/s, which, according to Huang, is more data per second than the entire Internet shifts about. That particular figure, he says, is 900 Tb/s (note: small b = bits, large B = bytes), so the spine is transferring about 16% more data.
That does sound like a ridiculously impressive figure, but it’s somewhat meaningless. Not least being that other sources claim the peak Internet data transfer rate is over 1,200 Tb/s.
Then you have things like Nvidia’s RTX 5090, which has a peak L1 cache transfer rate of 50 TB/s – sure, that’s way less than an NVLink spine, but it’s also just one GPU. Multiply it by 72, the number of GPUs the spine serves, and you have an absurd total transfer rate of 3,600 TB/s.
What I’m getting at is that if one searches around enough, it’s easy to find examples of equally bonkers data transfer rates. I’m not dismissing what NVLink can do – it’s clearly impressive stuff – but the muted reception to Huang’s “moves more traffic than the entire Internet” is somewhat indicative of how much people are used to hearing tech CEO’s say such things.
What is impressive, though, is the fact that NVLink Fusion isn’t just for Nvidia CPUs and GPUs. Other ‘semi-custom’ processors from vendors such as Qualcomm, Fujitsu, and perhaps even AMD and Intel, will be able to utilise NVLink to hook up their CPUs to a mountain of Nvidia GPUs.
