NVIDIA continues to turn out more powerful graphics processing units (GPUs). The company’s Blackwell chips feature more than 200 billion transistors and the high-speed interconnect bandwidth that AI workloads and hyperscale data centers require. Now, the NVIDIA Vera Rubin platform is taking things to another level to facilitate more advanced reasoning by AI applications while lowering the cost of AI inference. NVIDIA says the platform can reduce inference cost per token by up to 10 times compared with Blackwell.

On the networking side, NVIDIA Spectrum-6 ASIC Ethernet switches are designed to remove networking bottlenecks from AI and cloud computing workloads. The goal is to transform Ethernet into a high-performance scale-out fabric that can keep GPUs fed with data. Broadcom, too, has released the Tomahawk 6 to improve network performance for AI training and inference. It supports up to 102.4 Tb/s of switching capacity on a single chip and 1.6TbE connectivity.

Similarly, Cisco has released new chips for high-end switches and routers to support data centers filled with GPUs. The Cisco Silicon One G300 chip supports backend networking between racks of GPUs. Its packet processing architecture incorporates buffering and queuing capabilities that help manage traffic and reduce congestion. The G300 provides 102.4 Tbps of bandwidth with 200 Gbps SerDes connectivity.

Ultra-High Processing Means Ultra-High Heat

These are just a few examples of developments on the chip innovation front. There are many more. Chips like these are designed to increase the amount of processing that can be accomplished within individual GPUs and to network together multiple GPUs and server racks to accomplish more work in less time. It is all about pushing the boundaries of processing to accommodate next-generation AI capabilities.

Processing at the level necessary for the applications being run in AI data centers generates immense amounts of heat. In some cases, it goes well beyond what can be dealt with by air cooling alone. As a result, some of these chips and switches are incorporating cold plates. What we are seeing in the market now is that cold plate designs are no longer focused solely on keeping GPUs cool. The networking systems that work in tandem with GPUs are so sophisticated and conduct such intense levels of processing that some are starting to require liquid cooling. The highest-end systems might be fully liquid cooled, while others use partial liquid cooling on specific chips and components, and others remain air cooled.

That is likely to be the pattern going forward as data centers and chip designs continue to advance. The most intense areas of processing will increasingly require liquid cooling to remove heat. In parallel, air-cooling systems will remain a vital element in maintaining the broader data center environment. It is about matching the cooling approach to processing density, thermal requirements, and cost. Liquid cooling becomes increasingly important as rack and processing densities rise, but it will continue to be supplemented by air cooling, even in many high-density environments.

Further, many data centers will continue to support traditional workloads rather than shifting entirely to AI. They will not necessarily be deploying ultra-high-density racks, and these facilities will continue to rely heavily on air cooling.