DESIGN & BUILD
The bandwidth requirements are massive. 400G networking, which seemed cutting-edge just a few years ago, is rapidly giving way to 800G and 1.6T connections. The Ultra Ethernet Consortium, formed by Arista, Broadcom, Intel, Meta and Microsoft, is developing standards specifically for AI workloads that could challenge InfiniBand’ s dominance in highperformance computing.
Software-defined networking( SDN) has become essential for managing these complex topologies. Hyperscale operators use SDN controllers to automatically configure network paths, balance traffic loads, and respond to failures without human intervention. This automation is critical given the scale of these networks: a single hyperscale facility might contain hundreds of thousands of network ports.
The power constraint crisis The industry’ s growth trajectory has created an unprecedented challenge that threatens to constrain AI development: power availability. US data centre power demand hit 46,000 MW in Q3 2024, and projections show another 35 GW will be needed by 2030: the equivalent to adding 35 nuclear power plants.
Gartner expects that 40 % of existing AI data centres will hit power constraints by 2027 with Northern Virginia, historically the dominant hyperscale market, hitting power grid limits that are forcing operators to look elsewhere.
The power requirements for AI workloads are fundamentally different from traditional computing. Training large language models requires sustained high power consumption across thousands of GPUs running in parallel for weeks or months. Unlike web servers that can scale up and down based on demand, AI training workloads run at full capacity continuously.
This has led to innovative approaches to power procurement. Hyperscale operators are increasingly partnering directly with utilities to develop new generation capacity, often focusing on renewable sources to meet sustainability commitments. Amazon has signed agreements for over 15 GW of renewable
“ Trainium2 chips offer 30-40 % better price-performance than current GPU-powered compute instances generally available today”
Andy Jassy,
CEO, Amazon datacentremagazine. com 153