Scale-Up Ethernet Networking: What You Need To Know For 2025

In the era of explosive AI growth, networking technology stands at the forefront of innovation. Traditional data center fabrics are giving way to specialized architectures designed to handle the immense computational demands of modern machine learning models. Among these, scale-up Ethernet networking emerges as a game-changer, promising to unify vast arrays of accelerators into cohesive supercomputing units. This shift addresses the limitations of proprietary interconnects, fostering an open, scalable ecosystem that major tech giants are rallying behind.

Understanding Scale-Up vs. Scale-Out in AI Contexts

AI workloads, particularly training large language models, demand unprecedented inter-processor communication. Scale-out architectures connect resources across multiple racks or buildings, ideal for distributed tasks but plagued by higher latency due to multi-hop paths. In contrast, scale-up designs integrate dozens to thousands of XPUs—graphics processing units, tensor processing units, or similar accelerators—within a single pod or rack. This setup allows direct memory access between devices, mimicking a single massive processor.

Key distinctions include:

• Latency Profile: Scale-up offers one-hop access, slashing delays to microseconds.
• Bandwidth Density: Intra-rack links push terabits per second per device.
• Workload Suitability: Perfect for collective operations like all-reduce in model training.

With AI clusters projected to encompass millions of XPUs, scale-up forms the foundational building block, aggregating into larger scale-across fabrics for exascale computing.

Why Ethernet is Poised to Dominate AI Scale-Up

Historically, proprietary solutions like NVLink or InfiniBand held sway in high-performance computing due to their tailored low-latency features. However, Ethernet’s universality, cost advantages, and evolving standards are flipping the script. Industry leaders including Broadcom, Cisco, Nvidia, Meta, and Arista are investing heavily in Ethernet adaptations for AI.

Ethernet’s strengths for scale-up include:

• Open Standards: Governed by IEEE, ensuring multi-vendor interoperability and avoiding lock-in.
• Maturity: Decades of refinement yield robust tools for congestion control and fault tolerance.
• Scalability: Speeds from 400Gbps to 1.6Tbps support exploding data volumes.
• Efficiency: Reduced transceiver counts and power draw compared to alternatives.

Projections indicate Ethernet powering the majority of the world’s largest AI superclusters by 2028, driven by collaborative efforts to retrofit it for tight-knit accelerator fabrics.

Core Technologies Driving Ethernet Scale-Up

Several frameworks are adapting Ethernet’s layers for AI demands. At the physical layer, high-speed PHYs enable dense connectivity. Transport enhancements introduce lossless protocols, while endpoint designs optimize XPU interactions.

ESUN (Ethernet for Scale-Up Networking) orchestrates these, defining network behaviors and XPU interfaces. SUE-Transport refines headers like AI Forwarding Headers (AFH) for minimal overhead, paired with Credit-Based Flow Control (CBFC) and Link Layer Retry (LLR) for reliability.

Technical Deep Dive: Lossless Ethernet for AI

AI scale-up mandates zero packet loss, as even minor drops cascade into training failures. Enhanced Ethernet employs Explicit Congestion Notification (ECN), Priority Flow Control (PFC), and advanced buffering to achieve near-lossless operation. Cut-through switching minimizes latency by forwarding frames before full receipt.

For a typical 10MW data center hosting 6,000 XPUs, scale-up pods of 100-512 devices per rack reduce optical links dramatically. Aggregate bandwidth matches high-bandwidth memory (HBM) evolution, projected to exceed 10TB/s per XPU by 2027.

Scale-across extends this across buildings via de-buffering switches with line-rate encryption, preserving performance over longer distances.

Industry Momentum and Collaborative Efforts

The push for open Ethernet gained traction at events like ONUG’s Fall 2025 AI Networking Summit. Broadcom forecasted Ethernet’s dominance in hyperscale AI three years out. HPE, Marvell, and Synopsys contribute silicon and IP, while OCP incubates SUE as an open blueprint compatible with SONiC OS.

This coalition contrasts with closed ecosystems, promoting innovation through layered modularity: PHY for speed, L2 for forwarding, and application-specific transports.

Performance Metrics and Real-World Gains

Benchmarks highlight Ethernet scale-up’s edge:

• Latency: Sub-100ns tail latency vs. 500ns+ in scale-out.
• Throughput: 800Gbps ports scale to 51.2Tbps switches.
• Power: 30-50% lower than proprietary optics.
• Cost: Leverages commoditized supply chains.

In shared-memory paradigms, UALink over Ethernet enables atomic operations across pods, slashing software overhead for memory-intensive models.

Challenges and Future Roadmap

Despite progress, hurdles remain: standardizing endpoint memory models, optimizing for 1.6Tbps+, and integrating with existing scale-out spines. ESUN’s iterative approach targets network functionality (traffic handling) and XPU endpoints (scheduling, partitioning).

Upcoming milestones include 224G PHY deployments and CPO (co-packaged optics) for denser fabrics. By 2028, Ethernet could underpin 70% of AI backend networks, blending scale-up pods into trillion-parameter training runs.

Implications for Data Center Operators

For enterprises building AI infra, Ethernet scale-up offers flexibility: mix copper/fiber media, backward-compatible upgrades, and vendor-agnostic deployments. It future-proofs against XPU diversity from AMD, Intel, Nvidia, and beyond.

Frequently Asked Questions

What differentiates scale-up from scale-out networking?

Scale-up tightly couples XPUs in one rack for ultra-low latency; scale-out links distant racks for broader distribution.

Is Ethernet reliable enough for AI training?

Yes, with ESUN/SUE enhancements providing lossless, in-order delivery rivaling InfiniBand.

Who supports ESUN?

Major players like Broadcom, Cisco, Meta, Nvidia, Arista, HPE, and Marvell.

When will Ethernet scale-up be mainstream?

Projections point to 2028 for hyperscale adoption.

Does it support shared memory?

UALink enables it over Ethernet, unifying pod memory domains.