Transforming Ethernet from Underdog to Champion for AI Inference

Modern AI inference is increasingly constrained by KV cache movement across prefill, decode, and storage tiers rather than GPU compute. This session demonstrates how a lightweight software layer enables RDMA-class performance on standard Ethernet networks without application changes, supporting high-performance disaggregated vLLM inference. Live benchmarks highlight improvements in time-to-first-token (TTFT) and inter-token latency (ITL). Discover more: https://www.amd.com/en/corporate/events/advancing-ai/s ...