Application Trends and Challenges - AI training models have evolved from simple single-GPU setups to complex architectures requiring tensor parallelism, pipeline parallelism, and mixture of experts (MoE) [1][3][4] - Inference applications demand low latency where every microsecond impacts user experience and costs, requiring the elimination of staging buffers and direct GPU-to-GPU High Bandwidth Memory (HBM) writes [5][6][16] - Large-scale training running across tens of thousands of GPUs faces critical challenges including network fabric congestion, job bootstrap time increasing to tens of seconds or minutes, and memory resource consumption on High Bandwidth Memory (HBM) and qubits [7][8][9] Rekall Library Innovations - AMD collective communication library Rekall utilizes system Direct Memory Access (SDMA) copy engines to move data across GPUs, freeing up compute units for matrix-matrix multiplication and achieving significant speedups [10][11][15] - Single-node optimizations for small messages implement one-shot and two-shot algorithms, achieving significant speedups across different collectives with a maximum speedup of 3.7% on an 8-GPU MI350 node [17][19][20] - Multi-node communication implements a topology-aware hierarchical algorithm that achieves a speedup of up to 3.6% (or 3.6 GB) across 16 nodes with 8 GPUs (MI350) per node [21][23][24] - GPU-initiated communication enables compute and communication fusion, using a single Compute Unit (CU) via SDMA to achieve higher or similar performance compared to default algorithms using up to 64 CUs [25][26][28][29] - Congestion-aware spray traffic (CAS) technology measures round-trip times across queue pairs to dynamically distribute load, achieving bandwidth close to optimal and a speedup of up to 1.7% in multi-node all-to-all operations [35][36][37][39] Roxamen Runtime Performance - Roxamen acts as a GPU-initiated runtime implementing OpenCMN APIs, utilizing SDMA with a single Compute Unit (CU) to deliver up to 35 times higher bandwidth for larger messages on 8 MI355 GPUs compared to default algorithms using 64 CUs [41][42][43] - Supports multiple Network Interface Cards (NICs)—including AMD Pensando Polara, Thor, and Connect7—achieving high bus bandwidth close to 50 gigabytes per second and highly competitive latencies for 8-byte messages [44][45][46]