Summary of Key Points from the Conference Call Industry and Company Overview - The discussion revolves around **Agentic AI** frameworks, which enhance traditional Large Language Models (LLMs) by integrating decision-making orchestrators and external tools, transforming them into autonomous problem solvers [2][4]. Core Insights and Arguments - **Agentic AI Workloads**: The paper profiles five representative agentic AI workloads: **Haystack RAG**, **Toolformer**, **ChemCrow**, **LangChain**, and **SWE-Agent**. These workloads are analyzed for latency, throughput, and energy metrics, highlighting the significant role of CPUs in these metrics compared to GPUs [3][10][20]. - **Latency Contributions**: Tool processing on CPUs can account for up to **90.6%** of total latency in agentic workloads, indicating a need for joint CPU-GPU optimization rather than focusing solely on GPU improvements [10][34]. - **Throughput Bottlenecks**: Throughput is bottlenecked by both CPU factors (coherence, synchronization, core over-subscription) and GPU factors (memory capacity and bandwidth). This dual limitation affects the performance of agentic AI systems [10][45]. - **Energy Consumption**: At large batch sizes, CPU dynamic energy consumption can reach up to **44%** of total dynamic energy, emphasizing the inefficiency of CPU parallelism compared to GPU [10][49]. Important but Overlooked Content - **Optimizations Proposed**: The paper introduces two key optimizations: 1. **CPU and GPU-Aware Micro-batching (CGAM)**: This method aims to improve performance by capping batch sizes and using micro-batching to optimize latency [11][50]. 2. **Mixed Agentic Workload Scheduling (MAWS)**: This approach adapts scheduling strategies for heterogeneous workloads, balancing CPU-heavy and LLM-heavy tasks to enhance overall efficiency [11][58]. - **Profiling Insights**: The profiling of agentic AI workloads reveals that tool processing, rather than LLM inference, is the primary contributor to latency, which is a critical insight for future optimizations [32][34]. - **Diverse Computational Patterns**: The selected workloads represent a variety of applications and computational strategies, showcasing the breadth of agentic AI systems and their real-world relevance [21][22]. Conclusion - The findings underscore the importance of a CPU-centric perspective in optimizing agentic AI frameworks, highlighting the need for comprehensive strategies that address both CPU and GPU limitations to enhance performance, efficiency, and scalability in AI applications [3][10][11].