Core Viewpoint - The article emphasizes the importance of distinguishing between LLM Inference and LLM Serving, as the rapid development of LLM-related technologies has led to confusion in the industry regarding these concepts [1][3]. Summary by Sections LLM Inference and LLM Serving Concepts - LLM Inference refers to the process of running a trained LLM to generate predictions or outputs based on user inputs, focusing on the execution of the model itself [5]. - LLM Serving is oriented towards user and client needs, addressing the challenges of using large language models through IT engineering practices [7]. Characteristics and Responsibilities - LLM Inference is computation-intensive and typically requires specialized hardware like GPUs or TPUs [4]. - The responsibility of LLM Inference includes managing the model's runtime state and execution, while LLM Serving encompasses end-to-end service processes, including request handling and model management [10]. Technical Frameworks - vLLM is highlighted as a typical implementation framework for LLM Inference, optimizing memory usage during service inference [5][7]. - Kserve is presented as an example of LLM Serving, providing capabilities for model versioning and standardized service experiences across different machine learning frameworks [7][10]. Model Context Protocol (MCP) - MCP is described as a standardized protocol that connects AI models to various data sources and tools, functioning as a bridge between LLM Inference and LLM Serving [11][12]. - The architecture of MCP suggests that it plays a role similar to LLM Serving while also addressing aspects of LLM Inference [12][16]. Future Development of MCP - The article predicts that MCP will evolve to enhance authentication, load balancing, and infrastructure services, while clearly delineating the functions of LLM Inference and LLM Serving [17].