Core Insights - The article discusses the emergence of various large language models (LLMs) such as Gemini, GPT, Qwen, Llama, and DeepSeek, highlighting the availability of over 182,000 models on Hugging Face. It identifies two main concerns: persistent performance issues and the distinct advantages and disadvantages of different LLMs [2][3][4][5][6]. LLM Ensemble Concept - The concept of "LLM Ensemble" is introduced, suggesting that instead of relying on a single LLM based on performance rankings, it is more beneficial to consider multiple LLMs simultaneously to leverage their diverse strengths [1]. Post-hoc Ensemble Methods - The article categorizes post-hoc ensemble methods into two types: 1. Selection-then-regeneration methods, which depend heavily on task-specific training data and require fine-tuning a large model, limiting their flexibility [8][9]. 2. Similarity-based selection methods, which are mostly unsupervised and select responses based on similarity metrics, though they are criticized for their simplistic design [2][3]. LLM-PeerReview Framework - The LLM-PeerReview framework is proposed as a simple, unsupervised LLM ensemble method inspired by academic peer review processes. It consists of three sequential modules: Scoring, Reasoning, and Selection [7][12]. Scoring Process - The scoring process utilizes multiple LLMs as judges to evaluate responses to the same prompt, employing a novel "Flipped-triple scoring trick" to mitigate biases inherent in traditional scoring methods [12][13][14]. Reasoning and Selection - Reasoning involves aggregating scores from multiple judges, with two versions: a simple average and a weighted version that considers the review quality of different LLMs. Selection focuses on identifying the highest-scoring response from a pool of candidates [12][15]. Experimental Results - The LLM-PeerReview and its weighted variant LLM-PeerReview-W significantly outperform individual LLMs and existing ensemble baselines, achieving average performance improvements of 6.9% and 7.3% over advanced methods like Smoothie-Global [24]. Method Advantages - The LLM-PeerReview framework is characterized by its unsupervised nature, interpretability, and applicability across various tasks, including both Exact-Match Generation and Open-Ended Generation tasks [17]. Efficiency Analysis - The framework allows for a reduction in the number of evaluators to improve efficiency while maintaining performance quality, contrasting with traditional debate-based methods that require multiple rounds of evaluation [21]. Conclusion - LLM-PeerReview is presented as a transparent and effective ensemble method that mimics the peer review process, demonstrating significant advantages over existing models and methods in terms of performance and flexibility [26].