Core Viewpoint - The article discusses the importance of identifying key reasoning steps in large language models (LLMs) to enhance their interpretability, debuggability, and safety [2][6]. Group 1: Research Methods - The authors propose three complementary methods to analyze the reasoning process of LLMs, aiming to identify critical steps known as "thought anchors" that significantly influence subsequent reasoning [6][13]. - The first method is a black-box approach that measures the impact of sentences on final answers through counterfactual analysis, comparing the answer distributions with and without specific sentences [9][18]. - The second method is a white-box approach that identifies key sentences through attention patterns, revealing how these sentences affect the reasoning trajectory [10][24]. - The third method is a causal attribution approach that directly measures causal relationships between sentences by suppressing attention to specific sentences and observing the impact on subsequent logits [11][29]. Group 2: Findings and Implications - Each method provides evidence for the existence of thought anchors, which are crucial reasoning steps that disproportionately affect the reasoning process [13][15]. - The research indicates that planning generation and uncertainty management sentences consistently exhibit higher counterfactual importance compared to other sentence categories, supporting the idea that high-level organizational sentences can anchor and guide reasoning trajectories [23][25]. - The authors provide an open-source tool for visualizing the outputs of these methods, which can aid in debugging reasoning failures and identifying sources of unreliability [14][15]. Group 3: Case Study - The article includes a case study demonstrating the practical application of the proposed methods, using a specific problem involving the conversion of a hexadecimal number to binary [34][36]. - The resampling method reveals the initial incorrect reasoning trajectory and key turning points, highlighting the importance of specific sentences in achieving the correct answer [37][39]. - Attention analysis shows that the model's reasoning process is organized into distinct computational modules, with key sentences driving the flow of information and resolving contradictions [40][42].