X @Avi Chawla

Training Methodology - Large Language Models initiate from randomly initialized weights where token selection remains near uniform, resulting in the generation of gibberish text [2] - The pre-training stage consumes nearly all training compute by processing a large text corpus to repeatedly predict the next word, imparting grammar, world facts, and raw reasoning abilities [2] - The model acts as a text continuation engine during pre-training rather than directly answering questions [1][3] Fine-Tuning and Alignment - Instruction fine-tuning utilizes pairs of instructions and good responses to teach the model formatting conventions, such as answering questions or summarizing requests [3] - OpenAI leveraged roughly 13,000 human-written examples for GPT-3 during instruction fine-tuning, compared against a pre-training corpus comprising hundreds of billions of words [3] - Preference fine-tuning employs human evaluators to select the superior output between two responses, training a separate reward model to update the main language model [4][5] Advanced Reasoning and Limitations - Reasoning fine-tuning applies programmed correctness checks for domains like mathematics and code without requiring human preference signals [6] - DeepSeek trained the R1 model using GRPO, allowing emergent behaviors such as intermediate step processing and self-correction to manifest spontaneously during training [8] - Stage 4 reasoning fine-tuning remains restricted to domains where programs can automatically verify answers, contrasting with support replies or summaries that lack a deterministic correct output [8][9]