# API Distillation Classic [[Distillation]] trains a small "student" model to imitate a larger "teacher" model, usually with access to the teacher's internal probabilities. API distillation is the black-box version: you only have the teacher's text outputs, bought through its API, and you train your own model on them. ## How it works 1. Send a large, well-chosen set of prompts to a [[Frontier Models|frontier model]] (hard reasoning, coding, tool use). 2. Collect the responses, often including step-by-step reasoning. 3. Filter and score them (this is where [[Evaluation and Verification Infrastructure]] matters). 4. Use the resulting dataset for supervised fine-tuning or as a starting point for reinforcement learning on your own model. The student does not need the teacher's weights or its pretraining budget. It needs good prompts, the output, and good filtering. ## Why it is the first link in the loop - Frontier capability leaves the frontier lab **as output**, which is exactly what an API sells. - It is cheap relative to pretraining, so it compounds for well-optimised labs. - It is hard to police. Terms of service typically forbid training competing models on outputs, but detection relies on usage patterns rather than any technical barrier. > [!note] The lever it gives frontier labs > Closing or restricting API access to the best models is the one move that cuts this link. It does nothing about capability already embedded in released [[Open Weights]]. Related: [[Pretraining, Post-Training and Fine-Tuning]], [[The Cross-Pacific AI Chessboard]]