s1: Simple test-time scaling

Closed reasoning models made a loud claim: spend more tokens at test time and accuracy rises. The public recipe was fuzzy. Do you need millions of reasoning traces? A custom search loop? A secret verifier?

Muennighoff et al. chase the opposite extreme. They build s1K — only 1,000 questions with reasoning traces, filtered for difficulty, diversity, and quality — then supervised-finetune Qwen2.5-32B-Instruct into s1-32B. At inference they add one controller: budget forcing. Cap thinking by inserting an end-of-thinking delimiter early, or lengthen thinking by suppressing that delimiter and appending the word Wait when the model tries to stop. Extra waits often trigger a self-check that repairs a wrong path.

What this paper explains

Language models can improve by training bigger, or by giving a fixed model more inference compute. The second path — test-time scaling — showed up dramatically in systems like OpenAI’s o1, but those methods were not fully public.

This paper asks for the simplest open recipe that still works:

1. How small can a high-signal reasoning SFT set be if you curate ruthlessly?
2. How do you control thinking length at decode time without a search tree or external process reward model?

Their answers are s1K (data) and budget forcing (controller). Together they produce s1-32B, which the abstract reports as exceeding o1-preview by up to 27% on competition math (MATH and AIME24), with AIME24 moving from 50% to 57% when budget forcing is used to extrapolate beyond the no-intervention baseline.

Prior limits

• Train-time only scaling. Bigger models and more pretraining tokens help, but they do not give you a dial at serving time for “think longer on this hard problem.”
• Opaque closed recipes. Demonstrating that test-time compute helps is not the same as publishing a minimal method others can copy.
• Huge reasoning corpora. Many replication efforts lean on large distilled trace sets. The authors instead stress sample efficiency: three curation axes (difficulty, diversity, quality) validated by ablations.
• Uncontrollable generation length. Naively sampling until the model stops gives uneven thinking budgets. You cannot cleanly ask for “half as much” or “2× more” think time without a controller.

The mechanism

s1K curation. Start from a large pool of candidate questions and traces, then keep 1,000 that jointly maximize:

• Difficulty — problems the base model does not already nail.
• Diversity — coverage across domains / problem types.
• Quality — clean reasoning traces and answers (their distillation / filtering pipeline).

Ablations in the paper argue that dropping any axis hurts more than shrinking the set randomly.

Budget forcing (the Daily’s core visual). After SFT, s1-32B emits a thinking segment, then an answer segment, separated by an end-of-thinking delimiter.

1. Cap (less compute). When a token budget is hit, forcibly insert the end-of-thinking delimiter so the model must transition to the answer.
2. Stretch (more compute). When the model tries to emit the end-of-thinking delimiter too early, suppress it and append Wait (possibly multiple times). Generation continues; the model often revisits earlier steps and corrects mistakes.

That is the whole controller. No tree search. No learned process reward model in the loop — just delimiter surgery plus a literal wait token.

Algorithm / figure walkthrough

1. Supervised-finetune an instruct model on s1K so it reliably produces think → delimiter → answer traces.
2. At inference, choose a thinking budget $B$ (tokens or a Wait count).
3. Stream tokens in the thinking phase.
4. If length approaches $B$ from below and the model tries to stop: suppress the delimiter, append Wait, continue.
5. If length hits $B$: insert the end-of-thinking delimiter and let the answer phase run.
6. Grade only the final answer span.

The opening animation tells the same story in three beats: a short think that stops early, a forced Wait that reopens the trace, and a longer think that lands a corrected answer.

What to notice when reading

• Two contributions, one Daily focus. Data curation (s1K) matters for sample efficiency; budget forcing is the interactive mechanism to teach.
• Perfect controllability claim. Among scaling methods they compare, budget forcing is highlighted for a clear positive slope versus thinking length — treat that as the authors’ reported comparison, not a universal law.
• Wait is content, not magic. It works because the finetuned model learned to treat continuation after hesitation as a chance to re-check; the string alone on an untrained base model is not the paper’s claim.
• Metrics are competition math. Headline numbers are on MATH / AIME24-style evaluations in their setup — not a claim about every NLP task.

Results and evidence

Numbers below are from the paper’s abstract and reported experiments (arXiv:2501.19393), not independent re-runs.

Ablations (§5) separate data choices (difficulty / diversity / quality) from test-time choices (how you stretch or cut thinking). Budget forcing is reported as the cleanest control knob among methods they plot for scaling.

Limitations

Compressed from scope and what the paper does not claim:

• Competition-math wins are not automatic transfer to every reasoning domain.
• Budget forcing assumes a model that already emits an explicit think/answer structure after SFT.
• Forcing ever-longer traces has latency and cost; the accuracy slope eventually saturates in practice.
• Curation quality for s1K still requires careful filtering — “1,000 random traces” is not the method.
• Comparisons to closed o1-class systems depend on public eval harnesses and snapshots; treat relative percentages as paper-reported, not timeless leaderboard truth.

How to read the paper

1. Abstract — s1K, budget forcing, 27% / 50%→57% headlines.
2. §2 — how s1K is built (difficulty, diversity, quality).
3. §3 — budget forcing mechanics (cap vs stretch with Wait).
4. §4 — s1-32B vs o1-preview and related baselines.
5. §5 — data and test-time ablations; which knobs actually move the curve.
6. §6 — discussion on simple reasoning recipes.

Knowledge check

Keep reading / Sources

• Paper: s1: Simple test-time scaling (arXiv:2501.19393)
• Code / model / data: github.com/simplescaling/s1
• Related Fanout Daily: InstructGPT (SFT as behavior shaping)
• Related Fanout Daily: Knowledge distillation (trace distillation lineage)
• Related Fanout Daily: Speculative decoding (another inference-time compute pattern — different mechanism)