FanoutResearch papers
Foundational AI papers
A curated reading sequence of papers that shaped modern AI.
- 01 · Shannon 1948
A Mathematical Theory of Communication
The founding document of information theory.
- 02 · Rosenblatt 1958
The Perceptron
The first learning machine.
- 03 · Rumelhart, Hinton & Williams 1986
Learning representations by back-propagating errors
Popularized the algorithm deep learning runs on.
- 04 · Krizhevsky et al. 2012
ImageNet Classification with Deep CNNs (AlexNet)
The deep learning big bang.
- 05 · Sutskever et al. 2014
Sequence to Sequence Learning
Encoder–decoder thinking.
- 06 · He et al. 2015
Deep Residual Learning (ResNet)
Skip connections unlock depth.
- 07 · Vaswani et al. 2017
Attention Is All You Need
The Transformer.
- 08 · Devlin et al. 2018
BERT
Bidirectional pretraining.
- 09 · Kaplan et al. 2020
Scaling Laws for Neural Language Models
Power laws for loss.
- 10 · Ouyang et al. 2022
InstructGPT
RLHF becomes a recipe.
- 11 · Hoffmann et al. 2022
Chinchilla
Compute-optimal scaling.