Skip to main content
Fanout
Overview

AI Research archive

A structured path from math fundamentals to building LLMs from scratch. 12 modules, 108 lessons, covering everything from derivatives to transformers to reinforcement learning.

Study progress
Free access

Read all 108 lessons

Every lesson in the path is written and open in this clone. Start at module 01 and work through the whole curriculum.

Start reading
Curriculum structure
01Math Fundamentals15 lessons02Core AI Intuitions4 lessons03PyTorch Fundamentals9 lessons04TensorFlow Fundamentals27 lessons05Neural Network from Scratch7 lessons06Transformers3 lessons07Reinforcement Learning5 lessons08LLM From Scratch4 lessons09Write Research Paper1 lessons10How to Fine-Tune Models5 lessons11Machine Learning Operations (MLOps)25 lessons12Bonus Lessons3 lessons
Continue learning

Functions

Math Fundamentals — 0 of 15 lessons done

0% completeResume

Full curriculum

01Math FundamentalsThe mathematical foundations you need for AI research — from functions and derivatives to information theory and SVD.15 lessons
  1. Functionsread
  2. Derivativesread
  3. Vectorsread
  4. Gradientsread
  5. Matricesread
  6. Derivation Rules & Examplesread
  7. The Chain Ruleread
  8. Backprop in Pythonread
  9. The Jacobian Matrixread
  10. Hadamard Product (Element-wise Op)read
  11. Entropy & Information Theoryread
  12. KL Divergenceread
  13. Singular Value Decomposition (SVD)read
  14. Moving Averages (EMA)read
  15. More Math Lessonsread
02Core AI IntuitionsBuild deep intuition for the core operations that power all of AI — dot products, softmax, broadcasting, and norms.4 lessons
  1. Similarity With Dot Productread
  2. Softmax Probabilitiesread
  3. Tensor Broadcastingread
  4. L1 vs L2 Normsread
03PyTorch FundamentalsMaster the tensor operations that are the building blocks of every neural network implementation.9 lessons
  1. Creating Tensorsread
  2. Matrix Multiplicationread
  3. Transposing Tensorsread
  4. flatten, reshape, view, squeeze, unsqueezeread
  5. Indexing and Slicingread
  6. cat, stackread
  7. Special Tensors (eye, rand, arange, linspace)read
  8. Autograd: Automatic Differentiationread
  9. 7 PyTorch Tasks (Advanced)read
04TensorFlow FundamentalsLearn TensorFlow from the ground up — linear models, CNNs, transfer learning, adversarial examples, NLP, reinforcement learning, and more.27 lessons
  1. Simple Linear Modelread
  2. Convolutional Neural Networkread
  3. Pretty Tensorread
  4. Layers APIread
  5. Keras APIread
  6. Save & Restoreread
  7. Ensemble Learningread
  8. CIFAR-10read
  9. Inception Modelread
  10. Transfer Learningread
  11. Video Dataread
  12. Fine-Tuningread
  13. Adversarial Examplesread
  14. Adversarial Noise for MNISTread
  15. Visual Analysisread
  16. Visual Analysis for MNISTread
  17. Deep Dreamread
  18. Style Transferread
  19. TensorFlow GPU vs CPUread
  20. Reinforcement Learningread
  21. Estimator APIread
  22. TFRecords & Dataset APIread
  23. Hyper-Parameter Optimizationread
  24. Natural Language Processingread
  25. Machine Translationread
  26. Image Captioningread
  27. Time-Series Predictionread
05Neural Network from ScratchBuild neural networks from the ground up — single neurons, layers, training loops, normalization, and optimization.7 lessons
  1. Single Neuron From Scratchread
  2. Building a Layerread
  3. Implementing a Networkread
  4. RMSNormread
  5. Learning Rate, Decayread
  6. Adam Optimizerread
  7. Neural Network From Scratchread
06TransformersThe architecture that changed everything — attention mechanisms, self-attention, and building GPT from scratch.3 lessons
  1. Attention Mechanism Explainedread
  2. Self Attention from Scratchread
  3. GPT From Scratchread
07Reinforcement LearningHow agents learn from interaction — from basic environments to PPO and modern LLM reasoning techniques.5 lessons
  1. Agents & Environmentsread
  2. Policy Gradients (REINFORCE)read
  3. Deep Q-Learning (DQN)read
  4. PPO, LLM Reasoning, Importance Ratio, Advantageread
  5. Qwen 3 GSPO & DeepSeek GRPO — LLM Reasoningread
08LLM From ScratchBuild state-of-the-art large language models from scratch — LLaMA 4, DeepSeek V3, Qwen 3, and more.4 lessons
  1. Llama 4 From Scratchread
  2. DeepSeek V3 From Scratchread
  3. Qwen 3 From Scratchread
  4. Self-Study LLMread
09Write Research PaperThe complete workflow from coding experiments to writing and publishing an AI research paper.1 lessons
  1. Code, Write & Publish AI Research Paperread
10How to Fine-Tune ModelsFrom adapters to full fine-tuning runs — dataset preparation, training configuration, and evaluation of the result.5 lessons
  1. Prompting vs Fine-Tuningread
  2. Adapters and LoRAread
  3. QLoRA and Quantized Trainingread
  4. Building a Fine-Tuning Datasetread
  5. Evaluating a Fine-Tuneread
11Machine Learning Operations (MLOps)Shipping models to production: data and pipeline versioning, deployment, monitoring, and reproducibility.25 lessons
  1. What MLOps Actually Isread
  2. Reproducible Environmentsread
  3. Experiment Trackingread
  4. Data Versioningread
  5. ML Pipeline with DVC & AWS S3read
  6. Training Pipelines and Orchestrationread
  7. Model Registryread
  8. Packaging Models for Servingread
  9. Batch vs Online Inferenceread
  10. Model Serving with FastAPIread
  11. Containerizing a Model Serviceread
  12. CI for Machine Learningread
  13. CD and Deployment Strategiesread
  14. Canary and Shadow Deploymentsread
  15. Monitoring Data Driftread
  16. Monitoring Concept Driftread
  17. Model Performance Monitoringread
  18. Logging and Observabilityread
  19. Alerting and On-Call for MLread
  20. Feature Storesread
  21. Retraining Triggersread
  22. Cost Management for Training and Servingread
  23. Security and Access Controlread
  24. Governance and Model Cardsread
  25. Incident Response for ML Systemsread
12Bonus LessonsExtra lessons added over time.3 lessons
  1. Train LLM — Sequence Length vs Batch Sizeread
  2. SwiGLU — Better Neural Networksread
  3. 100x AI Reasoning — Tiny Recursive Modelread

Content gets updated with time.