ML from Scratch · 5 lectures

ML from Scratch

Machine learning built from first principles, with pen-and-paper maths, small from-scratch code and live figures: decision trees and random forests, bagging, boosting and XGBoost, then neural networks from the perceptron through backpropagation to modern training.

1. Decision Trees (CART) and Random ForestsRecursive yes/no questions, the impurity measures that choose them, why a single deep tree overfits, and how averaging many randomised trees fixes it.
2. Bagging vs Boosting, and XGBoost from the Objective UpTwo opposite ways to combine weak learners: average independent ones, or build them in sequence to fix each other's mistakes. Gradient boosting as gradient descent, and the second-order maths inside XGBoost.
3. Neural Networks I: From the Perceptron to the Multilayer NetworkA neuron is a weighted sum and a squashing function. One of them can only draw a straight line; stack them with a non-linearity in between and they can draw anything.
4. Neural Networks II: Backpropagation from ScratchThe chain rule, organised so that one backward sweep gives every gradient. Worked by hand on a tiny network, then vectorised.
5. Neural Networks III: Training That Actually WorksLosses, mini-batches, momentum and Adam, sensible initialisation, normalisation and regularisation. The practical toolkit that turns backprop into a model that generalises.