How to learn machine learning

Build the computational vocabulary every model uses. You work with vectors, matrices, and linear systems before touching a dataset, because the normal equation and principal components only make sense when you can see dimensions. This sits first because later stages reference these objects constantly.

• vectors and vector spaces
• matrix multiplication
• dot products and norms
• linear independence and rank
• matrix inverse and pseudo-inverse
• eigendecomposition
• singular value decomposition

You can now: Given a small feature matrix, compute by hand the model weights that minimise squared error using the normal equation.

Estimation language comes before model training because a model's output is a probability claim. You learn maximum likelihood, Bayes, and the Gaussian thoroughly enough to derive simple estimators by hand. This stage makes cross-entropy and evaluation metrics in later stages legible.

• random variables
• expectation and variance
• conditional probability
• Bayes' rule
• maximum likelihood estimation
• Gaussian distribution
• sampling distribution of the mean

You can now: Derive the maximum likelihood estimate for the mean of a Gaussian from five sample values and state its variance.

Optimisation turns a mathematical model into something you can fit. Here you derive gradients for squared error and logistic loss, then implement gradient descent from scratch. Sitting after probability means you understand what is being minimised; sitting before models means every training run is interpretable.

• mean squared error
• cross-entropy loss
• gradient descent
• learning rate and step size
• stochastic gradient descent
• convexity and local minima
• momentum
• mini-batch training

You can now: Compute the gradient of mean squared error with respect to weights by hand for a linear model with three features, then implement one gradient descent step in Python.

Now you apply the previous maths to benchmark models. You learn each model's assumption, loss function, and failure mode, not just its API. This stage sits after optimisation because you can already see how each model is trained; it sits before evaluation because models must be understood separately from metrics.

• linear regression
• logistic regression
• k-nearest neighbours
• decision trees
• naive Bayes
• support vector machines
• random forests
• gradient boosting

You can now: Train and evaluate each of five models on a held-out dataset, then explain which one should generalise best for a given data shape.

Evaluation and regularisation are where most practitioners fail, so this stage isolates them. You learn to split data without leakage, estimate generalisation with cross-validation, and control capacity with L1/L2. Placing this after core models means you already have working models to break and fix.

• train/test split
• k-fold cross-validation
• bias-variance trade-off
• overfitting and underfitting
• L1 and L2 regularisation
• data leakage detection
• precision, recall, ROC AUC
• class imbalance correction

You can now: Find a deliberately introduced data leakage in a preprocessing pipeline and show how the reported accuracy changes after removal.

Unsupervised methods and end-to-end pipelines connect the individual pieces. You reduce dimensions, cluster, and build a full preprocessing-to-prediction pipeline. This stage is last because unsupervised techniques rely on the linear algebra and probability from stages one and two, plus your debugging instincts from stage five.

• k-means clustering
• principal component analysis
• singular value decomposition
• t-SNE for visualisation
• feature scaling and imputation
• end-to-end modelling pipelines
• model interpretation with SHAP
• ablation testing

You can now: Build a pipeline from raw CSV to trained classifier, including scaling, imputation, and a PCA step, and justify each component with a before/after metric.