Writing / Machine learning

Machine learning,
from the foundations.

Understand the idea, follow the mathematics, and work through an example. Notes for learning deeply and returning when the details get blurry.

My aim with these notes is to make the reasoning visible: what problem a method solves, what its assumptions mean, where its equations come from, and how to use them. The starting point is my machine learning lectures, expanded into explanations I can return to long after a particular exam.

I am beginning with convolutional networks and the statistical-learning topics that follow them. Prerequisites are explained where they are needed. Probability, statistics, linear algebra, and calculus will grow into linked notes of their own.

The learning sequence

Each available chapter includes intuition, derivations, worked examples, diagrams, practice with hidden solutions, and a compact revision section. The seven chapters below form the current learning sequence; further foundation notes will be linked as they are developed.

1. 01 · Available

   Understanding convolutional networks

   Images as arrays; convolution and cross-correlation; channels, sharing, stride, padding, pooling, activation functions, receptive fields, shared gradients, and classical filters.

   Explore a sliding kernel · Twelve practice problems · Revision checklist

2. 02 · Available

   Training convolutional networks

   Complete convolutional backpropagation, interpret AlexNet, derive dropout scaling, and understand batch normalization in training and inference.

   Inspect dropout masks · Explore batch normalization · Fourteen practice problems

3. 03 · Available

   Probability and multivariate Gaussians

   Build probability from outcomes and events, derive distribution moments, then understand covariance, Gaussian geometry, conditioning, and transformations.

   Explore conditional probability · Explore Gaussian geometry · Eighteen practice problems

4. 04 · Available

   Statistical regression and Gaussian classification

   Derive Gaussian estimates, connect regression to least squares, and build a Gaussian classifier from class priors, means, and pooled covariance.

   Fit a regression example · Explore class posteriors · Twenty practice problems

5. 05 · Available

   Naive Bayes, from words to evidence

   Derive conditional independence, constrained word estimates, and smoothing. Fit count and presence models to the same corpus, then explain their predictions and limits.

   Explore count estimation · Compare document models · Eighteen practice problems

6. 06 · Available

   Factor models and PCA, finding the smaller space

   Derive latent covariance, orthogonal projections, the eigenvector solution, reconstruction error, and SVD. Connect eigenfaces, image patches, cosine patterns, and document factors.

   Explore projections · Explore patch compression · Twenty-four practice problems

7. 07 · Available

   Decision trees, from splits to pruning

   Derive Gini, entropy and weighted gain; grow classification and regression trees; compare validation pruning and complexity costs; understand the forest introduction.

   Explore numerical splits · Explore pruning choices · Twenty-six practice problems

Foundations, exactly where they are needed

A prerequisite should help explain an idea rather than become a long detour before the idea. The available chapters contain these self-contained bridges:

• Vectors, arrays, dot products, and affine neurons.
• Counting discrete positions and interpreting a floor.
• Derivatives, the chain rule, and why gradients can shrink.
• Partial derivatives and accumulation through shared parameters.
• Expectation and probability-weighted averages through dropout.
• Mean, variance, standard deviation, and normalization.
• Chain-rule paths through shared batch statistics.
• Outcomes, events, and random variables.
• Expectations, variances, and covariance identities.
• Covariance matrices, positive definiteness, and dimensions.
• Likelihoods and the variable being optimized.
• Trace identities and Gaussian covariance estimation.
• Log-density ratios, decision boundaries, and posterior probabilities.
• Conditional independence and the probability chain rule.
• Constrained probability estimation and count units.
• Pseudocounts, support, and normalization.
• Orthogonal coordinates, projections, and squared distances.
• Eigenvectors, covariance geometry, and a constrained maximum.
• Matrix rank, singular values, and low-rank approximation.
• Logarithmic surprise, entropy and concavity.
• Additive counts and bottom-up optimization.

The probability chapter now develops those foundations directly; later notes will deepen the algebra, calculus, and statistical modeling connections.

Three ways to use a chapter

Learn it

Read in order. Pause at each checkpoint, draw the diagram yourself, and reproduce the small calculation. If an equation feels mysterious, go back to the meaning and dimensions of each symbol.

Revise it

Attempt the practice problems with solutions closed. Record the reason for a mistake: a missing assumption, a shape error, a forgotten rule, or a concept that is not yet clear. Retry the problem later and use the revision checklist to track what you can explain unaided.

Explain it

Use the interview questions to practice short explanations backed by an example or derivation. Learn the conditions under which a statement is true. “Convolution shares weights” is useful; showing where that sharing appears in both the parameter count and the gradient is stronger.

How these notes are grounded

The lecture slides guide the learning goals and coverage. Explanations add missing steps, prerequisite context, original diagrams, and original practice questions. Supplementary connections are identified where they extend beyond the immediate lecture. Each chapter acknowledges its sources and states where the next topic begins.

The notes are developed one chapter at a time, with time for questions and corrections. If a statement is wrong or a derivation skips the step that matters, that is something to fix in the explanation.