Start here

A reading path through the published terms for a first course in applied machine learning, in the order a course meets them: what the data and the model are, how a model is trained and judged, and the methods built on that. Each term links its entry; py marks a term whose Python demo recomputes what the entry states.

1. Data, model and loss
What a data point, a feature and a label are, and what a model and a loss function do with them.
data py data point py feature py label py tabular data py model py hypothesis py loss py
2. Training and validation
Learning a hypothesis from a training set and judging it on data points that did not enter training.
empirical risk minimization (ERM) py training set py training error py validation set py validation error py test set py generalization py independent and identically distributed assumption (i.i.d. assumption) py
3. Linear models and gradient descent
The first model that is trained by an optimization algorithm one can read step by step.
linear regression py logistic regression py classification py confusion matrix py accuracy py baseline py gradient descent (GD) py stochastic gradient descent (SGD) py learning rate py
4. Overfitting and regularization
Why a small training error can mislead, and the three ways to counter it.
overfitting py regularization py hyperparameter py k-fold cross-validation (k-fold CV) py data leakage py
5. Non-linear models
Deep nets keep gradient descent on a non-convex loss; decision trees drop it for a greedy search.
artificial neural network (ANN) py decision tree py support vector machine (SVM) py
6. Ensembles
Combining many hypotheses, and auditing what the combination relies on.
ensemble py bagging py boosting py random forest py feature importance py
7. Unsupervised and self-supervised learning
Structure without labels: clusters, components, and representations learned from the data alone.
clustering py k-means py expectation–maximization (EM) py Gaussian mixture model (GMM) py principal component analysis (PCA) py dimensionality reduction py autoencoder py feature learning py self-supervised learning py pretraining py fine-tuning py
8. Distribution shift
A hypothesis honest on its own data and wrong about the world.
distribution shift py transfer learning py
9. Reinforcement learning
Learning from rewards instead of labels.
reinforcement learning (RL) py reward py policy (reinforcement learning) py online learning py
10. Putting it together
From raw data to a deployed predictor.
machine learning pipeline (ML pipeline) py large language model (LLM) py

Instructors assembling their own list: the instructors page and the Seminar-Baukasten configurator.