Experimental foundations

Core concepts

Learn the ideas that determine whether an experiment is trustworthy. Each lesson uses a small notebook to make an abstract failure mode visible.

9 concepts

01
beginner

Train, Validation, and Test Sets

Training data fits the model, validation data chooses the design, and test data estimates the performance of the finished design.

data splittingevaluation
02
beginnerafter train validation test

Generalization

Generalization is the ability to perform well on new examples from the conditions where the model will actually be used.

generalizationevaluationmodel complexity
03
intermediateafter generalization

Bias and Variance

Bias is systematic error from an overly restrictive model; variance is instability caused by sensitivity to the particular training sample.

generalizationmodel complexity
04
beginnerafter train validation test

Metrics

A metric compresses model behavior into a number, so choosing one means choosing which mistakes matter.

classification metricsevaluation
05
beginnerafter train validation test

Data Leakage

Leakage lets training or evaluation use information that will not exist when a real prediction is made.

leakagedata splittingevaluation
06
intermediateafter bias variance

Regularization

Regularization makes some fitted solutions preferable to others so the learned behavior is more likely to survive new data.

regularizationgeneralizationmodel complexity
07
intermediateafter train validation test

Loss and Optimization

The loss defines what counts as a training mistake; the optimizer searches for parameters that reduce that loss.

loss functionsoptimization
08
beginner

The Perceptron

A perceptron adds weighted inputs and turns the result into one of two decisions.

Neural networks / Deep learningperceptron
09
intermediateafter perceptron

Backpropagation

Backpropagation sends the prediction error backward through a network to measure how much each parameter contributed to it.

Neural networks / Deep learningbackpropagationoptimization