Module 33

The finer points of training

Advanced module (after graduation): overfitting, regularization, early stopping, learning rate, dropout

When you finish this module you can say:

I can see that a model is memorizing from its training and validation error, reduce it with regularization, early stopping, a learning-rate schedule and dropout, and measure honestly which setting helped.

By the end of this module you will be able to diagnose an overfitting model by looking at a validation set, fix it with regularization, early stopping and learning-rate settings, and report in a table what worked.

Lessons

6 lessons · ≈ 60 min

1. Seeing memorization10 min
2. Regularization: a penalty on the weights10 min
3. Early stopping10 min
4. Learning rate: the length of a step10 min
5. Dropout and normalization10 min
6. Project: rescue a network10 min · project

Practice exercises

Short writing exercises, independent of the lessons: Go to the exercises

1. The best penalty size
2. Where to stop, where the best is
3. Normalize the columns