Module 25

Trees and ensembles

Advanced module (after graduation): Gini, depth, forest, boosting, honest tuning

When you finish this module you can say:

I can work out by hand how a decision tree picks its question, build a forest and boosting, choose settings without looking at the test data, and measure which column matters in two ways.

By the end of this module you will be able to compare tree, forest and boosting models honestly on the same data and choose their settings without looking at the test data.

Lessons

6 lessons · ≈ 67 min

1. How a tree picks a question10 min
2. Depth and memorization10 min
3. Forest: the vote of many trees11 min
4. Boosting: learning by correcting errors12 min
5. Searching for settings honestly12 min
6. Project: which column matters12 min · project

Practice exercises

Short writing exercises, independent of the lessons: Go to the exercises

1. The size of a tree
2. Rows that are never picked
3. The best learning rate