Module 15

Neural network from scratch

neuron, gradient, backpropagation, training

When you finish this module you can say:

I can build a neural network with no library and train it with backpropagation, and I can explain what every line of a PyTorch training loop does.

By the end of this module you'll be able to train a neural network that recognizes handwritten digits, entirely from scratch with NumPy, no AI library involved.

Lessons

6 lessons · ≈ 60 min

1. The neuron: weight, bias, activation8 min
2. Loss and slope9 min
3. micrograd: backpropagation11 min
4. A multi-layer network and the training loop10 min
5. A vectorized network with NumPy10 min
6. Project: digit recognizer12 min · project

Practice exercises

Short writing exercises, independent of the lessons: Go to the exercises

1. A layer of three neurons
2. Find the bottom of the valley
3. Update the weights in place
4. Leaky ReLU
5. One-hot encoding