Build & train a neural net

On top of the engine, build Neuron / Layer / MLP and a gradient-descent training loop. Train a tiny network until the loss drops — you just trained a neural net from scratch.

Python~60 min

Use this templateFree · your own copy on GitHub

What you’ll build

A working automatic-differentiation engine — the same idea behind PyTorch — in about 100 lines. By the end, a real trainable neuron whose gradients you can prove are correct.

engine.pyPython

a = Value(2.0)
b = Value(-3.0)
L = (a * b + Value(10.0)).tanh()

L.backward()      # fills in every gradient automatically
a.grad            # how much L moves when you nudge a

The tests decide when you’re done — not the tutor. When they pass, you’ve built it.

What you’ll learn

Build Neuron, Layer, and MLP classes on top of your autograd engine
Write a gradient-descent training loop from scratch
Compute a loss, backpropagate it, and nudge every weight toward a better answer
Train the network until the loss visibly drops
See exactly what a forward pass, backward pass, and weight update really are

Neural networksMulti-layer perceptronGradient descentLoss functionsForward and backward passTraining loop

Before you start: Do the autograd engine first — this builds directly on the Value class you wrote there.

Meet your tutor

It won’t hand you the answer. It asks the one question that makes the next line obvious — and checks your work by running the tests.

you

i don’t get why we even need gradients

tutor

before I answer — picture nudging a up by a hair. what happens to the loss?

you

it goes down a little?

tutor

exactly — that “how much” is the gradient. let’s make the code measure it. open engine.py and find where * is defined…

A taste of the tutoring style

You're at the end

See the full Neural Networks: Zero to Hero roadmap