L13 | English F3 Student Learning Log

Persona: I'm a Form 3 (Grade 9) student and I only read English. I know basic Python (variables, for loops, lists) and school algebra up to quadratics. I learn from the official pages, the videos with captions, and the book. Kaggle notebook steps are written as a practice log that follows the official notebooks.

What I understood (in my own words)

A neural net is many layers of functions stacked together.
Chain rule: the outer rate of change times the inner rate of change.
Backprop starts from the final loss and works back layer by layer to find how each parameter should change.
Computers overflow on huge numbers, so we need tricks.

Where I got stuck, and how I got unstuck

Where I got stuck	How I solved it
Chain rule is senior-school maths	Gears helped: if gear A turns B three times and B turns C twice, A turns C six times. Rates multiply.
The log-sum-exp formula	I remember the purpose: subtract the max first so numbers never get huge, and the answer is unchanged.

My answers to the exercises

Q1. y = (3x + 1)². Use the chain rule to find dy/dx at x = 1.

Outer u² has derivative 2u; inner 3x + 1 has derivative 3. dy/dx = 2(3x + 1) × 3. At x = 1: 2 × 4 × 3 = 24.

Q2. In your own words, why does backpropagation work backwards from the loss?

The loss is at the end, and each parameter's effect passes through every later layer to reach it. Working backwards reuses the parts already computed, so all gradients come out in one pass instead of recomputing for each parameter.

Q3. What happens with math.exp(1000) in Python? How does the log-sum-exp trick avoid it?

You get an OverflowError because the number is too big for a float. Log-sum-exp subtracts the largest value first (so the biggest becomes 0), takes exponents safely, then adds the subtracted amount back. The result is the same.

My Zettelkasten card

Card｜The chain rule works like gears, each layer driving the next

Backpropagation sounds scary, but it's the chain rule. The final loss depends on every layer, and each layer's effect is 'this layer's rate of change' times 'the rates of all later layers'. Multiplying backwards tells you which way to nudge every parameter.

We did a gear-train experiment in science. Turn the first gear once: how many turns does the last one make? Multiply all the gear ratios. Back then it was just a calculation; now I see gradients flowing through a network the same way. The code in this lesson is long, though. I understood the backward pass for one layer, but I need more time for the multi-layer version.

My next step: compute the total ratio for gears of 2, 3 and 4, then write out the chain rule for y = (3x + 1)² alongside it.

🔗 Related notes to grow next: → Physics: 'Gear ratios are the chain rule in metal' → Debugging: 'If you're lost, print every intermediate value' → Maths: 'Computer numbers have limits'

Back to lesson note：L13_反向傳播與MLP_202611041526