L03 | English F3 Student Learning Log

Persona: I'm a Form 3 (Grade 9) student and I only read English. I know basic Python (variables, for loops, lists) and school algebra up to quadratics. I learn from the official pages, the videos with captions, and the book. Kaggle notebook steps are written as a practice log that follows the official notebooks.

What I understood (in my own words)

A model is just a function with some numbers inside (parameters) that we can adjust.
Training means finding the best parameters: guess, measure the loss, check the gradient, take a small step that lowers the loss.
The gradient is the slope. The slope of a tangent from maths class finally has a use!
ReLU is very simple (negatives become 0), but adding lots of them can draw any curve, like making a circle from many short straight lines.
You can even build a neural net in Excel. It's slow, but GPUs are great at matrix multiplication, so they're much faster.

Where I got stuck, and how I got unstuck

Where I got stuck	How I solved it
I haven't learned derivatives yet	Jeremy says for now I only need to know the gradient tells you which way is downhill and how steep it is. I watched the Khan Academy video he mentioned.
Why do we subtract the gradient?	I sketched y = x². At x = 3 the slope is positive, so going right goes uphill. To go downhill I move left, which means subtracting.
MMULT in Excel kept giving errors	The matrix sizes had to match: the columns of the first must equal the rows of the second.

My answers to the exercises

Q1. Loss L(w) = w², current w = 3, learning rate 0.1. Find the gradient, w after one update, and w after a second update.

Gradient = 2 × 3 = 6. First update: w = 3 − 0.1 × 6 = 2.4. Second: gradient = 4.8, so w = 2.4 − 0.48 = 1.92. w keeps moving towards 0, where the loss is lowest.

Q2. ReLU(x) = max(0, x). Compute ReLU(−2), ReLU(0), ReLU(3). Why can adding many ReLUs make curved shapes?

ReLU(−2) = 0, ReLU(0) = 0, ReLU(3) = 3. Each ReLU is a line with one bend. Many of them bending at different points add up to a wiggly curve.

Q3. What happens if the learning rate is too large? Too small?

Too large: each step jumps over the bottom, so the loss bounces around or even grows. Too small: you creep along and training takes forever.

Q4. A model predicts [2, 4]; the true values are [3, 3]. Compute the MSE.

Errors: −1 and 1. Squares: 1 and 1. Mean: (1 + 1) / 2 = 1. MSE = 1.

Q5. In the Titanic data, Fare ranges 0–500 while Age is about 0–80. Why take the log of Fare and normalize?

Without it, Fare's large numbers would dominate the sum and the other columns would barely matter. Log squashes the long tail (log(1+9) ≈ 2.3, log(1+99) ≈ 4.6), and normalizing puts all columns on a similar scale so training is stable.

My Zettelkasten card

Card｜Gradient descent is walking downhill blindfolded, one small step at a time

A neural network learns by repeatedly asking 'which way makes the loss smaller?' and taking a small step that way. The gradient gives the direction and steepness, and the learning rate sets the step size. Behind the 'intelligence' is a very simple repeated move.

It reminded me of a hike when we lost the trail at dusk. We couldn't see far, so we felt which way the ground sloped down and kept walking until we reached the road. Big strides on steep parts made me slip; tiny steps would have taken all night. That's exactly the learning rate. One thing still bugs me: a small dip on the hillside could fool you into thinking you've reached the bottom. The lesson says this is less of a problem in deep learning than you'd expect, and I want to find out why.

My next step: do gradient descent on y = w² in a spreadsheet with three learning rates and chart how w changes.

🔗 Related notes to grow next: → Maths: 'Slope is the language of how fast things change' → Learning: 'Small quick steps beat waiting for a perfect plan' → Life: 'When lost, check direction first, then choose your stride'

Back to lesson note：L03_神經網路基礎_202611041526