L24 | English F3 Student Learning Log

Persona: I'm a Form 3 (Grade 9) student and I only read English. I know basic Python (variables, for loops, lists) and school algebra up to quadratics. I learn from the official pages, the videos with captions, and the book. Kaggle notebook steps are written as a practice log that follows the official notebooks.

What I understood (in my own words)

Attention lets each position 'look at' every other position and decide which matter most.
The query is what I'm looking for, the key is each position's label, and the value is the actual content.
Multi-head attention is like several pairs of eyes looking for different relationships at once.
Adding a condition like 'which class to generate' lets us choose a T-shirt or a shoe.

Where I got stuck, and how I got unstuck

Where I got stuck	How I solved it
Q, K and V felt abstract	The library analogy made it click (see exercise 1).
Sin/cos time embeddings	I think of them as many clock hands turning at different speeds that together describe one moment; every moment has a unique combination.

My answers to the exercises

Q1. Use 'finding a book in a library' as an analogy for query, key and value.

The query is the topic I want, e.g. 'volcanoes'. The keys are the labels on each book's spine. I compare my topic with every label and read more pages from the more relevant books. The values are what's actually inside the books.

Q2. Convolution only looks at nearby pixels. How is attention different?

A convolution kernel only covers nearby pixels, so it takes many layers to see far away. Attention compares every position with every other position in one go, so it can link distant related parts directly, like the laces on one side of a shoe and the heel on the other.

Q3. What's the difference between an unconditional and a conditional diffusion model?

An unconditional model generates randomly; you can't choose what appears. A conditional model takes an extra input (e.g. the class 'shoe') so you can say what to generate.

My Zettelkasten card

Card｜Attention is really about knowing where to focus

Attention lets every position in the model look at the whole image, then decide how much to take from each place based on relevance. It doesn't spread focus evenly; it concentrates on what matters most. That's the core of why Transformers handle both language and images.

It reminds me of being a goalkeeper. When I started, I tried to watch everything: the ball, the striker's eyes, the other players. My reactions were slow. My coach told me to watch the striker's hips and planted foot, because they predict the shot direction best. That's like matching the query with keys to focus attention on what's most relevant. People's attention is limited, but a model can run many heads looking at different things at once. That's where models beat us.

Next training session, I'll watch only the striker's hips for five minutes and record how often I guess the shot direction correctly.

🔗 Related notes to grow next: → Sport: 'Where you look sets how fast you react' → AI: 'Similarity decides attention' → Learning: 'Read with your own query in mind'

Back to lesson note：L24_注意力與Transformer_202611041526