Python 必備知識與範例|Python You Need for fast.ai

中文:fast.ai 官方要求「大約一年的寫程式經驗」。這份筆記把課程真正用到的 Python 拆成 16 個主題,依「Part 1 必備 → Part 2 必備」排序,每個主題都附課程出處、範例和小練習。 English: fast.ai expects "about a year of coding experience". This note splits the Python actually used in the course into 16 topics, ordered from Part 1 essentials to Part 2 essentials. Each topic has where it appears in the course, a runnable example and a short exercise.

只識中文的同學

  • 每段的 English 那一行可以跳過,只讀「中文:」那一行。
  • 變數、串列、函式這些詞的生活比喻在 白話小辭典 第 1 部分,先看比喻再看程式會容易很多。
  • 第一週每天讀哪幾個主題,見 新手從這裡開始。

怎麼用這份筆記|How to use this note

  1. 在 Kaggle 開一個新 notebook(Create → New Notebook),每個範例貼進一格,按 Shift + Enter 執行。 Open a new Kaggle notebook, paste each example into a cell, press Shift + Enter.
  2. 先自己改數字、改字,看結果怎麼變,再做小練習。答案在每節最後的摺疊區。 Change the numbers and text first, see what changes, then do the exercise. Answers are folded at the end of each section.
  3. 進度建議:主題 1 到 10 學完再開始 L01;主題 11 到 13 在 L03 到 L05 之間補;主題 14 到 16 是 Part 2(L09 之後)才需要。 Suggested pace: finish topics 1 to 10 before L01; topics 11 to 13 between L03 and L05; topics 14 to 16 only for Part 2 (L09+).
#主題 Topic何時需要 When課程出處 Where in the course
0Jupyter/Kaggle 操作 Notebook basicsL01 前每一課
1變數與資料型別 Variables & typesL01 前每一課
2字串與 f-string StringsL01 前L01 搜尋關鍵字、L04 組合文字
3串列、切片 Lists & slicingL01 前L01 urls[0]、L05 df[:5]
4字典、元組 Dicts & tuplesL01 前L01 predict 回傳三個值、L07
5條件與迴圈 if / forL01 前L01 for o in searches:
6串列生成式 ComprehensionsL01 前L02、L05、L11 到處都是
7函式 FunctionsL01 前L01 get_y=parent_label、L03 quad
8import 與函式庫 ImportsL01 前L01 from fastai.vision.all import *
9檔案路徑 pathlibL01 前L01 path/o
10錯誤訊息怎麼讀 Reading errorsL01 前每一課
11NumPy 與廣播 Arrays & broadcastingL03 到 L05L03、L05、L11
12pandas 表格 DataFramesL04 到 L06L04、L05、L06、L07
13PyTorch 張量與梯度 Tensors & gradientsL03 到 L05L03、L05、L13
14類別與物件 ClassesPart 2(L07 開始用到)L07、L13 到 L16
15迭代器、產生器 Iterators & generatorsPart 2L10、L14
16裝飾器、*args/**kwargs、partialPart 2L14、L15、L16

0|Jupyter/Kaggle 操作|Notebook basics

中文:notebook 由一格一格的「儲存格(cell)」組成。程式格執行後,最後一行的值會自動顯示。開頭加 ! 代表執行終端機指令,例如安裝套件。 English: A notebook is made of cells. When a code cell runs, the value of its last line is displayed automatically. A leading ! runs a shell command, such as installing a package.

# 每一課第一格都會看到這行:安裝/更新 fastai
# You will see this in the first cell of every lesson: install or update fastai
!pip install -Uqq fastai

1 + 2          # 最後一行自動顯示 3 / the last line is shown: 3
快捷鍵 Shortcut作用 Action
Shift + Enter執行並跳到下一格 Run and move to next cell
Esc 再按 A/B在上方/下方新增一格 Add cell above / below
Esc 再按 D D刪除這一格 Delete cell
函式名後加 ?看說明,例如 print? Show help
函式名後加 ??看原始碼,例如 parent_label?? Show source

Kaggle 要先在 Settings → Accelerator 選 GPU,帳號要先做手機驗證,否則訓練會非常慢。 On Kaggle, choose GPU under Settings → Accelerator (phone verification required), or training will be very slow.


1|變數與資料型別|Variables & types

中文:變數是貼在值上的名字。常見型別:整數 int、小數 float、字串 str、布林 bool。 English: A variable is a name attached to a value. Common types: int, float, str, bool.

epochs = 3              # int 整數
lr = 0.002              # float 小數(learning rate 學習率)
label = "bird"          # str 字串
is_bird = True          # bool 布林
print(type(epochs), type(lr), type(label), type(is_bird))

prob = 0.9987
print(round(prob * 100, 1))   # 99.9
小練習 Exercise

建立變數 n_images = 200、valid_pct = 0.2,算出驗證集有幾張圖。 Create n_images = 200 and valid_pct = 0.2; compute how many images go to validation.

答案 Answer:n_images * valid_pct → 40.0;要整數用 int(n_images * valid_pct) → 40。


2|字串與 f-string|Strings & f-strings

中文:f-string 用 f"...{變數}..." 把變數放進文字裡,課程常用來組搜尋關鍵字或印出結果。 English: An f-string f"...{var}..." puts values inside text. The course uses it to build search terms and print results.

o = "forest"
print(f"{o} photo")            # forest photo(L01 搜尋關鍵字 / L01 search term)

is_bird, _, probs = "bird", 0, [0.0013, 0.9987]
print(f"This is a: {is_bird}.")
print(f"Probability it's a bird: {probs[1]:.4f}")   # 小數 4 位 / 4 decimals

name = "  Bird Photo  "
print(name.strip().lower())    # "bird photo"
print("anchor; target".split("; "))   # ['anchor', 'target'](L04 切文字)
小練習 Exercise

給 kinds = "grizzly",印出 grizzly bear photo。 Given kinds = "grizzly", print grizzly bear photo.

答案 Answer:print(f"{kinds} bear photo")


3|串列與切片|Lists & slicing

中文:串列(list)是有順序的一排資料,從 0 開始編號。切片 a[開始:結束] 不含結束位置。 English: A list is an ordered sequence, indexed from 0. Slicing a[start:stop] excludes stop.

searches = ["forest", "bird"]
print(searches[0])        # forest
print(searches[-1])       # bird(-1 是最後一個 / last item)

nums = [10, 20, 30, 40, 50]
print(nums[1:3])          # [20, 30]
print(nums[:2])           # [10, 20] 前兩個 / first two
print(nums[::2])          # [10, 30, 50] 每隔一個 / every other
print(len(nums))          # 5

nums.append(60)           # 加到最後 / add to end
print(nums)
小練習 Exercise

urls 有 200 個網址,怎麼取出前 6 個? urls has 200 items; how do you get the first 6?

答案 Answer:urls[:6]


4|字典與元組|Dicts & tuples

中文:字典(dict)用「鍵 → 值」查資料;元組(tuple)是不能改的串列,函式常用它一次回傳好幾個值,可以「拆包」接收。 English: A dict maps keys to values. A tuple is an unchangeable list; functions often return several values as a tuple, which you can unpack.

vocab = {0: "bird", 1: "forest"}
print(vocab[0])                     # bird
vocab[2] = "cat"                    # 新增 / add
for k, v in vocab.items():
    print(k, v)

# L01:learn.predict 回傳三個值,用拆包接收
# L01: learn.predict returns three values; unpack them
result = ("bird", 0, [0.9987, 0.0013])
is_bird, idx, probs = result
print(is_bird, probs[idx])

# 不需要的值習慣用 _ 接 / use _ for values you don't need
is_bird, _, probs = result
小練習 Exercise

建立字典 price = {"apple": 5, "egg tart": 12},印出蛋撻價格,再加入 "milk tea": 20。 Build the dict, print the egg tart price, then add milk tea.

答案 Answer:print(price["egg tart"]);price["milk tea"] = 20


5|條件與迴圈|if & for

中文:for 對每個項目重複做事;if 依條件決定做不做。Python 用「縮排」表示哪些行屬於迴圈或條件,縮排錯就會出錯。 English: for repeats for each item; if decides whether to run. Python uses indentation to mark blocks; wrong indentation causes errors.

# L01 下載圖片的結構(簡化)/ simplified L01 download loop
searches = ["forest", "bird"]
for o in searches:
    print(f"downloading {o} photos ...")

# range(n) 產生 0 到 n-1 / range(n) gives 0..n-1
for epoch in range(3):
    loss = 1 / (epoch + 1)
    print(f"epoch {epoch}: loss={loss:.3f}")

prob = 0.97
if prob > 0.5:
    print("bird")
else:
    print("not bird")
小練習 Exercise

用迴圈印出 1 到 10 中所有偶數。 Print every even number from 1 to 10 with a loop.

答案 Answer:

for i in range(1, 11):
    if i % 2 == 0:
        print(i)

6|串列生成式|List comprehensions

中文:把「建立空串列 → for 迴圈 → append」濃縮成一行。課程程式碼大量使用,一定要看得懂。 English: Compresses "empty list → for loop → append" into one line. The course uses it everywhere, so you must be able to read it.

nums = [1, 2, 3, 4, 5]

# 一般寫法 / long way
squares = []
for x in nums:
    squares.append(x * x)

# 生成式 / comprehension
squares = [x * x for x in nums]
evens = [x for x in nums if x % 2 == 0]
print(squares, evens)

# 字典生成式 / dict comprehension
lengths = {w: len(w) for w in ["bird", "forest"]}
print(lengths)            # {'bird': 4, 'forest': 6}
小練習 Exercise

把 files = ["a.jpg", "b.png", "c.jpg"] 中的 .jpg 檔挑出來。 Keep only the .jpg files.

答案 Answer:[f for f in files if f.endswith(".jpg")]


7|函式|Functions

中文:def 定義函式,return 回傳結果。參數可以有預設值。fastai 很常「把函式當參數傳進去」,例如 get_y=parent_label(注意沒有括號)。 English: def defines a function, return gives back a result. Parameters can have defaults. fastai often passes a function as an argument, like get_y=parent_label (note: no parentheses).

# L03:二次函數 / quadratic function from L03
def quad(a, b, c, x):
    return a * x**2 + b * x + c

print(quad(3, 2, 1, 1.5))      # 10.75

# 預設值 / default value
def resize_note(size=400):
    return f"resize to {size}px"
print(resize_note(), resize_note(192))

# 函式當參數 / function as an argument
def parent_label(path):
    return path.split("/")[-2]        # 取資料夾名 / folder name

def label_all(paths, get_y):
    return [get_y(p) for p in paths]

print(label_all(["dest/bird/1.jpg", "dest/forest/2.jpg"], parent_label))

# lambda:一行的小函式 / one-line function
double = lambda x: x * 2
print(double(5))
小練習 Exercise

寫 mse(preds, targets),回傳兩個串列的均方誤差(L03 損失函數)。 Write mse(preds, targets) returning mean squared error (the L03 loss).

答案 Answer:

def mse(preds, targets):
    return sum((p - t) ** 2 for p, t in zip(preds, targets)) / len(preds)
print(mse([1, 2, 3], [1, 2, 5]))   # 1.333...

8|import 與函式庫|Imports & libraries

中文:別人寫好的工具放在「函式庫」裡,用 import 帶進來。from x import * 會把全部名稱帶進來,fastai 課程刻意這樣寫,讓 notebook 簡短。 English: Libraries hold tools other people wrote; import brings them in. from x import * imports every name; the fastai course does this on purpose to keep notebooks short.

import math
print(math.sqrt(16))

import numpy as np          # 慣例縮寫 / conventional alias
import pandas as pd

from pathlib import Path    # 只拿一個名稱 / import one name

# 課程第一課會看到 / seen in L01:
# from fastai.vision.all import *
# from duckduckgo_search import DDGS

9|檔案路徑 pathlib|File paths

中文:Path 物件代表檔案或資料夾,可以用 / 接路徑,比自己拼字串安全。L01 的 path/o 就是這個。 English: A Path object represents a file or folder; / joins paths safely. L01's path/o uses this.

from pathlib import Path

path = Path("bird_or_not")
dest = path / "forest"              # bird_or_not/forest
print(dest, dest.name, dest.parent)

dest.mkdir(parents=True, exist_ok=True)   # 建資料夾 / make folder
(dest / "note.txt").write_text("hello")
print([p.name for p in dest.iterdir()])   # 列出檔案 / list files
print(Path("dest/bird/1.jpg").parent.name)   # bird(就是 parent_label 的做法)

10|錯誤訊息怎麼讀|Reading error messages

中文:錯誤訊息要「從最下面往上讀」。最後一行告訴你錯誤種類,往上找到你自己寫的那一行。把最後一行貼到 Google 或 forums.fast.ai 搜尋,通常有人問過。 English: Read errors bottom-up. The last line names the error; scroll up to find your own line. Paste the last line into Google or forums.fast.ai; someone has usually asked before.

錯誤 Error常見原因 Usual cause
NameError: name 'x' is not defined前面定義 x 的那格還沒執行 The cell that defines it wasn't run
IndentationError縮排不一致 Inconsistent indentation
TypeError型別不對,例如字串加數字 Wrong type, e.g. str + int
IndexError: list index out of range索引超過長度 Index past the end
KeyError字典沒有這個鍵 Missing dict key
ModuleNotFoundError套件沒裝,先 !pip install Package not installed
CUDA out of memoryGPU 記憶體不夠,調小 batch size(L07)GPU memory full
try:
    print(10 / 0)
except ZeroDivisionError as e:
    print("出錯了 / error:", e)

11|NumPy 與廣播|NumPy & broadcasting

中文:NumPy 陣列可以一次對整組數字運算,不用寫迴圈。「廣播」讓形狀不同的陣列也能運算,例如每一列都乘上同一組係數(L05)。 English: NumPy arrays compute on whole groups of numbers without loops. Broadcasting lets arrays of different shapes combine, e.g. multiplying every row by the same coefficients (L05).

import numpy as np

a = np.array([1, 2, 3])
print(a * 10, a.sum(), a.mean())

m = np.array([[1, 2, 3],
              [4, 5, 6]])
print(m.shape)               # (2, 3):2 列 3 欄 / 2 rows, 3 cols
coeffs = np.array([0.5, 1.0, 2.0])
print(m * coeffs)            # 廣播:每列都乘 coeffs / broadcast to every row
print((m * coeffs).sum(axis=1))   # 每列加總 = 每個乘客的預測 / per-row sum (L05 predictions)
print(m @ coeffs)            # 矩陣乘法,結果一樣 / matrix multiply, same result
print(np.log1p(np.array([0, 9, 99])))   # L05 處理票價用的 log / log used for Fare
小練習 Exercise

x = np.array([[1, 2], [3, 4]]),算每一欄的平均。 Compute the mean of each column.

答案 Answer:x.mean(axis=0) → [2., 3.]


12|pandas 表格|pandas DataFrames

中文:pandas 的 DataFrame 就是程式裡的 Excel 表格。L04 讀專利資料、L05 讀鐵達尼號、L06 隨機森林、L07 電影評分都用它。 English: A pandas DataFrame is a spreadsheet in code. Used for patents (L04), Titanic (L05), random forests (L06), and movie ratings (L07).

import numpy as np
import pandas as pd

df = pd.DataFrame({
    "Name": ["Ann", "Bob", "Cat", "Dan"],
    "Age": [22, None, 35, 58],
    "Fare": [7.25, 71.3, 8.05, 30.0],
    "Survived": [0, 1, 1, 0],
})
# 真實課程用 df = pd.read_csv("train.csv") / the course uses read_csv

print(df.head())             # 前幾列 / first rows
print(df.describe())         # 統計摘要 / summary stats
print(df.isna().sum())       # 每欄缺幾個 / missing per column

df["Age"] = df["Age"].fillna(df["Age"].mode()[0])   # L05:用眾數補缺值 / fill with mode
df["LogFare"] = np.log1p(df["Fare"])                 # 新增欄位 / new column
print(df[df["Survived"] == 1])                       # 篩選列 / filter rows
print(df.groupby("Survived")["Fare"].mean())         # 分組平均 / group mean
小練習 Exercise

找出 Fare 大於 20 的乘客姓名。 Show names of passengers whose Fare is above 20.

答案 Answer:df[df["Fare"] > 20]["Name"]


13|PyTorch 張量與梯度|Tensors & gradients

中文:張量(tensor)像 NumPy 陣列,但能放到 GPU 上,還能自動算梯度。L03 用它做梯度下降:算損失 → backward() 算梯度 → 往梯度反方向更新參數。 English: A tensor is like a NumPy array that can live on the GPU and compute gradients automatically. L03 uses it for gradient descent: compute loss → backward() → move parameters against the gradient.

import torch

t = torch.tensor([[1., 2.], [3., 4.]])
print(t.shape, t.mean(), t @ t)

# L03 的梯度下降縮小版 / a tiny version of L03 gradient descent
xs = torch.linspace(-2, 2, 20)
ys = 3 * xs**2 + 2 * xs + 1                 # 真正的答案 / the true function

params = torch.tensor([1.0, 1.0, 1.0], requires_grad=True)   # a, b, c 從亂猜開始
lr = 0.01
for step in range(200):
    a, b, c = params
    preds = a * xs**2 + b * xs + c
    loss = ((preds - ys) ** 2).mean()       # MSE 損失
    loss.backward()                         # 算梯度 / compute gradients
    with torch.no_grad():
        params -= lr * params.grad          # 更新 / update
        params.grad.zero_()                 # 清空梯度 / reset
print(params, loss.item())                  # 會接近 [3, 2, 1] / approaches [3, 2, 1]
小練習 Exercise

把 lr 分別改成 0.1、0.5、0.0001 各跑一次,損失有什麼不同?(L03 學習率的觀念) Try lr = 0.1, 0.5 and 0.0001. How does the loss change? (L03 learning rate idea)

答案 Answer:0.1 剛好,損失接近 0,參數變成 [3, 2, 1];0.5 太大,每一步跳過頭,損失爆掉變成 nan;0.0001 太小,200 步後損失還有約 12。 0.1 is just right (loss near 0, parameters reach [3, 2, 1]); 0.5 is too big and overshoots until the loss becomes nan; 0.0001 is too small and the loss is still about 12 after 200 steps.


14|類別與物件|Classes & objects(Part 2,L07 起)

中文:類別是「做物件的模具」。__init__ 在建立物件時執行,self 指物件自己。PyTorch 模型都是繼承 nn.Module 的類別,forward 定義資料怎麼流過模型。 English: A class is a mould for objects. __init__ runs when an object is created; self is the object itself. Every PyTorch model is a class inheriting nn.Module; forward defines how data flows through it.

class Dog:
    def __init__(self, name):
        self.name = name
    def say(self):
        return f"{self.name}: woof"

d = Dog("Lucky")
print(d.say())

# L07:協同過濾的點積模型 / L07 dot-product model
import torch
from torch import nn

class DotProduct(nn.Module):
    def __init__(self, n_users, n_movies, n_factors):
        super().__init__()                         # 先跑父類別的 init / run parent init
        self.user_factors = nn.Embedding(n_users, n_factors)
        self.movie_factors = nn.Embedding(n_movies, n_factors)
    def forward(self, x):
        users = self.user_factors(x[:, 0])
        movies = self.movie_factors(x[:, 1])
        return (users * movies).sum(dim=1)

model = DotProduct(n_users=10, n_movies=20, n_factors=5)
x = torch.tensor([[0, 3], [2, 7]])                 # (使用者, 電影) / (user, movie)
print(model(x))                                    # 呼叫 model(x) 就是執行 forward

特殊方法(dunder):__init__、__call__、__len__、__getitem__、__repr__。L10、L14 會自己寫,例如讓物件可以用 obj[i] 取值。 Dunder methods let objects support obj[i], len(obj), obj(x); written by hand in L10 and L14.

class Dataset:
    def __init__(self, xs, ys): self.xs, self.ys = xs, ys
    def __len__(self): return len(self.xs)
    def __getitem__(self, i): return self.xs[i], self.ys[i]

ds = Dataset([1, 2, 3], ["a", "b", "c"])
print(len(ds), ds[1])          # 3 (2, 'b')

15|迭代器與產生器|Iterators & generators(Part 2)

中文:yield 讓函式「一次給一個」,不用一口氣把全部資料放進記憶體。L14 用它寫 DataLoader,一次吐出一個批次(batch)。 English: yield makes a function hand out one item at a time instead of building everything in memory. L14 uses it to write a DataLoader that yields one batch at a time.

def batches(items, bs):
    for i in range(0, len(items), bs):
        yield items[i:i + bs]

for b in batches(list(range(10)), bs=4):
    print(b)          # [0,1,2,3] [4,5,6,7] [8,9]

it = iter([10, 20, 30])
print(next(it), next(it))   # 10 20

16|裝飾器、*args/**kwargs、partial(Part 2)

中文:*args 收任意多個位置參數,**kwargs 收任意多個具名參數,fastai 用來「把參數往下傳」。partial 先固定部分參數。裝飾器(@)是包住另一個函式、加上額外行為的函式,L16 的回呼(callback)系統會用到。 English: *args / **kwargs collect extra arguments and pass them along. partial fixes some arguments in advance. A decorator (@) wraps a function to add behaviour; L16's callback system uses this idea.

def train(model, epochs=1, **kwargs):
    print(model, epochs, kwargs)
train("resnet18", 3, lr=0.01, wd=0.1)

from functools import partial
def power(x, n): return x ** n
square = partial(power, n=2)
print(square(5))      # 25

import time
def timer(f):
    def wrapper(*args, **kwargs):
        t0 = time.time()
        out = f(*args, **kwargs)
        print(f"{f.__name__} took {time.time() - t0:.3f}s")
        return out
    return wrapper

@timer
def slow_sum(n):
    return sum(range(n))
slow_sum(1_000_000)

自我檢核|Self-check before L01

能不用查資料做到下面五件事,就可以開始 L01。 If you can do these five things without looking anything up, you're ready for L01.

  1. 寫一個 for 迴圈,把串列中每個字串加上 " photo"。 Loop over a list and add " photo" to each string.
  2. 用串列生成式從 range(20) 挑出 3 的倍數。 Use a comprehension to keep multiples of 3 from range(20).
  3. 寫函式 accuracy(preds, labels),回傳答對比例。 Write accuracy(preds, labels) returning the fraction correct.
  4. 用 Path 建立 data/cat 資料夾,並列出 data 底下的資料夾名稱。 Create data/cat with Path and list folder names inside data.
  5. 故意寫錯一行讓它出 NameError,讀懂錯誤訊息,再修好。 Trigger a NameError on purpose, read it, and fix it.
參考答案 Answers
words = ["bird", "forest"]
print([w + " photo" for w in words])

print([i for i in range(20) if i % 3 == 0])

def accuracy(preds, labels):
    return sum(p == l for p, l in zip(preds, labels)) / len(labels)
print(accuracy(["bird", "forest", "bird"], ["bird", "bird", "bird"]))   # 0.666...

from pathlib import Path
Path("data/cat").mkdir(parents=True, exist_ok=True)
print([p.name for p in Path("data").iterdir() if p.is_dir()])

延伸資源|Further resources

相關筆記|Related notes

Built with LogoFlowershow