Warm-up · Activity 1 of 7
Warm-up from the last lesson: what does this print?
from decimal import Decimal
print(repr(Decimal("0.10") + Decimal("0.20")))// A4.5 · ~40 min · Advanced
Weak references and a ten times faster report
After this lesson you can use weakref.ref and WeakValueDictionary to share objects without keeping them alive, and make a slow report ten times faster, checked and measured.
Lesson 5 of 5 in A4 Performance and numbers
PreviousEnd of the module
You will be able to
- Create weak references with weakref.ref, and know what a dead reference returns and which objects allow one
- Build a cache with weakref.WeakValueDictionary that shares objects in use and lets unused ones go
- Speed up a report step by step: measure, find the hot spot, fix it, check the result and compare a ratio
Predict · Activity 2 of 7
Predict before you read on: what does this print?
import gc import weakref class Report: pass r = Report() ref = weakref.ref(r) alive = ref() is r del r gc.collect() print(alive, ref())Practice · Activity 3 of 7
Fill in the class so that the cache drops a report as soon as nothing else uses it.
cache = weakref.____()cache = weakref.()Practice · Activity 4 of 7
Which values can be weakly referenced? What does this print?
import weakref class Rows(list): pass ok = [] for value in (Rows([1, 2]), [1, 2], 42, "text"): try: weakref.ref(value) ok.append(type(value).__name__) except TypeError: pass print(" ".join(ok))Practice · Activity 5 of 7
Put the steps for speeding up a slow report into order.
- 1.Profile it with cProfile, sorted by cumulative time
- 2.Check that the new version returns the same result
- 3.Fix the hot spot, such as a list scan inside a loop
- 4.Time the current version with timeit.repeat and keep the minimum
- 5.Time it again and compare the ratio with a threshold
Brain teaser · Activity 6 of 7
Brain teaser. What does this print?
import gc import weakref class Report: pass cache = weakref.WeakValueDictionary() cache["kept"] = kept = Report() cache["temp"] = Report() gc.collect() print(sorted(cache))Apply · Activity 7 of 7
Mini-task. Take a function that looks things up in a list inside a loop, from your own code or like slow() in the sample. Write a fast version with a dict built once. Print whether both return the same result, then time both with timeit.repeat, keep the minimum of each, and print only whether the ratio is at least 10. Never print raw seconds.
Check your work against this list
Build it yourself
Read the worked example, then write the exercises. Your code runs in your browser or on your computer and is never uploaded.
Worked example
The slow report, made fast
The slow report scans the customer list for every order and adds float amounts. The fast one builds a dict by id once and adds exact Decimals. The program first checks that both find the same customers and shows the totals; then it times both and prints only whether the ratio clears 10; the real ratio is between 30 and 70. Finally a WeakValueDictionary shares the finished report without keeping it alive.
main.py
import gc
import timeit
import weakref
from decimal import Decimal
class Customer:
def __init__(self, cid: int, name: str) -> None:
self.cid = cid
self.name = name
class Report:
def __init__(self, totals: dict[str, Decimal]) -> None:
self.totals = totals
customers = [Customer(i, f"customer {i}") for i in range(3000)]
orders = [(i % 3000, "0.10") for i in range(9000)]
def slow_totals() -> dict[str, float]:
totals: dict[str, float] = {}
for cid, amount in orders:
customer = next(c for c in customers if c.cid == cid) # scans the list
totals[customer.name] = totals.get(customer.name, 0.0) + float(amount)
return totals
def fast_totals() -> dict[str, Decimal]:
by_id = {c.cid: c for c in customers} # one pass, then O(1) lookups
totals: dict[str, Decimal] = {}
for cid, amount in orders:
name = by_id[cid].name
totals[name] = totals.get(name, Decimal("0")) + Decimal(amount)
return totals
# 1. Same report? Compare before timing anything.
slow, fast = slow_totals(), fast_totals()
print("same customers:", slow.keys() == fast.keys())
print("customer 7:", slow["customer 7"], "vs", fast["customer 7"])
# 2. Measured: best of 3 runs each, reported only as a ratio.
t_slow = min(timeit.repeat(slow_totals, number=1, repeat=3))
t_fast = min(timeit.repeat(fast_totals, number=1, repeat=3))
print("at least 10x faster:", t_slow / t_fast >= 10)
# 3. A weak cache: reports are shared while in use, never kept alive by it.
cache: weakref.WeakValueDictionary[str, Report] = weakref.WeakValueDictionary()
report = Report(fast)
cache["2026-09"] = report
print("cached while in use:", cache.get("2026-09") is report)
del report
gc.collect()
print("cached after del:", "2026-09" in cache)
Run it with
python main.pyOutput
same customers: True
customer 7: 0.30000000000000004 vs 0.30
at least 10x faster: True
cached while in use: True
cached after del: False- The slow report works through about 13 million generator steps; the fast one does 3,000 dict inserts and 9,000 lookups.
- The float total is already off after three orders; the Decimal total is exactly 0.30.
- The ratio is compared with 10, well below the measured 30 to 70, so the check holds on a busy machine too.
- After del and gc.collect() nothing uses the report, so the cache has dropped it.
Change it and run it
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Exercises
Exercise 1 of 3
Step 1: one pass over the customers
customer_names(orders, customers) returns the customer name for each (customer id, amount) order. The starter scans the customer list for every order, which the profile of a big report shows as the hot spot. Build a dict from id to customer once, then look each order up in it. A test counts how often the customer list is looped over: at most once.
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Hints
Hint 1
The inner for loop runs once per order: 500 orders mean 500 passes over the customers.
Hint 2
Build by_id = {customer.cid: customer for customer in customers} before the loop over orders.
Hint 3
Then each name is by_id[cid].name, and a list comprehension over orders is enough.
Show a solution
One way to solve it. Yours can look different and still pass the checks.
class Customer:
def __init__(self, cid: int, name: str) -> None:
self.cid = cid
self.name = name
def customer_names(orders: list[tuple[int, str]], customers: list[Customer]) -> list[str]:
"""The customer's name for every (customer id, amount) order, in order."""
by_id = {customer.cid: customer for customer in customers}
return [by_id[cid].name for cid, _amount in orders]
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
class Customer:
def __init__(self, cid: int, name: str) -> None:
self.cid = cid
self.name = name
def customer_names(orders: list[tuple[int, str]], customers: list[Customer]) -> list[str]:
"""The customer's name for every (customer id, amount) order, in order."""
names = []
for cid, _amount in orders:
for customer in customers:
if customer.cid == cid:
names.append(customer.name)
break
return names
test_main.py
from main import Customer, customer_names
class CountingList(list):
"""A list that counts how often it is looped over."""
loops = 0
def __iter__(self):
CountingList.loops += 1
return super().__iter__()
def test_names():
"""Each order gets its customer's name, in order"""
customers = [Customer(1, "Ada"), Customer(2, "Bo")]
got = customer_names([(2, "5.00"), (1, "3.50"), (2, "1.00")], customers)
assert got == ["Bo", "Ada", "Bo"], f"got {got!r}, expected ['Bo', 'Ada', 'Bo']"
def test_one_pass_over_customers():
"""500 orders loop over the customer list once, not once per order"""
customers = CountingList(Customer(i, f"c{i}") for i in range(100))
orders = [(i % 100, "1.00") for i in range(500)]
CountingList.loops = 0
got = customer_names(orders, customers)
assert got[:2] == ["c0", "c1"] and len(got) == 500, "the names are wrong"
assert CountingList.loops <= 1, f"the customer list was looped over {CountingList.loops} times; build a dict by id once"
def test_no_orders():
"""No orders: an empty list"""
got = customer_names([], [Customer(1, "Ada")])
assert got == [], f"got {got!r}, expected []"
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testExercise 2 of 3
Step 2: exact totals
revenue(orders) adds up the amounts per customer id. The starter adds floats and converts at the end, so three orders of 0.10 become 0.3000000000000000444089209850062616169452667236328125. Add Decimals built from the amount strings instead, starting each total at Decimal("0"), and keep the customers in the order they first appear.
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Hints
Hint 1
Converting to Decimal at the end is too late: the float error is already in the sum.
Hint 2
Change the totals to dict[int, Decimal] and add Decimal(amount) to totals.get(cid, Decimal("0")).
Hint 3
A dict keeps insertion order, so the first-seen order comes for free; return totals directly.
Show a solution
One way to solve it. Yours can look different and still pass the checks.
from decimal import Decimal
def revenue(orders: list[tuple[int, str]]) -> dict[int, Decimal]:
"""Exact revenue per customer id from (customer id, amount) orders."""
totals: dict[int, Decimal] = {}
for cid, amount in orders:
totals[cid] = totals.get(cid, Decimal("0")) + Decimal(amount)
return totals
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
from decimal import Decimal
def revenue(orders: list[tuple[int, str]]) -> dict[int, Decimal]:
"""Exact revenue per customer id from (customer id, amount) orders."""
totals: dict[int, float] = {}
for cid, amount in orders:
totals[cid] = totals.get(cid, 0.0) + float(amount)
return {cid: Decimal(total) for cid, total in totals.items()}
test_main.py
from decimal import Decimal
from main import revenue
def test_exact_cents():
"""Three orders of 0.10 are exactly 0.30"""
got = revenue([(7, "0.10"), (7, "0.10"), (7, "0.10")])
assert got == {7: Decimal("0.30")}, f"got {got!r}, expected {{7: Decimal('0.30')}}"
assert str(got[7]) == "0.30", f"got {str(got[7])!r}, expected '0.30' with both places"
def test_per_customer():
"""Each customer id gets its own total, in first-seen order"""
got = revenue([(2, "19.99"), (1, "5.00"), (2, "0.01")])
assert list(got) == [2, 1], f"keys {list(got)!r}, expected [2, 1]"
assert got[2] == Decimal("20.00") and got[1] == Decimal("5.00"), f"got {got!r}"
def test_no_orders():
"""No orders: an empty dict"""
assert revenue([]) == {}, "no orders should give {}"
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testExercise 3 of 3
Step 3: a cache that forgets
ReportCache.get(key, build) returns the cached report for key, or calls build() and stores the result. The starter stores reports in a dict, so no report is ever freed. Store them in a weakref.WeakValueDictionary: a report still in use is shared and built only once, and once nobody uses it, the entry disappears and the next get builds it again.
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Hints
Hint 1
Only the line in __init__ has to change; get and __len__ work the same on a WeakValueDictionary.
Hint 2
import weakref, then self._reports: weakref.WeakValueDictionary[str, Report] = weakref.WeakValueDictionary().
Hint 3
get() on a WeakValueDictionary returns None for a dropped entry, so the existing if report is None branch rebuilds it.
Show a solution
One way to solve it. Yours can look different and still pass the checks.
import weakref
from collections.abc import Callable
from decimal import Decimal
class Report:
def __init__(self, totals: dict[int, Decimal]) -> None:
self.totals = totals
class ReportCache:
"""Shares reports that are still in use, without keeping any of them alive."""
def __init__(self) -> None:
self._reports: weakref.WeakValueDictionary[str, Report] = weakref.WeakValueDictionary()
def get(self, key: str, build: Callable[[], Report]) -> Report:
report = self._reports.get(key)
if report is None:
report = build()
self._reports[key] = report
return report
def __len__(self) -> int:
return len(self._reports)
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
from collections.abc import Callable
from decimal import Decimal
class Report:
def __init__(self, totals: dict[int, Decimal]) -> None:
self.totals = totals
class ReportCache:
"""Shares reports that are still in use, without keeping any of them alive."""
def __init__(self) -> None:
self._reports: dict[str, Report] = {}
def get(self, key: str, build: Callable[[], Report]) -> Report:
report = self._reports.get(key)
if report is None:
report = build()
self._reports[key] = report
return report
def __len__(self) -> int:
return len(self._reports)
test_main.py
import gc
from decimal import Decimal
from main import Report, ReportCache
class Builder:
"""Builds a report and counts how often it had to."""
def __init__(self):
self.calls = 0
def __call__(self):
self.calls += 1
return Report({1: Decimal("0.30")})
def test_shares_live_report():
"""While a report is in use, get returns the same object and builds once"""
cache, build = ReportCache(), Builder()
first = cache.get("2026-09", build)
second = cache.get("2026-09", build)
assert second is first, "get built a second report although the first is still in use"
assert build.calls == 1, f"build was called {build.calls} times, expected 1"
def test_does_not_keep_alive():
"""Once nobody uses the report, the cache lets it go"""
cache = ReportCache()
report = cache.get("2026-09", Builder())
del report
gc.collect()
assert len(cache) == 0, f"the cache still holds {len(cache)} report(s); store them in a weakref.WeakValueDictionary"
def test_rebuilds_after_drop():
"""A dropped report is built again on the next get"""
cache, build = ReportCache(), Builder()
cache.get("2026-09", build)
gc.collect()
report = cache.get("2026-09", build)
assert build.calls == 2, f"build was called {build.calls} times, expected 2"
assert report.totals == {1: Decimal("0.30")}, "the rebuilt report has the wrong totals"
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testCommon mistakes
A weak reference to a list
import weakref
rows = [1, 2, 3]
ref = weakref.ref(rows)
What Python prints
TypeError: cannot create weak reference to 'list' objectWhy, and the fix
list, dict, int, str and tuple do not support weak references. Wrap the data in an instance of your own class, or subclass list: class Rows(list): pass, then weakref.ref(Rows([1, 2, 3])) works.
Using a dead reference without a check
import gc
import weakref
class Report:
def __init__(self) -> None:
self.name = "September"
ref = weakref.ref(Report())
gc.collect()
print(ref().name)
What Python prints
AttributeError: 'NoneType' object has no attribute 'name'Why, and the fix
The report had no strong reference, so it is gone and ref() returns None. Call the reference once, keep the result in a variable, and check it: report = ref(), then if report is not None: use report.name.
Reading a dropped cache entry with []
import gc
import weakref
class Report:
pass
cache = weakref.WeakValueDictionary()
cache["temp"] = Report()
gc.collect()
print(cache["temp"])
What Python prints
KeyError: 'temp'Why, and the fix
Entries of a WeakValueDictionary vanish when their value dies, even between two lines of your code. Use cache.get("temp"), which returns None, and rebuild the value when that happens, as ReportCache.get does.
Python in the browser: Pyodide 314.0.7, MPL-2.0. Licence and source
Exit ticket
5 questions, no hints. Score 80% or more to complete the lesson.
Finish every activity above to unlock the exit ticket.