Warm-up · Activity 1 of 7
Warm-up from regular expressions, which the log grep will need. What does this print?
import re
print(re.match(r"(\w+) (\w+)", "ERROR db").group(2))// I4.6 · ~40 min · Intermediate
After this lesson you can log from your own programs with loggers, levels, handlers and formatters, and you have built loggrep, a command-line tool that searches log files with re, logging and argparse.
Lesson 6 of 6 in I4 The standard library for real programs
You will be able to
Warm-up · Activity 1 of 7
import re
print(re.match(r"(\w+) (\w+)", "ERROR db").group(2))Predict · Activity 2 of 7
import logging
logging.info("starting")
logging.warning("disk almost full")Practice · Activity 3 of 7
logging.basicConfig(level=logging.____, stream=sys.stdout, format="%(levelname)s %(message)s")Practice · Activity 4 of 7
import logging
import sys
logging.basicConfig(stream=sys.stdout, format="%(name)s [%(levelname)s] %(message)s")
logging.getLogger("shop.cart").error("empty cart")Practice · Activity 5 of 7
Brain teaser · Activity 6 of 7
import logging
import sys
logging.basicConfig(stream=sys.stdout, format="root: %(message)s")
log = logging.getLogger("app")
log.addHandler(logging.StreamHandler(sys.stdout))
log.warning("disk full")Apply · Activity 7 of 7
Check your work against this list
Read the worked example, then write the exercises. Your code runs in your browser or on your computer and is never uploaded.
Worked example
The module logs through its own logger, shop. basicConfig sends INFO and above to the screen in a custom format; stream=sys.stdout only makes the log visible here as normal output. A second handler on the root logger writes errors, and only errors, to errors.log in a different format. The debug message is dropped, because the level is INFO.
main.py
import logging
import sys
log = logging.getLogger("shop")
def checkout(cart: dict[str, int]) -> int:
log.debug("cart has %d items", len(cart))
if not cart:
log.warning("empty cart, nothing to do")
return 0
total = sum(cart.values())
log.info("checkout total %d cents", total)
return total
logging.basicConfig(level=logging.INFO, format="%(levelname)-7s %(name)s: %(message)s", stream=sys.stdout)
# A second handler: errors only, into a file, with its own format.
errors = logging.FileHandler("errors.log", encoding="utf-8")
errors.setLevel(logging.ERROR)
errors.setFormatter(logging.Formatter("%(name)s|%(levelname)s|%(message)s"))
logging.getLogger().addHandler(errors)
checkout({"bread": 250, "coffee": 890})
checkout({})
log.error("payment service unreachable")
errors.close()
print("errors.log:")
with open("errors.log", encoding="utf-8") as f:
print(f.read(), end="")
Run it with
python main.pyOutput
INFO shop: checkout total 1140 cents
WARNING shop: empty cart, nothing to do
ERROR shop: payment service unreachable
errors.log:
shop|ERROR|payment service unreachableTab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Exercise 1 of 3
The log grep reads lines like 2026-09-29 10:17:12 ERROR db: connection lost. Write parse_line(line), which returns (time, level, message) or None for a line that is not in this form, such as a traceback line. Write level_ok(level, minimum), which is True when level is at least as severe as minimum. The starter splits at spaces and compares level names as text, where "ERROR" < "WARNING".
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
A regular expression with three groups fits: date and time, a word in capitals, and the rest. Use fullmatch, after rstrip("\n").
logging.getLevelNamesMapping() maps "DEBUG" to 10 up to "CRITICAL" to 50. Compare those numbers.
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)"); return None when LINE.fullmatch(...) is None.
One way to solve it. Yours can look different and still pass the checks.
import logging
import re
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)")
def parse_line(line: str) -> tuple[str, str, str] | None:
match = LINE.fullmatch(line.rstrip("\n"))
if match is None:
return None
return match.group(1), match.group(2), match.group(3)
def level_ok(level: str, minimum: str) -> bool:
numbers = logging.getLevelNamesMapping()
return numbers.get(level, 0) >= numbers[minimum]
if __name__ == "__main__":
print(parse_line("2026-09-29 10:17:12 ERROR db: connection lost\n"))
print(parse_line("Traceback (most recent call last):\n"))
print(level_ok("ERROR", "WARNING"), level_ok("INFO", "WARNING"))
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
def parse_line(line: str) -> tuple[str, str, str] | None:
date, time, level, message = line.split(" ", 3)
return f"{date} {time}", level, message
def level_ok(level: str, minimum: str) -> bool:
return level >= minimum
if __name__ == "__main__":
print(parse_line("2026-09-29 10:17:12 ERROR db: connection lost\n"))
print(parse_line("Traceback (most recent call last):\n"))
print(level_ok("ERROR", "WARNING"), level_ok("INFO", "WARNING"))
test_main.py
from main import level_ok, parse_line
def test_parse_line():
"""A log line splits into time, level and message"""
got = parse_line("2026-09-29 10:17:12 ERROR db: connection lost\n")
want = ("2026-09-29 10:17:12", "ERROR", "db: connection lost")
assert got == want, f"parse_line returned {got!r}"
def test_not_a_log_line():
"""A traceback line or an empty line gives None"""
for line in ["Traceback (most recent call last):\n", "\n", " File \"app.py\", line 3\n"]:
got = parse_line(line)
assert got is None, f"for {line!r} parse_line returned {got!r}, expected None"
def test_level_ok():
"""ERROR passes a WARNING minimum, INFO does not"""
assert level_ok("ERROR", "WARNING") is True, "level_ok('ERROR', 'WARNING') should be True"
assert level_ok("INFO", "WARNING") is False, "level_ok('INFO', 'WARNING') should be False"
def test_level_ok_edges():
"""The minimum itself passes, CRITICAL passes everything"""
assert level_ok("DEBUG", "DEBUG") is True, "level_ok('DEBUG', 'DEBUG') should be True"
assert level_ok("CRITICAL", "ERROR") is True, "level_ok('CRITICAL', 'ERROR') should be True"
assert level_ok("DEBUG", "INFO") is False, "level_ok('DEBUG', 'INFO') should be False"
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyExercise 2 of 3
Step 1 is done in the starter. Write grep(lines, pattern, minimum="DEBUG", ignore_case=False): it returns the log lines, without "\n", whose level passes minimum and whose message matches the regular expression pattern. ignore_case=True makes the search ignore case. Lines that are not log lines are skipped, and each is reported with log.debug("line %d skipped: not a log line", number), counting from 1. Run it to search app.log (second tab).
Tab indents and Shift+Tab outdents. To leave the editor with the keyboard, press Esc, then Tab.
The first run downloads Python for your browser (up to 6.5 MB) and keeps it cached. Your code stays on your device.
Compile the pattern once with re.compile(pattern, flags), where flags is re.IGNORECASE or 0.
Loop with enumerate(lines, start=1), so you have the line number for the debug message.
For each line: parse_line; if None, log and continue; otherwise keep the line when level_ok(level, minimum) and regex.search(message).
One way to solve it. Yours can look different and still pass the checks.
import logging
import re
from collections.abc import Iterable
log = logging.getLogger("loggrep")
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)")
def parse_line(line: str) -> tuple[str, str, str] | None:
match = LINE.fullmatch(line.rstrip("\n"))
if match is None:
return None
return match.group(1), match.group(2), match.group(3)
def level_ok(level: str, minimum: str) -> bool:
numbers = logging.getLevelNamesMapping()
return numbers.get(level, 0) >= numbers[minimum]
def grep(lines: Iterable[str], pattern: str, minimum: str = "DEBUG", ignore_case: bool = False) -> list[str]:
regex = re.compile(pattern, re.IGNORECASE if ignore_case else 0)
found = []
for number, line in enumerate(lines, start=1):
parsed = parse_line(line)
if parsed is None:
log.debug("line %d skipped: not a log line", number)
continue
stamp, level, message = parsed
if level_ok(level, minimum) and regex.search(message):
found.append(line.rstrip("\n"))
return found
if __name__ == "__main__":
logging.basicConfig(level=logging.DEBUG, format="%(levelname)s: %(message)s")
with open("app.log", encoding="utf-8") as f:
for line in grep(f, r"timeout|lost", minimum="INFO", ignore_case=True):
print(line)
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
import logging
import re
from collections.abc import Iterable
log = logging.getLogger("loggrep")
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)")
def parse_line(line: str) -> tuple[str, str, str] | None:
match = LINE.fullmatch(line.rstrip("\n"))
if match is None:
return None
return match.group(1), match.group(2), match.group(3)
def level_ok(level: str, minimum: str) -> bool:
numbers = logging.getLevelNamesMapping()
return numbers.get(level, 0) >= numbers[minimum]
def grep(lines: Iterable[str], pattern: str, minimum: str = "DEBUG", ignore_case: bool = False) -> list[str]:
return [line.rstrip("\n") for line in lines if pattern in line]
if __name__ == "__main__":
logging.basicConfig(level=logging.DEBUG, format="%(levelname)s: %(message)s")
with open("app.log", encoding="utf-8") as f:
for line in grep(f, r"timeout|lost", minimum="INFO", ignore_case=True):
print(line)
test_main.py
import unittest
from main import grep
LOG = [
"2026-09-29 10:15:02 INFO web: server started on port 8000\n",
"2026-09-29 10:16:03 WARNING db: slow query took 2.4 s\n",
"Traceback (most recent call last):\n",
"2026-09-29 10:17:12 ERROR db: connection lost\n",
"2026-09-29 10:18:55 ERROR web: request Timeout for /api/orders\n",
]
def test_regex():
"""The pattern is a regular expression, searched in the message"""
got = grep(LOG, r"lost|slow")
want = ["2026-09-29 10:16:03 WARNING db: slow query took 2.4 s", "2026-09-29 10:17:12 ERROR db: connection lost"]
assert got == want, f"grep(LOG, 'lost|slow') returned {got!r}"
def test_minimum_level():
"""minimum='ERROR' leaves out the WARNING line"""
got = grep(LOG, "db:", minimum="ERROR")
assert got == ["2026-09-29 10:17:12 ERROR db: connection lost"], f"with minimum='ERROR' grep returned {got!r}"
def test_ignore_case():
"""ignore_case=True finds Timeout for the pattern timeout"""
assert grep(LOG, "timeout") == [], "without ignore_case, timeout should not match Timeout"
got = grep(LOG, "timeout", ignore_case=True)
assert got == ["2026-09-29 10:18:55 ERROR web: request Timeout for /api/orders"], f"with ignore_case=True grep returned {got!r}"
def test_skipped_line_is_logged():
"""The traceback line is skipped with a DEBUG message on the logger loggrep"""
with unittest.TestCase().assertLogs("loggrep", level="DEBUG") as logs:
grep(LOG, "Traceback")
assert logs.output == ["DEBUG:loggrep:line 3 skipped: not a log line"], f"the logger loggrep recorded {logs.output!r}"
app.log
2026-09-29 10:15:02 INFO web: server started on port 8000
2026-09-29 10:15:40 DEBUG db: pool size 5
2026-09-29 10:16:03 WARNING db: slow query took 2.4 s
2026-09-29 10:17:12 ERROR db: connection lost
Traceback (most recent call last):
2026-09-29 10:17:13 INFO db: reconnected after timeout
2026-09-29 10:18:55 ERROR web: request timeout for /api/orders
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyExercise 3 of 3
Finish loggrep with main(argv=None) -> int: python main.py PATTERN FILE [FILE ...] [--level LEVEL] [-i] [-v]. Print each match as "file: line". --level takes DEBUG to CRITICAL; -v sets logging to DEBUG with basicConfig, otherwise WARNING. A missing file is logged with log.error("%s: no such file", name), and the other files are still searched. Return 0 if anything matched, 1 if nothing, 2 after a missing file. Run it on your own computer, for example python main.py timeout app.log -i -v.
This exercise needs Python on your computer (the browser version cannot run it). The files and commands are below.
Add --level with choices=LEVELS and default="DEBUG", and -i and -v with action="store_true".
Call logging.basicConfig(level=..., format="%(levelname)s: %(message)s") right after parse_args; it writes to stderr, so it never mixes with the matches.
Wrap the open in try/except FileNotFoundError, remember failed = True and continue. At the end: 2 if failed, else 0 if anything matched, else 1.
One way to solve it. Yours can look different and still pass the checks.
import argparse
import logging
import re
import sys
from collections.abc import Iterable
log = logging.getLogger("loggrep")
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)")
LEVELS = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
def parse_line(line: str) -> tuple[str, str, str] | None:
match = LINE.fullmatch(line.rstrip("\n"))
if match is None:
return None
return match.group(1), match.group(2), match.group(3)
def level_ok(level: str, minimum: str) -> bool:
numbers = logging.getLevelNamesMapping()
return numbers.get(level, 0) >= numbers[minimum]
def grep(lines: Iterable[str], pattern: str, minimum: str = "DEBUG", ignore_case: bool = False) -> list[str]:
regex = re.compile(pattern, re.IGNORECASE if ignore_case else 0)
found = []
for number, line in enumerate(lines, start=1):
parsed = parse_line(line)
if parsed is None:
log.debug("line %d skipped: not a log line", number)
continue
stamp, level, message = parsed
if level_ok(level, minimum) and regex.search(message):
found.append(line.rstrip("\n"))
return found
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="loggrep", description="Search log files.")
parser.add_argument("pattern", help="regular expression to search the messages for")
parser.add_argument("files", nargs="+", help="log files to search")
parser.add_argument("--level", default="DEBUG", choices=LEVELS, help="lowest level to show")
parser.add_argument("-i", "--ignore-case", action="store_true")
parser.add_argument("-v", "--verbose", action="store_true", help="log what loggrep does")
args = parser.parse_args(argv)
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING, format="%(levelname)s: %(message)s")
matched = False
failed = False
for name in args.files:
try:
with open(name, encoding="utf-8") as f:
found = grep(f, args.pattern, args.level, args.ignore_case)
except FileNotFoundError:
log.error("%s: no such file", name)
failed = True
continue
for line in found:
print(f"{name}: {line}")
matched = matched or bool(found)
if failed:
return 2
return 0 if matched else 1
if __name__ == "__main__":
sys.exit(main())
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
import argparse
import logging
import re
import sys
from collections.abc import Iterable
log = logging.getLogger("loggrep")
LINE = re.compile(r"(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([A-Z]+) (.*)")
LEVELS = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
def parse_line(line: str) -> tuple[str, str, str] | None:
match = LINE.fullmatch(line.rstrip("\n"))
if match is None:
return None
return match.group(1), match.group(2), match.group(3)
def level_ok(level: str, minimum: str) -> bool:
numbers = logging.getLevelNamesMapping()
return numbers.get(level, 0) >= numbers[minimum]
def grep(lines: Iterable[str], pattern: str, minimum: str = "DEBUG", ignore_case: bool = False) -> list[str]:
regex = re.compile(pattern, re.IGNORECASE if ignore_case else 0)
found = []
for number, line in enumerate(lines, start=1):
parsed = parse_line(line)
if parsed is None:
log.debug("line %d skipped: not a log line", number)
continue
stamp, level, message = parsed
if level_ok(level, minimum) and regex.search(message):
found.append(line.rstrip("\n"))
return found
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="loggrep", description="Search log files.")
parser.add_argument("pattern", help="regular expression to search the messages for")
parser.add_argument("files", nargs="+", help="log files to search")
args = parser.parse_args(argv)
for name in args.files:
with open(name, encoding="utf-8") as f:
for line in grep(f, args.pattern):
print(f"{name}: {line}")
return 0
if __name__ == "__main__":
sys.exit(main())
test_main.py
import contextlib
import io
import unittest
from main import main
LOG = (
"2026-09-29 10:16:03 WARNING db: slow query took 2.4 s\n"
"2026-09-29 10:17:12 ERROR db: connection lost\n"
"2026-09-29 10:17:13 INFO db: reconnected after timeout\n"
)
def run(argv):
with open("t_app.log", "w", encoding="utf-8") as f:
f.write(LOG)
out = io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(io.StringIO()):
try:
status = main(argv)
except SystemExit as stop:
status = stop.code
return status, out.getvalue().splitlines()
def test_match():
"""Matching lines are printed with the file name, status 0"""
status, lines = run(["db", "t_app.log", "--level", "WARNING"])
want = ["t_app.log: 2026-09-29 10:16:03 WARNING db: slow query took 2.4 s", "t_app.log: 2026-09-29 10:17:12 ERROR db: connection lost"]
assert (status, lines) == (0, want), f"main returned {status!r} and printed {lines!r}"
def test_no_match():
"""No matching line gives status 1 and no output"""
status, lines = run(["disk", "t_app.log"])
assert (status, lines) == (1, []), f"main returned {status!r} and printed {lines!r}"
def test_ignore_case():
"""-i finds the lowercase timeout for TIMEOUT"""
status, lines = run(["-i", "TIMEOUT", "t_app.log"])
assert status == 0 and len(lines) == 1, f"with -i main returned {status!r} and printed {lines!r}"
def test_missing_file():
"""A missing file is logged as an error on loggrep, status 2"""
with unittest.TestCase().assertLogs("loggrep", level="ERROR") as logs:
status, lines = run(["db", "missing.log", "t_app.log"])
assert status == 2, f"with a missing file main returned {status!r}, expected 2"
assert logs.output == ["ERROR:loggrep:missing.log: no such file"], f"the logger loggrep recorded {logs.output!r}"
assert len(lines) == 3, f"the other file should still be searched, but main printed {lines!r}"
app.log
2026-09-29 10:15:02 INFO web: server started on port 8000
2026-09-29 10:15:40 DEBUG db: pool size 5
2026-09-29 10:16:03 WARNING db: slow query took 2.4 s
2026-09-29 10:17:12 ERROR db: connection lost
Traceback (most recent call last):
2026-09-29 10:17:13 INFO db: reconnected after timeout
2026-09-29 10:18:55 ERROR web: request timeout for /api/orders
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyimport logging
logging.getLogger("app").setLevel("info")
What Python prints
ValueError: Unknown level: 'info'Why, and the fix
setLevel and basicConfig(level=...) accept a level name only in capitals, "INFO", because that is how the levels are registered. Better, use the constant logging.INFO: a typo there is an AttributeError at once instead of a confusing ValueError.
import logging
import sys
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("{levelname}: {message}"))
What Python prints
ValueError: Invalid format '{levelname}: {message}' for '%' styleWhy, and the fix
By default, a Formatter and basicConfig(format=...) use %-style fields: "%(levelname)s: %(message)s". Braces work only if you also pass style="{". The check runs when the formatter is created, so the mistake shows up at once.
import logging
handler = logging.FileHandler("logs/app.log")
logging.getLogger("app").addHandler(handler)
What Python prints
No such file or directoryWhy, and the fix
FileHandler creates the file, but not the folder around it, so it raises FileNotFoundError. Create the folder first: Path("logs").mkdir(exist_ok=True), with from pathlib import Path.
Python in the browser: Pyodide 314.0.7, MPL-2.0. Licence and source
5 questions, no hints. Score 80% or more to complete the lesson.
Finish every activity above to unlock the exit ticket.