Warm-up · Activity 1 of 7
// I5.5 · ~40 min · Intermediate
pyproject.toml and an installable package
By the end you can turn a folder of code into an installable package with a pyproject.toml, a src layout, a command-line entry point and tests, installed in editable mode.
Lesson 5 of 5 in I5 Testing and project tooling
You will be able to
- Write pyproject.toml: [build-system], name, version, requires-python
- Lay out a package in src/ and install it in editable mode with pip install -e .
- Expose a main(argv) -> int function as a console script and test it
Predict · Activity 2 of 7
Predict it before reading on. What does this print?
import tomllib text = """ [project] name = "wordstat" version = "0.1.0" [project.scripts] wordstat = "wordstat.cli:main" """ data = tomllib.loads(text) print(data["project"]["scripts"])Practice · Activity 3 of 7
Fill in the table name that turns wordstat into a command when the package is installed.
[project.____] wordstat = "wordstat.cli:main"[project.]Practice · Activity 4 of 7
Match each file of the wordstat project to its job.
Practice · Activity 5 of 7
An entry point is written module:function. What does this print?
module, _, func = "wordstat.cli:main".partition(":") print(module, func)Brain teaser · Activity 6 of 7
Puzzle. Which keys does the project table have?
import tomllib data = tomllib.loads(""" [project] name = "wordstat" version = "0.1.0" [project.scripts] wordstat = "wordstat.cli:main" """) print(sorted(data["project"]))Apply · Activity 7 of 7
Small task. Write the pyproject.toml for a project called notes-cli: the package notes_cli lives in src/, needs Python 3.10 or newer, builds with setuptools and installs a command notes that calls main in notes_cli/cli.py. Then, in an activated environment on your machine, run python -m pip install -e . and type notes.
Check your work against this list
Build it yourself
Read the worked example, then write the exercises. Your code runs in your browser or on your computer and is never uploaded.
Worked example
A package skeleton, read the way pip reads it
This program writes a small src-layout project into a temporary folder: pyproject.toml, the package and a test. It reads pyproject.toml with tomllib, then does by hand what an installed console script does: it puts src on sys.path, which python -m pip install -e . would do for you, splits the entry point at the colon, imports the module and calls the function.
main.py
import importlib
import sys
import tempfile
import tomllib
from pathlib import Path
FILES = {
"pyproject.toml": """[build-system]
requires = ["setuptools >= 77.0.3"]
build-backend = "setuptools.build_meta"
[project]
name = "wordstat"
version = "0.1.0"
requires-python = ">=3.10"
[project.scripts]
wordstat = "wordstat.cli:main"
""",
"src/wordstat/__init__.py": '__version__ = "0.1.0"\n',
"src/wordstat/cli.py": """def main() -> int:
print("hello from wordstat")
return 0
""",
"tests/test_cli.py": """from wordstat.cli import main
def test_main_returns_0() -> None:
assert main() == 0
""",
}
root = Path(tempfile.mkdtemp())
for name, text in FILES.items():
(root / name).parent.mkdir(parents=True, exist_ok=True)
(root / name).write_text(text, encoding="utf-8")
print("files:", sorted(p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_file()))
# tomllib.load needs the file opened in binary mode.
with open(root / "pyproject.toml", "rb") as f:
project = tomllib.load(f)["project"]
print("project:", project["name"], project["version"], project["requires-python"])
# pip install -e . makes src/ importable; here we add it to sys.path by hand.
sys.path.insert(0, str(root / "src"))
command, target = next(iter(project["scripts"].items()))
module_name, _, function_name = target.partition(":")
function = getattr(importlib.import_module(module_name), function_name)
print(f"the {command} command runs {module_name}.{function_name}()")
status = function()
print("exit status:", status)
Run it with
python main.pyOutput
files: ['pyproject.toml', 'src/wordstat/__init__.py', 'src/wordstat/cli.py', 'tests/test_cli.py']
project: wordstat 0.1.0 >=3.10
the wordstat command runs wordstat.cli.main()
hello from wordstat
exit status: 0- Only pyproject.toml sits at the root with the tests; the importable code is under src/.
- tomllib.load gets the file opened with "rb"; a text-mode file raises TypeError.
- The last three lines are what typing wordstat does after pip install -e ., except that sys.exit gets the status.
Exercises
Exercise 1 of 3
Step 1: the pyproject.toml
Build step 1 of 3 of the wordstat package. PYPROJECT holds the text of its pyproject.toml. Add a [build-system] table for setuptools, requires-python = ">=3.10", and a [project.scripts] table that makes the command wordstat call wordstat.cli:main. The tests read it with tomllib. On your machine, save the text as pyproject.toml in the project folder.
This exercise needs Python on your computer (the browser version cannot run it). The files and commands are below.
Hints
Hint 1
Tables start with a header on its own line: [build-system], [project], [project.scripts].
Hint 2
The Packaging User Guide gives the setuptools values: requires = ["setuptools >= 77.0.3"] and build-backend = "setuptools.build_meta".
Hint 3
Strings in TOML need quotes: requires-python = ">=3.10" and wordstat = "wordstat.cli:main".
Show a solution
One way to solve it. Yours can look different and still pass the checks.
import tomllib
PYPROJECT = """
[build-system]
requires = ["setuptools >= 77.0.3"]
build-backend = "setuptools.build_meta"
[project]
name = "wordstat"
version = "0.1.0"
description = "Show the most common words in a text file"
requires-python = ">=3.10"
[project.scripts]
wordstat = "wordstat.cli:main"
"""
if __name__ == "__main__":
data = tomllib.loads(PYPROJECT)
print(data["project"]["name"], data["project"]["version"])
print("scripts:", data["project"].get("scripts", {}))
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
import tomllib
# Add a [build-system] table for setuptools, requires-python = ">=3.10",
# and a [project.scripts] table with wordstat = "wordstat.cli:main".
PYPROJECT = """
[project]
name = "wordstat"
version = "0.1.0"
description = "Show the most common words in a text file"
"""
if __name__ == "__main__":
data = tomllib.loads(PYPROJECT)
print(data["project"]["name"], data["project"]["version"])
print("scripts:", data["project"].get("scripts", {}))
test_main.py
import tomllib
import main
def load():
try:
return tomllib.loads(main.PYPROJECT)
except tomllib.TOMLDecodeError as err:
raise AssertionError(f"PYPROJECT is not valid TOML: {err}") from None
def test_build_system():
"""[build-system] names what to install and the backend to build with"""
build = load().get("build-system")
assert build is not None, "add a [build-system] table"
assert build.get("requires"), "[build-system] needs requires, a list such as [\"setuptools >= 77.0.3\"]"
assert build.get("build-backend") == "setuptools.build_meta", "for setuptools, build-backend is \"setuptools.build_meta\""
def test_name_and_version():
"""[project] has the name wordstat and a version"""
project = load().get("project", {})
assert project.get("name") == "wordstat", f"name is {project.get('name')!r}; it must be \"wordstat\""
assert project.get("version"), "add a version, such as \"0.1.0\""
def test_requires_python():
"""requires-python declares the minimum Python, 3.10"""
project = load().get("project", {})
assert project.get("requires-python") == ">=3.10", f"requires-python is {project.get('requires-python')!r}; use \">=3.10\""
def test_console_script():
"""[project.scripts] maps the command wordstat to wordstat.cli:main"""
scripts = load().get("project", {}).get("scripts", {})
assert scripts.get("wordstat") == "wordstat.cli:main", f"[project.scripts] has {scripts}; add wordstat = \"wordstat.cli:main\""
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyExercise 2 of 3
Step 2: the command-line entry point
Step 2 of 3. main.py is src/wordstat/cli.py; the package files hold top_words. Finish main(argv): add an option -n/--top (an int, default 3) and pass it to top_words, and when the file does not exist print wordstat: no such file: PATH to stderr and return 1 instead of crashing. Return 0 on success. Locally, after pip install -e ., run wordstat story.txt -n 2.
This exercise needs Python on your computer (the browser version cannot run it). The files and commands are below.
Hints
Hint 1
parser.add_argument("-n", "--top", type=int, default=3) makes args.top an int.
Hint 2
Wrap the read_text call in try/except FileNotFoundError.
Hint 3
In the except block: print(f"wordstat: no such file: {args.path}", file=sys.stderr) and return 1.
Show a solution
One way to solve it. Yours can look different and still pass the checks.
import argparse
import sys
from pathlib import Path
from wordstat.core import top_words
def main(argv: list[str] | None = None) -> int:
"""Entry point of the wordstat command: wordstat = "wordstat.cli:main"."""
parser = argparse.ArgumentParser(prog="wordstat", description="Show the most common words in a file.")
parser.add_argument("path", help="the text file to read")
parser.add_argument("-n", "--top", type=int, default=3, help="how many words to show")
args = parser.parse_args(argv)
try:
text = Path(args.path).read_text(encoding="utf-8")
except FileNotFoundError:
print(f"wordstat: no such file: {args.path}", file=sys.stderr)
return 1
for word, count in top_words(text, args.top):
print(f"{count} {word}")
return 0
if __name__ == "__main__":
sys.exit(main())
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
import argparse
import sys
from pathlib import Path
from wordstat.core import top_words
def main(argv: list[str] | None = None) -> int:
"""Entry point of the wordstat command: wordstat = "wordstat.cli:main"."""
parser = argparse.ArgumentParser(prog="wordstat", description="Show the most common words in a file.")
parser.add_argument("path", help="the text file to read")
# Add -n/--top (an int, default 3), and return 1 with a message on
# stderr when the file does not exist.
args = parser.parse_args(argv)
text = Path(args.path).read_text(encoding="utf-8")
for word, count in top_words(text):
print(f"{count} {word}")
return 0
if __name__ == "__main__":
sys.exit(main())
test_main.py
import contextlib
import io
import tempfile
from pathlib import Path
import main
TEXT = "The cat sat. The cat ran! The end."
def run(*args):
out, err = io.StringIO(), io.StringIO()
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp, "story.txt")
path.write_text(TEXT, encoding="utf-8")
argv = [str(path) if arg == "FILE" else arg for arg in args]
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
status = main.main(argv)
return status, out.getvalue(), err.getvalue()
def test_prints_top_three():
"""wordstat FILE prints the three most common words and returns 0"""
status, out, _ = run("FILE")
assert out == "3 the\n2 cat\n1 sat\n", f"printed {out!r}"
assert status == 0, f"returned {status!r}; return 0 on success"
def test_top_option():
"""-n 1 and --top 2 choose how many words to print"""
_, out, _ = run("-n", "1", "FILE")
assert out == "3 the\n", f"with -n 1 printed {out!r}"
_, out, _ = run("--top", "2", "FILE")
assert out == "3 the\n2 cat\n", f"with --top 2 printed {out!r}"
def test_missing_file():
"""A missing file prints a message on stderr and returns 1"""
try:
status, out, err = run("no-such-file.txt")
except FileNotFoundError:
raise AssertionError("FileNotFoundError escaped main; catch it and return 1") from None
assert status == 1, f"returned {status!r}; return 1 when the file is missing"
assert "no-such-file.txt" in err, f"stderr was {err!r}; name the missing file there"
assert out == "", f"stdout should stay empty, got {out!r}"
wordstat/__init__.py
"""Show the most common words in a text."""
__version__ = "0.1.0"
wordstat/core.py
from collections import Counter
def top_words(text: str, n: int = 3) -> list[tuple[str, int]]:
"""The n most common words, lowercased and without punctuation, most common first."""
words = [word.strip(".,!?;:\"'()").lower() for word in text.split()]
return Counter(word for word in words if word).most_common(n)
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyExercise 3 of 3
Step 3: the tests
Step 3 of 3. main.py is tests/test_core.py. It has one test for top_words; add at least three focused tests: that case is ignored, that punctuation is stripped, and that n limits how many words come back. The checks run your tests against the real top_words and against three broken versions, one per rule. Locally, run them with python -m unittest from the project root after pip install -e .
This exercise needs Python on your computer (the browser version cannot run it). The files and commands are below.
Hints
Hint 1
One behaviour per test: def test_ignores_case(self) -> None: with a single assertEqual.
Hint 2
Pick inputs where the rule matters: "The the THE" for case, "end. end!" for punctuation, "a b c d" with n=2 for the limit.
Hint 3
self.assertEqual(top_words("The the THE", 1), [("the", 3)])
Show a solution
One way to solve it. Yours can look different and still pass the checks.
import unittest
from wordstat.core import top_words
class TestTopWords(unittest.TestCase):
def test_most_common_first(self) -> None:
self.assertEqual(top_words("b a b", 2), [("b", 2), ("a", 1)])
def test_ignores_case(self) -> None:
self.assertEqual(top_words("The the THE", 1), [("the", 3)])
def test_strips_punctuation(self) -> None:
self.assertEqual(top_words("end. end!", 1), [("end", 2)])
def test_n_limits_the_result(self) -> None:
self.assertEqual(len(top_words("a b c d", 2)), 2)
def test_empty_text(self) -> None:
self.assertEqual(top_words("", 3), [])
if __name__ == "__main__":
suite = unittest.TestLoader().loadTestsFromTestCase(TestTopWords)
unittest.TextTestRunner(verbosity=2).run(suite)
Run it on your computer
Install Python 3.14 or newer. Save these files in one folder, open a terminal in that folder, and run the commands below.
main.py
import unittest
from wordstat.core import top_words
class TestTopWords(unittest.TestCase):
# Add one focused test per behaviour: case is ignored, punctuation is
# stripped, and n limits how many words come back.
def test_most_common_first(self) -> None:
self.assertEqual(top_words("b a b", 2), [("b", 2), ("a", 1)])
if __name__ == "__main__":
suite = unittest.TestLoader().loadTestsFromTestCase(TestTopWords)
unittest.TextTestRunner(verbosity=2).run(suite)
test_main.py
import unittest
from collections import Counter
import main
def run_against(implementation):
saved = main.top_words
main.top_words = implementation
try:
result = unittest.TestResult()
unittest.TestLoader().loadTestsFromTestCase(main.TestTopWords).run(result)
finally:
main.top_words = saved
return result
def keeps_case(text, n=3):
words = [word.strip(".,!?;:\"'()") for word in text.split()]
return Counter(word for word in words if word).most_common(n)
def keeps_punctuation(text, n=3):
return Counter(word.lower() for word in text.split()).most_common(n)
def ignores_n(text, n=3):
words = [word.strip(".,!?;:\"'()").lower() for word in text.split()]
return Counter(word for word in words if word).most_common()
BROKEN = {
"case is not ignored": keeps_case,
"punctuation is kept": keeps_punctuation,
"n is ignored": ignores_n,
}
def test_correct_code_passes():
"""At least 4 tests, and all pass with the real top_words"""
result = run_against(main.top_words)
failing = [test.id() for test, _ in result.failures + result.errors]
assert result.wasSuccessful(), f"these tests fail on correct code: {failing}"
assert result.testsRun >= 4, f"{result.testsRun} tests ran; write at least 4 focused tests"
def test_every_bug_is_caught():
"""Each of three broken versions makes a test fail"""
missed = [bug for bug, broken in BROKEN.items() if run_against(broken).wasSuccessful()]
if missed:
assert False, f"no test fails when {missed[0]}; add a test for it"
wordstat/__init__.py
"""Show the most common words in a text."""
__version__ = "0.1.0"
wordstat/core.py
from collections import Counter
def top_words(text: str, n: int = 3) -> list[tuple[str, int]]:
"""The n most common words, lowercased and without punctuation, most common first."""
words = [word.strip(".,!?;:\"'()").lower() for word in text.split()]
return Counter(word for word in words if word).most_common(n)
On macOS and Linux, type python3 wherever these commands say python, as in the first lesson.
Run the program:
python main.pyRun the checks (needs learnrun.py in the same folder):
python learnrun.py testDownload learnrun.pyCommon mistakes
A version without quotes
import tomllib
PYPROJECT = """
[project]
name = "wordstat"
version = 0.1.0
"""
print(tomllib.loads(PYPROJECT)["project"]["version"])
What Python prints
tomllib.TOMLDecodeError: Expected newline or end of document after a statement (at line 4, column 14)Why, and the fix
In TOML, 0.1.0 is not a value: 0.1 is read as a number and the second dot makes no sense after it. Versions, names and version specifiers are strings, so quote them: version = "0.1.0", requires-python = ">=3.10". The line and column in the message point at the problem.
Opening pyproject.toml in text mode
import tomllib
from pathlib import Path
Path("pyproject.toml").write_text('[project]\nname = "wordstat"\n', encoding="utf-8")
with open("pyproject.toml") as f:
print(tomllib.load(f)["project"]["name"])
What Python prints
TypeError: File must be opened in binary mode, e.g. use `open('foo.toml', 'rb')`Why, and the fix
tomllib.load needs a file opened in binary mode, so it can handle the encoding itself: open("pyproject.toml", "rb"). For text you already have as a str, use tomllib.loads instead.
Importing a src-layout package that is not installed
# Run from the project root, before python -m pip install -e .
from pathlib import Path
Path("src/wordstat").mkdir(parents=True, exist_ok=True)
Path("src/wordstat/__init__.py").write_text('__version__ = "0.1.0"\n', encoding="utf-8")
import wordstat
print(wordstat.__version__)
What Python prints
ModuleNotFoundError: No module named 'wordstat'Why, and the fix
With the src layout the package lives in src/wordstat, and src is not on sys.path, so running from the project root cannot find it. That is deliberate: it stops you testing a copy that is not what users install. Activate your environment and run python -m pip install -e . once; after that, import wordstat works everywhere in that environment.
Python in the browser: Pyodide 314.0.7, MPL-2.0. Licence and source
Exit ticket
5 questions, no hints. Score 80% or more to complete the lesson.
Finish every activity above to unlock the exit ticket.