TEST 101 · Testing what a pipeline claims

PYTHON 102 ended with a package another project can import. This course asks what that package promises and writes the tests that hold it to the promise. The code under test is PANDAS 101's dirty lot cleaned by plain Python: clean_price turns "18,500" into 18500 and refuses "ask", clean_rows turns the 24 listings into dicts with a None where the odometer or the rating is missing, total_value and damage_rate sum the lot, and write_report writes three lines to a file. The tool is pytest, run as a subprocess from the notebook the way a terminal would run it, on test files the lessons write to disk, so the summary line, the exit code and the failure report are the real ones.

Eight lessons take a test from one assert to a suite scored against seven injected bugs, and the last one names the bug no test can catch. Every number in the prose was produced by the cell above it.

How this course teaches

Every lesson is the same twenty six cells, and nine of them are yours.

• A prediction you commit to before the cell runs. It is graded on the reasoning, not the guess, and being wrong here is the point.
• Warmups: a one line blank or a two to four line exercise under the theory it practices, each with a four rung hint ladder behind it, where the last rung explains and still does not hand over the code.
• An exercise that is broken when you open it.
• A diagnose cell: a test that runs, passes, and proves nothing. Something below it refuses the green run, usually by installing a bug the test should have caught and watching it stay green.
• A challenge that ends in a sentence you write. The tutor grades the sentence, which means a green tick you earned for the wrong reason can be taken back.

No cell in this course passes in the state it ships. That is deliberate, and it is checked mechanically before the course is published.

The particular danger of this subject is a test that passes and holds nothing. assert result == result is green on a module that multiplies every price by ten. A test whose expected value was printed by the pipeline and pasted in is green on doubled prices, and a damage rate checked against a rate computed from the same rows stays green when every flag is inverted. A test that writes its report into the project folder passes and leaves the file for the next run to read. A test on eight random rows passes about one run in three. pytest.raises(Exception) passes on the wrong error. Every diagnose cell in the course is one of those, and every cross check is the thing that refuses it: a mutant the test has to go red on, a folder listing after the run, twenty runs that all have to be green, and a survivor table that says which claim has no test.

What you will be able to do

• Write a function and a test file, run pytest on it, read the summary line and the exit code, and know why a file the folder walk never collects is not a test.
• Break a function on purpose and watch the suite catch it, tell a test from a tautology by the bug that separates them, and install and restore a mutant without touching the original.
• Cover the edge cases of a function in one parametrized test with named rows, run one row alone with -k, and read which row failed from the report.
• Hold a pipeline to claims whose expected values come from outside it: the file's line count, a type on every row, a total read out of the raw text, a row a person typed.
• Give tests their data through a fixture that is rebuilt per test and their files through tmp_path, compose fixtures, and prove a suite leaves the folder as it found it.
• Compare floats within a tolerance you state, measure how often a random test passes before fixing it, seed the draw, and count the expected value from the rows the seed picks.
• Expect an error by class and by message, tell a crash before the refusal from a missing error and from the wrong words, and turn a failure report into one sentence per test.
• Score a suite against injected bugs, read the survivors as the claims without a test, write those claims, and name the mutant no test can catch as equivalent rather than untested.

The lessons

1. The first test. clean_price, a test file written from the notebook, pytest in a fresh interpreter, 1 passed in 0.01s, and the exit codes 0, 1 and 5. A folder whose only file is named check_ ends with exit 5 and no tests ran, however correct the file. The diagnose is a script that calls a run green because the word failed is absent, over a run that collected nothing.

2. Break it on purpose. A mutant that returns ten times the price fails one test of three: assert result == result and assert clean_price("18,500") pass on it. A mutant that swaps strip() for lower() keeps the spaces, and the whitespace test that goes red on it is the proof the whitespace test tests whitespace. The diagnose is a suite of three asserts true of any integer, 3 passed and a mutation score of zero.

3. Parametrize. Four cases in one test, ids in the report, -k to run the two-comma row alone with 3 deselected, and an edge table with zero, leading-zeros, trailing-comma and stray-commas. The diagnose is a loop inside one test that stops at its first failing case and reports one broken case where the parametrized file reports two.

4. Testing a claim. Four claims about clean_rows: the row count is the file's line count less one, every damage flag is a bool, the total equals the quoted prices in the raw text, A-101 equals a dict a person typed. expected = clean_rows(LINES) then asserting the same call caught none of three mutants. A golden file is a test once a person has read it. The challenge injects one mutant per claim and each fails exactly its own test.

5. Fixtures and tmp_path. A fixture is rebuilt per test: the test that appends sees four prices, the next sees three. tmp_path is a folder that vanishes; a test that wrote test101_report.txt by bare name passed and left the file behind, and leaves_files lists what a run added to the folder. The challenge composes report on lot and tmp_path and leaves nothing.

6. Floats and randomness. 0.1 + 0.2 == 0.3 fails with 0.30000000000000004 printed; the average price is 27926.08695652174, within a cent of 27926.09 under approx(abs=0.01) and not within a thousandth. An unseeded eight-row damage rate equals the lot's in 365 seeds of 1000. Seed 1 picks A-105, A-119, A-103, A-109, A-104, A-116, A-115 and A-123, three damaged, and the test asserts the ids and the count. random.seed in one test leaks into the next; a property over ten seeds holds what every draw must satisfy. The diagnose is the flaky test run twenty times.

7. Exceptions and failure reports. pytest.raises(ValueError, match="not a price"), and the three ways it fails: clean_price(None) crashes with AttributeError before the refusal, clean_price("8300") reports DID NOT RAISE, and match="empty" against not a price: '' reports the regex pair. explain turns a report into one E line per failed test. A table of four bad inputs records the crashes nobody designed. The diagnose is raises(Exception), green on a module that raises KeyError.

8. Three claims. Seven mutants: a comma turned into a dot, an inverted damage flag, a missing km or rating turned into zero, a rounded report line, a KeyError in place of the ValueError, and if row["price"] in place of is not None. The starter suite catches four; two one-line claims about A-102 and A-106 take it to six; the seventh survives every suite because a zero adds nothing to a sum. The diagnose is a damage rate test whose expected value came from the pipeline. The challenge scores at six of seven and names the equivalent mutant.

What you need

• A Python kernel, 3.9 or later, with pytest: pip install pytest. No server, no network; every test runs in a fresh interpreter on this machine.
• About 450 minutes across the eight lessons, at 55 to 60 each.
• PYTHON 102, for modules, writing a .py file from a notebook, and running a subprocess. PANDAS 101 is where the lot comes from, but the course reads it with plain Python.