tests.md (1921B)
1 # Test integrity 2 3 You are given whole test functions the change adds or alters. The question is not 4 whether they are tidy. The question is whether they can fail. 5 6 - `cannot-fail` — nothing in the body could make the test fail. An assertion that 7 compares a literal to itself, or that only checks a key exists without checking 8 what is under it, tests nothing. 9 - `skips-in-normal-conditions` — the test skips when a tool, file or environment 10 variable is missing, and that absence is ordinary rather than exceptional. A 11 skipping test reports success having checked nothing. Tests that hold code against 12 an external authority must fail when the authority is unavailable. 13 - `name-overclaims` — the name says it checks a property the body does not check. 14 - `passes-on-a-stub` — the test would still pass if the function under test returned 15 its input unchanged, or returned a zero value. Say which. 16 - `asserts-the-shape-not-the-value` — it checks that a field or key is present but 17 never checks its content, so a wrong value passes. 18 19 Report the specific assertion at fault, and what to assert instead. Ignore style, 20 naming and table-versus-loop questions entirely. 21 22 A test that skips itself is marked with the line it skips on; your question about 23 it is only whether the absence it skips on is ordinary. A test with no assertion 24 at all is measured before you read, and is not yours to report unless the rule is 25 another one. Where the functions the tests call are shown, judge `passes-on-a-stub` 26 against the function's body, not against the test alone. 27 28 ## Reading across languages 29 30 A test body arrives whole however the language runs it: a Go `Test` function, a 31 call to `test` or `it` in TypeScript, an `@(test)` procedure in Odin. The skip 32 shapes differ with them (`t.Skip`, `test.skip`, a test returning early) but the 33 rule does not: a test that checks nothing and reports success is a finding in 34 each.