Loomworks — standing note: a check is not trusted until it has been observed failing — v0.1

Version. 0.1 Date. 2026-08-07 Author. Claude Code (CR-2026-182 execution session). Operator: Marvin Percival. Charter. standing-notes/dunin7-standing-authorization-charter-v0_1. Origin. Four instances in the CR-2026-175→182 sequence of a check that reported green while unable to catch anything. Reads with. standing-notes/loomworks-standing-note-safety-mechanisms-that-blind-v0_1; standing-notes/loomworks-standing-note-spend-context-built-at-the-gate-v0_1.

The rule

Any newly written check — test, guard, gate, assertion, lint rule — is not trusted until it has been observed failing for the reason it exists.

Not "tests pass." Not "the gate is wired." Observed failing, deliberately, for its own reason, and then observed passing again once the thing is restored.

This already applied to CI gates by practice (CR-2026-167 broke every gate before trusting it; CR-2026-173 and CR-2026-174 proved the mypy and ruff gates red-then-green on real infrastructure). It is now general. The sequence below shows the failure is not specific to CI.

Why: four instances in one sequence

| # | The check | Why it could not catch anything | Caught by | |---|---|---|---| | 1 | The route-dependency guard (CR-2026-178) | A request-based version returns 200 for a route that 422s every real caller — conftest stubs the very dependency that breaks it | break-and-observe | | 2 | The mypy baseline (CR-2026-181) | Four already-fixed errors still authorized; the floor had drifted looser than reality, so a regression reintroducing any of them would have passed | reading the regeneration diff | | 3 | The non-erasure guard (CR-2026-181) | Not vacuous — it fired correctly. The temptation was to exempt it for a variable named truncated, which would have made it vacuous for every future case | refusing the exemption | | 4 | The selector-mounting test (CR-2026-182) | Passed twice while incapable of failing: first the harness had no workspaces so a different guard suppressed the control anyway; then it rendered a bare <div> and never rendered the component under test | break-and-observe |

Break-and-observe caught three of the four. The one it did not — the drifted baseline — was caught by reading a diff, and is the reason the regeneration rule exists (scripts/check_mypy_baseline.py). That is worth stating plainly: break-and-observe is necessary and not sufficient. A check can also rot after being correctly proven, and nothing about proving it once prevents that.

What makes this failure mode different

A vacuous check is indistinguishable from a passing one by reading the result. Every other defect class announces itself somewhere — a red test, an error, a wrong value on a screen. This one produces the exact output of success, and produces it reliably, which is worse than producing nothing: it is positive evidence for a claim nobody checked.

Instance 4 is the sharpest illustration. Two consecutive versions of the same test passed. Both looked like coverage. Neither could fail. The second was more elaborate than the first — mocking workspaces to remove a confound — and still tested nothing, because it never rendered the component. Effort spent on a check is not evidence the check works.

How to apply

1. Break the thing the check exists to catch, and watch the check fail. Not a nearby thing — the actual thing. If the check guards "route X is reachable", make route X unreachable.
2. Read the failure message. It must name the real reason. A check that fails with an incidental TypeError where an assertion was expected is telling you it fired for the wrong cause — that happened in CR-2026-176 and the assertions were rewritten before committing.
3. Restore, and watch it pass. Half the proof is that it goes green again for the right reason; a check that fails permanently is not evidence either.
4. For checks over a whole surface, assert non-vacuity in the check itself. The route guard asserts it inspected >100 routes, so it cannot silently become a no-op if route collection changes. A count assertion costs one line and survives refactors that quietly empty the input.
5. When a guard is inconvenient, move your code, not the guard. Instance 3: a local variable was renamed rather than the non-erasure guard exempted. An exemption is permanent and applies to every future case; a rename costs nothing and applies to one.
6. Ratcheting checks need a re-proof discipline, not only a first proof. A baseline, allowlist, or snapshot can loosen after it was correctly established. Whatever regenerates it must be read, not run and committed.

The general form

> Treat a green check as a claim, not as evidence — until you have seen it go red on purpose.

The three standing notes now point at one thing from different angles: a mechanism can be correct and still hide what it was meant to reveal (spend context at the gate), a mechanism that makes something safe in one dimension can blind you in another (test isolation, unit coverage), and — here — a mechanism can report success while structurally incapable of reporting anything else. In all three, the artifact that looked like assurance was produced by the same machinery that suppressed the warning.

DUNIN7 — Done In Seven LLC — Miami, Florida Loomworks — standing note: observe the failure before trusting the check — v0.1 — 2026-08-07