DUNIN7 · LOOMWORKS · RECORD
record.dunin7.com
Status Current
Path substrate/cleanup/loomworks-substrate-hygiene-scoping-note-v0_2.html
Loomworks · Substrate

Substrate Hygiene Scoping Note — v0.2

Version: 0.2
Date: 2026-05-24
Author: Marvin Percival (DUNIN7), prepared via Claude.ai
Audience: The Operator (Marvin). Companion to the FORAY integration locations inventory v0.1.

What changed from v0.1

v0.2 absorbs five corrections surfaced by CC's Step 1 inventory of FORAY integration locations in the engine. The v0.1 framing was substantially right; the corrections sharpen specific claims that the casual scoping language got wrong.

Five changes from v0.1:

Change 1 — Attestation column survives renamed

v0.1 said "the Phase 25 wiring gets removed" — meaning all three _foray_* columns. The inventory surfaced that the attestation column carries Memory-essential WebAuthn proof on engagement commits (written by stamp_commit_attestation in commit_orchestration.py). It's not FORAY-readiness; it's Memory data under a FORAY-decorated name. Survives renamed (e.g., commit_webauthn_attestation).

Change 2 — Three sibling CRs, not one

v0.1 described the work as a single cleanup arc. The inventory surfaced honest complexity: the three tables have very different scope (audit half a day, Phase 25 wiring half a day, credit substrate multi-day) and no cross-CR dependencies. Three sibling CRs that ship independently is the right shape, ordered smallest first (audit → Phase 25 wiring → credit substrate).

Change 3 — Reserved-location module home is src/loomworks/foray/

v0.1 didn't name a specific location. CC's recommendation is a new top-level module src/loomworks/foray/ — forward-looking, becomes the home for actual FORAY integration when it eventually lands. Today it holds the reserved emitter and the relocated _ANCHOR_PRIORITY registry. Clean separation from audit/ and credit/ modules.

Change 4 — _ANCHOR_PRIORITY registry growth pressure

v0.1 mentioned the registry abstractly. The inventory enumerated it: 19 entries today (across Phases 25/31/53/54/55/58/59), each phase adding some. This adds a real timing argument: if cleanup is deferred and more phases land, the registry keeps growing. Worth pinning this work before the next event-kind addition rather than after.

Change 5 — PostgreSQL trigger is part of the cleanup surface

v0.1 focused on application code. The inventory surfaced that the credit substrate cleanup also touches a PostgreSQL trigger (credit.update_balance_on_flow) — a DDL change, not just code. Whatever replaces credit.foray_action_flows needs a corresponding trigger replacement for the balance projection to keep working.

What's getting cleaned up

The three wrong-pattern items remain:

1. credit.foray_action_flows — credit transactions in FORAY shape. 12 constructor sites (11 in flows.py, 1 in proposal_applier.py), a PostgreSQL balance-update trigger (credit.update_balance_on_flow), and two downstream readers (the conversion observer for referral-idempotency; the reconciliation evaluator for drift detection) that scan JSONB metadata. Multi-day work — its own CR.

2. audit.foray_events — narrative events (primarily Companion setting changes). 1 helper writer called from 3 sites, no downstream readers, no PostgreSQL triggers. Half a day — smallest CR, ships first.

3. memory_events Phase 25 readiness wiring — three columns + the _ANCHOR_PRIORITY registry + the _foray namespace injection logic. Two of the three columns (content_hash, foray_tx_ref) are FORAY-readiness and get removed. The third column (attestation) survives renamed because it carries Memory-essential data. Half a day.

Three sibling Change Requests, ordered

CR-A — Audit cleanup (first, smallest)

First to ship. Smallest scope, no downstream readers, validates the reserved-location pattern before applying it to bigger surfaces.

  • Replace audit.foray_events with a clean audit.events table (drop foray_ prefix)
  • Update the helper writer (1 file) and its 3 call sites
  • Add _foray_reserved_emit("audit.<event_kind>", payload) at each call site
  • Drop audit.foray_events after migration
  • Update 4 test files
CR-B — Phase 25 wiring cleanup (second)

Confirms the pattern works for memory_events (the most foundational surface) and crucially handles the attestation-column rename.

  • Drop content_hash and foray_tx_ref columns from memory_events (both FORAY-readiness)
  • Rename attestation column to commit_webauthn_attestation (Memory-essential, decorated name corrected)
  • Update stamp_commit_attestation in commit_orchestration.py to use the new column name
  • Remove the _foray namespace injection logic at events.py:209-220
  • Move the _ANCHOR_PRIORITY registry to src/loomworks/foray/ (renamed if appropriate — _FORAY_INTEGRATION_PRIORITY is a candidate)
  • Add _foray_reserved_emit("memory.<event_kind>", payload) at the memory_events injection point
  • Update 11 test files (1 dedicated test file plus 10 with incidental references)
CR-C — Credit substrate cleanup (third, largest)

Largest scope, has downstream readers and a PostgreSQL trigger to migrate.

  • Design replacement table (clean credit.events or similar — see Step 2 work for the design)
  • Replace credit.update_balance_on_flow trigger with a clean equivalent that reads from the new table
  • Update 12 constructor sites across flows.py and proposal_applier.py
  • Update the two downstream readers (conversion observer, reconciliation evaluator) to query the new table — these get cleaner because the new table has typed columns instead of JSONB-extract queries
  • Add _foray_reserved_emit("credit.<event_kind>", payload) at each call site
  • Drop credit.foray_action_flows after migration
  • Update 8 test files
  • Multi-day work

The three CRs share no dependencies — each can be drafted, executed, and shipped independently. The ordering is for risk management (build confidence in the pattern starting small) and timing pressure (the _ANCHOR_PRIORITY registry growth pressure argues for not deferring CR-B too long).

The reserved-location pattern

The pattern from v0.1 stands:

  1. A clearly-named function call_foray_reserved_emit(event_kind, payload) — that does nothing today
  2. The function body is a no-op until FORAY integration eventually opens
  3. A consistent comment marker at every call site — # FORAY_RESERVED_LOCATION: <event_kind> — for grep-ability

New in v0.2: module placement. The _foray_reserved_emit function lives in src/loomworks/foray/__init__.py. The new top-level module also holds:

Clean separation from audit/ and credit/ modules — those modules are about Loomworks' own operational concerns; foray/ is about Loomworks' integration with the FORAY external service (today reserved, eventually live).

What this cleanup does NOT do

What this cleanup also addresses

The v0.1 Step 6 named "sweep for other dead weight" as a separate piece of work. CC's inventory surfaced one specific candidate that should be considered alongside the three sibling CRs:

The _ANCHOR_PRIORITY registry's 19 entries are growth pressure. Phases 25/31/53/54/55/58/59 each added entries. If a future memory event kind lands before CR-B ships, the registry grows further, and the eventual cleanup absorbs more state. This argues for scheduling CR-B before the next phase that would add an event kind.

Other sweep candidates (Contributor retirement, KV unification, identifier scheme, job table base) remain as v0.1 named them — separate concerns, sequenced based on the development assessment's Phase A / Phase B / Phase C framing.

What I'm asking from you to proceed

1. Confirm the v0.2 framing is right. The five corrections from v0.1 are the load-bearing changes. If any of them don't land as expected, tell me and v0.3 absorbs.

2. Confirm the three-CR ordering. CR-A (audit) → CR-B (Phase 25 wiring) → CR-C (credit substrate). My read on the timing-pressure argument suggests CR-B might want to land soon after CR-A rather than waiting through CR-C, given the registry's growth pressure. Tell me if you want to adjust the ordering.

3. Confirm Step 2 starts with CR-A. CR-A is the half-day audit cleanup, smallest scope, validates the pattern. Step 2 of the cleanup work is the kickoff for designing CR-A's replacement table. Producing now in parallel with this v0.2 absorption.

A note on the cleanup discipline

The Step 1 inventory was meant to surface corrections to v0.1's framing, and it did. This is the discovery-record discipline working as intended. Without the inventory, we would have stripped a column carrying operational WebAuthn proof, framed the cleanup as one CR when it's three, and missed the PostgreSQL trigger.

v0.2 is not v0.1's failure; it's v0.1's completion. The cheap-corrections-now-rather-than-expensive-corrections-later principle.