DUNIN7 · LOOMWORKS · RECORD
record.dunin7.com
Record/Substrate/Cleanup/phase-63-cr-credit-substrate-cleanup-v0_1.html
Status Current
Path substrate/cleanup/phase-63-cr-credit-substrate-cleanup-v0_1.html
←Back
Loomworks · Phase 63 · Change Request

Phase 63 Change Request — Credit substrate cleanup (CR-C) — v0.1

CR identifier. CR-2026-XXX (next number in sequence; CR drafter confirms against engine repo docs/phase-crs/ listing before archival).

Version. 0.1

Date. 2026-05-24

Status. Drafted; awaiting Operator approval. No pre-flight Step 0 yet — Step 0 runs at CR execution time.

Author. Claude Code (CR drafter) on direction from Marvin Percival (DUNIN7 Operator).

Phase number. 63 (Phase 62 CR-B closed and tagged phase-62-memory-events-cleanup at engine commit ead29a8; Phase 63 is the next available).

Companion documents.

Baseline. loomworks-engine ead29a8 (CR-B merge). Alembic head 0072. Test count 2670 pytest passing at baseline.

1. Background and motivation

CR-A (Phase 61 — audit cleanup) and CR-B (Phase 62 — memory_events Phase 25 wiring) have closed. The reserved-location pattern is validated; src/loomworks/foray/ is established and populated; per-call-site discipline scales 3 sites → 1 site cleanly.

CR-C is the third and largest sibling CR. It addresses the credit substrate:

CR-C transforms credit.foray_action_flows into credit.flows (clean Loomworks-natural name), extracts three typed columns from the JSONB (event_kind, turn_event_id, converted_person_id) to replace JSONB-extract queries in the readers, renames the JSONB column from metadata to extra_metadata to retire the SQLAlchemy reserved-name workaround, renames timestamp to occurred_at (matching CR-A and CR-B conventions), preserves the PostgreSQL trigger (it binds to the table OID and references column names CR-C does not change), preserves the intra-credit FK (transparent through RENAME TABLE), and adds _foray_reserved_emit at all 12 constructor sites per the per-call-site discipline.

CR-C closes Phase A of the development assessment. The FORAY-shape cleanup arc completes when CR-C ships.

The discovery trajectory: scoping note v0.1 → CC's Step 1 inventory → scoping note v0.2 (absorbed corrections) → CR-A close → CR-B close → this CR (CR-C).

2. Scope

2.1 In-scope

Engine-only work. The complete set:

2.2 Out-of-scope

2.3 Scope-boundary check

If CC finds itself implementing any of the out-of-scope items, it has crossed a scope boundary — halt and surface per §10.

3. Architectural decisions

All decisions trace to the design document. CC consumes them; does not relitigate.

Decision Setting Source
D1 Table name. credit.flows (drop foray_action_ prefix; preserve "flow" vocabulary the codebase uses). Design §3.1
D2 In-place transform via RENAME. PostgreSQL trigger and intra-credit FK survive transparently; minimal DDL. Design §3.6 + §4.1 + §8.1
D3 Preserve the PostgreSQL trigger. No DDL touches it. The trigger references column names CR-C does not change. Design §3.6 + §8.2 + §8.3
D4 Three new typed columns: event_kind (NOT NULL), turn_event_id (NULL), converted_person_id (NULL). Design §3.2
D5 9-value event_kind taxonomy. Explicit at every writer site; no derivation. legacy_unknown for historical rows that match no pattern. Design §3.3 + §8.4
D6 metadata column renamed to extra_metadata. Python attribute matches DB column; retires the SQLAlchemy reserved-name workaround. Design §3.2 + §8.6
D7 timestamp column renamed to occurred_at. Same convention as CR-A and CR-B. Design §3.2
D8 extra_metadata Python attribute (was flow_metadata). Test code keyword-argument updates throughout. Design §8.6
D9 3 readers refactored to typed-column queries. Includes the inline reader inside write_referral_credit_flow (inventory said 2 readers; grep found 3). Design §2.6 + §5.3
D10 Two new indexes. (event_kind, converted_person_id) and (event_kind, turn_event_id). Both small (filtered kinds are minorities). Design §3.4 + §8.9
D11 No new FKs. Intra-credit last_flow_id FK survives the rename; no cross-base FKs added per CR-A precedent. Design §3.5 + §8.7
D12 Per-call-site reserved-emit, 12 emits per insert pattern. Validates the discipline at scale-up (3 → 1 → 12). Design §5.5 + §8 (cross-cutting)
D13 Reserved-slot count: 2 slots. Most likely consumption: test scope expansion within densely-referencing files (test_phase_47_credit_substrate.py has 21 construct sites; test_phase_48_conversion_detection.py has 29 references). Design §9.6

4. Substrate changes

4.1 Alembic migration 0073_credit_flows_cleanup.py

Single migration. Revision 0073, revises 0072.

Upgrade (in-place transform):

  1. op.rename_table("foray_action_flows", "flows", schema="credit") — table rename; trigger + FK + indexes follow transparently
  2. op.alter_column("flows", "metadata", new_column_name="extra_metadata", schema="credit") — rename JSONB column
  3. op.alter_column("flows", "timestamp", new_column_name="occurred_at", schema="credit") — rename timestamp column
  4. op.add_column("flows", sa.Column("event_kind", sa.String(64), nullable=True), schema="credit") — nullable initially for backfill
  5. op.add_column("flows", sa.Column("turn_event_id", UUID(as_uuid=True), nullable=True), schema="credit")
  6. op.add_column("flows", sa.Column("converted_person_id", UUID(as_uuid=True), nullable=True), schema="credit")
  7. UPDATE to backfill event_kind from JSONB: sql UPDATE credit.flows SET event_kind = COALESCE( extra_metadata->>'event', CASE WHEN extra_metadata->>'pipeline_stage' IN ('classify','respond') THEN 'consumption_token' WHEN extra_metadata->>'pipeline_stage' = 'credit_debit' THEN 'consumption_credit' WHEN extra_metadata ? 'grant_id' THEN 'issuance' WHEN extra_metadata ? 'shape_event_id' THEN 'corrective' ELSE 'legacy_unknown' END );
  8. UPDATE to backfill turn_event_id from extra_metadata->>'turn_event_id'
  9. UPDATE to backfill converted_person_id from extra_metadata->>'converted_person_id'
  10. op.alter_column("flows", "event_kind", nullable=False, schema="credit") — tighten to NOT NULL after backfill
  11. op.create_index("ix_credit_flows_kind_converted_person", "flows", ["event_kind", "converted_person_id"], schema="credit")
  12. op.create_index("ix_credit_flows_kind_turn_event", "flows", ["event_kind", "turn_event_id"], schema="credit")

Downgrade:

Reverse: drop the two new indexes; drop the three new columns; rename occurred_at → timestamp; rename extra_metadata → metadata; rename table flows → foray_action_flows.

The downgrade loses the typed-column data but the original JSONB still carries the source values (the typed columns were extracted-not-moved). Production rollback path is fail-forward via a new migration.

4.2 Trigger preservation

The credit.update_balance_on_flow function and trg_balance_update trigger are NOT touched by the migration. Pre-flight (Step 0) confirms the trigger fires on inserts to credit.foray_action_flows; post-migration verification confirms the same trigger fires on inserts to credit.flows.

4.3 FK preservation

The fk_credit_asset_balances_last_flow constraint on credit.asset_balances.last_flow_id references the renamed table transparently. The constraint name does not change.

4.4 Data preservation

The 2 rows in dev DB (count verified at design time) preserve through the migration:

Production rows: count and shape verified at Step 0 pre-flight.

4.5 ORM model update

src/loomworks/credit/models.py:

5. Code changes

5.1 src/loomworks/credit/flows.py — 11 constructor site updates

Each ForayActionFlowRow(...) construct becomes CreditFlowRow(...). Each call adds:

Each db.add(...) call is followed by a _foray_reserved_emit call:

db.add(flow := CreditFlowRow(...))
# FORAY_RESERVED_LOCATION: credit.<event_kind>
_foray_reserved_emit(
    f"credit.{event_kind}",
    {
        "flow_id": str(flow.id),
        "transaction_id": str(transaction_id),
        "asset_id": asset_id,
        "quantity": quantity,
    },
)

(The walrus assignment captures the row instance for the reserved-emit payload; if walrus syntax isn't appropriate in a given site, the row is constructed first then db.add(...) is called separately.)

Per-function specifics in design doc §5.2.

5.2 src/loomworks/credit/proposal_applier.py:399 — 1 constructor site

5.3 Reader refactoring — 3 sites

Reader 1 — conversion_observer._conversion_credit_already_written (src/loomworks/credit/conversion_observer.py:75-85):

Replace JSONB-extract predicates with typed-column predicates:

select(CreditFlowRow.transaction_id).where(
    CreditFlowRow.event_kind == "referral_credit",
    CreditFlowRow.converted_person_id == converted_person_id,
)

Reader 2 — reconciliation_evaluator._sum_recorded_consumption_for_turn (src/loomworks/credit/reconciliation_evaluator.py:211-222):

Add explicit event_kind = 'consumption_credit' filter; replace flow_metadata["turn_event_id"].astext with typed-column predicate:

select(CreditFlowRow.quantity).where(
    CreditFlowRow.event_kind == "consumption_credit",
    CreditFlowRow.from_party == str(person_id),
    CreditFlowRow.to_party == "dunin7",
    CreditFlowRow.asset_id == credit_asset_id,
    CreditFlowRow.turn_event_id == turn_event_id,
)

Reader 3 — write_referral_credit_flow inline (src/loomworks/credit/flows.py:399-423):

Same refactor pattern as Reader 1. Both branches of the conditional (Phase 48 vs Phase 47 fallback) use typed-column predicates.

5.4 Comment + docstring sweeps

Mechanical text updates in:

No behavior changes; just text updates.

5.5 Module init updates

src/loomworks/credit/__init__.py — if it exports ForayActionFlowRow directly, update to CreditFlowRow. (Step 0 verifies.)

6. Test changes

6.1 tests/conftest.py

Single line: credit.foray_action_flows → credit.flows in the TRUNCATE chain.

6.2 tests/test_phase_47_credit_substrate.py — densest test surface

21 construct sites + 24 total references. Mechanical changes throughout:

Estimated 75 line changes.

6.3 tests/test_phase_48_reconciliation_evaluator.py

1 raw-SQL INSERT helper update (line 166): rewrite the INSERT to use the new column names + add the new typed columns (event_kind, turn_event_id, converted_person_id as appropriate for the helper's purpose — likely 'consumption_credit' with turn_event_id populated). Plus any ORM-level references in the rest of the file (mostly mechanical column renames).

6.4 tests/test_phase_48_conversion_detection.py

29 references — most are read-side ORM queries. Mechanical updates:

The queries get cleaner. Some tests may have parametrized fixtures that need adapting.

Estimated 30 line changes.

6.5 tests/test_phase_50_conversion_credit_override.py

17 references — similar shape to test_phase_48_conversion_detection. Same refactor pattern.

Estimated 20 line changes.

6.6 tests/test_phase_49_* (three files)

Smaller surface (2 + 5 + 1 references). Mechanical updates: class rename, column renames, typed-column predicates where reader-side queries exist.

Estimated 15 line changes total.

6.7 New test file

tests/test_foray_reserved_emit_for_credit.py — 2-3 tests asserting the reserved-emit hook fires at credit flow writes with the correct namespaced kind and payload shape. Pattern matches CR-B's test_foray_reserved_emit_for_memory.py.

Estimated 50 lines.

6.8 Total test churn estimate

~180-220 lines across 8 modified files + 1 new file. Mechanical but voluminous.

6.9 No test file renames

Unlike CR-A and CR-B, no test file has a phase-25-style filename framing that's invalidated by the cleanup. Test names like test_phase_47_credit_substrate.py describe their phase-origin which remains historically correct.

7. Migration plan

7.1 Pre-migration verification (Step 0)

Before alembic upgrade head:

  1. Confirm engine baseline at ead29a8 (the CR-B merge)
  2. Confirm alembic head is 0072
  3. Query the current row count, distinct event-kind distribution, and trigger binding: sql SELECT COUNT(*) FROM credit.foray_action_flows; SELECT COALESCE(metadata->>'event', metadata->>'pipeline_stage', 'unknown') AS kind, COUNT(*) FROM credit.foray_action_flows GROUP BY 1 ORDER BY 2 DESC; SELECT tgname FROM pg_trigger WHERE tgrelid = 'credit.foray_action_flows'::regclass;
  4. Verify no views reference credit.foray_action_flows
  5. Run baseline pytest; record passing count (~2670)

7.2 Apply migration on both DBs

.venv/bin/alembic upgrade head
DATABASE_URL='postgresql+asyncpg://playground@localhost:5432/playground_test' .venv/bin/alembic upgrade head

CR-A established the test-DB-needs-its-own-migration pattern; CR-B confirmed.

7.3 Verify post-upgrade

7.4 Round-trip verification

alembic downgrade -1 → alembic upgrade head. Same data preservation pattern as CR-A and CR-B.

7.5 Rollback plan

8. Order of operations

Step 0 — Pre-flight

  1. Pull loomworks-engine main; confirm HEAD at ead29a8
  2. Confirm Alembic head is 0072
  3. Run baseline test suite: .venv/bin/pytest -q; record count (~2670)
  4. Run the §7.1 verification queries; record row count + event-kind distribution
  5. Verify the inventory's findings still hold (12 constructor sites, 3 reader sites, 8 test files)
  6. Create branch phase-63-credit-substrate-cleanup from main

If any check fails, CC halts and surfaces.

Step 1 — Migration scaffold

Write migrations/versions/0073_credit_flows_cleanup.py per §4.1. Do not run yet.

Step 2 — Apply migration + verify

Apply on both dev + test DBs. Run §7.3 verifications. If anything fails (trigger doesn't fire, row count drops, schema mismatch), halt and surface.

Round-trip verify: alembic downgrade -1 → alembic upgrade head preserves all data.

Step 3 — Update ORM model

Apply §5.1 changes to src/loomworks/credit/models.py. Verify imports across the credit module still resolve.

Step 4 — Refactor the 3 readers

Apply §5.3 changes:

Run focused tests on the reader files (tests/test_phase_48_conversion_detection.py, tests/test_phase_48_reconciliation_evaluator.py) to catch any reader-behavior subtleties early.

Step 5 — Update the 12 constructor sites

Apply §5.1 + §5.2 changes (the 11 sites in flows.py + the 1 site in proposal_applier.py). Each site gets:

12 sites total. Per-site review.

Step 6 — Comment + docstring sweep

Apply §5.4 mechanical text updates across the credit module + src/loomworks/api/app.py:301.

Step 7 — Update tests

Apply §6 changes to the 8 test files. Add tests/test_foray_reserved_emit_for_credit.py. Run full test suite.

Expected: ~2670 ± 5 passing tests (some test counts may shift slightly because the typed-column predicates may merge or split parametrized cases). The 2 pre-existing failures from CR-A and CR-B baselines remain.

If tests fail, CC halts and surfaces.

Step 8 — Documentation sweep

grep -rn "ForayActionFlow\|foray_action_flows" --include="*.py" --include="*.md" src/ tests/ docs/

Each hit should be either in the migration 0073 (appropriate historical reference) or already updated. Anything else gets corrected.

Step 9 — Commit chain

5-7 commits on branch phase-63-credit-substrate-cleanup:

  1. "Phase 63 CR-C — Migration 0073 + ORM model rename (CreditFlowRow)"
  2. "Phase 63 CR-C — Refactor three readers from JSONB-extract to typed-column queries"
  3. "Phase 63 CR-C — Update 12 constructor sites with new typed columns + reserved-emit"
  4. "Phase 63 CR-C — Comment + docstring sweep across credit module"
  5. "Phase 63 CR-C — Update tests for credit.flows rename and typed-column queries"
  6. (optional) Methodology methodology-tightening commits as needed

Checkpoint A — Operator confirms

CC reports back:

Operator reviews; either approves close or requests adjustments.

Close — Tag and merge

On Operator approval:

  1. Tag phase-63-credit-substrate-cleanup at branch HEAD
  2. Push branch + tag to origin
  3. Merge to main
  4. Land implementation notes directly into loomworks-record/substrate/cleanup/

9. Acceptance criteria

At Checkpoint A, all of the following must be true:

10. Risk assessment

10.1 Trigger doesn't survive rename (very unlikely)

Risk. A PostgreSQL version edge case unbinds the trigger from the renamed table.

Mitigation. Step 0 records the trigger binding via pg_trigger; Step 2 verifies same binding post-rename. If unbound, halt and recreate via the downgrade-path DDL.

10.2 Backfill query misses a real production pattern

Risk. Production rows have JSONB shapes the §4.1 step-7 backfill doesn't cover; rows land at event_kind = 'legacy_unknown' when they should have been one of the explicit values.

Mitigation. Step 0 runs the backfill query as SELECT first; surfaces the distribution before applying as UPDATE. If unexpected, extend the backfill query before Step 2.

10.3 Reader 2 behavior change (added event_kind filter)

Risk. Reader 2's refactored query adds an explicit event_kind = 'consumption_credit' filter that wasn't in the original. Could subtly differ from the JSONB-extract version for edge cases.

Mitigation. Reader 2 originally filtered by (from_party, to_party, asset_id, turn_event_id). The new filter adds event_kind. For production data, the credit-debit flow is identified by from_party=<person>, to_party='dunin7', asset_id=<credit asset> AND event_kind='consumption_credit' after CR-C. The original implicit identification by metadata.turn_event_id will now use the typed column. The tests in test_phase_48_reconciliation_evaluator.py exercise this; passing tests confirm no behavior change.

10.4 12-site reserved-emit ceremony

Risk. 12 near-identical 5-line emit blocks across two files might feel like ceremony where CR-A's 3 sites felt natural.

Mitigation. If the per-call-site discipline feels wrong at this scale, halt and surface for v0.2 of the design. Don't centralize unilaterally — D12 was the explicit decision.

10.5 Test-pattern complexity in test_phase_47_credit_substrate.py

Risk. 21 construct sites in one test file may have parametrized patterns that don't map cleanly to the new typed-column kwargs.

Mitigation. Reserved slot 1 anticipates this. If a test pattern resists mechanical adaptation, halt and surface.

10.6 extra_metadata keyword rename breaks parametrized tests

Risk. Tests may parametrize across flow_metadata kwargs explicitly; the rename to extra_metadata requires careful sweep.

Mitigation. grep -rn "flow_metadata=" tests/ enumerates every site. Mechanical sweep.

10.7 Reserved-slot budget

Per D13, slot 1 likely absorbed by test pattern complexity, slot 2 by reader behavior subtleties. If a third surprise emerges, halt and surface for v0.2.

10.8 Multi-day execution

CR-C is the multi-day CR. The execution span risks context loss between sessions. Each step's checkpoint should be a stable checkpoint (branch + tests passing) so resumption is clean.

11. Carry-forward

11.1 No CR-D

CR-C closes Phase A of the development assessment. There is no CR-D. The carry-forward is to Phase B (Memory-environment cleanup) and Phase C (Memory work).

11.2 To Phase B

11.3 To future FORAY integration

When FORAY anchoring eventually opens:

11.4 Methodology consolidation pass

After CR-C closes, a methodology consolidation pass should lift the patterns validated across the three-CR arc. Candidates:

  1. Per-call-site reserved-emit discipline — three-instance evidence at 3 / 1 / 12 sites
  2. Test-filename-to-current-substrate alignment — CR-B promoted to methodology candidate at three instances; CR-C had no renames but the pattern is well-established
  3. In-place transform vs. drop-and-recreate — CR-A drop-and-recreate; CR-B in-place rename with FK; CR-C in-place transform with FK + trigger. Rule emerging: rename when downstream bindings exist; recreate when independent
  4. Inventory-grep gap pattern — CR-A no undercounts; CR-B undercounted by 4 phase-specific tests; CR-C inventory's "8 files" matched count but per-file density wasn't surfaced. Pattern: inventories enumerate references but undercount cognitive density per reference

12. Document trailer

This CR is drafted by Claude Code on 2026-05-24 from:

No CR-execution work begins until the Operator approves both this CR and the companion design document.

DUNIN7 · Done In Seven LLC · Miami, Florida

record.dunin7.com prototype · Generated 2026-07-08