DUNIN7 · LOOMWORKS · RECORD
record.dunin7.com
Record/Substrate/Cleanup/loomworks-cr-b-memory-events-cleanup-design-v0_1.html
Status Current
Path substrate/cleanup/loomworks-cr-b-memory-events-cleanup-design-v0_1.html
←Back
Loomworks · Substrate · Cleanup

CR-B Memory Events Cleanup Design — v0.1

Version. 0.1

Date. 2026-05-24

Author. Claude Code, on direction from Marvin Percival (DUNIN7 Operator)

Companion documents.

Baseline. loomworks-engine 616bab9 (CR-A merge). Alembic head 0071.

1. Purpose

CR-B — Phase 25 wiring cleanup on memory_events — is the second of three sibling Change Requests from the substrate hygiene scoping note v0.2. It removes the FORAY-readiness columns that have never been operationally used, renames the attestation column to make its Memory-essential WebAuthn-proof role explicit, moves the _ANCHOR_PRIORITY registry into the foray module CR-A established, and replaces the _foray namespace injection logic with the per-call-site reserved-emit pattern CR-A validated.

The work is medium-scope: one Alembic migration (column drops + true column rename + data preserved), six code-site updates, twelve affected test files. Approximately half a day of work. The pattern inherits cleanly from CR-A; deviations are surfaced explicitly in §8.

2. Current state — memory_events Phase 25 wiring

2.1 Columns added by Phase 25

Migration 0047_phase_25_engagement_creation_ui.py added three columns to memory_events:

Column Type Nullable Origin Non-null rows today (dev DB)
content_hash TEXT YES Phase 25 494 (written on every event since Phase 25 landed)
attestation JSONB YES Phase 25 1 (one real WebAuthn proof on an engagement-committed event)
foray_tx_ref TEXT YES Phase 25 0 (never written; pure readiness)

No indexes were added on any of the three columns. No foreign keys. No triggers. The columns are pure additive shape.

2.2 content_hash

Computed by: compute_content_hash(payload: dict) -> str at src/loomworks/memory/events.py:168-180. SHA-256 hex digest of json.dumps(payload, sort_keys=True, separators=(",", ":")) after excluding any pre-existing _foray namespace.

Written by: append_event at src/loomworks/memory/events.py:271. Every event row written since Phase 25 landed carries a non-NULL content_hash.

Read by: Nothing outside events.py itself (the value also lands inside the _foray namespace block at line 249 as a denormalized copy). No application code, no projector, no analytics, no test query asserts content from this column for operational purposes — the dedicated Phase 25 test asserts only that the column is populated.

Confirmation of unused status: grep -rn "content_hash" src/ returns hits in events.py (write site + comments + function), storage/object_store.py (a completely separate content_hash for object-store blobs; same name, different concern), and uploads/upload_event.py (yet another distinct content_hash for the upload-pathway model). None of these reference the memory_events.content_hash column.

2.3 attestation

Written by: stamp_commit_attestation(engagement_id, attestation, db) at src/loomworks/engagement/commit_orchestration.py:109-149. Updates the JSONB column on the most recent engagement_committed or engagement_committed_divergent row for the given engagement.

Called from: Two sites in src/loomworks/api/routers/engagements.py — line 536 (engagement-commit endpoint) and line 632 (the divergent-commit endpoint).

Helper: src/loomworks/engagement/commit_attestation.py:142 — attestation: dict[str, Any] parameter in the verify-then-stamp helper.

Data shape: dict with WebAuthn fields (authenticator data, signature, clientDataJSON, public-key credential id, etc.). The actual verified attestation payload from a WebAuthn ceremony.

This column carries Memory-essential operational data, not FORAY readiness. The "FORAY-decorated name" framing in the scoping note v0.2 is correct: the column lives on the FORAY-readiness column trio added by Phase 25 because attestation lands as part of an "anchorable" engagement-committed event, but the data it stores is WebAuthn proof that Memory uses to prove the commit's authority — independent of any FORAY anchoring.

2.4 foray_tx_ref

Never written, never read. Defined on the ORM (src/loomworks/memory/events.py:104) and referenced in a comment (line 99). Pure FORAY-readiness for a future "the FORAY transaction reference that anchored this event" column. Zero non-NULL rows in production.

2.5 _ANCHOR_PRIORITY registry

src/loomworks/memory/events.py:112-165. A module-level dict mapping event_kind strings → priority labels ("critical", "high", "standard", "low"). Today's 19 entries:

Event kind Priority Origin
engagement_committed critical Phase 25
engagement_committed_divergent critical Phase 25
membership_created high Phase 25
agent_registered high Phase 25
seed_drafted standard Phase 25
seed_amended standard Phase 25
discovery_to_seed_extracted standard Phase 53 P53-D12
seed_committed_from_brief standard Phase 54 P54-D5
engagement_created_from_assistance standard Phase 55 P55-D8
finding_produced standard Phase 25
finding_addressed standard Phase 25
induction_cycle_recorded standard Phase 25
candidate_engagement_discarded low Phase 25
conversation_started standard Phase 31
operator_turn standard Phase 31
companion_turn standard Phase 31
key_provided high Phase 25 (authority)
upload_event_received standard Phase 58
manual_content_contributed standard Phase 59 Sub-arc 4a

Verified count: 19. Matches the scoping note v0.2 figure exactly.

2.6 _foray namespace injection

src/loomworks/memory/events.py:236-255. Inside append_event, after computing the content hash:

content_hash = compute_content_hash(payload)
anchor_priority = _ANCHOR_PRIORITY.get(event_kind)
if anchor_priority is not None:
    payload["_foray"] = {
        "anchorable": True,
        "anchor_priority": anchor_priority,
        "content_hash": content_hash,
        "attestation_ref": None,
    }

The injection mutates the in-memory payload before it lands in the JSONB column. Downstream consumers that walk the payload dict see the _foray namespace; the DB column content_hash is the source of truth for the hash, and the in-payload copy is a denormalization.

2.7 Test surface

Twelve test files reference the affected columns / registry / helper:

Dedicated Phase 25 tests (2 files, 11 test functions):

Incidental references (10 files):

These files do not test the Phase 25 wiring; they reference attestation or content_hash in passing (typically because they exercise event-writing flows that trigger the wiring; their assertions are about other event content).

3. Target state — design decisions

3.1 Column changes

Drop content_hash — TEXT column, written but never read. The 494 non-null values are write-only data; losing them costs nothing operationally. compute_content_hash function becomes unused after the drop and is also removed (§3.5).

Drop foray_tx_ref — TEXT column, never written. Zero data loss.

Rename attestation → commit_webauthn_attestation — true column rename via op.alter_column(..., new_column_name=...). Preserves the 1 non-NULL row (the real WebAuthn proof) and any indexes (there are none today). All read/write sites update to the new name.

Rationale for the rename name: commit_webauthn_attestation is verbose but explicit. The column carries (a) WebAuthn proof (b) recorded at engagement commit. Both pieces are load-bearing — naming it just webauthn_attestation would lose the "this is the commit-time proof" provenance; naming it commit_attestation would lose the "this is specifically a WebAuthn cryptographic proof" specificity. The full name is the right name.

3.2 Registry move

_ANCHOR_PRIORITY moves from src/loomworks/memory/events.py to a new file src/loomworks/foray/anchor_priority.py. The 19 entries come along unchanged. The dict structure stays.

Rename: keep as _ANCHOR_PRIORITY. The scoping note v0.2 floated _FORAY_INTEGRATION_PRIORITY as a candidate. After designing the move:

Recommendation: move without rename. The module path provides the FORAY context; the symbol name stays terse and matches its content. If, during CR-B execution, the unrenamed symbol feels confusing alongside the new module path, that surfaces a v0.2 decision point.

3.3 Injection removal + reserved-emit

The current _foray namespace injection block at events.py:236-255 does two things:

  1. Compute content_hash and store it in the DB column (line 243, 271)
  2. Conditionally inject the _foray namespace into the in-payload dict (lines 245-255)

CR-B removes both:

Replace with reserved-emit call at the write site, per the per-call-site discipline CR-A validated:

# FORAY_RESERVED_LOCATION: memory.<event_kind>
_foray_reserved_emit(
    f"memory.{event_kind}",
    {"event_id": str(event.event_id), "anchor_priority": _ANCHOR_PRIORITY.get(event_kind)},
)

The emit fires for every event regardless of anchor priority (the priority is data inside the payload, not a gate on whether to emit). When FORAY integration eventually lands, the real implementation can decide which events to anchor based on the anchor_priority field.

Placement: after the row lands in the session and after the post-append hooks fire — at the very end of append_event, right before return event. This matches CR-A's pattern of placing the reserved-emit outside the persistence try/except so its semantics is independent.

3.4 stamp_commit_attestation update

The function body at src/loomworks/engagement/commit_orchestration.py:109-149 reads/writes the attestation column. After the rename:

UPDATE memory_events
   SET commit_webauthn_attestation = :att
 WHERE event_id = (
     SELECT event_id FROM memory_events
      WHERE engagement_id = :eid
        AND event_kind IN (
            'engagement_committed',
            'engagement_committed_divergent'
        )
      ORDER BY engagement_version DESC
      LIMIT 1
 )

The function name stamp_commit_attestation stays — it describes the operation, not the column. The function's docstring updates to reference the new column name.

3.5 compute_content_hash resolution

After the content_hash column drops and the _foray namespace injection is removed, compute_content_hash has zero callers in production code.

Decision: remove the function. Loomworks does not use content hashing for any operational purpose (dedup, idempotency, etc.). The function existed solely to feed the FORAY-readiness column.

If a future ship reintroduces a content-hashing requirement (FORAY integration; cryptographic deduplication; replay-attack detection), the function can be rewritten then with the actual semantics needed. Today's function was speculative; remove it.

3.6 ORM model update

MemoryEventRow at src/loomworks/memory/events.py:62-104:

3.7 No PostgreSQL trigger

No trigger exists today on these columns; none added by CR-B. Memory events are append-only; the projector views are maintained by apply_event_to_view and friends in the application layer.

4. Migration plan

4.1 One Alembic migration

0072_memory_events_phase_25_wiring_cleanup.py. Revision 0072, revises 0071.

Upgrade:

  1. ALTER TABLE memory_events RENAME COLUMN attestation TO commit_webauthn_attestation
  2. ALTER TABLE memory_events DROP COLUMN content_hash
  3. ALTER TABLE memory_events DROP COLUMN foray_tx_ref

Order matters: rename first (preserves the 1 row), then drop the two unused columns. Single transaction (Alembic default).

4.2 Downgrade

Reverse the upgrade:

  1. ALTER TABLE memory_events ADD COLUMN foray_tx_ref TEXT NULL
  2. ALTER TABLE memory_events ADD COLUMN content_hash TEXT NULL
  3. ALTER TABLE memory_events RENAME COLUMN commit_webauthn_attestation TO attestation

The downgrade re-adds the dropped columns as NULL — the original 494 content_hash values are not regenerated (lost in upgrade; ok because no consumer needs them; if future work needs hashes, recomputing from payload is straightforward).

Like CR-A: production rollback path is fail-forward with a new migration; the downgrade is for local-dev iteration.

4.3 Index / FK considerations

No indexes on the three columns → nothing to drop or recreate. The pkey, uniqueness constraints, and FK to engagements all remain untouched.

4.4 No data transformation needed

The 1 row of real attestation data is preserved through the rename. The 494 rows of content_hash lose the column but no downstream consumer reads it. The 0 rows of foray_tx_ref data is, trivially, no data.

5. Code changes

5.1 New file — src/loomworks/foray/anchor_priority.py

"""Anchor priority registry — relocated from memory.events by CR-B.

Maps memory event kinds to anchor priority labels. When FORAY
integration eventually opens, this guides anchor-batching decisions.

Established by CR-A's foray module foundation (Phase 61); populated
here by CR-B (Phase 62). The dict structure stays unchanged from its
original location at memory/events.py.

Future event kinds extend this dict by adding a new key/value pair;
the prior practice of adding entries at the bottom of the dict in
phase-origin order is preserved.
"""
from __future__ import annotations


_ANCHOR_PRIORITY: dict[str, str] = {
    "engagement_committed": "critical",
    # ... 18 more entries with the same phase-origin comments as today
}

The full dict + comments from events.py:107-165 move verbatim.

5.2 Update src/loomworks/foray/__init__.py

Add _ANCHOR_PRIORITY re-export:

from loomworks.foray.anchor_priority import _ANCHOR_PRIORITY
from loomworks.foray.reserved_emit import _foray_reserved_emit

__all__ = ["_ANCHOR_PRIORITY", "_foray_reserved_emit"]

This way callers can import either from the submodule (loomworks.foray.anchor_priority) or from the package (loomworks.foray). Today no production code outside events.py imports the registry; tests in the Phase 25 dedicated file import it from events.py and will switch to import from loomworks.foray.

5.3 src/loomworks/memory/events.py

Substantive simplification:

Approximate diff size: ~80 lines removed, ~10 lines added. Net negative.

5.4 src/loomworks/engagement/commit_orchestration.py

stamp_commit_attestation:

No other changes; function signature, error handling, return type all unchanged.

5.5 src/loomworks/engagement/commit_attestation.py

The attestation: dict[str, Any] parameter at line 142 is a function-level variable name, not a column name. It can stay as attestation (the variable holds the WebAuthn attestation payload during verification, before it's stored in the renamed column). No change.

5.6 src/loomworks/api/routers/engagements.py

Six references to attestation in this file at lines 103, 508, 519, 536, 604, 615, 632 — these are all about the request body's body.attestation Pydantic field and the function-call kwarg attestation= to stamp_commit_attestation. None of them are column references. No change.

5.7 src/loomworks/api/schemas.py

The attestation: "AttestationPayload | None" = None fields at lines 289 and 302 are Pydantic request-body shape definitions, not column references. No change.

6. Test impact

6.1 tests/test_phase_25_content_hash_and_foray.py — substantial reshape

This file existed solely to test the Phase 25 wiring being removed. All three tests reference dropped/removed functionality:

If a new test for the registry's existence is desired, add a small test_foray_anchor_priority_registry.py with one assertion that the registry imports and has the expected ~19 entries. Don't over-test a static data structure.

Decision: delete test_phase_25_content_hash_and_foray.py. Optionally add a one-test test_foray_anchor_priority_registry.py smoke file. Inheriting CR-A's no-over-testing-no-ops principle.

6.2 tests/test_phase_25_commit_attestation.py — column rename + file rename

The file tests stamp_commit_attestation and the WebAuthn attestation flow. Direct column references inside the tests:

File rename: test_phase_25_commit_attestation.py → test_commit_webauthn_attestation.py. Match the column rename. Drops the "Phase 25" framing (the test is now about the commit-time WebAuthn attestation, not Phase 25 wiring).

Test logic preserved; the file's purpose (cover the WebAuthn challenge store + verify-then-stamp flow) is exactly what should be tested, regardless of where the column lives in the schema.

6.3 Incidental references in 10 other test files

For each, identify whether the reference is to the attestation column (needs rename) or to a function parameter / Pydantic field / variable named attestation (no change). Most references will be the latter — these tests are about engagement-creation flows, not about the column itself.

Approach during execution: grep "attestation"\|content_hash\|foray_tx_ref\|_ANCHOR_PRIORITY across the 10 files; for each hit, classify (column vs not-column) and update only the column refs. Expected delta: small (5-15 total line changes across 10 files).

6.4 tests/conftest.py

The TRUNCATE chain includes memory_events (no column-specific reference). No update needed.

6.5 New tests

One optional test: test_foray_reserved_emit_called_on_memory_event_write — asserts the reserved-emit hook fires at the append_event write site. Mock loomworks.foray._foray_reserved_emit; trigger an event write; assert the mock was called with "memory.<event_kind>".

This test surfaces if the reserved-emit call is ever accidentally removed or if its arguments drift. Worth adding (one test, ~20 lines) given the reserved-emit pattern is the load-bearing piece of CR-B.

7. Build sequencing (preview)

The full step-by-step plan lives in the companion CR document. Summary:

  1. Step 0 — pre-flight (verify baseline at 616bab9; verify inventory still matches; alembic head 0071)
  2. Step 1 — write Alembic migration 0072
  3. Step 2 — move _ANCHOR_PRIORITY to src/loomworks/foray/anchor_priority.py; update foray/__init__.py exports
  4. Step 3 — update events.py (drop dict + function + columns + injection; add reserved-emit call)
  5. Step 4 — update stamp_commit_attestation to use the renamed column
  6. Step 5 — apply migration; verify schema; round-trip downgrade/upgrade
  7. Step 6 — update tests (delete test_phase_25_content_hash_and_foray.py; rename + update test_phase_25_commit_attestation.py; incidental refs in 10 files; optional new test)
  8. Step 7 — documentation sweep
  9. Step 8 — commit chain (3-5 commits on branch phase-62-memory-events-cleanup)
  10. Checkpoint A — Operator confirms before close
  11. Close — tag phase-62-memory-events-cleanup

8. Architectural decisions worth surfacing

8.1 Registry rename — keep _ANCHOR_PRIORITY, don't rename to _FORAY_INTEGRATION_PRIORITY

The scoping note v0.2 floated the rename. After design analysis: the new module path (src/loomworks/foray/anchor_priority.py) provides the FORAY context; renaming the symbol costs readability across phase-origin comments and assertion sites without adding clarity. Decision documented in §3.2.

If the move-without-rename feels confusing during execution, surface for v0.2 of this design.

8.2 compute_content_hash — remove, don't preserve

The function exists solely to feed the dropped column. Removing it is correct under "code-not-needed should not exist." If a future ship reintroduces a content-hashing requirement with concrete semantics (FORAY anchoring; dedup; replay protection), it gets a fresh function with the actual semantics. Today's function was speculative.

8.3 Reserved-emit at end of append_event, not at every write call site

CR-A had three call sites (three orchestration paths into write_setting_change_event). CR-B has one write site (append_event is the single code path; the docstring at events.py:1-5 documents this). One write site → one reserved-emit call. The per-call-site discipline still applies; "every call site" reduces to "one call" here.

This is structurally simpler than CR-A and validates the discipline scales both directions: 3 → 1 → 12 (CR-C will have 12). No deviation.

8.4 attestation rename — verbose but explicit

commit_webauthn_attestation is 26 characters where attestation was 11. The rename carries both pieces of context (commit-time + WebAuthn). Shorter alternatives (commit_attestation, webauthn_attestation) each drop one piece. The cost of the longer name is line length in queries; the cost of a shorter name is ambiguity. Pick clarity.

8.5 Test file rename + delete

The dedicated test files' framings (test_phase_25_*) reflect the Phase 25 origin. After the cleanup, the names should reflect what the tests cover:

CR-A established the precedent: the audit test file renamed from test_voice_tune_setting_foray_audit.py to test_voice_tune_setting_audit.py. CR-B follows.

8.6 ORM-vs-DB FK posture (inherited from CR-A)

memory_events already has a real ORM-level ForeignKey to engagements(id) (line 84). This pre-dates CR-A's FK deviation. CR-B does not touch this FK — it's load-bearing (the EngagementRow mapping at lines 38-59 was added specifically to make the FK resolvable inside the MemoryEventRow Base).

The CR-A FK deviation pattern applies only to audit's independent DeclarativeBase. memory_events shares its Base with EngagementRow in the same module; the FK resolves fine. No deviation to surface.

9. Risk assessment

CR-B's risks and mitigations:

9.1 ORM rename surfaces unexpected coupling

Risk. Some test or callsite reaches into MemoryEventRow.attestation directly (e.g., via raw SQL or ORM attribute access). The rename misses it; the test or callsite fails at runtime.

Mitigation. Step 7 documentation sweep runs grep -rn "\.attestation\b\|memory_events.attestation" --include="*.py" after the rename. Any hit that's not a function-parameter name (which keeps using attestation) gets updated. Tests catch the rest.

9.2 _ANCHOR_PRIORITY import path drift

Risk. Test code imports _ANCHOR_PRIORITY from loomworks.memory.events. After the move, the import breaks.

Mitigation. The dedicated test file (test_phase_25_content_hash_and_foray.py) is being deleted; its import goes away with it. The other 10 test files don't import _ANCHOR_PRIORITY directly (verified via grep). Step 7 sweep confirms no import-path regressions.

9.3 Reserved-emit at end of append_event vs error path

Risk. If the post-append hooks raise (they don't today — fire_post_append_hooks swallows hook exceptions per the docstring at events.py:320-325), the reserved-emit never fires. Is that the desired semantics?

Mitigation. Yes — the reserved-emit represents "an event landed successfully and is now persistent." If anything in the success path failed, the event didn't fully land, and emitting would be premature. Place the reserved-emit at the very end, after hooks fire, after return event would otherwise execute. This matches "the event is persisted and the post-append hooks have run" semantics.

9.4 Migration column drop fails in production due to view dependency

Risk. The materialized views (current_memory_objects, shape_events, etc.) might reference content_hash or attestation in their definitions; DROP COLUMN would fail.

Mitigation. Step 0 pre-flight runs a check: SELECT viewdef FROM pg_views WHERE schemaname = 'public' AND viewdef ILIKE '%content_hash%' OR viewdef ILIKE '%attestation%' OR viewdef ILIKE '%foray_tx_ref%'. If any view references the columns, surface before running the migration.

9.5 The 1 attestation row is on a row that gets pruned mid-migration

Risk. If a concurrent process deletes engagements during the migration (it doesn't in production — Loomworks never deletes — but local dev / test fixtures might), the rename happens on no-data and the single proof row vanishes.

Mitigation. Single-transaction migration; PostgreSQL's ALTER TABLE ... RENAME COLUMN is atomic. Concurrent writes during the migration are non-issues for a rename. Production has only one engine instance; not a concern.

9.6 Reserved slot budget — 2 slots

Same as CR-A. Most likely consumption candidates:

If a third surprise emerges, halt and surface for v0.2.

10. Carry-forward

10.1 To CR-C (credit substrate cleanup)

10.2 To future FORAY integration

10.3 Deferred from CR-B explicitly

10.4 Methodology notes

The CR-A→CR-B pattern validation: at this point the discipline scales (3 sites → 1 site) cleanly. The third instance (CR-C, 12 sites) will validate scale-up. If CR-C surfaces friction at 12 that wasn't visible at 1 or 3, that's a methodology candidate for the next consolidation.

11. Confidence

Ready for Operator review and CR execution.

The pattern inherits cleanly from CR-A. Column rename is a true rename (no data loss). Column drops are safe (no read sites, no view dependencies expected). Registry move + injection removal are straightforward. compute_content_hash removal is a clean dead-code deletion. Test impact is well-scoped: 2 dedicated files (one deleted, one renamed-and-updated) + 10 incidental files with small individual changes.

The single design call that could go either way is the registry rename (§3.2). Recommendation: keep as-is. If the Operator prefers the rename, that's a one-line decision and CR-B can pivot.

Estimated CR-execution time: 3-5 hours including test run-up and tag-and-close.

DUNIN7 · Done In Seven LLC · Miami, Florida

record.dunin7.com prototype · Generated 2026-07-08