This is the professional substrate review you asked for after reading the inventory. Version 0.1 was written from the engine's current implementations of FORAY. Version 0.2 is rewritten after the FORAY project provided a protocol-level primer correcting several substantive misunderstandings I had about FORAY itself.
The review's structure, judgment of what's working, and most recommendations stay the same. What changes is the FORAY section — substantially — and the entity-model diagrams that depend on it.
Both v0.1 and the primer remain in the record. v0.1 documents the position before protocol grounding; the primer documents the protocol grounding itself; v0.2 absorbs the primer and revises accordingly. The trajectory matters because the v0.1 → v0.2 correction is exactly the kind of discovery the project's discipline says to preserve.
I had FORAY wrong at the protocol level. v0.1 treated FORAY as if the three Loomworks tables I'd seen were the protocol. They aren't. They're three incomplete implementations of one component of the protocol. The actual FORAY protocol is a five-construct grammar — Arrangements, Accruals, Anticipations, Actions (the four "A"s forming a typed DAG), plus Attestations as a structurally distinct fifth.
My "unify into one substrate table" recommendation was wrong. The protocol-respecting target is four tables, not one. Collapsing into a single polymorphic table would erase the typed reference structure that the protocol uses for validation. The right shape — per the primer — is foray_transactions (envelope) + foray_components (the four DAG types) + foray_attestations (the cross-cutting fifth) + memory_events (cleaned of readiness wiring).
I conflated "attestation" with "audit row." The attestation JSONB column on memory_events (a WebAuthn assertion at commit time) is a small, single-purpose thing. The protocol's Attestation construct is a structurally distinct fifth component type with its own lifecycle, validity periods, polymorphic subject references, and attestation chains. Same word, different concepts.
I missed the three-layer storage architecture entirely. Layer 1 (Kaspa anchoring), Layer 2 (queryable substrate), Layer 3 (sealed archive). Loomworks is implementing Layer 2 only. Layer 1 belongs to the FORAY anchor service; Layer 3 is out of scope unless litigation-grade preservation joins the roadmap.
The diagnosis of three siblings sharing a grammar but not a shape was correct. The instinct to strip readiness wiring from memory_events was correct. The headline that "the foundation is not broken; drift is around the core, not inside it" still holds. Five of the six priorities (DB boundary, Contributor retirement, KV unification, identifier scheme, job table base) are unaffected by the FORAY correction and stand exactly as v0.1 described them.
The discovery-record discipline asks that corrections be visible alongside the corrected position. v0.1 stays in the record. The primer stays in the record. v0.2 absorbs both and shows the trajectory. A future reader (you, me, or someone else later) can see how the substrate review got from "unify into one table" to "four tables grounded in the protocol" without having to reverse-engineer the path.
The foundation is not broken. The core architectural commitment — memory_events as a canonical append-only event log with typed projections — is exactly right, and it's been protected consistently across 70 migrations. The foundation work was done well.
What's drifted around the foundation is real, and the FORAY drift specifically runs deeper than v0.1 saw. The three sibling FORAY tables aren't just three shapes of one substrate — they're three incomplete implementations of one component (the Layer 2 queryable store) of a five-construct protocol that the engine has only partially modeled. Properly grounding FORAY means rebuilding to four tables that honor the protocol's typed DAG and its cross-cutting Attestation primitive.
The window to do this cleanly is now. Data is wipeable. Existing FORAY usage in the engine is essentially placeholder. Building the protocol-respecting substrate now is one focused phase; doing it after production data accumulates is migration work plus backward-compatibility cost.
My professional advice: two architectural pieces of work first, then proceed. Rebuild FORAY against the protocol (four tables, not one), and tighten the database-level type discipline so the substrate enforces what the application currently only validates. The other four cleanups (KV stores, identifier scheme, Contributor retirement, job table base) are smaller and can ride along or wait.
Before naming what to fix, name what's right. Unchanged from v0.1.
Every change to every MemoryObject is a row in memory_events. Typed views (Assertion, ShapeEvent, RenderEvent, Manifestation, RenderComposition) project subsets into scalar columns for queryability, but the event log is the source of truth and everything else is rebuildable from it. This is the strongest design coherence in the substrate, and it's why "knowledge accumulates and never disappears" works as more than a slogan.
Whatever else changes, this stays. The pattern is correct. The primer reinforces this — the FORAY-respecting design keeps memory_events as the event log, just cleaned of the readiness-wiring columns that don't belong there.
Engagements are the spine of the substrate. Persons get memberships to them; events scope to them; the dashboard aggregates across them; the Companion participates within them. The decision to make Engagement the universal adapter has held up across 70 migrations.
The Engagement's table itself has accumulated cruft (more on that below), but the conceptual centrality of "engagement is the unit" is exactly right.
Nine empty-DDL migrations exist purely as lineage records when a Pydantic vocabulary was extended without a schema change. Heavy data migrations duplicate slices of append_event inline to stay self-contained against future code evolution. This is intentional discipline — the migration chain is documentation, not just schema state.
This pattern is unusual and correct. It means the substrate's history is interrogable. Don't lose it in any cleanup.
This section is essentially new in v0.2. It replaces v0.1's "three siblings sharing a grammar" framing with the protocol-grounded picture from the primer. The primer is the authoritative document for protocol detail; this section summarizes what matters for the substrate review.
FORAY is a five-construct protocol. Four constructs form a typed Directed Acyclic Graph; the fifth is structurally distinct.
The four DAG components describe the transaction. The cross-cutting fifth (Attestations) describes claims about the transaction. The reference shapes differ structurally — DAG components have typed forward references to earlier kinds; Attestations have a single polymorphic subject_refs[] that can point at any component including other attestations.
This is why "one polymorphic table over all five" is the wrong unification target. The protocol's typing carries validation semantics — an Action can reference Anticipations; an Anticipation cannot reference Actions; an Attestation can reference anything. Collapsing into one shape hides this type information from the database.
v0.1 recommended one unified substrate table with a polymorphic event_type discriminator. The primer is explicit: that path was considered and rejected at the protocol level because it collapses typed DAG references with polymorphic subject references, which loses protocol-level type information.
The correct shape is foray_transactions (the envelope) + foray_components (one polymorphic table over the four DAG kinds, with typed reference columns) + foray_attestations (sibling table for the cross-cutting fifth) + memory_events (cleaned of readiness-wiring, returns to being a pure append-only event log).
FORAY architecturally separates three storage layers. Loomworks is implementing one of them.
| Layer | Where | What it holds | Loomworks scope |
|---|---|---|---|
| Layer 1 | Kaspa | Envelope hash + Merkle root + anchor metadata (~500 bytes) | FORAY anchor service |
| Layer 2 | Operational store (PostgreSQL for Loomworks) | Full components + audit_data + attestations (~15-20 KB per transaction) | Yes — this is what we build |
| Layer 3 | Arweave + Glacier | Sealed encrypted archive for litigation/dispute (~5 KB compressed) | Out of scope (unless litigation-grade preservation joins roadmap) |
The blockchain is not a database. It's a cryptographic commitment store that enables off-chain logic to operate with on-chain guarantees. Loomworks holds the queryable substrate; the FORAY anchor service (separate component, not built into the engine) hands the envelope to Kaspa.
Two diagrams. The first shows the substrate as it is — the same diagram as v0.1, since the current state didn't change. The second is rewritten to reflect the four-table FORAY shape from the primer.
The recommended shape:
foray_transactions holds the envelope (one row per transaction). foray_components holds the four DAG kinds with typed reference columns. foray_attestations holds the cross-cutting fifth with polymorphic subject_refs. memory_events stays as event log, cleaned of the Phase 25 readiness wiring.settings table replacing credentials, engagement_api_keys, system_config, person_settings.The priorities themselves are mostly unchanged from v0.1. Priority 1 is rewritten to reflect the protocol-grounded target. The other five are unchanged.
What it is: Implement the four-table FORAY substrate from the primer. foray_transactions (envelope, one row per transaction). foray_components (one polymorphic table over ARR/ACC/ANT/ACT with typed reference columns and the dual foray_core / audit_data JSONB separation). foray_attestations (sibling table for the cross-cutting fifth with polymorphic subject_refs[] and validity-period semantics). Strip the Phase 25 readiness-wiring columns from memory_events so it returns to being a pure append-only event log.
Retire credit.foray_action_flows and audit.foray_events. The existing rows migrate into foray_components (or, for the parts that are events about state rather than state itself, into memory_events). The existing readiness-wiring columns on memory_events become derived state — readiness is computed from foray_components.*_refs[] and status rather than stored as columns on the event log.
Why first: Highest leverage piece of substrate work in the project. Every future audit-shaped feature (Phase 63's audit viewer is a current example, and there will be more) will reach for FORAY. Getting the substrate right once removes the parallel-solution risk permanently. Doing it wrong locks in three-tables-for-one-concept indefinitely.
Why now: The data permission is real. All existing FORAY data is essentially test data. Rebuild to the protocol now; the alternative is migration plus backward-compatibility cost when real data accumulates.
What unblocks: Phase 63's audit wiring and audit viewer execute against a coherent substrate. Future business transactions (when Loomworks starts modeling real ARR/ACC/ANT/ACT transactions, not just engine setting-changes) plug into a protocol-respecting substrate. The FORAY anchor service integration becomes a clean hand-off rather than a question of which table the envelope comes from.
The estimate increased from v0.1's "2-3 days" to "4-5 days" because the protocol-respecting target is a meaningfully larger piece of substrate than v0.1's single-table sketch — four tables to design, four schemas to enforce, plus the derived-readiness logic that replaces today's column-based readiness wiring.
What it is: A focused discipline pass. (a) Add DB CHECK constraints to every state column that has a Pydantic Literal so the database enforces the same vocabulary the application validates. (b) Eliminate ORM stub proliferation by establishing one authoritative ORM mapping per table; consumers import from it. (c) Reconcile the specific ORM/migration disagreement on memory_events.actor_kind (and any others discovered).
Unchanged from v0.1. The discipline gap is real and the fix is cheap.
What it is: The contributors table is a Phase 3 artifact that should have retired when persons + memberships landed in Phase 14. Migrate every code path using contributors to person-based equivalents; drop the table.
Unchanged from v0.1.
What it is: Collapse credentials, engagement_api_keys, system_config, person_settings into one settings table with polymorphic scope.
Unchanged from v0.1.
What it is: Drop zero-padding (E0001 → E1), reserve E1-E999 for foundational engagements, regular allocation starts at E1000. Voice recognizer regex update, dev data reset.
Unchanged from v0.1.
What it is: Shared base for the eight job tables — common status enum, shared base columns, per-agent specifics in subclass tables or discriminator columns.
Unchanged from v0.1.
Total estimated work across all six priorities: ~10-12 days of CC time. About one day longer than v0.1's estimate because of the FORAY scope increase. Still bounded; still substantial but tractable. Each piece closes cleanly in priority order.
The primer arrived with three explicit pointers the FORAY project asked me to surface in the substrate review. They're worth carrying into v0.2 verbatim because they name points of tension where the substrate review and the published protocol documentation might disagree.
The FORAY documentation at foray.dunin7.com describes the model as "4-Component Model (4A) + Attestations Extension", with Attestations as an optional fifth for third-party validation. The primer reframes Attestation as a native cross-cutting capability — the fifth structurally distinct construct.
Both framings produce the same JSON output. The difference is conceptual. The primer's framing is what drives the four-table substrate shape. A future FORAY spec update (perhaps v4.2) is expected to resolve this. Until then, if someone reads the public spec and sees "4A + Extension" while this review says "five constructs," the apparent disagreement is real but resolvable.
The instinct to put all five constructs in one polymorphic table is natural and would simplify the SQL surface. The primer's §4 is explicit about why this is wrong at the protocol level — typed DAG references and polymorphic subject references are structurally different, and collapsing them hides type information from the database.
If someone (later, or in a different conversation) pushes back on the four-table shape with "wouldn't one table be simpler," the answer is in the primer: it would be simpler at the cost of pushing all type-checking into application code where it's easier for the next person to forget. The two-state-table shape is the protocol-respecting minimum.
Layer 1 (Kaspa anchoring) is the FORAY anchor service's responsibility — a separate component that Loomworks hands envelopes off to. Layer 3 (sealed archive for litigation-grade preservation) is out of scope entirely unless that requirement joins the roadmap.
Loomworks is implementing Layer 2 — the queryable operational store. That's the right scope. If the roadmap later adds requirements that touch Layer 1 directly (Loomworks calling Kaspa) or Layer 3 (sealed archive), the substrate shape may need extension. But starting Layer 2-only is the protocol-respecting minimum.
The six priorities above are bounded. A few places where I'm choosing conservatism over ambition, with the reasoning surfaced. Unchanged from v0.1.
Some substrate reviews end with "rebuild from scratch." This one doesn't, because the core architectural decisions are right. Building a new substrate would be six months of work to arrive at roughly the same place architecturally, minus the migration history that documents how you got there. Bad trade. The right move is targeted refactor against a substrate whose foundations are already correct.
Things like "engagements.state" carrying lifecycle states while the Engagement MemoryObject's own .state field carries version-chain state — same word, different concepts — are real terminology drift. Mostly cosmetic; behaviors are correct. Worth doing in your v0.21 absorption work, not as a substrate priority.
The substrate review treats the engine as if it will be deployed eventually. Deployment is its own scoping conversation.
Realistic estimate, slightly larger than v0.1's because of the FORAY scope correction. FORAY rebuild is the biggest piece (~4-5 days). DB boundary tightening is one day. Contributor retirement is one-to-two days. KV unification is one-to-two. Identifier scheme is half a day. Job table base is one-to-two days, eventually.
I'd suggest treating each as its own phase. By the time real Operator data starts accumulating, the foundation is in the shape you want.
Three options for sequencing. Mostly unchanged from v0.1 but with the FORAY rebuild now understood as a larger piece than v0.1 framed it.
The editor work is small and independent; it can land in a day or two. After that, the substrate phases run in priority order with FORAY first.
The architecturally pure move. FORAY rebuild first; editor friction stays for now. Phase 63 reopens after the substrate work is solid.
Phase 63's editor improvements live entirely in record.dunin7.com with no engine dependency. The substrate work lives entirely in loomworks-engine. The two can proceed in parallel without conflict.
I lean Option C, with one caveat: the FORAY rebuild is now a larger piece than v0.1 framed (4-5 days vs 2-3). If parallel work splits your attention enough that the substrate work suffers, sequential is the better path.
The conversation that should happen now is which of A/B/C you prefer, and whether the four-table FORAY shape feels right to you as the protocol-grounded target. Once those settle, the next step is scoping the first piece. Phase 64 (FORAY rebuild) becomes a real piece of work with the primer as its protocol grounding and this review as its architectural orientation.
This review depends on two earlier documents in the substrate section. The substrate inventory v0.1 describes the current state of the engine substrate. The FORAY protocol primer v0.1 describes FORAY at the protocol level. v0.1 of this review (without the FORAY correction) also stays in the record as the prior position. Together the four documents form the substrate analysis arc for 2026-05-23 and 2026-05-24.