The Loomworks Substrate

0.What is Loomworks?

Loomworks is a knowledge-work engine. It hosts long-lived engagements — persistent spaces where people and AI agents accumulate, refine, and produce structured knowledge together. Every contribution is preserved; nothing is erased.

An engagement starts with a seed — a founding document declaring what the work is, who consumes it, what voice it takes, and what success looks like. Once the seed is committed, people and agents add assertions (statements with provenance) into the engagement's Memory. The Memory grows; it never shrinks. Corrections supersede prior assertions but don't replace them.

From an engagement's Memory the engine produces three downstream artifacts in a pipeline:

• Manifestation — a snapshot bookmark of Memory at a particular moment.
• Shaping — selected, recast material prepared for a specific consumer.
• Rendering — the final-form artifact (a document, specification, storybook, reference).

People interact with the engine through a per-Operator Companion — an AI conversational partner that can converse, classify intent, propose actions, surface notifications, and write to a private personal memory. The Companion runs on credits; consumption is metered per token through a small accounting substrate.

1.Who's who — the identity layer

Loomworks tracks persons (global identity, one row per natural human) joined to engagements through memberships, with stackable designations declaring what each person does in that engagement.

WebAuthn credentials, recovery codes, and TOTP secrets all live on the person; the substrate supports the full passkey-and-recovery flow.

2.The engagement spine

The engagements table is the root row for every space in the system. It started with four columns in Phase 1; it now carries ten, accumulated through ten migrations.

3.Memory: the event log and its projections

Everything an engagement "remembers" is a row in memory_events — the canonical append-only log. Each row is one version of one object (an assertion, a relationship, a shape event, a render event, a manifestation, a composition, a consideration, a cadence firing, a key-provided record, …).

Sixteen-plus object types are registered in the type registry. They all write to the same log. They are projected into:

• current_memory_objects — generic current-state view of every object.
• Five typed views — shape_events_view, render_events_view, manifestation_view, external_production_records_view, render_compositions_view. Each unpacks JSONB payload into scalar columns to make queries fast.

The projections are all rebuildable from memory_events. The event log is the source of truth; views are derived.

4.FORAY — three siblings, one grammar

FORAY is the substrate's record-of-action concept. It currently has three concrete surfaces. They share a grammar but not a shape — and the Operator has flagged this as architecturally worth examining.

What's shared across the three

• shared A transaction id (called transaction_id, tx_id, or foray_tx_ref — three names, two types).
• shared An actor (encoded three different ways: polymorphic VARCHAR, UUID-only, or UUID + kind discriminator).
• shared A payload JSONB (called metadata or payload).
• shared A timestamp.

What isn't shared

• divergent Row shape (value vs narrative vs object-version).
• divergent Trigger discipline (balance-update on credit; none on the others).
• divergent Read paths (credit has many; audit has none yet; memory_events has none).
• divergent Schema placement (credit / audit / public).
• divergent Discriminator (asset_id / event_type / event_kind — three taxonomies, all free-text).

5.Credit and accounting

The credit substrate landed across three migrations covering Phases 47, 48, and 49.

Plus eleven lifecycle columns added to persons across two phases (tier, account status, expiration, referral, exhaustion preferences, three near-exhaustion timestamps, previous responder model).

6.Conversation and the Companion

The Companion is a per-person AI conversational layer. Every turn is logged in conversation_turns:

• Role — operator or companion.
• Content — the turn text.
• Classified intent — only on companion turns. Roughly 18 intent labels including add_knowledge, remember_about_me, create_engagement_active, save_filter, tune_setting, general_conversation.
• Input mode — text or voice. The substrate distinguishes voice-originated turns at the database level.
• Completeness-check prefix — recursion guard for voice listening's silence-submit feature.

Companion notifications (companion_notifications) are proactive messages: informational alerts and approval cards live in the same table, distinguished by whether approval_status is non-null. Lifecycle: pending → delivered → seen → dismissed.

7.Workspaces, tags, saved filters

The Operator can organise engagements with three primitives:

• Workspaces — named groupings, one Operator owns them.
• Tags — free-form strings applied to engagements. No taxonomy table, no case-folding, no length cap.
• Saved filters — named predicates of the form {"all_of": [<predicate>, …]}. Three system-defined filters seed for every new person: Everywhere, Recent, Active.

8.Operational job tables

Eight tables across Phases 4-10 track agent dispatch and background work. They share a broad shape but no shared base — each was introduced when its agent landed.

Four different status vocabularies for what is effectively the same lifecycle.

9.Settings, secrets, side-tables

The substrate has four key-value stores, each introduced at a different phase for a different actor scope:

They are not unified. Each uses its own scope encoding.

Other side-tables: shape_event_title (Operator label per shape), render_specialist_binding (durable agent-registry entry, the only persisted agent registration), uploaded_files (binary file metadata; bytes on disk), email_send_attempts (per-send observability), migration_events (Phase 14 identity-migration audit).

10.Bootstrap-seeded engagements

Three engagements have deterministic, fixed UUIDs and ship with the substrate:

• 00000000-…-000000000001 — administrative engagement. Seeds live here by convention.
• 00000000-…-000000000002 — Loomworks commons. The universal-commons engagement every signed-up person joins. Visibility automatic.
• 00000000-…-0000000000e2 — E2E test sandbox. Hosts Playwright mutation traffic so live engagements stay clean.

Plus a per-person Personal engagement created at signup — invisible, Memory-only, visibility personal, hosts the Companion's per-person knowledge.

11.How the substrate has evolved

70 migrations across roughly six weeks of development time (April 13 – May 20, 2026). The migration history itself has shape:

• Nine empty-DDL migrations (0006, 0010, 0017, 0019, 0020, 0023, 0024, 0025, 0027) exist as Alembic lineage records — they document a Pydantic Literal extension that didn't require schema change. The migration chain is treated as searchable provenance.
• Eleven heavy data migrations seed founding content into the substrate (the Loomworks induction, the five founding assertions, the personal-engagement induction for existing persons, the display-number backfill, etc.). They duplicate slices of append_event inline so they stay self-contained.
• Each phase generally bundles its schema changes into one migration, but Phase 16 split into four — a pattern that recurs when an addendum lands after the main migration.

12.Cross-cutting findings

These are places where the structure shows that it grew rather than was designed up-front. They are observations, not judgements.

Engagements ORM stubs proliferate

At least six different layers each declare their own minimal _EngagementRow mapping. Each declares a subset of the engagements row's ten columns. They are silently drifting from the actual table shape; nothing enforces consistency.

Engagement state is dual-specified and disagrees

The Pydantic Engagement.state Literal admits ("active", "suspended", "archived"); the database has no CHECK constraint; raw SQL inserts 'candidate'. Neither enumeration fully describes the in-use value set.

FORAY readiness columns are written-but-not-read

Three of four Phase-25 columns (content_hash, foray_tx_ref, the _foray payload sub-block) have no consumers. Only attestation is fully wired.

Four KV stores, four scope conventions

credentials (scope-keyed), engagement_api_keys (per-engagement), system_config (system), person_settings (per-person). Not unified.

Polymorphism by VARCHAR vs UUID typing

The substrate encodes "who did this" three different ways across its three audit-shaped surfaces: polymorphic VARCHAR (credit), UUID-only NOT NULL (audit), UUID + kind discriminator (memory_events).

State enumerations share verbs but not vocabularies

At least seven state columns across MemoryObject types and projection views use overlapping verbs (active / retired / confirmed / superseded / current / produced / held / committed / …). "Current" on Manifestation means the same thing as "confirmed" on ShapeEvent, under different names.

Companion notifications carries two row shapes

Informational notifications (Phase 44) and approval cards (Phase 45) share one table with six nullable-by-shape columns, discriminated on whether approval_status is non-null.

Five tables with audit-log shape

The substrate has five append-only event-log-shaped tables (memory_events, migration_events, credit.foray_action_flows, audit.foray_events, plus per-event tables like email_send_attempts). Five tables, five concrete shapes, one recurring grammar.

Display identifier reshape is queued

Current format (E#### zero-padded) was introduced recently; the Operator wants to move to unbounded form. The underlying sequence is already unbounded — only the format expression and assignment policy need to change.

Summary in three sentences

Loomworks is a knowledge-work engine built on a single append-only event log with typed projections — an architecture that honours its core promise that knowledge accumulates and never disappears. After 70 migrations across roughly six weeks of phased development, the substrate covers identity, engagement lifecycle, memory and projections, credit and accounting, conversation and the Companion, settings, audit, grouping, and operational bookkeeping. The picture is coherent at the level of grammar; at the level of shapes it shows the seams of incremental growth — the FORAY substrate splits into three siblings, four key-value stores exist side by side, status enumerations repeat without sharing, and several columns are wired ahead of integrations that haven't landed.