Version: v0.2
Date: 2026-08-16
Arc: Engagement-recall scoping arc (build list v1.02 queue head; Ruling 1)
Grounding: Step 0 findings loomworks-engagement-recall-step-0-findings-v0_1.md (branch engagement-recall-step-0, engine main cff1099); CR-2026-129 v0.2; queued directions §18.2 (v0.31); personal-memory posture note v0.1 §4.3; demonstrating-companion-answers standing note v0.1
Status: RULED. Operator accepted D1–D6 as leaned, 2026-08-16. CR drafting is unblocked.
Changes from v0.1: Ruling record added below; section 4 decisions marked ruled; section 6 updated. No design content changed.
This note scopes the semantic-recall foundation effort — the ruled queue head. The target is the engagement-scoped question-answering path: today, when an Operator asks the Companion about past notes, the system hands the responder the fifty most recent committed notes and nothing about the question selects anything. The effort replaces that recency window with retrieval by meaning: the question is embedded, the engagement's committed notes are searched by vector similarity, and the answer is composed server-side over exactly the retrieved set, with each item carrying its provenance and the answer stating what was searched. Five design leans are already ruled. The Step 0 inspection grounded them against the code and surfaced six decisions that need a ruling before the CR is drafted — the largest being that a transactional embed would put a network call inside the commit transaction, a property no commit currently has, so the embed hook should be asynchronous. When this lands, the demonstration rule's above-window bar lifts: live recall demonstration becomes permitted for engagements of any size (commons still excluded).
Operator ruling, 2026-08-16: all six decisions accepted as leaned — D1 async post-commit embed (lean 3 mechanics amended, correction preserved in D1's text); D2 OpenAI embeddings, env-level key; D3 per-version embedding rows; D4 loud fallback; D5 deployed-target pgvector verification as CR Step 0 task; D6 LLM-relevance-judge rerank deferred, filed as a queued direction. The five leans of the arc's opening ruling stand as grounded in section 2.
Facts below are from the findings note; file:line references are engine main cff1099.
The target path is settled. The foundation effort re-aims _route_ask_about_past_input (router.py:1646) — the engagement-scoped dispatcher. It refuses without an engagement and reads the turn's engagement's committed assertions. The personal sibling _route_recall_personal (router.py:1685) is untouched and inherits later, per posture note §4.3. The CR-129/posture-note wording discrepancy resolves in the posture note's favor; both texts are preserved side by side in the findings' corrections register, not smoothed.
The V5 failure mode holds at current main. _PAST_INPUT_LIMIT = 50 (router.py:152), pure recency ordering in list_assertions (assertions.py:807–875), the question never passed in, zero search-index artifacts in source or migration history, pg_extension holding only plpgsql. What post-dates V5: the boundary is now stated deterministically to model and reader (CR-2026-169 D-4), and per the CR-179 measurement no real engagement currently exceeds the window (largest: 13 committed). The effort is being built ahead of the scale that breaks the current path, not after.
The schema offers a clean seam. Assertions live as JSONB in an append-only event log (memory_events) plus a projector-maintained current-version projection (current_memory_objects, PK (engagement_id, object_id)). CR-2026-224's findings_view is the fresh in-repo precedent for a projector-maintained sibling table. Corrections are walkable: revision mints a new version, retraction mints a retracted version, no version is ever deleted; but the projection holds only highest versions — older versions live only in the event log, and was_revision_of points backward.
Infrastructure facts. pgvector is not installed and not available on DUNIN7-M4 (Homebrew PostgreSQL 16.12; no vector row in pg_available_extensions) — an install step for both playground_dev and playground_test is a real budgeted item, and the deployed target's posture is unverified from the engine repo. The openai SDK (>=2.36.0) is already a dependency for Whisper transcription, so an OpenAI-embeddings lane needs no new package. The Anthropic API offers no embeddings endpoint. No local-model precedent exists.
Provenance is stored in full and stripped before the responder. Every assertion version carries full provenance; the route reduces each item to item_id + display_number + content, and AnswerSource carries item_id + display_number only. Widening is a deserialization away — no new query, no storage change. The stated-scope sentence pattern already exists twice (the D-4 boundary line; the personal path's showing-count confession).
A third mechanism exists. find_files (CR-2026-137) and correction_check (CR-2026-145) are an established LLM-as-relevance-judge pattern — one LLM call over a numbered candidate list. This is the current in-repo answer to "find the relevant thing." Its role in the design is decision D6.
brew install pgvector + CREATE EXTENSION vector on dev and test, deployed target verification owed (D5).total_count, retrieval_limit, engagement id, per-item timestamps) are already cheap; widening the route's item shape and AnswerSource is the work. The server-composed pattern exists on the personal path (CR-129); this brings it to the engagement path over the retrieved set.WHERE engagement_id filter today and remains so under vector search. Commons excluded (demonstration rule); cross-engagement stays in Shaping; personal inherits later.The build, as scoped (subject to D1–D6):
assertion_embeddings), keyed (engagement_id, object_id, object_version), holding the vector column plus embedding-model identifier and embedded-at timestamp. FK discard policy declared per the CR-215 census fence. Migration creates the extension and the table; pgvector installed on dev and test databases as a budgeted pre-step.assertion_committed (and on revision/retraction versions), an embed job is dispatched asynchronously after the commit transaction, following the drift-agent dispatch + job-row precedent (assertions.py:370 onward). Commit latency and availability never depend on the embedding API. A backfill pass embeds existing committed assertions (85 in dev; trivially small)._route_ask_about_past_input embeds the incoming question, retrieves top-K by vector similarity over the engagement's committed assertions (highest versions for the primary answer set), walks corrections where a retrieved item is superseded (surfacing successor/predecessor honestly), and composes the readout server-side over exactly the retrieved set — each item carrying its provenance thread, the readout stating retrieval scope ("searched all N notes on this engagement by meaning; showing the K most relevant"). AnswerSource widened to carry the thread.D1 — Embed hook shape: asynchronous post-commit. This amends ruled lean 3's "embed at assertion-commit in the same flow." Correction preserved: the ruled form assumed the transactional projector seat (CR-224 precedent); the findings show every current projector is pure SQL, and a transactional embed would make commit latency and availability depend on an external network API — a property no commit currently has. The amended form: embed dispatched at commit time, asynchronously, drift-agent pattern, with the read side honest about any un-embedded backlog. Lean: async post-commit.
D2 — Embedding provider and key lane. OpenAI embeddings via the already-present SDK (no new dependency; model selection — e.g. text-embedding-3-small — settled in the CR). Key lane: env-level platform config, the Whisper precedent — recall is a platform capability, not a per-person billable service, at this stage; the Phase 64 metering convention gets the CR-223 before/after treatment in the CR. Rejected alternative, recorded: local embedding model — no precedent in the dependency posture, and it would add model-hosting weight to DUNIN7-M4 and the deployed target for no trust gain (vectors still live in the engine's Postgres either way). Lean: OpenAI embeddings, env-level key.
D3 — Per-version embedding rows. Key the sibling table by (engagement_id, object_id, object_version) so superseded versions are retrievable for the corrections walk without event-log scans at query time, and successor-finding gets an indexed path. Rejected alternative, recorded: highest-version-only rows — cheaper, but it makes lean 3's corrections surfacing an event-log walk on the hot path. Lean: per-version.
D4 — Loud fallback. Embedding-unavailable falls back to the recency window with the existing boundary line plus an explicit not-a-semantic-search statement. Rejected alternative, recorded: refuse the question outright — honest but hostile; the windowed answer with a true scope statement serves the Operator better. Lean: fallback with loud statement.
D5 — Deployed-target pgvector verification. The production Postgres posture is stated nowhere in the engine repo; whether the deployed target permits CREATE EXTENSION vector is unverified. Fold verification into the CR's Step 0 rather than a separate arc. Lean: CR Step 0 task.
D6 — LLM-as-relevance-judge role: deferred. Vector retrieval is the selection mechanism in the first build. The existing per-question LLM-judge pattern (find_files, correction_check) is recorded as a candidate rerank stage over the vector top-K, filed as a queued direction, not built now. Lean: defer.
One flag, not a decision: if any future backfill or background embed worker is ever pointed at the personal engagement (when personal inherits), that worker must be checked against B-109's fence before wiring. The engagement-scoped first build does not touch the fence's territory.
Personal recall (inherits later, §4.3). Universal commons (excluded by the demonstration rule regardless). Cross-engagement retrieval (Shaping's territory, non-transitive default). Message-stream conversation search (separate scoping note, distinct from recall). The §18.5 memory=recall seed reframe (awaiting separate Operator commission; strengthened, not performed, by this work). Any change to the personal path's _PERSONAL_RECALL_LIMIT wholesale posture.
D1–D6 are ruled (this version). The CR is drafted next (next chat, per fresh-chat-per-role): schema migration + pgvector install pre-step, async embed dispatch + backfill, retrieval + corrections walk + widened readout, loud fallback, demonstration-rule amendment, tests green at each step, Checkpoint A halt. This note and its ruled successor land in loomworks-record as canonical.
DUNIN7 — Done In Seven LLC Engagement recall foundation effort — scoping note — v0.2 — 2026-08-16 — RULED