Loomworks — Artifact ingestion: description and tags — Scoping note v0.3

0. What changed from v0.1 and v0.2

v0.2 → v0.3. v0.2 §5.8 and §7 treated batch ingest — throughput, retries, idempotency across a folder — as a build item inside the CR. The Operator proposed a folder-upload pre-flight: interrogate the folder, report its volume and cost, propose an upload strategy with a plain-terms explanation of every strategy including manual splitting, and let the Operator confirm or back out. That is Operator-authority applied to volume, and it turns the credit-spend confirmation from an inherited posture into a designed surface. v0.3 adds §5.10 Folder pre-flight, D10, two Step 0 items, and moves batch mechanics from §7 to "inside D10's CR". Nothing else changes.

v0.1 → v0.2.

v0.1 recommended (§5.3, D3) that tags be metadata on the description assertion, with a separate artifact–term relation deferred to the domain layer. The Operator asked where the repository was that lets artifacts be located and organized regardless of engagement and domain. It was not there. Metadata on an assertion is engagement-scoped by construction, and the buffer review (R4) and cross-engagement location both need a store that is not. The deferral was wrong.

v0.2 adds §5.9 The tag repository — a substrate-wide term registry plus an artifact–term relation — restates D3 as a schema-shape decision, adds D9 on normalization versus tree structure, and adds Step 0 items for existing vocabulary storage. Nothing else in v0.1 is changed. The prior position is named in §11.

1. What this note scopes

Two additions to artifact ingestion, and one retirement.

• A description assertion per ingested artifact, generated by the Companion, stored beside — not instead of — the extracted content.
• Tags applied to that description assertion, drawn from engagement and domain vocabularies, re-matched when a new focus appears.
• A tag repository — a substrate-wide term registry and an artifact–term relation — so that artifacts can be located and organized across engagements and domains, under grant.
• A folder pre-flight — before a folder is transformed, the Companion reports its volume and cost, proposes an upload strategy, and the Operator confirms or backs out.
• Retirement of the scale guard in summarization.py, whose only job the description assertion now does properly.

Plus the consequence for the buffer engagement: seed-triggered review matches descriptions and tags, never raw extraction.

2. Where it came from

The thread ran in one session on 2026-08-25, starting from an external post on "compiled second brains" (the Karpathy LLM Wiki pattern). The trajectory, with corrections named:

1. The post's compiler model was read against the four rooms. Its divergences — overwrite instead of corrections-preserved, autonomous filing instead of Operator-commits, no provenance, one reader, no engagement/general boundary — are the load-bearing differences.
2. Operator position: the engagement focuses one human; a domain focuses many. Any approach that begins with the artifacts misses that focus. Archiving with tags is a system hoping for relevance. (Prior Claude framing — "the human brings focus to the engagement" — was reversed. The engagement and domain focus the human, not the other way round.)
3. Downsides named honestly: focus at entry loses the artifact whose relevance arrives later; placing is per-artifact work. Both are the same seam, and the answer is a buffer at the edge that is refused the name "memory".
4. Operator proposal: a catch-all engagement the Companion reviews from time to time. This closes quick-capture investigation open question 5 (§3.6 and §8 of loomworks-quick-capture-engagement-investigation-v0_1.md).
5. Operator proposal: at ingest, a plain-English description and a normalized tag set. Reconciled with point 2: a tag invented from the artifact is a description of the artifact (the hoping kind); a tag drawn from a focus's vocabulary is a join key. The normalized set is the union of declared vocabularies, not a standalone taxonomy.
6. Step 0 inspection ordered: does ingest produce a description today? Answer: no. The Operator's observation of the Companion describing an artifact at upload was the card, not the record — a hope for persistence, not persistence.
7. Correction preserved: the description was first ruled as a new field on the source record. Reversed the same day: a field cannot hold a correction. A description is an assertion. Ruled by the Operator: "assertion, not field."
8. Scope correction preserved (continued at 9): the Legacy Software engagement (retiring experts, code files, interpretation) was the example that made the correction visible. The Operator directed that the method is for all artifact ingestion across engagement types. Every item was re-checked for where the example had leaked in; the tag-vocabulary sentence was generalized, and the scale-guard finding was widened from "not on code" to "retire".
9. Correction preserved (v0.2): v0.1 filed tags as assertion metadata and deferred a cross-cutting relation. The Operator asked where the repository was that locates artifacts regardless of engagement and domain. Reversed: the term registry and the artifact–term relation are in scope from the start. The Operator then asked whether the repository should normalize as far as possible or use a tree. Ruled for recommendation: normalize the words; do not build one tree; let each vocabulary draw its own structure over shared terms.
10. Correction preserved (v0.3): batch-ingest mechanics were filed as a build item, not a decision. The Operator asked for an explanation and then proposed the folder pre-flight. Reversed: volume is an Operator decision at the moment of upload, made on a full report with the strategies explained. Mechanics (retries, idempotency) stay inside the CR; the choice of strategy does not.

3. Step 0 finding

From the inspection report (engine 2f41aa7, paths under src/loomworks/):

• The ingest route is operator_upload (api/routers/uploads.py:668). It writes three things: the raw bytes to object storage; one held assertion per file, grammar_element="definition", content = extracted text or its summary (uploads.py:965–972); and the source record UploadEventReceived (uploads/upload_event.py:182), which keeps the verbatim extraction in transformed_content_inline.
• No description is stored anywhere. No column resembling description, summary, abstract, caption, gist, narrative, tags, keywords, labels, or topics exists in the live schema apart from one unrelated scheduler field. Confirmed against information_schema.columns, not only source greps.
• Two mechanisms generate prose, and both substitute it for content rather than storing it beside content: the scale guard (uploads/summarization.py, threshold 2,000 chars at line 32; its own docstring: "a scale guard, not a general summarization feature") and the image vision skill (uploads/skills/image_vision_analysis.py:60–65, which genuinely describes an image, but as the extraction output).
• No tags of any kind. Ingest writes six assertion-metadata keys, none topical. The only classification is purpose classification, which selects an extraction chain.
• The Companion turn after upload is persisted but by post-CR-2026-141 decision names files rather than describing them (uploads.py:604–612). The card payload content_summary is a truncation of raw extraction for rehydration (uploads.py:397–411), not a generated description.
• Every engine upload route requires an engagement (engagement_id: uuid.UUID = Path(...) at uploads.py:670). An un-focused upload cannot arise at the engine. Whether the Operator Layer queues or redirects an upload with no engagement selected was not inspected.

4. Rulings already landed (2026-08-25)

These were ruled in conversation before this note and are recorded here as its foundation.

• R1 — Buffer engagement. A dedicated holding-area engagement is the destination for anything that arrives with no engagement. Engagement in mechanism, buffer in role, thin seed by construction. Not memory; not Manifested. Closes quick-capture open question 5.
• R2 — Description is an assertion, not a field. Prior position (field on the source record) named and reversed. A field overwrites; an assertion holds a correction.
• R3 — Tags come from vocabularies. Engagement vocabulary (Phase 40 Operator vocabulary schemas), domain-adopted terms, and review-proposed candidates. A candidate is not a key until a focus adopts it. No standalone taxonomy.
• R4 — Review is triggered by new focus. Seed creation or domain vocabulary change re-matches buffer descriptions and tags against the new focus. Periodic sweep permitted, secondary.
• R5 — Placement is Companion-proposes, Operator-commits. A moved assertion keeps its trail.
• R6 — General scope. The method applies to every ingested artifact across engagement types.

5. The proposed shape

5.1 The description assertion

At ingest, alongside the existing extraction assertion, the Companion authors a second held assertion:

• Grammar element: description (new; see D1 on how grammar elements are declared).
• Content: one short plain-English paragraph answering "what is this artifact?" — subject, kind, and one or two things a reader would want to know before opening it. Not a summary of the content; a description of the thing.
• Provenance, outward: the source record (upload_event_id), the original filename, the extraction method — the same keys the extraction assertion carries today.
• Provenance, inward: the extraction assertion it was inferred from. The description is an inference over extracted content, and the record should say so.
• Origin: Companion. AI origin marked in the record, invisible as clerk on the surface — the seed's posture.
• Lifecycle: held, then committed by the Operator, like any other contribution. At alpha this is the standard held-plus-commit path; auto-commit for trusted categories is a later delegation-contract question (Phase 45 shape), not this note's.

For images, the vision skill's prose stays where it is as the extraction output. The description assertion is generated from it like any other extraction. Two records, two roles, even when the prose is similar.

5.2 Human correction

When a person — the Operator, a contributor, a retiring expert — corrects the description, that is a new assertion against the same source with the person as origin. It supersedes the Companion's for recall. The Companion's stays. Trust axis is tie-to-source: a person's description of a document they know outranks an inference over its text.

This is the mechanism that carries the value in expert-heavy engagements: the correction is the knowledge the engagement exists to capture, and it lands as a first-class assertion with its own provenance rather than as an edit to a field.

5.3 Tags

Tags are applied to the description assertion and recorded in the tag repository (§5.9) as artifact–term relations. v0.1 said "metadata on the description assertion, not a separate table"; reversed in v0.2 — see §0.

• Sources, in order of authority: the engagement's declared vocabulary (Phase 40); terms the engagement's domain has adopted; candidates proposed by the Companion from the description. Which source dominates varies by engagement type; the mechanism does not.
• Description first, tags second. The Companion writes the description, then proposes tags by matching it against current vocabularies. Tags are never invented first and the description back-filled.
• Candidates are not keys. A proposed tag that matches no vocabulary is stored as a candidate. It becomes a key when a focus adopts it — the elevation pathway working in reverse, a word rising from the buffer into a vocabulary.
• Correction preserved. A person changing the tags is a new version of the description assertion. No smoothing.
• Re-tag on new focus. When a seed is created or a domain vocabulary changes, descriptions are re-matched against the new words and new tag proposals are surfaced. Tags assigned at ingest are only as good as the focuses that existed at ingest.

5.4 Per-artifact only

Ingest produces one description and one tag set per artifact. It does not attempt a module overview, a subsystem map, a project picture, or any cross-artifact synthesis. Those are Manifestation — Memory organized at a moment in time. Diagrammatic overviews are render-types — Rendering Mode A over Manifested descriptions. Keeping this boundary keeps the upgrade small and the rooms honest.

5.5 The card is backed by the assertion

UploadResultCard today shows content_summary, a truncation. After the upgrade it shows the description assertion — the same surface, now backed by something that persists and can be corrected. This does not reverse CR-2026-141: the Companion's spoken narration still names files; the description is written to the record, not spoken on the turn.

5.6 The buffer becomes reviewable

Seed-triggered review (R4) matches the new seed's vocabulary and intent against buffer descriptions and tags. Proposals surface at the Operator's next chat open in the form quick-capture already landed for engagement creation: "these three look like FarmGuard." The Operator commits placement. Raw extraction is never what review reads.

5.7 Retire the scale guard

The scale guard's job was to keep a held assertion from carrying a 40-page verbatim. Once a description assertion exists, that job is done properly: the description carries the prose; the extraction assertion can carry the content or a pointer to the verbatim on the source record. The guard has no remaining reason, and its current behaviour is destructive for any content whose verbatim is the value — code, contracts, transcripts, datasets, reports. Retire it for all content types; do not scope it by type. (See D2 for the extraction-assertion content question that retirement opens.)

5.8 Cost, named once

One model call per artifact for the description, and one match pass for tags. At scale — a folder upload of thousands of files — the cost is reported to the Operator before anything runs (§5.10), and the mechanics of running it (throughput, retries, idempotency on content_hash and upload_event_id) are build items inside the CR. The credit system's model identity applies: the lean is the smallest capable model for description at scale and a larger one for any review pass. The choice is D4.

5.9 The tag repository

Two structures, substrate-wide, outside any single engagement's schema.

The term registry. One normalized set of terms. Each term carries a canonical form, its surface forms (aliases), who first proposed it, when, and which engagements and domains have adopted it as a key. A term no focus has adopted is a candidate. Normalization is the registry's job: case, whitespace, punctuation, singular form, alias merge. The Companion proposes a merge; the Operator commits it — merging terms is a state transition on something the Operator owns. A merged alias keeps its history; the record shows the two were once separate.

The artifact–term relation. One row per application: the artifact (source record and description assertion), the term, who applied it (Companion or person), its status (proposed or committed), and the engagement it was applied in. This is the table v0.1 deferred.

No global tree. The registry is flat. Structure — deductions is narrower than payroll — is declared by a vocabulary owner (an engagement or a domain) and is scoped to that vocabulary. Two focuses may draw different trees over the same words without conflict. The registry never infers structure; a suggested relation is a candidate proposed to a vocabulary owner and committed or not. The result is a shared flat term set with many local trees over it — the thesaurus model, not the taxonomy model. One global tree would force placement decisions nobody with authority made, and would rot the way the compiler post says personal wikis rot.

Location runs through grants. The registry is global. The relation is queryable across engagements, but only within what the asker holds a grant to. A tag can tell an asker that an artifact exists in a scope they cannot read; it cannot show them the artifact. That is the Memory access axis doing its job, and it is the line between locating regardless of engagement and reading regardless of engagement.

What it enables. Seed-triggered buffer review (R4) matches a new seed against the whole relation, not one engagement's metadata. The Companion's cross-engagement intelligence (Source C in the expertise note) gains the substrate it has lacked. Domain adoption of a term is a registry write, not a new mechanism.

What it is not. Not a taxonomy. Not a relevance engine. It records words, who claimed them, and which artifacts carry them. Relevance is still a relation to a focus; the repository only makes that relation findable across scopes.

5.10 Folder pre-flight

When a folder is offered for upload, nothing is transformed until the Operator has seen a report and chosen a strategy.

The report. File count; total size; breakdown by type; how many descriptions will be generated; credit cost at the model chosen under D4; rough elapsed time; and how many held assertions the Operator will have to review afterwards. The last figure is reported with the same weight as the credit figure — review burden is the cost that arrives later.

The strategies, each explained in plain terms on the surface.

• Full ingest now. Every file extracted and described, in the background, with progress visible. Suited to folders under the volume threshold.
• Extract now, describe later. Files are extracted and their source records written; descriptions run as a later pass the Operator starts. Suited when the content is needed immediately and the descriptions can wait.
• Store only, promote later. Bytes are stored with no processing (the existing store-only route); the Operator promotes selected files to full ingest later (D6). Suited when only part of the folder matters.
• Split the folder. The Operator divides the folder into smaller ones and uploads them in sequence. The explanation says why this is often the best choice: each batch stays reviewable; credit spend is visible per step; the Operator can stop after the first batch if the descriptions are not what they wanted; a mistake in one batch does not touch the others.

Confirm or back out. Confirming runs the chosen strategy. Backing out writes nothing to Memory — no assertion, no source record. An event-log entry recording that a pre-flight was declined is acceptable; it is not an artifact state.

The threshold. A default volume threshold, held as configuration, above which the pre-flight recommends splitting. The Operator may override per upload. This follows the seed's own posture for volume: the open sign-up path carries a limit; authorized lift raises it.

Where the explanation lives. The strategy explanations are Operator-facing content and follow plain-terms discipline. They belong in a voice loader alongside the existing credit and quick-capture voice patterns, not in prompt text.

What the pre-flight is not. It is not a gate that prevents upload. The Operator can always confirm. It surfaces and signals; the Operator decides.

6. Decisions for Operator ruling

Each carries a recommendation. A single word confirms; a sentence changes it.

• D1 — Grammar element declaration. How is description added as a grammar element? Step 0 must confirm whether grammar_element is a registered enum (Phase 38 specification grammar declaration) or a free string. Recommendation: register it, wherever the registry is; a free-string addition would be a source-contract hole.
• D2 — Extraction assertion content after scale-guard retirement. Verbatim content in the assertion, or a pointer to transformed_content_inline on the source record? Recommendation: pointer, with a short lead excerpt, so held-assertion size stays bounded without rewriting content.
• D3 — Tag repository schema shape. v0.1 asked whether to build a relation table at all and recommended not yet; reversed in v0.2. The question now: does the registry and relation live in its own schema alongside the engagement schemas (the infrastructure-engagement pattern the credit system established), or as substrate tables outside any engagement? Recommendation: own schema, infrastructure-engagement pattern — the credit system already proved that shape and it gives the registry a Memory, a Companion, and FORAY emission for free.
• D4 — Model selection for description at scale. Recommendation: Haiku-class for ingest description; Sonnet-class for seed-triggered review proposals. Recorded as credit-asset identity, not as a prompt constant.
• D5 — Review triggers. Seed creation and domain vocabulary change as primary; periodic sweep secondary. Recommendation: build the seed-creation trigger first; sweep is a config flag off at alpha.
• D6 — Store-only route. Does store-only (no detection, no transformation) get a description? Recommendation: no. Store-only is a deliberate opt-out of ingest processing; a description would contradict it. The Operator can promote a store-only file to full ingest later if that path exists (Step 0 to confirm).
• D7 — Candidate-tag adoption path. Where does a candidate become a key? Recommendation: the engagement vocabulary editor for engagement adoption; the domain layer, when built, for domain adoption. Not a new surface.
• D8 — Operator Layer un-selected upload. Not inspected. Recommendation: Step 0 read of the Operator Layer before CR drafting, to establish whether an un-focused upload can reach the buffer at all today or only via quick-capture.
• D9 — Normalization versus tree. Recommendation: flat normalized registry with alias merge; broader/narrower relations declared per vocabulary and scoped to it; no global tree; the registry never infers structure. Confirm or amend.
• D10 — Folder pre-flight. Operator-proposed; recorded as the shape in §5.10. Open sub-points for ruling: (a) default volume threshold — recommendation: 200 files, configuration, overridable per upload; (b) whether backing out leaves an event-log entry — recommendation: yes, event only; (c) whether single-file uploads get a lighter pre-flight (cost line only) or none — recommendation: none at alpha; the existing single-file flow is unchanged.

7. Out of scope

• Manifestation-room synthesis across artifacts (module overviews, system maps). B-5 territory.
• Diagrammatic renders. Render-type territory.
• Domain-layer mechanics — how a domain declares vocabulary, who may adopt a term on its behalf. The elevation pathway investigation holds this. The registry write that records an adoption is in scope; the authority model for who performs it is not.
• Batch-ingest mechanics — retries, idempotency on content_hash and upload_event_id, concurrency against the API rate limit. Build items inside D10's CR. v0.2 also placed the choice of strategy and the spend confirmation here; v0.3 moves those to §5.10 and D10.
• Auto-commit of descriptions for trusted categories. Phase 45 delegation-contract shape, later.

8. Seed and specification check

Checked against loomworks-candidate-seed-v0_14.md and settled methodology.

• Memory accumulates with provenance; corrections preserved, not smoothed. The description-as-assertion ruling is the direct expression of this. A field would have violated it.
• Operator-authority over state transitions. Held-plus-commit for descriptions; Companion-proposes, Operator-commits for buffer placement. No automatic moves.
• AI origin marked, invisible as clerk. Companion-authored descriptions carry origin in the record; the surface shows the description, not a badge.
• Only show what is available. Tag candidates that no focus has adopted are stored, not surfaced as controls.
• Provenance threads — flag. The seed v0.14 drafter's notes record that the provenance-threads candidate is queued against the seed, not absorbed. This note uses "outward" and "inward" thread language as posture. The mechanism available today is assertion metadata plus the source-record linkage. The CR must build on what exists and name the thread vocabulary as the posture it is implementing, not as a substrate feature already present.
• Memory access axis and grants. Cross-engagement location through the tag repository is bounded by grant. The repository must not become a side channel that reveals content across scopes; it reveals existence of a relation, and only where the asker holds a grant. This is a constraint on CR-B's query surface.
• Plain-terms discipline. "Description" and "tags" are plain terms and may appear on Operator-facing surfaces as-is. "Grammar element" stays in code-translation contexts.

No conflict with the seed found. One dependency on a queued candidate, flagged above.

9. Step 0 items before CR drafting

Carry-forwards from the inspection and from §6. Read-only, against the live engine and Operator Layer.

1. How grammar_element is declared and validated (D1).
2. Whether the assertion model can hold a pointer to the source record's verbatim without schema change (D2).
3. The Phase 40 vocabulary schema shape — where declared terms live, how they are read (D3, D7).
4. Whether a store-only file can be promoted to full ingest (D6).
5. Operator Layer behaviour when no engagement is selected at upload (D8).
6. Whether personal_engagement_id on host_account is the right anchor for the buffer engagement or the buffer is a distinct engagement per account (R1 mechanism).
7. Batch upload path (folder_manifest, parent_upload_event_id) — where a per-file description call fits without changing the folder contract.
8. Whether any term, tag, or vocabulary table exists today in any schema — the inspection's live-schema query found no column named tags/keywords/labels/topics, but Phase 40 vocabulary schemas are stored somewhere; find where, and whether that storage can host the registry or must be superseded by it.
9. The infrastructure-engagement pattern as built for the credit system — what a new infrastructure engagement needs (schema, Memory, Companion, FORAY seam) — as the candidate shape for the tag repository (D3).
10. Whether the folder manifest is available before transformation begins — the inspection saw folder_manifest on the source record, but not where it is first read — so the pre-flight can count without processing.
11. Whether any pre-flight or confirmation seam exists in the Operator Layer folder-upload flow today, and how the store-only route is reached from it.

10. What happens next

1. Operator rules D1–D10.
2. Step 0 inspection brief issued to Claude Code covering §9.
3. CR drafting. Likely three CRs: CR-A description assertion, grammar element, card rebinding, scale-guard retirement; CR-B tag repository (term registry, artifact–term relation, alias merge), vocabulary matching, buffer review trigger, grant-bounded cross-engagement query. CR-B depends on CR-A's assertion existing. CR-B may itself split if the repository as an infrastructure engagement (D3) is large enough to warrant its own CR before matching and review are built on it. CR-C folder pre-flight (§5.10, D10), which depends on CR-A for the description count and cost figures and can otherwise proceed in parallel with CR-B.
4. The comparison note that started this thread (loomworks-llm-wiki-comparison-investigation-v0_1) is a separate document and will cite this note.

11. Trajectory notes for the Discovery record

Positions taken and set aside, in order:

• Set aside: the human brings focus to engagements and domains. Replaced by: engagements and domains focus the human.
• Set aside: archiving with tags, as a system hoping for relevance. Kept in restricted form: tags as join keys drawn from focus vocabularies; tags invented from artifacts remain the hoping kind and are held as candidates only.
• Set aside: description as a field on the source record. Replaced by: description as an assertion. Ruled "assertion, not field."
• Set aside: scale-guard retirement scoped to code. Replaced by: retirement for all content types, because the guard's reason disappears once a description exists.
• Set aside: Legacy Software as the frame of the upgrade. Kept as: the example that exposed the correction; the method is general.
• Set aside (v0.1): tags as metadata on the description assertion, relation table deferred. Replaced by (v0.2): a substrate-wide term registry and artifact–term relation from the start, because location across engagements and the buffer review both need it.
• Set aside (v0.2): a global tag tree. Replaced by: a flat normalized registry with vocabulary-owned local structure — the thesaurus model.
• Set aside (v0.2): batch-ingest as a build item only. Replaced by (v0.3): the folder pre-flight — volume, cost, and review burden reported; strategies explained; Operator confirms or backs out. Mechanics stay inside the CR.
• Crystallized as new: recall is a function of the store plus the focus the asker stands in. A general-purpose second brain has a ceiling because it has no focus to recall from. This tightens the Memory=recall reframe and belongs with it when that reframe is drafted into the seed.