Upload Pathway — Ground Truth (v0_1)

Version. 0.1 Date. 2026-07-01 Type. Investigation / grounding. Read-only ground truth for scoping a Room-1 completion effort: file / folder / image intake into Memory. No fixes, no design decisions — ground truth only. Base. engine main @ 2abcfd1 (post-CR-2026-129, Room 1 closed). Supersedes/extends. Reconciles against [[loomworks-upload-pathway-investigation-v0_1]] (architectural, 2026-05-08), [[loomworks-upload-vision-conversion-bug-diagnosis-v0_2]] (2026-06-02), the Phase 59 CR (CR-2026-092 v0.4), and queued-directions §15.3.

Target end state (what this grounds against)

• Upload a file (any type), a folder, or an image → it becomes Memory.
• Images: the system INTERPRETS the image (describes its contents), and that description becomes a held assertion the Operator confirms (machine describes → proposes → human approves), flowing through the same capture→confirm→commit grammar as a spoken note.
• Non-image files/folders: the file is STORED and its content is interpreted into a recallable description/summary that also becomes Memory.

Headline

There are two image rails. The old one drafted a Memory assertion; the current one doesn't. That split — not a broken vision skill — explains both the Operator's "it worked before" recollection and the live "0%, no description" symptom.

| | Rail A — Phase 16 /contributions (older) | Rail B — Phase 58/59 /uploads (current chat/composer) | |---|---|---| | Endpoint | POST /engagements/{id}/contributions (contributions.py) | POST /engagements/{id}/uploads (uploads.py:441) | | Image vision | describe_image_skill (skills/image_description.py) | image_vision_analysis (uploads/skills/, claude-sonnet-4-6) | | Lands in Memory? | YES — creates a held Assertion + lineage (contributions.py:280) | NO — assertions_drafted=[] hardcoded (uploads.py:661) | | Storage | local disk | MinIO / S3 object store |

The frontend chat/composer upload moved onto Rail B. Vision still fires, but its output stops at event grain and never becomes an assertion. This is a rail switch that dropped the Memory-landing — not a regression inside the vision code. Git history confirms image_vision_analysis was born complete (9405cb7), refined (516a005), and fixed not disabled (bd05d5c); no commit ever stubbed it or removed a key. A live upload on 2026-06-02 shows the vision call actually firing.

1. What works

• Multi-file + folder upload (frontend). /chat's UploadAttachmentButton has both a multi-file and a webkitdirectory folder picker; the in-engagement composer's AttachButton is file-only by choice. Both → useEngagementUpload → postUpload → POST /engagements/{id}/uploads (multipart, field files). Caveat: folder structure is flattened to basenames — the client sends file.name, not webkitRelativePath (uploads.ts:38; deliberate per AttachButton.tsx:11-13).
• The engine /uploads endpoint is code-complete (Phase 59 landed; tag phase-59-upload-pathway-completion): batch list[UploadFile], 50 MB chunked-streaming cap → 413, rate-limited 10/min, detection registry, per-skill key resolver.
• Storage is real — MinIOObjectStore.put_blob (minio_backend.py, not a stub; auto-creates bucket). Default backend minio at localhost:9000 (config.py:58). Dev caveat: empty default creds and no in-repo MinIO service definition — a live MinIO must be running or put_blob → 503.
• Interpretation skills are real, complete code — image_vision_analysis (two-stage Claude vision, HEIC/AVIF→JPEG), image_ocr (Tesseract→vision fallback), image_metadata (Pillow), plus pdf/docx/text/audio extraction. Accepted types = detection-registry rules (extension + magic bytes): txt/md, pdf, docx, 7 audio formats, 9 image formats.
• The assertion/held-tray path to reuse is healthy — add_assertion → held → commit_assertion, the Companion's capture→confirm→commit grammar (_route_add_knowledge, router.py:255). Rail A already proves upload→assertion is possible (describe_image_skill → held assertion).

2. What's broken (the 0% surface — not what it looks like)

The "image_metadata 0% / image_ocr 0% / image_vision_analysis 0%, no description" picker is not skill confidence — skills have no confidence field (they emit reliability ∈ succeeded/unreliable/failed). Those percentages are the executor's chain-disambiguation heuristic (per_candidate_scores):

• All three image skills register the same image-family detection rule → an image yields 3 candidate chains (executor.py).
• With >1 candidate, the executor runs a keyword-mention purpose classifier over accompanying_message / purpose_declaration. With no message (or no keyword hit) it scores every candidate 0.0 and returns status="needs_clarification" before any skill runs (executor.py:211-219, 520-527).
• So 0% across all three = the disambiguation dead-ended; no skill executed → no description. The blank "I think this is —" is that needs_clarification surface.

Distinct from the key-gate: a missing Anthropic key would surface transformation_failed with an error message, not the 0% picker. Vision's key is DB-resolved via system_config.loomworks_llm_key / engagement key — not an env var (config.py has only openai_api_key).

Two independent problems: (a) multi-candidate image routing dead-ends at 0% before running anything; (b) even past that, vision needs a DB-stored loomworks_llm_key.

3. What's absent

• Upload → Memory assertion, on Rail B. Confirmed at every layer: assertions_drafted=[] hardcoded (uploads.py:661), zero add_assertion calls anywhere in uploads/, no downstream listener converting upload_event_received into an assertion. Uploaded content lives only as (a) an object-store blob and (b) an upload_event_received row in memory_events — unreachable by recall (recall_personal / ask_about_past_input read committed assertions only). The model field (upload_event.py:250-251) and executor docstrings (executor.py:476) describe drafting held assertions — documented as intent, never wired.
• The image→held-assertion→confirm loop (machine describes → proposes → human approves through capture→confirm→commit) — absent on Rail B.
• Non-image content-to-memory — extraction runs, but the summary/description likewise stops at event grain, never an assertion.
• True folder structure — flattened to basenames; directory hierarchy is not transmitted or stored.

4. Phase 59 reconciliation + prior findings

• Phase 59 (CR-2026-092 v0.4) landed — but was scoped engine-only and never included assertion-drafting. P59-D3 was amended (V9) to push narration/consumption to Phase 60; there is no sub-arc for drafting assertions from upload events. The gap is by omission of scope, not a failure.
• This exact gap is already an open, unsequenced finding: queued-directions v0_28 §15.3 — "Upload content does not reach assertion grain." Verbatim: assertions_drafted hardcoded [], no consumer produces assertions, "an Operator cannot affirm, correct, or compose an individual claim extracted from an uploaded file the way they can a claim contributed through conversation." Status: Open, not sequenced.
• Two prior investigations corroborate: [[loomworks-upload-pathway-investigation-v0_1]] (the intended discipline — uploads don't write Memory directly; the Companion drafts held assertions, the Operator commits) and [[loomworks-upload-vision-conversion-bug-diagnosis-v0_2]] (vision executes end-to-end when infra + key present; the earlier 500 was MinIO down, not a vision regression).

Salvageable vs net-new (for scoping)

• Reuse as-is: storage (MinIO/S3), all interpretation skills (vision/ocr/metadata/pdf/docx/text/audio), the add_assertion→held→commit grammar, and Rail A's proven upload→assertion pattern (image_description.py) as the reference.
• Fix: the multi-candidate image routing that dead-ends at 0% (one detection rule vs three competing skills); ensure loomworks_llm_key is populated for vision; the folder-structure flattening (if hierarchy matters).
• Net-new (the core of the target end state): wire Rail B's executor output → held assertion (assertions_drafted → add_assertion), so an interpreted image/file description flows into the capture→confirm→commit loop and becomes recallable Memory. This is the one substantive missing piece; everything upstream (storage, interpretation) already exists.

Key code references

• Endpoint: src/loomworks/api/routers/uploads.py:441 (operator_upload), registration :721-732, the assertions_drafted=[] gap :661.
• Executor: src/loomworks/uploads/executor.py (detection→narrow→skill→ExecutorResult; :118-121 "not responsible for writing to memory"; :211-219, 520-527 the needs_clarification 0% path).
• Skills: src/loomworks/uploads/skills/{image_vision_analysis,image_ocr,image_metadata,pdf_extraction,docx_extraction,text_extraction,audio_transcription}.py; base skills/base.py.
• Vision: src/loomworks/uploads/vision/claude_vision.py (run_vision, claude-sonnet-4-6). Credentials: uploads/skill_credentials.py:58 (loomworks_llm_key), cascade api_keys/resolve.py.
• Storage: storage/{factory,object_store,minio_backend,s3_backend}.py; config.py:58-63.
• Rail A: api/routers/contributions.py:280, skills/image_description.py.
• Recall (reads committed assertions only): orchestration/router.py:1110-1115 (ask_about_past_input), :1166-1171 (recall_personal).
• Frontend: UploadAttachmentButton.tsx, AttachButton.tsx, useEngagementUpload.ts, lib/api/uploads.ts.

Read-only. Nothing was modified. This is grounding for scoping — no fixes, no design decisions.