DUNIN7 · LOOMWORKS · RECORD
record.dunin7.com
Record/Session Handoffs/loomworks-session-handoff-2026-06-02-v0_1.md
Status Current
Path session-handoffs/loomworks-session-handoff-2026-06-02-v0_1.md
←Back

Session handoff — upload-pathway threads (v0.1)

Version: 0.1 Date: 2026-06-02 Audience: A fresh Claude chat with no access to this environment — everything needed is inlined below. Origin: A cleanup session that began verifying CR-2026-097 and branched into an upload-500 investigation. Four threads were left open. None is merged or pushed.

0. Environment facts (for whoever picks this up)

Thread 1 — Part-1 fix: image_vision_analysis converts before the vision call

Branch: fix-upload-vision-conversion-missing, single commit bd05d5c, off main (813f13f). State: NOT pushed (no upstream configured), NOT merged into main. Working tree currently on this branch. Tests green (the new regression test + the Phase 58/59/60 upload-vision suite: 70 passed, zero regressions).

Why: src/loomworks/uploads/skills/image_vision_analysis.py::transform claimed (docstring only) to convert HEIC/HEIF/AVIF/TIFF to JPEG before the Vision call, but never called the converter. For those formats detect_vision_media_type returns None, so media_type fell through to the default "image/jpeg" and the raw, unconverted bytes were sent to Vision mislabeled as JPEG. (NB: this is a real latent bug, but it is not the 500 — see Thread 2. Its true symptom is a transformation_failed/HTTP-200, because execute_upload catches TransformationError at executor.py:567 and returns a result rather than raising.)

What changed (diff main..bd05d5c, 3 files, +188/−20): 


--- a/src/loomworks/files/conversion.py
+++ b/src/loomworks/files/conversion.py
@@ def needs_jpeg_conversion(file_path: Path) -> bool: return ... @@
-def convert_to_jpeg(file_path: Path) -> bytes:
-    """Decode the file with Pillow and return JPEG bytes. ..."""
-    try: from pillow_heif import register_heif_opener
-    ... register_heif_opener(); from PIL import Image ...
-    try:
-        with Image.open(file_path) as img:
-            ... img.save(buf, format="JPEG", quality=90); return buf.getvalue()
-    except ImageConversionError: raise
-    except Exception as exc:
-        raise ImageConversionError(f"Could not convert {file_path.name} to JPEG: {exc}") from exc
+def _prepare_pillow():
+    """Register the HEIF opener and return the Pillow Image module. ..."""
+    try: from pillow_heif import register_heif_opener
+    ... register_heif_opener(); from PIL import Image ...; return Image
+
+def _to_jpeg(source, *, source_name: str) -> bytes:
+    """Decode source (path or file-like) to JPEG bytes. ..."""
+    Image = _prepare_pillow()
+    try:
+        with Image.open(source) as img:
+            ... img.save(buf, format="JPEG", quality=90); return buf.getvalue()
+    except ImageConversionError: raise
+    except Exception as exc:
+        raise ImageConversionError(f"Could not convert {source_name} to JPEG: {exc}") from exc
+
+def convert_to_jpeg(file_path: Path) -> bytes:
+    """... delegates to _to_jpeg ..."""
+    return _to_jpeg(file_path, source_name=file_path.name)
+
+def convert_to_jpeg_bytes(data: bytes, *, source_name: str = "<in-memory image>") -> bytes:
+    """Bytes-accepting sibling of convert_to_jpeg. ..."""
+    return _to_jpeg(io.BytesIO(data), source_name=source_name)

--- a/src/loomworks/uploads/skills/image_vision_analysis.py
+++ b/src/loomworks/uploads/skills/image_vision_analysis.py
@@ imports @@
+from pathlib import Path
@@ inside transform(), after the late import of run_vision @@
+    from loomworks.files.conversion import (
+        ImageConversionError, convert_to_jpeg_bytes, needs_jpeg_conversion,
+    )
+    image_bytes = content
     media_type = "image/jpeg"
     if filename:
-        detected = detect_vision_media_type(filename)
-        if detected is not None:
-            media_type = detected
+        if needs_jpeg_conversion(Path(filename)):
+            try:
+                image_bytes = convert_to_jpeg_bytes(content, source_name=filename)
+            except ImageConversionError as exc:
+                raise TransformationError(
+                    f"image_vision_analysis could not convert {filename!r} to JPEG ...") from exc
+            media_type = "image/jpeg"
+        else:
+            detected = detect_vision_media_type(filename)
+            if detected is not None:
+                media_type = detected
@@ both run_vision(...) calls @@
-            image_bytes=content,
+            image_bytes=image_bytes,

Next decision for this thread: push + open PR, or hold. It is correct and green, but not the fix for the reported 500 — confirm that's understood before merging (it shouldn't be sold as "fixes the upload 500").

Thread 2 — Record-note correction (the "→ 500" attribution is WRONG)

The true cause of the upload 500, captured empirically (real server traceback, not inferred): 


loomworks.storage.object_store.ObjectStoreError: MinIO bucket setup failed for 'loomworks-uploads'
  at  src/loomworks/storage/minio_backend.py:60   (_ensure_bucket)
  ← minio_backend.py:53 (__init__)
  ← src/loomworks/storage/factory.py:60  (build_object_store)
  ← src/loomworks/api/deps.py:84  (get_object_store)   ← FastAPI dependency
  ← fastapi/dependencies/utils.py:678  (solve_dependencies)
underlying: urllib3 MaxRetryError HTTPConnectionPool(host='localhost', port=9000): Connection refused

The note that needs rewriting is /Users/dunin7/loomworks-record/investigations/loomworks-upload-vision-conversion-bug-diagnosis-v0_1.md (committed 39f058f). Specifically wrong: Item-1 title ("…(the 500)"), the causal chain lines 38–49 (the → uploads.py:419 → HTTP 500 step), the key-line-ref line 62, and the "Fix part 2" framing line 67. Item 2 (AVIF double-classification) and the conversion-never-called finding remain valid — just reclassify Item 1's symptom as transformation_failed/HTTP-200, and add a new item for the real MinIO-down 500. Verbatim current text: 


# Upload vision-conversion bug — diagnosis (v0.1)

**Status.** Verified diagnosis. Fix part 1 in progress on engine branch `fix-upload-vision-conversion-missing` (off `main`).
**Date.** 2026-06-02
**Environment.** Engine repo `DUNIN7/loomworks-engine`, `main` baseline. Found while tracing an upload 500 during CR-2026-097 verification.
**Method.** Pure-function reproduction — synthesized a 32×24 AVIF with PIL, exercised `convert_to_jpeg` and `image_vision_analysis.transform` (with `run_vision` stubbed; no API/network/DB). All line refs verified against `main`.

---

## Item 1 — `image_vision_analysis.transform` never converts; ships raw bytes mislabeled as `image/jpeg` (the 500)

**Symptom.** Uploading an AVIF (and, by the same path, HEIC / HEIF / TIFF) returns **HTTP 500** `"Executor failed unexpectedly: image_vision_analysis call 1 failed; call 2 not attempted"`.

**Root cause.** `src/loomworks/uploads/skills/image_vision_analysis.py::transform` claims (docstring, lines 193 & 195) to convert via `loomworks.files.conversion.convert_to_jpeg` "when needed" — but **the code never calls it**. `convert_to_jpeg` appears only in the docstring; there is no conversion and no `_needs_conversion` consultation in the skill.

The media-type selection (lines 212–216):

media_type = "image/jpeg" # default if filename: detected = detect_vision_media_type(filename) if detected is not None: media_type = detected 



For any conversion-needing format, `detect_vision_media_type(filename)` returns `None`, so `media_type` silently falls through to the default `"image/jpeg"`, and the **raw, unconverted bytes** are sent to the Vision API (lines 220–225 / 265–269) mislabeled as JPEG.

Verified per-extension behaviour:

| Upload | `detect_vision_media_type` | `_needs_conversion` | media_type sent | bytes sent |
|---|---|---|---|---|
| `.avif` | `None` | **True** | `image/jpeg` ⚠️ | raw AVIF, unconverted |
| `.heic` | `None` | True | `image/jpeg` ⚠️ | raw HEIC |
| `.heif` | `None` | True | `image/jpeg` ⚠️ | raw HEIF |
| `.tiff` | `None` | True | `image/jpeg` ⚠️ | raw TIFF |
| `.jpg/.png/.webp` | correct | False | correct | correct ✅ |

**Causal chain (end to end).**

AVIF upload → uploads.py operator_upload → execute_upload → image_vision_analysis.transform → detect_vision_media_type(".avif") = None → media_type defaults to "image/jpeg" → run_vision(image_bytes=<raw AVIF>, media_type="image/jpeg") ← mislabeled, unconverted → Anthropic rejects malformed payload → ClaudeVisionError (claude_vision.py:167/182) → caught at transform:226 → raise TransformationError → uploads.py:419 blanket except Exception → HTTP 500 "Executor failed unexpectedly: …" 



**Proven vs inferred.** Proven deterministically by the repro: `convert_to_jpeg` works for AVIF (produced 646 valid JPEG bytes, header `\xff\xd8\xff\xe0`), and `transform` never calls it — it sends raw AVIF bytes labelled `image/jpeg`. The final Anthropic-rejection step is inferred (no key/network in the repro), but the mislabeling is a defect regardless of how the API responds, and it matches a 500.

**Dependency theory ruled out.** `pillow_heif` 1.3.0 is installed and `PIL.features.check('avif')` is `True` on this machine — AVIF/HEIC decode works. The 500 is **not** a missing decoder.

**Key line refs.**
- `src/loomworks/uploads/skills/image_vision_analysis.py:193,195` — docstring claims conversion
- `…image_vision_analysis.py:212–216` — media_type default + detect fallthrough (the bug)
- `…image_vision_analysis.py:220–225, 264–269` — `run_vision(image_bytes=content, media_type=media_type)` (raw bytes)
- `…image_vision_analysis.py:226` — `ClaudeVisionError` → `TransformationError`
- `src/loomworks/uploads/vision/claude_vision.py:38` — native media types (no AVIF)
- `…claude_vision.py:54` — convert-first extension set
- `src/loomworks/api/routers/uploads.py:419–423` — blanket `except Exception` → 500 (see Item, fix part 2)
- `src/loomworks/files/conversion.py:58` — `convert_to_jpeg` (works; just never invoked by the skill)

**Fix part 1 (this branch).** Make `transform` actually convert when `_needs_conversion(filename)` is true, via a **bytes-accepting variant** of `convert_to_jpeg` (no temp-file staging), then send the converted bytes with `media_type="image/jpeg"`. Regression test: synthesize AVIF, assert `transform` converts and hands `run_vision` JPEG bytes + `image/jpeg` (stubbed `run_vision` captures the payload).

**Fix part 2 (held — decision pending).** Narrow the executor's blanket `except Exception` (`uploads.py:419`) so genuine format/conversion failures map to **415/422**, reserving 500 for true unexpected faults. Today every executor error — including clean format rejections — masquerades as a 500.

---

## Item 2 — AVIF is double-classified in `conversion.py` (separate inconsistency)

In `src/loomworks/files/conversion.py`, `.avif` appears in **both**:

- line 23 — `_BROWSER_RENDERABLE` (returned **untouched**): `{.jpg, .jpeg, .png, .gif, .webp, .avif}`
- line 31 — JPEG-conversion set `JPEG_CONVERSION_EXTS`: `{.heic, .heif, .tif, .tiff, .avif}`

So AVIF sits in both the "leave alone" set and the "must convert" set — contradictory. Whichever set a given code path consults first determines AVIF's routing; the file-retrieval `?format=jpeg` path and the vision path can therefore disagree on whether AVIF needs conversion. `convert_to_jpeg` also only calls `register_heif_opener()` (HEIF), relying on native PIL for AVIF — which works here but is undocumented as a dependency assumption.

**Held — decision pending.** Cleanup deferred with fix part 2. Resolve which set AVIF should belong to (likely: browser-renderable inline, but convertible when a JPEG is explicitly required) and make the two paths agree.

---

## Scope note

This is **outside CR-2026-097**. It was found during that CR's verification but is an independent upload-pathway defect; it lives on its own branch (`fix-upload-vision-conversion-missing`), not the CR WIP branch. Relates to the prior upload-pathway-cascade work (engine `040d675`).

Correction to make (when you do it): keep Item 2 and the "transform never converts" finding; demote Item 1's symptom from "HTTP 500" to "transformation_failed (HTTP 200, swallowed by executor.py:567)"; add a new Item documenting the real 500 = MinIO-down at deps.py:84/minio_backend.py:60 (now resolved + made durable); fix the uploads.py:419/fix part 2 framing (that except isn't even on the 500 path). Bump to v0.2.

Thread 3 — Two clarification-card findings (NOT yet recorded anywhere)

Context: an image upload with no confident chain match comes back needs_clarification. The OL renders AskOperatorToClarifyCard. An Operator clicked Confirm; the UI showed "Noted — skipped." Engine state showed the card writes nothing (no chain selected, no skill ran, no version bump) — by design, per the card's own header: "Resolution callback is v1 local-state only — engine re-dispatch is a future-phase capability."

Finding A — resolutionLabel null-skill → "skipped" fallthrough (frontend cosmetic bug)

File: /Users/dunin7/loomworks/src/components/chat/AskOperatorToClarifyCard.tsx

The Confirm button passes classification.classified_label as the skill; when the classifier produced all-zero scores, classified_label is null, so it calls resolve("confirm", null). resolutionLabel has no branch for confirm + null skill, so it falls through to "skipped": 


// Confirm button (lines ~78–85):
onClick={() => resolve("confirm", classification.classified_label)}   // classified_label === null here

// resolutionLabel (lines 107–114):
function resolutionLabel(action: ClarifyAction, skill: string | null): string {
  if (action === "confirm" && skill) return `confirmed ${skill}`;   // skill null → FALSE
  if (action === "pick") return "you'll pick a different skill";
  return "skipped";                                                  // ← confirm+null lands here
}

Verdict: NOT mis-wired to skip, NOT an intentional "confirm-with-all-zero-scores → skip". The action recorded in local state is genuinely "confirm"; only the label is wrong. Fix: add a confirm-with-null-skill branch (e.g. "confirmed"). Note the card is only used at UploadResultCard.tsx:165 and is passed no onResolved, so all three actions are engine no-ops today regardless.

Finding B — accuracy-framing emitted with a null classified_label (engine; touches a settled commitment)

File: /Users/dunin7/loomworks-engine/src/loomworks/uploads/executor.py, classify_purpose_for_chains (line 186).

framing_kind defaults to "accuracy" (executor.py:109, upload_event.py:123). Only the tied-score branch sets framing_kind="purpose" (line 281). The two no-confident-candidate branches leave the default: 


# no purpose declaration (lines 211–219): classified_chain=None, above_threshold=False  → framing defaults "accuracy"
# no keyword match  (lines 241–252): max_score==0 → classified_chain=None, above_threshold=False → framing defaults "accuracy"
# tied top score    (lines 254–282): classified_chain=None → framing_kind="purpose"   ← only explicit set
# unique match      (lines 284+):    classified_chain=X, above_threshold=True

So when there is no confident candidate, the engine emits framing_kind="accuracy" with classified_label=None. The OL accuracy card (AskOperatorToClarifyCard.tsx, rendered when framing_kind==="accuracy") then shows "I want to make sure I read this right. I think this is —." — an accuracy-question with nothing to confirm. The accuracy framing is defined for "the classifier has a candidate it's confident enough to propose" (card header + CR-2026-093 v0.3 §13.1 "two-surface separation": accuracy-question vs purpose/discovery). Emitting accuracy with a null label violates that settled separation — the no-match case should arguably use framing_kind="purpose" (discovery), like the tie case. This is the one finding here that touches a settled commitment.

Thread 4 — Object-store outage should be 503, not bare 500 (open question)

Open question: When the object-store dependency get_object_store (deps.py:84) fails because MinIO/S3 is unreachable, the upload endpoint returns a bare HTTP 500 (unhandled, in dependency resolution). Should this instead be a clean 503 Service Unavailable?

Context to decide it:

Settled-commitment callout (which threads touch them)

Before changing clarification/upload behaviour, check the settled foundation doc: /Users/dunin7/loomworks-record/candidate-seeds/loomworks/loomworks-candidate-seed-v0_9.md.

Quick status table

| Thread | Where | State | |---|---|---| | 1. Part-1 conversion fix | engine branch fix-upload-vision-conversion-missing (bd05d5c) | committed, green; not pushed, not merged | | 2. Record-note correction | loomworks-record/investigations/loomworks-upload-vision-conversion-bug-diagnosis-v0_1.md (39f058f) | needs rewrite to MinIO-down truth; bump to v0.2 | | 3A. resolutionLabel fallthrough | loomworks/src/components/chat/AskOperatorToClarifyCard.tsx:107–114 | unrecorded; trivial frontend fix | | 3B. accuracy-framing + null label | loomworks-engine/src/loomworks/uploads/executor.py:186 (classify_purpose_for_chains) | unrecorded; touches settled §13.1 separation | | 4. object-store-outage → 503 | loomworks-engine/src/loomworks/api/deps.py:84 + uploads.py:443 precedent | open question; "Part 2 / 4xx" framing is dropped |

DUNIN7 · Done In Seven LLC · Miami, Florida

record.dunin7.com prototype · Generated 2026-07-07