Upload vision-conversion bug — diagnosis (v0.1)

Status. Verified diagnosis. Fix part 1 in progress on engine branch fix-upload-vision-conversion-missing (off main). Date. 2026-06-02 Environment. Engine repo DUNIN7/loomworks-engine, main baseline. Found while tracing an upload 500 during CR-2026-097 verification. Method. Pure-function reproduction — synthesized a 32×24 AVIF with PIL, exercised convert_to_jpeg and image_vision_analysis.transform (with run_vision stubbed; no API/network/DB). All line refs verified against main.

Item 1 — image_vision_analysis.transform never converts; ships raw bytes mislabeled as image/jpeg (the 500)

Symptom. Uploading an AVIF (and, by the same path, HEIC / HEIF / TIFF) returns HTTP 500 "Executor failed unexpectedly: image_vision_analysis call 1 failed; call 2 not attempted".

Root cause. src/loomworks/uploads/skills/image_vision_analysis.py::transform claims (docstring, lines 193 & 195) to convert via loomworks.files.conversion.convert_to_jpeg "when needed" — but the code never calls it. convert_to_jpeg appears only in the docstring; there is no conversion and no _needs_conversion consultation in the skill.

The media-type selection (lines 212–216):

media_type = "image/jpeg"                 # default
if filename:
    detected = detect_vision_media_type(filename)
    if detected is not None:
        media_type = detected

For any conversion-needing format, detect_vision_media_type(filename) returns None, so media_type silently falls through to the default "image/jpeg", and the raw, unconverted bytes are sent to the Vision API (lines 220–225 / 265–269) mislabeled as JPEG.

Verified per-extension behaviour:

| Upload | detect_vision_media_type | _needs_conversion | media_type sent | bytes sent | |---|---|---|---|---| | .avif | None | True | image/jpeg ⚠️ | raw AVIF, unconverted | | .heic | None | True | image/jpeg ⚠️ | raw HEIC | | .heif | None | True | image/jpeg ⚠️ | raw HEIF | | .tiff | None | True | image/jpeg ⚠️ | raw TIFF | | .jpg/.png/.webp | correct | False | correct | correct ✅ |

Causal chain (end to end).

AVIF upload
 → uploads.py operator_upload → execute_upload → image_vision_analysis.transform
   → detect_vision_media_type(".avif") = None  → media_type defaults to "image/jpeg"
   → run_vision(image_bytes=<raw AVIF>, media_type="image/jpeg")   ← mislabeled, unconverted
     → Anthropic rejects malformed payload → ClaudeVisionError      (claude_vision.py:167/182)
   → caught at transform:226 → raise TransformationError
 → uploads.py:419 blanket `except Exception` → HTTP 500
   "Executor failed unexpectedly: …"

Proven vs inferred. Proven deterministically by the repro: convert_to_jpeg works for AVIF (produced 646 valid JPEG bytes, header \xff\xd8\xff\xe0), and transform never calls it — it sends raw AVIF bytes labelled image/jpeg. The final Anthropic-rejection step is inferred (no key/network in the repro), but the mislabeling is a defect regardless of how the API responds, and it matches a 500.

Dependency theory ruled out. pillow_heif 1.3.0 is installed and PIL.features.check('avif') is True on this machine — AVIF/HEIC decode works. The 500 is not a missing decoder.

Key line refs.

• src/loomworks/uploads/skills/image_vision_analysis.py:193,195 — docstring claims conversion
• …image_vision_analysis.py:212–216 — media_type default + detect fallthrough (the bug)
• …image_vision_analysis.py:220–225, 264–269 — run_vision(image_bytes=content, media_type=media_type) (raw bytes)
• …image_vision_analysis.py:226 — ClaudeVisionError → TransformationError
• src/loomworks/uploads/vision/claude_vision.py:38 — native media types (no AVIF)
• …claude_vision.py:54 — convert-first extension set
• src/loomworks/api/routers/uploads.py:419–423 — blanket except Exception → 500 (see Item, fix part 2)
• src/loomworks/files/conversion.py:58 — convert_to_jpeg (works; just never invoked by the skill)

Fix part 1 (this branch). Make transform actually convert when _needs_conversion(filename) is true, via a bytes-accepting variant of convert_to_jpeg (no temp-file staging), then send the converted bytes with media_type="image/jpeg". Regression test: synthesize AVIF, assert transform converts and hands run_vision JPEG bytes + image/jpeg (stubbed run_vision captures the payload).

Fix part 2 (held — decision pending). Narrow the executor's blanket except Exception (uploads.py:419) so genuine format/conversion failures map to 415/422, reserving 500 for true unexpected faults. Today every executor error — including clean format rejections — masquerades as a 500.

Item 2 — AVIF is double-classified in conversion.py (separate inconsistency)

In src/loomworks/files/conversion.py, .avif appears in both:

• line 23 — _BROWSER_RENDERABLE (returned untouched): {.jpg, .jpeg, .png, .gif, .webp, .avif}
• line 31 — JPEG-conversion set JPEG_CONVERSION_EXTS: {.heic, .heif, .tif, .tiff, .avif}

So AVIF sits in both the "leave alone" set and the "must convert" set — contradictory. Whichever set a given code path consults first determines AVIF's routing; the file-retrieval ?format=jpeg path and the vision path can therefore disagree on whether AVIF needs conversion. convert_to_jpeg also only calls register_heif_opener() (HEIF), relying on native PIL for AVIF — which works here but is undocumented as a dependency assumption.

Held — decision pending. Cleanup deferred with fix part 2. Resolve which set AVIF should belong to (likely: browser-renderable inline, but convertible when a JPEG is explicitly required) and make the two paths agree.

Scope note

This is outside CR-2026-097. It was found during that CR's verification but is an independent upload-pathway defect; it lives on its own branch (fix-upload-vision-conversion-missing), not the CR WIP branch. Relates to the prior upload-pathway-cascade work (engine 040d675).