DUNIN7 · LOOMWORKS · RECORD
record.dunin7.com
Status Current
Path investigations/loomworks-upload-vision-conversion-bug-diagnosis-v0_2.md

Upload vision-conversion bug — diagnosis (v0.2)

Version: 0.2 (supersedes v0.1 / commit 39f058f). v0.1 misattributed the upload 500 to the AVIF-conversion path; v0.2 corrects it to the real cause (MinIO down). The original attribution is preserved struck-through in the Correction Notice below so the trajectory is recoverable. Status. Corrected diagnosis. Two distinct defects, one real outage. Fix part 1 (conversion) on engine branch fix-upload-vision-conversion-missing (bd05d5c, off main); the real 500 (MinIO down) is resolved + made durable. Date. 2026-06-02 (v0.1), corrected 2026-06-02 (v0.2). Environment. Engine repo DUNIN7/loomworks-engine. v0.1 reasoned from a pure-function repro (no API/network/DB); v0.2 adds the actual server traceback captured from a live upload against an isolated main-code instance. Line refs re-verified against live code 2026-06-02 — all held, no drift.


⚠️ Correction notice — what v0.2 changes and why

v0.1 ran a pure-function reproduction (run_vision stubbed, no real HTTP/network). It correctly found that transform never converts AVIF, then inferred that this produced the HTTP 500. A later session captured the real server traceback from an actual upload and the inference was wrong:

Superseded v0.1 attribution (struck through, retained for trajectory):

> ~~## Item 1 — image_vision_analysis.transform never converts; ships raw bytes mislabeled as image/jpeg (the 500)~~ > > ~~Symptom. Uploading an AVIF … returns HTTP 500 "Executor failed unexpectedly: image_vision_analysis call 1 failed; call 2 not attempted".~~ > > ~~Causal chain (end to end).~~ > ``` > ~~AVIF upload → … → run_vision(image_bytes=<raw AVIF>, media_type="image/jpeg")~~ > ~~ → Anthropic rejects → ClaudeVisionError → caught at transform:226 → raise TransformationError~~ > ~~→ uploads.py:419 blanket except Exception → HTTP 500 "Executor failed unexpectedly: …"~~ > ``` > ~~Proven vs inferred. … the mislabeling … matches a 500.~~ ← the "matches a 500" inference was wrong > ~~Dependency theory ruled out. … The 500 is not a missing decoder.~~ ← true that it's not a decoder; but the framing assumed an AVIF-path 500 that did not exist > ~~Key line refs: src/loomworks/api/routers/uploads.py:419–423 — blanket except Exception → 500 (see fix part 2)~~ ← that except is not on the 500's path > ~~Fix part 2 (held). Narrow the executor's blanket except Exception (uploads.py:419) so genuine format/conversion failures map to 415/422…~~ ← moot: that except never sees the 500; replaced by the 503 question (Item 3)

Everything in v0.1 about transform never converting, convert_to_jpeg working for AVIF, the per-extension table, and the AVIF double-classification remains true and is carried forward below — only the symptom classification (500 → 200) and the root-cause-of-the-500 (conversion → MinIO) are corrected.


Item 1 — image_vision_analysis.transform never converts; ships raw bytes mislabeled as image/jpeg

This is a real latent defect, but its symptom is transformation_failed (HTTP 200), NOT a 500.

Root cause. src/loomworks/uploads/skills/image_vision_analysis.py::transform claims (docstring) to convert via loomworks.files.conversion.convert_to_jpeg "when needed" — but the code never called it. For any conversion-needing format, detect_vision_media_type(filename) returns None, so media_type falls through to the default "image/jpeg" and the raw, unconverted bytes are sent to Vision mislabeled as JPEG.

Verified per-extension behaviour (unchanged from v0.1):

| Upload | detect_vision_media_type | needs_jpeg_conversion | media_type sent | bytes sent | |---|---|---|---|---| | .avif | None | True | image/jpeg ⚠️ | raw AVIF, unconverted | | .heic | None | True | image/jpeg ⚠️ | raw HEIC | | .heif | None | True | image/jpeg ⚠️ | raw HEIF | | .tiff | None | True | image/jpeg ⚠️ | raw TIFF | | .jpg/.png/.webp | correct | False | correct | correct ✅ |

Corrected symptom chain.


AVIF upload (with MinIO up)
 → operator_upload → execute_upload → image_vision_analysis.transform
   → media_type defaults to "image/jpeg"; raw AVIF bytes sent to run_vision (mislabeled)
   → Vision rejects → ClaudeVisionError (claude_vision.py:167/182)
   → caught at transform:226 → raise TransformationError
 → execute_upload CATCHES TransformationError (executor.py:567)
   → returns ExecutorResult(status="transformation_failed")
 → operator_upload persists it; endpoint returns HTTP 200 with per-file status="transformation_failed"

Dependency theory (decoders) ruled out, still true. pillow_heif 1.3.0 is installed and PIL.features.check('avif') is True; convert_to_jpeg produces valid JPEG for AVIF (646 bytes, header \xff\xd8\xff\xe0). The skill simply never invoked it.

Key line refs (re-verified 2026-06-02).

Fix part 1 (branch fix-upload-vision-conversion-missing, bd05d5c). transform now converts when needs_jpeg_conversion(Path(filename)) is true, via a bytes-accepting convert_to_jpeg_bytes (no temp-file staging), and sends image/jpeg; conversion failure → TransformationError. Regression test synthesizes an AVIF and asserts transform hands run_vision JPEG bytes + image/jpeg (stubbed run_vision). Committed, green; not pushed, not merged.


Item 2 — The actual upload 500: MinIO down (object-store dependency fails before the handler)

Real server traceback (captured against a live main-code instance):


loomworks.storage.object_store.ObjectStoreError: MinIO bucket setup failed for 'loomworks-uploads'
  at  src/loomworks/storage/minio_backend.py:60   (_ensure_bucket, raise)
  ← src/loomworks/storage/minio_backend.py:53      (__init__ → self._ensure_bucket())
  ← src/loomworks/storage/factory.py:60            (build_object_store → return MinIOObjectStore(...))
  ← src/loomworks/api/deps.py:84                    (get_object_store → return build_object_store())  [FastAPI dependency]
  ← fastapi/dependencies/utils.py:678              (solve_dependencies)
underlying: urllib3.exceptions.MaxRetryError HTTPConnectionPool(host='localhost', port=9000): Connection refused

Why it is a bare 500, not the uploads.py:419 message. get_object_store is a FastAPI Depends, resolved in solve_dependencies before operator_upload's body executes. The ObjectStoreError is unhandled there, so FastAPI/Starlette emits the default 500 with body Internal Server Error and no detail — distinct from the in-handler uploads.py:421–422 "Executor failed unexpectedly: {exc}". The blanket except at uploads.py:419 is not on this path.

File-type independent. A control JPEG POST returned the identical bare 500 — confirming the cause is the object-store dependency, not anything about AVIF/format/conversion.

Resolution (done). MinIO was simply not running on :9000 (power-loss casualty, never restarted). It was started against its existing data dir /Users/dunin7/minio-data, bucket loomworks-uploads, with the engine's .env creds — after which the same upload returned HTTP 200 and landed an upload_event_received event + a MinIO blob. It is now durable via LaunchAgent ~/Library/LaunchAgents/com.dunin7.minio.plist (RunAtLoad + KeepAlive), so the outage will not recur on reboot.

Line refs re-verified 2026-06-02 (all matched, no drift): minio_backend.py:53 (__init___ensure_bucket()), minio_backend.py:55 (def _ensure_bucket), minio_backend.py:60 (raise ObjectStoreError(...)), factory.py:30 (build_object_store), factory.py:60 (return MinIOObjectStore(...)), deps.py:68 (get_object_store), deps.py:84 (return build_object_store()).


Item 3 — AVIF double-classified in conversion.py (separate inconsistency; unchanged from v0.1)

In src/loomworks/files/conversion.py, .avif appears in both:

So AVIF sits in both the "leave alone" and "must convert" sets — contradictory; the retrieval ?format=jpeg path and the vision path can disagree on whether AVIF needs conversion. convert_to_jpeg also only calls register_heif_opener() (HEIF), relying on native PIL for AVIF (works here, undocumented assumption). Held — decision pending.


Follow-up: object-store outage should be a clean 503, not a bare 500 (replaces v0.1's "fix part 2")

v0.1's "fix part 2" (narrow uploads.py:419 → 415/422 for format failures) is moot — that except isn't on the 500 path, and format failures already come back as HTTP-200 transformation_failed. The real follow-up: the get_object_store dependency failure (deps.py:84minio_backend.py:60) should map to 503 Service Unavailable, consistent with the existing precedent where operator_upload maps put_blob's ObjectStoreError → 503 (uploads.py:444). Tracked as a separate open question (do not fold into the conversion branch).


Scope note

Independent of CR-2026-097 (found during its verification). Fix part 1 lives on its own branch, not the CR WIP branch. Relates to prior upload-pathway-cascade work (engine 040d675).