Loomworks — Gaussian Splat Ingest and Render Investigation — v0.1
Plain-language summary
What this document does. Records a considered position on whether and how Loomworks would handle Gaussian Splat data — taking it in (ingest) and producing something a reader can view (render). It is an opinion held in reserve, not a plan.
What a Gaussian Splat is. A capture format for 3D scenes. A real place or object is photographed from many angles, and the result is a dense field of millions of small 3D "Gaussians" — each one a point in space carrying position, shape, colour, and transparency. Played back in a viewer, the field reconstructs the scene so you can move a camera around inside it. It is how photorealistic 3D capture is increasingly done — closer to a recorded hologram than to a hand-built 3D model.
What changed. Nothing. First document on the topic.
Decision needed. None now. The note exists so that if someone asks "could Loomworks do this," there is a worked answer rather than an improvised one.
In scope. How splat data would sit in Memory; how it would leave through Rendering; where it conflicts with current commitments; a recommendation.
Out of scope. Building anything. No change request, no phase, no spec.
The bottom line first
Worth doing, scoped narrowly, deferred until a real engagement needs it.
- Ingest: a splat is an opaque captured artifact referenced from Memory — not a field of millions of assertions. Memory carries the assertion that this engagement holds a splat capture of subject X, at time T, from source S. The geometry is stored behind that reference, not decomposed.
- Render: a splat viewer is the first Mode B render-type Loomworks would declare. Rendering produces the specification — which capture, which opening camera, which scene settings — and a downstream viewer surface does the actual graphics work. The engine does not draw splats.
- Why defer: building splat support speculatively earns nothing and pulls a heavy real-time-graphics surface into the product to maintain. Wait for an engagement that needs it.
Ground truth this rests on
Confirmed from the canonical record, not assumed:
- Seed v0.12 (2026-06-07). Thirteen declared render-types, all Mode A. Zero Mode B render-types declared yet. The seed names Mode B as the path for future external-production render-types but none exist. Memory's declared contribution kinds are: typed, spoken, extracted from a document, described from an image. No content kind, contribution kind, or origin thread names capture, sensor, point-cloud, 3D, mesh, scan, or anything spatial or geometric.
- Manifest v0.51 (2026-06-11). No open item, deferred entry, or residue touches rendering content kinds, viewers, WebGL/WebGPU, 3D, splats, or interactive render surfaces.
- Queued-directions v0.26 (2026-06-11). No queued direction for Gaussian Splat, point cloud, 3D capture, or interactive viewer. One adjacent mention: the multi-stage-extraction entry (Category 1) names a 3D model file as a possible future ingest input — file → orthographic projection → vision-based text description. That is the inverse of splat render work, but it is the natural home for part of the ingest half.
Ingest — how a splat would sit in Memory
The rule it must not break
Memory accumulates knowledge as assertions, each carrying provenance. The seed's standing discipline — the same one that keeps the engagement seed from being decomposed into Memory assertions — applies here directly: a splat is an opaque artifact, not a million assertions.
Decomposing a splat into per-Gaussian rows would be the same category error as decomposing the seed. Millions of geometry points are not knowledge the recall question operates on. Nobody asks Memory "what is the opacity of Gaussian number 4,812,003." They ask "do we have a capture of the north field, when was it taken, what does it link to, has the subject changed since."
How it fits cleanly
A splat enters as capture-origin material, referenced. Memory holds:
- The assertion that the engagement holds a splat capture of a named subject, at a time, from a source — with full provenance (which capture device or session, which scope, who contributed it).
- A reference to the splat artifact itself, stored as a blob behind that reference. The geometry never becomes Memory content.
This fits the provenance model with no strain. The origin thread is sensor/capture, marked like any other origin. The trust axis is tie-to-source, exactly as for a document or an image.
The connection to existing direction
The queued-directions Category 1 entry already contemplates a 3D model file as an extraction input — running it through orthographic projection to a vision-based text description. A splat fits the same family from the opposite end: the splat itself stays an opaque referenced artifact, and a text description of what it captures (derived the same way, if useful) can become an ordinary Memory assertion alongside the reference. The ingest half does not need a new pathway invented for it — it extends the capture/extraction thinking already in the record.
What is new at ingest
One thing only: a content kind for opaque captured artifacts — a blob reference with capture provenance, distinct from the four existing contribution kinds (typed, spoken, document-extracted, image-described). This is the seed-touching part of the ingest story and would need the Operator's sign-off before it landed, because it adds to the declared contribution kinds the seed names.
Render — how a splat would leave through Rendering
Where the real caution is
Splat playback is a live GPU viewer: WebGL or WebGPU, interactive, continuous, camera-driven. That is fundamentally unlike every render-type declared today, all of which produce a settled artifact a reader receives once (a document, a page, a conversation turn).
Reading it against the seed's two modes
The seed defines Rendering in two modes, and calls the boundary load-bearing:
- Mode A — Loomworks-owned production. The Render is the artifact: a PDF, docx, xlsx, Markdown, conversation turn, HTML page. The specialist runs inside Loomworks.
- Mode B — Specification to external production system. Loomworks produces the specification; an external system consumes it and produces the artifact. Worked case: an REQ spec consumed by Claude Code to produce an application.
A splat viewer is not Mode A. Forcing the Loomworks Rendering room to rasterise splats would pull a heavy real-time graphics engine inside the product for one content kind. Don't.
It is Mode B, cleanly. Rendering produces the specification — which splat reference to load, the opening camera position, scene and lighting parameters, any annotations to overlay. A downstream viewer surface (a render-specialist, per the seed's own term) consumes that specification and does the GPU work. This matches Mode B's existing shape exactly: Loomworks specifies, an external production environment realises.
The significance
There are zero Mode B render-types declared today. A splat viewer would be the first. That is not a small footnote — the seed says a Mode B addition requires defining the specification grammar (where a Mode A addition only requires registering a specialist). So the render half is not "add a render-type"; it is "exercise Mode B for the first time, and define its specification grammar." That is real methodology surface, and it is the reason this is worth recording rather than improvising later.
What this respects
- Only show what is available. The viewer is a declared render-type with a real specification behind it, or it does not appear. No placeholder 3D pane.
- Lineage. A Mode B splat Render records what it was produced from — which Shape, which capture reference, what configuration — exactly as the seed requires of every Render.
- The engine stays clean. Graphics live in the viewer surface, not the engine.
Alternatives considered and set aside
- Decompose the splat into Memory assertions. Rejected — category error, same class as decomposing the seed. Geometry is not knowledge the recall question operates on.
- Render splats inside the engine as Mode A. Rejected — drags a real-time GPU graphics dependency into the engine for one content kind; breaks the "engine stays clean" line.
- Invent a parallel ingest pathway for 3D/capture. Rejected — the capture/extraction thinking in queued-directions Category 1 already covers the family; extend it rather than fork it.
- Treat a splat the same as the existing "described from an image" kind. Rejected — an image description is a single derived assertion; a splat is a large opaque artifact that must be referenced and stored, not just described. It needs its own opaque-captured-artifact content kind.
Recommendation
If the day comes that an engagement needs it:
- Ingest — declare one new Memory content kind: opaque captured artifact (blob reference + capture provenance). Splat stored behind a reference; an optional vision-derived text description can ride alongside as an ordinary assertion.
- Render — declare the first Mode B render-type: a splat-viewer specification. Define its specification grammar (capture reference, opening camera, scene parameters, overlays). A downstream viewer surface consumes it.
- Until then — hold this note. Do not build. Speculative splat support earns nothing and adds a graphics surface to maintain.
Both the new content kind and the first Mode B render-type touch the seed and require Operator sign-off before they land. Nothing here proposes that sign-off now.
Conflicts surfaced (per standing rule — raised, not resolved)
- Adds to declared contribution kinds. The seed names four (typed, spoken, document-extracted, image-described). An opaque-captured-artifact kind is a fifth. Seed amendment, Operator decision — flagged, not assumed.
- First exercise of Mode B. No Mode B render-type exists. The seed requires defining the specification grammar for any Mode B addition. This is methodology surface, not a routine render-type registration — flagged so it is not mistaken for the lighter Mode A path.
Neither is a defect. Both are decisions the Operator would own if and when this moves.