Loomworks — Axis 11 Conversation Pagination change-request — v0.1

Plain-language summary

Today, opening an engagement loads only the most recent 50 conversation turns and there is no way to see older ones — the substrate already supports paging back, but the surface never asks. This change-request adds a "load earlier conversation" control to the in-engagement surface (CenterPane). When older turns exist, the control appears; clicking it loads the next page of older turns and folds them into the conversation, keeping what you were reading visually still.

It is frontend only. The engine endpoint already exposes the before cursor and the has_more flag (confirmed at Step 0a); the client type already carries has_more (src/lib/types.ts:90) — the surface simply stops throwing it away and starts using it. Nothing touches Memory — pagination is a pure read of conversation history that already exists; it writes nothing, derives nothing, touches no provenance.

A live-browser check at Step 0b established the one fact the build hinges on: the page grows; the inner conversation container does not scroll. On real CenterPane with 54 turns the document was 5128px tall in a 900px viewport, the window was the scroller, and the conversation <div> — despite overflow-y:auto — was exactly as tall as its content and refused to scroll. So scroll-position preservation is keyed to the window, not to any container scrollTop.

What lands.

• A "load earlier conversation" control in CenterPane, gated on has_more: it appears only when older turns exist, and disappears at the start of history. (The settled §6 Q2 decision: explicit control, not infinite scroll.)
• CenterPane stops discarding res.has_more in its history-fetch .then (CenterPane.tsx:357–380) and tracks it in state.
• Clicking the control fetches the next older page — …&before=<oldest raw turn's created_at> — and prepends those turns into CenterPane's turns state.
• Window scroll-position preservation on prepend, using the window scroll model observed at 0b (capture document height + window.scrollY before; restore window.scrollTo(0, scrollY + heightDelta) after) so the turns you were reading stay put when older ones load above them.
• A dedup-by-turn_id guard on prepend, so a turn can never render twice.
• The existing [turns]-keyed scroll effect (CenterPane.tsx:429–437), which assumes a container scroller that does not exist, is neutralized for the load-earlier path so it cannot fight the window-scroll restore.

What you (the Operator) see after this ships.

Open an engagement with a long history (E0005 Loomworks has 96 of your turns). The most recent 50 load as today. Where the older turns live in the flow (below, in the default newest-first order), a "Load earlier conversation" control sits. Click it — the next page of older turns folds in, the turns you were looking at don't jump, and the control stays until you reach the very first turn, then it goes away.

What this change-request does not do.

• No engine change. before and has_more already exist (conversation_history.py:63, 116); the response shape already carries has_more (schemas.py:475). Frontend only.
• No Memory contact. No write, no derivation, no provenance change. Pure read.
• No infinite scroll. Explicit control only (settled §6 Q2).
• No ChatView change. ChatView shares only the trivial fetch URL, not the turn-state or scroll logic; it inherits pagination later through the ChatView → CenterPane port (queued-directions §15.1), not a shared hook now (settled §6 Q1 / Step 0c).
• No summarization, time-windowing, room-segmentation, or held-state work. Deferred per the scoping note.
• No composer-residue fix. Carved out.

Estimated effort. One checkpoint. A single frontend component plus its first test file. No migration, no substrate, no new endpoint.

Step 0 — Codebase inspection block

Step 0 of the handoff (0a/0b/0c) is already executed and accepted; this block is the re-confirm before building, to catch any drift since b77b987.

Step 0.1 — Pre-flight

cd /Users/dunin7/loomworks && git status && git log --oneline -1
cd /Users/dunin7/loomworks-engine && git status && git log --oneline -1

Both working trees clean; frontend at b77b987, engine at 1109fcf (or later). If either has moved, re-read the two fetch sites and the endpoint before proceeding.

Step 0.2 — Re-confirm the endpoint contract (engine, read-only)

sed -n '56,156p' /Users/dunin7/loomworks-engine/src/loomworks/orchestration/routers/conversation_history.py
sed -n '475,483p' /Users/dunin7/loomworks-engine/src/loomworks/orchestration/schemas.py

Confirm still true: before: Optional[datetime] query param applied as created_at < before (line ~104); has_more = len(rows) > limit over a limit+1 fetch (lines ~112–116); response {turns: ConversationTurn[], has_more: bool} (schemas.py:475), turns oldest-first within the window.

Step 0.3 — Re-confirm the client fetch + types

sed -n '349,409p' "/Users/dunin7/loomworks/src/app/operator/engagement/[engagement_address]/CenterPane.tsx"
sed -n '90,93p' /Users/dunin7/loomworks/src/lib/types.ts
sed -n '628,644p' "/Users/dunin7/loomworks/src/app/operator/engagement/[engagement_address]/CenterPane.tsx"

Confirm still true: the mount fetch is …?project_id=…&limit=50, maps res.turns, and discards res.has_more (CenterPane.tsx:357–380); the client ConversationHistoryResponse carries has_more (types.ts:90–93); turns state is ascending (oldest at index 0) and orderedTurns reverses only for newest_first (CenterPane.tsx:628–632).

Step 0.4 — Halt-or-proceed

If 0.2/0.3 match, proceed to Section A. If anything has drifted (the endpoint, the response shape, the order semantics, the filter), halt and report — the build below is written against these exact shapes.

Section A — The "load earlier" control (has_more gating)

A.1 — Track has_more instead of discarding it

In the mount fetch (CenterPane.tsx:357 .then), after setTurns(...), also set new state hasMore from res.has_more. Add:

• const [hasMore, setHasMore] = useState(false);
• In the mount .then: setHasMore(res.has_more);
• On the reset paths that setTurns([]) (mount start line 353, the 404 branch line 395): also setHasMore(false).

A.2 — Track the pagination cursor as the oldest raw created_at

The client filters turns (_filterNonConversationTurns, CenterPane.tsx:257/359), so the oldest visible turn may be newer than the oldest raw turn of the loaded window. The cursor must be the oldest raw turn's created_at, or filtered turns would be re-fetched and the cursor could stall.

• const oldestCursorRef = useRef<string | null>(null);
• In the mount .then, before filtering: oldestCursorRef.current = res.turns[0]?.created_at ?? null; (the response is oldest-first, so res.turns[0] is the raw oldest of the window).
• On reset paths: oldestCursorRef.current = null;

A.3 — Render the control at the older-turn edge, gated on hasMore

Older turns live at the edge of the flow that depends on messageOrder (CenterPane.tsx:628–632):

• newest_first (default, observed live): older turns render at the bottom → the control renders after the grouped list.
• oldest_first: older turns render at the top → the control renders before the grouped list.

Render, only when hasMore is true and a load is not in flight, a button: "Load earlier conversation" (data-testid="load-earlier"), with a loading state ("Loading earlier…", disabled) while a fetch is in flight and an inline error slot for a failed load. When hasMore is false the control is absent (end of history — "only show what is available").

Section B — Fetch, prepend, and the dedup guard

B.1 — The load-earlier fetch

A loadEarlier callback, guarded against concurrent runs (const [loadingEarlier, setLoadingEarlier] = useState(false); plus an in-flight ref):

const before = oldestCursorRef.current;
if (!before || loadingEarlier || !hasMore) return;
setLoadingEarlier(true);
api<ConversationHistoryResponse>(
  `/operator/conversation-history?project_id=${encodeURIComponent(engagement.project_id)}`
  + `&limit=50&before=${encodeURIComponent(before)}`,
)

before is the raw created_at string the client received from the engine; URL-encoded, it round-trips through the same ISO-8601 (de)serialization FastAPI uses to parse Optional[datetime] (signature-grounding, pre-flight shape 2). Same project-scoped path, same limit=50, plus the cursor.

B.2 — Prepend with the dedup-by-turn_id guard

On success:

1. Update the cursor to the new page's raw oldest before filtering: oldestCursorRef.current = res.turns[0]?.created_at ?? oldestCursorRef.current;
2. setHasMore(res.has_more);
3. Filter the page through _filterNonConversationTurns and map to DisplayTurn exactly as the mount fetch does (CenterPane.tsx:359–379) — same shape, no divergence.
4. Prepend with dedup: older turns go at the front of the ascending turns array, skipping any turn_id already present:

setTurns((prev) => {
  const seen = new Set(prev.map((t) => t.turn_id));
  const fresh = mappedOlder.filter((t) => !seen.has(t.turn_id));
  return [...fresh, ...prev];   // older turns are smaller created_at → front
});

The Set guard makes the prepend idempotent — a double-click, a React StrictMode double-invoke, or a retry can never double-render a turn.

On error: leave turns and the cursor unchanged, surface the inline error in the control's slot; the Operator can retry. finally: setLoadingEarlier(false).

B.3 — Known limitation (the created_at-cursor / id-tiebreak edge) — accepted at v0.1

The engine cursor is created_at < before (strict), while its ordering tiebreaks on id DESC within an equal created_at (conversation_history.py:104, 108–110). Two turns sharing an exact created_at across a page boundary therefore have a real edge:

• The dedup guard (B.2) fully prevents duplicates.
• It does not prevent a skip: if a turn shares the boundary turn's exact created_at but sorts after it by id, the strict < cursor excludes it and it is never loaded.

A complete fix is a composite (created_at, id) cursor, which is an engine change and therefore out of this frontend-only scope. Accepted at v0.1 because conversation turns are written sequentially with distinct sub-second created_at values in practice (the boundary collision is rare), and the dedup guard already makes a future move to an inclusive (<= + id) cursor safe. Flagged as the one residual correctness gap (see pre-flight findings). If it ever bites, the fix is engine-side and small.

Section C — Window scroll-position preservation, and neutralizing the inert effect

C.1 — Preserve position on prepend, keyed to the window (per the 0b observation)

The window is the scroller (0b: page grows, container does not scroll). When older turns insert above the current viewport — the oldest_first case, control at top — the view would jump down by the height of the inserted content unless corrected. Correct it on the window:

• Immediately before applying the prepend setTurns, capture prevHeight = document.documentElement.scrollHeight and prevY = window.scrollY.
• In a useLayoutEffect that runs after the prepend commits (gated by a "restore pending" ref set in loadEarlier), compute document.documentElement.scrollHeight - prevHeight and call window.scrollTo(0, prevY + delta). Running in useLayoutEffect (not useEffect) applies the correction before paint, so there is no visible flash.

When older turns insert below the viewport — the newest_first default, control at bottom — the turns being read do not move, so no window correction is applied (applying the delta there would wrongly push the view down toward the new content). Gate the restore to the at-top insertion case (messageOrder === "oldest_first"). State this gating explicitly in code comments.

C.2 — Neutralize the existing [turns]-keyed scroll effect for the load-earlier path

CenterPane.tsx:429–437 runs on every turns change and sets conversationRef.current.scrollTop. Step 0b confirmed this is inert — the container does not scroll, so scrollTop stays 0 (measured: forced scrollTop = 99999 → read back 0). It cannot actively move the window today, but it must not fight the C.1 restore, and if the layout is ever height-bounded it would. Gate it so it does not run on a load-earlier-driven turns change:

• Set a ref (e.g. prependInFlightRef.current = true) in loadEarlier before the prepend; in the effect at 429–437, early-return when it is set, and clear it in C.1's layout effect after the restore.

The effect's existing behavior for new turns (send / receive, which append to the end) is unchanged.

Section D — Tests

CenterPane has no test file today (confirmed: no CenterPane under tests/). This CR adds the first one — tests/components/engagement/CenterPane.pagination.test.tsx — following the repo's established pattern (tests/components/responsive.test.tsx, tests/components/chat/ChatView.test.tsx): jsdom + React Testing Library, providers mocked (AuthProvider, SSEProvider, next/navigation), and vi.spyOn(globalThis, "fetch").mockResolvedValue(jsonResponse({ turns, has_more })).

Tests:

1. has_more gates the control. Mount with has_more: false → no load-earlier control. Mount with has_more: true → control present.
2. Load-earlier prepend. Mount (page 1, has_more: true), click load-earlier, second fetch resolves an older page → the older turns render in order, and the fetch URL carries &before=<page-1 oldest raw created_at> (assert the exact query string).
3. End of history. When the load-earlier page returns has_more: false → the control disappears after that load.
4. Dedup-by-turn_id. The older page overlaps the loaded set by one turn_id → that turn renders exactly once (assert a single element for it).
5. Cursor uses raw oldest, not visible oldest. Page 1's raw oldest turn is one the filter drops → the before on the next fetch is that raw turn's created_at, not the oldest visible one.
6. Window scroll preservation logic. jsdom has no layout engine (no real scrollHeight/getBoundingClientRect heights), so this test asserts the mechanism, not pixels: stub document.documentElement.scrollHeight to differ before/after the prepend and window.scrollY, spy on window.scrollTo, and assert it is called with prevY + (after − before) in the oldest_first case — and not called in the newest_first case. (State this jsdom constraint in the test file header.)
7. Inert effect neutralized. Assert the load-earlier prepend does not trigger the conversationRef.scrollTop reset path (e.g. spy/guard), so C.2's gate holds.

All existing vitest tests continue to pass.

Section E — Verification

Checkpoint (frontend).

cd /Users/dunin7/loomworks
npm test
npm run build

New CenterPane pagination tests plus the full existing suite pass; build succeeds.

Smoke test (real surface, real data).

1. Sign in as Marvin Percival (dev /auth, person 6d867b23…) and open /operator/engagement/E0005 (Loomworks — 96 of his turns).
2. Confirm the newest 50 load as today and a "Load earlier conversation" control is present at the older-turn edge (bottom, in the default newest-first order).
3. Click it. Confirm: older turns fold in; the turns you were reading do not jump; the control remains while has_more holds.
4. Keep clicking to the start of history. Confirm the control disappears at the first turn (no further has_more).
5. Confirm no turn renders twice anywhere in the scrollback.
6. (Optional) Toggle to oldest-first ("show oldest first" to the Companion), reload, and confirm the control sits at the top and the view stays anchored when older turns load above.

Halt for Operator visual confirmation.

Section F — Acceptance criteria

1. With older turns present, a load-earlier control appears, gated on has_more; it is absent at the start of history.
2. Clicking it fetches …&before=<oldest raw created_at>&limit=50 and prepends the older page into turns.
3. No turn ever renders twice (dedup-by-turn_id).
4. The turns being read do not visually jump when older turns load above them (window-scroll preservation, oldest_first); no spurious scroll when they load below (newest_first).
5. The existing [turns]-keyed scroll effect does not fire on the load-earlier path.
6. No engine change; no Memory write/derivation/provenance contact.
7. New CenterPane pagination tests cover §D 1–7; all existing tests pass; build succeeds.
8. The Operator confirms by eye on E0005 that loading earlier feels stable.

Section G — Out of scope

• Engine changes — before/has_more already exist; the composite-cursor fix for §B.3 is explicitly deferred engine-side.
• ChatView — inherits pagination later via the ChatView → CenterPane port, not here.
• Infinite scroll — explicit control only.
• Summarization, time-windowing, room-segmentation, held-state — deferred per the scoping note.
• The ChatView composer-scrolls-off residue — carved out, filed separately.
• Memory contact of any kind.

Section H — Pre-flight findings (five-shape review)

This review is appended to the CR itself; the consolidated finding list is in the accompanying pre-flight report. The load-bearing finding is §B.3 (the created_at-cursor skip edge), surfaced and accepted with a named engine-side future fix. No blockers.

Operator approval

Approved for execution.
- Marvin Percival
- [timestamp]

Once approval is recorded, Claude Code executes the single frontend checkpoint, halting at the smoke test for visual confirmation. Not approved yet — pre-flight only.