Chat window loading

Voice: On the user side, scrolling far back in a conversation, jumping to a question, then returning to the latest should feel smooth. Technically this is full store + a bounded front-end window, not stuffing every bubble into memory at once.

Truth: this page; model context compaction: 对话历史与上下文压缩机制解析 (the display window ≠ context sent to the model).

Date: 2026-07-29 Status: Done (Web main conversation + question index + mobile window; auxiliary surfaces take the latest page) Related code:

• Backend: backend/internal/store/chat.go, backend/internal/api/handlers/chat.go (History / HistoryTurns)
• Web: client/web/src/chatMessageWindow.ts, client/web/src/useChatSessions.ts, client/web/src/WorkChatMessageThread.tsx, client/web/src/ChatTurnNav.tsx
• Mobile: client/mobile/src/lib/chatMessageWindow.ts, client/mobile/app/(app)/chat/[sessionId].tsx

1. Problem

Under long-form agents (literary writing and similar), a single reply can be tens of thousands of characters; when a conversation reaches hundreds to thousands of rounds:

Compare WeChat: the store may have years of messages; the screen only mounts a nearby slice; when paging older messages, load and recycle the other side; search/locate an item by swapping the window then focusing.

2. Layer boundaries

flowchart LR
  subgraph ui [Front-end display window]
    W[About 80–150 items]
  end
  subgraph api [History API]
    B[before / after / around]
  end
  subgraph db [Database]
    M[chat_messages full set]
  end
  subgraph llm [Model context]
    S[Rolling summary + recent originals]
  end
  W --> B --> M
  M --> S

• DB: keep everything (no TTL); deleting a conversation cascades messages.
• Front-end window: bounded; bidirectional load + recycle.
• Model: independent rolling summary; the front-end window is not required to cover everything sent to the model.

3. Window state machine

Constants (implementation may tune):

• WINDOW_PAGE_SIZE = 10 (first open / return-to-bottom only pulls the latest slice)
• WINDOW_MAX = 40 (if over, recycle the opposite side by load direction)
• TURN_NAV_PAGE_SIZE = 12 (right-side question nav; scrolling to the top auto-loads older)

Transitions

4. API contract

4.1 Message window

GET /api/v1/chat/history

Response:

{
  "items": [ /* ChatMessage chronological */ ],
  "session_id": "...",
  "older_cursor": "…or null",
  "newer_cursor": "…or null",
  "at_live_edge": true,
  "next_cursor": "…same value as older_cursor (compat with old clients)"
}

Cursors remain opaque server encodings (created_at + id).

4.2 Question-nav index

GET /api/v1/chat/history/turns

Opening pulls only the latest page; does not chain requests on mount. The next page is requested when the user scrolls up to the top of the nav, or taps “earlier” at the top.

Response:

{
  "items": [{ "id": "…", "preview": "first-line truncated preview", "created_at": "…" }],
  "session_id": "...",
  "older_cursor": "…or null",
  "next_cursor": "…same value as older_cursor"
}

Web right-side question nav uses this API for pagination; tapping an id outside the window then around-swaps the message window.

5. Scroll keep-position and recycle

Anchor keep-position

1. Before load: find the first visible bubble in the message viewport by data-message-id (or id=work-chat-user-…), record offset = el.getBoundingClientRect().top - viewport.top.
2. After writing items (useLayoutEffect): locate the same id again, scrollTop += (newTop - oldTop) so the view does not jump.
3. suppressAutoScroll: while loading, do not chase the bottom.

Recycle

• After loadOlder, if items.length > WINDOW_MAX: delete extra items from the tail, generate newerCursor from the new last item, atLiveEdge=false.
• After loadNewer over the window: delete from the head, update olderCursor.

6. Versus streaming / edit-resend

• Streaming bubble: lives in document flow outside the window list (does not enter virtual-row height estimates); only when atLiveEdge is the user usually reading at the bottom.
• Stream stick-to-bottom: stick once when the streaming bubble appears after send; after that, chase tokens only while the user is still at the bottom. After the user scrolls up (wheel/touch/drag the bar), stop chasing; stream content continues to append outside the viewport, and the finished answer is not force-pulled.
• New assistant message while not at live edge: do not stuff it into the window; keep the “back to bottom” entry, resetToLatest.
• Send: if !atLiveEdge, ensureAtLiveEdge / resetToLatest first, then send.
• Edit-resend / truncate: after truncate, whole-window refreshHistory, then send, to avoid a dirty window.

7. Surface coverage

8. Acceptance checklist

1. Continuously loading older: in-memory items.length stays near WINDOW_MAX.
2. Top load does not visually jump; tapping the first in-window nav item lands on the matching “you” bubble.
3. around jump: swap window first then focus; no large blank / stacked bubbles.
4. Back to bottom / send: atLiveEdge=true, stream is readable; after scrolling up during stream, no force-pull; after returning to bottom, chase can continue.
5. A thousand-round conversation is tested as “window + API” only; do not require rendering a thousand DOM nodes at once.
6. Right-side nav can load earlier questions independently (does not depend on window body).
7. Mobile top-load keeps memory bounded; sending while not at latest first returns to latest.

9. Revision history