Compaction: a button, and automatically when the window fills

A long conversation eventually just stops working. Compaction summarises the
earlier turns and sends the summary in their place.

The messages are kept. They stay in the transcript behind a collapsed
divider and simply stop being part of the request, which is what makes the
button safe to press and automatic compaction safe to have at all: a summary
that came out badly is a bad turn, not a lost conversation.

Stored on the Chat, not as a synthetic Message. A synthetic row needs a
role -- `system` breaks the one-system-message rule the moment build_messages
emits it beside the harness, and user/assistant makes it a turn people can
edit, regenerate from and copy, indistinguishable from a real one in all
four places a bubble is rendered. Worse, "editing rewinds, it does not
branch" would silently delete it and leave no marker that compaction had
happened at all.

The summary goes out as a user turn and an assistant turn, not one. A
leading assistant breaks templates requiring the first non-system message to
be user; a lone leading user produces user, user whenever the kept history
starts on a user turn -- which it always does, because the cutoff lands on a
finished reply.

compacted_through_id is a plain id rather than a foreign key: migrations.py
compiles only the column type, so a REFERENCES clause would exist on a fresh
database and not on an upgraded one, and a constraint half the fleet has is
worse than none. cutoff_message validates it on every read instead, and a
rewind past the boundary clears it.

Compacting again summarises only the delta, with the previous summary
supplied to be subsumed. Re-summarising the whole chat each time grows
quadratically and eventually exceeds the window it is protecting.

Automatically at the top of _run, not in post_message: that route's contract
is to return immediately and leave the slow part to a resumable connection,
and it also means build_request is called once, after compaction, with no
second assembly path. The trigger is the last reply's recorded usage plus an
estimate of the new turn -- retrospective because true prompt_tokens are only
knowable after a response, plus the delta because otherwise fifty thousand
characters pasted into the composer overflow a window that read 90% last
turn. It never fires when the context length is unknown. It does fire on
estimated counts, which is safe here precisely because nothing is lost.

_maybe_compact never raises: a failure logs and sends the uncompacted
request. A `status` event says "Summarising earlier messages…" in the
meantime, because a silent multi-second pause before the first token is what
a hang looks like.

The wording is three fragments under Admin - Prompts. Clearing task.compact
turns compaction off entirely.

Also adds compaction.moment(): SQLite does not store the offset, so a row
loaded from disk is naive while one in the session's identity map keeps its
tzinfo, and comparing the two raises. Every comparison here is between
exactly those.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-01 01:02:02 +02:00
parent 314cc946d7
commit 17f3fa1946
17 changed files with 1068 additions and 12 deletions
+10
View File
@@ -47,6 +47,13 @@ be a different project, not a refactor.
- [x] **Unread indicator** — a green dot and a toast when a reply lands while
you were elsewhere
- [x] Folders, arbitrarily nested; deleting one keeps the chats inside it
- [x] Per-reply metrics — tokens, context used as a percentage, tokens/second,
live while streaming and kept afterwards. Estimated with a `~` when the
endpoint reports no usage
- [x] Compaction — a button, and automatically at a configurable percentage of
the model's context. Summarised turns are kept and collapsed, not deleted
- [x] Temporary chats — never listed, swept after a day, with a Keep button
- [x] An admin-only request inspector beside the thread
### Tools
- [x] **Tool calling** — one reply is a bounded loop of requests, not one
@@ -126,6 +133,9 @@ be a different project, not a refactor.
- [x] Defaults in code and overrides in the database, so improving a default
still reaches an instance that never edited it
### Suggestions
- [x] Admin-managed cards on the new-chat screen; three seeded once at startup
### Interface
- [x] **Installable** — manifest, generated PWA icons, a service worker for the
shell and a themed offline page. The worker deliberately never touches