Compaction: a button, and automatically when the window fills

A long conversation eventually just stops working. Compaction summarises the
earlier turns and sends the summary in their place.

The messages are kept. They stay in the transcript behind a collapsed
divider and simply stop being part of the request, which is what makes the
button safe to press and automatic compaction safe to have at all: a summary
that came out badly is a bad turn, not a lost conversation.

Stored on the Chat, not as a synthetic Message. A synthetic row needs a
role -- `system` breaks the one-system-message rule the moment build_messages
emits it beside the harness, and user/assistant makes it a turn people can
edit, regenerate from and copy, indistinguishable from a real one in all
four places a bubble is rendered. Worse, "editing rewinds, it does not
branch" would silently delete it and leave no marker that compaction had
happened at all.

The summary goes out as a user turn and an assistant turn, not one. A
leading assistant breaks templates requiring the first non-system message to
be user; a lone leading user produces user, user whenever the kept history
starts on a user turn -- which it always does, because the cutoff lands on a
finished reply.

compacted_through_id is a plain id rather than a foreign key: migrations.py
compiles only the column type, so a REFERENCES clause would exist on a fresh
database and not on an upgraded one, and a constraint half the fleet has is
worse than none. cutoff_message validates it on every read instead, and a
rewind past the boundary clears it.

Compacting again summarises only the delta, with the previous summary
supplied to be subsumed. Re-summarising the whole chat each time grows
quadratically and eventually exceeds the window it is protecting.

Automatically at the top of _run, not in post_message: that route's contract
is to return immediately and leave the slow part to a resumable connection,
and it also means build_request is called once, after compaction, with no
second assembly path. The trigger is the last reply's recorded usage plus an
estimate of the new turn -- retrospective because true prompt_tokens are only
knowable after a response, plus the delta because otherwise fifty thousand
characters pasted into the composer overflow a window that read 90% last
turn. It never fires when the context length is unknown. It does fire on
estimated counts, which is safe here precisely because nothing is lost.

_maybe_compact never raises: a failure logs and sends the uncompacted
request. A `status` event says "Summarising earlier messages…" in the
meantime, because a silent multi-second pause before the first token is what
a hang looks like.

The wording is three fragments under Admin - Prompts. Clearing task.compact
turns compaction off entirely.

Also adds compaction.moment(): SQLite does not store the offset, so a row
loaded from disk is naive while one in the session's identity map keeps its
tzinfo, and comparing the two raises. Every comparison here is between
exactly those.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/services/generation.py

@@ -22,9 +22,12 @@ import time
 from dataclasses import dataclass, field
 from datetime import UTC, datetime, timedelta
 
+from sqlalchemy import select
+
 from lembas.db.models import ROLE_ASSISTANT, ROLE_USER, Chat, Message, User
 from lembas.db.session import session_scope
 from lembas.services import chat as chat_service
+from lembas.services import compaction as compaction_service
 from lembas.services import metrics as metrics_service
 from lembas.services import prompts as prompts_service
 from lembas.services import tokens
@@ -95,6 +98,10 @@ class Generation:
     # woken individually: with a 100ms cadence a short poll is simpler than
     # future bookkeeping, and cannot drop a wakeup.
     version: int = 0
+    # What the reply is doing when it is not producing tokens. Shown in the
+    # streaming bubble, because a silent multi-second pause before the first
+    # token is what a hang looks like.
+    status: str = ""
     # Number of browsers currently watching. Decides whether a finished reply
     # counts as unread.
     followers: int = 0
@@ -214,6 +221,13 @@ async def _run(generation: Generation) -> None:
     title_prompt = ""
 
     try:
+        # Before the request is assembled, so build_request is called once and
+        # what goes out is the compacted conversation -- there is no second
+        # assembly path. Here rather than in post_message because that route's
+        # whole contract is to return immediately, and a three-second
+        # summarisation in front of it would break exactly that.
+        await _maybe_compact(generation)
+
         with session_scope() as db:
             chat = db.get(Chat, generation.chat_id)
             message = db.get(Message, generation.message_id)
@@ -387,6 +401,77 @@ async def _run(generation: Generation) -> None:
         generation.touch()
 
 
+async def _maybe_compact(generation: Generation) -> None:
+    """Summarise the earlier turns if the window is about to be full.
+
+    Never raises. A failed compaction logs and sends the uncompacted request,
+    which either works or fails upstream with a message that says what actually
+    happened -- refusing to answer because the summariser was unavailable would
+    be a worse trade.
+
+    The awaited call is deliberately outside any session, the same shape titling
+    uses: read everything needed, close, ask, reopen to write.
+    """
+    try:
+        with session_scope() as db:
+            chat = db.get(Chat, generation.chat_id)
+            message = db.get(Message, generation.message_id)
+            if chat is None or message is None:
+                return
+
+            pending = _pending_text(db, message)
+            if not compaction_service.should_compact(db, chat, pending=pending):
+                return
+
+            template = prompts_service.resolve(db, "task.compact")
+            upto = compaction_service.last_complete(db, chat)
+            if not template.strip() or upto is None:
+                return
+
+            endpoint, model_id = chat_service.resolve_endpoint(db, chat)
+            transcript = compaction_service.transcript(db, chat, upto=upto)
+            previous = compaction_service.previous_summary_block(chat)
+            upto_id = upto.id
+
+        generation.status = "Summarising earlier messages…"
+        generation.touch()
+
+        summary = await chat_service.summarise_for_compaction(
+            endpoint,
+            model_id,
+            transcript=transcript,
+            previous_summary=previous,
+            template=template,
+        )
+        if not summary:
+            return
+
+        with session_scope() as db:
+            chat = db.get(Chat, generation.chat_id)
+            upto = db.get(Message, upto_id)
+            if chat is None or upto is None:
+                return
+            compaction_service.apply(chat, summary=summary, upto=upto)
+            db.commit()
+            log.info("chat %s compacted automatically through %s", chat.id, upto_id)
+    except Exception:  # noqa: BLE001 - the reply matters more than the tidy-up
+        log.exception("automatic compaction failed for chat %s", generation.chat_id)
+    finally:
+        generation.status = ""
+        generation.touch()
+
+
+def _pending_text(db, message: Message) -> str:
+    """The user turn this reply is answering, for the size estimate."""
+    previous = db.scalars(
+        select(Message)
+        .where(Message.chat_id == message.chat_id, Message.created_at < message.created_at)
+        .order_by(Message.created_at.desc())
+        .limit(1)
+    ).first()
+    return previous.content if previous is not None else ""
+
+
 def _question_from(payload: dict) -> str:
     """The last thing the user said, for auto-titling."""
     for entry in reversed(payload.get("messages", [])):