Jaroslav Beneš
17f3fa1946
A long conversation eventually just stops working. Compaction summarises the earlier turns and sends the summary in their place. The messages are kept. They stay in the transcript behind a collapsed divider and simply stop being part of the request, which is what makes the button safe to press and automatic compaction safe to have at all: a summary that came out badly is a bad turn, not a lost conversation. Stored on the Chat, not as a synthetic Message. A synthetic row needs a role -- `system` breaks the one-system-message rule the moment build_messages emits it beside the harness, and user/assistant makes it a turn people can edit, regenerate from and copy, indistinguishable from a real one in all four places a bubble is rendered. Worse, "editing rewinds, it does not branch" would silently delete it and leave no marker that compaction had happened at all. The summary goes out as a user turn and an assistant turn, not one. A leading assistant breaks templates requiring the first non-system message to be user; a lone leading user produces user, user whenever the kept history starts on a user turn -- which it always does, because the cutoff lands on a finished reply. compacted_through_id is a plain id rather than a foreign key: migrations.py compiles only the column type, so a REFERENCES clause would exist on a fresh database and not on an upgraded one, and a constraint half the fleet has is worse than none. cutoff_message validates it on every read instead, and a rewind past the boundary clears it. Compacting again summarises only the delta, with the previous summary supplied to be subsumed. Re-summarising the whole chat each time grows quadratically and eventually exceeds the window it is protecting. Automatically at the top of _run, not in post_message: that route's contract is to return immediately and leave the slow part to a resumable connection, and it also means build_request is called once, after compaction, with no second assembly path. The trigger is the last reply's recorded usage plus an estimate of the new turn -- retrospective because true prompt_tokens are only knowable after a response, plus the delta because otherwise fifty thousand characters pasted into the composer overflow a window that read 90% last turn. It never fires when the context length is unknown. It does fire on estimated counts, which is safe here precisely because nothing is lost. _maybe_compact never raises: a failure logs and sends the uncompacted request. A `status` event says "Summarising earlier messages…" in the meantime, because a silent multi-second pause before the first token is what a hang looks like. The wording is three fragments under Admin - Prompts. Clearing task.compact turns compaction off entirely. Also adds compaction.moment(): SQLite does not store the offset, so a row loaded from disk is naive while one in the session's identity map keeps its tzinfo, and comparing the two raises. Every comparison here is between exactly those. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
41 lines
1.5 KiB
HTML
41 lines
1.5 KiB
HTML
{% from "_macros.html" import icon %}
|
|
{#
|
|
The whole thread. Returned after a rewind or a compaction, either of which
|
|
changes an arbitrary number of messages at once -- replacing the lot is
|
|
simpler and less error-prone than working out which individual bubbles to
|
|
remove. Also included by chat/index.html, so the conversation is described in
|
|
exactly one place.
|
|
|
|
Deliberately has no root element: it is swapped with innerHTML into #thread,
|
|
and an outerHTML swap would take the container with it.
|
|
#}
|
|
{% if compacted %}
|
|
{# Summarised turns are kept and still readable -- they have only stopped being
|
|
sent. A compaction that summarised badly is then a bad turn rather than a
|
|
lost conversation. #}
|
|
<details class="compacted">
|
|
<summary class="compacted__summary">
|
|
{{ icon("archive", "icon--sm") }}
|
|
<span>{{ compacted | length }} earlier messages, summarised</span>
|
|
{{ icon("chevron-down", "icon--sm reasoning__chevron") }}
|
|
</summary>
|
|
<p class="compacted__note">
|
|
These are no longer sent to the model; a summary of them goes instead. They
|
|
are kept here so nothing is lost.
|
|
</p>
|
|
<div class="compacted__body">
|
|
{% for message in compacted %}
|
|
{% with body_html = bodies.get(message.id, "") %}
|
|
{% include "chat/_message.html" %}
|
|
{% endwith %}
|
|
{% endfor %}
|
|
</div>
|
|
</details>
|
|
{% endif %}
|
|
|
|
{% for message in messages %}
|
|
{% with body_html = bodies.get(message.id, "") %}
|
|
{% include "chat/_message.html" %}
|
|
{% endwith %}
|
|
{% endfor %}
|