Compaction: a button, and automatically when the window fills
A long conversation eventually just stops working. Compaction summarises the earlier turns and sends the summary in their place. The messages are kept. They stay in the transcript behind a collapsed divider and simply stop being part of the request, which is what makes the button safe to press and automatic compaction safe to have at all: a summary that came out badly is a bad turn, not a lost conversation. Stored on the Chat, not as a synthetic Message. A synthetic row needs a role -- `system` breaks the one-system-message rule the moment build_messages emits it beside the harness, and user/assistant makes it a turn people can edit, regenerate from and copy, indistinguishable from a real one in all four places a bubble is rendered. Worse, "editing rewinds, it does not branch" would silently delete it and leave no marker that compaction had happened at all. The summary goes out as a user turn and an assistant turn, not one. A leading assistant breaks templates requiring the first non-system message to be user; a lone leading user produces user, user whenever the kept history starts on a user turn -- which it always does, because the cutoff lands on a finished reply. compacted_through_id is a plain id rather than a foreign key: migrations.py compiles only the column type, so a REFERENCES clause would exist on a fresh database and not on an upgraded one, and a constraint half the fleet has is worse than none. cutoff_message validates it on every read instead, and a rewind past the boundary clears it. Compacting again summarises only the delta, with the previous summary supplied to be subsumed. Re-summarising the whole chat each time grows quadratically and eventually exceeds the window it is protecting. Automatically at the top of _run, not in post_message: that route's contract is to return immediately and leave the slow part to a resumable connection, and it also means build_request is called once, after compaction, with no second assembly path. The trigger is the last reply's recorded usage plus an estimate of the new turn -- retrospective because true prompt_tokens are only knowable after a response, plus the delta because otherwise fifty thousand characters pasted into the composer overflow a window that read 90% last turn. It never fires when the context length is unknown. It does fire on estimated counts, which is safe here precisely because nothing is lost. _maybe_compact never raises: a failure logs and sends the uncompacted request. A `status` event says "Summarising earlier messages…" in the meantime, because a silent multi-second pause before the first token is what a hang looks like. The wording is three fragments under Admin - Prompts. Clearing task.compact turns compaction off entirely. Also adds compaction.moment(): SQLite does not store the offset, so a row loaded from disk is naive while one in the session's identity map keeps its tzinfo, and comparing the two raises. Every comparison here is between exactly those. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -156,6 +156,16 @@ VARIABLES: tuple[Variable, ...] = (
|
||||
),
|
||||
Variable("question", "Question", "The first message. Chat title task only."),
|
||||
Variable("answer", "Answer", "The first reply. Chat title task only."),
|
||||
Variable(
|
||||
"transcript",
|
||||
"Transcript",
|
||||
"The turns being summarised, oldest first. Compaction task only.",
|
||||
),
|
||||
Variable(
|
||||
"previous_summary",
|
||||
"Earlier summary",
|
||||
"The summary from a previous compaction, if there was one. Compaction task only.",
|
||||
),
|
||||
)
|
||||
|
||||
VARIABLE_NAMES = frozenset(variable.name for variable in VARIABLES)
|
||||
@@ -764,6 +774,77 @@ BUILTIN: tuple[Fragment, ...] = (
|
||||
"Assistant: {{answer}}"
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="task.compact",
|
||||
label="Compaction summary",
|
||||
group=GROUP_TASKS,
|
||||
order=410,
|
||||
variables=("transcript", "previous_summary"),
|
||||
hint="A separate one-message request, not part of any chat. Clear it to "
|
||||
"turn compaction off entirely: the button says so and nothing is "
|
||||
"summarised automatically.",
|
||||
default=(
|
||||
"Summarise the conversation below so it can be carried forward after the "
|
||||
"earlier turns are dropped from your context. This is a working record, "
|
||||
"not a report for a reader.\n"
|
||||
"\n"
|
||||
"Keep, under these headings and in this order:\n"
|
||||
"\n"
|
||||
"## What we are doing\n"
|
||||
"The goal, and where we have got to.\n"
|
||||
"\n"
|
||||
"## Decisions\n"
|
||||
"Anything settled, and why. A decision without its reason gets argued "
|
||||
"again.\n"
|
||||
"\n"
|
||||
"## Facts established\n"
|
||||
"Names, numbers, versions, file paths, URLs and identifiers, copied "
|
||||
"exactly. Do not round them, paraphrase them or reconstruct one from "
|
||||
"memory — if it is not in the transcript, leave it out.\n"
|
||||
"\n"
|
||||
"## Open threads\n"
|
||||
"What is unfinished, and what was about to happen next.\n"
|
||||
"\n"
|
||||
"Leave out pleasantries, retracted ideas and anything already superseded. "
|
||||
"Do not answer the conversation: you are recording it. Write in the "
|
||||
"language of the conversation, and stay under 500 words.\n"
|
||||
"\n"
|
||||
"{{previous_summary}}\n"
|
||||
"\n"
|
||||
"## Transcript\n"
|
||||
"\n"
|
||||
"{{transcript}}"
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="task.compact_lead",
|
||||
label="How a summary is introduced",
|
||||
group=GROUP_TASKS,
|
||||
order=420,
|
||||
hint="Sits in front of the summary, in the turn that replaces the "
|
||||
"messages no longer being sent. Without it a model reads the summary as "
|
||||
"something the person has just typed.",
|
||||
default=(
|
||||
"Here is a summary of the earlier part of this conversation. Those "
|
||||
"messages are no longer in your context. Treat this summary as an "
|
||||
"accurate record of them and rely on it rather than on what you can no "
|
||||
"longer see; if it does not cover something you need, say so instead of "
|
||||
"filling the gap."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="task.compact_ack",
|
||||
label="The model's acknowledgement",
|
||||
group=GROUP_TASKS,
|
||||
order=430,
|
||||
hint="One assistant turn after the summary, so the conversation still "
|
||||
"alternates user, assistant, user. Several chat templates reject a "
|
||||
"history that does not.",
|
||||
default=(
|
||||
"Understood. I have the summary of the earlier turns and will carry on "
|
||||
"from there."
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
register_source(_builtin_source)
|
||||
|
||||
Reference in New Issue
Block a user