Regenerate actually regenerates

`ensure` is keyed on message_id and idempotent on purpose -- a page load
finding an unfinished reply must attach to it rather than start a second
one, and `_follow` calls it too. But finished generations linger in the
registry for KEEP_FINISHED so a follower arriving at the last moment still
gets the final frames, and regenerate is the only caller that reuses an
existing Message row instead of creating a new one. So `ensure` handed back
the finished generation: no request was made, `_follow` replayed the old
answer, and the `done` frame re-rendered a streaming shell because the row
said incomplete. That is the reconnect loop, and the Send button stuck on
Stop. It appeared to work after five minutes only by accident, and only
sometimes: `_prune` sat below the early return, so it was unreachable for
exactly the message that needed it.

`restart()` is the explicit opposite of `ensure`, and regenerate calls it.
`_prune` moves above the lookup.

Cancelling a live predecessor makes its `finally:` run `_persist` on the
same row, which would overwrite the reply that replaced it. `_persist` now
refuses when another generation owns the message -- "someone else owns this
row now", not "this one is registered", so a direct call still writes.

Three things found next door, all in the same area and all bugs:

  - `done` was set before `_persist` committed, while `_follow`'s docstring
    claimed the opposite. `_follow` breaks out the instant it sees the flag
    and re-renders the bubble from the row, so the row has to be right
    first. Harmless today, a guaranteed loss once metrics land there.
  - Live reasoning duplicated quadratically. The frame carries the whole
    block each time, exactly as `render` and `tools` do, but the target
    swapped it `beforeend`.
  - `sse.KEEPALIVE` was defined and never yielded. A model thinking for
    ninety seconds emits nothing, and an idle connection is what a proxy
    closes.

There was no test for regenerate at all. There is now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-01 00:28:19 +02:00
parent 1906919ee2
commit 0df21d23af
4 changed files with 337 additions and 6 deletions
+52 -3
View File
@@ -120,18 +120,48 @@ def ensure(chat_id: str, message_id: str) -> Generation:
Idempotent, because more than one thing can ask for it: the route that
created the message, and any page load that finds the message unfinished.
`_prune` runs first, not after the lookup. Below it, a stale entry could
never expire: the early return is the only path a repeated id takes, so the
sweep was unreachable for exactly the message that needed it.
"""
_prune()
existing = _RUNNING.get(message_id)
if existing is not None:
return existing
_prune()
generation = Generation(chat_id=chat_id, message_id=message_id)
_RUNNING[message_id] = generation
_TASKS[message_id] = asyncio.create_task(_run(generation))
return generation
def restart(chat_id: str, message_id: str) -> Generation:
"""Produce this reply again, discarding any finished attempt at it.
`ensure` is idempotent on purpose, and that is load-bearing: a page load
finding an unfinished reply must attach to it rather than start a second
one, and `_follow` calls it too. Regeneration is the one caller that means
the opposite.
It is also the one caller that reuses an existing Message row -- blanked and
marked incomplete -- rather than creating a new one. The finished Generation
for that id is still in the registry, because finished ones linger
KEEP_FINISHED so a follower arriving at the last moment still gets the final
frames. `ensure` handed that one straight back: no request was made,
`_follow` replayed the previous answer, and the `done` frame re-rendered a
streaming shell because the row said incomplete. That was the reconnect loop,
and the Send button stuck on Stop.
"""
previous = _RUNNING.pop(message_id, None)
task = _TASKS.pop(message_id, None)
if previous is not None and not previous.done:
previous.cancel = True
if task is not None:
task.cancel()
return ensure(chat_id, message_id)
async def shutdown() -> None:
"""Stop every running generation, keeping what each has produced."""
for task in list(_TASKS.values()):
@@ -295,10 +325,14 @@ async def _run(generation: Generation) -> None:
)
title = title or chat_service.fallback_title(question)
# Written *before* `done`, because `_follow` breaks out of its loop the
# moment it sees that flag and immediately re-renders the bubble from
# the row. The other order left a window in which the finished frame
# showed the previous turn's stored values.
_persist(generation, title, time.monotonic() - started)
generation.done = True
generation.finished_at = datetime.now(UTC)
generation.touch()
_persist(generation, title, time.monotonic() - started)
def _question_from(payload: dict) -> str:
@@ -319,7 +353,21 @@ def _question_from(payload: dict) -> str:
def _persist(generation: Generation, title: str, elapsed: float) -> None:
"""Write the finished reply, name the chat, and set the unread flag."""
"""Write the finished reply, name the chat, and set the unread flag.
A generation another one has replaced may not write. A regeneration cancels
its predecessor, whose `finally:` then runs this on the same row -- and it
would overwrite the fresh reply with the abandoned one.
The test is "someone else owns this row now", not "this one is registered":
an unregistered generation still writes, because that is a direct call
rather than a superseded one.
"""
owner = _RUNNING.get(generation.message_id)
if owner is not None and owner is not generation:
log.debug("skipping persist for superseded generation %s", generation.message_id)
return
try:
with session_scope() as db:
message = db.get(Message, generation.message_id)
@@ -364,5 +412,6 @@ __all__ = [
"ensure",
"get",
"request_stop",
"restart",
"shutdown",
]