A ceiling that was a schedule, and a reply that ended in silence

Reported: an ordinary chat with a small local model researching a question
well -- six searches, each one informed by the last -- stopped at the round
limit and produced no answer at all. Two separate faults, and the second is
the serious one.

The limit was 5 and it should not have been a working number. It was 1 once,
and the note beside it already said why that was wrong: a count low enough to
be reached by ordinary work is a schedule, not a ceiling, and it overrides the
model's judgement on every turn instead of catching a runaway. Five was the
same mistake with a larger number. It is 0 now -- no ceiling, falling back to
MAX_TOOL_ROUNDS as a runaway backstop, which is the shape `Limits.steps`
already had for an agent chat. What bounds an ordinary chat is the context
window, which is a real limit rather than a guess at how much looking-up a
question deserves. An administrator who wants a ceiling can still set one.

The worse fault: *every* budget ended the reply where it was noticed. That is
survivable for a model that narrates as it works and produces nothing at all
for one that goes straight to tool calls -- an empty bubble with a red line
under it, and everything it had gathered thrown away. `_wrap_up` withdraws the
tools and asks once more instead. What it found is in the transcript either
way; one request turns it into an answer. Same move `plan_submit` makes, and
the reason the loop now runs to `budget + 2`: the round at the budget notices,
the one after it answers. The event stays, because an answer the model chose to
give and one it gave because it ran out of room read identically otherwise.

`_too_big` is the one exception and stays a hard stop. It *is* the finding that
there is no room for another request, so a wrap-up round would be the same
overflow with an upstream error in place of an explanation.

`core.keep_working` was gated on the agent family and is now gated on
`unbounded`, the exact complement of `round_budget` -- so an ordinary chat with
no ceiling is told to work until the job is done rather than being told nothing,
and is never told it has a budget of two hundred, which it would ration.

The regression test asserts the reply is not empty, and fails with `'' ==
'Here is what I found.'` against the old code -- which is exactly what was seen.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 11:11:57 +02:00
parent a56ee16ee3
commit 9b65adf388
8 changed files with 285 additions and 69 deletions
+37 -10
View File
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 1399 tests, ~87s
pytest # 1403 tests, ~87s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
@@ -1039,15 +1039,42 @@ number only catches the case where it never says so: a small model that has
decided searching is the answer, searching until the context runs out at a full
request each.
It was briefly 1, and that is the lesson. One is low enough to stop being a
ceiling and start being a schedule — it overrode the model's judgement on every
turn rather than catching a runaway. Worse, several built-ins are **two-step
pairs**: `knowledge_get` and `notes_get` read a document *"by the id a search
returned"*, so a ceiling of one left the library searchable and not readable.
`settings_store.chat_rounds()` is the number now, default 5, **0 meaning no
ceiling** (the loop falls back to `MAX_TOOL_ROUNDS`, a runaway backstop).
`tools.MAX_ROUNDS` is only the fallback for callers with no session, and a test
pins the two equal.
It was 1, then 5, and it is **0 by default now** — no ceiling, the loop falling
back to `MAX_TOOL_ROUNDS` as a runaway backstop, which is the shape
`Limits.steps` already had for an agent chat. Both numbers were the same mistake
at different scales: low enough to be reached by ordinary work is low enough to
be a schedule rather than a ceiling, overriding the model's judgement on every
turn instead of catching a runaway. One left the library searchable and not
readable, several built-ins being **two-step pairs**`knowledge_get` and
`notes_get` read a document *"by the id a search returned"*. Five ended a small
local model's genuinely good piece of research at its sixth search. What bounds
an ordinary chat is the context window, which is a real limit rather than a
guess at how much looking-up a question deserves. `settings_store.chat_rounds()`
is the number; `tools.MAX_ROUNDS` is only the fallback for callers with no
session, and a test pins the two equal.
**A budget must never end a reply in silence.** Every one of them used to
`break` where it was noticed, leaving whatever prose the model had emitted —
which for a model that goes straight to tool calls is *nothing*, so the reader
got an empty bubble with a red line under it and the whole reply thrown away.
`_wrap_up` withdraws the tools and asks once more instead: what was gathered is
in the transcript either way, and one request turns it into an answer. That is
the move `plan_submit` already makes — a turn should not end mid-sentence — and
it is why the loop runs to `budget + 2`: the round at `budget` notices the
overrun, the one after it answers. The event still goes in the transcript,
because an answer the model chose to give and one it gave because it ran out of
room read identically otherwise. `_too_big` is the single exception and stays a
hard stop: it *is* the finding that there is no room for another request, so a
wrap-up round would be the same overflow with an upstream error in place of an
explanation.
**`core.rounds` and `core.keep_working` are gated on complements, so exactly one
appears.** `round_budget` is set only when an administrator has put a ceiling on
an ordinary chat; `unbounded` is set precisely when it is not, and an agent chat
always has the second. Neither renders anywhere — they exist to be `requires`.
A model told it has a budget rations it and stops early to report progress, and
one told to keep going does the work; those are different sentences rather than
the same sentence with a different number in it.
An agent chat is sized by `agent/policy.py:Limits` instead, where **`steps` is a
runaway backstop and not a working budget**. It was 40 and it was reached; a step