A ceiling that was a schedule, and a reply that ended in silence

Reported: an ordinary chat with a small local model researching a question
well -- six searches, each one informed by the last -- stopped at the round
limit and produced no answer at all. Two separate faults, and the second is
the serious one.

The limit was 5 and it should not have been a working number. It was 1 once,
and the note beside it already said why that was wrong: a count low enough to
be reached by ordinary work is a schedule, not a ceiling, and it overrides the
model's judgement on every turn instead of catching a runaway. Five was the
same mistake with a larger number. It is 0 now -- no ceiling, falling back to
MAX_TOOL_ROUNDS as a runaway backstop, which is the shape `Limits.steps`
already had for an agent chat. What bounds an ordinary chat is the context
window, which is a real limit rather than a guess at how much looking-up a
question deserves. An administrator who wants a ceiling can still set one.

The worse fault: *every* budget ended the reply where it was noticed. That is
survivable for a model that narrates as it works and produces nothing at all
for one that goes straight to tool calls -- an empty bubble with a red line
under it, and everything it had gathered thrown away. `_wrap_up` withdraws the
tools and asks once more instead. What it found is in the transcript either
way; one request turns it into an answer. Same move `plan_submit` makes, and
the reason the loop now runs to `budget + 2`: the round at the budget notices,
the one after it answers. The event stays, because an answer the model chose to
give and one it gave because it ran out of room read identically otherwise.

`_too_big` is the one exception and stays a hard stop. It *is* the finding that
there is no room for another request, so a wrap-up round would be the same
overflow with an upstream error in place of an explanation.

`core.keep_working` was gated on the agent family and is now gated on
`unbounded`, the exact complement of `round_budget` -- so an ordinary chat with
no ceiling is told to work until the job is done rather than being told nothing,
and is never told it has a budget of two hundred, which it would ration.

The regression test asserts the reply is not empty, and fails with `'' ==
'Here is what I found.'` against the old code -- which is exactly what was seen.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 11:11:57 +02:00
parent 20040f53a8
commit 27b94c385d
7 changed files with 248 additions and 59 deletions
+16 -6
View File
@@ -25,7 +25,7 @@ AUDIO = "audio"
# Used when nothing is stored. `services/tools.py:MAX_ROUNDS` is the same number
# and exists for callers with no session -- this module is where the setting is
# read, and the two are asserted equal by a test so they cannot drift.
DEFAULT_CHAT_ROUNDS = 5
DEFAULT_CHAT_ROUNDS = 0
SEARCH = "search"
PROMPTS = "prompts"
AGENTS = "agents"
@@ -53,11 +53,21 @@ def _general_defaults() -> dict[str, Any]:
# model saying it has what it needs. This only catches the case where it
# never says so.
#
# Five rather than one because several built-in tools are two-step pairs
# -- knowledge_get and notes_get read a document "by the id a search
# returned" -- so a ceiling of one makes the second half unreachable and
# the library searchable but not readable. Zero means no ceiling.
"max_chat_rounds": 5,
# Zero, meaning no ceiling, and the loop falls back to MAX_TOOL_ROUNDS
# as a runaway backstop -- the same shape `Limits.steps` has for an agent
# chat. It was 1, then 5, and both were the same mistake at different
# scales: a number low enough to be reached by ordinary work is not a
# ceiling, it is a schedule, and it overrides the model's judgement on
# every turn rather than catching a runaway. Five was reached by a small
# local model doing a genuinely good piece of research -- six searches,
# each one informed by the last -- and the reply ended there.
#
# What actually bounds an ordinary chat is the context window
# (`CONTEXT_HEADROOM`), which is a real limit rather than a guess at how
# much looking-up a question deserves. An administrator who wants a
# ceiling can still set one, and `core.rounds` then tells the model it
# has one; with none, `core.keep_working` tells it to work until done.
"max_chat_rounds": 0,
}