A ceiling for a chat, and a nudge for an agent that stops early
MAX_ROUNDS = 1 was wrong, and wrong in a way worth writing down. The loop already ends the moment a round comes back with no tool calls -- that is the model saying it has what it needs, and it is the termination condition every agentic harness uses. A round limit was never a schedule; it exists to catch the case where the model never says so. One is low enough to stop being a ceiling and start being a schedule: it overrode the model's judgement on every single turn. And it broke something concrete. Several built-ins are two-step pairs -- knowledge_get and notes_get read a document "by the id a search returned" -- so one round left the library searchable and not readable. That is not an edge case, it is the library working at half depth, and I understated it as "cannot search the web and then read a result" when the change went in. It is a setting now, under General, default 5, with 0 meaning no ceiling. The loop and the harness both read settings_store.chat_rounds, so the model is never told a budget that is not its own; tools.MAX_ROUNDS is the fallback for callers with no session and a test pins the two equal. core.rounds goes back to naming the number, and vanishes entirely when there is no ceiling rather than promising zero rounds. The other half of "let it decide how long to go": an agent reply that ends while its plan still has open tasks is asked once to carry on. Only against a plan, because that is the one thing there is to be objectively wrong about -- a model with no plan that says it has finished is believed, and arguing with it would be guessing. At most twice in a row, with the count reset the moment it calls a tool again, so the bound is on consecutive stops rather than on stops in total. Never in Plan mode and never past plan_submit, which ends the turn on purpose. Giving up is recorded as an event rather than left silent. The model's own words go back with the nudge, which turned up a real bug on the way: ReasoningSplitter holds back a few characters against a <think> tag split across chunks, so round_text at the end of a round was missing its tail. That text is echoed as an assistant turn for tool rounds too, so a model has been occasionally asked to continue from a transcript where it trailed off mid-sentence. Flushed per round now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
|
||||
lembas secret-key # generate LEMBAS_SECRET_KEY
|
||||
lembas create-admin # create or promote an admin
|
||||
|
||||
pytest # 1195 tests, ~70s
|
||||
pytest # 1206 tests, ~70s
|
||||
# PLAN.md tracks what is and is not built
|
||||
ruff check . # lint (line length 100)
|
||||
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
|
||||
@@ -767,14 +767,22 @@ search is enabled, the user has `tools.web_search`, **and** the model is flagged
|
||||
`tools` — sending a `tools` array to an endpoint without support fails the whole
|
||||
request, exactly as images do without `vision`.
|
||||
|
||||
**An ordinary chat gets ONE round; an agent chat runs until the work is done.**
|
||||
`MAX_ROUNDS` is 1. A plain conversation asking a question is one round of looking
|
||||
things up and then an answer, and the rounds after that were a small model that
|
||||
had decided searching was the answer searching until the context ran out, at a
|
||||
full request each. Several tools can still be called *within* that round, which
|
||||
is the thing worth telling the model. The trade is real and worth naming: a plain
|
||||
chat can no longer search and then read one of the results, because reading is a
|
||||
second round — that is what an agent chat is for.
|
||||
**A round ceiling is a ceiling, not a schedule.** The loop ends the moment a
|
||||
round comes back with no tool calls — that is the model saying it has what it
|
||||
needs, and it is the same termination condition every agentic harness uses. The
|
||||
number only catches the case where it never says so: a small model that has
|
||||
decided searching is the answer, searching until the context runs out at a full
|
||||
request each.
|
||||
|
||||
It was briefly 1, and that is the lesson. One is low enough to stop being a
|
||||
ceiling and start being a schedule — it overrode the model's judgement on every
|
||||
turn rather than catching a runaway. Worse, several built-ins are **two-step
|
||||
pairs**: `knowledge_get` and `notes_get` read a document *"by the id a search
|
||||
returned"*, so a ceiling of one left the library searchable and not readable.
|
||||
`settings_store.chat_rounds()` is the number now, default 5, **0 meaning no
|
||||
ceiling** (the loop falls back to `MAX_TOOL_ROUNDS`, a runaway backstop).
|
||||
`tools.MAX_ROUNDS` is only the fallback for callers with no session, and a test
|
||||
pins the two equal.
|
||||
|
||||
An agent chat is sized by `agent/policy.py:Limits` instead, where **`steps` is a
|
||||
runaway backstop and not a working budget**. It was 40 and it was reached; a step
|
||||
@@ -782,7 +790,9 @@ count low enough to be the thing that ends a reply is a count that ends it
|
||||
halfway. What actually bounds one is the wall clock and `completion_tokens`.
|
||||
These are different sentences rather than the same sentence with a different
|
||||
number in it, which is why `core.rounds` and `core.keep_working` are two
|
||||
fragments gated on `round_budget` rather than one with `{{max_rounds}}` in it.
|
||||
fragments gated on `round_budget` — blank in an agent chat, and blank again when
|
||||
an administrator has set no ceiling, so the fragment vanishes rather than
|
||||
promising zero rounds.
|
||||
|
||||
**The token ceiling would have worked on OpenAI and silently done nothing
|
||||
elsewhere.** `generation.completion_tokens` is only populated when the endpoint
|
||||
@@ -791,6 +801,25 @@ estimate is computed once, in `_run`'s `finally:`, long after the loop that need
|
||||
it. `_written()` takes `max(reported, estimated)` so the limit fires everywhere.
|
||||
The worst kind of limit is one that looks configured.
|
||||
|
||||
**A model that stops is believed, unless its own plan says otherwise.**
|
||||
`core.keep_working` is the cheap half of stopping-halfway; `generation._nudge`
|
||||
is the other half, and it only fires where there is something objective to check
|
||||
against — an open task on the chat's plan. No plan means nothing to be wrong
|
||||
about, so a model with none that says it has finished is taken at its word. It
|
||||
is asked at most `MAX_NUDGES` times **in a row** (the count resets the moment a
|
||||
tool is called again), never in Plan mode, and never past `plan_submit` — that
|
||||
ends the turn deliberately and nudging it would argue with the point of the
|
||||
mode. Giving up is recorded as an event rather than left silent. The model's own
|
||||
words go back with the nudge, or it is asked to carry on from a transcript in
|
||||
which it never spoke.
|
||||
|
||||
**A round's text is flushed before it is echoed back.** `ReasoningSplitter`
|
||||
holds back a few characters against a `<think>` tag split across chunks, so
|
||||
`round_text` at the end of a round was missing its last words — and that text is
|
||||
echoed as an assistant turn, both for a tool round and for a nudge. A turn
|
||||
missing its tail is one the model is asked to continue from having apparently
|
||||
trailed off mid-sentence.
|
||||
|
||||
**A chat can narrow what it may use, and can never widen it.** `Chat.scope_json`
|
||||
is filtered inside `resolve_tools` *after* the capability, permission and
|
||||
instance gates — exactly as `chat.knowledge_bases` narrows `knowledge_search` —
|
||||
|
||||
Reference in New Issue
Block a user