One round for a chat, as many as it takes for an agent
Two different jobs were sharing one number. A plain conversation asking a question is one round of looking things up and then an answer; the rounds after that were a small model that had decided searching was the answer searching until the context ran out, at a full request each. MAX_ROUNDS is 1 now. Several tools can still be called within that round, which is the thing worth telling the model. The trade is real and worth naming: a plain chat can no longer search and then read one of the results, because reading is a second round. That is what an agent chat is for. An agent chat is sized by Limits instead, where steps is now a runaway backstop and not a working budget. It was 40 and it was reached -- a step count low enough to be the thing that ends a reply is a count that ends it halfway. What bounds one now is the wall clock and a new completion-token ceiling, with zero meaning no ceiling, the same convention index_chars already uses. That ceiling would have been decorative. generation.completion_tokens is only populated when the endpoint sends a usage block, and llama.cpp, Ollama and friends never do; the fallback estimate is computed once, in _run's finally, long after the loop that needs it. So _written takes the larger of reported and estimated, and there is a test that runs the whole thing against a stream reporting no usage at all. A limit that works on OpenAI and silently does nothing everywhere else is the worst kind: one that looks configured. core.rounds could not stay one fragment. "You get at most N rounds" is not the same sentence with a different number in it -- a model told it has a budget rations it and stops early to report progress, which is exactly the behaviour that strands a long piece of work. So it splits: core.rounds keeps the one-round case and gates on a new round_budget variable that _agent_values blanks, and core.keep_working says the other thing to an agent chat. A queued message during a one-round reply is now never taken mid-reply -- there is no work under way to steer -- and falls through to _drain, which gives it a reply of its own. No code change went with that; it falls out of the guard, and there is a test so that "it happens to work" and "it is meant to work" stop looking the same. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -127,15 +127,22 @@
|
||||
<section class="card">
|
||||
<h2 class="card__title">What one reply may spend</h2>
|
||||
<p class="field__hint">
|
||||
Three separate bounds, because they fail differently: steps stop a loop,
|
||||
the clock stops one slow command eating an afternoon, and output stops a
|
||||
model filling its own context with build logs and having no room to answer.
|
||||
Four separate bounds, because they fail differently: the clock stops one
|
||||
slow command eating an afternoon, tool output stops a model filling its own
|
||||
context with build logs and having no room to answer, written tokens stop
|
||||
one that keeps going, and the step count is a backstop against a runaway.
|
||||
</p>
|
||||
|
||||
<div class="field">
|
||||
<label class="field__label" for="max_steps">Most rounds of tool calls</label>
|
||||
<input class="input" id="max_steps" name="max_steps"
|
||||
value="{{ values.max_steps }}" inputmode="numeric">
|
||||
<label class="field__label" for="max_completion_tokens">
|
||||
Most a reply may write
|
||||
</label>
|
||||
<input class="input" id="max_completion_tokens" name="max_completion_tokens"
|
||||
value="{{ values.max_completion_tokens }}" inputmode="numeric">
|
||||
<p class="field__hint">
|
||||
In tokens, across every round of one reply. This is the bound that
|
||||
normally ends a long piece of work. Zero means no ceiling.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label class="field__label" for="max_wall_seconds">Longest a reply may take</label>
|
||||
@@ -148,6 +155,16 @@
|
||||
<input class="input" id="max_total_output_bytes" name="max_total_output_bytes"
|
||||
value="{{ values.max_total_output_bytes }}" inputmode="numeric">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label class="field__label" for="max_steps">Most rounds of tool calls</label>
|
||||
<input class="input" id="max_steps" name="max_steps"
|
||||
value="{{ values.max_steps }}" inputmode="numeric">
|
||||
<p class="field__hint">
|
||||
A backstop, not a working budget. An agent reply is meant to run until
|
||||
the task is done, so a number low enough to be what stops it is a number
|
||||
that stops it halfway. Use the token ceiling above for a real limit.
|
||||
</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
|
||||
Reference in New Issue
Block a user