One round for a chat, as many as it takes for an agent
Two different jobs were sharing one number. A plain conversation asking a question is one round of looking things up and then an answer; the rounds after that were a small model that had decided searching was the answer searching until the context ran out, at a full request each. MAX_ROUNDS is 1 now. Several tools can still be called within that round, which is the thing worth telling the model. The trade is real and worth naming: a plain chat can no longer search and then read one of the results, because reading is a second round. That is what an agent chat is for. An agent chat is sized by Limits instead, where steps is now a runaway backstop and not a working budget. It was 40 and it was reached -- a step count low enough to be the thing that ends a reply is a count that ends it halfway. What bounds one now is the wall clock and a new completion-token ceiling, with zero meaning no ceiling, the same convention index_chars already uses. That ceiling would have been decorative. generation.completion_tokens is only populated when the endpoint sends a usage block, and llama.cpp, Ollama and friends never do; the fallback estimate is computed once, in _run's finally, long after the loop that needs it. So _written takes the larger of reported and estimated, and there is a test that runs the whole thing against a stream reporting no usage at all. A limit that works on OpenAI and silently does nothing everywhere else is the worst kind: one that looks configured. core.rounds could not stay one fragment. "You get at most N rounds" is not the same sentence with a different number in it -- a model told it has a budget rations it and stops early to report progress, which is exactly the behaviour that strands a long piece of work. So it splits: core.rounds keeps the one-round case and gates on a new round_budget variable that _agent_values blanks, and core.keep_working says the other thing to an agent chat. A queued message during a one-round reply is now never taken mid-reply -- there is no work under way to steer -- and falls through to _drain, which gives it a reply of its own. No code change went with that; it falls out of the guard, and there is a test so that "it happens to work" and "it is meant to work" stop looking the same. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -128,6 +128,13 @@ VARIABLES: tuple[Variable, ...] = (
|
||||
Variable("user_name", "User's name", "The name of the person in the conversation."),
|
||||
Variable("model_name", "Model", "The display name of the model answering."),
|
||||
Variable("max_rounds", "Tool rounds", "How many rounds of tool calls one reply may take."),
|
||||
Variable(
|
||||
"round_budget",
|
||||
"Round budget applies",
|
||||
"Set in an ordinary chat and blank in an agent chat. Nothing renders it; "
|
||||
"it exists so a fragment can say `requires=('round_budget',)` and appear "
|
||||
"for one and not the other.",
|
||||
),
|
||||
Variable(
|
||||
"memory_limit",
|
||||
"Memory length",
|
||||
@@ -572,19 +579,64 @@ BUILTIN: tuple[Fragment, ...] = (
|
||||
"permission first."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="core.tool_list",
|
||||
label="What you have",
|
||||
group=GROUP_CORE,
|
||||
order=105,
|
||||
when_tools=True,
|
||||
variables=("tool_names",),
|
||||
requires=("tool_names",),
|
||||
hint="The names of the tools offered on THIS request, which is not the "
|
||||
"same as the tools that exist -- a chat can narrow them, a model's "
|
||||
"capabilities can, a permission can. A model that has to discover its "
|
||||
"own list by calling something and being told it does not exist spends "
|
||||
"a round finding out, and in an ordinary chat that round is the whole "
|
||||
"reply. It is also what stops a model hunting for a skill when there "
|
||||
"are none.",
|
||||
default=(
|
||||
"The tools you have on this request are: {{tool_names}}. That is the whole "
|
||||
"list. Anything not named there does not exist here — calling it costs a "
|
||||
"round and returns nothing."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="core.rounds",
|
||||
label="The round budget",
|
||||
group=GROUP_CORE,
|
||||
order=110,
|
||||
when_tools=True,
|
||||
variables=("max_rounds",),
|
||||
hint="A model that plans six searches gets cut off after three. Better it "
|
||||
"knows the budget than discovers it.",
|
||||
requires=("round_budget",),
|
||||
hint="An ordinary chat only. It gets ONE round of tool calls, and the "
|
||||
"thing worth saying about one round is 'ask for everything at once' — "
|
||||
"which is different in kind from what is true of an agent chat's two "
|
||||
"hundred, not a different number in the same sentence. So this is "
|
||||
"gated on `round_budget`, which `_agent_values` blanks, and the agent "
|
||||
"case is its own fragment below.",
|
||||
default=(
|
||||
"You get at most {{max_rounds}} rounds of tool calls before you have to "
|
||||
"answer with what you have. Several tools can be called in one round. Plan "
|
||||
"within that budget: two careful searches beat six that run out halfway."
|
||||
"You get one round of tool calls, and then you have to answer with what "
|
||||
"came back. Ask for everything you need at once — several tools can be "
|
||||
"called in the same round. If what comes back is not enough, say what you "
|
||||
"would look up next rather than answering as though it were."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="core.keep_working",
|
||||
label="Working until it is done",
|
||||
group=GROUP_CORE,
|
||||
order=111,
|
||||
families=("agent",),
|
||||
hint="An agent chat only, and the counterpart to the round budget above. "
|
||||
"A model told it has a budget rations it and stops early to report "
|
||||
"progress; the step count here is a runaway backstop, not an "
|
||||
"allowance, and saying so is what makes a long piece of work run.",
|
||||
default=(
|
||||
"Keep working until the task is actually done. You are not rationing a "
|
||||
"round budget: call tools as many times as the work needs, one step "
|
||||
"informing the next. What ends a reply is finishing it, being stopped, or "
|
||||
"running past the time and output an administrator allowed — and if that "
|
||||
"happens you are told so and can be asked to carry on. Do not stop halfway "
|
||||
"to report progress and wait to be told to continue."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
|
||||
Reference in New Issue
Block a user