One round for a chat, as many as it takes for an agent

Two different jobs were sharing one number. A plain conversation asking a
question is one round of looking things up and then an answer; the rounds after
that were a small model that had decided searching was the answer searching
until the context ran out, at a full request each. MAX_ROUNDS is 1 now. Several
tools can still be called within that round, which is the thing worth telling
the model.

The trade is real and worth naming: a plain chat can no longer search and then
read one of the results, because reading is a second round. That is what an
agent chat is for.

An agent chat is sized by Limits instead, where steps is now a runaway backstop
and not a working budget. It was 40 and it was reached -- a step count low
enough to be the thing that ends a reply is a count that ends it halfway. What
bounds one now is the wall clock and a new completion-token ceiling, with zero
meaning no ceiling, the same convention index_chars already uses.

That ceiling would have been decorative. generation.completion_tokens is only
populated when the endpoint sends a usage block, and llama.cpp, Ollama and
friends never do; the fallback estimate is computed once, in _run's finally,
long after the loop that needs it. So _written takes the larger of reported and
estimated, and there is a test that runs the whole thing against a stream
reporting no usage at all. A limit that works on OpenAI and silently does
nothing everywhere else is the worst kind: one that looks configured.

core.rounds could not stay one fragment. "You get at most N rounds" is not the
same sentence with a different number in it -- a model told it has a budget
rations it and stops early to report progress, which is exactly the behaviour
that strands a long piece of work. So it splits: core.rounds keeps the
one-round case and gates on a new round_budget variable that _agent_values
blanks, and core.keep_working says the other thing to an agent chat.

A queued message during a one-round reply is now never taken mid-reply -- there
is no work under way to steer -- and falls through to _drain, which gives it a
reply of its own. No code change went with that; it falls out of the guard, and
there is a test so that "it happens to work" and "it is meant to work" stop
looking the same.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/api/admin_agents.py

@@ -61,9 +61,10 @@ async def save_agents(
     default_timeout: int = Form(60),
     max_timeout: int = Form(600),
     max_output_bytes: int = Form(64 * 1024),
-    max_steps: int = Form(40),
+    max_steps: int = Form(200),
     max_wall_seconds: int = Form(900),
     max_total_output_bytes: int = Form(1024 * 1024),
+    max_completion_tokens: int = Form(200_000),
     approval_timeout: int = Form(900),
     allow_default: str = Form(""),
     deny_default: str = Form(""),
@@ -75,6 +76,8 @@ async def save_agents(
     terminal_integration: bool = Form(False),
     index_enabled: bool = Form(False),
     index_chars: int = Form(2000),
+    instructions_enabled: bool = Form(False),
+    instructions_chars: int = Form(4000),
 ) -> Response:
     settings_store.update(
         db,
@@ -86,9 +89,11 @@ async def save_agents(
             "default_timeout": min(max(default_timeout, 1), 3600),
             "max_timeout": min(max(max_timeout, 1), 3600),
             "max_output_bytes": min(max(max_output_bytes, 1024), 1024 * 1024),
-            "max_steps": min(max(max_steps, 1), 200),
+            "max_steps": min(max(max_steps, 1), 1000),
             "max_wall_seconds": min(max(max_wall_seconds, 30), 7200),
             "max_total_output_bytes": min(max(max_total_output_bytes, 4096), 8 * 1024 * 1024),
+            # Floor of 0, not 1: zero is how "no ceiling" is said.
+            "max_completion_tokens": min(max(max_completion_tokens, 0), 5_000_000),
             "approval_timeout": min(max(approval_timeout, 60), 3600),
             "allow_default": _lines(allow_default),
             "deny_default": _lines(deny_default),

src/lembas/services/agent/policy.py

@@ -99,14 +99,26 @@ class Decision:
 class Limits:
     """What one agent reply may spend.
 
-    Three axes because they fail differently. Steps stop a loop; wall clock
-    stops a single slow command eating an afternoon; output stops a model
-    filling its own context with build logs and having no room left to answer.
+    Four axes because they fail differently. Wall clock stops a single slow
+    command eating an afternoon; `output_bytes` stops a model filling its own
+    context with build logs and having no room left to answer; and
+    `completion_tokens` stops one that keeps writing.
+
+    `steps` is the odd one out. It is a **runaway backstop, not a working
+    budget** -- an agent reply is meant to run until the task is finished, and a
+    step count low enough to be the thing that ends it is a count that ends it
+    halfway. It was 40, which is a working budget, and it was reached. Anything
+    that wants a real ceiling should set `completion_tokens`, which measures
+    what a long reply actually costs.
+
+    `completion_tokens` of 0 means no ceiling, the same convention `index_chars`
+    uses in the settings store.
     """
 
-    steps: int = 40
+    steps: int = 200
     wall_seconds: float = 900.0
     output_bytes: int = 1024 * 1024
+    completion_tokens: int = 200_000
 
 
 def subject(tool_name: str, command: str = "") -> str | None:

src/lembas/services/agent/session.py

@@ -141,9 +141,12 @@ def resolve(db: DBSession, chat: Chat, user: User | None) -> AgentContext | None
         allow=tuple(values.get("allow_default") or ()),
         deny=tuple(values.get("deny_default") or ()),
         limits=Limits(
-            steps=int(values.get("max_steps") or 40),
+            steps=int(values.get("max_steps") or 200),
             wall_seconds=float(values.get("max_wall_seconds") or 900),
             output_bytes=int(values.get("max_total_output_bytes") or 1024 * 1024),
+            # `or 0` would turn a deliberate 0 into the default, and 0 is how an
+            # administrator says "no ceiling". `agents()` has already clamped it.
+            completion_tokens=int(values.get("max_completion_tokens", 200_000) or 0),
         ),
         timeout=float(values.get("default_timeout") or 60),
         max_timeout=float(values.get("max_timeout") or 600),

src/lembas/services/generation.py

@@ -379,6 +379,10 @@ async def _run(generation: Generation) -> None:
                 if generation.output_bytes > limits.output_bytes:
                     _gave_up(generation, "with too much output to read")
                     break
+                written = _written(generation)
+                if limits.completion_tokens and written > limits.completion_tokens:
+                    _gave_up(generation, f"after writing about {written:,} tokens")
+                    break
             accumulator = tools_service.ToolCallAccumulator()
             # Text the model produced in *this* round, needed separately from
             # generation.content when echoing the assistant turn back.
@@ -446,13 +450,13 @@ async def _run(generation: Generation) -> None:
                 # chat allowed forty steps stopped after three and said it had
                 # taken forty. Two numbers, one of them wrong, in code whose
                 # whole job is to say what happened.
+                howmany = "one round" if budget == 1 else f"{budget} rounds"
                 generation.tool_events.append(
                     {
                         "name": calls[0]["name"],
                         "status": "error",
                         "error": (
-                            f"Stopped after {budget} rounds of tool calls "
-                            f"without an answer."
+                            f"Stopped after {howmany} of tool calls without an answer."
                         ),
                     }
                 )

src/lembas/services/harness.py

@@ -125,6 +125,11 @@ def context_variables(
         "user_name": (user.name or "") if user is not None else "",
         "model_name": "",
         "max_rounds": str(tools_service.MAX_ROUNDS),
+        # Not rendered anywhere. It is the gate on `core.rounds`: an ordinary
+        # chat gets one round and is told to ask for everything at once, an
+        # agent chat is told to keep going, and those are different sentences
+        # rather than the same sentence with a different number in it.
+        "round_budget": str(tools_service.MAX_ROUNDS),
         "memory_limit": str(memories_service.MAX_MEMORY_CHARS),
         "tool_names": _tool_names(offered),
         "memories": memories_service.block(db, user) if "memory" in families else "",
@@ -181,6 +186,10 @@ def _agent_values(db: DBSession, chat, user) -> dict[str, str]:
         "agent_mode": policy.MODE_GUIDANCE.get(context.mode, ""),
         "agent_rewound": rewound,
         "max_rounds": str(context.limits.steps),
+        # Blanked, which is what makes `core.rounds` vanish here: `steps` is a
+        # runaway backstop and telling a model it has a budget of two hundred
+        # invites it to ration one.
+        "round_budget": "",
         "project_files": _project_files(db, chat, context, settings_store, index_service),
     }
 

src/lembas/services/prompts.py

@@ -128,6 +128,13 @@ VARIABLES: tuple[Variable, ...] = (
     Variable("user_name", "User's name", "The name of the person in the conversation."),
     Variable("model_name", "Model", "The display name of the model answering."),
     Variable("max_rounds", "Tool rounds", "How many rounds of tool calls one reply may take."),
+    Variable(
+        "round_budget",
+        "Round budget applies",
+        "Set in an ordinary chat and blank in an agent chat. Nothing renders it; "
+        "it exists so a fragment can say `requires=('round_budget',)` and appear "
+        "for one and not the other.",
+    ),
     Variable(
         "memory_limit",
         "Memory length",
@@ -572,19 +579,64 @@ BUILTIN: tuple[Fragment, ...] = (
             "permission first."
         ),
     ),
+    Fragment(
+        key="core.tool_list",
+        label="What you have",
+        group=GROUP_CORE,
+        order=105,
+        when_tools=True,
+        variables=("tool_names",),
+        requires=("tool_names",),
+        hint="The names of the tools offered on THIS request, which is not the "
+        "same as the tools that exist -- a chat can narrow them, a model's "
+        "capabilities can, a permission can. A model that has to discover its "
+        "own list by calling something and being told it does not exist spends "
+        "a round finding out, and in an ordinary chat that round is the whole "
+        "reply. It is also what stops a model hunting for a skill when there "
+        "are none.",
+        default=(
+            "The tools you have on this request are: {{tool_names}}. That is the whole "
+            "list. Anything not named there does not exist here — calling it costs a "
+            "round and returns nothing."
+        ),
+    ),
     Fragment(
         key="core.rounds",
         label="The round budget",
         group=GROUP_CORE,
         order=110,
         when_tools=True,
-        variables=("max_rounds",),
-        hint="A model that plans six searches gets cut off after three. Better it "
-        "knows the budget than discovers it.",
+        requires=("round_budget",),
+        hint="An ordinary chat only. It gets ONE round of tool calls, and the "
+        "thing worth saying about one round is 'ask for everything at once' — "
+        "which is different in kind from what is true of an agent chat's two "
+        "hundred, not a different number in the same sentence. So this is "
+        "gated on `round_budget`, which `_agent_values` blanks, and the agent "
+        "case is its own fragment below.",
         default=(
-            "You get at most {{max_rounds}} rounds of tool calls before you have to "
-            "answer with what you have. Several tools can be called in one round. Plan "
-            "within that budget: two careful searches beat six that run out halfway."
+            "You get one round of tool calls, and then you have to answer with what "
+            "came back. Ask for everything you need at once — several tools can be "
+            "called in the same round. If what comes back is not enough, say what you "
+            "would look up next rather than answering as though it were."
+        ),
+    ),
+    Fragment(
+        key="core.keep_working",
+        label="Working until it is done",
+        group=GROUP_CORE,
+        order=111,
+        families=("agent",),
+        hint="An agent chat only, and the counterpart to the round budget above. "
+        "A model told it has a budget rations it and stops early to report "
+        "progress; the step count here is a runaway backstop, not an "
+        "allowance, and saying so is what makes a long piece of work run.",
+        default=(
+            "Keep working until the task is actually done. You are not rationing a "
+            "round budget: call tools as many times as the work needs, one step "
+            "informing the next. What ends a reply is finishing it, being stopped, or "
+            "running past the time and output an administrator allowed — and if that "
+            "happens you are told so and can be asked to carry on. Do not stop halfway "
+            "to report progress and wait to be told to continue."
         ),
     ),
     Fragment(

src/lembas/services/settings_store.py

@@ -66,9 +66,19 @@ def _agents_defaults() -> dict[str, Any]:
         "max_timeout": 600,
         "max_output_bytes": 64 * 1024,
         # Per reply. See services/agent/policy.py:Limits.
-        "max_steps": 40,
+        #
+        # `max_steps` is a runaway backstop rather than a working budget: an
+        # agent reply is meant to run until the task is done, and a step count
+        # low enough to be the thing that stops it is a count that stops it
+        # halfway. What actually bounds a long reply is the wall clock and
+        # `max_completion_tokens`.
+        "max_steps": 200,
         "max_wall_seconds": 900,
         "max_total_output_bytes": 1024 * 1024,
+        # How much the model may *write* in one reply, across every round.
+        # Zero means no ceiling, which is a thing somebody may want and has no
+        # other way of being said -- the same convention as `index_chars`.
+        "max_completion_tokens": 200_000,
         # How long a reply waits for someone to answer. Clamped on read: a zero
         # here would park a background task forever.
         "approval_timeout": 900,
@@ -261,4 +271,11 @@ def agents(db: DBSession) -> dict[str, Any]:
     # directory for the file picker, but put none of it in the prompt", which
     # is a reasonable thing to want and has no other way of being said.
     values["index_chars"] = min(max(int(values.get("index_chars") or 0), 0), 20_000)
+    values["instructions_chars"] = min(
+        max(int(values.get("instructions_chars") or 0), 0), 20_000
+    )
+    # Zero is meaningful here too: no ceiling on what one reply may write.
+    values["max_completion_tokens"] = min(
+        max(int(values.get("max_completion_tokens") or 0), 0), 5_000_000
+    )
     return values

src/lembas/services/tools.py

@@ -45,16 +45,31 @@ from lembas.services.search.base import SearchError
 
 log = logging.getLogger(__name__)
 
-# How many times a model may call tools before it has to answer with words.
-# Not a safety limit so much as a termination one: a small model that has
-# decided searching is the answer will otherwise search until the context runs
-# out, and each round costs a full request.
-MAX_ROUNDS = 3
+# How many times a model may call tools before it has to answer with words, in
+# an ORDINARY chat. An agent chat is sized by `agent/policy.py:Limits.steps`
+# instead, which is two orders of magnitude larger, because an agent reply is
+# meant to run until the work is done.
+#
+# One, deliberately. A plain conversation asking a question is one round of
+# looking things up and then an answer; the rounds after that were a small model
+# that had decided searching was the answer searching until the context ran out,
+# at a full request each. Several tools can still be called *within* that round,
+# which is the thing worth telling the model -- see `core.rounds`.
+#
+# The trade is real and worth naming: a chat can no longer search and then read
+# one of the results, because reading is a second round. That is what an agent
+# chat is for.
+MAX_ROUNDS = 1
 
 # Tool families, matching the per-model capability flags and the permission
 # keys. The three names differ by prefix only, which is deliberate: adding a
 # family means adding one entry here and one permission.
 FAMILY_SEARCH = "web_search"
+# Reading one page, given its address. Its own family rather than part of
+# `web_search`: an administrator may reasonably want a model that can look
+# things up but not follow an arbitrary URL it read somewhere, and the SSRF
+# surface is entirely on this side.
+FAMILY_FETCH = "fetch"
 FAMILY_KNOWLEDGE = "knowledge"
 FAMILY_NOTES = "notes"
 FAMILY_MEMORY = "memory"

src/lembas/web/templates/admin/agents.html

@@ -127,15 +127,22 @@
   <section class="card">
     <h2 class="card__title">What one reply may spend</h2>
     <p class="field__hint">
-      Three separate bounds, because they fail differently: steps stop a loop,
-      the clock stops one slow command eating an afternoon, and output stops a
-      model filling its own context with build logs and having no room to answer.
+      Four separate bounds, because they fail differently: the clock stops one
+      slow command eating an afternoon, tool output stops a model filling its own
+      context with build logs and having no room to answer, written tokens stop
+      one that keeps going, and the step count is a backstop against a runaway.
     </p>
 
     <div class="field">
-      <label class="field__label" for="max_steps">Most rounds of tool calls</label>
-      <input class="input" id="max_steps" name="max_steps"
-             value="{{ values.max_steps }}" inputmode="numeric">
+      <label class="field__label" for="max_completion_tokens">
+        Most a reply may write
+      </label>
+      <input class="input" id="max_completion_tokens" name="max_completion_tokens"
+             value="{{ values.max_completion_tokens }}" inputmode="numeric">
+      <p class="field__hint">
+        In tokens, across every round of one reply. This is the bound that
+        normally ends a long piece of work. Zero means no ceiling.
+      </p>
     </div>
     <div class="field">
       <label class="field__label" for="max_wall_seconds">Longest a reply may take</label>
@@ -148,6 +155,16 @@
       <input class="input" id="max_total_output_bytes" name="max_total_output_bytes"
              value="{{ values.max_total_output_bytes }}" inputmode="numeric">
     </div>
+    <div class="field">
+      <label class="field__label" for="max_steps">Most rounds of tool calls</label>
+      <input class="input" id="max_steps" name="max_steps"
+             value="{{ values.max_steps }}" inputmode="numeric">
+      <p class="field__hint">
+        A backstop, not a working budget. An agent reply is meant to run until
+        the task is done, so a number low enough to be what stops it is a number
+        that stops it halfway. Use the token ceiling above for a real limit.
+      </p>
+    </div>
   </section>
 
   <section class="card">