Agent chats run commands, and stop to ask first

The four tools an agent chat has -- shell_run, file_read, file_write,
file_list -- and the mode table wired into the loop that decides which of
them stop for approval. Verified end to end against a real Kali container
over SSH: the card shows the command, allowing it runs it there, and the
file it writes is visible from outside.

The mode is enforced in `_authorise`, in the generation loop, server-side,
keyed on each tool's declared risk. Not in the prompt: a model is told
which mode it is in so it behaves sensibly, but everything it reads -- a
web page, a README, the output of the last command -- is untrusted, and a
rule written only into a system message is one a poisoned file can argue
with. Within an agent chat every call goes through the table, including
the built-in ones, because notes_edit writes and Plan mode meaning "look
but do not touch" has to mean that too.

Two things this turned up.

The runners re-check the mode as a backstop, and that backstop refused the
very thing a person had just approved -- the mode says "ask", and asking
was exactly what happened. Approval is now threaded per call, on a copy of
the context, because a round runs its calls together and only some of them
were allowed.

And the harness said nothing at all, because `registry` maps an offered
tool *name* back to a family and did not know the agent tools existed. So
shell_run resolved to no family and the fragment naming the machine, the
directory and the mode was never admitted. The same omission cost custom
tools their guidance once already; there is a test for it now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-02 00:08:48 +02:00
parent 6a849dc1ec
commit b6aab8de55
14 changed files with 1579 additions and 29 deletions
+70
View File
@@ -134,6 +134,27 @@ VARIABLES: tuple[Variable, ...] = (
"The character limit on a single remembered fact.",
),
Variable("tool_names", "Tool names", "The tools offered on this request, comma separated."),
Variable(
"agent_target",
"Agent machine",
"The connection an agent chat acts on. Empty in an ordinary chat.",
),
Variable(
"agent_dir",
"Project directory",
"Where commands start on that machine, and what relative paths mean.",
),
Variable(
"agent_mode",
"Agent mode",
"Which of Manual, Edit, Auto or Plan is in force, and what it permits.",
),
Variable(
"agent_rewound",
"Rewound at",
"When an agent chat was last edited or regenerated. Empty otherwise, "
"which is what keeps the note about it out of every other reply.",
),
Variable(
"memories",
"Memories",
@@ -754,6 +775,55 @@ BUILTIN: tuple[Fragment, ...] = (
"Read one with skill_get before following it."
),
),
Fragment(
key="tool.agent",
label="Acting on a machine",
group=GROUP_TOOLS,
order=250,
families=("agent",),
variables=("agent_target", "agent_dir", "agent_mode"),
requires=("agent_target",),
hint="Appears in an agent chat. Says which machine, which directory and "
"what the mode permits -- none of which can go in a tool description, "
"because those are schema and cannot change per chat.",
default=(
"### Acting on {{agent_target}}\n"
"\n"
"- You are working on **{{agent_target}}**, in `{{agent_dir}}`. That is "
"where commands start and what a relative path is measured from. "
"Nothing you do reaches the machine LLeMbas itself runs on.\n"
"- **Each command is a fresh shell.** A `cd` in one call is gone by the "
"next, so pass `cwd` instead of chaining directory changes.\n"
"- Nothing can answer a prompt. Pass the flags that make a command "
"non-interactive — `-y`, `--no-input`, `--yes` — rather than waiting "
"for it to ask. On a Debian-derived system `apt-get install` needs an "
"`apt-get update` first or it reports the package as missing.\n"
"- Look before you write. Read a file before replacing it, and list a "
"directory before guessing at a path.\n"
"- {{agent_mode}}\n"
"- If something is refused, say what you were going to do and ask. Do "
"not look for another way round it."
),
),
Fragment(
key="tool.agent_rewound",
label="After a rewind",
group=GROUP_CONTEXT,
order=330,
families=("agent",),
requires=("agent_rewound",),
variables=("agent_rewound", "agent_target"),
hint="Only after a turn in an agent chat was edited or regenerated. The "
"transcript rewinds; the machine does not.",
default=(
"### This conversation was rewound\n"
"\n"
"Turns were edited or regenerated {{agent_rewound}}, but "
"{{agent_target}} was not. Files created or changed by steps no longer "
"in the transcript are still there. Check before assuming anything is "
"unmade."
),
),
# --- Tasks ---------------------------------------------------------------
Fragment(
key="task.title",