Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Agent chats run commands, and stop to ask first

Browse Source 
The four tools an agent chat has -- shell_run, file_read, file_write,
file_list -- and the mode table wired into the loop that decides which of
them stop for approval. Verified end to end against a real Kali container
over SSH: the card shows the command, allowing it runs it there, and the
file it writes is visible from outside.

The mode is enforced in `_authorise`, in the generation loop, server-side,
keyed on each tool's declared risk. Not in the prompt: a model is told
which mode it is in so it behaves sensibly, but everything it reads -- a
web page, a README, the output of the last command -- is untrusted, and a
rule written only into a system message is one a poisoned file can argue
with. Within an agent chat every call goes through the table, including
the built-in ones, because notes_edit writes and Plan mode meaning "look
but do not touch" has to mean that too.

Two things this turned up.

The runners re-check the mode as a backstop, and that backstop refused the
very thing a person had just approved -- the mode says "ask", and asking
was exactly what happened. Approval is now threaded per call, on a copy of
the context, because a round runs its calls together and only some of them
were allowed.

And the harness said nothing at all, because `registry` maps an offered
tool *name* back to a family and did not know the agent tools existed. So
shell_run resolved to no family and the fragment naming the machine, the
directory and the mode was never admitted. The same omission cost custom
tools their guidance once already; there is a test for it now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-02 00:08:48 +02:00
parent 191394fa08
commit a064407fa7
14 changed files with 1579 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/services/generation.py
+211 -22
View File
@@ -21,7 +21,7 @@ import json
import logging
import time
import uuid
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from datetime import UTC, datetime, timedelta
from sqlalchemy import select
@@ -34,6 +34,7 @@ from lembas.services import interaction, tokens
from lembas.services import metrics as metrics_service
from lembas.services import prompts as prompts_service
from lembas.services import tools as tools_service
from lembas.services.agent import policy as agent_policy
from lembas.services.llm.openai_client import (
LLMError,
chunk_usage,
@@ -120,6 +121,10 @@ class Generation:
# Seconds spent waiting for a person, cumulative. Taken off the wall-clock
# budget so that thinking time is the model's and not the reader's.
waited: float = 0.0
# How much tool output this reply has handed back, against the agent budget.
# A model that fills its own context with build logs has no room left to
# answer with.
output_bytes: int = 0
def touch(self) -> None:
self.version += 1
@@ -273,6 +278,9 @@ async def _run(generation: Generation) -> None:
endpoint = model_id = None
needs_title = False
title_prompt = ""
# Bound before the try, because the finally clears the credential on it and
# a chat that has been deleted returns before it would otherwise be set.
tool_context = None
try:
# Before the request is assembled, so build_request is called once and
@@ -313,8 +321,25 @@ async def _run(generation: Generation) -> None:
generation.prompt_estimate = tokens.estimate_request(payload)
for round_number in range(tools_service.MAX_ROUNDS + 1):
limits = tool_context.agent.limits if tool_context.agent else None
budget = limits.steps if limits else tools_service.MAX_ROUNDS
for round_number in range(budget + 1):
generation.rounds = round_number + 1
# Checked between rounds, never mid-stream: cutting a reply off in
# the middle of a sentence to enforce a budget produces garbage, and
# Stop already covers the mid-stream case. Time spent waiting for a
# person is subtracted -- somebody who thinks for ten minutes about
# one command should not thereby spend the whole allowance.
if limits is not None and round_number:
spent = (time.monotonic() - started) - generation.waited
if spent > limits.wall_seconds:
_gave_up(generation, f"after {spent / 60:.0f} minutes")
break
if generation.output_bytes > limits.output_bytes:
_gave_up(generation, "with too much output to read")
break
accumulator = tools_service.ToolCallAccumulator()
# Text the model produced in *this* round, needed separately from
# generation.content when echoing the assistant turn back.
@@ -380,8 +405,8 @@ async def _run(generation: Generation) -> None:
"name": calls[0]["name"],
"status": "error",
"error": (
f"Stopped after {tools_service.MAX_ROUNDS} rounds of tool "
f"calls without an answer."
f"Stopped after {budget} rounds of tool calls "
f"without an answer."
),
}
)
@@ -397,20 +422,23 @@ async def _run(generation: Generation) -> None:
# together under a semaphore, and four people-shaped pauses inside
# that gather would queue behind each other invisibly -- see
# services/interaction.py.
decided = await _authorise(generation, tool_context, calls)
decided, allowed = await _authorise(generation, tool_context, calls)
if generation.stopped:
break
generation.status = _tool_status(calls)
generation.touch()
try:
outcomes = await _run_calls(tool_context, calls, decided=decided)
outcomes = await _run_calls(
tool_context, calls, decided=decided, allowed=allowed
)
finally:
generation.status = ""
generation.touch()
for call, outcome in zip(calls, outcomes, strict=True):
generation.tool_events.append(outcome.event)
generation.output_bytes += len(outcome.content)
messages.append(tools_service.tool_turn(call, outcome.content))
generation.touch()
@@ -463,6 +491,14 @@ async def _run(generation: Generation) -> None:
)
title = title or chat_service.fallback_title(question)
# The decrypted SSH credential dies with the reply rather than with the
# object holding it. A finished Generation lingers KEEP_FINISHED so a
# follower arriving at the last moment still gets the final frames, and
# a private key should not sit in memory for five minutes waiting on
# that.
if tool_context is not None and getattr(tool_context, "agent", None) is not None:
tool_context.agent.clear()
# Written *before* `done`, because `_follow` breaks out of its loop the
# moment it sees that flag and immediately re-renders the bubble from
# the row. The other order left a window in which the finished frame
@@ -539,6 +575,25 @@ async def _maybe_compact(generation: Generation) -> None:
MAX_PARALLEL_TOOLS = 4
def _gave_up(generation, why: str) -> None:
"""Stop, and leave something in the transcript saying why.
A reply that simply stopped would look like the model losing interest. The
event is the same shape the out-of-rounds branch uses, so it renders with
everything else.
"""
generation.tool_events.append(
{
"name": "budget",
"kind": "agent",
"status": "error",
"results": [],
"error": f"Stopped {why}. Ask again to carry on from here.",
}
)
generation.touch()
def _tool_status(calls: list[dict]) -> str:
"""What to show while tools run.
@@ -550,6 +605,81 @@ def _tool_status(calls: list[dict]) -> str:
return f"Running {len(calls)} tools…"
def _arguments_of(call: dict) -> dict:
try:
args = json.loads(call["arguments"] or "{}")
except json.JSONDecodeError:
return {}
return args if isinstance(args, dict) else {}
def _describe(name: str, args: dict) -> tuple[str, str]:
"""What an approval card says about one call: a title, and the detail.
The detail is the thing being agreed to -- the command line, the path -- and
is shown verbatim and escaped. A summary that paraphrased it would be a card
approving something other than what runs.
"""
if name == "shell_run":
return "Run a command", str(args.get("command") or "")
if name == "file_write":
return "Write a file", str(args.get("path") or "")
if name == "file_read":
return "Read a file", str(args.get("path") or "")
if name == "file_list":
return "List a directory", str(args.get("path") or "")
detail = ", ".join(f"{k}={v!r}" for k, v in list(args.items())[:4])
return f"Use {name}", detail[:400]
def _approvals(context, calls: list[dict]) -> list[interaction.Item]:
"""The calls in this round that a person has to allow before they run.
Only in an agent chat: `context.agent` is None everywhere else, and an
ordinary conversation behaves exactly as it did. Within one, *every* call
goes through the table, including the built-in ones -- `notes_edit` writes,
and Plan mode meaning "look but do not touch" has to mean that too.
"""
agent = getattr(context, "agent", None)
if agent is None:
return []
book = context.tools if context.tools is not None else tools_service.REGISTRY
items: list[interaction.Item] = []
for index, call in enumerate(calls):
tool = book.get(call["name"])
if tool is None or tool.risk == tools_service.RISK_ASK:
continue # unknown names are refused by run_tool; questions are their own card
args = _arguments_of(call)
command = str(args.get("command") or "") if call["name"] == "shell_run" else ""
decision = agent_policy.decide(
mode=agent.mode,
risk=tool.risk,
tool_name=call["name"],
command=command,
allow=agent.allow,
deny=agent.deny,
)
if decision.verdict == agent_policy.ALLOW:
continue
title, detail = _describe(call["name"], args)
items.append(
interaction.Item(
index=index,
key=f"a{index}",
kind=interaction.KIND_APPROVAL,
tool_name=call["name"],
title=f"{title} on {agent.label}",
detail=detail,
reason=decision.reason,
)
)
return items
def _ask_items(context, calls: list[dict]) -> list[interaction.Item]:
"""Which of this round's calls need a person, and what to show about each.
@@ -565,12 +695,7 @@ def _ask_items(context, calls: list[dict]) -> list[interaction.Item]:
tool = book.get(call["name"])
if tool is None or tool.risk != tools_service.RISK_ASK:
continue
try:
args = json.loads(call["arguments"] or "{}")
except json.JSONDecodeError:
args = {}
if not isinstance(args, dict):
args = {}
args = _arguments_of(call)
for asked in _questions_in(args):
options = [str(o).strip() for o in (asked.get("options") or []) if str(o).strip()]
@@ -617,7 +742,9 @@ def _questions_in(args: dict) -> list[dict]:
return out
async def _authorise(generation, context, calls: list[dict]) -> dict[int, ToolOutcome]:
async def _authorise(
generation, context, calls: list[dict]
) -> tuple[dict[int, ToolOutcome], set[int]]:
"""Which of this round's calls may run, and what the others answer instead.
Returns outcomes keyed by the call's index. Every index the caller does not
@@ -625,10 +752,17 @@ async def _authorise(generation, context, calls: list[dict]) -> dict[int, ToolOu
the runner being reached at all. That is what keeps
`zip(calls, outcomes, strict=True)` aligned -- an endpoint matching on
`tool_call_id` pairs the wrong content with the right id otherwise.
Also returns the indices a person explicitly allowed, so the runners can be
told. They re-check the mode as a backstop and would otherwise refuse the
very thing that was just approved -- the mode says "ask", and asking is what
happened.
"""
items = _ask_items(context, calls)
questions = _ask_items(context, calls)
approvals = _approvals(context, calls)
items = [*approvals, *questions]
if not items:
return {}
return {}, set()
timeout = float(context.interaction_timeout or 900)
pause = interaction.build(uuid.uuid4().hex, items, timeout=timeout)
@@ -638,14 +772,56 @@ async def _authorise(generation, context, calls: list[dict]) -> dict[int, ToolOu
if reply.ended:
generation.stopped = True
return {}
return {}, set()
# Grouped back by call, because one `ask_user` call may have carried several
# questions and the endpoint expects exactly one tool turn per call.
decided: dict[int, ToolOutcome] = {}
allowed: set[int] = set()
# An approval that came back as a refusal answers its call without the
# runner being reached; one that came back allowed is simply left out, which
# is how `_run_calls` is told to go ahead.
for item in approvals:
if reply.permitted:
allowed.add(item.index)
continue
decided[item.index] = _not_allowed(item, reply)
# Questions are grouped back by call, because one `ask_user` call may have
# carried several and the endpoint expects exactly one tool turn per call.
grouped: dict[int, list[interaction.Item]] = {}
for item in items:
for item in questions:
grouped.setdefault(item.index, []).append(item)
return {index: _answered(asked, reply) for index, asked in grouped.items()}
for index, asked in grouped.items():
decided[index] = _answered(asked, reply)
return decided, allowed
def _not_allowed(item: interaction.Item, reply: interaction.Reply) -> ToolOutcome:
"""What the model is told when a person declined, or never answered.
Told plainly, and told to stop rather than to try again: a model that reads
"not allowed" as "not allowed *that way*" will spend the rest of the reply
looking for a way round, which is the opposite of what the refusal meant.
"""
event = {
"name": item.tool_name,
"kind": "agent",
"label": item.title,
"query": item.detail,
"results": [],
}
if reply.outcome == interaction.EXPIRED:
return ToolOutcome(
"Nobody answered, so this was not run. Stop and say what you were "
"about to do and why.",
{**event, "status": "error", "error": "Not answered."},
)
return ToolOutcome(
"They declined this. Do not try it another way — say what you were "
"going to do and ask what they would prefer.",
{**event, "status": "error", "error": "Declined."},
)
def _answered(items: list[interaction.Item], reply: interaction.Reply) -> ToolOutcome:
@@ -688,7 +864,11 @@ def _answered(items: list[interaction.Item], reply: interaction.Reply) -> ToolOu
async def _run_calls(
context, calls: list[dict], *, decided: dict[int, ToolOutcome] | None = None
context,
calls: list[dict],
*,
decided: dict[int, ToolOutcome] | None = None,
allowed: set[int] | None = None,
) -> list:
"""Run one round's calls together, results in call order.
@@ -713,8 +893,17 @@ async def _run_calls(
# occupies its index, because the tool turns have to line up.
if decided and index in decided:
return decided[index]
# A per-call copy for anything a person allowed, so the runner's own
# check does not undo their decision. A copy rather than a flag on the
# shared context, because a round runs its calls together and only some
# of them were approved.
ctx = context
if allowed and index in allowed and getattr(context, "agent", None) is not None:
ctx = replace(context, agent=context.agent.as_approved())
async with limit:
return await tools_service.run_tool(context, call["name"], call["arguments"])
return await tools_service.run_tool(ctx, call["name"], call["arguments"])
return list(await asyncio.gather(*(one(i, c) for i, c in enumerate(calls))))