Correcting a command before allowing it

An approval card was Allow, Always, or Don't. A model proposing the right
command with one flag wrong therefore cost a whole round trip to explain in
prose. There is an Edit button on it now.

Where the edit lands is the whole of the feature, and it is one line.
`arguments` is the list `_run_calls` hands to `run_tool` as `parsed=`, and
`run_tool` never re-parses -- so writing into it inside `_authorise` is the
only mutation the runner can see. Editing the Item would do nothing: it is
frozen and display-only.

Two things had to move with it. The raw `call["arguments"]` string is rewritten
beside the parsed dict, and the assistant turn is now built *after* `_authorise`
rather than before it -- the old order told the model it ran what it proposed
while something else ran, and every later round would have reasoned from a
transcript that was quietly false. And `_remember_always` reads the edit, or
"always allow this" would store a standing permission for a command nobody
approved; it still derives the pattern itself through `policy.subject`, which
yields nothing for a composed command line.

Nothing is re-checked against the mode or the lists, and that is not a shortcut.
The deny list resolves to ASK rather than to a refusal -- it means "always ask
about this" -- so a person who has typed the command and pressed Allow is
exactly the asking it was demanding, and re-asking would put the same card up
with no way past it. It is the line the terminal panel already draws.

The box is only offered where the detail *is* an argument and can be put back:
a tool with no entry in `tool_labels.DETAIL_KEYS` gets a `k=repr(v)` summary,
and a box there would silently change nothing. Both halves are always in the
DOM with one hidden, rather than the field being created on click -- a field
that does not exist until a handler runs is a field that submits nothing if
the handler fails, and this one decides what runs on somebody's machine.

The transcript says "edited by you". Attributing somebody's own typing to a
model is the same misattribution as the other way round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 08:40:51 +02:00
parent ec12c3a981
commit ab2e74974b
7 changed files with 620 additions and 40 deletions
+37 -17
View File
@@ -1330,15 +1330,6 @@ async def answer_interaction(
form = await request.form()
verdict = str(form.get("verdict") or "").strip()
# Read and recorded *before* resolving: `interaction.wait_for` clears
# `generation.pending` in its `finally`, so a moment later there is nothing
# left to remember and "always" would quietly mean "once".
remembered = 0
if verdict == interaction.ALLOW_ALWAYS:
remembered = _remember_always(
db, chat, generation_service.pending_items(chat.id, interaction_id)
)
answers: dict[str, str] = {}
for field, value in form.multi_items():
kind, _, key = str(field).partition(".")
@@ -1350,6 +1341,23 @@ async def answer_interaction(
elif kind == "choice" and written:
answers.setdefault(key, written)
# Read and recorded *before* resolving: `interaction.wait_for` clears
# `generation.pending` in its `finally`, so a moment later there is nothing
# left to remember and "always" would quietly mean "once".
#
# `answers` is gathered first because an approval card can now carry a
# corrected command, and "always allow this" has to mean the command that is
# about to run rather than the one the model asked for. Remembering the
# proposed one would grant a standing permission nobody approved.
remembered = 0
if verdict == interaction.ALLOW_ALWAYS:
remembered = _remember_always(
db,
chat,
generation_service.pending_items(chat.id, interaction_id),
answers=answers,
)
answered = generation_service.answer(
chat.id,
interaction_id,
@@ -1377,15 +1385,23 @@ async def answer_interaction(
return response
def _remember_always(db: DBSession, chat: Chat, items) -> int:
def _remember_always(
db: DBSession, chat: Chat, items, *, answers: dict[str, str] | None = None
) -> int:
"""Record what "always allow" was said about. Returns how many were new.
The pattern is derived **here**, from the item that was approved, and never
taken from the request -- the endpoint accepts an interaction id and a
verdict and nothing else. `agent_policy.subject` is the same normaliser
`decide` matches with, so what is stored is exactly what will be compared
later; it returns None for a command line carrying a shell metacharacter,
which is precisely the shape that must never become a standing permission.
The pattern is derived **here**, and still never taken from the request as a
pattern: `answers` carries the command a person may have corrected on the
card, and it goes through `agent_policy.subject` exactly as `item.detail`
does. That is the same normaliser `decide` matches with, so what is stored
is exactly what will be compared later; it returns None for a command line
carrying a shell metacharacter, which is precisely the shape that must never
become a standing permission.
Reading the edit matters rather than being a nicety. Somebody who corrects a
command and presses "always allow" has approved the corrected one, and
storing what the model originally asked for would be a standing permission
for something nobody ever agreed to.
A tool name for everything that is not a command, which is the convention
the shipped `allow_default` already uses: `file_read` and `file_list` are
@@ -1393,12 +1409,16 @@ def _remember_always(db: DBSession, chat: Chat, items) -> int:
"""
scope = dict(chat.scope_json or {})
entries = list(scope.get("allow") or [])
written = answers or {}
added = 0
for item in items:
if item.kind != interaction.KIND_APPROVAL or len(entries) >= MAX_SCOPE_KEYS:
continue
pattern = agent_policy.subject(item.tool_name, item.detail)
detail = item.detail
if item.editable:
detail = (written.get(item.key) or "").strip() or item.detail
pattern = agent_policy.subject(item.tool_name, detail)
if not pattern or pattern in entries:
continue
entries.append(pattern)
+100 -21
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import time
import uuid
@@ -585,15 +586,6 @@ async def _run(generation: Generation) -> None:
generation.touch()
break
messages = [
# The **raw** arguments string, not the parsed dict: the
# endpoint has to see back exactly what it sent, or an
# id-matching server pairs its own call with something it does
# not recognise.
*payload["messages"],
tools_service.assistant_turn(calls, "".join(round_text)),
]
# Parsed once, here, and shared by everything below: the approval
# card, `policy.decide`, and the runner. See `_arguments_for`.
arguments = _arguments_for(tool_context, calls)
@@ -602,10 +594,27 @@ async def _run(generation: Generation) -> None:
# together under a semaphore, and four people-shaped pauses inside
# that gather would queue behind each other invisibly -- see
# services/interaction.py.
decided, allowed = await _authorise(generation, tool_context, calls, arguments)
decided, allowed, edited = await _authorise(
generation, tool_context, calls, arguments
)
if generation.stopped:
break
messages = [
# The **raw** arguments string, not the parsed dict: the
# endpoint has to see back exactly what it sent, or an
# id-matching server pairs its own call with something it does
# not recognise.
#
# Built after `_authorise` rather than before it, because a
# command corrected on the approval card is written back into
# `calls` there. The other order sent the model the command it
# proposed while a different one ran, and every later round
# reasoned from a transcript that was quietly false.
*payload["messages"],
tools_service.assistant_turn(calls, "".join(round_text)),
]
generation.status = _tool_status(calls)
generation.touch()
try:
@@ -616,7 +625,14 @@ async def _run(generation: Generation) -> None:
generation.status = ""
generation.touch()
for call, outcome in zip(calls, outcomes, strict=True):
for index, (call, outcome) in enumerate(zip(calls, outcomes, strict=True)):
if index in edited:
# A command somebody corrected on the card is theirs, not
# the model's. Shown as such, for the same reason a plan
# goes back quoted and attributed: text must not arrive
# wearing an authorship it does not have, in either
# direction.
outcome.event["edited"] = True
generation.tool_events.append(outcome.event)
generation.output_bytes += len(outcome.content)
messages.append(tools_service.tool_turn(call, outcome.content))
@@ -1102,6 +1118,12 @@ def _approvals(context, calls: list[dict], arguments: list[dict]) -> list[intera
detail=detail,
reason=decision.reason,
purpose=agent_tools.why_of(args),
# A card showing one argument can offer to correct it. A model
# proposing the right command with one flag wrong is the common
# case, and Allow-or-Don't makes that a whole round trip to
# explain. Anything whose detail is a summary rather than a
# value cannot be put back and is not offered the box.
editable=bool(tool_labels.DETAIL_KEYS.get(call["name"])),
)
)
return items
@@ -1171,12 +1193,13 @@ def _questions_in(args: dict) -> list[dict]:
async def _authorise(
generation, context, calls: list[dict], arguments: list[dict]
) -> tuple[dict[int, ToolOutcome], set[int]]:
) -> tuple[dict[int, ToolOutcome], set[int], set[int]]:
"""Which of this round's calls may run, and what the others answer instead.
Returns outcomes keyed by the call's index. Every index the caller does not
find here is cleared to run; every index it does find is answered without
the runner being reached at all. That is what keeps
Returns outcomes keyed by the call's index, the indices a person allowed,
and the indices whose command they corrected on the way. Every index the
caller does not find in the first is cleared to run; every index it does
find is answered without the runner being reached at all. That is what keeps
`zip(calls, outcomes, strict=True)` aligned -- an endpoint matching on
`tool_call_id` pairs the wrong content with the right id otherwise.
@@ -1184,12 +1207,17 @@ async def _authorise(
told. They re-check the mode as a backstop and would otherwise refuse the
very thing that was just approved -- the mode says "ask", and asking is what
happened.
A command corrected on the card is written back into `arguments` **in
place**, because that same list is what `_run_calls` hands to `run_tool` as
`parsed=` and `run_tool` never re-parses. Editing the item would do nothing:
`Item` is display-only and frozen. This is the one place the two meet.
"""
questions = _ask_items(context, calls, arguments)
approvals = _approvals(context, calls, arguments)
items = [*approvals, *questions]
if not items:
return {}, set()
return {}, set(), set()
timeout = float(context.interaction_timeout or 900)
pause = interaction.build(uuid.uuid4().hex, items, timeout=timeout)
@@ -1199,19 +1227,25 @@ async def _authorise(
if reply.ended:
generation.stopped = True
return {}, set()
return {}, set(), set()
decided: dict[int, ToolOutcome] = {}
allowed: set[int] = set()
edited: set[int] = set()
# An approval that came back as a refusal answers its call without the
# runner being reached; one that came back allowed is simply left out, which
# is how `_run_calls` is told to go ahead.
for item in approvals:
if reply.permitted:
allowed.add(item.index)
if not reply.permitted:
decided[item.index] = _not_allowed(item, reply)
continue
decided[item.index] = _not_allowed(item, reply)
if _apply_edit(calls, arguments, item, reply) != item.detail:
# So the transcript can say the command was changed before it ran.
# Without it a reader scrolling back sees a command attributed to
# the model that the model never wrote.
edited.add(item.index)
allowed.add(item.index)
# Questions are grouped back by call, because one `ask_user` call may have
# carried several and the endpoint expects exactly one tool turn per call.
@@ -1221,7 +1255,52 @@ async def _authorise(
for index, asked in grouped.items():
decided[index] = _answered(asked, reply)
return decided, allowed
return decided, allowed, edited
def _apply_edit(
calls: list[dict],
arguments: list[dict],
item: interaction.Item,
reply: interaction.Reply,
) -> str:
"""Put a corrected command back where the runner will find it.
Returns what is going to run, edited or not, so the caller can record the
right thing. Two writes, and both are needed:
`arguments[index]` is what `run_tool` is handed as `parsed=`, and it never
re-parses -- so this is the only write that reaches the runner. Editing the
item would do nothing at all: `Item` is frozen and display-only.
`call["arguments"]`, the raw string, is rewritten beside it, because that is
what goes back to the endpoint as the assistant turn. Otherwise the model is
told it ran what it proposed rather than what actually ran, and every later
round reasons from a transcript that is quietly false.
Nothing is re-checked against the mode or the lists. That is the same line
the terminal panel and the directory browser draw, and here it is not even
close: the deny list resolves to ASK rather than to a refusal -- it means
"always ask about this" -- and a person who has typed the command themselves
and pressed Allow is exactly the asking it was demanding. Re-asking would
put the same card up again with no way past it. The instance's list still
governs the *model*: a pattern remembered by "always allow" is checked by
`decide`, where a deny hit wins before the allow list is even read.
"""
if not item.editable:
return item.detail
edited = reply.answer_to(item)
key = tool_labels.DETAIL_KEYS.get(item.tool_name)
if not edited or edited == item.detail or not key:
return item.detail
arguments[item.index] = {**arguments[item.index], key: edited}
calls[item.index] = {
**calls[item.index],
"arguments": json.dumps(arguments[item.index]),
}
return edited
def _not_allowed(item: interaction.Item, reply: interaction.Reply) -> ToolOutcome:
+6
View File
@@ -85,6 +85,12 @@ class Item:
purpose: str = ""
options: tuple[str, ...] = ()
allow_free_text: bool = True
# Whether `detail` can be corrected before this is allowed. Only where the
# detail *is* one argument and can be put back where it came from -- a tool
# with no entry in `tool_labels.DETAIL_KEYS` gets a `k=repr(v)` summary that
# cannot be parsed back, and offering a box that silently changed nothing
# would be worse than offering none.
editable: bool = False
@dataclass
+12
View File
@@ -521,6 +521,18 @@
white-space: pre-wrap;
overflow-wrap: anywhere;
}
/* The same command, correctable. Sized and spaced like the <pre> it replaces
so pressing Edit does not make the card jump. */
.interaction__edit {
width: 100%;
margin: 0 0 var(--sp-2);
padding: var(--sp-3);
font-family: var(--font-mono);
font-size: var(--text-xs);
white-space: pre-wrap;
overflow-wrap: anywhere;
resize: vertical;
}
.interaction__reason { margin: 0; color: var(--ink-muted); font-size: var(--text-xs); }
/* The model's account of what it is about to do. Above the command and quieter
than the title, so the command stays the thing being agreed to. */
@@ -68,7 +68,7 @@
{% else %}
{% for item in ask.items %}
<div class="interaction__question">
<div class="interaction__question" x-data="{ editing: false }">
<p class="interaction__title">{{ item.title }}</p>
{% if item.purpose %}
{# The model's own account of what this is for, above the thing itself.
@@ -78,7 +78,44 @@
<p class="interaction__purpose">It says: {{ item.purpose }}</p>
{% endif %}
{% if item.detail %}
<pre class="interaction__detail">{{ item.detail }}</pre>
{% if item.editable %}
{#
The command, correctable before it runs. A model proposing the right
thing with one flag wrong is the common case, and Allow-or-Don't
makes that a round trip to explain in prose.
Both halves are always in the DOM and one is hidden, rather than the
box being created when Edit is pressed: a field that does not exist
until a click is a field that submits nothing if the click handler
ever fails, and this one decides what runs on somebody's machine.
The textarea is disabled while hidden so an untouched card cannot
post a `text.` field at all — that field means "this was edited",
and an empty one arriving would be indistinguishable from a command
somebody cleared.
#}
<div x-show="!editing">
<pre class="interaction__detail">{{ item.detail }}</pre>
<button class="btn btn--sm" type="button"
@click="editing = true; $nextTick(() => $refs.edit{{ item.key }}.focus())">
{{ icon("pencil", "icon--sm") }} Edit
</button>
</div>
<div x-show="editing" x-cloak>
<label class="visually-hidden" for="edit-{{ item.key }}">
Change this before it runs
</label>
<textarea class="textarea interaction__edit" id="edit-{{ item.key }}"
name="text.{{ item.key }}" rows="3" spellcheck="false"
x-ref="edit{{ item.key }}"
:disabled="!editing">{{ item.detail }}</textarea>
<p class="interaction__reason">
Allow runs what is in the box. It is not checked against this
chat's rules again — you typed it.
</p>
</div>
{% else %}
<pre class="interaction__detail">{{ item.detail }}</pre>
{% endif %}
{% endif %}
{% if item.reason %}
<p class="interaction__reason">{{ item.reason }}</p>
@@ -62,6 +62,15 @@
</span>
{% endif %}
{% if event.edited %}
{# Corrected on the approval card before it ran, so what is shown above is
the reader's command and not the model's. Said out loud rather than
left to be inferred: attributing somebody's own typing to a model is
the same misattribution as the other way round, and a transcript read
back a week later has nothing else to go on. #}
<span class="badge">edited by you</span>
{% endif %}
{% if event.why %}
{# What the model said this call was for. In the summary rather than the
body because the body is collapsed: in Auto mode nothing stops for