Two controls that did nothing, and instructions worth reading
**Switching mode mid-reply did nothing.** The mode was snapshotted when the reply began, so changing to Auto during a long agent reply went on asking about every call until the next turn. The same snapshot held the chat's allow list, which means "Always allow this" was accepted, written to the row, and then ignored for the rest of the reply that had just asked about it -- the same bug, in the quieter place nobody reported. `agent/session.py:refresh` re-reads exactly those two, between rounds and never within one. A round's calls are authorised together, so a switch must not retroactively approve what is already queued -- which is the property the reply-long snapshot was protecting by accident, and the reason this is not simply moved into `_authorise`. It mutates in place, because `as_approved` copies field references and a replacement would leave the round's approved copy pointing at the old context. **The composer's highlighting stayed behind after sending.** htmx fires afterSwap and afterSettle *before* afterRequest, and the composer empties itself from `hx-on::after-request` -- so every repaint ran while the box still held the message. It repaints on afterRequest and on `reset` as well now, deferred a frame: a form's reset event fires before its fields are actually cleared, so reading the value in the same turn paints the text that is about to vanish. Driven under a DOM stub reproducing htmx's real ordering, and confirmed to fail without the fix. **plan_update, audited.** It never said to mark a task `doing`, so the plan only ever showed work already finished, which is the opposite of "what somebody reads to see where you are". It never said several changes fit in one call, so a model spends a round per task. And `done` now means checked rather than written. **New: core.engineering**, an agent-chat fragment about conduct rather than about any language -- run what you write, find the project's own build and test commands rather than guessing, read before editing, change one thing at a time, read the error instead of guessing at a fix, do not broaden an except to make output clean, and say what you did not check. Every line is about the gap between having written something and knowing it works, which is the gap a model closes by asserting. That pushed the shipped harness to within 1,300 characters of its ceiling, where crossing it silently severs the project's own AGENTS.md. The ceiling is 20,000 and the test pins a margin as well as a fit -- the headroom is also where an administrator's own wording goes, and an override is usually longer than the default it replaces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -160,6 +160,37 @@ def _allow_for(chat: Chat) -> tuple[str, ...]:
|
||||
return tools_service.scoped_allow(chat)
|
||||
|
||||
|
||||
def refresh(db: DBSession, agent: AgentContext) -> AgentContext:
|
||||
"""Re-read the two things a person can change while a reply is running.
|
||||
|
||||
The mode and the chat's own allow list, and nothing else. Everything else on
|
||||
the context is fixed for the life of a chat (the connection, the directory)
|
||||
or is an instance setting nobody is editing mid-reply.
|
||||
|
||||
Called once per round rather than once per reply. The reply-long snapshot it
|
||||
replaces made both controls do nothing until the next turn: switching to
|
||||
Auto during a long agent reply went on asking about every call, and
|
||||
"Always allow this" was accepted, written to the row, and then ignored for
|
||||
the rest of the reply that had just asked. Both look exactly like a control
|
||||
that does not work, because for that reply they were.
|
||||
|
||||
Once per *round* and not more often, because a round's calls are authorised
|
||||
together: what is already queued was decided under the mode that was in
|
||||
force when it was queued, and switching to Auto must not retroactively
|
||||
approve it. Mutated in place -- `as_approved` copies field references, so a
|
||||
replacement here would leave the approved copy of this round pointing at the
|
||||
old one.
|
||||
"""
|
||||
chat = db.get(Chat, agent.chat_id)
|
||||
if chat is None:
|
||||
return agent
|
||||
|
||||
agent.mode = chat.agent_mode if chat.agent_mode in policy.MODES else policy.MODE_MANUAL
|
||||
instance = settings_store.agents(db)
|
||||
agent.allow = (*(instance.get("allow_default") or ()), *_allow_for(chat))
|
||||
return agent
|
||||
|
||||
|
||||
def resolve(db: DBSession, chat: Chat, user: User | None) -> AgentContext | None:
|
||||
"""This chat's agent setup, or None if it has none it can use.
|
||||
|
||||
|
||||
@@ -1015,14 +1015,23 @@ def tool_defs(context: AgentContext | None = None) -> list[ToolDef]:
|
||||
name="plan_update",
|
||||
family=FAMILY_AGENT,
|
||||
description=(
|
||||
"Keep the plan current while you carry it out. Call it when a "
|
||||
"task finishes, when something you find changes what needs doing, "
|
||||
"and when a task turns out to be unnecessary — as you go, not at "
|
||||
"the end. The plan is what somebody reads to see where you are.\n"
|
||||
"Keep the plan current while you carry it out. The plan is what "
|
||||
"somebody reads to see where you are, so it has to be updated as "
|
||||
"you go and not written up at the end.\n"
|
||||
"\n"
|
||||
"Quote the ids from the plan in your prompt: tasks are t1, t2 and "
|
||||
"so on, objectives are o1. This does not end your turn; carry on "
|
||||
"with the work afterwards."
|
||||
"Mark a task 'doing' when you start it and 'done' when you have "
|
||||
"checked it actually works — not when you have written the code "
|
||||
"for it. Use 'dropped' for a task that turned out to be "
|
||||
"unnecessary, and say why in its note. Add tasks the plan did "
|
||||
"not anticipate as you discover them.\n"
|
||||
"\n"
|
||||
"One call carries as many changes as you like: finishing one "
|
||||
"task and starting the next is a single call, not two. Use the "
|
||||
"ids exactly as they appear in the plan above — tasks are t1, "
|
||||
"t2 and so on, objectives o1, phases p1.\n"
|
||||
"\n"
|
||||
"This does not end your turn and is not a progress report to "
|
||||
"stop after. Carry straight on with the work."
|
||||
),
|
||||
parameters={
|
||||
"type": "object",
|
||||
|
||||
@@ -36,6 +36,7 @@ from lembas.services import metrics as metrics_service
|
||||
from lembas.services import prompts as prompts_service
|
||||
from lembas.services import tools as tools_service
|
||||
from lembas.services.agent import policy as agent_policy
|
||||
from lembas.services.agent import session as agent_session
|
||||
from lembas.services.agent import tools as agent_tools
|
||||
from lembas.services.llm.openai_client import (
|
||||
LLMError,
|
||||
@@ -632,6 +633,15 @@ async def _run(generation: Generation) -> None:
|
||||
# card, `policy.decide`, and the runner. See `_arguments_for`.
|
||||
arguments = _arguments_for(tool_context, calls)
|
||||
|
||||
# The mode and the chat's allow list, re-read. Both are things a
|
||||
# person changes *while watching this reply*, and both were
|
||||
# snapshotted for its whole life -- so switching to Auto went on
|
||||
# asking about every call, and "Always allow this" was stored and
|
||||
# then ignored until the next turn. Between rounds, never within
|
||||
# one: what this round has already queued was decided under the mode
|
||||
# that was in force when it was queued.
|
||||
_refresh_agent(tool_context)
|
||||
|
||||
# Decided before anything runs, never during. A round's calls run
|
||||
# together under a semaphore, and four people-shaped pauses inside
|
||||
# that gather would queue behind each other invisibly -- see
|
||||
@@ -952,6 +962,23 @@ def _gave_up(generation, why: str) -> None:
|
||||
generation.touch()
|
||||
|
||||
|
||||
def _refresh_agent(context) -> None:
|
||||
"""Pick up a mode or an allow-list change made while this reply is running.
|
||||
|
||||
Its own short session: `ToolContext` is a session-free snapshot precisely so
|
||||
nothing in a tool holds a live one, and this is one primary-key lookup plus
|
||||
a settings read on a loop that is already doing network work per round.
|
||||
|
||||
Silent on failure. A chat deleted mid-reply is not a reason to fail the
|
||||
reply, and the reply is about to end anyway.
|
||||
"""
|
||||
agent = getattr(context, "agent", None)
|
||||
if agent is None:
|
||||
return
|
||||
with contextlib.suppress(Exception), session_scope() as db:
|
||||
agent_session.refresh(db, agent)
|
||||
|
||||
|
||||
def _wrap_up(generation, why: str, payload: dict, *, name: str = "budget") -> tuple[list, dict]:
|
||||
"""A budget has run out. Withdraw the tools and ask for an answer.
|
||||
|
||||
|
||||
@@ -64,7 +64,19 @@ log = logging.getLogger(__name__)
|
||||
# so they are bounded whatever this is. What this bounds is the *fragments*
|
||||
# growing without anybody noticing -- so it is set above the sum of what those
|
||||
# budgets grant, with room for the plan and the memories beside them.
|
||||
MAX_HARNESS_CHARS = 16000
|
||||
#
|
||||
# 20,000 rather than 16,000, which the shipped set had grown to within 1,300
|
||||
# characters of. A ceiling this close to the content is one the next fragment
|
||||
# crosses, and crossing it is silent: `assemble` cuts the tail, and the tail is
|
||||
# the project's own AGENTS.md. `tests/test_harness.py` pins a margin now as well
|
||||
# as a fit, so the room is a fact rather than a hope.
|
||||
MAX_HARNESS_CHARS = 20000
|
||||
|
||||
# How much of the ceiling the shipped fragments may occupy at full budget. The
|
||||
# rest is headroom for an administrator's own wording, which is the thing this
|
||||
# limit exists to leave room for -- an override is usually longer than the
|
||||
# default it replaces, not shorter.
|
||||
HARNESS_MARGIN = 0.2
|
||||
|
||||
# How many attached filenames to name in the prompt. Enough to show what the
|
||||
# tags will look like, few enough that a chat with thirty files does not spend
|
||||
|
||||
@@ -685,6 +685,42 @@ BUILTIN: tuple[Fragment, ...] = (
|
||||
"not stop halfway to report progress and wait to be told to continue."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="core.engineering",
|
||||
label="Working on code",
|
||||
group=GROUP_CORE,
|
||||
order=112,
|
||||
families=("agent",),
|
||||
hint="An agent chat only, where there is a machine to check things on. "
|
||||
"Every line here is about the gap between having written something and "
|
||||
"knowing it works, which is the one a model closes by asserting rather "
|
||||
"than by testing: the failure is not bad code, it is confident code "
|
||||
"nobody ran. Deliberately about *conduct* rather than about any "
|
||||
"language — style belongs to the project, and its own AGENTS.md is "
|
||||
"where a project says so.",
|
||||
default=(
|
||||
"- Working on code, on this machine:\n"
|
||||
" - Run what you write. A script you have not run is a draft, and "
|
||||
"“this should work” is not a result. If you cannot run it, say that "
|
||||
"plainly rather than implying you did.\n"
|
||||
" - Find out how the project is built, tested and linted before "
|
||||
"guessing — a README, a Makefile, a pyproject or package.json — and use "
|
||||
"what is there rather than a command you would have chosen.\n"
|
||||
" - Read a file before changing it, and match what is around you: the "
|
||||
"naming, the error handling, the way the existing code is laid out. Code "
|
||||
"that reads as though it came from somewhere else is a cost even when it "
|
||||
"works.\n"
|
||||
" - Change one thing, check it, then change the next. A dozen edits "
|
||||
"checked at the end leave you without the one that broke it.\n"
|
||||
" - Read what a failure actually says. Guessing at a fix and running it "
|
||||
"again is slower than reading the error once, and it hides the cause.\n"
|
||||
" - Do not silence a problem to make output clean: a broadened except, a "
|
||||
"removed assertion or a skipped test buys a green run and keeps the bug.\n"
|
||||
" - Say what you did and what you checked, including what you could not "
|
||||
"check. If something is still broken, say so — being told a job is "
|
||||
"finished when it is not is worse than being told it is hard."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
key="core.objective",
|
||||
label="Working to an objective",
|
||||
@@ -1115,13 +1151,16 @@ BUILTIN: tuple[Fragment, ...] = (
|
||||
"the point of it is being able to see where things are while they are "
|
||||
"still moving.",
|
||||
default=(
|
||||
"- There is a plan for this work, set out below. Keep it current: call "
|
||||
"plan_update when a task or a phase finishes, when something you find "
|
||||
"changes what needs doing, and when a task turns out to be unnecessary. "
|
||||
"Do it as you go rather than at the end — the plan is what somebody reads "
|
||||
"to see where you are. If what you find makes the plan wrong rather than "
|
||||
"merely incomplete, say so and ask with ask_user rather than quietly "
|
||||
"planning something else."
|
||||
"- There is a plan for this work, set out below. Keep it current with "
|
||||
"plan_update as you go rather than at the end: mark a task “doing” when "
|
||||
"you start it and “done” once you have checked it works, drop one that "
|
||||
"turns out to be unnecessary, and add work the plan did not anticipate "
|
||||
"when you find it. Several changes go in one call. The plan is what "
|
||||
"somebody reads to see where you are, so a plan updated only at the end "
|
||||
"is a report rather than a plan. Updating it is bookkeeping, not a "
|
||||
"milestone — carry straight on with the work afterwards. If what you find "
|
||||
"makes the plan wrong rather than merely incomplete, say so and ask with "
|
||||
"ask_user rather than quietly planning something else."
|
||||
),
|
||||
),
|
||||
Fragment(
|
||||
|
||||
@@ -440,6 +440,22 @@
|
||||
document.addEventListener("htmx:afterSwap", paint);
|
||||
document.addEventListener("htmx:afterSettle", paint);
|
||||
|
||||
/* And after the *request*, a frame later, which is the one that matters on
|
||||
send. htmx fires afterSwap and afterSettle before afterRequest, and the
|
||||
composer empties itself from `hx-on::after-request` -- so every repaint
|
||||
above ran while the box still held the message, and the highlight stayed
|
||||
behind over an empty field until the next keystroke repainted it.
|
||||
|
||||
A frame later for two reasons: `form.reset()` fires its `reset` event
|
||||
*before* the fields are actually cleared, and reading the value in the same
|
||||
turn would paint the text that is about to disappear. */
|
||||
function repaintSoon() {
|
||||
if (window.requestAnimationFrame) window.requestAnimationFrame(paint);
|
||||
else setTimeout(paint, 0);
|
||||
}
|
||||
document.addEventListener("htmx:afterRequest", repaintSoon);
|
||||
document.addEventListener("reset", repaintSoon);
|
||||
|
||||
document.addEventListener("click", function (event) {
|
||||
if (!event.target.closest(".composer-menu") &&
|
||||
!event.target.closest("[data-composer-input]")) hide();
|
||||
|
||||
Reference in New Issue
Block a user