Let a command run in the background instead of being killed

An agent command is one blocking conn.run over a per-call connection, killed the
moment it hits its timeout -- so a ten-minute apt install is impossible, which is
exactly what a user hit. This is the substrate for running it detached instead:
the model can ask for background=true, or a command that outlasts its timeout is
kept running rather than killed, and either way the model gets tools to read and
stop it. Opt-in, off by default, under Admin -> Agents; off is byte-for-byte the
old behaviour.

The mechanism has to survive the connection closing (that is the whole premise
of the per-call model), so a job is a setsid-detached process on the far side,
redirected to a remote logfile and an exit-file; LLeMbas reconnects, as always,
to read it later. services/agent/jobs.py holds the wrappers.

Three things in those wrappers are load-bearing and each was got wrong in the
first sketch:

- The command never touches a quoted shell context. sh -c '<cmd>' shatters the
  instant the command contains a quote -- git commit -m 'fix', awk '{…}', sed
  's/…/…/' are the common case, and it is an injection hole besides. So the
  command is base64-encoded in Python and decoded on the far side into a script
  file; it is bytes, never shell syntax.
- The child records its own pid via $$ as its first act, under setsid where it
  is the session leader, so job_stop can kill the whole process group. echo $!
  from the launcher captures the wrong pid.
- The command's exit status comes from the exit-file, never the wrapper's own
  status -- which is ~0 from its trailing rm. Reading the wrapper's status would
  mark every job a success.

A command that finishes in time is indistinguishable from a foreground one --
same output, same wording; the difference shows only when it does not, where
instead of "stopped after Ns" it becomes a job id. Auto-convert is its own
sub-switch: with it off, a timeout stays a hard stop and nothing is left
running, because routing the plain case through the detached wrapper would leave
an orphan running past a stop an administrator asked for.

New agent tools job_output/job_list/job_stop, offered only when the feature is
on (the plan_submit gating pattern); job_stop is RISK_EXECUTE since it kills a
process. A job's files are namespaced by the calling chat's id and the wrappers
are always built from it, so a model in one chat cannot even name another's job.

Tested against a real local /bin/sh rather than the fake echo-the-command sshd
fixture, because the shell logic -- setsid, base64, the wait loop, the child
surviving the wait being cut off -- is the whole of the risk. The auto-wake that
prompts the model back when a job finishes is the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-03 14:15:49 +02:00
parent bf9287493b
commit 89d2d6ebfd
8 changed files with 1033 additions and 31 deletions
+16
View File
@@ -137,6 +137,21 @@ def _agents_defaults() -> dict[str, Any]:
# one thing there is to be objectively wrong about -- a model with no
# plan that says it has finished is believed.
"nudge_unfinished": True,
# Whether a command may run detached, keep running after the reply ends,
# and be checked on later. Off by default, and off means byte-for-byte
# the old behaviour: a command that times out is killed. See
# services/agent/jobs.py.
"background_enabled": False,
# The sub-switch: a timed-out command is left running as a job instead
# of killed. Off leaves the timeout a hard stop and offers only the
# model's explicit `background=true`.
"background_on_timeout": True,
# Whether the model is woken with the result when a job finishes, rather
# than only seeing it when it next runs of its own accord.
"background_notify": True,
# Most background jobs watched at once. Each is a periodic reconnect to
# the far side, so it is a real cost, not a scruple.
"background_max_jobs": 5,
}
@@ -323,4 +338,5 @@ def agents(db: DBSession) -> dict[str, Any]:
values["max_completion_tokens"] = min(
max(int(values.get("max_completion_tokens") or 0), 0), 5_000_000
)
values["background_max_jobs"] = min(max(int(values.get("background_max_jobs") or 0), 1), 100)
return values