Let a command run in the background instead of being killed

An agent command is one blocking conn.run over a per-call connection, killed the
moment it hits its timeout -- so a ten-minute apt install is impossible, which is
exactly what a user hit. This is the substrate for running it detached instead:
the model can ask for background=true, or a command that outlasts its timeout is
kept running rather than killed, and either way the model gets tools to read and
stop it. Opt-in, off by default, under Admin -> Agents; off is byte-for-byte the
old behaviour.

The mechanism has to survive the connection closing (that is the whole premise
of the per-call model), so a job is a setsid-detached process on the far side,
redirected to a remote logfile and an exit-file; LLeMbas reconnects, as always,
to read it later. services/agent/jobs.py holds the wrappers.

Three things in those wrappers are load-bearing and each was got wrong in the
first sketch:

- The command never touches a quoted shell context. sh -c '<cmd>' shatters the
  instant the command contains a quote -- git commit -m 'fix', awk '{…}', sed
  's/…/…/' are the common case, and it is an injection hole besides. So the
  command is base64-encoded in Python and decoded on the far side into a script
  file; it is bytes, never shell syntax.
- The child records its own pid via $$ as its first act, under setsid where it
  is the session leader, so job_stop can kill the whole process group. echo $!
  from the launcher captures the wrong pid.
- The command's exit status comes from the exit-file, never the wrapper's own
  status -- which is ~0 from its trailing rm. Reading the wrapper's status would
  mark every job a success.

A command that finishes in time is indistinguishable from a foreground one --
same output, same wording; the difference shows only when it does not, where
instead of "stopped after Ns" it becomes a job id. Auto-convert is its own
sub-switch: with it off, a timeout stays a hard stop and nothing is left
running, because routing the plain case through the detached wrapper would leave
an orphan running past a stop an administrator asked for.

New agent tools job_output/job_list/job_stop, offered only when the feature is
on (the plan_submit gating pattern); job_stop is RISK_EXECUTE since it kills a
process. A job's files are namespaced by the calling chat's id and the wrappers
are always built from it, so a model in one chat cannot even name another's job.

Tested against a real local /bin/sh rather than the fake echo-the-command sshd
fixture, because the shell logic -- setsid, base64, the wait loop, the child
surviving the wait being cut off -- is the whole of the risk. The auto-wake that
prompts the model back when a job finishes is the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-03 14:15:49 +02:00
parent a5fa982ae3
commit 3fc3449726
8 changed files with 1033 additions and 31 deletions
+13
View File
@@ -86,6 +86,16 @@ class AgentContext:
# read the same stale plan from the database and the second would lose the
# first. This snapshot is what they actually merge into.
plan: dict[str, Any] = field(default_factory=dict)
# Whether commands may run detached. When off, `shell_run` is byte-for-byte
# what it always was and the `job_*` tools are not offered -- a command that
# times out is killed, as before. When on, a command can be launched in the
# background (or converted to one when it times out) and the model gets the
# tools to check on it. `on_timeout` is the sub-switch for the auto-convert.
background: bool = False
background_on_timeout: bool = True
# Whether a finished job wakes the model on its own, rather than only being
# seen when it next runs. Read by the wording here and by the watcher.
background_notify: bool = True
def executor(self) -> Executor:
return ssh_service.SshExecutor(self.spec, self.project_dir)
@@ -179,6 +189,9 @@ def resolve(db: DBSession, chat: Chat, user: User | None) -> AgentContext | None
timeout=float(values.get("default_timeout") or 60),
max_timeout=float(values.get("max_timeout") or 600),
max_output=int(values.get("max_output_bytes") or 64 * 1024),
background=bool(values.get("background_enabled")),
background_on_timeout=bool(values.get("background_on_timeout", True)),
background_notify=bool(values.get("background_notify", True)),
spec=ssh_service.spec_from(profile),
)