Wake the model when a background job finishes
The other half of background execution: a job that finishes while nobody is looking prompts the model back with its result, rather than sitting unread until the model happens to run again. The vehicle is the queue, because it is the only wiring that already delivers a turn into or after a reply. A per-job poller notices completion and calls jobs.wake. If a reply is being written the completion is left queued for that reply's _inject/_drain; if the chat is idle a fresh reply is started to answer it -- the send_queued_now move. All of it under a per-chat lock with no await between the running-check and ensure, so two jobs finishing at once cannot each spin up a generation: the second sees the first's reply already live and leaves its completion for it. That is the invariant the queue exists to hold, reached from outside a request for the first time. The completion is a user-role turn whose content names itself a machine event -- "A background job you started has finished" -- not a bare person turn. _inject sends a queued turn verbatim, so the framing cannot live there; it lives in the words, the way execute_plan quotes the plan, and a tool.background fragment tells the model these arrive and are a machine event rather than the person speaking. The poller reconnects a fresh connection each tick rather than holding one open -- holding one is the exact live-connection state the whole ssh.py/base.py design forbids, and poll is self-healing besides. Bounded by background_max_jobs and a six-hour ceiling, after which the remote job may keep running but we stop watching it. A Job table, and here the terminal/generation "lost on restart" precedent does NOT transfer: those are seconds long with a human watching, a background job is hours long with nobody watching -- the one case a restart forgetting it would silently break the feature's whole promise. So the row lets a lifespan startup hook rehydrate the watcher and wake as if nothing happened. Cancelling a watcher never stops the detached remote job; it runs on and is picked back up. Tested end to end against a real local shell: launch a detached command, poll it to completion through a watcher, and assert the model was woken with the exit code and output -- plus the lock proving two simultaneous completions start one reply, not two. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -100,4 +100,34 @@ class SshProfile(UUIDPrimaryKey, Timestamps, Base):
|
||||
return f"<SshProfile {self.name} {self.address}>"
|
||||
|
||||
|
||||
__all__ = ["AUTH_KEY", "AUTH_METHODS", "AUTH_PASSWORD", "SshProfile"]
|
||||
class Job(Timestamps, Base):
|
||||
"""A command left running on the far side after the reply that started it.
|
||||
|
||||
The durable record behind `services/agent/jobs.py`, which otherwise keeps
|
||||
only an in-process registry lost on restart. A background job runs for
|
||||
minutes to hours with nobody watching -- exactly the case a restart must not
|
||||
forget -- so the row lets a startup hook re-poll the job's deterministic
|
||||
exit-file and wake the model as if nothing had happened.
|
||||
|
||||
The id is `jobs`'s own short hex, not a UUIDPrimaryKey, because the same id
|
||||
names the files on the machine and is quoted back by the model.
|
||||
"""
|
||||
|
||||
__tablename__ = "agent_jobs"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(32), primary_key=True)
|
||||
chat_id: Mapped[str] = mapped_column(
|
||||
String(32), ForeignKey("chats.id", ondelete="CASCADE"), index=True, nullable=False
|
||||
)
|
||||
command: Mapped[str] = mapped_column(Text, default="")
|
||||
# running | done | killed | lost. `lost` means it stopped without an exit
|
||||
# code being recorded -- killed out of band, or the host rebooted under it.
|
||||
status: Mapped[str] = mapped_column(String(16), default="running", nullable=False)
|
||||
exit_status: Mapped[int | None] = mapped_column(Integer)
|
||||
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<Job {self.id} {self.status}>"
|
||||
|
||||
|
||||
__all__ = ["AUTH_KEY", "AUTH_METHODS", "AUTH_PASSWORD", "Job", "SshProfile"]
|
||||
|
||||
Reference in New Issue
Block a user