From 2f09d8363dec816ac6075b5003f868560b1af7e9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jaroslav=20Bene=C5=A1?= Date: Wed, 5 Aug 2026 21:31:36 +0200 Subject: [PATCH] Something can happen because time passed, and land somewhere worth reading Nothing in LLeMbas ever happened on its own. Every reply was downstream of somebody pressing Send, and the one exception -- jobs.wake, waking a chat when a background job finishes -- was downstream of a command they had run. PLAN.md never listed scheduling as unbuilt because services/chat.py:618 had recorded it as a decision: "a scheduler is a whole new concern for a single-worker application". This is that concern, taken on deliberately, plus the two places its output goes. Reports first, because it is useful with no scheduling at all. A report is not a Chat with one Message in it: it has no turns and no reply, it is read top to bottom, and it must be writable with no chat behind it -- being the fallback for a run whose own chat has gone. As a Chat it would need a sidebar row per daily report, a title that regenerates itself, a composer to suppress and a bubble with a rewind button around something that is not a turn. The section's character is enforced by absence: nothing under reports/ includes the composer or renders chat/_message.html, so there is no sse-connect anywhere and nothing on those pages *can* start a generation. The test reads that off the OpenAPI schema, not by walking app.routes -- this FastAPI keeps an included router wrapped rather than flattening it, so the walk finds nothing and the assertion passes for the wrong reason. rule.py is pure, total, and was finished before anything called it. No session, no wall clock, nothing that raises: validate clamps what it recognises, drops what it does not, and answers {} for prose -- at which point the caller shows the manual form. It had to be that way because the compile step's output is model output that becomes a *timer*, which is the sharpest case of hard rule 6 here. The invariant, pinned: anything validate accepts has a computable next occurrence. A schedule that can never fire looks exactly like a working one on every screen it appears on. Wall-clock and elapsed time are kept apart because they mean different things. at.times are wall-clock in the owner's zone, so 15:00 stays 15:00 across a daylight-saving change -- that is what "every Monday at 3PM" means. every is elapsed real time, so six hours stays six hours across a 23- or 25-hour day -- that is what a timer means. Conflating them gets one of the two wrong twice a year. A time inside the spring-forward gap fires at the first minute that exists; left to zoneinfo's own resolution it lands an hour away wearing a wall-clock time that did not happen, and a daily 02:30 report vanishing once a year on a machine nobody watches is the failure this file is arranged around. The ticker claims and commits *before* it fires. The other order is a hot loop: a firing that raises is retried every tick for ever against whatever it was that failed, and the only symptom is load. Its blanket except is copied from the terminal reaper for a sharper reason -- a ticker that dies on one bad row stops every schedule on the instance and says nothing at all. No request fails, no reply errors, no dot appears. The reports simply stop. Three rules that look like bugs from outside: a firing arriving while the chat is still answering queues rather than starting a second reply, and past max_queued is skipped with the reason on the row; Run now does not advance next_fire_at, or testing a schedule silently consumes the run it was testing; resuming recomputes from now, or a schedule paused for a month fires the instant it comes back, once per occurrence it missed. Catching up lives in the sweep and not in a startup hook, because a suspended host and a long stall reproduce "its time passed while nothing was running" with no restart to hang one on. services/wake.py is the lock discipline extracted rather than copied. A finished job and a due schedule are the same problem, and both depend on there being no await between the running_for check and the writes; two lock dictionaries for one invariant is how one of them drifts. jobs.wake is now a caller that supplies wording, and _completion_text stayed exactly where it was because tool.background quotes its opening sentence. A scheduled run has no reader, so ask_user is withdrawn from resolve_tools rather than merely discouraged in core.unattended -- a rule living only in a system message is one a page the model just read can argue with, and a parked question holds the reply for the whole approval_timeout with nobody to answer it. For the same reason a task chat may not be an agent chat in v1: Manual, Edit and Plan all stop to ask on RISK_EXECUTE, so the only two outcomes would be unattended execution and a reply that stalls. That deserves its own pass. Messages is bounded in the request and unbounded on disk. Only the latest chunk is sent; everything else stays exactly where it was written. Nothing is folded into text and nothing is deleted -- the visible conversation is identical either way, so destroying the older rows would buy only disk, against being irreversible and losing every attachment and tool call in the range, and it would contradict the rule compaction already holds. should_compact refuses this kind for the matching reason: two mechanisms narrowing one transcript is how a summary ends up summarising a summary. The history route is the mirror of thread_tail and keeps its four properties; the fifth is its own, that prepending moves the scroll position, so app.js records scrollHeight before the swap and adds the difference back after. An empty Chat.kind meant "both sides of the switch" and had been read as "no filter" since there were only two of them. The sidebar passes "" precisely when agent chats are switched off -- so the moment a third kind existed, every task chat and every Messages conversation appeared in somebody's ordinary chat list, on exactly the instances whose owners would never think to look. KINDS stays the two-sided fork, because set_sidebar_kind validates against it and a third entry there makes the tree filterable to a side with no button to leave it; ALL_KINDS is what a row may be. Both narrowings are pinned, because they are two implementations of one rule and only one of them is SQL. Per-user timezone had to exist for any of this: harness.py:179 was telling every reader the *server's* idea of the date, which is survivable while the answer is prose and stops being survivable the moment somebody says "every Monday at 3" and something has to work out when that is. Three things were caught by a test being wrong rather than by the code being wrong. The task-chat "no composer" assertions were passing against a page rendering its no-models-configured branch. A permission test asserted the same thing twice because the administrator bypasses every permission. And every Messages test passed with default_model never called, because none of them configured a model -- so the pair it returns was being assigned straight to model_id, and SQLite refuses a tuple in a String column. The fixtures now say why they exist. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 157 ++++++ PLAN.md | 62 ++- src/lembas/api/admin_models.py | 1 + src/lembas/api/admin_schedules.py | 76 +++ src/lembas/api/chats.py | 32 ++ src/lembas/api/messages.py | 124 +++++ src/lembas/api/pages.py | 80 ++- src/lembas/api/preferences.py | 21 + src/lembas/api/reports.py | 85 +++ src/lembas/api/schedules.py | 368 +++++++++++++ src/lembas/db/migrations.py | 1 + src/lembas/db/models/__init__.py | 36 ++ src/lembas/db/models/chat.py | 30 +- src/lembas/db/models/report.py | 68 +++ src/lembas/db/models/schedule.py | 85 +++ src/lembas/main.py | 34 ++ src/lembas/security/permissions.py | 27 + src/lembas/services/agent/jobs.py | 60 +-- src/lembas/services/chat.py | 16 + src/lembas/services/compaction.py | 10 +- src/lembas/services/harness.py | 51 +- src/lembas/services/messages.py | 156 ++++++ src/lembas/services/prompts.py | 156 +++++- src/lembas/services/reports.py | 159 ++++++ src/lembas/services/schedule/__init__.py | 17 + src/lembas/services/schedule/clock.py | 111 ++++ src/lembas/services/schedule/compile.py | 218 ++++++++ src/lembas/services/schedule/rule.py | 506 ++++++++++++++++++ src/lembas/services/schedule/runner.py | 301 +++++++++++ src/lembas/services/schedule/ticker.py | 210 ++++++++ src/lembas/services/schedules.py | 224 ++++++++ src/lembas/services/settings_store.py | 52 ++ src/lembas/services/tool_labels.py | 16 + src/lembas/services/tools.py | 193 ++++++- src/lembas/services/wake.py | 120 +++++ src/lembas/web/static/css/admin.css | 26 + src/lembas/web/static/css/chat.css | 15 + src/lembas/web/static/js/app.js | 30 ++ src/lembas/web/templates/admin/_layout.html | 4 + src/lembas/web/templates/admin/schedules.html | 117 ++++ src/lembas/web/templates/chat/index.html | 16 +- .../web/templates/messages/_history.html | 35 ++ src/lembas/web/templates/messages/index.html | 112 ++++ .../templates/partials/_sidebar_sections.html | 49 ++ .../web/templates/partials/sidebar.html | 1 + src/lembas/web/templates/reports/_layout.html | 43 ++ src/lembas/web/templates/reports/detail.html | 47 ++ src/lembas/web/templates/reports/index.html | 60 +++ src/lembas/web/templates/schedules/_form.html | 128 +++++ .../web/templates/schedules/_layout.html | 39 ++ .../web/templates/schedules/_strip.html | 68 +++ src/lembas/web/templates/schedules/edit.html | 37 ++ src/lembas/web/templates/schedules/index.html | 54 ++ src/lembas/web/templates/schedules/new.html | 78 +++ src/lembas/web/templates/settings.html | 31 ++ tests/conftest.py | 30 ++ tests/test_agent_policy.py | 4 + tests/test_chat.py | 7 +- tests/test_messages.py | 273 ++++++++++ tests/test_reports.py | 241 +++++++++ tests/test_schedule_compile.py | 339 ++++++++++++ tests/test_schedule_rule.py | 373 +++++++++++++ tests/test_schedule_ticker.py | 450 ++++++++++++++++ tests/test_schedules_ui.py | 373 +++++++++++++ tests/test_sidebar_sections.py | 116 ++++ 65 files changed, 6991 insertions(+), 68 deletions(-) create mode 100644 src/lembas/api/admin_schedules.py create mode 100644 src/lembas/api/messages.py create mode 100644 src/lembas/api/reports.py create mode 100644 src/lembas/api/schedules.py create mode 100644 src/lembas/db/models/report.py create mode 100644 src/lembas/db/models/schedule.py create mode 100644 src/lembas/services/messages.py create mode 100644 src/lembas/services/reports.py create mode 100644 src/lembas/services/schedule/__init__.py create mode 100644 src/lembas/services/schedule/clock.py create mode 100644 src/lembas/services/schedule/compile.py create mode 100644 src/lembas/services/schedule/rule.py create mode 100644 src/lembas/services/schedule/runner.py create mode 100644 src/lembas/services/schedule/ticker.py create mode 100644 src/lembas/services/schedules.py create mode 100644 src/lembas/services/wake.py create mode 100644 src/lembas/web/templates/admin/schedules.html create mode 100644 src/lembas/web/templates/messages/_history.html create mode 100644 src/lembas/web/templates/messages/index.html create mode 100644 src/lembas/web/templates/partials/_sidebar_sections.html create mode 100644 src/lembas/web/templates/reports/_layout.html create mode 100644 src/lembas/web/templates/reports/detail.html create mode 100644 src/lembas/web/templates/reports/index.html create mode 100644 src/lembas/web/templates/schedules/_form.html create mode 100644 src/lembas/web/templates/schedules/_layout.html create mode 100644 src/lembas/web/templates/schedules/_strip.html create mode 100644 src/lembas/web/templates/schedules/edit.html create mode 100644 src/lembas/web/templates/schedules/index.html create mode 100644 src/lembas/web/templates/schedules/new.html create mode 100644 tests/test_messages.py create mode 100644 tests/test_reports.py create mode 100644 tests/test_schedule_compile.py create mode 100644 tests/test_schedule_rule.py create mode 100644 tests/test_schedule_ticker.py create mode 100644 tests/test_schedules_ui.py create mode 100644 tests/test_sidebar_sections.py diff --git a/CLAUDE.md b/CLAUDE.md index 23396df..a1f2a87 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -94,6 +94,9 @@ src/lembas/ terminal.py the terminal panel's WebSocket, and its two locks audio.py transcribe, speak, voice discovery library.py knowledge, notes, skills pages; memory CRUD + messages.py the one long conversation, and paging back through it + reports.py the Reports feed, and one report on its own page + schedules.py the Scheduled list, the rule form, and a task's controls files.py upload, serve, remove attachments preferences.py per-user theme, default model, password, audio db/ @@ -118,6 +121,16 @@ src/lembas/ patch.py (applying a unified diff, and rendering one) audio.py OpenAI-shaped /v1/audio/* client fetch.py URL retrieval, HTML to text, the SSRF guard + messages.py the one conversation per person: bounded in the request, + unbounded on disk + reports.py filing a finished piece of work, and finding it again + schedules.py making, changing and stopping a schedule + schedule/ work that happens because time passed: clock.py (whose + "now"), rule.py (the recurrence, pure and total), + ticker.py (the loop and the claim), runner.py (firing), + compile.py (plain words into a rule) + wake.py starting a reply from outside a request -- one lock + discipline, shared by finished jobs and by schedules sharing.py one visibility rule for every library store prompts.py every injected prompt fragment, and {{variables}} metrics.py tokens, context percentage and tokens/second @@ -1206,6 +1219,150 @@ The composer keeps its own because it does more: it follows the selected profile's default directory until somebody picks their own, which only means something while a chat is being created. +**Messages is bounded in the request and unbounded on disk.** One conversation +per person, meant to run for years, so it cannot all be sent -- `build_messages` +takes the last `LIVE_CHUNK` turns and nothing before them. **Nothing is folded +into text and nothing is deleted.** The visible conversation is identical either +way, so destroying the older rows would buy only disk, against being +irreversible and losing every attachment and tool call in the range -- and it +would contradict the rule compaction already holds, that hiding turns is not +deleting them. `compaction.should_compact` refuses this kind for the matching +reason: two mechanisms narrowing one transcript is how a summary ends up +summarising a summary. + +The consequence is worth stating rather than discovering: past the live chunk +the model genuinely does not see what was said. + +`GET /api/messages/history?before=` is the mirror of `thread_tail` and keeps its +four properties -- 204 on a cursor it cannot place, the comparison in SQL with an +`id` tie-breaker (without which a row sharing the cursor's microsecond can never +be reached, and a message that cannot be scrolled back to is gone), an explicit +`hx-target`, and a sentinel outside the composer form. The fifth is its own: +**prepending moves the scroll position**, so `app.js` records `scrollHeight` +before the swap and adds the difference back after. Without it the reader is +dragged up the page the instant the sentinel fires, which reads as a browser bug. + +**`TemplateResponse` injects nothing.** The history route renders +`chat/_message.html` outside `render()`, so `user` *and* `chat` are passed by +hand -- the same reason the SSE path does. Missing either is a 500 on scroll from +a page that rendered perfectly. + +**A schedule is claimed before it is fired, and that order is the design.** +`ticker.sweep` moves the row on -- `fired_count`, `last_fire_at`, the next +`next_fire_at` -- and **commits** before a single firing is awaited. The other +order is a hot loop: a firing that raises is retried every tick for ever against +whatever it was that failed, and the only symptom is load. A sweep lock stops two +overlapping passes claiming the same row, because a firing awaits a model and can +take minutes. Exhaustion *disables*: a rule with nothing left returns `None` and +the row is switched off rather than examined for ever. + +The blanket `except` around the loop is copied from `terminal._reaper_loop` for a +sharper reason than the reaper has. **A ticker that dies on one bad row stops +every schedule on the instance and says nothing** -- no request fails, no reply +errors, no dot appears. The reports simply stop. + +**`rule.py` is pure, total and tested before anything calls it.** No session, no +wall clock, nothing that raises. `validate` is this feature's `nh3.clean`: the +compile step's output is *model output that becomes a timer*, so it clamps what +it recognises, drops what it does not, and answers `{}` for prose -- at which +point the route shows the manual form rather than writing a schedule that can +never fire. The invariant, pinned in the tests, is that **anything `validate` +accepts has a computable next occurrence**; a schedule that can never fire looks +exactly like a working one on every screen it appears on. + +Wall-clock and elapsed time are deliberately different. `at.times` are wall-clock +in the owner's zone, so 15:00 stays 15:00 across a daylight-saving change -- +that is what "every Monday at 3PM" means. `every` is elapsed real time, so six +hours stays six hours across a 23- or 25-hour day -- that is what a timer means. +Conflating them gets one of the two wrong twice a year. A time inside the +spring-forward gap fires at the first minute that exists rather than being +skipped, because a daily report vanishing once a year on a machine nobody watches +is exactly the failure this file is arranged around; `zoneinfo`'s own resolution +yields an instant an hour away wearing a wall-clock time that did not happen. + +**`services/wake.py` is one lock discipline with two callers.** A finished +background job and a due schedule are the same problem -- put a turn into a chat +from outside any request and get it answered -- and both depend on there being no +`await` between the `running_for` check and the writes. Two lock dictionaries for +one invariant is how one of them drifts, so `jobs.wake` is now a caller that +supplies wording. `_completion_text` stayed where it was, because +`tool.background` quotes its opening sentence to the model. + +**Three rules around firing each look like a bug from outside.** A firing +arriving while the chat still answers the previous one *queues* rather than +starting a second reply -- but `_drain` takes one per reply, so the queue is +bounded and past `max_queued` the firing is skipped with the reason on the row. +**Run now does not advance `next_fire_at`**, or testing a schedule would silently +consume the run it was testing. **Resuming recomputes from now**, or a schedule +paused for a month fires the instant it comes back, once for every occurrence it +missed. + +**A task chat is created with its schedule, and that is the one place "chats are +created lazily" is bent.** The lazy rule exists so an opened-and-abandoned chat +never appears in the sidebar; a task chat is not opened and abandoned, because +creating it *is* the act -- and it has to exist before a first firing that may be +days away with nobody present to make one. Removing a schedule keeps the chat by +default and turns it back into an ordinary one: deleting a transcript as a side +effect of removing a timer is the destructive default this codebase avoids, and a +`KIND_TASK` chat with no schedule behind it would appear in no list at all. + +**A task chat may not be an agent chat, in v1.** Scheduling one means running +commands on a timer with nobody watching -- and since Manual, Edit and Plan all +stop to ask on `RISK_EXECUTE`, the only two outcomes are unattended execution and +a reply that stalls until `approval_timeout`. Neither is a feature. That deserves +its own pass with a mode built for it. + +**A task chat has no composer, and the suppression is by absence.** +`chat/index.html` includes `schedules/_strip.html` instead. `chat/_composer.html` +is the only thing that posts a message, so its absence *is* the guarantee -- a +hidden one would still be a form anybody could post to, the same reason Reports +has no route that would accept one. + +**An empty `kind` means both sides of the switch, and never "no filter".** For +as long as there were exactly two kinds those were the same sentence, and the +sidebar leant on it: `Folder.visible_chats` read `not kind or chat.kind == kind` +and `sidebar_context` added its `where` only when `kind` was truthy. `kind` is +`""` precisely when the Chat/Agent switch is *absent* — an instance with agent +chats turned off — so the moment a third kind existed, every conversation +belonging to a section rather than to the tree appeared in somebody's ordinary +chat list, on exactly the instances whose owners would never think to look. + +So `KINDS` stays the two-sided switch and `ALL_KINDS` is what a row may be. +**`KINDS` must not grow**: `api/preferences.py:set_sidebar_kind` validates +against it, and a third entry there makes the tree filterable to a side with no +button to leave it — the "one side of a fork nobody can move" failure the +`sidebar_split` guard already exists to prevent. Both narrowings filter against +`KINDS`, and both are pinned in `tests/test_sidebar_sections.py`, because they +are two implementations of one rule and only one of them is SQL: fixing the +query alone leaves a task chat filed in a folder showing up anyway. + +`/api/chats/unread` narrows the same way and for a sharper reason — a section +gets **one dot for the section**, not one per conversation inside it, so forty +task chats must not mean forty out-of-band spans aimed at elements that are not +on the page. htmx says nothing at all when an OOB target is missing, so that +would be silent waste rather than a visible bug. + +**A report is not a chat with one message in it.** It has a title, a body, a +time and a source; it is read top to bottom and never answered; and it must be +writable with no chat behind it at all, being the fallback destination for +scheduled work whose own chat has gone. As a `Chat` it would need a sidebar row +per daily report, a `title_generated` flag, an `unread` flag, a composer to +suppress and a bubble with an avatar and a rewind button around something that +is not a turn. It is the line `services/library/` already draws from the other +side, and `services/reports.py` is deliberately thinner than the library stores: +no sharing (a report records what somebody's own model did for them) and no +revisions (it describes a moment, not a document being worked on). + +The section's character is enforced by absence rather than by suppression: +`reports/*.html` never includes the composer and never renders +`chat/_message.html`, so there is no `sse-connect` anywhere on those pages and +nothing on them *can* start a generation. `tests/test_reports.py` asserts both +the markup and, from the OpenAPI schema, that no route under `/reports` or +`/api/reports` accepts anything but the delete. Read the schema and not +`app.routes` — this FastAPI keeps an included router wrapped rather than +flattening it, so walking the routes finds nothing and the assertion passes for +the wrong reason. + **The sidebar shows one kind at a time.** `Chat.kind` distinguishes an agent chat everywhere except the one place a person looked. The switch is stored on the account, and three things about it are not the obvious version. It lives diff --git a/PLAN.md b/PLAN.md index 8c99d64..127deee 100644 --- a/PLAN.md +++ b/PLAN.md @@ -8,7 +8,8 @@ that would be expensive to revisit. Kept current as work lands; the detail of with web search, custom HTTP tools and MCP servers, agent chats that work on a machine over SSH, a knowledge library, notes, memory and skills, speech in and out, image generation over ComfyUI, users and groups, model administration, -installable as an app. 1676 tests, `ruff` clean. +installable as an app, reports, messages, and scheduled work that runs on its +own. 1805 tests, `ruff` clean. --- @@ -239,6 +240,55 @@ be a different project, not a refactor. contents come along, with the path and the machine, so the model knows exactly which file it was handed +### Scheduling +- [x] **Schedules** — work that runs because time passed rather than because + somebody asked just now. Fire once or repeat; a fixed number of runs or + until stopped; a timer ("every ten minutes") or a calendar ("every Monday + at 3PM"), and the two compose into "every other Monday" +- [x] **Wall-clock and elapsed time are kept apart**, because they mean + different things: a calendar time stays 15:00 across a daylight-saving + change, while a six-hourly timer stays six hours. A time that does not + exist on a spring-forward day fires at the first minute that does +- [x] **Per-user timezone**, so "every Monday" means the reader's Monday. The + harness tells them their own time now, not the server's +- [x] **Scheduled** — one chat per task, replied into each time it comes round. + No composer: run it now, pause it, edit it, remove it +- [x] A missed run **catches up once** and then resumes. A week of downtime owes + one report, not a hundred and sixty-eight +- [x] Claim before firing, so a run that fails moves the schedule on rather than + retrying every tick for ever; and "Run now" deliberately does *not* consume + the run it was testing +- [x] **Say it in your own words** — a model turns "every Monday morning, check + the build" into a recurrence and an instruction that reads on its own, + and shows it back for approval before anything is saved. Anything it + cannot work out lands in the same form, filled in as far as it got +- [x] A scheduled run knows nobody is watching: `ask_user` is **withdrawn**, not + merely discouraged, because a question with no one to answer it holds the + reply until it times out + +### Messages +- [x] **Messages** — one conversation per person that is meant to run for + years. It opens on the most recent turns and pages older ones in as you + scroll up +- [x] **Bounded in the request, unbounded on disk.** Only the latest chunk is + sent to the model; everything else stays exactly where it was written. + Nothing is folded into text and nothing is deleted +- [x] Anything scheduled can post here, and the schedules that do are listed + beside the conversation rather than two pages away + +### Reports +- [x] **Reports** — a section of its own for finished work: an investigation + written up, an account of what an agent chat changed, whatever a schedule + leaves behind. Filed with `report_write`, searched with FTS5, read on its + own page +- [x] **Nothing here can be replied to**, and that is the section rather than a + restriction on it. No composer, no route that accepts a message, and + nothing on either page that renders the streaming shell — so there is + nothing that could start a generation +- [x] Its own family, permission and capability flag, so a model that keeps + notes need not file reports and a model that files reports need not have + a library at all + ### Audio - [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped `/v1/audio/transcriptions` endpoint. The recording never touches disk @@ -344,6 +394,16 @@ Running several workers needs that state in the database or a broker, because the request following a reply would not necessarily land in the process writing it. +The schedule ticker is now the strongest reason this is not merely a +convenience. It is in-process like the rest, so **two workers means two tickers +and every schedule firing twice**. The claim that prevents a double-fire is a +Python lock plus a write committed in the same transaction, not `SELECT ... FOR +UPDATE`, which SQLite does not have. Scheduling also makes downtime visible in a +way nothing else here does: a dropped reply is one somebody watched fail, while +a missed run is one nobody saw at all — which is what the catch-up in the sweep +is for, and why it lives there rather than in a startup hook (a suspended host +or a long stall reproduces it with no restart to hang one on). + **A restart abandons replies in flight.** Shutdown cancels them and keeps what each had. There is no resume. diff --git a/src/lembas/api/admin_models.py b/src/lembas/api/admin_models.py index 6f2d80a..d5efd21 100644 --- a/src/lembas/api/admin_models.py +++ b/src/lembas/api/admin_models.py @@ -44,6 +44,7 @@ TOOL_CAPABILITIES = ( ("tool_custom", "Custom tools"), ("tool_mcp", "MCP servers"), ("tool_ask", "Ask the reader"), + ("tool_report", "Reports"), ("tool_image", "Image generation"), ("tool_agent", "Agent execution"), ) diff --git a/src/lembas/api/admin_schedules.py b/src/lembas/api/admin_schedules.py new file mode 100644 index 0000000..8f5edfd --- /dev/null +++ b/src/lembas/api/admin_schedules.py @@ -0,0 +1,76 @@ +"""Scheduling administration: whether work may run on its own, and how much. + +Everything here is clamped again in `settings_store.schedules` on the way out. +That is not belt and braces for its own sake: a value stored by an earlier +release, or edited into the database by hand, has to be survivable too, and the +same argument `agents` and `images` already make. What this page adds is telling +somebody *why* a number matters at the moment they change it. +""" + +from __future__ import annotations + +import logging + +from fastapi import APIRouter, Form, Request, Response, status +from fastapi.responses import RedirectResponse +from sqlalchemy import func, select + +from lembas.api.deps import AdminUser, Db +from lembas.db.models import Schedule +from lembas.services import settings_store +from lembas.web.templating import render + +log = logging.getLogger(__name__) + +router = APIRouter(prefix="/admin/schedules", tags=["admin-schedules"]) + + +@router.get("") +async def schedules_page(request: Request, db: Db, user: AdminUser, saved: bool = False): + total = int(db.scalar(select(func.count()).select_from(Schedule)) or 0) + active = int( + db.scalar( + select(func.count()).select_from(Schedule).where(Schedule.enabled.is_(True)) + ) + or 0 + ) + return render( + request, + "admin/schedules.html", + { + "values": settings_store.schedules(db), + # Shown because turning the switch off does not delete anything, and + # an administrator who has just done so should be able to see what + # has stopped rather than infer it. + "total": total, + "active": active, + "saved": saved, + }, + ) + + +@router.post("") +async def save_schedules( + db: Db, + user: AdminUser, + enabled: bool = Form(False), + tick_seconds: int = Form(30), + max_per_user: int = Form(20), + max_concurrent: int = Form(3), + min_interval_seconds: int = Form(60), + max_queued: int = Form(3), +) -> Response: + settings_store.update( + db, + { + "enabled": enabled, + "tick_seconds": tick_seconds, + "max_per_user": max_per_user, + "max_concurrent": max_concurrent, + "min_interval_seconds": min_interval_seconds, + "max_queued": max_queued, + }, + key=settings_store.SCHEDULES, + ) + log.info("scheduling %s by %s", "enabled" if enabled else "disabled", user.email) + return RedirectResponse("/admin/schedules?saved=1", status_code=status.HTTP_303_SEE_OTHER) diff --git a/src/lembas/api/chats.py b/src/lembas/api/chats.py index f6b11a3..1b3d401 100644 --- a/src/lembas/api/chats.py +++ b/src/lembas/api/chats.py @@ -19,6 +19,8 @@ from lembas.api.deps import Db, RequiredUser, require_permission from lembas.db.models import ( KIND_AGENT, KIND_CHAT, + KIND_MESSAGES, + KINDS, ROLE_ASSISTANT, ROLE_USER, Chat, @@ -37,6 +39,7 @@ from lembas.services import generation as generation_service from lembas.services import interaction, settings_store, sse from lembas.services import metrics as metrics_service from lembas.services import prompts as prompts_service +from lembas.services import reports as reports_service from lembas.services import steps as steps_service from lembas.services import tokens as tokens_service from lembas.services import tools as tools_service @@ -705,6 +708,12 @@ async def unread_poll(db: Db, user: RequiredUser) -> Response: # A temporary chat has no sidebar row, so a dot has nowhere to # land and the toast would name a chat nobody can navigate to. Chat.temporary.is_(False), + # And neither has a conversation belonging to a section rather + # than to the tree. Those get one dot per *section*, below -- + # forty task chats must not mean forty out-of-band spans aimed + # at elements that are not on the page. htmx says nothing at all + # when an OOB target is missing, so this would be silent waste. + Chat.kind.in_(KINDS), ) ) ) @@ -721,6 +730,29 @@ async def unread_poll(db: Db, user: RequiredUser) -> Response: for c in chats ) + # One dot for the whole Reports section, carried by this poll rather than by + # a second timer of its own. Sent on every tick including empty, because it + # has to be able to clear: a dot that survived reading the last report would + # be news that cannot be dismissed. + if permissions.has(db, user, "reports.use"): + waiting = reports_service.unread_count(db, user) + markup += ( + '' + ) + + # The Messages conversation, read from the row rather than created: this + # runs every ten seconds on every open page, and `for_user` would write one + # for every account that has never opened the section. + conversation = db.scalars( + select(Chat).where(Chat.user_id == user.id, Chat.kind == KIND_MESSAGES) + ).first() + markup += ( + '' + ) + response = HTMLResponse(markup) if fresh: # HX-Trigger carries the toast; ui.js listens for it. diff --git a/src/lembas/api/messages.py b/src/lembas/api/messages.py new file mode 100644 index 0000000..990e2b1 --- /dev/null +++ b/src/lembas/api/messages.py @@ -0,0 +1,124 @@ +"""Messages: one conversation per person, read backwards on demand. + +The page is the ordinary chat shell with two differences: it opens on the most +recent turns rather than on all of them, and above them sits a sentinel that +fetches the page before whenever it is scrolled into view. + +That sentinel is the mirror of `GET /api/chats/{id}/tail`, which polls forwards, +and it keeps the same four properties for the same reasons — most of all +answering **204 to a cursor it cannot place** rather than falling back to "the +oldest hundred", which would prepend a block the page already holds. +""" + +from __future__ import annotations + +import logging + +from fastapi import APIRouter, Request, Response, status + +from lembas.api.deps import Db, RequiredUser +from lembas.api.pages import _chat_context, sidebar_context +from lembas.db.models import Message, Schedule +from lembas.services import messages as messages_service +from lembas.services import schedules as schedules_service +from lembas.services.markdown import render_markdown +from lembas.services.schedule import clock +from lembas.services.schedule import rule as rule_service +from lembas.web.templating import render + +log = logging.getLogger(__name__) + +router = APIRouter(tags=["messages"]) + + +def _bodies(messages: list[Message]) -> dict[str, str]: + """Markdown rendered server-side, keyed by id, as `chat_detail` does.""" + return {m.id: render_markdown(m.content) for m in messages if m.role == "user"} + + +@router.get("/messages") +async def messages_page(request: Request, db: Db, user: RequiredUser): + conversation = messages_service.for_user(db, user) + live = messages_service.live_messages(db, conversation) + + # The schedules that post in here, listed beside the conversation because + # this is where somebody would look for them -- a schedule whose output + # arrives in this thread and whose controls are two pages away is one nobody + # will find when they want to stop it. + posting = list( + db.scalars( + schedules_service.visible(user) + .where(Schedule.target == "messages") + .order_by(Schedule.created_at.desc()) + ) + ) + zone = clock.zone_for(user) + + return render( + request, + "messages/index.html", + { + "chat": conversation, + "messages": live, + "compacted": [], + "bodies": _bodies(live), + "inherited_prompt": "", + "inherited_from": "", + "more_before": bool(live) and messages_service.has_more_before( + db, conversation, live[0] + ), + "oldest_id": live[0].id if live else "", + "schedules": [ + { + "row": row, + "summary": rule_service.describe(row.rule_json or {}, zone=zone), + } + for row in posting + ], + **_chat_context(db, user, conversation), + **sidebar_context(db, user), + }, + ) + + +@router.get("/api/messages/history") +async def messages_history( + request: Request, db: Db, user: RequiredUser, before: str = "" +) -> Response: + """The page of turns immediately before `before`, oldest first. + + 204 rather than a fallback whenever the cursor cannot be placed: an absent + one, one from another chat, one belonging to a message that has gone. The + alternative -- answering with the oldest page -- would prepend a block the + reader is already looking at, and a duplicated transcript is something only + a reload can reconcile. + """ + conversation = messages_service.for_user(db, user) + cursor = db.get(Message, before) if before else None + if cursor is None or cursor.chat_id != conversation.id: + return Response(status_code=status.HTTP_204_NO_CONTENT) + + page = messages_service.older_than(db, conversation, cursor) + if not page: + return Response(status_code=status.HTTP_204_NO_CONTENT) + + from lembas.web.templating import templates + + return templates.TemplateResponse( + request, + "messages/_history.html", + { + "messages": page, + "bodies": _bodies(page), + "more_before": messages_service.has_more_before(db, conversation, page[0]), + "oldest_id": page[0].id, + # `render()` injects `user` and friends; `TemplateResponse` does + # not, and `chat/_message.html` dereferences both `user` and `chat` + # -- the same reason the SSE path passes them by hand. Missing + # either is a 500 on scroll and nothing at all on the page that + # rendered fine. + "user": user, + "chat": conversation, + **_chat_context(db, user, conversation), + }, + ) diff --git a/src/lembas/api/pages.py b/src/lembas/api/pages.py index 0cc9755..e25dd7b 100644 --- a/src/lembas/api/pages.py +++ b/src/lembas/api/pages.py @@ -2,21 +2,35 @@ from __future__ import annotations +from zoneinfo import available_timezones + from fastapi import APIRouter, HTTPException, Request, Response, status from fastapi.responses import FileResponse, JSONResponse, RedirectResponse from sqlalchemy import select from sqlalchemy.orm import Session as DBSession from lembas.api.deps import Db, RequiredUser -from lembas.db.models import KIND_CHAT, KINDS, Chat, Folder, KnowledgeBase, Message, User +from lembas.db.models import ( + KIND_CHAT, + KIND_MESSAGES, + KIND_TASK, + KINDS, + Chat, + Folder, + KnowledgeBase, + Message, + User, +) from lembas.security import permissions from lembas.services import audio as audio_service from lembas.services import canvas as canvas_service from lembas.services import chat as chat_service from lembas.services import compaction as compaction_service +from lembas.services import reports as reports_service from lembas.services import settings_store from lembas.services import suggestions as suggestions_service from lembas.services.library import documents as documents_service +from lembas.services.schedule import clock from lembas.web.templating import STATIC_DIR, render router = APIRouter(tags=["pages"]) @@ -141,6 +155,7 @@ _GATE_LABELS = { "memory": "Memory", "skills": "Skills", "ask": "Asking you questions", + "report": "Filing reports", "agent": "Running commands", "custom": "Custom tools", "mcp": "MCP servers", @@ -309,9 +324,13 @@ def sidebar_context(db: DBSession, user: User) -> dict: Chat.folder_id.is_(None), Chat.archived.is_(False), Chat.temporary.is_(False), + # `kind` empty means "both sides of the switch", never "no filter" -- + # see `Folder.visible_chats`. Task chats and the Messages conversation + # have sections of their own and must never appear in this list, and + # the case that reaches here with "" is precisely an instance with + # agents disabled, where nobody would ever see the leak coming. + Chat.kind.in_((kind,) if kind else KINDS), ) - if kind: - narrowed = narrowed.where(Chat.kind == kind) unfiled = list( db.scalars(narrowed.order_by(Chat.pinned.desc(), Chat.updated_at.desc())) ) @@ -326,6 +345,22 @@ def sidebar_context(db: DBSession, user: User) -> dict: # not there on any of them. The picker lists every model in the # administrator's order, pinned or not; pinning is not ordering. "pinned_models": [m for m in chat_service.available_models(db, user) if m.pinned], + # Whether the Reports entry starts with its dot showing. Only the first + # paint: from then on `/api/chats/unread` moves it out of band, the same + # deal a chat row's dot has. Counted rather than existence-checked + # because the same query answers both and a count is what a title would + # want if this ever grows one. + "unread_reports": reports_service.unread_count(db, user), + # Read rather than created, for the reason the poll does the same: this + # runs on every page, and `messages.for_user` would write a conversation + # for every account that has never opened the section. + "unread_messages": bool( + db.scalar( + select(Chat.unread).where( + Chat.user_id == user.id, Chat.kind == KIND_MESSAGES + ) + ) + ), "sidebar_kind": kind, # Whether the switch is worth showing at all. A two-way switch with one # useful side is worse than no switch: it offers a view that is empty by @@ -571,12 +606,45 @@ async def chat_detail(request: Request, db: Db, user: RequiredUser, chat_id: str "bodies": bodies, "inherited_prompt": inherited, "inherited_from": inherited_from, + **_schedule_context(db, user, chat), **_chat_context(db, user, chat), **sidebar_context(db, user), }, ) +def _schedule_context(db: DBSession, user: User, chat: Chat) -> dict: + """What the strip below a task chat needs. + + Empty for every other kind, so the three keys exist unconditionally and the + template can ask about `schedule` without a `default(false)` -- the same + reason `audio_service.template_flags` is passed by all four bubble + renderers rather than by whichever one remembered. + + `schedule` being None on a task chat is a real state, not an error: removing + a schedule keeps its chat by default, and the strip says so. + """ + from lembas.services import schedules as schedules_service + + if chat is None or chat.kind != KIND_TASK: + return {"schedule": None, "schedule_summary": "", "schedule_next": None} + + schedule = schedules_service.for_chat(db, chat) + if schedule is None: + return {"schedule": None, "schedule_summary": "", "schedule_next": None} + + zone = clock.zone_for(user) + return { + "schedule": schedule, + "schedule_summary": schedules_service.describe(schedule, owner=user), + "schedule_next": ( + clock.as_utc(schedule.next_fire_at).astimezone(zone) + if schedule.next_fire_at + else None + ), + } + + @router.get("/settings") async def settings_page( request: Request, @@ -606,6 +674,12 @@ async def settings_page( "voice_error": voice_error, "memories": memories_service.all_for(db, user), "memory_limit": memories_service.MAX_MEMORY_CHARS, + # Sorted rather than left in set order, because a list of six + # hundred zones that is not alphabetical is one nobody can use. + "timezones": sorted(available_timezones()), + "timezone": clock.name_for(user), + "server_timezone": str(clock.server_zone()), + "local_now": clock.now_for(user).strftime("%H:%M on %A %-d %B"), **context, **sidebar_context(db, user), }, diff --git a/src/lembas/api/preferences.py b/src/lembas/api/preferences.py index 24c9ab8..564c78b 100644 --- a/src/lembas/api/preferences.py +++ b/src/lembas/api/preferences.py @@ -12,6 +12,7 @@ from lembas.api.deps import Db, RequiredUser from lembas.config import settings from lembas.security.passwords import hash_password, validate_password, verify_password from lembas.security.sessions import COOKIE_NAME, create_session, revoke_all_for_user +from lembas.services.schedule import clock log = logging.getLogger(__name__) @@ -38,6 +39,26 @@ async def set_theme(db: Db, user: RequiredUser, theme: str = Body(..., embed=Tru return {"ok": True, "theme": theme} +@router.post("/timezone") +async def set_timezone(db: Db, user: RequiredUser, timezone: str = Form("")) -> Response: + """Which zone this person's schedules fire in, and what time they are told it is. + + Empty is a real answer -- "whatever the server is set to" -- rather than an + unset field, which is why it is stored as "" instead of being removed. An + unrecognised name is refused rather than stored and fallen back from later: + a schedule that quietly fires in the wrong zone is the failure this whole + field exists to prevent, and the one place to catch it is the write. + """ + chosen = (timezone or "").strip() + if chosen and not clock.known(chosen): + return RedirectResponse( + "/settings?error=timezone", status_code=status.HTTP_303_SEE_OTHER + ) + user.settings_json = {**(user.settings_json or {}), clock.SETTING_KEY: chosen} + db.commit() + return RedirectResponse("/settings?saved=timezone", status_code=status.HTTP_303_SEE_OTHER) + + # Which CSS variables a browser is allowed to set from here, and how far. An # open dict would let a page store anything under somebody's account and have # it read back on every load; a width outside these bounds would hand them a diff --git a/src/lembas/api/reports.py b/src/lembas/api/reports.py new file mode 100644 index 0000000..1136dde --- /dev/null +++ b/src/lembas/api/reports.py @@ -0,0 +1,85 @@ +"""Reports: a feed of finished work, and one report on its own page. + +List-plus-detail, the same shape as the library — and for the same reason, since +an instance running a daily schedule accumulates reports faster than anything +else here. + +**There is no composer on either page, and no route below accepts a message.** +That is the whole character of the section rather than an omission: a report is +addressed to the reader and cannot be answered, and the way to be sure of that +is for the machinery that would answer to be absent. Nothing here renders +`chat/_message.html`, so there is no `sse-connect` anywhere on these pages and +nothing on them can start a generation. +""" + +from __future__ import annotations + +import logging + +from fastapi import APIRouter, Depends, HTTPException, Request, status +from fastapi.responses import RedirectResponse, Response + +from lembas.api.deps import Db, RequiredUser, require_permission +from lembas.api.library import PAGE_SIZE, _page +from lembas.api.pages import sidebar_context +from lembas.db.models import Report +from lembas.services import reports as reports_service +from lembas.services.markdown import render_markdown +from lembas.web.templating import render + +log = logging.getLogger(__name__) + +router = APIRouter(dependencies=[Depends(require_permission("reports.use"))], tags=["reports"]) + + +@router.get("/reports") +async def reports_list(request: Request, db: Db, user: RequiredUser, q: str = "", page: int = 1): + if q.strip(): + rows = reports_service.search(db, user, q, limit=PAGE_SIZE) + pager = {"page": 1, "pages": 1, "total": len(rows)} + else: + rows, pager = _page( + db, reports_service.visible(user).order_by(Report.created_at.desc()), page + ) + return render( + request, + "reports/index.html", + { + "section": "reports", + "reports": rows, + "q": q, + "pager": pager, + **sidebar_context(db, user), + }, + ) + + +@router.get("/reports/{report_id}") +async def report_detail(request: Request, db: Db, user: RequiredUser, report_id: str): + report = reports_service.get(db, report_id, user) + if report is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That report is not available.") + # Opening one is what reading it means. Done before rendering so the dot on + # the way in and the dot on the way back to the list agree -- the poller + # would otherwise re-announce a report the reader is looking at. + reports_service.mark_read(db, report) + return render( + request, + "reports/detail.html", + { + "section": "reports", + "report": report, + # Model output, through the one path allowed to emit HTML. + "body_html": render_markdown(report.body), + **sidebar_context(db, user), + }, + ) + + +@router.post("/api/reports/{report_id}/delete") +async def delete_report(db: Db, user: RequiredUser, report_id: str) -> Response: + report = reports_service.get(db, report_id, user) + if report is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That report is not available.") + reports_service.delete(db, report) + return RedirectResponse("/reports", status_code=status.HTTP_303_SEE_OTHER) diff --git a/src/lembas/api/schedules.py b/src/lembas/api/schedules.py new file mode 100644 index 0000000..ad04b7f --- /dev/null +++ b/src/lembas/api/schedules.py @@ -0,0 +1,368 @@ +"""Scheduled: the list, the setup form, and one task chat's controls. + +A schedule's own chat is rendered by the ordinary chat page — same transcript, +same tail poller, same canvas — with the composer replaced by a strip of +controls. That is the whole reason `KIND_TASK` reuses `Chat` and `Message` +rather than growing tables of its own. + +The rule form here is the **manual** one, and it is not a fallback in the +apologetic sense: it is what makes "an empty override means off" safe for the +compile step in Phase 3. Clearing `task.schedule_compile` must switch off the +*compiling*, not the feature. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime + +from fastapi import APIRouter, Depends, Form, HTTPException, Request, status +from fastapi.responses import RedirectResponse, Response + +from lembas.api.deps import Db, RequiredUser, require_permission +from lembas.api.pages import sidebar_context +from lembas.db.models import TARGET_CHAT, TARGET_MESSAGES, TARGET_REPORT, Schedule +from lembas.services import chat as chat_service +from lembas.services import schedules as schedules_service +from lembas.services.schedule import clock, runner +from lembas.services.schedule import rule as rule_service +from lembas.web.templating import render + +log = logging.getLogger(__name__) + +router = APIRouter( + dependencies=[Depends(require_permission("schedule.use"))], tags=["schedules"] +) + +# What the setup form may ask for, in the order they are offered. +OFFERED_TARGETS = ( + (TARGET_CHAT, "Its own chat"), + (TARGET_REPORT, "Reports"), + (TARGET_MESSAGES, "Messages"), +) + +REPEAT_ONCE = "once" +REPEAT_EVERY = "every" +REPEAT_CALENDAR = "calendar" + + +def _rule_from_form(form) -> dict: + """Build a rule dict out of the setup form's fields. + + Deliberately builds the *raw* shape and hands it to `rule.validate` rather + than validating here: there is one normaliser, it is total, and it is the + same one a model's compiled output will go through in Phase 3. Two + validators would be two ideas of what a legal schedule is. + """ + repeat = str(form.get("repeat") or REPEAT_ONCE) + raw: dict = {} + + when = str(form.get("start_date") or "").strip() + at_time = str(form.get("start_time") or "").strip() or "09:00" + if when: + raw["start"] = f"{when}T{at_time}:00" + + if repeat == REPEAT_EVERY: + unit = str(form.get("every_unit") or "hours") + try: + amount = int(form.get("every_amount") or 1) + except (TypeError, ValueError): + amount = 1 + raw["every"] = {unit: amount} + # A timer with no start begins now. Said here rather than in the rule + # module, which has no clock by design. + raw.setdefault("start", datetime.now(tz=UTC).isoformat()) + + elif repeat == REPEAT_CALENDAR: + times = [t.strip() for t in str(form.get("times") or "09:00").split(",") if t.strip()] + raw["at"] = { + "weekdays": [int(d) for d in form.getlist("weekdays") if str(d).isdigit()], + "times": times, + } + days = str(form.get("month_days") or "").strip() + if days: + raw["at"]["days"] = [int(d) for d in days.split(",") if d.strip().isdigit()] + + try: + count = int(form.get("count") or 0) + except (TypeError, ValueError): + count = 0 + if count > 0: + raw["count"] = count + + until = str(form.get("until") or "").strip() + if until: + raw["until"] = f"{until}T23:59:00" + + return raw + + +def _form_values( + *, schedule: Schedule | None = None, compiled=None +) -> dict: + """Everything `schedules/_form.html` renders, from whichever source there is. + + One dict for both pages, because they are the same fields: an existing row + on the edit page, and what the compile proposed on the new one. The form + reads only this, so what a model suggested is displayed through exactly the + same path as what is stored -- there is no branch in the template that could + show one of them differently. + """ + if compiled is not None: + values = _rule_defaults_from(compiled.rule) + values.update( + title=compiled.title, instruction=compiled.instruction, target=compiled.target + ) + return values + values = _rule_defaults_from((schedule.rule_json if schedule else {}) or {}) + values.update( + title=schedule.title if schedule else "", + instruction=schedule.instruction if schedule else "", + target=schedule.target if schedule else TARGET_CHAT, + ) + return values + + +def _rule_defaults_from(rule: dict) -> dict: + """What the form should show for a rule. + + Derived from the *normalised* rule, so the form and the engine cannot + disagree about what is stored -- an edit screen showing something other + than what runs is the same failure as a label that names the wrong tool. + Shared by the edit page and by the compile's review step, so what a model + proposed is displayed through exactly the same path as what is saved. + """ + rule = rule or {} + at = rule.get("at") or {} + every = rule.get("every") or {} + if at: + repeat = REPEAT_CALENDAR + elif every: + repeat = REPEAT_EVERY + else: + repeat = REPEAT_ONCE + minutes = int(every.get("minutes") or 0) + unit, amount = "minutes", minutes + for size, name in ((10080, "weeks"), (1440, "days"), (60, "hours")): + if minutes and not minutes % size: + unit, amount = name, minutes // size + break + return { + "repeat": repeat, + "every_unit": unit, + "every_amount": amount or 1, + "weekdays": at.get("weekdays") or [], + "times": ", ".join(at.get("times") or []), + "month_days": ", ".join(str(d) for d in at.get("days") or []), + "count": rule.get("count") or 0, + } + + +def _context(db, user, schedule: Schedule | None, *, error: str = "") -> dict: + return { + "section": "scheduled", + "schedule": schedule, + "targets": OFFERED_TARGETS, + "weekday_names": list( + enumerate(("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")) + ), + "form": _form_values(schedule=schedule), + "error": error, + "models": chat_service.available_models(db, user), + "timezone": clock.name_for(user) or str(clock.server_zone()), + **sidebar_context(db, user), + } + + +# --- The list ------------------------------------------------------------------ +@router.get("/scheduled") +async def scheduled_list(request: Request, db: Db, user: RequiredUser): + rows = list( + db.scalars(schedules_service.visible(user).order_by(Schedule.created_at.desc())) + ) + zone = clock.zone_for(user) + return render( + request, + "schedules/index.html", + { + "section": "scheduled", + "schedules": [ + { + "row": row, + "summary": rule_service.describe(row.rule_json or {}, zone=zone), + "next": clock.as_utc(row.next_fire_at).astimezone(zone) + if row.next_fire_at + else None, + } + for row in rows + ], + **sidebar_context(db, user), + }, + ) + + +@router.get("/scheduled/new") +async def new_schedule(request: Request, db: Db, user: RequiredUser, error: str = ""): + """One question: what do you want to schedule? + + The detail comes from the compile. The manual form is on the same page + behind a disclosure, so somebody who already knows exactly when it should + run does not have to describe it in prose and hope. + """ + return render( + request, + "schedules/new.html", + {**_context(db, user, None, error=error), "compiled": None, "described": ""}, + ) + + +@router.post("/api/schedules/describe") +async def describe_schedule(request: Request, db: Db, user: RequiredUser): + """Work a plain-language request into a schedule, and show it back. + + Deliberately a *review* step rather than creating the schedule outright. + The whole point of the compile is that a model chose the timing, and a + timing nobody looked at is exactly the standing instruction this codebase + refuses to create silently elsewhere. + + Nothing here can fail into an error page: a cleared fragment, an endpoint + that is down, prose instead of JSON and a rule that means nothing all end at + the same place, which is the form with the reader's own words in it and a + line saying what to finish. + """ + from lembas.services import prompts as prompts_service + from lembas.services.schedule import compile as compile_service + + form = await request.form() + described = str(form.get("request") or "").strip() + + template = prompts_service.resolve(db, "task.schedule_compile") + resolved = compile_service.endpoint_for(db, user) + if resolved is None: + compiled = compile_service.Compiled( + instruction=described, + title=described[:80], + reason="There is no model configured to work this out, so fill it in yourself.", + ) + else: + endpoint, model_id = resolved + compiled = await compile_service.compile_request( + endpoint, model_id, described, template=template, user=user + ) + + context = _context(db, user, None) + # The compiled values become the form's values, so the reader edits what the + # model proposed rather than being shown it beside an empty form. + context["form"] = _form_values(compiled=compiled) + return render( + request, + "schedules/new.html", + { + **context, + "compiled": compiled, + "described": described, + "summary": rule_service.describe(compiled.rule, zone=clock.zone_for(user)) + if compiled.rule + else "", + }, + ) + + +@router.get("/scheduled/{schedule_id}/edit") +async def edit_schedule( + request: Request, db: Db, user: RequiredUser, schedule_id: str, error: str = "" +): + schedule = schedules_service.get(db, schedule_id, user) + if schedule is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.") + return render(request, "schedules/edit.html", _context(db, user, schedule, error=error)) + + +# --- Writing -------------------------------------------------------------------- +@router.post("/api/schedules") +async def create_schedule(request: Request, db: Db, user: RequiredUser) -> Response: + form = await request.form() + try: + schedule = schedules_service.create( + db, + owner=user, + title=str(form.get("title") or ""), + instruction=str(form.get("instruction") or ""), + request=str(form.get("instruction") or ""), + rule=_rule_from_form(form), + target=str(form.get("target") or TARGET_CHAT), + model_id=str(form.get("model_id") or ""), + ) + except schedules_service.ScheduleError as error: + # Back to the form with the reason, rather than a 400 nobody can act on. + return RedirectResponse( + f"/scheduled/new?error={error}", status_code=status.HTTP_303_SEE_OTHER + ) + return RedirectResponse(f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER) + + +@router.post("/api/schedules/{schedule_id}") +async def save_schedule( + request: Request, db: Db, user: RequiredUser, schedule_id: str +) -> Response: + schedule = schedules_service.get(db, schedule_id, user) + if schedule is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.") + form = await request.form() + try: + schedules_service.update( + db, + schedule, + owner=user, + title=str(form.get("title") or ""), + instruction=str(form.get("instruction") or ""), + rule=_rule_from_form(form), + target=str(form.get("target") or TARGET_CHAT), + ) + except schedules_service.ScheduleError as error: + return RedirectResponse( + f"/scheduled/{schedule_id}/edit?error={error}", + status_code=status.HTTP_303_SEE_OTHER, + ) + return RedirectResponse(f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER) + + +@router.post("/api/schedules/{schedule_id}/toggle") +async def toggle_schedule( + db: Db, user: RequiredUser, schedule_id: str, enabled: str = Form("") +) -> Response: + schedule = schedules_service.get(db, schedule_id, user) + if schedule is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.") + schedules_service.set_enabled( + db, schedule, owner=user, enabled=enabled not in ("", "0", "false") + ) + return RedirectResponse( + f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER + ) + + +@router.post("/api/schedules/{schedule_id}/run") +async def run_schedule(db: Db, user: RequiredUser, schedule_id: str) -> Response: + """Fire it now, without consuming the run it was scheduled for. + + `runner.run_now` is a different entry point from the ticker's for exactly + that reason -- testing a schedule must not skip the real one. + """ + schedule = schedules_service.get(db, schedule_id, user) + if schedule is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.") + chat_id = schedule.chat_id + await runner.run_now(schedule_id) + return RedirectResponse(f"/chat/{chat_id}", status_code=status.HTTP_303_SEE_OTHER) + + +@router.post("/api/schedules/{schedule_id}/delete") +async def delete_schedule( + db: Db, user: RequiredUser, schedule_id: str, keep_chat: str = Form("1") +) -> Response: + schedule = schedules_service.get(db, schedule_id, user) + if schedule is None: + raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.") + schedules_service.delete(db, schedule, keep_chat=keep_chat not in ("", "0", "false")) + return RedirectResponse("/scheduled", status_code=status.HTTP_303_SEE_OTHER) diff --git a/src/lembas/db/migrations.py b/src/lembas/db/migrations.py index 4ebac50..431808b 100644 --- a/src/lembas/db/migrations.py +++ b/src/lembas/db/migrations.py @@ -121,6 +121,7 @@ FTS_INDEXES: tuple[tuple[str, str, tuple[str, ...]], ...] = ( ("documents_fts", "documents", ("title", "description", "extracted_text")), ("notes_fts", "notes", ("title", "body")), ("skills_fts", "skills", ("name", "description", "body")), + ("reports_fts", "reports", ("title", "summary", "body")), ) diff --git a/src/lembas/db/models/__init__.py b/src/lembas/db/models/__init__.py index b59f167..eeff9c7 100644 --- a/src/lembas/db/models/__init__.py +++ b/src/lembas/db/models/__init__.py @@ -20,8 +20,11 @@ from lembas.db.models.attachment import ( ) from lembas.db.models.canvas import ScratchDoc from lembas.db.models.chat import ( + ALL_KINDS, KIND_AGENT, KIND_CHAT, + KIND_MESSAGES, + KIND_TASK, KINDS, ROLE_ASSISTANT, ROLE_SYSTEM, @@ -52,6 +55,23 @@ from lembas.db.models.library import ( SkillRevision, chat_knowledge_bases, ) +from lembas.db.models.report import ( + SOURCE_CHAT, + SOURCE_MANUAL, + SOURCE_SCHEDULE, + SOURCES, + Report, +) +from lembas.db.models.schedule import ( + ORIGIN_MODEL, + ORIGIN_USER, + ORIGINS, + TARGET_CHAT, + TARGET_MESSAGES, + TARGET_REPORT, + TARGETS, + Schedule, +) from lembas.db.models.setting import Setting from lembas.db.models.suggestion import Suggestion from lembas.db.models.tool import ( @@ -84,12 +104,15 @@ __all__ = [ "AUTH_METHODS", "AUTH_PASSWORD", "AUTHOR_USER", + "ALL_KINDS", "Attachment", "KINDS", "KIND_AGENT", "KIND_CHAT", "KIND_DOCUMENT", "KIND_IMAGE", + "KIND_MESSAGES", + "KIND_TASK", "KIND_TEXT", "PRINCIPAL_GROUP", "PRINCIPAL_USER", @@ -111,8 +134,21 @@ __all__ = [ "SECRET_NONE", "SECRET_PLACEMENTS", "SECRET_QUERY", + "ORIGINS", + "ORIGIN_MODEL", + "ORIGIN_USER", + "SOURCES", + "SOURCE_CHAT", "SOURCE_LINK", + "SOURCE_MANUAL", + "SOURCE_SCHEDULE", "SOURCE_UPLOAD", + "TARGETS", + "TARGET_CHAT", + "TARGET_MESSAGES", + "TARGET_REPORT", + "Report", + "Schedule", "Chat", "Job", "Connection", diff --git a/src/lembas/db/models/chat.py b/src/lembas/db/models/chat.py index 54b0098..bd3eb40 100644 --- a/src/lembas/db/models/chat.py +++ b/src/lembas/db/models/chat.py @@ -26,8 +26,28 @@ ROLE_TOOL = "tool" # chat is pointed at a machine before it starts and stays pointed there. KIND_CHAT = "chat" KIND_AGENT = "agent" + +# The two sides of the sidebar's Chat/Agent switch, and nothing else. +# `KINDS` must NOT grow: `api/preferences.py:set_sidebar_kind` validates against +# it, so a third entry would make the tree filterable to a side with no button +# to leave it -- the "one side of a fork nobody can move" failure the +# `sidebar_split` guard already exists to prevent. KINDS = (KIND_CHAT, KIND_AGENT) +# Conversations that belong to a section of their own rather than to the tree. +# A Messages conversation is one per person; a task chat belongs to a schedule +# and is reached through Scheduled. Neither is ever listed among the chats, so +# neither is a side of the switch. +KIND_MESSAGES = "messages" +KIND_TASK = "task" + +# What a row's `kind` may actually be. Every listing that means "the sidebar +# tree" filters on KINDS; every check that means "is this a real value" uses +# this. Reading `kind == ""` as "no filter" is what leaks a task chat into the +# ordinary list on an instance with agents switched off, where the sidebar +# passes "" precisely because there is no switch to read. +ALL_KINDS = (*KINDS, KIND_MESSAGES, KIND_TASK) + # Duplicated from services/agent/policy.py rather than imported: a model module # importing a service would invert the dependency, and this is only the column # default. policy.MODES is the vocabulary; this is what a row starts as. @@ -90,14 +110,20 @@ class Folder(UUIDPrimaryKey, Timestamps, Base): folder branch went through the relationship and filtered nothing. `kind` narrows to one side of the sidebar's Chat/Agent switch. Empty - means both, which is what every caller outside the sidebar wants. + means *both sides of the switch* -- which is not the same as "no filter", + and the difference only became visible once a third kind existed. An + instance with agents disabled passes "" because there is no switch to + read, so a bare `not kind` would list every task chat and the Messages + conversation among somebody's ordinary chats. Those have sections of + their own and are never in the tree. Ordered like the unfiled list: pinned first, then most recently touched. """ + wanted = (kind,) if kind else KINDS kept = [ chat for chat in self.chats - if not chat.archived and not chat.temporary and (not kind or chat.kind == kind) + if not chat.archived and not chat.temporary and chat.kind in wanted ] kept.sort(key=lambda chat: chat.updated_at, reverse=True) kept.sort(key=lambda chat: not chat.pinned) diff --git a/src/lembas/db/models/report.py b/src/lembas/db/models/report.py new file mode 100644 index 0000000..37650ce --- /dev/null +++ b/src/lembas/db/models/report.py @@ -0,0 +1,68 @@ +"""Reports: what was found, written down once and never replied to.""" + +from __future__ import annotations + +from sqlalchemy import Boolean, ForeignKey, String, Text +from sqlalchemy.orm import Mapped, mapped_column + +from lembas.db.base import Base, Timestamps, UUIDPrimaryKey + +# Where a report came from. Not a foreign key to anything -- see `source_id`. +SOURCE_SCHEDULE = "schedule" +SOURCE_CHAT = "chat" +SOURCE_MANUAL = "manual" +SOURCES = (SOURCE_SCHEDULE, SOURCE_CHAT, SOURCE_MANUAL) + + +class Report(UUIDPrimaryKey, Timestamps, Base): + """A finished piece of work, filed. + + Deliberately not a `Chat` with one `Message` in it. A report is read top to + bottom and never answered, so everything a conversation carries -- a + composer, a sidebar row, a title that regenerates itself, a bubble with an + avatar and a rewind button -- would be machinery to suppress rather than + machinery to use. It is the same line `services/library/` already draws + between a note and a chat: a durable artefact is not a turn. + + It must also be writable with no chat behind it at all, being the fallback + destination for a scheduled run whose own chat has gone. + + `body` is Markdown written by a model and goes through + `services/markdown.py` like everything else from an endpoint. Hard rule 6 + applies here exactly as it does in a transcript. + """ + + __tablename__ = "reports" + + owner_id: Mapped[str] = mapped_column( + String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True + ) + title: Mapped[str] = mapped_column(String(300), nullable=False) + # One line for the list page, so a feed of forty reports can be read without + # opening any of them. Written by the model beside the body; falls back to + # the body's first line when it did not bother. + summary: Mapped[str] = mapped_column(String(500), default="") + body: Mapped[str] = mapped_column(Text, default="") + + source: Mapped[str] = mapped_column(String(16), default=SOURCE_MANUAL, nullable=False) + # The chat or the schedule this came out of, kept so a report can say where + # it was made. Deliberately not a ForeignKey: `migrations.py` compiles the + # column type only, so a REFERENCES clause would exist on a fresh database + # and not on an upgraded one -- the same reason `Chat.compacted_through_id` + # and `Folder.ssh_profile_id` are plain ids. Both are validated on read, and + # the row outliving what it points at is normal rather than exceptional: a + # report is worth keeping after the chat that produced it has been deleted. + source_id: Mapped[str] = mapped_column(String(32), default="") + schedule_id: Mapped[str] = mapped_column(String(32), default="") + model_id: Mapped[str] = mapped_column(String(300), default="") + + # NOT NULL with a scalar default so `migrations._add_column_sql` can backfill + # it if this column is ever added to a table that already has rows. + unread: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False) + # Why a run produced nothing worth reading. A scheduled report that failed + # is still a report -- one that silently did not appear is indistinguishable + # from a schedule that never fired. + error: Mapped[str] = mapped_column(Text, default="") + + def __repr__(self) -> str: + return f"" diff --git a/src/lembas/db/models/schedule.py b/src/lembas/db/models/schedule.py new file mode 100644 index 0000000..bc96288 --- /dev/null +++ b/src/lembas/db/models/schedule.py @@ -0,0 +1,85 @@ +"""Schedules: what should happen later, and where its result goes.""" + +from __future__ import annotations + +from datetime import datetime + +from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text +from sqlalchemy.orm import Mapped, mapped_column + +from lembas.db.base import Base, Timestamps, UUIDPrimaryKey +from lembas.db.types import JSONDict + +# Where a firing's result is delivered. Chosen per schedule rather than fixed by +# the screen it was made on: Reports has to stay reachable from anywhere, being +# the fallback, and a schedule somebody wants moved from its own chat to Reports +# should not have to be built again. +TARGET_CHAT = "chat" +TARGET_REPORT = "report" +TARGET_MESSAGES = "messages" +TARGETS = (TARGET_CHAT, TARGET_REPORT, TARGET_MESSAGES) + +# Who made it. Kept because "why is this running?" is a question with two very +# different answers, and one of them is "a model decided to". +ORIGIN_USER = "user" +ORIGIN_MODEL = "model" +ORIGINS = (ORIGIN_USER, ORIGIN_MODEL) + + +class Schedule(UUIDPrimaryKey, Timestamps, Base): + """One standing instruction and when it comes due. + + The row carries no recurrence logic at all: `rule_json` is read by + `services/schedule/rule.py`, which is pure and knows nothing about rows. + What lives here is the bookkeeping the ticker needs to claim a firing + without doing it twice. + """ + + __tablename__ = "schedules" + + user_id: Mapped[str] = mapped_column( + String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True + ) + title: Mapped[str] = mapped_column(String(200), nullable=False, default="") + + # What the reader actually typed, kept verbatim and for ever. The compile + # rewrites it into `instruction`, and "what did I actually ask for" has to + # survive that -- both so the edit form can show it and so a recompile has + # something to work from other than its own previous output. + request: Mapped[str] = mapped_column(Text, default="") + # What is sent when it fires. The compiled form: standalone, since it is + # read with no conversation around it. + instruction: Mapped[str] = mapped_column(Text, default="") + + rule_json: Mapped[dict] = mapped_column(JSONDict, default=dict) + + target: Mapped[str] = mapped_column(String(16), default=TARGET_CHAT, nullable=False) + # The chat this fires into. Deliberately not a ForeignKey -- `migrations.py` + # compiles the column type only, so a REFERENCES clause would exist on a + # fresh database and not on an upgraded one. Validated on read, and a + # dangling value disables the schedule rather than raising every tick. + chat_id: Mapped[str] = mapped_column(String(32), default="") + model_id: Mapped[str] = mapped_column(String(300), default="") + + enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False) + + # The ticker's entire query. Nullable because "nothing more to do" is a real + # state -- a spent count, a closed window, a calendar matching nothing -- + # and is different from "due at the epoch". + next_fire_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True) + last_fire_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + # Stamped when a firing starts and cleared when it finishes, so a run that + # died halfway says so instead of looking like one that never happened. + claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + + fired_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False) + # Why the last run did not work. Shown on the schedule's own page: a + # schedule that silently stopped producing anything is indistinguishable + # from one that was never due. + last_error: Mapped[str] = mapped_column(Text, default="") + + origin: Mapped[str] = mapped_column(String(16), default=ORIGIN_USER, nullable=False) + compiled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + + def __repr__(self) -> str: + return f"" diff --git a/src/lembas/main.py b/src/lembas/main.py index 40e1754..5d7b30a 100644 --- a/src/lembas/main.py +++ b/src/lembas/main.py @@ -19,6 +19,7 @@ from lembas.api import ( admin_images, admin_models, admin_prompts, + admin_schedules, admin_search, admin_suggestions, admin_tools, @@ -31,8 +32,11 @@ from lembas.api import ( files, folders, library, + messages, pages, preferences, + reports, + schedules, terminal, ) from lembas.api.deps import RedirectToLogin, is_htmx, login_redirect @@ -98,6 +102,28 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]: except Exception: # noqa: BLE001 - a job that cannot be rehydrated is not fatal log.exception("could not rehydrate background jobs") + # Schedules. `release_claims` first, because a firing interrupted by the + # last shutdown left a claim stamp that would otherwise read as permanently + # running. Then the ticker, started here rather than lazily like the + # terminal reaper: a schedule can be due at startup with nobody logged in, + # which is most of the point of having one. Inside the loop, so its tasks + # land in this event loop. + # + # Catching up on what was missed is deliberately NOT done here. It lives in + # the sweep, because a suspended laptop, a paused container and a long stall + # all reproduce "its time passed while nothing was running" with no restart + # for a startup hook to hang on. + try: + from lembas.services.schedule.ticker import release_claims + from lembas.services.schedule.ticker import start as start_ticker + + released = release_claims() + if released: + log.info("released %s interrupted schedule claim(s)", released) + start_ticker() + except Exception: # noqa: BLE001 - scheduling failing must not block startup + log.exception("could not start the schedule ticker") + log.info("LLeMbas %s starting on http://%s:%s", __version__, settings.host, settings.port) log.info("data directory: %s", settings.data_dir.resolve()) yield @@ -107,7 +133,11 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]: from lembas.services.agent.jobs import shutdown as stop_jobs from lembas.services.agent.terminal import shutdown as stop_terminals from lembas.services.generation import shutdown as stop_generations + from lembas.services.schedule.ticker import shutdown as stop_ticker + # Before the generations, so nothing new is fired into a chat whose reply is + # about to be cancelled and persisted. + await stop_ticker() await stop_generations() # Open shells have nothing to persist: whatever was running on the far side # is cut off mid-command. Every deploy does this, and the panel is told why @@ -142,12 +172,16 @@ def create_app() -> FastAPI: app.include_router(files.router) app.include_router(folders.router) app.include_router(library.router) + app.include_router(messages.router) + app.include_router(reports.router) + app.include_router(schedules.router) app.include_router(agents.router) app.include_router(admin.router) app.include_router(admin_users.router) app.include_router(admin_models.router) app.include_router(admin_audio.router) app.include_router(admin_search.router) + app.include_router(admin_schedules.router) app.include_router(admin_images.router) app.include_router(admin_prompts.router) app.include_router(admin_suggestions.router) diff --git a/src/lembas/security/permissions.py b/src/lembas/security/permissions.py index 6e87328..7995848 100644 --- a/src/lembas/security/permissions.py +++ b/src/lembas/security/permissions.py @@ -163,6 +163,33 @@ PERMISSION_DEFS: tuple[PermissionDef, ...] = ( True, "Chat", ), + PermissionDef( + "schedule.use", + "Schedule work", + "Set things to run later, on their own — once, or on a repeating " + "timetable. This spends model time with nobody at the keyboard, so it " + "is a capability chosen on purpose rather than one everybody has.", + False, + "Scheduling", + ), + PermissionDef( + "reports.use", + "Keep reports", + "Read the Reports section: finished pieces of work filed for them to " + "read later, by a model that was asked for one or by something that ran " + "while they were away.", + True, + "Reports", + ), + PermissionDef( + "tools.report", + "File reports", + "Let a model write a report when it finishes a piece of work, and read " + "back ones it filed earlier. A report is addressed to the reader and " + "cannot be replied to, so this costs nothing but a place to put things.", + True, + "Reports", + ), PermissionDef( "audio.transcribe", "Dictate messages", diff --git a/src/lembas/services/agent/jobs.py b/src/lembas/services/agent/jobs.py index 6098b94..be67649 100644 --- a/src/lembas/services/agent/jobs.py +++ b/src/lembas/services/agent/jobs.py @@ -73,9 +73,6 @@ LAUNCH_GRACE = 10.0 _JOBS: dict[str, JobState] = {} # One watcher task per job being polled to completion. _WATCHERS: dict[str, asyncio.Task] = {} -# One lock per chat, so two jobs finishing at once cannot each start a reply -- -# see `wake`. -_WAKE_LOCKS: dict[str, asyncio.Lock] = {} # Stop watching a job after this. The remote process may keep running; we simply # stop holding a watcher for it and mark it lost. A job that runs longer than @@ -615,13 +612,6 @@ async def _watch( # --- Waking the model ---------------------------------------------------------- -def _lock(chat_id: str) -> asyncio.Lock: - lock = _WAKE_LOCKS.get(chat_id) - if lock is None: - lock = _WAKE_LOCKS[chat_id] = asyncio.Lock() - return lock - - def _completion_text( job_id: str, command: str, status: str, exit_status: int | None, output: str ) -> str: @@ -651,48 +641,18 @@ async def wake( `queued` for that reply's `_inject`/`_drain` to deliver; if the chat is idle, a fresh reply is started to answer it, the `send_queued_now` move. - The whole thing is under a per-chat lock, and there is no `await` between the - running-check and starting the reply, so two jobs finishing at once cannot - each spin up a generation -- the second sees the first's reply already live - and leaves its completion for it. That is the invariant the queue exists to - hold, reached here from outside a request. - - The completion is a user-role turn whose *content* names itself a machine - event -- `_inject` sends a queued turn verbatim, so the framing cannot live - there; it lives in the words, the way `execute_plan` quotes the plan. + The lock discipline that makes that safe lives in `services/wake.py`, which + is the one copy of it -- schedules need the identical rule, and two lock + dictionaries for one invariant is how one of them drifts. What stays here is + the *wording*, because `tool.background` quotes `_completion_text`'s opening + sentence to the model and rewording it would break that instruction with + nothing anywhere to notice. """ - from lembas.db.models import ROLE_ASSISTANT, ROLE_USER, Chat - from lembas.db.session import session_scope - from lembas.services import chat as chat_service - from lembas.services import generation as generation_service + from lembas.services import wake as wake_service - content = _completion_text(job_id, command, status, exit_status, output) - async with _lock(chat_id): - running = generation_service.running_for(chat_id) is not None - assistant_id = "" - try: - with session_scope() as db: - chat = db.get(Chat, chat_id) - if chat is None: - return - # `machine` changes the bubble and nothing else: the role stays - # `user` because `_inject` sends a queued turn verbatim and the - # request must carry a user turn, and the framing the *model* - # reads is in the words `_completion_text` wrote. What it stops - # is the transcript claiming the reader typed this. - chat_service.create_message( - db, chat, ROLE_USER, content, queued=running, machine=True - ) - if not running: - assistant = chat_service.create_message( - db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id - ) - assistant_id = assistant.id - except Exception: # noqa: BLE001 - a failed wake must not crash the watcher - log.exception("could not wake chat %s for job %s", chat_id, job_id) - return - if assistant_id: - generation_service.ensure(chat_id, assistant_id) + await wake_service.wake_chat( + chat_id, _completion_text(job_id, command, status, exit_status, output) + ) # --- Rehydration and shutdown -------------------------------------------------- diff --git a/src/lembas/services/chat.py b/src/lembas/services/chat.py index d9400b6..b2b9070 100644 --- a/src/lembas/services/chat.py +++ b/src/lembas/services/chat.py @@ -10,6 +10,7 @@ from sqlalchemy import func, select from sqlalchemy.orm import Session as DBSession from lembas.db.models import ( + KIND_MESSAGES, ROLE_ASSISTANT, ROLE_SYSTEM, ROLE_USER, @@ -265,6 +266,21 @@ def build_messages( select(Message).where(Message.chat_id == chat.id).order_by(Message.created_at) ).all() + # The Messages conversation never ends, so it cannot all be sent. Only the + # most recent turns go; everything before them stays on screen and out of + # the request. One branch, and the bound is applied before the loop rather + # than inside it so the filters below still see a contiguous tail. + # + # Not compaction: that summarises with a model call and a threshold, on a + # conversation somebody decided to shorten. This is mechanical, lossless and + # permanent, which is why `compaction.should_compact` refuses this kind -- + # two mechanisms fighting over one transcript is how you get a summary of a + # summary. + if chat.kind == KIND_MESSAGES: + from lembas.services import messages as messages_service + + history = history[-messages_service.LIVE_CHUNK :] + for message in history: if upto is not None and message.id == upto.id: break diff --git a/src/lembas/services/compaction.py b/src/lembas/services/compaction.py index f7ea707..caca3b6 100644 --- a/src/lembas/services/compaction.py +++ b/src/lembas/services/compaction.py @@ -25,7 +25,7 @@ from datetime import UTC, datetime from sqlalchemy import select from sqlalchemy.orm import Session as DBSession -from lembas.db.models import ROLE_ASSISTANT, Chat, Message +from lembas.db.models import KIND_MESSAGES, ROLE_ASSISTANT, Chat, Message from lembas.services import metrics as metrics_service from lembas.services import settings_store, tokens @@ -187,6 +187,14 @@ def should_compact(db: DBSession, chat: Chat, *, pending: str = "") -> bool: if limit <= 0: return False + # The Messages conversation bounds its own request mechanically, in + # `build_messages`. Two mechanisms narrowing one transcript is how a summary + # ends up summarising a summary -- and this one would be summarising turns + # that are already outside the request, which achieves nothing at the cost + # of a model call and a divider on a page that has no divider. + if chat.kind == KIND_MESSAGES: + return False + last = last_complete(db, chat) if last is None: return False diff --git a/src/lembas/services/harness.py b/src/lembas/services/harness.py index 7ec9f11..adfc55f 100644 --- a/src/lembas/services/harness.py +++ b/src/lembas/services/harness.py @@ -33,16 +33,16 @@ clearing those fragments in the admin page restores it exactly. from __future__ import annotations import logging -from datetime import datetime from typing import Any from sqlalchemy import select from sqlalchemy.orm import Session as DBSession -from lembas.db.models import User +from lembas.db.models import KIND_TASK, User from lembas.services import prompts, settings_store from lembas.services.library import memories as memories_service from lembas.services.library import skills as skills_service +from lembas.services.schedule import clock log = logging.getLogger(__name__) @@ -176,11 +176,22 @@ def context_variables( offered = tools or [] families = _families(db, offered) - stamp = datetime.now().astimezone() + # The reader's zone, not the server's. Telling somebody in another country + # that it is Tuesday when it is Wednesday where they are was survivable + # while the answer was only ever prose; it stops being survivable the moment + # they can say "every Monday at 3" and something has to work out when that + # is. `zone_for` falls back to the server's, so an instance where nobody has + # set one behaves exactly as it always did. + stamp = clock.now_for(user) values: dict[str, str] = { "today": stamp.strftime("%A %-d %B %Y"), "now": stamp.strftime("%A %-d %B %Y, %H:%M (UTC%z)"), + # Named so a model working out a schedule can say which zone it meant, + # and so `core.today` can carry it without a second fragment. Empty when + # nobody has chosen one, which drops the line rather than printing the + # server's zone as though it were a decision. + "timezone": clock.name_for(user), "instance_name": str(settings_store.get(db, "instance_name") or "LLeMbas"), "user_name": (user.name or "") if user is not None else "", "model_name": "", @@ -230,6 +241,12 @@ def context_variables( "agent_instructions": "", "agent_instructions_file": "", "plan": "", + # Empty everywhere but a scheduled task's own chat, which is what makes + # it the gate on `core.unattended` as well as the content of + # `context.schedule`. Two fragments, one variable, and no way for the + # warning to appear without the thing it warns about. + "schedule_instruction": "", + "schedule_summary": "", } if chat is not None: @@ -251,9 +268,37 @@ def context_variables( if "agent" in families: values.update(_agent_values(db, chat, user)) + # Not gated on a family: a scheduled task has no tools of its own, and + # the thing that must reach the model is precisely that nobody is + # reading. One primary-key lookup, the same deal `plan` gets. + if chat.kind == KIND_TASK: + values.update(_schedule_values(db, chat, user)) + return values +def _schedule_values(db: DBSession, chat, user) -> dict[str, str]: + """What a scheduled task's chat is for, and how often it comes round. + + A task chat accumulates every run, so by the tenth the original instruction + is far out of sight up the transcript. Put back in front of the model each + turn rather than left to be inferred -- exactly what `Chat.plan_message_id` + exists to do for a plan. + """ + from lembas.services import schedules as schedules_service + + schedule = schedules_service.for_chat(db, chat) + if schedule is None: + # The schedule was removed and its chat kept. There is nothing standing + # to say, so the fragments vanish rather than describing a timer that no + # longer exists. + return {} + return { + "schedule_instruction": schedule.instruction or schedule.request or "", + "schedule_summary": schedules_service.describe(schedule, owner=user), + } + + def _agent_values(db: DBSession, chat, user) -> dict[str, str]: """What an agent chat's harness needs to say about where it is.""" from lembas.services import plans as plans_service diff --git a/src/lembas/services/messages.py b/src/lembas/services/messages.py new file mode 100644 index 0000000..579aaad --- /dev/null +++ b/src/lembas/services/messages.py @@ -0,0 +1,156 @@ +"""Messages: one long-running conversation per person. + +Signal-shaped rather than chat-shaped. There is exactly one of these per +account, it is never titled, never filed and never deleted, and it is meant to +run for years — which is the whole difficulty, because a conversation that never +ends cannot all be sent to a model. + +**What is stored and what is used are different things, and only the second is +bounded.** Every turn is kept, for ever, and scrolling up shows all of them +exactly as they were written. What reaches the model is the most recent +`LIVE_CHUNK` turns and nothing before them. + +**Nothing is folded into text and nothing is deleted**, and that is a +deliberate reading of "compressed and history only". The visible conversation +would be identical either way, so the only thing destroying the older turns +would buy is disk — against which it is irreversible, it loses every attachment +and tool call in the folded range, and it contradicts the rule this codebase +already holds for compaction: *hiding turns is not deleting them*. Bounding the +request achieves the whole of what the feature needs. If the rows ever do need +folding, it is one function against this same boundary and the pages above it do +not change. + +The consequence is worth stating plainly rather than discovering: **a Messages +conversation is infinite on screen and finite in the request.** Past the live +chunk the model genuinely does not see what was said, and it is told so. +""" + +from __future__ import annotations + +import logging + +from sqlalchemy import func, select +from sqlalchemy.orm import Session as DBSession + +from lembas.db.models import KIND_MESSAGES, Chat, Message, User + +log = logging.getLogger(__name__) + +# How many turns reach the model. The "latest chunk", and deliberately larger +# than a page of history: it is the part that has to be enough to hold a +# conversation in, while the rest only has to be readable. +LIVE_CHUNK = 40 + +# How many older turns one scroll-up fetches. Bigger than the live chunk because +# reading back is cheap -- no tokens, no request, just rows. +HISTORY_PAGE = 100 + + +def for_user(db: DBSession, user: User) -> Chat: + """This person's Messages conversation, made if it is not there yet. + + The second deliberate exception to "chats are created lazily", and for a + different reason than a task chat's: a schedule can post in here before + anybody has ever opened the page, and `wake_chat` needs a row to write to. + Get-or-create rather than a startup sweep, so an account that never opens + Messages never grows one. + """ + from lembas.services import chat as chat_service + + existing = db.scalars( + select(Chat) + .where(Chat.user_id == user.id, Chat.kind == KIND_MESSAGES) + .order_by(Chat.created_at) + ).first() + if existing is not None: + return existing + + # `default_model` answers with the *pair* -- the model id and the connection + # it was reached through -- because a chat stores both and resolving the + # second later would pick whichever connection happens to offer the id. + # Unpacked rather than assigned, which is the mistake this comment exists to + # stop being made again: assigning the tuple straight to `model_id` writes a + # tuple into a String column and SQLite refuses the insert. + chosen = chat_service.default_model(db, user) + model_id, connection_id = chosen if chosen else ("", None) + + conversation = Chat( + user_id=user.id, + kind=KIND_MESSAGES, + title="Messages", + # Titling never runs on this one: there is no first exchange to name and + # the name is fixed. Set so nothing downstream has to special-case it. + title_generated=True, + model_id=model_id, + connection_id=connection_id, + ) + db.add(conversation) + db.commit() + return conversation + + +def count(db: DBSession, chat: Chat) -> int: + return int( + db.scalar(select(func.count()).select_from(Message).where(Message.chat_id == chat.id)) + or 0 + ) + + +def live_messages(db: DBSession, chat: Chat, *, limit: int = LIVE_CHUNK) -> list[Message]: + """The most recent turns, oldest first. + + Fetched newest-first and reversed rather than offset from the start: an + offset would have to be recomputed from a count on every request, and would + be wrong the moment a turn arrived between the two queries. + """ + newest = db.scalars( + select(Message) + .where(Message.chat_id == chat.id) + .order_by(Message.created_at.desc(), Message.id.desc()) + .limit(limit) + ).all() + return list(reversed(newest)) + + +def older_than( + db: DBSession, chat: Chat, cursor: Message, *, limit: int = HISTORY_PAGE +) -> list[Message]: + """The page of turns immediately before `cursor`, oldest first. + + The comparison is done in SQL with an `id` tie-breaker, exactly as + `thread_tail` does going the other way. That is not decoration: under a bare + `<`, a row sharing the cursor's microsecond can never be reached, and a + message that cannot be scrolled back to is a message that is gone. + """ + rows = db.scalars( + select(Message) + .where( + Message.chat_id == chat.id, + (Message.created_at < cursor.created_at) + | ((Message.created_at == cursor.created_at) & (Message.id < cursor.id)), + ) + .order_by(Message.created_at.desc(), Message.id.desc()) + .limit(limit) + ).all() + return list(reversed(rows)) + + +def has_more_before(db: DBSession, chat: Chat, cursor: Message) -> bool: + """Whether the sentinel should be rendered again above a page. + + Asked separately rather than by fetching one extra row, because the answer + is needed *after* the page has been reversed and the extra row would have to + be trimmed off the wrong end. + """ + return ( + db.scalar( + select(func.count()) + .select_from(Message) + .where( + Message.chat_id == chat.id, + (Message.created_at < cursor.created_at) + | ((Message.created_at == cursor.created_at) & (Message.id < cursor.id)), + ) + ) + or 0 + ) > 0 diff --git a/src/lembas/services/prompts.py b/src/lembas/services/prompts.py index fe6f955..fd3479f 100644 --- a/src/lembas/services/prompts.py +++ b/src/lembas/services/prompts.py @@ -124,6 +124,24 @@ class Variable: VARIABLES: tuple[Variable, ...] = ( Variable("today", "Today's date", "The current date, written out in full."), Variable("now", "Date and time", "The current date and time, with the offset from UTC."), + Variable( + "schedule_instruction", + "Scheduled instruction", + "In a scheduled task's chat: what it is to do each time it runs. Empty " + "everywhere else, which is what makes it the gate on the unattended " + "guidance as well as its content.", + ), + Variable( + "schedule_summary", + "Schedule", + "In a scheduled task's chat: how often it runs, in words.", + ), + Variable( + "timezone", + "Timezone", + "The reader's timezone, as an IANA name. Empty when they have not chosen " + "one, in which case the times above are the server's.", + ), Variable("instance_name", "Instance name", "What this installation is called."), Variable("user_name", "User's name", "The name of the person in the conversation."), Variable("model_name", "Model", "The display name of the model answering."), @@ -248,6 +266,18 @@ VARIABLES: tuple[Variable, ...] = ( ), Variable("question", "Question", "The first message. Chat title task only."), Variable("answer", "Answer", "The first reply. Chat title task only."), + Variable( + "request", + "The request", + "What somebody said they wanted to happen, in their own words. " + "Working out a schedule only.", + ), + Variable( + "targets", + "Destinations", + "Where a scheduled run's result may be sent, as a list of the values " + "that are accepted. Working out a schedule only.", + ), Variable( "transcript", "Transcript", @@ -573,13 +603,16 @@ BUILTIN: tuple[Fragment, ...] = ( label="Today's date", group=GROUP_CORE, order=20, - variables=("today",), + variables=("today", "timezone"), hint="A model has no clock. Without this it cannot tell whether what it " - "recalls is current, and will not think to check.", + "recalls is current, and will not think to check. The timezone line " + "carries its own variable, so it disappears on an instance where nobody " + "has chosen one rather than announcing the server's as a decision.", default=( "Today is {{today}}. Your training data stops well before this, so treat " "anything time-sensitive as something to check rather than something you " - "already know." + "already know.\n" + "- Times the person gives you are in {{timezone}} unless they say otherwise." ), ), Fragment( @@ -1117,6 +1150,72 @@ BUILTIN: tuple[Fragment, ...] = ( ), ), # --- Context ------------------------------------------------------------- + Fragment( + key="core.unattended", + label="Nobody is watching", + group=GROUP_CORE, + order=35, + requires=("schedule_instruction",), + hint="Only in a scheduled task's chat. The point a model cannot work " + "out for itself is that there is no reader — so the usual moves of " + "asking what was meant, or stopping to check, end the run having done " + "nothing. This is the prompt half; the enforcement is that `ask_user` " + "is not offered here at all, because a rule living only in a system " + "message is one a page the model just read can argue with.", + default=( + "- This chat runs on a schedule and nobody is necessarily reading it. You " + "cannot ask a question and wait for an answer: there is no one to answer, " + "and the run would simply end. Where something is ambiguous, choose the " + "most reasonable reading, do the work, and say plainly in your reply what " + "you assumed and what you would want confirmed. Finish what you were asked " + "to do in this one reply." + ), + ), + Fragment( + key="context.schedule", + label="What this task is for", + group=GROUP_CONTEXT, + order=310, + variables=("schedule_instruction", "schedule_summary"), + requires=("schedule_instruction",), + hint="A task chat accumulates every run, so by the tenth the original " + "instruction is far out of sight. This puts it back in front of the " + "model each turn, the same way the current plan is — one lookup, and no " + "guessing from the transcript.", + default=( + "## This scheduled task\n" + "It runs: {{schedule_summary}}\n" + "Each time, you are to: {{schedule_instruction}}\n" + "Earlier runs are above. Say what has changed since the last one rather " + "than repeating it, unless there is nothing above to compare with." + ), + ), + Fragment( + key="tool.report", + label="Reports", + group=GROUP_TOOLS, + order=246, + families=("report",), + hint="Appears when the report tools are offered. The whole of what a " + "model cannot infer from the schema is the audience: a report is read " + "somewhere else, later, by somebody who cannot answer it. Everything " + "else here follows from that — write it whole, do not end on a " + "question, and do not file one for a two-line answer that has already " + "been given in the conversation.", + default=( + "- You can file a report with report_write: a finished piece of work, kept " + "where the person will find it later. Write one when you are asked for one, " + "and when you finish something long enough that its result is worth keeping — " + "an investigation, an account of what you changed, a summary of what you " + "found. Do not file one for an answer you have just given in two lines; the " + "conversation already holds that. A report is read on its own, away from this " + "chat and possibly long afterwards, and the person cannot reply to it — so " + "say what you were asked, what you found and what you conclude, refer to " + "nothing above, and end on a finding rather than a question. report_search " + "and report_get read back ones filed earlier, which is worth doing before a " + "recurring report so this one can say what changed." + ), + ), Fragment( key="context.knowledge_scope", label="Which knowledge bases", @@ -1518,6 +1617,57 @@ BUILTIN: tuple[Fragment, ...] = ( "from there." ), ), + Fragment( + key="task.schedule_compile", + label="Working out a schedule", + group=GROUP_TASKS, + order=440, + variables=("request", "now", "timezone", "targets"), + hint="One request, made once, when somebody describes something they " + "want to happen later. It turns their words into a recurrence and into " + "an instruction that reads sensibly with no conversation around it — " + "which is how it will be read, days later, by a model that was not " + "there when it was typed. Clearing this switches off the *working out*, " + "not scheduling: the setup screen then asks for the time in its own " + "fields, with the reader's words already filled in. The reply is parsed " + "leniently and anything unusable falls back to that same form, so a " + "model that answers in prose costs a moment rather than a broken " + "schedule.", + default=( + "Turn the request below into a schedule. Reply with one JSON object and " + "nothing else — no commentary, no code fence.\n" + "\n" + "It is currently {{now}} ({{timezone}}). Times you write are in that zone.\n" + "\n" + "The object has these keys:\n" + '- "title": a short name for this, five words or fewer.\n' + '- "instruction": what should be done each time it runs, written out in ' + "full. It will be read on its own, with none of this conversation around " + "it and nobody available to answer a question about it, so say everything " + "it needs. Write it as an instruction, not as a description.\n" + '- "target": where the result goes — one of: {{targets}}. Use "report" ' + "when the point is something to read later, and \"chat\" otherwise.\n" + '- "schedule": an object saying when, with these optional keys:\n' + ' "start": an ISO timestamp for the first (or only) run.\n' + ' "every": one of {"minutes": n}, {"hours": n}, {"days": n}, ' + '{"weeks": n} — a plain timer.\n' + ' "at": {"weekdays": [0-6, Monday is 0], "days": [1-31], ' + '"months": [1-12], "times": ["HH:MM"]} — a calendar. Leave a list out ' + "to mean every one of them.\n" + ' "count": how many times in total, if they said a number.\n' + ' "until": an ISO timestamp to stop after, if they gave one.\n' + "\n" + 'Use "every" for "in ten minutes" or "every six hours". Use "at" for ' + '"every Monday at 3" or "daily at nine". Use both only for something ' + 'like "every other Tuesday". For a one-off, give "start" alone.\n' + "\n" + "If they did not say when, guess the most ordinary reading rather than " + "leaving it out — daily at 09:00 for something described as daily.\n" + "\n" + "The request:\n" + "{{request}}" + ), + ), ) register_source(_builtin_source) diff --git a/src/lembas/services/reports.py b/src/lembas/services/reports.py new file mode 100644 index 0000000..cb941b9 --- /dev/null +++ b/src/lembas/services/reports.py @@ -0,0 +1,159 @@ +"""Reports: filing a finished piece of work, and finding it again. + +A report is written and read; it is never answered. That is the whole shape of +the thing, and it is why this store is deliberately thinner than +`services/library/`: there is no sharing, because a report is a record of what +somebody's own model did on their behalf, and no revisions, because a report +describes a moment rather than a document being worked on. + +`sharing.visible_to` is therefore absent on purpose rather than forgotten. If +reports ever become shareable, `RESOURCE_TYPES` is where that starts, and every +listing here has to go through the helper -- six independently written +permission checks is how one of them ends up written slightly differently. +""" + +from __future__ import annotations + +import logging + +from sqlalchemy import func, select +from sqlalchemy.orm import Session as DBSession + +from lembas.db.models import SOURCE_MANUAL, SOURCES, Report, User +from lembas.services.library.fts import search_ids + +log = logging.getLogger(__name__) + +INDEX = "reports_fts" + +MAX_TITLE_CHARS = 300 +MAX_SUMMARY_CHARS = 500 +MAX_BODY_CHARS = 60_000 +SNIPPET_CHARS = 400 + + +def visible(user: User | None): + """Every report this person owns. + + Takes no session because it builds a query rather than running one, and + takes `None` to mean nobody so an unauthenticated caller gets an empty + result instead of an exception -- the same shape `sharing.visible_to` has, + so a later move to shared reports is a change of one line here. + """ + if user is None: + return select(Report).where(Report.id.is_(None)) + return select(Report).where(Report.owner_id == user.id) + + +def get(db: DBSession, report_id: str, user: User | None) -> Report | None: + report = db.get(Report, report_id) + if report is None or user is None or report.owner_id != user.id: + return None + return report + + +def recent(db: DBSession, user: User | None, *, limit: int = 20) -> list[Report]: + return list(db.scalars(visible(user).order_by(Report.created_at.desc()).limit(limit))) + + +def search(db: DBSession, user: User | None, needle: str, *, limit: int = 20) -> list[Report]: + """Reports matching `needle`, best match first. + + Ids come back from FTS and the rows are re-ordered by hit position, exactly + as the library stores do -- the index knows about ranking and the ORM query + knows about ownership, and neither is asked to do the other's job. + """ + hits = search_ids(db, INDEX, needle, limit=limit * 4) + if not hits: + return [] + order = {hit.id: position for position, hit in enumerate(hits)} + rows = list(db.scalars(visible(user).where(Report.id.in_(list(order))))) + rows.sort(key=lambda report: order.get(report.id, len(order))) + return rows[:limit] + + +def unread_count(db: DBSession, user: User | None) -> int: + if user is None: + return 0 + return int( + db.scalar( + select(func.count()).select_from(Report).where( + Report.owner_id == user.id, Report.unread.is_(True) + ) + ) + or 0 + ) + + +def _first_line(body: str) -> str: + """A summary for a model that did not write one. + + Markdown headings are stripped rather than shown: a list of reports all + beginning "# " reads as a bug, and the heading is nearly always the title + again. + """ + for line in (body or "").splitlines(): + stripped = line.strip().lstrip("#").strip() + if stripped: + return stripped[:MAX_SUMMARY_CHARS] + return "" + + +def create( + db: DBSession, + *, + owner: User, + title: str, + body: str, + summary: str = "", + source: str = SOURCE_MANUAL, + source_id: str = "", + schedule_id: str = "", + model_id: str = "", + error: str = "", + unread: bool = True, +) -> Report: + """File a report. + + Trimming happens here rather than at the column so an over-long write from + a tool is filed short with everything else intact, instead of failing the + turn -- the rule `memories` already follows. + + `unread` defaults to True because every caller that matters is something + that happened without the reader present. A report somebody typed themselves + passes False. + """ + report = Report( + owner_id=owner.id, + title=(title.strip() or "Untitled report")[:MAX_TITLE_CHARS], + summary=(summary.strip() or _first_line(body))[:MAX_SUMMARY_CHARS], + body=(body or "").strip()[:MAX_BODY_CHARS], + source=source if source in SOURCES else SOURCE_MANUAL, + source_id=source_id or "", + schedule_id=schedule_id or "", + model_id=model_id or "", + error=error or "", + unread=unread, + ) + db.add(report) + db.commit() + return report + + +def mark_read(db: DBSession, report: Report) -> Report: + if report.unread: + report.unread = False + db.commit() + return report + + +def delete(db: DBSession, report: Report) -> None: + db.delete(report) + db.commit() + + +def snippet(report: Report) -> str: + text = (report.summary or report.body or "").strip() + if len(text) <= SNIPPET_CHARS: + return text + return text[:SNIPPET_CHARS].rstrip() + "…" diff --git a/src/lembas/services/schedule/__init__.py b/src/lembas/services/schedule/__init__.py new file mode 100644 index 0000000..8cf258e --- /dev/null +++ b/src/lembas/services/schedule/__init__.py @@ -0,0 +1,17 @@ +"""Scheduling: what should happen later, and what makes it happen. + +Four modules, split by what each of them is allowed to touch: + +- `clock.py` -- whose idea of "now" is in force. No session, no rows. +- `rule.py` -- the recurrence spec, and when it next comes due. Pure and + total: it never raises, never opens a session and never reads + the wall clock, which is what lets it be tested exhaustively + before anything calls it. +- `ticker.py` -- the loop that notices a schedule is due, and claims it. +- `runner.py` -- what actually happens when one fires. + +The order matters and is the phasing: everything upstream of `ticker.py` can be +got wrong quietly, so it is settled first. +""" + +from __future__ import annotations diff --git a/src/lembas/services/schedule/clock.py b/src/lembas/services/schedule/clock.py new file mode 100644 index 0000000..ec3f79c --- /dev/null +++ b/src/lembas/services/schedule/clock.py @@ -0,0 +1,111 @@ +"""Whose idea of "now" is in force. + +Until schedules existed, nothing here needed a timezone: `harness.py` stamped +`datetime.now().astimezone()` and every reader was told the *server's* idea of +the date. That is harmless when the answer is prose and wrong the moment a +person says "every Monday at 3" and something has to work out when that is. + +One resolver, because the model compiling a schedule, the screen echoing it back +and the ticker firing it must agree about what Monday means. A disagreement here +does not raise -- it fires at the wrong time, which is the kind of wrong nobody +can debug from the outside. + +Deliberately no new column. The zone lives in `user.settings_json["timezone"]` +beside the theme, empty meaning "whatever the server is set to" -- which is the +honest default for the single-user instance this mostly runs on, and is a real +answer rather than a prompt to go and choose one. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime, tzinfo +from zoneinfo import ZoneInfo, ZoneInfoNotFoundError, available_timezones + +from lembas.db.models import User + +log = logging.getLogger(__name__) + +SETTING_KEY = "timezone" + + +def server_zone() -> tzinfo: + """What the machine is set to, as a real tzinfo. + + `astimezone()` on a naive stamp attaches the system zone, which is what the + harness has always used. Read once per call rather than cached: a host whose + zone changes under a long-running process is rare, and a cache that gets it + wrong is worse than the lookup. + """ + return datetime.now().astimezone().tzinfo or UTC + + +def known(name: str) -> bool: + """Whether this is a zone name Python can actually resolve. + + `available_timezones()` reads the system database and is not cheap, so it is + only consulted for a value that is about to be stored. Everything on the + read path goes through `zone_for`, which simply falls back. + """ + return bool(name) and name in available_timezones() + + +def resolve(name: str) -> tzinfo: + """A zone by name, falling back to the server's rather than raising. + + A stored name can stop resolving -- the tz database is a system package and + a zone can be renamed out from under a row. Falling back means a schedule + fires an hour out at worst; raising means it does not fire at all and the + ticker logs an exception nobody reads. + """ + if not name: + return server_zone() + try: + return ZoneInfo(name) + except (ZoneInfoNotFoundError, ValueError, OSError): + log.warning("unknown timezone %r, falling back to the server's", name) + return server_zone() + + +def name_for(user: User | None) -> str: + """The stored name, or "" meaning the server's. Never resolved here -- + the settings form wants the raw value so an unset zone shows as unset.""" + if user is None: + return "" + return str((user.settings_json or {}).get(SETTING_KEY) or "") + + +def zone_for(user: User | None) -> tzinfo: + """The zone a schedule of this person's fires in, and the one the harness + should tell them the time in.""" + return resolve(name_for(user)) + + +def now_for(user: User | None) -> datetime: + """Aware, in the reader's zone.""" + return datetime.now(tz=zone_for(user)) + + +def to_utc(moment: datetime, *, zone: tzinfo) -> datetime: + """A wall-clock stamp in `zone`, as an instant. + + Naive input is *interpreted* in `zone`; aware input is converted, so a + caller that already knows the offset cannot have it silently reassigned. + """ + if moment.tzinfo is None: + moment = moment.replace(tzinfo=zone) + return moment.astimezone(UTC) + + +def as_utc(moment: datetime) -> datetime: + """An instant, whatever it arrived as. + + SQLite does not store the offset, so a row read back from disk is naive + while one still in the session's identity map keeps its tzinfo, and + comparing the two raises -- the same trap `compaction.moment` exists for. + A naive stamp from the database is UTC by construction, because that is what + every column here is written with. + """ + if moment.tzinfo is None: + return moment.replace(tzinfo=UTC) + return moment.astimezone(UTC) diff --git a/src/lembas/services/schedule/compile.py b/src/lembas/services/schedule/compile.py new file mode 100644 index 0000000..568c5f5 --- /dev/null +++ b/src/lembas/services/schedule/compile.py @@ -0,0 +1,218 @@ +"""Turning "remind me every Monday to check the build" into a schedule. + +One request, once, when a schedule is created. It does two things a person +should not have to do by hand: work out the recurrence, and rewrite the +description into something that reads sensibly with **no conversation around +it** — because that is how it will be read, days later, by a model that was not +present when it was typed. + +Three rules hold this up: + +- **The rule goes through `rule.validate` and nothing else.** That function is + total and clamping, and this is the reason it had to be: what arrives here is + model output that becomes a *timer*. There is one normaliser, shared with the + manual form, so there cannot be two ideas of what a legal schedule is. +- **A compile that fails is not an error.** It hands back what it could work out + and the caller shows the manual form with the reader's own words in it. A + model that answers in prose must never quietly produce a schedule that never + fires. +- **Clearing `task.schedule_compile` switches off the compiling, not the + feature.** That is what makes "an empty override means off" safe here, and it + is only safe because the manual form exists. `task.compact` set the precedent + that clearing a fragment kills a feature, so this one says otherwise in its + own hint. + +Deliberately no `response_format`. Several local endpoints reject unknown +parameters outright, and this is exactly the `apply_effort` lesson: a request +that 400s here would be the compile silently switching itself off. +""" + +from __future__ import annotations + +import json +import logging +import re +from dataclasses import dataclass, field +from datetime import UTC, datetime + +from lembas.db.models import TARGET_CHAT, TARGETS, Chat, User +from lembas.services.llm.openai_client import Endpoint, LLMError, complete +from lembas.services.reasoning import strip_reasoning +from lembas.services.schedule import clock +from lembas.services.schedule import rule as rule_service + +log = logging.getLogger(__name__) + +# Enough for a small model that thinks before answering. The title lesson +# applies: too small is not a shorter answer, it is no answer, because the +# thinking consumes the budget and content comes back empty. +MAX_TOKENS = 900 +MAX_REQUEST_CHARS = 2000 + +_FENCE = re.compile(r"```(?:json)?\s*(.*?)```", re.DOTALL) + + +@dataclass(frozen=True) +class Compiled: + """What the compile worked out. `ok` is False when the reader must finish + the job by hand -- the fields are still filled in as far as they went.""" + + ok: bool = False + title: str = "" + instruction: str = "" + target: str = TARGET_CHAT + rule: dict = field(default_factory=dict) + reason: str = "" + + +def _payload(raw: str) -> dict: + """The first JSON object in a reply, however it was wrapped. + + Lenient for the reason `tools.parse_arguments` is: a small model sends + something close to the shape rather than the shape, and refusing it costs a + whole round trip to end up showing the manual form anyway. + """ + text = (raw or "").strip() + fenced = _FENCE.search(text) + if fenced: + text = fenced.group(1).strip() + if not text.startswith("{"): + start, end = text.find("{"), text.rfind("}") + if start == -1 or end <= start: + return {} + text = text[start : end + 1] + try: + parsed = json.loads(text) + except (ValueError, TypeError): + return {} + return parsed if isinstance(parsed, dict) else {} + + +def render_prompt(template: str, *, request: str, user: User | None) -> str: + """Fill the fragment in. Separate so a test can read what was asked.""" + from lembas.services import prompts as prompts_service + + zone = clock.zone_for(user) + now = datetime.now(tz=UTC).astimezone(zone) + return prompts_service.substitute( + template, + { + "request": request[:MAX_REQUEST_CHARS], + "now": now.strftime("%A %-d %B %Y, %H:%M"), + "timezone": clock.name_for(user) or str(clock.server_zone()), + "targets": ", ".join(TARGETS), + }, + ) + + +async def compile_request( + endpoint: Endpoint, + model_id: str, + request: str, + *, + template: str, + user: User | None = None, +) -> Compiled: + """Work a plain-language request into a schedule. + + Never raises. Every failure -- a cleared fragment, an endpoint that is down, + prose instead of JSON, a rule that normalises to nothing -- comes back as + `ok=False` with whatever was salvageable, and the route shows the manual form. + """ + plain = (request or "").strip() + if not plain: + return Compiled(reason="Say what you want to happen.") + if not template.strip(): + # An administrator cleared the fragment. That switches off the + # *compiling*: the reader fills the form in themselves, with their own + # words already in it. + return Compiled(instruction=plain, title=plain[:80], reason="") + + prompt = render_prompt(template, request=plain, user=user) + body = { + "model": model_id, + "messages": [{"role": "user", "content": prompt}], + "max_tokens": MAX_TOKENS, + "temperature": 0.2, + } + try: + raw = await complete(endpoint, body) + except LLMError as exc: + log.info("schedule compile failed: %s", exc) + return Compiled( + instruction=plain, + title=plain[:80], + reason="The model could not be reached, so fill this in yourself.", + ) + + # A model that thinks inline puts its reasoning in `content`, which is the + # field `complete` hands back verbatim -- the same trap auto-titling hit. + answered, _ = strip_reasoning(raw) + payload = _payload(answered) + if not payload: + return Compiled( + instruction=plain, + title=plain[:80], + reason="The model did not answer with a schedule, so fill this in yourself.", + ) + + raw_rule = payload.get("schedule") or payload.get("rule") or {} + if isinstance(raw_rule, dict): + # A model asked for "every six hours" writes `{"every": {"hours": 6}}` + # and nothing else, which is the natural reading and cannot fire: a + # timer measures from a start, and `rule.py` has no clock to invent one. + # Filled in here, exactly as the manual form's `_rule_from_form` does, + # so the two paths agree about what a startless timer means. A model + # that puts `start` at the top level instead is read the same way rather + # than being told its schedule means nothing. + raw_rule = dict(raw_rule) + if raw_rule.get("every") and not raw_rule.get("start"): + raw_rule["start"] = payload.get("start") or datetime.now(tz=UTC).isoformat() + clean = rule_service.validate(raw_rule) + title = str(payload.get("title") or "").strip() or plain[:80] + instruction = str(payload.get("instruction") or "").strip() or plain + target = str(payload.get("target") or TARGET_CHAT) + if target not in TARGETS: + target = TARGET_CHAT + + if not clean: + return Compiled( + title=title, + instruction=instruction, + target=target, + reason="The model could not work out when this should run — say when below.", + ) + if rule_service.next_after(clean, datetime.now(tz=UTC), zone=clock.zone_for(user)) is None: + # Normalised, but with nothing left to fire. Refused for the same reason + # `schedules.create` refuses it: a schedule that can never run looks + # exactly like a working one on every screen it appears on. + return Compiled( + title=title, + instruction=instruction, + target=target, + rule=clean, + reason="That time has already passed — say when it should run.", + ) + + return Compiled(ok=True, title=title, instruction=instruction, target=target, rule=clean) + + +def endpoint_for(db, user: User) -> tuple[Endpoint, str] | None: + """A connection and model to compile with, or None if there is none. + + Built on a throwaway `Chat` that is never added to a session, exactly as + `agent/draft.py` does: `resolve_endpoint` reads `model_id` and + `connection_id` and nothing else, so it works unchanged and did not have to + learn what a compile is. + """ + from lembas.services import chat as chat_service + + models = chat_service.available_models(db, user) + if not models: + return None + chosen = next((m for m in models if m.pinned), models[0]) + stand_in = Chat(user_id=user.id, model_id=chosen.model_id, connection_id=chosen.connection_id) + try: + return chat_service.resolve_endpoint(db, stand_in) + except LLMError: + return None diff --git a/src/lembas/services/schedule/rule.py b/src/lembas/services/schedule/rule.py new file mode 100644 index 0000000..a29e1cf --- /dev/null +++ b/src/lembas/services/schedule/rule.py @@ -0,0 +1,506 @@ +"""When a schedule next comes due. + +Pure and total. Nothing here opens a session, reads the wall clock or raises: +every function takes what it needs and answers, so the whole of this module can +be tested exhaustively before anything calls it. That is deliberate, because +everything downstream fails *quietly* -- a schedule that never fires looks +exactly like a working one on the list page, and a schedule that fires an hour +out looks like nothing at all until somebody notices the report is late. + +## The shape + +Plain cron cannot say "ten minutes from now, five times", so the rule is a dict +with two independent generators and a bound: + + { + "start": "2026-08-05T14:30:00Z", # first candidate instant, UTC + "every": {"minutes": 10}, # a stride + "at": {"weekdays": [0], # 0 = Monday + "days": [1, 15], # day of the month + "months": [1, 4, 7, 10], + "times": ["15:00"]}, # wall-clock, in the owner's zone + "count": 5, # total firings, 0 = unbounded + "until": "2026-12-31T00:00:00Z" # last instant, "" = unbounded + } + +`every` and `at` compose, and the four combinations are the whole vocabulary: + + every at meaning + ----- ---- ------------------------------------------------------------ + - - fire once, at `start` + x - a timer: start, start + every, start + 2*every, ... + - x a calendar: every matching wall-clock moment after `start` + x x a calendar with a stride: matching moments, every Nth kept + +## Timezone, and why the two halves differ + +`at.times` are **wall-clock** in the owner's zone: 15:00 stays 15:00 across a +DST change, because that is what "every Monday at 3PM" means to the person who +said it. `every` durations are **elapsed real time**: ten minutes is ten +minutes, and a six-hourly timer must not skip or double on a 23- or 25-hour day. +Those are different meanings, not an inconsistency, and conflating them is how +one of the two comes out wrong twice a year. + +A wall-clock time that does not exist (the hour skipped on a spring-forward day) +fires at the first instant that does, rather than being skipped -- a daily report +vanishing once a year is precisely the silent failure this file exists to avoid. +One that occurs twice on a fall-back day fires on the first, once. + +## Not expressible + +Said plainly, because the gap is the point: "the last Friday of the month", "the +third Monday", "weekdays except holidays", "the Nth business day", sub-minute +intervals, sunrise-relative times, and any conditional firing ("only if the +build is red"). The first two are what people will actually ask for; the rule is +JSON, so an `nth` key inside `at` adds them later with no migration. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime, timedelta, tzinfo + +log = logging.getLogger(__name__) + +# The stride units, and how many seconds each is worth. Months are absent on +# purpose: a month is not a duration, and "every month" is `at: {days: [n]}`, +# which is what somebody means by it. +UNITS: dict[str, int] = { + "minutes": 60, + "hours": 3600, + "days": 86400, + "weeks": 604800, +} + +# Bounds. Every one of these is a clamp rather than a rejection, because the +# rule can arrive from a *model* -- the compile step's output is model output +# that becomes a timer, and `validate` is this feature's `nh3.clean`. +MIN_INTERVAL_SECONDS = 60 +MAX_INTERVAL_SECONDS = 366 * 86400 +MAX_COUNT = 10_000 +MAX_TIMES = 24 +MAX_HORIZON_DAYS = 366 * 5 + +# How far ahead a calendar search will walk before giving up. A rule asking for +# 31 February matches nothing, and a search with no bound would spin for ever +# inside the ticker. Days rather than iterations, so the limit is a statement +# about the schedule rather than about the loop. +SEARCH_DAYS = 366 * 4 + +WEEKDAYS = (0, 1, 2, 3, 4, 5, 6) + + +# --- Reading a rule ------------------------------------------------------------ +def _int(value: object, *, low: int, high: int, default: int = 0) -> int: + try: + number = int(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return default + return max(low, min(number, high)) + + +def _stamp(value: object) -> datetime | None: + """An ISO instant, or None. Naive input is read as UTC. + + `fromisoformat` handles a trailing Z from Python 3.11, but a model writes + all sorts of things, so anything unparseable is simply absent. + """ + if isinstance(value, datetime): + return value if value.tzinfo else value.replace(tzinfo=UTC) + if not isinstance(value, str) or not value.strip(): + return None + try: + parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00")) + except ValueError: + return None + return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC) + + +def _times(value: object) -> list[tuple[int, int]]: + """Wall-clock times as (hour, minute), sorted and deduplicated. + + Accepts "15:00", "15:00:30" and "9:5", because a model writes all three, and + a rule refused for its punctuation is a round trip spent on nothing. + """ + if isinstance(value, str): + value = [value] + if not isinstance(value, (list, tuple)): + return [] + found: set[tuple[int, int]] = set() + for item in list(value)[:MAX_TIMES]: + if not isinstance(item, str) or ":" not in item: + continue + hour, _, rest = item.strip().partition(":") + minute = rest.partition(":")[0] + try: + pair = (int(hour), int(minute)) + except ValueError: + continue + if 0 <= pair[0] <= 23 and 0 <= pair[1] <= 59: + found.add(pair) + return sorted(found) + + +def _numbers(value: object, *, low: int, high: int) -> list[int]: + if isinstance(value, int) and not isinstance(value, bool): + value = [value] + if not isinstance(value, (list, tuple)): + return [] + found: set[int] = set() + for item in value: + if isinstance(item, bool): + continue + try: + number = int(item) # type: ignore[arg-type] + except (TypeError, ValueError): + continue + if low <= number <= high: + found.add(number) + return sorted(found) + + +def _every(value: object) -> dict[str, int]: + """A stride, clamped to something that can actually be run. + + An interval under a minute is refused rather than clamped to a minute: the + ticker's own granularity is coarser than that, so honouring it is impossible + and pretending to would be a schedule that silently runs late for ever. + Clamped up, because "every 10 seconds" from a model means "often", and often + is a minute. + """ + if not isinstance(value, dict): + return {} + seconds = 0 + for unit, size in UNITS.items(): + seconds += _int(value.get(unit), low=0, high=MAX_INTERVAL_SECONDS) * size + if seconds <= 0: + return {} + seconds = max(MIN_INTERVAL_SECONDS, min(seconds, MAX_INTERVAL_SECONDS)) + return {"minutes": seconds // 60} + + +def validate(rule: object) -> dict: + """Normalise a rule, or return {} for one that cannot be made sense of. + + **Total on purpose.** The compile step hands this whatever a model wrote, so + it drops what it does not recognise and clamps what it does, and never + raises. `{}` is the honest answer for prose, for a cron string, for an empty + object -- and the caller's job is then to show the manual form rather than + write a schedule that never fires. A schedule that can never fire is + indistinguishable from a working one on every screen it appears on, which is + this feature's flagship silent failure. + + The invariant worth holding on to, and pinned in the tests: **anything this + returns non-empty has a computable next occurrence.** + """ + if not isinstance(rule, dict): + return {} + + every = _every(rule.get("every")) + raw_at = rule.get("at") if isinstance(rule.get("at"), dict) else {} + at = { + "weekdays": _numbers(raw_at.get("weekdays"), low=0, high=6), + "days": _numbers(raw_at.get("days"), low=1, high=31), + "months": _numbers(raw_at.get("months"), low=1, high=12), + "times": [f"{hour:02d}:{minute:02d}" for hour, minute in _times(raw_at.get("times"))], + } + # A calendar with no time of day has no time of day. Midnight is the only + # defensible reading and it is what every cron-like thing does, so it is + # filled in rather than making the whole `at` block meaningless. + if any(at[key] for key in ("weekdays", "days", "months")) and not at["times"]: + at["times"] = ["00:00"] + if not at["times"]: + at = {} + + start = _stamp(rule.get("start")) + until = _stamp(rule.get("until")) + count = _int(rule.get("count"), low=0, high=MAX_COUNT) + + # A one-shot is `start` and nothing else, so without a start there is + # nothing to fire and nothing to infer -- unlike a calendar, which is + # perfectly meaningful from now onwards. + if not every and not at and start is None: + return {} + # A window that closes before it opens produces nothing, which is a rule + # that cannot fire rather than one that fires oddly. + if start is not None and until is not None and until < start: + return {} + + normalised: dict = {} + if start is not None: + normalised["start"] = start.astimezone(UTC).isoformat() + if every: + normalised["every"] = every + if at: + normalised["at"] = {key: value for key, value in at.items() if value} + normalised["at"]["times"] = at["times"] + if count: + normalised["count"] = count + if until is not None: + normalised["until"] = until.astimezone(UTC).isoformat() + return normalised + + +# --- When it next comes due ----------------------------------------------------- +def _interval(rule: dict) -> timedelta: + return timedelta(minutes=int((rule.get("every") or {}).get("minutes") or 0)) + + +def _matches(moment: datetime, at: dict) -> bool: + """Whether a local date satisfies the calendar constraints. + + Empty means "every", per field, which is what makes `{"times": ["09:00"]}` + read as "daily at nine" without having to enumerate seven weekdays. + """ + weekdays = at.get("weekdays") or [] + days = at.get("days") or [] + months = at.get("months") or [] + if weekdays and moment.weekday() not in weekdays: + return False + if days and moment.day not in days: + return False + return not (months and moment.month not in months) + + +def _wall(day: datetime, hour: int, minute: int, zone: tzinfo) -> datetime: + """A wall-clock time on a given local day, as an instant. + + Two DST cases, both handled here rather than left to `zoneinfo`'s defaults: + + - **The hour that does not exist.** On a spring-forward day, 02:30 is not a + time. Constructing it anyway yields something that does not round-trip, so + the gap is detected by comparing and the result is pushed to the first + instant that does exist. Skipping the day instead is how a daily report + disappears once a year. + - **The hour that happens twice.** `fold=0` picks the first, and the + advance-past-the-last-fire rule upstream is what stops the second being + taken as a separate occurrence. + """ + naive = day.replace(hour=hour, minute=minute, second=0, microsecond=0, tzinfo=None) + local = naive.replace(tzinfo=zone, fold=0) + # A time inside the spring-forward gap does not survive the round trip. + if local.astimezone(UTC).astimezone(zone).replace(tzinfo=None) != naive: + # Walk forward a minute at a time to the far side of the gap. Gaps are + # an hour at most in every zone the database has ever carried, so this + # is bounded and cheap; adding the offset difference directly would + # assume the size of a gap this code has no business knowing. + for extra in range(1, 181): + candidate = (naive + timedelta(minutes=extra)).replace(tzinfo=zone, fold=0) + round_trip = candidate.astimezone(UTC).astimezone(zone).replace(tzinfo=None) + if round_trip == naive + timedelta(minutes=extra): + return candidate.astimezone(UTC) + return local.astimezone(UTC) + + +def _calendar_after(rule: dict, after: datetime, *, zone: tzinfo) -> datetime | None: + """The first calendar occurrence strictly after `after`.""" + at = rule.get("at") or {} + times = [tuple(int(part) for part in value.split(":")) for value in at.get("times") or []] + if not times: + return None + + local = after.astimezone(zone) + day = local.replace(hour=0, minute=0, second=0, microsecond=0) + for _ in range(SEARCH_DAYS): + if _matches(day, at): + for hour, minute in times: + moment = _wall(day, hour, minute, zone) + if moment > after: + return moment + day += timedelta(days=1) + # Re-anchor to local midnight: adding a day across a DST boundary + # otherwise leaves the cursor an hour either side of it, and the day + # after a fall-back would be searched from 23:00 the previous evening. + day = day.astimezone(zone).replace(hour=0, minute=0, second=0, microsecond=0) + return None + + +def _exhausted(rule: dict, moment: datetime, fired: int) -> bool: + count = int(rule.get("count") or 0) + if count and fired >= count: + return True + until = _stamp(rule.get("until")) + return bool(until and moment > until) + + +def next_after( + rule: dict, after: datetime, *, zone: tzinfo, fired: int = 0 +) -> datetime | None: + """The next instant this rule comes due, strictly after `after`. + + `None` means never again: the count is spent, the window has closed, or the + calendar matches nothing inside the search horizon. A caller seeing `None` + disables the schedule -- exhaustion switches off, it does not loop. + + `fired` is how many times it has already run, and is what makes `count` + work without the rule having to carry mutable state. + """ + if not isinstance(rule, dict) or not rule: + return None + count = int(rule.get("count") or 0) + if count and fired >= count: + return None + + after = after.astimezone(UTC) + start = _stamp(rule.get("start")) + every = _interval(rule) + at = rule.get("at") or {} + + moment: datetime | None + if at: + # A calendar never fires before its start, so the search begins at + # whichever of the two is later. + floor = max(after, start - timedelta(microseconds=1)) if start else after + moment = _calendar_after(rule, floor, zone=zone) + if moment is not None and every: + # A stride over a calendar keeps every Nth match. Counted from the + # start rather than from `after`, so "every other Monday" means the + # same two Mondays whenever it is asked. + stride = max(1, int(round(every.total_seconds() / 86400)) or 1) + if stride > 1 and start is not None: + elapsed = (moment.astimezone(zone).date() - start.astimezone(zone).date()).days + skipped = 0 + while elapsed % stride and skipped < SEARCH_DAYS: + moment = _calendar_after(rule, moment, zone=zone) + if moment is None: + break + elapsed = ( + moment.astimezone(zone).date() - start.astimezone(zone).date() + ).days + skipped += 1 + elif every: + if start is None: + return None + if after < start: + moment = start + else: + # Absolute arithmetic, deliberately: a timer measures elapsed time, + # so it must not shift when the offset does. Computed rather than + # stepped, so a schedule idle for a year costs one division. + elapsed = (after - start).total_seconds() + steps = int(elapsed // every.total_seconds()) + 1 + moment = start + every * steps + else: + # A one-shot. Due exactly once, and only if it has not already run -- + # `fired` is what stops it being re-offered for ever once its moment has + # passed, since `start > after` is false from then on. + if start is None or fired: + return None + moment = start if start > after else None + + if moment is None or _exhausted(rule, moment, fired): + return None + return moment + + +def advance( + rule: dict, *, after: datetime, now: datetime, zone: tzinfo, fired: int = 0 +) -> tuple[bool, datetime | None]: + """Catch up on a schedule whose time passed while nothing was running. + + Answers two things at once: whether it is owed a firing *now*, and when it + should next come due. The pair is one function because the second depends on + the first -- a caller that asked separately would have to decide what + "next" means for a schedule it has just decided to fire. + + **A missed run collapses to one.** The next occurrence returned is the first + one strictly after `now`, not the one after the slot that was missed -- so a + host switched off for a week comes back owing one report rather than a + hundred and sixty-eight. That is the whole reason this is not just + `next_after`. + + It is called from the *sweep* rather than only at startup, because a + suspended laptop, a paused container and a long stall all reproduce the + same situation with no restart to hang a startup hook on. + """ + due = next_after(rule, after, zone=zone, fired=fired) + if due is None: + return False, None + if due > now: + return False, due + # Overdue. Fire once, and resume from wherever the rule is now -- counting + # this firing, so `count` is spent by what actually ran. + return True, next_after(rule, now, zone=zone, fired=fired + 1) + + +# --- Saying it back ------------------------------------------------------------- +_DAY_NAMES = ("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday") +_MONTH_NAMES = ( + "January", "February", "March", "April", "May", "June", + "July", "August", "September", "October", "November", "December", +) + + +def _join(words: list[str]) -> str: + if len(words) <= 1: + return "".join(words) + return f"{', '.join(words[:-1])} and {words[-1]}" + + +def _ordinal(number: int) -> str: + if 10 <= number % 100 <= 20: + return f"{number}th" + return f"{number}{ {1: 'st', 2: 'nd', 3: 'rd'}.get(number % 10, 'th') }" + + +def _duration(delta: timedelta) -> str: + minutes = int(delta.total_seconds() // 60) + for size, unit in ((10080, "week"), (1440, "day"), (60, "hour"), (1, "minute")): + if minutes >= size and not minutes % size: + amount = minutes // size + return f"{amount} {unit}{'s' if amount != 1 else ''}" + return f"{minutes} minute{'s' if minutes != 1 else ''}" + + +def describe(rule: dict, *, zone: tzinfo) -> str: + """One line saying what this rule does, in the reader's own zone. + + Not decoration. It is what the setup screen echoes back before anything is + saved, what the list page shows beside each schedule, and what the harness + tells a model about its own chat. A row reading "Every Monday at 3PM" over a + rule that fires daily is the same class of failure as three places + disagreeing about a tool's name -- and this one is the reader's only view of + a decision that happens while they are not looking. + """ + rule = rule or {} + if not rule: + return "Never" + + at = rule.get("at") or {} + every = _interval(rule) + parts: list[str] = [] + + if at: + times = _join(list(at.get("times") or [])) + when = [] + if at.get("weekdays"): + when.append(_join([_DAY_NAMES[day] for day in at["weekdays"]])) + if at.get("days"): + when.append(f"the {_join([_ordinal(day) for day in at['days']])}") + if at.get("months"): + when.append(f"of {_join([_MONTH_NAMES[month - 1] for month in at['months']])}") + parts.append( + f"Every {' '.join(when)} at {times}" if when else f"Every day at {times}" + ) + # A stride over a calendar is a qualifier rather than a rewording: + # "Every Monday at 15:00, skipping to every 14 days" is clumsy but true, + # and inventing "every other Monday" for it would be a phrase that stops + # being true the moment the stride is not two. + stride_days = int(every.total_seconds() // 86400) if every else 0 + if stride_days > 1: + parts.append(f"but only every {stride_days} days") + elif every: + parts.append(f"Every {_duration(every)}") + else: + start = _stamp(rule.get("start")) + local = start.astimezone(zone) if start else None + return f"Once, on {local.strftime('%-d %B %Y at %H:%M')}" if local else "Once" + + count = int(rule.get("count") or 0) + if count: + parts.append(f"{count} time{'s' if count != 1 else ''}") + until = _stamp(rule.get("until")) + if until: + parts.append(f"until {until.astimezone(zone).strftime('%-d %B %Y')}") + + return ", ".join(parts) diff --git a/src/lembas/services/schedule/runner.py b/src/lembas/services/schedule/runner.py new file mode 100644 index 0000000..2372bee --- /dev/null +++ b/src/lembas/services/schedule/runner.py @@ -0,0 +1,301 @@ +"""What happens when a schedule fires. + +Every schedule fires the same way — a turn into a chat, answered by the ordinary +generation loop — and the *target* decides only what becomes of the finished +reply. One mechanism, three deliveries: + +- `chat` leave it there. The reply is the point, and it is already in the + task chat where somebody will read it. +- `report` copy it into a `Report` and keep the chat out of the way. +- `messages` copy it into the reader's Messages conversation, as an assistant + turn marked `machine`. Copied rather than moved: the task chat is + the working area and keeps the tool calls, the steps and the + metrics; Messages gets the answer. + +The alternative — a one-shot `complete()` in the shape of `generate_title` — was +rejected because it has no tools and no rounds, which is useless for the case +this feature exists for. "Give me a daily news report" needs to search the web. + +**Nothing in `services/generation.py` changes.** The waiting happens here, in a +task per firing, which is the shape `jobs._watch` already established. Making +generation aware of schedules would mean a branch inside `_persist`, and that is +the single writer with one rule. +""" + +from __future__ import annotations + +import asyncio +import contextlib +import logging +from datetime import UTC, datetime + +from lembas.db.models import ( + ROLE_ASSISTANT, + TARGET_CHAT, + TARGET_MESSAGES, + TARGET_REPORT, + Chat, + Message, + Schedule, + User, +) +from lembas.db.session import session_scope +from lembas.services import reports as reports_service +from lembas.services import wake as wake_service +from lembas.services.schedule import clock +from lembas.services.schedule import rule as rule_service + +log = logging.getLogger(__name__) + +# How long to wait for a firing's reply before giving up on delivering it. The +# reply itself is not cancelled -- it goes on and lands in its chat, which is +# where a task chat's output belongs anyway. What times out is only *this* +# task's interest in copying the result somewhere. +DELIVERY_TIMEOUT = 3600.0 +# How often the waiter looks. Coarse on purpose: nothing is watching this, and a +# report arriving three seconds late costs nobody anything. +POLL_SECONDS = 3.0 + + +def _preamble(schedule: Schedule, *, zone, due_at: datetime | None) -> str: + """The turn a firing puts into the chat. + + Names itself a scheduled event in *words*, because the role stays `user` -- + `_inject` sends a queued turn verbatim and `build_messages` must keep seeing + a user turn. The framing therefore cannot live in the role, exactly as it + cannot for a finished background job. + + The scheduled time is stated as well as the actual one, so a run caught up + after an outage can say so rather than reporting stale news as current. + """ + now = datetime.now(tz=UTC).astimezone(zone) + lines = [ + "This turn was started by a schedule, not by the person — " + "they are not necessarily at the keyboard.", + "", + f"[schedule: {schedule.title or 'untitled'}] " + f"{rule_service.describe(schedule.rule_json or {}, zone=zone)}", + f"It is now {now.strftime('%A %-d %B %Y, %H:%M')}.", + ] + if due_at is not None: + late = (datetime.now(tz=UTC) - clock.as_utc(due_at)).total_seconds() + if late > 600: + local = clock.as_utc(due_at).astimezone(zone) + lines.append( + f"This run was due at {local.strftime('%A %-d %B, %H:%M')} and is late — " + "say so if it makes any of what follows out of date." + ) + lines += ["", schedule.instruction or schedule.request or ""] + return "\n".join(lines) + + +async def _await_reply(chat_id: str, message_id: str) -> None: + """Wait for one generation to finish. + + Polled rather than awaited on the task itself: `generation` owns its + registry and its tasks, and reaching into either from here would couple this + to internals whose whole job is to be replaceable. A poll costs nothing at + this interval and cannot deadlock. + """ + from lembas.services import generation as generation_service + + waited = 0.0 + while waited < DELIVERY_TIMEOUT: + running = generation_service.running_for(chat_id) + # `running_for` already excludes a finished generation, so `None` is the + # ordinary end of this loop. The id check is what stops us waiting on + # somebody's *next* reply in the same chat, which would otherwise happen + # whenever a queued turn is drained straight after ours. + if running is None or running.message_id != message_id: + return + await asyncio.sleep(POLL_SECONDS) + waited += POLL_SECONDS + log.warning("gave up waiting for the reply to schedule message %s", message_id) + + +def _finished_reply(db, chat_id: str, message_id: str) -> Message | None: + message = db.get(Message, message_id) + if message is None or message.chat_id != chat_id: + return None + if not message.complete or message.error: + return None + return message + + +async def deliver(schedule_id: str, message_id: str, *, since: datetime) -> None: + """Put a finished reply where the schedule said it should go. + + `since` is the moment the firing began, and it is what tells a report the + model filed itself apart from one filed on a previous run. + """ + with session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is None: + return + target = schedule.target + chat_id = schedule.chat_id + + if target == TARGET_CHAT: + # Already where it belongs. Stated rather than left to fall through, so + # a reader of this function does not have to infer the common case. + return + + await _await_reply(chat_id, message_id) + + with session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is None: + return + owner = db.get(User, schedule.user_id) + if owner is None: + return + message = _finished_reply(db, chat_id, message_id) + + if target == TARGET_REPORT: + # If the model filed one itself with `report_write`, that is the + # report and this must not file a second. The tool stamps + # `source_id` with the chat, which is what makes them the same run; + # `since` is what makes it *this* run. Both stamps go through + # `as_utc` because one comes from a row read back from SQLite (which + # loses the offset) and the other is still in memory -- comparing + # the two raises, the trap `compaction.moment` exists for. + already = reports_service.recent(db, owner, limit=5) + if any( + r.source_id == chat_id and clock.as_utc(r.created_at) >= clock.as_utc(since) + for r in already + ): + return + if message is None: + reports_service.create( + db, + owner=owner, + title=schedule.title or "Scheduled run", + body="", + source="schedule", + source_id=chat_id, + schedule_id=schedule.id, + error="The run did not produce a reply.", + ) + return + reports_service.create( + db, + owner=owner, + title=schedule.title or "Scheduled run", + body=message.content or "", + source="schedule", + source_id=chat_id, + schedule_id=schedule.id, + model_id=message.model_id or "", + ) + return + + if target == TARGET_MESSAGES: + if message is None: + schedule.last_error = "The run did not produce anything to post." + db.commit() + return + # Copied in as an assistant turn rather than moved, because the task + # chat is the working area and holds the tool calls, the steps and + # the metrics -- the Messages conversation gets the answer. Marked + # `machine` for the same reason a job completion is: the reader did + # not write it, and the bubble should not imply they did. + from lembas.services import chat as chat_service + from lembas.services import messages as messages_service + + conversation = messages_service.for_user(db, owner) + chat_service.create_message( + db, + conversation, + ROLE_ASSISTANT, + message.content or "", + model_id=message.model_id or "", + machine=True, + ) + conversation.unread = True + conversation.unread_notified = False + db.commit() + + +async def fire(schedule_id: str, *, due_at: datetime | None = None) -> None: + """Run one schedule now. + + Never raises: the ticker calls this and one bad schedule must not stop the + others. Anything that goes wrong is written to `last_error`, where the + schedule's own page shows it — a run that failed silently is + indistinguishable from one that was never due. + """ + from lembas.services import settings_store + + try: + with session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is None: + return + owner = db.get(User, schedule.user_id) + chat = db.get(Chat, schedule.chat_id) if schedule.chat_id else None + if owner is None: + return + if chat is None or chat.user_id != owner.id: + # The chat was deleted, or never belonged to this owner. Stop + # rather than fire into nothing on every tick from now on. + schedule.enabled = False + schedule.last_error = "Its chat no longer exists, so it has been switched off." + db.commit() + return + + limit = int(settings_store.schedules(db).get("max_queued") or 3) + zone = clock.zone_for(owner) + content = _preamble(schedule, zone=zone, due_at=due_at) + chat_id = chat.id + model_id = schedule.model_id or chat.model_id + began = datetime.now(tz=UTC) + schedule.claimed_at = began + schedule.last_error = "" + db.commit() + + # Outside the session: a chat already carrying a backlog is one whose + # replies are slower than its schedule, and adding to it makes that + # permanently worse. `_drain` takes one queued turn per reply. + if wake_service.queued_count(chat_id) >= limit: + with session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is not None: + schedule.last_error = ( + "Skipped: the previous run was still going, and turns are " + "already waiting in its chat." + ) + schedule.claimed_at = None + db.commit() + return + + message_id = await wake_service.wake_chat(chat_id, content, model_id=model_id) + + with session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is not None: + schedule.claimed_at = None + db.commit() + + if message_id: + await deliver(schedule_id, message_id, since=began) + except asyncio.CancelledError: + raise + except Exception: # noqa: BLE001 - one bad schedule must not stop the rest + log.exception("schedule %s failed to fire", schedule_id) + with contextlib.suppress(Exception), session_scope() as db: + schedule = db.get(Schedule, schedule_id) + if schedule is not None: + schedule.last_error = "Something went wrong running this. See the log." + schedule.claimed_at = None + db.commit() + + +async def run_now(schedule_id: str) -> None: + """Fire a schedule because somebody pressed the button. + + **Deliberately does not advance `next_fire_at`.** Testing a schedule must + not consume the run it was testing -- somebody who presses this at 14:00 to + check a 15:00 report still expects the 15:00 one. The ticker owns advancing, + and it is the only thing that does. + """ + await fire(schedule_id) diff --git a/src/lembas/services/schedule/ticker.py b/src/lembas/services/schedule/ticker.py new file mode 100644 index 0000000..10c4e53 --- /dev/null +++ b/src/lembas/services/schedule/ticker.py @@ -0,0 +1,210 @@ +"""The loop that notices a schedule is due, and claims it. + +Modelled on `agent/terminal.py:_reaper_loop`, which is the only periodic task +this codebase had before now — including the blanket `except` around the sweep, +for a reason that is sharper here: **a ticker that dies on one bad row stops +every schedule on the instance, and says nothing.** Nothing else would notice. +There is no request failing, no reply erroring, no dot appearing. The reports +simply stop, and the first person to find out is whoever eventually wonders why. + +Started from the lifespan rather than lazily like the reaper. Lazy is right for +terminals — a shell only exists once somebody opened one — and wrong here: a +schedule can be due at startup with nobody logged in, which is most of the point. + +## Claiming, and why the order is the whole design + +One worker and one loop, so the risk is not two processes racing; it is two +*overlapping sweeps*, and a firing that raises being retried every tick for ever. +Three things answer that: + +1. A lock around the sweep, so a slow one (a firing awaits a model, which can + take minutes) cannot overlap the next tick. +2. **Advance, then fire.** The row is moved on and committed *before* anything + is awaited. A firing that dies has still consumed its slot, so the schedule + resumes at its next occurrence with the reason on the row — rather than + becoming a hot loop against an endpoint that is down. +3. `claimed_at` outliving a firing is what lets a run that never finished say so + instead of looking like one that never started. + +Exhaustion **disables**: a rule with nothing left returns `None`, and the row is +switched off rather than being re-examined for ever. +""" + +from __future__ import annotations + +import asyncio +import contextlib +import logging +from datetime import UTC, datetime, timedelta + +from sqlalchemy import select + +from lembas.db.models import Schedule, User +from lembas.db.session import session_scope +from lembas.services import settings_store +from lembas.services.schedule import clock, runner +from lembas.services.schedule import rule as rule_service + +log = logging.getLogger(__name__) + +_TICKER: asyncio.Task | None = None +_SWEEPING = asyncio.Lock() +# Live firings, so shutdown can wait for them rather than leaving a half-written +# reply and a `claimed_at` that never clears. +_FIRING: set[asyncio.Task] = set() + +# Fallback when nothing has been configured. `settings_store.schedules` clamps +# the stored value; this is only for a sweep that runs before anything is read. +TICK_SECONDS = 30.0 + + +def _due(now: datetime): + return ( + select(Schedule) + .where( + Schedule.enabled.is_(True), + Schedule.next_fire_at.is_not(None), + Schedule.next_fire_at <= now, + ) + .order_by(Schedule.next_fire_at) + ) + + +def claim(schedule: Schedule, *, now: datetime, zone) -> tuple[bool, datetime | None]: + """Move one schedule on, and say whether it is owed a firing. + + Pure bookkeeping on the row: it does not fire anything and does not commit, + so the caller decides the transaction boundary. The caller must commit + before awaiting. + """ + rule = schedule.rule_json or {} + after = clock.as_utc(schedule.next_fire_at) if schedule.next_fire_at else now + fire_now, following = rule_service.advance( + rule, + # A microsecond earlier, because `next_after` answers *strictly* after + # what it is given -- so handing it the stored due moment would return + # the one following and skip the firing that is actually owed. The + # alternative, making `next_after` inclusive, would break the far more + # common "give me the one after this one" call it exists for. + after=after - timedelta(microseconds=1), + now=now, + zone=zone, + fired=schedule.fired_count or 0, + ) + if fire_now: + schedule.fired_count = (schedule.fired_count or 0) + 1 + schedule.last_fire_at = now + schedule.next_fire_at = following + if following is None: + # Nothing left to do: a spent count, a closed window, a calendar that + # matches nothing inside the horizon. Switched off rather than left + # enabled with a null next time, which would read as "waiting" for ever. + schedule.enabled = False + return fire_now, following + + +async def sweep(*, now: datetime | None = None) -> int: + """One pass. Returns how many schedules were fired. + + Claims every due row and commits, then starts the firings — in that order, + and with the commit in between, which is the property `test_schedule_ticker` + checks by making a firing raise. + """ + now = now or datetime.now(tz=UTC) + to_fire: list[tuple[str, datetime]] = [] + + async with _SWEEPING: + with session_scope() as db: + if not settings_store.schedules(db).get("enabled"): + return 0 + limit = int(settings_store.schedules(db).get("max_concurrent") or 3) + for schedule in db.scalars(_due(now)): + try: + owner = db.get(User, schedule.user_id) + if owner is None: + # The account is gone; the CASCADE will take the row. + schedule.enabled = False + continue + due_at = clock.as_utc(schedule.next_fire_at) if schedule.next_fire_at else now + fire_now, _ = claim(schedule, now=now, zone=clock.zone_for(owner)) + if fire_now: + to_fire.append((schedule.id, due_at)) + except Exception: # noqa: BLE001 - one bad row must not stop the sweep + log.exception("could not claim schedule %s", schedule.id) + with contextlib.suppress(Exception): + schedule.enabled = False + schedule.last_error = "This schedule could not be read, so it was stopped." + # Committed before a single firing starts. This is the claim. + db.commit() + + if not to_fire: + return 0 + + semaphore = asyncio.Semaphore(max(1, limit)) + + async def _guarded(schedule_id: str, due_at: datetime) -> None: + async with semaphore: + await runner.fire(schedule_id, due_at=due_at) + + for schedule_id, due_at in to_fire: + task = asyncio.create_task(_guarded(schedule_id, due_at)) + _FIRING.add(task) + task.add_done_callback(_FIRING.discard) + return len(to_fire) + + +def _interval() -> float: + with contextlib.suppress(Exception), session_scope() as db: + return float(settings_store.schedules(db).get("tick_seconds") or TICK_SECONDS) + return TICK_SECONDS + + +async def _loop() -> None: + while True: + try: + await asyncio.sleep(_interval()) + await sweep() + except asyncio.CancelledError: + raise + except Exception: # noqa: BLE001 - the ticker must outlive one bad sweep + log.exception("the schedule ticker raised") + + +def start() -> None: + """Begin ticking, once. Idempotent, so a second call in one process is not a + second ticker firing everything twice.""" + global _TICKER + if _TICKER is None or _TICKER.done(): + _TICKER = asyncio.create_task(_loop()) + + +async def shutdown() -> None: + global _TICKER + if _TICKER is not None: + _TICKER.cancel() + with contextlib.suppress(asyncio.CancelledError, Exception): + await _TICKER + _TICKER = None + for task in list(_FIRING): + task.cancel() + for task in list(_FIRING): + with contextlib.suppress(asyncio.CancelledError, Exception): + await task + _FIRING.clear() + + +def release_claims() -> int: + """Clear `claimed_at` on rows whose firing did not survive the last run. + + A restart abandons a reply in flight -- that is already true of every + generation here -- so a schedule whose firing was interrupted would + otherwise carry a claim stamp for ever and read as permanently running. + """ + with session_scope() as db: + stuck = list(db.scalars(select(Schedule).where(Schedule.claimed_at.is_not(None)))) + for schedule in stuck: + schedule.claimed_at = None + schedule.last_error = "This run was interrupted by a restart." + if stuck: + db.commit() + return len(stuck) diff --git a/src/lembas/services/schedules.py b/src/lembas/services/schedules.py new file mode 100644 index 0000000..847522f --- /dev/null +++ b/src/lembas/services/schedules.py @@ -0,0 +1,224 @@ +"""Making, changing and stopping a schedule. + +The row-level half: what the routes and the tools both need, so neither has its +own idea of what creating a schedule involves. `services/schedule/` holds the +machinery — when it next comes due, and what happens when it does. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime + +from sqlalchemy import func, select +from sqlalchemy.orm import Session as DBSession + +from lembas.db.models import ( + KIND_TASK, + ORIGIN_USER, + ORIGINS, + TARGET_CHAT, + TARGETS, + Chat, + Schedule, + User, +) +from lembas.services import settings_store +from lembas.services.schedule import clock +from lembas.services.schedule import rule as rule_service + +log = logging.getLogger(__name__) + +MAX_TITLE_CHARS = 200 +MAX_INSTRUCTION_CHARS = 8000 + + +class ScheduleError(Exception): + """Something a person needs told, in words they can act on.""" + + +def visible(user: User | None): + if user is None: + return select(Schedule).where(Schedule.id.is_(None)) + return select(Schedule).where(Schedule.user_id == user.id) + + +def get(db: DBSession, schedule_id: str, user: User | None) -> Schedule | None: + schedule = db.get(Schedule, schedule_id) + if schedule is None or user is None or schedule.user_id != user.id: + return None + return schedule + + +def for_chat(db: DBSession, chat: Chat) -> Schedule | None: + """The schedule a task chat belongs to, if any.""" + return db.scalars(select(Schedule).where(Schedule.chat_id == chat.id)).first() + + +def count_for(db: DBSession, user: User) -> int: + return int( + db.scalar( + select(func.count()).select_from(Schedule).where(Schedule.user_id == user.id) + ) + or 0 + ) + + +def create( + db: DBSession, + *, + owner: User, + title: str, + instruction: str, + rule: dict, + request: str = "", + target: str = TARGET_CHAT, + model_id: str = "", + origin: str = ORIGIN_USER, +) -> Schedule: + """Write a schedule and the chat it fires into. + + **The chat is created here, with the schedule**, and this is the one place + "chats are created lazily" is deliberately bent. That rule exists so an + opened-and-abandoned chat never appears in the sidebar; a task chat is not + opened and abandoned, because creating it *is* the act. It also has to exist + before the first firing, which may be days away and will have nobody + present to make one. + + A task chat is never `KIND_AGENT`. Scheduling one would mean running + commands on a timer with nobody watching — and since every mode except Auto + stalls waiting for an approval that will not come, the only two outcomes are + "unattended execution" and "does nothing". That deserves its own pass with a + mode built for it, not a flag here. + """ + clean = rule_service.validate(rule) + if not clean: + raise ScheduleError( + "That does not describe a time anything could run at. " + "Say when it should happen — a date and time, or how often." + ) + if rule_service.next_after(clean, datetime.now(tz=UTC), zone=clock.zone_for(owner)) is None: + # Belt and braces over `validate`'s own invariant. A schedule that can + # never fire looks exactly like a working one on every screen it appears + # on, so it is refused at the only moment somebody is present to be told. + raise ScheduleError("That schedule has no next run — its time has already passed.") + + limit = int(settings_store.schedules(db).get("max_per_user") or 20) + if count_for(db, owner) >= limit: + raise ScheduleError( + f"You already have {limit} schedules, which is the most this instance allows. " + "Remove one before adding another." + ) + + chat = Chat( + user_id=owner.id, + kind=KIND_TASK, + title=(title.strip() or "Scheduled task")[:MAX_TITLE_CHARS], + model_id=model_id or "", + ) + db.add(chat) + db.flush() + + schedule = Schedule( + user_id=owner.id, + title=(title.strip() or "Scheduled task")[:MAX_TITLE_CHARS], + request=(request or "").strip()[:MAX_INSTRUCTION_CHARS], + instruction=(instruction or "").strip()[:MAX_INSTRUCTION_CHARS], + rule_json=clean, + target=target if target in TARGETS else TARGET_CHAT, + chat_id=chat.id, + model_id=model_id or "", + origin=origin if origin in ORIGINS else ORIGIN_USER, + enabled=True, + next_fire_at=rule_service.next_after( + clean, datetime.now(tz=UTC), zone=clock.zone_for(owner) + ), + compiled_at=datetime.now(tz=UTC), + ) + db.add(schedule) + db.commit() + return schedule + + +def update( + db: DBSession, + schedule: Schedule, + *, + owner: User, + title: str | None = None, + instruction: str | None = None, + rule: dict | None = None, + target: str | None = None, +) -> Schedule: + """Change a schedule. Absent arguments are left alone.""" + if title is not None and title.strip(): + schedule.title = title.strip()[:MAX_TITLE_CHARS] + if instruction is not None: + schedule.instruction = instruction.strip()[:MAX_INSTRUCTION_CHARS] + if target is not None and target in TARGETS: + schedule.target = target + if rule is not None: + clean = rule_service.validate(rule) + if not clean: + raise ScheduleError( + "That does not describe a time anything could run at. " + "Say when it should happen — a date and time, or how often." + ) + schedule.rule_json = clean + # Recomputed from now, and the count restarted: an edited schedule is a + # new intention, and carrying the old `fired_count` into a new `count` + # would silently spend most of it before the first run. + schedule.fired_count = 0 + schedule.next_fire_at = rule_service.next_after( + clean, datetime.now(tz=UTC), zone=clock.zone_for(owner) + ) + if schedule.next_fire_at is None: + raise ScheduleError("That schedule has no next run — its time has already passed.") + db.commit() + return schedule + + +def set_enabled(db: DBSession, schedule: Schedule, *, owner: User, enabled: bool) -> Schedule: + """Pause or resume. + + **Resuming recomputes from now**, never from the stored value. A schedule + paused for a month would otherwise come back due — and with catching-up in + the sweep, it would fire the moment it was switched on, having decided it + was owed a run from four weeks ago. + """ + schedule.enabled = bool(enabled) + if enabled: + schedule.last_error = "" + schedule.next_fire_at = rule_service.next_after( + schedule.rule_json or {}, + datetime.now(tz=UTC), + zone=clock.zone_for(owner), + fired=schedule.fired_count or 0, + ) + if schedule.next_fire_at is None: + schedule.enabled = False + schedule.last_error = "There are no runs left in this schedule." + db.commit() + return schedule + + +def delete(db: DBSession, schedule: Schedule, *, keep_chat: bool = True) -> None: + """Remove a schedule, and by default keep its transcript. + + Keeping is the default because deleting a conversation as a side effect of + removing a timer is exactly the destructive default this codebase avoids + elsewhere. The chat becomes an ordinary one so it is reachable again — a + `KIND_TASK` chat with no schedule behind it would be in no list at all. + """ + chat = db.get(Chat, schedule.chat_id) if schedule.chat_id else None + if chat is not None: + if keep_chat: + chat.kind = "chat" + else: + db.delete(chat) + db.delete(schedule) + db.commit() + + +def describe(schedule: Schedule, *, owner: User | None) -> str: + return rule_service.describe(schedule.rule_json or {}, zone=clock.zone_for(owner)) diff --git a/src/lembas/services/settings_store.py b/src/lembas/services/settings_store.py index a690277..b0f0217 100644 --- a/src/lembas/services/settings_store.py +++ b/src/lembas/services/settings_store.py @@ -30,6 +30,7 @@ SEARCH = "search" PROMPTS = "prompts" AGENTS = "agents" IMAGES = "images" +SCHEDULES = "schedules" def _general_defaults() -> dict[str, Any]: @@ -272,6 +273,36 @@ def _images_defaults() -> dict[str, Any]: } +def _schedules_defaults() -> dict[str, Any]: + """Scheduling: work that happens because time passed rather than because + somebody asked just now. + + Off until an administrator turns it on, for the reason agent execution is: + this spends model time — and, in an agent chat, runs commands — with nobody + at the keyboard, which is a capability somebody chooses on purpose rather + than one that arrives with an upgrade. + """ + return { + "enabled": False, + # How often the ticker looks. The rule's own granularity is a minute, so + # this bounds how late a firing can be; 30s costs one indexed SELECT. + "tick_seconds": 30, + # A ceiling per person, so one account cannot fill the ticker's sweep. + "max_per_user": 20, + # Firings running at once. Fifty schedules due at 09:00 must not open + # fifty generations against one endpoint. + "max_concurrent": 3, + # The floor `rule.validate` clamps an interval up to. Separate from the + # rule module's own hard minimum: an administrator may want a coarser + # floor than "a minute" without editing code. + "min_interval_seconds": 60, + # How many turns may pile up unanswered in one chat before a firing is + # skipped instead of queued. `_drain` takes one per reply, so an + # unbounded queue is a backlog that outlives the day that caused it. + "max_queued": 3, + } + + _DEFAULTS: dict[str, Any] = { GENERAL: _general_defaults, AUDIO: _audio_defaults, @@ -279,6 +310,7 @@ _DEFAULTS: dict[str, Any] = { PROMPTS: _prompts_defaults, AGENTS: _agents_defaults, IMAGES: _images_defaults, + SCHEDULES: _schedules_defaults, } @@ -411,6 +443,26 @@ def images(db: DBSession) -> dict[str, Any]: return values +def schedules(db: DBSession) -> dict[str, Any]: + """Scheduling settings, with the numbers clamped on read. + + Clamped here rather than at the save, for the reason `agents` gives: a value + stored by an earlier version cannot bite either. Every floor below is a + number that means something bad at zero -- a tick of 0 is a busy loop, a + concurrency of 0 is a ticker that claims firings and never runs them, and + both would look from the outside like scheduling simply not working. + """ + values = get_group(db, SCHEDULES) + values["tick_seconds"] = min(max(int(values.get("tick_seconds") or 30), 5), 300) + values["max_per_user"] = min(max(int(values.get("max_per_user") or 20), 1), 200) + values["max_concurrent"] = min(max(int(values.get("max_concurrent") or 3), 1), 20) + values["min_interval_seconds"] = min( + max(int(values.get("min_interval_seconds") or 60), 60), 86400 + ) + values["max_queued"] = min(max(int(values.get("max_queued") or 3), 1), 50) + return values + + def images_ready(db: DBSession) -> bool: """Whether image generation can actually happen. diff --git a/src/lembas/services/tool_labels.py b/src/lembas/services/tool_labels.py index 97f3474..d922520 100644 --- a/src/lembas/services/tool_labels.py +++ b/src/lembas/services/tool_labels.py @@ -62,6 +62,10 @@ LABELS: dict[str, str] = { "notes_edit": "Note updated", "notes_delete": "Note deleted", "scratch_write": "Canvas written", + # Reports. + "report_write": "Report filed", + "report_search": "Reports searched", + "report_get": "Report read", "image_generate": "Image", "memory_add": "Memory saved", "memory_forget": "Memory removed", @@ -95,6 +99,9 @@ ICONS: dict[str, str] = { "notes_edit": "pencil", "notes_delete": "trash", "scratch_write": "file-text", + "report_write": "pencil", + "report_search": "search", + "report_get": "file-text", "image_generate": "image", "memory_add": "star", "memory_forget": "trash", @@ -133,6 +140,9 @@ ACTIONS: dict[str, str] = { "notes_edit": "Change a note", "notes_delete": "Delete a note", "scratch_write": "Write in the canvas", + "report_write": "File a report", + "report_search": "Search reports", + "report_get": "Read a report", "image_generate": "Generate an image", "memory_add": "Remember something", "memory_forget": "Forget something", @@ -155,6 +165,12 @@ DETAIL_KEYS: dict[str, str] = { "web_search": "query", "knowledge_search": "query", "notes_search": "query", + "report_search": "query", + # The title, not the body: a card has room for a line and the body is the + # report. Editable for the same reason the image prompt is -- correcting + # what a report will be called before it is filed is cheap, and renaming + # one afterwards means finding it first. + "report_write": "title", "job_stop": "id", # The thing being agreed to is what will be drawn, not which sampler draws # it. Also what makes the box on the card editable: a prompt corrected diff --git a/src/lembas/services/tools.py b/src/lembas/services/tools.py index 726e604..7262e8d 100644 --- a/src/lembas/services/tools.py +++ b/src/lembas/services/tools.py @@ -32,9 +32,10 @@ from typing import Any from sqlalchemy import select from sqlalchemy.orm import Session as DBSession -from lembas.db.models import AUTHOR_MODEL, Chat, User +from lembas.db.models import AUTHOR_MODEL, KIND_TASK, SOURCE_CHAT, Chat, User from lembas.db.session import session_scope from lembas.services import prompts as prompts_service +from lembas.services import reports as reports_service from lembas.services import scratch as scratch_service from lembas.services import search as search_service from lembas.services import settings_store @@ -119,6 +120,16 @@ FAMILY_AGENT = "agent" # the whole cost of this one is somewhere else. FAMILY_IMAGE = "image" +# Filing a finished piece of work where the reader will find it later. +# Deliberately not part of `notes`, and the line is the one a note already +# draws from the other side: a note is something to be found again *by the +# model*, searched for mid-conversation and edited when it turns out to be +# wrong. A report is addressed to a person, read once, and never answered -- +# so it is the destination for work nobody was watching, which is exactly what +# a note is not. Narrowing notes off must not take it away, and turning it on +# must not hand out the notebook. +FAMILY_REPORT = "report" + # The built-in families, in the order they are offered. FAMILIES = ( FAMILY_SEARCH, @@ -130,6 +141,7 @@ FAMILIES = ( FAMILY_SCRATCH, FAMILY_ASK, FAMILY_IMAGE, + FAMILY_REPORT, FAMILY_AGENT, ) @@ -670,6 +682,93 @@ async def _run_memory_forget(context: ToolContext, args: dict[str, Any]) -> Tool ) +# --- Reports ----------------------------------------------------------------- +async def _run_report_write(context: ToolContext, args: dict[str, Any]) -> ToolOutcome: + title = str(args.get("title") or "").strip() + body = str(args.get("body") or "").strip() + summary = str(args.get("summary") or "").strip() + if not body: + return ToolOutcome( + "A report needs a body. Write what you found, not a note saying you found it.", + {"name": "report_write", "status": "error", "error": "Empty body."}, + ) + with session_scope() as db: + user = db.get(User, context.owner_id) + if user is None: + return ToolOutcome( + "That report could not be filed.", + {"name": "report_write", "status": "error", "error": "No such owner."}, + ) + report = reports_service.create( + db, + owner=user, + title=title, + body=body, + summary=summary, + source=SOURCE_CHAT, + source_id=context.chat_id or "", + model_id=context.model_id or "", + ) + return ToolOutcome( + f"Filed report {report.id} — {report.title!r}. " + "The reader will find it under Reports; they cannot reply to it there.", + { + "name": "report_write", + "query": report.title, + "status": "ok", + "results": [{"title": report.title, "id": report.id}], + }, + ) + + +async def _run_report_search(context: ToolContext, args: dict[str, Any]) -> ToolOutcome: + query = str(args.get("query") or "").strip() + with session_scope() as db: + user = db.get(User, context.owner_id) + found = ( + reports_service.search(db, user, query, limit=8) + if query + else reports_service.recent(db, user, limit=8) + ) + event = { + "name": "report_search", + "query": query, + "status": "ok", + "results": [{"title": r.title, "id": r.id} for r in found], + } + if not found: + return ToolOutcome("There are no reports matching that.", event) + lines = ["Reports:"] + for report in found: + when = report.created_at.strftime("%Y-%m-%d %H:%M") + lines.append( + f"\n[{report.id}] {when} — {report.title}\n{reports_service.snippet(report)}" + ) + lines.append("\nUse report_get with an id to read one in full.") + return ToolOutcome("\n".join(lines), event) + + +async def _run_report_get(context: ToolContext, args: dict[str, Any]) -> ToolOutcome: + with session_scope() as db: + user = db.get(User, context.owner_id) + report = reports_service.get(db, str(args.get("id") or ""), user) + if report is None: + return ToolOutcome( + "There is no such report.", + {"name": "report_get", "status": "error", "error": "Not found."}, + ) + when = report.created_at.strftime("%Y-%m-%d %H:%M") + return ToolOutcome( + f"{report.title}\nFiled {when}\n\n{report.body}", + { + "name": "report_get", + "query": report.title, + "status": "ok", + "results": [{"title": report.title, "id": report.id}], + }, + ) + + # --- Skills ------------------------------------------------------------------ async def _run_skill_get(context: ToolContext, args: dict[str, Any]) -> ToolOutcome: name = str(args.get("name") or "").strip() @@ -971,6 +1070,73 @@ REGISTRY: dict[str, ToolDef] = { run=_run_memory_forget, risk=RISK_WRITE, ), + ToolDef( + name="report_write", + family=FAMILY_REPORT, + description=( + "File a report: a finished piece of work, written for the person " + "to read later. Use this when you have been asked for one, and " + "when you finish a long piece of work whose result is worth " + "keeping — an investigation, a summary of what you found, an " + "account of what you changed. A report is read on its own, away " + "from this conversation and possibly long after it, and THE " + "READER CANNOT REPLY TO IT. So write it whole: say what you were " + "asked, what you found and what you conclude, and do not refer " + "to 'the above' or ask a question at the end." + ), + parameters=_object( + { + "title": { + **_STRING, + "description": ( + "One line naming what this is about, as it will appear " + "in a list of dozens. 'Build failures this week', not " + "'Report' or 'Results'." + ), + }, + "body": { + **_STRING, + "description": ( + "The report itself, in Markdown. Headings and lists are " + "rendered. This is the whole of what the reader gets, so " + "it should stand on its own with no further context." + ), + }, + "summary": { + **_STRING, + "description": ( + "One sentence for the list page, so the report can be " + "triaged without opening it. Say the finding, not the " + "subject: 'Three tests fail on ARM only', not 'About the " + "test failures'. Omit it and the first line of the body " + "is used instead." + ), + }, + }, + ["title", "body"], + ), + run=_run_report_write, + risk=RISK_WRITE, + ), + ToolDef( + name="report_search", + family=FAMILY_REPORT, + description=( + "Search reports filed earlier, yours and the reader's. With no " + "query, returns the most recent. Worth doing before writing a " + "recurring report, so this week's can say what changed since last " + "week's rather than repeating it." + ), + parameters=_object({"query": _STRING}, []), + run=_run_report_search, + ), + ToolDef( + name="report_get", + family=FAMILY_REPORT, + description="Read one report in full, by the id a search returned.", + parameters=_object({"id": _STRING}, ["id"]), + run=_run_report_get, + ), ToolDef( name="skill_get", family=FAMILY_SKILLS, @@ -1153,12 +1319,23 @@ def _family_allowed( # the shape `resolve_tools` already refuses for `skill_get` with an # empty library. `settings_store.images_ready` answers all three. return bool(allowed.get("tools.image") and images) - if gate in (FAMILY_CUSTOM, FAMILY_MCP, FAMILY_ASK, FAMILY_AGENT, FAMILY_SCRATCH): + if gate in ( + FAMILY_CUSTOM, + FAMILY_MCP, + FAMILY_ASK, + FAMILY_AGENT, + FAMILY_SCRATCH, + FAMILY_REPORT, + ): # Deliberately without `library.use`: an HTTP endpoint an administrator # wrote has nothing to do with this person's own documents and notes, # and requiring the library permission for it would be a coincidence of # naming rather than a rule. The same goes for being asked a question, - # and for a pad that belongs to this chat and goes nowhere else. + # for a pad that belongs to this chat and goes nowhere else, and for + # filing a report -- which is addressed to the reader rather than kept + # for the model, and is the fallback destination for scheduled work, so + # gating it behind the library would switch that off for anyone whose + # instance does not use one. return bool(allowed.get(f"tools.{gate}")) return bool(allowed.get(f"tools.{gate}") and allowed.get("library.use")) @@ -1293,6 +1470,16 @@ def resolve_tools(db: DBSession, chat: Chat, user: User | None) -> ToolSet: off = scoped_off(chat) empty_library = not skills_service.count_enabled(db, user, exclude=scoped_skills_off(chat)) + # A scheduled task runs with nobody present, so `ask_user` cannot work here: + # it pauses the reply and waits for a POST that will never come, until + # `approval_timeout` expires -- a run that silently does nothing for fifteen + # minutes and then gives up. Withdrawn from the offered set rather than + # merely discouraged in `core.unattended`, because a rule living only in a + # system message is one a page the model just read can argue with. The + # fragment is the half that stops it *planning* around a tool it has not got. + if chat is not None and chat.kind == KIND_TASK: + off = off | {FAMILY_ASK} + return ToolSet( tuple( tool diff --git a/src/lembas/services/wake.py b/src/lembas/services/wake.py new file mode 100644 index 0000000..ada7f7f --- /dev/null +++ b/src/lembas/services/wake.py @@ -0,0 +1,120 @@ +"""Starting a reply when nobody asked for one just now. + +Two things need this: a background job finishing on the far side, and a schedule +coming due. Both are the same problem — put a turn into a chat from outside any +request, and get it answered — and both depend on the same invariant, which is +subtle enough that having two copies of it is how one of them drifts. + +**The invariant.** A chat may have one generation running at a time. So the +check and the writes happen under a per-chat lock with **no `await` between +them**: two things waking one chat at the same moment cannot each spin up a +reply, because the second sees the first's already live and leaves its turn +`queued` for that reply's `_inject`/`_drain` to deliver. + +**The role is load-bearing.** The turn is written as `user`, because `_inject` +sends a queued turn verbatim and `build_messages` must keep seeing a user turn +where one belongs. What stops the transcript claiming the reader typed it is +`Message.machine`, which changes the bubble and nothing about the request. The +framing the *model* reads therefore has to live in the words -- which is why +every caller passes prose that names itself, the way `execute_plan` quotes a +plan rather than asserting it. +""" + +from __future__ import annotations + +import asyncio +import logging + +log = logging.getLogger(__name__) + +_LOCKS: dict[str, asyncio.Lock] = {} + + +def lock_for(chat_id: str) -> asyncio.Lock: + """The lock for one chat, made on first use. + + Not cleaned up: a lock is two pointers, chats are finite, and a reaper for + these would be able to delete one while a waiter held it. + """ + existing = _LOCKS.get(chat_id) + if existing is None: + existing = _LOCKS[chat_id] = asyncio.Lock() + return existing + + +async def wake_chat(chat_id: str, content: str, *, model_id: str = "") -> str: + """Put a turn into a chat and get it answered. Returns the assistant message + id if a reply was started, or "" if the turn was queued for one already + running. + + Never raises: a caller here is a watcher or a ticker, and one chat that + cannot be woken must not stop the others. + + `model_id` overrides the chat's own, which is what lets a schedule name the + model it wants without editing the chat. Empty means the chat decides. + """ + from lembas.db.models import ROLE_ASSISTANT, ROLE_USER, Chat + from lembas.db.session import session_scope + from lembas.services import chat as chat_service + from lembas.services import generation as generation_service + + assistant_id = "" + async with lock_for(chat_id): + # Read before the writes and with nothing awaited in between. This is + # the whole invariant; moving either half out of the lock, or awaiting + # anything between them, reintroduces two concurrent replies to one + # chat -- which is a Stop button pointing at whichever bubble happens to + # come first in the document. + running = generation_service.running_for(chat_id) is not None + try: + with session_scope() as db: + chat = db.get(Chat, chat_id) + if chat is None: + return "" + chat_service.create_message( + db, chat, ROLE_USER, content, queued=running, machine=True + ) + if not running: + assistant = chat_service.create_message( + db, + chat, + ROLE_ASSISTANT, + "", + complete_=False, + model_id=model_id or chat.model_id, + ) + assistant_id = assistant.id + except Exception: # noqa: BLE001 - a failed wake must not kill its caller + log.exception("could not wake chat %s", chat_id) + return "" + + # Outside the lock: `ensure` spawns a task that will want a session of its + # own, and holding this while it starts would serialise every wake on the + # instance behind the slowest one. + if assistant_id: + generation_service.ensure(chat_id, assistant_id) + return assistant_id + + +def queued_count(chat_id: str) -> int: + """How many turns are waiting to be delivered into this chat. + + `_drain` takes **one** per reply, deliberately -- two consecutive user turns + in a request is worse than a delay. So something firing repeatedly into a + chat that is answering slowly can build a backlog nobody asked for, and the + callers here use this to stop rather than to queue for ever. + """ + from sqlalchemy import func, select + + from lembas.db.models import Message + from lembas.db.session import session_scope + + with session_scope() as db: + return int( + db.scalar( + select(func.count()) + .select_from(Message) + .where(Message.chat_id == chat_id, Message.queued.is_(True)) + ) + or 0 + ) diff --git a/src/lembas/web/static/css/admin.css b/src/lembas/web/static/css/admin.css index 44bf28f..b26510a 100644 --- a/src/lembas/web/static/css/admin.css +++ b/src/lembas/web/static/css/admin.css @@ -491,3 +491,29 @@ a.tabs__tab { text-decoration: none; } .mode-list__row { display: flex; gap: var(--sp-3); align-items: baseline; } .mode-list__row dt { flex: 0 0 5rem; color: var(--ink); } .mode-list__row dd { margin: 0; color: var(--ink-muted); font-size: var(--text-sm); } + +/* --- The schedule setup form ------------------------------------------------ */ +/* + Which "when" fieldset is showing follows the radio, with no JavaScript. `:has()` + is what makes that possible, and it is the same move the "Something else" box + on an approval card makes: a control and the thing it reveals cannot fall out + of step if the stylesheet is the only thing relating them. + + Hidden with `display: none` on the container rather than `[hidden]`, which + would need `!important` here -- see the note in app.css about `.btn` being + `inline-flex`. Fields inside a hidden fieldset are still submitted, which is + correct: `_rule_from_form` reads only the keys the chosen repeat mode uses. +*/ +.schedule-repeat { + border: 1px solid var(--border); + border-radius: var(--radius-md); + padding: var(--sp-4); + margin-bottom: var(--sp-4); + display: flex; + flex-direction: column; + gap: var(--sp-3); +} +.schedule-repeat > legend { padding: 0 var(--sp-2); } +.schedule-repeat [data-repeat] { display: none; } +.schedule-repeat:has(input[value="every"]:checked) [data-repeat="every"] { display: block; } +.schedule-repeat:has(input[value="calendar"]:checked) [data-repeat="calendar"] { display: block; } diff --git a/src/lembas/web/static/css/chat.css b/src/lembas/web/static/css/chat.css index 671a35d..5c49cce 100644 --- a/src/lembas/web/static/css/chat.css +++ b/src/lembas/web/static/css/chat.css @@ -1630,3 +1630,18 @@ font-size: var(--text-xs); } .plan__note { color: var(--ink-faint); font-size: var(--text-xs); } + +/* --- Reading backwards ------------------------------------------------------ */ +/* + The row that fetches older messages when it comes into view. Given real height + so `revealed` fires reliably -- a zero-height element at the top of a scroll + container is intersected ambiguously, and a sentinel that never fires is a + history nobody can reach. +*/ +.history-sentinel { + display: flex; + align-items: center; + justify-content: center; + padding: var(--sp-4) 0; + min-height: 2.5rem; +} diff --git a/src/lembas/web/static/js/app.js b/src/lembas/web/static/js/app.js index 54571d2..8c0129f 100644 --- a/src/lembas/web/static/js/app.js +++ b/src/lembas/web/static/js/app.js @@ -144,6 +144,36 @@ true ); + /* --- Reading backwards ------------------------------------------------- + The mirror of `scrollThread`. Messages pages in older turns above the ones + on screen, and *prepending* moves everything down by the height of what + arrived -- so without this the reader is dragged up the page the instant + the sentinel fires, which reads as a browser bug rather than as a feature. + + Height is recorded before the swap and the difference added back after, so + whatever was under the reader's eye stays there. `overflow-anchor` is not + reliable across an htmx swap, and `stick` deliberately is not touched: + reading history is not following a reply, and re-arming it here would jump + to the bottom the moment the next page landed. */ + var heightBefore = -1; + + document.body.addEventListener("htmx:beforeSwap", function (event) { + var el = event.target; + if (!el || !el.classList || !el.classList.contains("history-sentinel")) return; + var thread = document.getElementById("thread-scroll"); + heightBefore = thread ? thread.scrollHeight : -1; + }); + + document.body.addEventListener("htmx:afterSwap", function (event) { + if (heightBefore < 0) return; + var thread = document.getElementById("thread-scroll"); + if (thread) { + thread.scrollTop += thread.scrollHeight - heightBefore; + placed = thread.scrollTop; + } + heightBefore = -1; + }); + /* `toggle` does not bubble, so this has to be registered in the CAPTURE phase. Without the third argument the listener is never called and the whole thing is silently dead in every browser -- the same shape of failure as a trigger diff --git a/src/lembas/web/templates/admin/_layout.html b/src/lembas/web/templates/admin/_layout.html index 34d9bc4..f7af8a5 100644 --- a/src/lembas/web/templates/admin/_layout.html +++ b/src/lembas/web/templates/admin/_layout.html @@ -55,6 +55,10 @@ {{ icon("sparkle", "icon--sm") }} Agents + + {{ icon("clock", "icon--sm") }} + Scheduling + {{ icon("server", "icon--sm") }} MCP servers diff --git a/src/lembas/web/templates/admin/schedules.html b/src/lembas/web/templates/admin/schedules.html new file mode 100644 index 0000000..0549943 --- /dev/null +++ b/src/lembas/web/templates/admin/schedules.html @@ -0,0 +1,117 @@ +{% extends "admin/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "schedules" %} + +{% block title %}Scheduling - LLeMbas{% endblock %} +{% block heading %}Scheduling{% endblock %} + +{% block admin_content %} +

+ A schedule runs a piece of work because time has passed + rather than because somebody asked just now — a daily summary, a check every + Monday, a reminder in an hour. Each one has its own chat and replies into it, + or files a report. People make their own under Scheduled; + what you decide here is whether the feature exists and what it may spend. +

+ +
+ {{ icon("shield", "icon--sm") }} + + This is the one thing here that spends model time with nobody watching. A + schedule pointed at a chat that can run commands would run them unattended, + so a scheduled task is never an agent chat — but everything else a model can + reach, it can reach on a timer. Give Schedule work under + Groups & permissions to the people who should have it; it is off for + everybody by default. + +
+ +{% if saved %} +
{{ icon("check", "icon--sm") }} Saved.
+{% endif %} + +
+ +
+

Switch

+
+ +

+ Off by default. Turning this off stops everything firing and deletes + nothing — schedules keep their place and resume when it is turned back + on. + {% if total %} + There {{ "is" if total == 1 else "are" }} {{ total }} + schedule{{ "" if total == 1 else "s" }} on this instance, + {{ active }} of them not paused. + {% endif %} +

+
+
+ +
+

How often it looks

+
+ + +

+ Seconds. This is how late a run can be, not how often anything happens: + the finest a schedule can be set to is one minute, so anything under + that buys nothing. One indexed query per tick. +

+
+ +
+ + +

+ Seconds. A floor on how often one schedule may come round. Raise it if + people are setting things to run more often than the work takes. +

+
+
+ +
+

What it may spend

+
+ + +

+ Refused at the point of creation, with the reason. Existing schedules + over a lowered limit keep running; only new ones are refused. +

+
+ +
+ + +

+ Fifty schedules due at nine o'clock must not open fifty replies against + one endpoint. The rest wait their turn rather than being dropped. +

+
+ +
+ + +

+ A schedule that comes round faster than its chat can answer would build + a backlog for ever. Past this, a run is skipped and says so on the + schedule rather than joining the queue. +

+
+
+ +
+ +
+
+{% endblock %} diff --git a/src/lembas/web/templates/chat/index.html b/src/lembas/web/templates/chat/index.html index dc2b763..27a64a2 100644 --- a/src/lembas/web/templates/chat/index.html +++ b/src/lembas/web/templates/chat/index.html @@ -352,7 +352,21 @@ hx-sync="this:drop"> {% endif %} - {% include "chat/_composer.html" %} + {# + A task chat has no composer, and that is the section rather than a + restriction on it: a schedule's chat is written into by the schedule and + read by a person, so there is nothing for them to send. The controls + that *do* apply — run it now, pause it, edit it — take the row's place. + + Suppressed by not including it, not by hiding it. `chat/_composer.html` + is the only thing that posts a message, so its absence is the guarantee; + a hidden one would still be a form anybody could post to. + #} + {% if chat and chat.kind == "task" %} + {% include "schedules/_strip.html" %} + {% else %} + {% include "chat/_composer.html" %} + {% endif %} {% endif %} diff --git a/src/lembas/web/templates/messages/_history.html b/src/lembas/web/templates/messages/_history.html new file mode 100644 index 0000000..3c1af21 --- /dev/null +++ b/src/lembas/web/templates/messages/_history.html @@ -0,0 +1,35 @@ +{# + One page of older turns, oldest first, plus a fresh sentinel above them when + there is more. + + No root element, and the sentinel replaces *itself* with this whole fragment + (`hx-swap="outerHTML"`), so each page lands exactly where the previous + sentinel was and the next one ends up at the top again. + + The turns are rendered by `chat/_message.html`, the same template as + everything else in the thread. That is what makes scrolling up look identical + to scrolling down: there is no second way of drawing a message here, so there + is nothing that could drift from it. +#} +{% if more_before %} +{# + `hx-target="this"` is stated rather than inherited. This fragment is swapped + into a page whose composer form carries `hx-target="#thread"`, and htmx + resolves that by walking up the DOM -- the jobs chip has already demonstrated + once what an unstated target does to a transcript. +#} +
+ Loading earlier messages… +
+{% endif %} + +{% for message in messages %} + {% with body_html = bodies.get(message.id, "") %} + {% include "chat/_message.html" %} + {% endwith %} +{% endfor %} diff --git a/src/lembas/web/templates/messages/index.html b/src/lembas/web/templates/messages/index.html new file mode 100644 index 0000000..868f2ec --- /dev/null +++ b/src/lembas/web/templates/messages/index.html @@ -0,0 +1,112 @@ +{% extends "base.html" %} +{% from "_macros.html" import icon, mark %} +{# + Messages: one conversation per person, opened at the most recent turns. + + The ordinary chat shell, with two differences. It opens on the live chunk + rather than on everything, and above that sits a sentinel that fetches the + page before it when scrolled into view. Everything else — the composer, the + bubbles, the tail poller, the streaming shell — is the same machinery, which + is the whole reason this is a `Chat` with a different `kind`. +#} + +{% block title %}Messages - LLeMbas{% endblock %} + +{% block head %} + +{% endblock %} + +{% block body_attrs %} data-authenticated="true"{% endblock %} + +{% block body %} +
+ {% include "partials/sidebar.html" %} + +
+
+

{{ icon("chat", "icon--sm") }} Messages

+
+ {% if schedules %} +
+ + {{ icon("clock", "icon--sm") }} {{ schedules | length }} scheduled + +
+
Posting into this conversation
+ {% for item in schedules %} + + + {{ item.row.title }} + {{ item.summary }} + + {% if not item.row.enabled %} + paused + {% endif %} + + {% endfor %} +
+
+ {% endif %} + {% if can.get("schedule.use") %} + {{ icon("plus", "icon--sm") }} Schedule + {% endif %} +
+
+ +
+
+ {% if not messages %} +
+ {{ mark(cls="empty__mark", uid="messages") }} +

Nothing said yet

+

+ One conversation that keeps going. Anything scheduled to write here + will arrive in it. +

+
+ {% endif %} + + {# + The sentinel goes above the turns and replaces itself with the page + before, plus a fresh sentinel. Rendered only when there is something + earlier, so a short conversation has no loading row at the top. + #} + {% if more_before %} +
+ Loading earlier messages… +
+ {% endif %} + + {% for message in messages %} + {% with body_html = bodies.get(message.id, "") %} + {% include "chat/_message.html" %} + {% endwith %} + {% endfor %} +
+
+ + {# + Outside `#thread` and outside the composer form, for the two reasons + `tests/test_chat_tail.py` already walks the chat page to enforce. + #} + + + {% include "chat/_composer.html" %} +
+
+{% endblock %} + +{% block scripts %} + + + +{% endblock %} diff --git a/src/lembas/web/templates/partials/_sidebar_sections.html b/src/lembas/web/templates/partials/_sidebar_sections.html new file mode 100644 index 0000000..3c8d4fc --- /dev/null +++ b/src/lembas/web/templates/partials/_sidebar_sections.html @@ -0,0 +1,49 @@ +{% from "_macros.html" import icon %} +{# + Reports, Messages and Scheduled: the three places activity lands that is not + a conversation you started. + + Placed below pinned models and above the folder tree, and deliberately + OUTSIDE `#sidebar-tree`. Pinned models and "New chat" are outside it for the + same reason: they are not part of the Chat/Agent fork, so they must not vanish + or be re-rendered when the switch moves. Their unread dots ride the existing + out-of-band `/api/chats/unread` response, which is also outside the tree. + + Not the footer, where Library and Connections live. Those are places you go to + configure or consult something; these receive activity while you are elsewhere + and therefore need a dot, and a dot down in the footer reads as a settings + notification rather than as news. Scheduled also grows, and the footer is for + fixed destinations. + + Each entry is gated on its own permission, so a section somebody cannot use is + absent rather than present and refusing. +#} + diff --git a/src/lembas/web/templates/partials/sidebar.html b/src/lembas/web/templates/partials/sidebar.html index 6a5d4e3..27e8107 100644 --- a/src/lembas/web/templates/partials/sidebar.html +++ b/src/lembas/web/templates/partials/sidebar.html @@ -21,6 +21,7 @@ diff --git a/src/lembas/web/templates/reports/_layout.html b/src/lembas/web/templates/reports/_layout.html new file mode 100644 index 0000000..010d5a4 --- /dev/null +++ b/src/lembas/web/templates/reports/_layout.html @@ -0,0 +1,43 @@ +{% extends "base.html" %} +{% from "_macros.html" import icon %} +{# + The Reports shell. + + Keeps the chat sidebar, like the library does: you come here from a + conversation to read what something filed and go straight back. + + Deliberately no tab bar. The library has three stores that belong together and + are switched between; this is one thing. A row of tabs with a single tab in it + is a control that answers a question nobody asked. + + There is no composer here and no element that fetches, which is the point of + the section: a report is read, never answered. +#} + +{% block head %} + + +{% endblock %} + +{% block body_attrs %} data-authenticated="true"{% endblock %} + +{% block body %} +
+ {% include "partials/sidebar.html" %} + +
+
+

{% block heading %}Reports{% endblock %}

+
{% block actions %}{% endblock %}
+
+ + {# `.page` content needs `.admin-scroll` around it or `.main`'s min-height:0 + leaves it overflowing the viewport with nothing to scroll. #} +
+
+ {% block reports_content %}{% endblock %} +
+
+
+
+{% endblock %} diff --git a/src/lembas/web/templates/reports/detail.html b/src/lembas/web/templates/reports/detail.html new file mode 100644 index 0000000..edb614f --- /dev/null +++ b/src/lembas/web/templates/reports/detail.html @@ -0,0 +1,47 @@ +{% extends "reports/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "reports" %} + +{% block title %}{{ report.title }} - LLeMbas{% endblock %} +{% block heading %}{{ report.title }}{% endblock %} +{% block actions %} +{{ icon("chevron-left", "icon--sm") }} All reports +{% endblock %} + +{% block reports_content %} +

+ Filed + + {%- if report.source == "schedule" %} by a schedule + {%- elif report.source == "chat" %} from a chat + {%- endif %} + {%- if report.model_id %} · {{ report.model_id }}{% endif %}. +

+ +{% if report.error %} +
+ {{ icon("warning", "alert__icon") }} + This run did not finish cleanly. {{ report.error }} +
+{% endif %} + +{# + `body_html` has been through services/markdown.py, which is the one path + allowed to emit HTML here. A report is model output and hard rule 6 applies to + it exactly as it does to a reply. +#} +
+ {{ body_html | safe }} +
+ +
+
+ +
+
+{% endblock %} diff --git a/src/lembas/web/templates/reports/index.html b/src/lembas/web/templates/reports/index.html new file mode 100644 index 0000000..4454c81 --- /dev/null +++ b/src/lembas/web/templates/reports/index.html @@ -0,0 +1,60 @@ +{% extends "reports/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "reports" %} + +{% block title %}Reports - LLeMbas{% endblock %} +{% block heading %}Reports{% endblock %} + +{% block reports_content %} +

+ Finished work, filed to be read later. A model writes one when you ask for it + or when it finishes something worth keeping, and anything running on a + schedule leaves its result here. Nothing on this page can be replied to. +

+ +
+ + + {% if q %}Clear{% endif %} +
+ +{% if not reports %} +
+ {{ icon("archive", "empty__mark") }} +

{{ "Nothing found" if q else "No reports yet" }}

+

+ {% if q %} + No report matches “{{ q }}”. + {% else %} + Ask a model to write one up when it has finished looking into something, + and it will appear here. + {% endif %} +

+
+{% else %} +
    + {% for report in reports %} +
  • +
    + + {{ report.title }} + +
    + + {% if report.summary %} — {{ report.summary }}{% endif %} +
    +
    +
    + {% if report.error %}failed{% endif %} + {% if report.source == "schedule" %}scheduled{% endif %} + {% if report.unread %}new{% endif %} +
    +
    • + {% endfor %} +
+{% include "library/_pager.html" %} +{% endif %} +{% endblock %} diff --git a/src/lembas/web/templates/schedules/_form.html b/src/lembas/web/templates/schedules/_form.html new file mode 100644 index 0000000..0345240 --- /dev/null +++ b/src/lembas/web/templates/schedules/_form.html @@ -0,0 +1,128 @@ +{% from "_macros.html" import icon %} +{# + The manual rule form, shared by new and edit. + + Every field maps to one key `services/schedule/rule.py:validate` understands, + and nothing here validates: there is one normaliser, it is total, and it is + the same one a compiled rule goes through. Two validators would be two ideas + of what a legal schedule is. + + The repeat fieldsets are shown and hidden by the radio's `:has()` selector in + admin.css — no JavaScript, and nothing that can fall out of step with the + control, which is the same reasoning the "Something else" box on an approval + card follows. +#} +
+ + +

What this appears as in the list.

+
+ +
+ + +

+ Written for a model that will read it with no conversation around it, so say + the whole thing. Nobody will be there to answer a question about it. +

+
+ +
+ + +
+ +
+ When + + + + + +
+ +
+ + +
+

+ Leave the date empty for a repeating schedule and it begins now. + Times are {{ timezone }}. +

+
+ +
+ +
+ + +
+
+ +
+ +
+ {% for index, name in weekday_names %} + + {% endfor %} +
+

Leave all of them unticked for every day.

+ + + +

+ One or more times, separated by commas. These are wall-clock times: 09:00 + stays 09:00 when the clocks change. +

+ + + +

Days of the month, if you want it narrower. Optional.

+
+
+ +
+ + +

Number of runs. Leave it at 0 to keep going until you stop it.

+
+ +
+ + +

Optional. Nothing runs after this date.

+
diff --git a/src/lembas/web/templates/schedules/_layout.html b/src/lembas/web/templates/schedules/_layout.html new file mode 100644 index 0000000..c71bba4 --- /dev/null +++ b/src/lembas/web/templates/schedules/_layout.html @@ -0,0 +1,39 @@ +{% extends "base.html" %} +{% from "_macros.html" import icon %} +{# + The Scheduled shell: the list and the setup forms. + + A schedule's own *chat* is not rendered here — it is the ordinary chat page + with its composer replaced, which is the whole reason a task chat is a `Chat`. +#} + +{% block head %} + + +{% endblock %} + +{% block body_attrs %} data-authenticated="true"{% endblock %} + +{% block body %} +
+ {% include "partials/sidebar.html" %} + +
+
+

{% block heading %}Scheduled{% endblock %}

+
{% block actions %}{% endblock %}
+
+ +
+
+ {% if error %} +
+ {{ icon("warning", "alert__icon") }} {{ error }} +
+ {% endif %} + {% block schedules_content %}{% endblock %} +
+
+
+
+{% endblock %} diff --git a/src/lembas/web/templates/schedules/_strip.html b/src/lembas/web/templates/schedules/_strip.html new file mode 100644 index 0000000..e06ac15 --- /dev/null +++ b/src/lembas/web/templates/schedules/_strip.html @@ -0,0 +1,68 @@ +{% from "_macros.html" import icon %} +{# + What sits where a task chat's composer would be. + + Three plain forms rather than htmx: each one changes what the page says about + itself — when it next runs, whether it is paused, a whole new reply — so a + redirect back is the honest response, and a refresh cannot re-submit it. + + Every button here is the reader acting on their own schedule, so nothing is + checked against the chat's mode. That is the same argument the terminal panel, + the directory browser and the jobs panel already make: the model is governed, + the person is not. +#} +
+
+ {% if schedule %} +
+
+ {{ schedule_summary }} +
+ {%- if not schedule.enabled %} + Paused. + {%- elif schedule_next %} + Next run {{ schedule_next.strftime("%a %-d %b, %H:%M") }}. + {%- else %} + Nothing left to run. + {%- endif %} + {%- if schedule.fired_count %} Run {{ schedule.fired_count }} times so far.{% endif %} + {%- if schedule.target == "report" %} Files a report each time.{% endif %} +
+ {% if schedule.last_error %} +
{{ schedule.last_error }}
+ {% endif %} +
+ +
+
+ +
+
+ + +
+ + {{ icon("gear", "icon--sm") }} Edit + +
+
+ {% else %} + {# + The schedule was removed but its chat was kept, which is the default when + somebody deletes one. Said plainly rather than left as an empty bar. + #} +

+ This chat belonged to a schedule that has been removed. It is kept as a + record of what was done. +

+ {% endif %} +
+
diff --git a/src/lembas/web/templates/schedules/edit.html b/src/lembas/web/templates/schedules/edit.html new file mode 100644 index 0000000..4ee3747 --- /dev/null +++ b/src/lembas/web/templates/schedules/edit.html @@ -0,0 +1,37 @@ +{% extends "schedules/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "scheduled" %} + +{% block title %}{{ schedule.title }} - LLeMbas{% endblock %} +{% block heading %}{{ schedule.title }}{% endblock %} +{% block actions %} +Open its chat +{% endblock %} + +{% block schedules_content %} +
+ {% include "schedules/_form.html" %} +
+ + Cancel +
+
+ +
+

Remove

+

+ Stops it running. Its chat is kept by default and becomes an ordinary one, + so the transcript of everything it has already done stays where it is. +

+
+ + +
+
+{% endblock %} diff --git a/src/lembas/web/templates/schedules/index.html b/src/lembas/web/templates/schedules/index.html new file mode 100644 index 0000000..5f535e4 --- /dev/null +++ b/src/lembas/web/templates/schedules/index.html @@ -0,0 +1,54 @@ +{% extends "schedules/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "scheduled" %} + +{% block title %}Scheduled - LLeMbas{% endblock %} +{% block heading %}Scheduled{% endblock %} +{% block actions %} + + {{ icon("plus", "icon--sm") }} New scheduled task + +{% endblock %} + +{% block schedules_content %} +

+ Work that runs on its own, whether or not you are here. Each one has its own + chat, and replies into it every time it comes round. +

+ +{% if not schedules %} +
+ {{ icon("clock", "empty__mark") }} +

Nothing scheduled

+

+ Set something to run later — a daily summary, a check every Monday + morning, a reminder in an hour. +

+
+{% else %} +
    + {% for item in schedules %} +
  • +
    + {{ item.row.title }} +
    + {{ item.summary }} + {%- if item.next and item.row.enabled %} + · next {{ item.next.strftime("%a %-d %b, %H:%M") }} + {%- endif %} + {%- if item.row.fired_count %} · run {{ item.row.fired_count }} times{% endif %} +
    + {% if item.row.last_error %} +
    {{ item.row.last_error }}
    + {% endif %} +
    +
    + {% if item.row.target == "report" %}files a report{% endif %} + {% if not item.row.enabled %}paused{% endif %} + Edit +
    +
    • + {% endfor %} +
+{% endif %} +{% endblock %} diff --git a/src/lembas/web/templates/schedules/new.html b/src/lembas/web/templates/schedules/new.html new file mode 100644 index 0000000..462823a --- /dev/null +++ b/src/lembas/web/templates/schedules/new.html @@ -0,0 +1,78 @@ +{% extends "schedules/_layout.html" %} +{% from "_macros.html" import icon %} +{% set section = "scheduled" %} + +{% block title %}New scheduled task - LLeMbas{% endblock %} +{% block heading %}What do you want to schedule?{% endblock %} +{% block actions %} +Cancel +{% endblock %} + +{% block schedules_content %} +{# + One question first, and the detail worked out from the answer. The manual + fields are on the same page rather than behind a second screen: somebody who + already knows exactly when it should run should not have to describe it in + prose and hope, and the fields are also where a compile that did not work out + lands — so they cannot be a fallback that only appears when something breaks. +#} +
+
+ + +

+ Times are read as {{ timezone }}. Say how often, and what should happen. +

+
+
+ +
+
+ +{% if compiled %} + {% if compiled.reason %} +
+ {{ icon("warning", "alert__icon") }} {{ compiled.reason }} +
+ {% elif summary %} +
+ {{ icon("check", "alert__icon") }} + + This will run {{ summary }}. Check it below and change + anything that is not what you meant. + +
+ {% endif %} +{% endif %} + +
+ {# + Pre-filled from the compile when there was one. `schedule` is None here, so + the shared fields fall back to their empty state — which is why the two + compiled text values are written in explicitly rather than left to it. + #} + {% include "schedules/_form.html" %} + + {% if models %} +
+ + +
+ {% endif %} + +
+ +
+
+{% endblock %} diff --git a/src/lembas/web/templates/settings.html b/src/lembas/web/templates/settings.html index d41ae34..b2bc292 100644 --- a/src/lembas/web/templates/settings.html +++ b/src/lembas/web/templates/settings.html @@ -177,6 +177,37 @@ +
+

Timezone

+

+ What time a model is told it is, and the zone anything you + schedule runs in. Leave it unset to follow the server. +

+ {# + A plain form posting to its own route, not the composer's + pattern: this page has no htmx target to swap into and a + redirect back is what stops a refresh re-submitting it. + #} +
+ + +
+ {% if saved == "timezone" %} +

Saved. It is {{ local_now }} where you are.

+ {% elif error == "timezone" %} +

That is not a timezone this server knows about.

+ {% else %} +

It is currently {{ local_now }} where you are.

+ {% endif %} +
+

Install as an app

diff --git a/tests/conftest.py b/tests/conftest.py index 96b4d77..7570d91 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -102,6 +102,36 @@ def fresh_terminal_registry() -> Iterator[None]: _clear() +@pytest.fixture(autouse=True) +def fresh_schedule_ticker() -> Iterator[None]: + """Stop the schedule ticker and forget any firings, for the same reason. + + The ticker is a module-level task like the terminal reaper, and a firing is + a task holding a chat id. One left running would wake up inside the next + test's event loop, against the next test's database, and fire something + nobody in that test has ever heard of. + + The wake locks go too: they are keyed on chat id, and `make_chat` recycles + ids freely across a session. + """ + from lembas.services import wake as wake_service + from lembas.services.schedule import ticker as ticker_service + + def _clear() -> None: + running = ticker_service._TICKER + if running is not None: + running.cancel() + ticker_service._TICKER = None + for task in list(ticker_service._FIRING): + task.cancel() + ticker_service._FIRING.clear() + wake_service._LOCKS.clear() + + _clear() + yield + _clear() + + @pytest.fixture(autouse=True) def fresh_project_index() -> Iterator[None]: """Empty the directory-listing cache between tests, for the third time. diff --git a/tests/test_agent_policy.py b/tests/test_agent_policy.py index 77f0d08..09da0ea 100644 --- a/tests/test_agent_policy.py +++ b/tests/test_agent_policy.py @@ -266,6 +266,10 @@ def test_the_builtins_that_change_things_say_so(): "memory_forget", "skill_create", "skill_edit", + # Filing a report writes a durable artefact of the reader's, the same + # class as a note. Plan mode meaning "look but do not touch" has to mean + # this too, even though what it touches is a page rather than a machine. + "report_write", } diff --git a/tests/test_chat.py b/tests/test_chat.py index b994f57..0310951 100644 --- a/tests/test_chat.py +++ b/tests/test_chat.py @@ -836,7 +836,12 @@ def test_the_unread_poll_reports_dots(client: TestClient, db, registered, make_c response = client.get("/api/chats/unread") assert f'id="unread-{chat_id}"' in response.text - assert "hidden" not in response.text + # This chat's own span, not the whole body: the response also carries the + # section dots, and one of those being hidden is right rather than wrong. + chat_dot = next( + span for span in response.text.split(" User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _fill(db, chat: Chat, count: int, *, day: int = 1) -> list[Message]: + """`count` alternating turns, a minute apart so the order is unambiguous.""" + start = datetime(2026, 1, day, tzinfo=UTC) + rows = [] + for index in range(count): + rows.append( + Message( + chat_id=chat.id, + role="user" if index % 2 == 0 else "assistant", + content=f"turn {index}", + created_at=start + timedelta(minutes=index), + complete=True, + ) + ) + db.add_all(rows) + db.commit() + return rows + + +# --- The conversation itself ------------------------------------------------------ +def test_there_is_exactly_one_per_person(client: TestClient, db, registered): + """Get-or-create, so a schedule can post here before anybody has opened the + page — the second deliberate exception to "chats are created lazily".""" + first = messages_service.for_user(db, _user(db)) + second = messages_service.for_user(db, _user(db)) + + assert first.id == second.id + assert first.kind == KIND_MESSAGES + assert len(db.scalars(select(Chat).where(Chat.kind == KIND_MESSAGES)).all()) == 1 + + +def test_it_is_not_in_the_chat_tree(client: TestClient, db, registered): + """It has a section of its own. This is the kind-leakage trap again, from + the other side.""" + from lembas.api.pages import sidebar_context + + conversation = messages_service.for_user(db, _user(db)) + listed = {c.id for c in sidebar_context(db, _user(db))["unfiled_chats"]} + + assert conversation.id not in listed + + +# --- What reaches the model ------------------------------------------------------- +def test_the_request_does_not_grow_with_the_conversation( + client: TestClient, db, registered +): + """The whole point. A conversation meant to run for years cannot all be + sent, and a request that grows until the endpoint refuses it is the failure + nobody sees coming — there is nothing wrong on screen right up until it + stops working. + """ + from lembas.services import chat as chat_service + + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 10) + short = chat_service.build_messages(db, conversation) + + _fill(db, conversation, 300) + long = chat_service.build_messages(db, conversation) + + assert len(short) == 10 + assert len(long) == messages_service.LIVE_CHUNK + assert len(long) < len(short) + 300 + + +def test_it_is_the_most_recent_turns_that_are_sent(client: TestClient, db, registered): + from lembas.services import chat as chat_service + + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, messages_service.LIVE_CHUNK + 20) + + payload = chat_service.build_messages(db, conversation) + bodies = [entry["content"] for entry in payload] + + assert bodies[-1] == f"turn {messages_service.LIVE_CHUNK + 19}" + assert "turn 0" not in bodies + + +def test_an_ordinary_chat_still_sends_everything(client: TestClient, db, registered): + """The bound is one branch on one kind. A chat is not silently truncated.""" + from lembas.services import chat as chat_service + + ordinary = Chat(user_id=_user(db).id, model_id="m") + db.add(ordinary) + db.commit() + _fill(db, ordinary, messages_service.LIVE_CHUNK + 20) + + assert len(chat_service.build_messages(db, ordinary)) == messages_service.LIVE_CHUNK + 20 + + +def test_compaction_never_fires_on_it(client: TestClient, db, registered): + """Two mechanisms narrowing one transcript is how a summary ends up + summarising a summary — and this one would be summarising turns that are + already outside the request.""" + from lembas.services import compaction as compaction_service + + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 200) + + assert compaction_service.should_compact(db, conversation) is False + + +# --- Nothing is lost --------------------------------------------------------------- +def test_every_turn_is_kept_however_old(client: TestClient, db, registered): + """Bounded in the request, unbounded on disk. Deliberately not folded into + text: the visible conversation would be identical either way, so the only + thing destroying the rows would buy is disk — against irreversibility.""" + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 250) + + assert messages_service.count(db, conversation) == 250 + + +# --- Reading backwards -------------------------------------------------------------- +def test_the_page_opens_on_the_latest_chunk(client: TestClient, db, registered): + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 120) + + body = client.get("/messages").text + + assert "turn 119" in body + assert "turn 0" not in body + assert "history-sentinel" in body + + +def test_a_short_conversation_has_no_sentinel(client: TestClient, db, registered): + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 5) + + assert "history-sentinel" not in client.get("/messages").text + + +def test_scrolling_up_returns_the_page_before(client: TestClient, db, registered): + conversation = messages_service.for_user(db, _user(db)) + rows = _fill(db, conversation, 200) + oldest_shown = rows[-messages_service.LIVE_CHUNK] + + response = client.get(f"/api/messages/history?before={oldest_shown.id}") + + assert response.status_code == 200 + assert f"turn {200 - messages_service.LIVE_CHUNK - 1}" in response.text + # The turn it was asked to go before is not repeated. + assert f">turn {200 - messages_service.LIVE_CHUNK}<" not in response.text + + +def test_a_cursor_it_cannot_place_is_answered_with_204( + client: TestClient, db, registered +): + """Never with "the oldest page": that would prepend a block the reader is + already looking at, and a duplicated transcript is something only a reload + can reconcile.""" + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 100) + + other = Chat(user_id=_user(db).id, model_id="m") + db.add(other) + db.commit() + stray = Message(chat_id=other.id, role="user", content="elsewhere") + db.add(stray) + db.commit() + + assert client.get("/api/messages/history").status_code == 204 + assert client.get("/api/messages/history?before=nope").status_code == 204 + assert client.get(f"/api/messages/history?before={stray.id}").status_code == 204 + + +def test_the_oldest_page_stops_rather_than_looping(client: TestClient, db, registered): + conversation = messages_service.for_user(db, _user(db)) + rows = _fill(db, conversation, 3) + + response = client.get(f"/api/messages/history?before={rows[0].id}") + assert response.status_code == 204 + + +def test_two_turns_sharing_a_timestamp_are_each_returned_once( + client: TestClient, db, registered +): + """The `id` tie-breaker. Under a bare `<`, a row sharing the cursor's + microsecond can never be reached — and a message that cannot be scrolled + back to is a message that is gone.""" + conversation = messages_service.for_user(db, _user(db)) + stamp = datetime(2026, 1, 1, tzinfo=UTC) + twins = [ + Message(chat_id=conversation.id, role="user", content=f"same {i}", created_at=stamp) + for i in range(2) + ] + db.add_all(twins) + db.commit() + # A later day, so the cursor is unambiguously after both twins -- otherwise + # the cursor shares their stamp and the test is about id ordering, which is + # random, rather than about the tie-breaker. + later = _fill(db, conversation, 2, day=2) + + page = messages_service.older_than(db, conversation, later[0]) + assert {m.content for m in page} == {"same 0", "same 1"} + + +def test_the_sentinel_names_its_own_target(client: TestClient, db, registered): + """It sits on a page whose composer form carries `hx-target="#thread"`, and + htmx resolves that by walking up the DOM. The jobs chip demonstrated once + what an unstated target does to a transcript.""" + conversation = messages_service.for_user(db, _user(db)) + _fill(db, conversation, 120) + body = client.get("/messages").text + + sentinel = next(chunk for chunk in body.split(" User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _file(db, **kwargs) -> Report: + from lembas.services import reports as reports_service + + fields = {"title": "A report", "body": "What I found.", **kwargs} + return reports_service.create(db, owner=_user(db), **fields) + + +# --- The store ---------------------------------------------------------------- +def test_a_report_is_filed_and_read_back(client: TestClient, db, registered): + from lembas.services import reports as reports_service + + report = _file(db, title="Build failures", body="# Heading\n\nThree tests fail.") + assert reports_service.get(db, report.id, _user(db)) is report + + +def test_a_missing_summary_falls_back_to_the_first_real_line(client: TestClient, db, registered): + """A list of forty reports all reading '# ' is a bug somebody has to explain.""" + report = _file(db, body="# Weekly build report\n\nThree tests fail on ARM.", summary="") + assert report.summary == "Weekly build report" + + +def test_a_report_belongs_to_its_owner_alone(client: TestClient, db, registered): + """There is no sharing here, which is a decision rather than an omission -- + so a second account must not reach the first one's reports.""" + from lembas.security.passwords import hash_password + from lembas.services import reports as reports_service + + report = _file(db) + stranger = User( + email="sam@shire.test", name="Sam", password_hash=hash_password("gardening-is-hard") + ) + db.add(stranger) + db.commit() + + assert reports_service.get(db, report.id, stranger) is None + assert reports_service.recent(db, stranger) == [] + + +def test_an_over_long_body_is_trimmed_rather_than_refused(client: TestClient, db, registered): + """The rule `memories` already follows: file it short with everything else + intact, instead of failing the turn that wrote it.""" + from lembas.services import reports as reports_service + + report = _file(db, body="x" * (reports_service.MAX_BODY_CHARS + 500)) + assert len(report.body) == reports_service.MAX_BODY_CHARS + + +# --- The tool ----------------------------------------------------------------- +@pytest.mark.anyio +async def test_report_write_actually_runs(client: TestClient, db, registered): + """Not that it is declared -- that it *runs*. + + `_run_scratch_write` read a field its own dataclass did not have and raised + AttributeError for the whole life of the feature, swallowed by `run_tool`'s + blanket except into a message that reads exactly like a model calling it + wrongly. The test that existed asserted the family and the risk, which are + properties of the declaration. + """ + from lembas.services import tools as tools_service + + context = tools_service.ToolContext(owner_id=_user(db).id, chat_id="", model_id="test-model") + outcome = await tools_service.run_tool( + context, "report_write", '{"title": "Findings", "body": "Three tests fail."}' + ) + + assert outcome.event["status"] == "ok", outcome.content + filed = db.scalars(select(Report)).all() + assert [r.title for r in filed] == ["Findings"] + assert filed[0].model_id == "test-model" + + +@pytest.mark.anyio +async def test_report_write_refuses_an_empty_body(client: TestClient, db, registered): + from lembas.services import tools as tools_service + + context = tools_service.ToolContext(owner_id=_user(db).id, chat_id="") + outcome = await tools_service.run_tool(context, "report_write", '{"title": "Nothing"}') + + assert outcome.event["status"] == "error" + assert db.scalars(select(Report)).all() == [] + + +@pytest.mark.anyio +async def test_report_search_finds_by_body_word(client: TestClient, db, registered): + """FTS tables are outside the model-driven schema sync, so this is also the + check that `reports_fts` was actually created and its triggers fire.""" + from lembas.services import tools as tools_service + + _file(db, title="Unrelated", body="Nothing about the thing.") + _file(db, title="The one", body="A palantir was involved.") + + context = tools_service.ToolContext(owner_id=_user(db).id, chat_id="") + outcome = await tools_service.run_tool(context, "report_search", '{"query": "palantir"}') + + assert [r["title"] for r in outcome.event["results"]] == ["The one"] + + +def test_the_index_is_backfilled_on_an_upgrade(client: TestClient, db, registered): + """The upgrade path, which the fresh-database case does not exercise. + + FTS tables are outside the model-driven schema sync -- they are not + SQLAlchemy models, so `sync_schema` cannot diff them. On an instance that + already has reports, `ensure_fts` has to create the index *and* backfill + what is already in the table; without the backfill every report filed before + the upgrade is invisible to search forever, and nothing says so. + """ + from sqlalchemy import text + + from lembas.db.migrations import ensure_fts + from lembas.db.session import get_engine + from lembas.services import reports as reports_service + + _file(db, title="Before the upgrade", body="A palantir was involved.") + + # Drop the index and its triggers, leaving the rows: an instance upgrading + # into this release looks exactly like this. + with get_engine().begin() as connection: + for suffix in ("_ai", "_ad", "_au"): + connection.execute(text(f"DROP TRIGGER IF EXISTS reports_fts{suffix}")) + connection.execute(text("DROP TABLE IF EXISTS reports_fts")) + + assert "reports_fts" in ensure_fts(get_engine()) + assert [r.title for r in reports_service.search(db, _user(db), "palantir")] == [ + "Before the upgrade" + ] + + +# --- The section's character -------------------------------------------------- +def test_the_reports_pages_carry_no_composer(client: TestClient, db, registered): + """Reports are read, never answered, and the way to be sure of that is for + the machinery that would answer to be absent. + + `chat/_message.html` is the state machine: an `sse-connect` anywhere on + these pages is a generation this section has no business starting. + """ + report = _file(db) + + for url in ("/reports", f"/reports/{report.id}"): + body = client.get(url).text + assert "sse-connect" not in body, url + assert "composer__form" not in body, url + # Nothing on the page can send anything anywhere. Checked as "no field + # named content" rather than by looking for a URL, because the sidebar + # legitimately links to sections that do have composers. + assert 'name="content"' not in body, url + + +def test_no_route_accepts_a_message_for_a_report(client: TestClient, registered): + """Asserted on the resolved routes rather than on the templates, because the + failure this guards against is somebody adding the endpoint first.""" + from lembas.main import app + + # Read from the OpenAPI schema rather than by walking `app.routes`: this + # FastAPI keeps an included router wrapped rather than flattening it, so the + # walk finds nothing at all and the assertion passes for the wrong reason. + writable = { + f"{method.upper()} {path}" + for path, methods in app.openapi()["paths"].items() + if path.startswith(("/reports", "/api/reports")) + for method in methods + if method.upper() in {"POST", "PATCH", "PUT"} + } + assert writable == {"POST /api/reports/{report_id}/delete"} + + +def test_a_report_body_cannot_smuggle_html(client: TestClient, db, registered): + """Model output, hard rule 6. Rendered through services/markdown.py, which + is the one path allowed to emit HTML here.""" + report = _file( + db, + body="\n\n[click](javascript:alert(1))", + ) + body = client.get(f"/reports/{report.id}").text + + assert "" not in body + # The link is left as inert text rather than becoming an anchor, which is + # the property that matters: what must never appear is the href. + assert 'href="javascript:' not in body + + +def test_opening_a_report_clears_its_dot(client: TestClient, db, registered): + report = _file(db) + assert report.unread is True + + client.get(f"/reports/{report.id}") + db.refresh(report) + assert report.unread is False + + +def test_the_unread_poll_carries_the_section_dot(client: TestClient, db, registered): + """Sent on every tick including empty, because it has to be able to clear. + A dot that survived reading the last report would be news nobody can dismiss. + """ + report = _file(db) + + showing = client.get("/api/chats/unread").text + dot = next(s for s in showing.split(" User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _endpoint() -> Endpoint: + return Endpoint(base_url="http://127.0.0.1:1/v1", api_key="", extra_headers={}) + + +def _template(db) -> str: + return prompts_service.resolve(db, "task.schedule_compile") + + +async def _compile(db, reply: str, monkeypatch, request: str = "every monday at 3"): + async def answer(endpoint, payload): + return reply + + monkeypatch.setattr(compile_service, "complete", answer) + return await compile_service.compile_request( + _endpoint(), "test-model", request, template=_template(db), user=_user(db) + ) + + +# --- What it can read ------------------------------------------------------------- +@pytest.mark.anyio +async def test_a_plain_json_answer_compiles(client: TestClient, db, registered, monkeypatch): + compiled = await _compile( + db, + json.dumps( + { + "title": "Build check", + "instruction": "Check the build and say what broke.", + "target": "report", + "schedule": {"at": {"weekdays": [0], "times": ["15:00"]}}, + } + ), + monkeypatch, + ) + + assert compiled.ok is True + assert compiled.title == "Build check" + assert compiled.target == "report" + assert compiled.rule["at"]["times"] == ["15:00"] + + +@pytest.mark.anyio +async def test_a_fenced_answer_compiles(client: TestClient, db, registered, monkeypatch): + """Small models fence their JSON however they were trained to. Refusing it + costs a whole round trip to end up showing the manual form anyway — the + same reasoning `tools.parse_arguments` already follows.""" + body = json.dumps( + {"title": "T", "instruction": "I", "schedule": {"at": {"times": ["09:00"]}}} + ) + compiled = await _compile( + db, f"Here you go:\n```json\n{body}\n```\nHope that helps!", monkeypatch + ) + + assert compiled.ok is True + assert compiled.rule["at"]["times"] == ["09:00"] + + +@pytest.mark.anyio +async def test_a_timer_with_no_start_begins_now( + client: TestClient, db, registered, monkeypatch +): + """"Every six hours" is written `{"every": {"hours": 6}}` and nothing else, + which is the natural reading and cannot fire on its own — a timer measures + from a start, and `rule.py` has no clock to invent one. Filled in here + exactly as the manual form does, or the commonest request of all compiles to + a schedule that never runs.""" + compiled = await _compile( + db, + json.dumps({"title": "T", "instruction": "I", "schedule": {"every": {"hours": 6}}}), + monkeypatch, + request="every six hours", + ) + + assert compiled.ok is True + assert compiled.rule["every"] == {"minutes": 360} + assert compiled.rule["start"] + + +@pytest.mark.anyio +async def test_thinking_is_stripped_before_parsing( + client: TestClient, db, registered, monkeypatch +): + """A model that thinks inline puts its reasoning in `content`, which is the + field `complete` hands back verbatim — the trap auto-titling hit.""" + body = json.dumps({"title": "T", "instruction": "I", "schedule": {"at": {"times": ["09:00"]}}}) + compiled = await _compile(db, f"Let me work this out…{body}", monkeypatch) + + assert compiled.ok is True + assert compiled.title == "T" + + +# --- How it fails ------------------------------------------------------------------ +@pytest.mark.anyio +async def test_prose_falls_back_to_the_readers_own_words( + client: TestClient, db, registered, monkeypatch +): + """Never a schedule nobody asked for. The reader's words survive so the form + is filled in rather than blank.""" + compiled = await _compile(db, "Sure! I'd suggest running that weekly.", monkeypatch) + + assert compiled.ok is False + assert compiled.instruction == "every monday at 3" + assert compiled.reason + + +@pytest.mark.anyio +async def test_a_rule_that_normalises_to_nothing_is_not_ok( + client: TestClient, db, registered, monkeypatch +): + """The compile's output is model output that becomes a *timer*, and this is + the reason `rule.validate` had to be total.""" + compiled = await _compile( + db, json.dumps({"title": "T", "instruction": "I", "schedule": "0 3 * * 1"}), monkeypatch + ) + + assert compiled.ok is False + assert compiled.rule == {} + assert "when" in compiled.reason + + +@pytest.mark.anyio +async def test_a_time_already_past_is_not_ok(client: TestClient, db, registered, monkeypatch): + compiled = await _compile( + db, + json.dumps( + {"title": "T", "instruction": "I", "schedule": {"start": "2020-01-01T09:00:00Z"}} + ), + monkeypatch, + ) + + assert compiled.ok is False + assert "already passed" in compiled.reason + + +@pytest.mark.anyio +async def test_an_endpoint_that_is_down_is_not_an_error( + client: TestClient, db, registered, monkeypatch +): + async def refuse(endpoint, payload): + raise LLMError("connection refused") + + monkeypatch.setattr(compile_service, "complete", refuse) + compiled = await compile_service.compile_request( + _endpoint(), "test-model", "daily at nine", template=_template(db), user=_user(db) + ) + + assert compiled.ok is False + assert compiled.instruction == "daily at nine" + + +@pytest.mark.anyio +async def test_clearing_the_fragment_switches_off_the_compiling_not_the_feature( + client: TestClient, db, registered, monkeypatch +): + """`task.compact` set the precedent that clearing a fragment kills a + feature. Here it must not: the manual form is what makes "an empty override + means off" safe, and no request is made at all.""" + called = False + + async def answer(endpoint, payload): + nonlocal called + called = True + return "{}" + + monkeypatch.setattr(compile_service, "complete", answer) + compiled = await compile_service.compile_request( + _endpoint(), "test-model", "daily at nine", template="", user=_user(db) + ) + + assert called is False + assert compiled.ok is False + assert compiled.instruction == "daily at nine" + assert compiled.reason == "" + + +def test_the_prompt_carries_the_readers_zone(client: TestClient, db, registered): + """The model works out "Monday at 3" and the ticker fires it. If they + disagree about the zone, nothing errors — it simply runs at the wrong time.""" + user = _user(db) + user.settings_json = {**(user.settings_json or {}), "timezone": "Asia/Tokyo"} + db.commit() + + prompt = compile_service.render_prompt( + _template(db), request="every monday at 3", user=user + ) + assert "Asia/Tokyo" in prompt + assert "every monday at 3" in prompt + + +# --- The review step ---------------------------------------------------------------- +def test_describing_shows_it_back_rather_than_creating_it( + client: TestClient, db, registered, monkeypatch +): + """A timing a model chose and nobody looked at is exactly the standing + instruction this codebase refuses to create silently elsewhere.""" + + async def answer(endpoint, payload): + return json.dumps( + { + "title": "Build check", + "instruction": "Check the build.", + "schedule": {"at": {"weekdays": [0], "times": ["15:00"]}}, + } + ) + + monkeypatch.setattr(compile_service, "complete", answer) + + response = client.post("/api/schedules/describe", data={"request": "mondays at 3"}) + + assert response.status_code == 200 + assert "Every Monday at 15:00" in response.text + assert "Build check" in response.text + # Shown, not saved. + assert db.scalars(select(Schedule)).all() == [] + + +def test_describing_with_no_model_configured_still_answers( + client: TestClient, db, registered +): + from lembas.db.models import Connection + + for connection in db.scalars(select(Connection)): + db.delete(connection) + db.commit() + + response = client.post("/api/schedules/describe", data={"request": "mondays at 3"}) + + assert response.status_code == 200 + assert "fill it in yourself" in response.text + + +# --- Unattended --------------------------------------------------------------------- +def test_ask_user_is_not_offered_in_a_task_chat(client: TestClient, db, registered): + """Enforced in `resolve_tools`, not merely discouraged in the prompt. + + A parked `ask_user` holds the reply for the whole `approval_timeout` with + nobody there to answer — a run that silently does nothing for fifteen + minutes and then gives up. A rule living only in a system message is one a + page the model just read can argue with. + """ + from lembas.db.models import Model + from lembas.services import tools as tools_service + + model = db.scalars(select(Model)).one() + model.capabilities_json = {"tools": True, "tool_ask": True} + db.commit() + settings_store.update(db, {"default_permissions": {"tools.ask": True}}) + + ordinary = Chat(user_id=_user(db).id, model_id="test-model") + task = Chat(user_id=_user(db).id, model_id="test-model", kind=KIND_TASK) + db.add_all([ordinary, task]) + db.commit() + + offered = {t.name for t in tools_service.resolve_tools(db, ordinary, _user(db)).defs} + assert "ask_user" in offered + + withdrawn = {t.name for t in tools_service.resolve_tools(db, task, _user(db)).defs} + assert "ask_user" not in withdrawn + + +def test_a_task_chat_is_told_what_it_is_for(client: TestClient, db, registered, monkeypatch): + """A task chat accumulates every run, so by the tenth the instruction is far + out of sight up the transcript.""" + from lembas.services import harness + from lembas.services import schedules as schedules_service + + schedule = schedules_service.create( + db, + owner=_user(db), + title="Build check", + instruction="Check the build and say what broke.", + rule={"at": {"weekdays": [0], "times": ["15:00"]}}, + ) + chat = db.get(Chat, schedule.chat_id) + + values = harness.context_variables(db, _user(db), [], chat) + assert values["schedule_instruction"] == "Check the build and say what broke." + assert values["schedule_summary"] == "Every Monday at 15:00" + + block = harness.compose(db, _user(db), [], chat) + assert "nobody is necessarily reading it" in block + assert "Check the build and say what broke." in block + + +def test_an_ordinary_chat_is_told_none_of_it(client: TestClient, db, registered): + """`core.unattended` and `context.schedule` are gated on the same variable, + so the warning cannot appear without the thing it warns about.""" + from lembas.services import harness + + chat = Chat(user_id=_user(db).id, model_id="test-model") + db.add(chat) + db.commit() + + block = harness.compose(db, _user(db), [], chat) + assert "nobody is necessarily reading" not in block + assert "This scheduled task" not in block diff --git a/tests/test_schedule_rule.py b/tests/test_schedule_rule.py new file mode 100644 index 0000000..94d74d1 --- /dev/null +++ b/tests/test_schedule_rule.py @@ -0,0 +1,373 @@ +"""The recurrence rule, on its own. + +`rule.py` is pure and total: no session, no wall clock, nothing that raises. So +it is tested exhaustively here, before anything calls it — which is the whole +reason it was built first. Everything downstream of it fails quietly. A schedule +that never fires looks exactly like a working one on every screen it appears on, +and one that fires an hour out looks like nothing at all until the report is +late. + +The DST cases are the ones worth reading. They are not hypotheticals: each of +them happens twice a year, on a machine nobody is watching. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from zoneinfo import ZoneInfo + +from lembas.services.schedule import rule as rule_service + +# A zone with an interesting spring and autumn, and one without. +PRAGUE = ZoneInfo("Europe/Prague") +UTC_ZONE = UTC + + +def at(text: str) -> datetime: + """A UTC instant from an ISO string, for readable expectations.""" + return datetime.fromisoformat(text).replace(tzinfo=UTC) + + +def local(text: str, zone=PRAGUE) -> datetime: + """A wall-clock stamp in a zone, as an instant.""" + return datetime.fromisoformat(text).replace(tzinfo=zone).astimezone(UTC) + + +# --- validate: total, clamping, never raising --------------------------------- +def test_validate_never_raises_on_anything(): + """The compile step hands this whatever a model wrote. Model output that + becomes a *timer* is the sharpest case of hard rule 6 in the codebase.""" + for junk in ( + None, 0, "", "every monday", [], {}, {"every": "often"}, + {"at": "3pm"}, {"count": "lots"}, {"start": "not a date"}, + {"every": {"minutes": -5}}, {"at": {"weekdays": [9, "x", None]}}, + {"until": "2020-01-01T00:00:00Z", "start": "2026-01-01T00:00:00Z"}, + {"at": {"times": ["25:99", "nope", ""]}}, + ): + assert isinstance(rule_service.validate(junk), dict) + + +def test_prose_and_cron_normalise_to_nothing(): + """`{}` is the honest answer, and the caller's cue to show the manual form + rather than write a schedule that can never fire.""" + assert rule_service.validate("0 3 * * 1") == {} + assert rule_service.validate({"cron": "0 3 * * 1"}) == {} + assert rule_service.validate({"every": {"seconds": 5}}) == {} + + +def test_anything_validate_accepts_can_actually_fire(): + """The flagship invariant. A rule that normalises to something non-empty but + has no next occurrence is a schedule indistinguishable from a working one on + the list page — which is this feature's worst silent failure.""" + now = at("2026-08-05T12:00:00") + candidates = [ + {"start": "2026-08-05T14:30:00Z"}, + {"start": "2026-08-05T14:30:00Z", "every": {"minutes": 10}}, + {"at": {"times": ["15:00"]}}, + {"at": {"weekdays": [0], "times": ["15:00"]}}, + {"at": {"days": [1, 15], "times": ["09:00"]}}, + {"at": {"months": [1, 7], "days": [1], "times": ["00:00"]}}, + {"at": {"weekdays": [0], "times": ["15:00"]}, "count": 5}, + {"start": "2026-08-05T00:00:00Z", "at": {"weekdays": [0], "times": ["15:00"]}, + "every": {"weeks": 2}}, + ] + for raw in candidates: + clean = rule_service.validate(raw) + assert clean, raw + assert rule_service.next_after(clean, now, zone=PRAGUE) is not None, clean + + +def test_an_interval_below_a_minute_is_raised_not_honoured(): + """The ticker's granularity is coarser than a second, so honouring it is + impossible and pretending to would run silently late for ever.""" + clean = rule_service.validate({"start": "2026-08-05T00:00:00Z", "every": {"minutes": 0}}) + assert "every" not in clean + + clean = rule_service.validate( + {"start": "2026-08-05T00:00:00Z", "every": {"minutes": 1}} + ) + assert clean["every"] == {"minutes": 1} + + +def test_a_calendar_with_no_time_gets_midnight(): + """Otherwise "every Monday" means nothing at all, and the whole `at` block + would have to be discarded.""" + clean = rule_service.validate({"at": {"weekdays": [0]}}) + assert clean["at"]["times"] == ["00:00"] + + +def test_a_window_that_closes_before_it_opens_is_refused(): + assert rule_service.validate( + {"start": "2026-08-05T00:00:00Z", "until": "2026-08-01T00:00:00Z", + "every": {"hours": 1}} + ) == {} + + +# --- The four combinations ------------------------------------------------------ +def test_a_one_shot_fires_once_and_then_never(): + clean = rule_service.validate({"start": "2026-08-05T14:30:00Z"}) + assert rule_service.next_after(clean, at("2026-08-05T12:00:00"), zone=PRAGUE) == at( + "2026-08-05T14:30:00" + ) + # Once it has run, `fired` is what stops it being offered again — its moment + # is in the past, so nothing else would. + assert rule_service.next_after( + clean, at("2026-08-05T12:00:00"), zone=PRAGUE, fired=1 + ) is None + assert rule_service.next_after(clean, at("2026-08-05T15:00:00"), zone=PRAGUE) is None + + +def test_a_timer_steps_from_its_start(): + clean = rule_service.validate( + {"start": "2026-08-05T12:00:00Z", "every": {"minutes": 10}} + ) + assert rule_service.next_after(clean, at("2026-08-05T11:00:00"), zone=PRAGUE) == at( + "2026-08-05T12:00:00" + ) + assert rule_service.next_after(clean, at("2026-08-05T12:00:00"), zone=PRAGUE) == at( + "2026-08-05T12:10:00" + ) + assert rule_service.next_after(clean, at("2026-08-05T12:05:00"), zone=PRAGUE) == at( + "2026-08-05T12:10:00" + ) + + +def test_a_timer_idle_for_a_year_costs_one_division(): + """Computed rather than stepped. A loop here would spin a hundred thousand + times inside the ticker for a schedule nobody touched.""" + clean = rule_service.validate( + {"start": "2020-01-01T00:00:00Z", "every": {"minutes": 1}} + ) + assert rule_service.next_after(clean, at("2026-08-05T12:00:30"), zone=PRAGUE) == at( + "2026-08-05T12:01:00" + ) + + +def test_every_monday_at_three_means_local_three(): + clean = rule_service.validate({"at": {"weekdays": [0], "times": ["15:00"]}}) + # 5 August 2026 is a Wednesday; the next Monday is the 10th. + found = rule_service.next_after(clean, at("2026-08-05T12:00:00"), zone=PRAGUE) + assert found == local("2026-08-10T15:00:00") + assert found.astimezone(PRAGUE).strftime("%A %H:%M") == "Monday 15:00" + + +def test_several_times_a_day_are_all_taken_in_order(): + clean = rule_service.validate({"at": {"times": ["09:00", "17:00"]}}) + first = rule_service.next_after(clean, local("2026-08-05T08:00:00"), zone=PRAGUE) + assert first == local("2026-08-05T09:00:00") + second = rule_service.next_after(clean, first, zone=PRAGUE) + assert second == local("2026-08-05T17:00:00") + third = rule_service.next_after(clean, second, zone=PRAGUE) + assert third == local("2026-08-06T09:00:00") + + +def test_a_stride_over_a_calendar_keeps_every_nth(): + """"Every other Monday" — and anchored on `start`, so it names the same two + Mondays whenever it is asked rather than depending on when you looked.""" + clean = rule_service.validate( + { + "start": "2026-08-10T00:00:00Z", + "at": {"weekdays": [0], "times": ["15:00"]}, + "every": {"weeks": 2}, + } + ) + first = rule_service.next_after(clean, at("2026-08-05T00:00:00"), zone=PRAGUE) + assert first == local("2026-08-10T15:00:00") + second = rule_service.next_after(clean, first, zone=PRAGUE) + assert second == local("2026-08-24T15:00:00") # not the 17th + + later = rule_service.next_after(clean, local("2026-08-20T00:00:00"), zone=PRAGUE) + assert later == local("2026-08-24T15:00:00") + + +def test_a_day_of_month_that_does_not_exist_every_month_still_finds_one(): + """31 is a real answer in January and no answer in February. The search must + walk on rather than concluding the rule is dead.""" + clean = rule_service.validate({"at": {"days": [31], "times": ["09:00"]}}) + found = rule_service.next_after(clean, local("2026-02-01T00:00:00"), zone=PRAGUE) + assert found == local("2026-03-31T09:00:00") + + +def test_an_impossible_calendar_answers_never_rather_than_spinning(): + """31 February matches nothing. Bounded by the search horizon, so the ticker + cannot be hung by one bad row.""" + clean = rule_service.validate( + {"at": {"months": [2], "days": [31], "times": ["09:00"]}} + ) + assert rule_service.next_after(clean, at("2026-08-05T00:00:00"), zone=PRAGUE) is None + + +# --- Daylight saving ------------------------------------------------------------ +def test_a_wall_clock_time_survives_spring_forward(): + """29 March 2026, Prague: 02:00 becomes 03:00 and 02:30 does not exist. + + A daily 02:30 report vanishing once a year, on a schedule nobody is + watching, is exactly the failure this whole module is arranged around. It + fires at the first instant that does exist instead. + """ + clean = rule_service.validate({"at": {"times": ["02:30"]}}) + found = rule_service.next_after(clean, local("2026-03-29T00:00:00"), zone=PRAGUE) + + assert found is not None + assert found.astimezone(PRAGUE).date().isoformat() == "2026-03-29" + # Exactly the first minute that exists, not "somewhere after". Left to + # zoneinfo's own resolution this reads 02:30+01:00 — an hour later as an + # instant, and a wall-clock time that did not happen. + assert found.astimezone(PRAGUE).strftime("%H:%M") == "03:00" + + +def test_a_wall_clock_time_fires_once_across_fall_back(): + """25 October 2026, Prague: 02:00–03:00 happens twice. Once, not twice.""" + clean = rule_service.validate({"at": {"times": ["02:30"]}}) + first = rule_service.next_after(clean, local("2026-10-24T12:00:00"), zone=PRAGUE) + assert first.astimezone(PRAGUE).date().isoformat() == "2026-10-25" + + # The next occurrence is the following day, not the repeat of the same hour. + second = rule_service.next_after(clean, first, zone=PRAGUE) + assert second.astimezone(PRAGUE).date().isoformat() == "2026-10-26" + # 25 real hours between them, because the clock went back in between. Taking + # the second 02:30 instead would make this one hour. + assert second - first == timedelta(hours=25) + + +def test_a_daily_calendar_holds_its_wall_clock_across_a_boundary(): + """The half that people mean by "every day at 9": 09:00 stays 09:00, and the + real interval between two firings is 23 or 25 hours.""" + clean = rule_service.validate({"at": {"times": ["09:00"]}}) + before = rule_service.next_after(clean, local("2026-03-28T00:00:00"), zone=PRAGUE) + after = rule_service.next_after(clean, before, zone=PRAGUE) + + assert before.astimezone(PRAGUE).hour == 9 + assert after.astimezone(PRAGUE).hour == 9 + assert after - before == timedelta(hours=23) + + +def test_a_timer_holds_its_interval_across_a_boundary(): + """The other half, and the opposite behaviour on purpose: six hours is six + hours, so a 23-hour day must not shift or double it.""" + clean = rule_service.validate( + {"start": "2026-03-28T00:00:00Z", "every": {"hours": 6}} + ) + cursor = local("2026-03-28T12:00:00") + steps = [] + for _ in range(6): + cursor = rule_service.next_after(clean, cursor, zone=PRAGUE) + steps.append(cursor) + + assert all(b - a == timedelta(hours=6) for a, b in zip(steps, steps[1:], strict=False)) + + +# --- Exhaustion ----------------------------------------------------------------- +def test_a_count_is_spent_and_then_it_is_over(): + """"Five times" meaning "for ever" is the failure. Exhaustion returns None, + which is the caller's cue to disable rather than to loop.""" + clean = rule_service.validate( + {"start": "2026-08-05T12:00:00Z", "every": {"minutes": 10}, "count": 3} + ) + now = at("2026-08-05T11:00:00") + assert rule_service.next_after(clean, now, zone=PRAGUE, fired=2) is not None + assert rule_service.next_after(clean, now, zone=PRAGUE, fired=3) is None + assert rule_service.next_after(clean, now, zone=PRAGUE, fired=99) is None + + +def test_an_until_closes_the_window(): + clean = rule_service.validate( + { + "start": "2026-08-05T12:00:00Z", + "every": {"days": 1}, + "until": "2026-08-08T00:00:00Z", + } + ) + assert rule_service.next_after(clean, at("2026-08-06T00:00:00"), zone=PRAGUE) == at( + "2026-08-06T12:00:00" + ) + assert rule_service.next_after(clean, at("2026-08-08T00:00:00"), zone=PRAGUE) is None + + +# --- Catching up ---------------------------------------------------------------- +def test_a_missed_run_collapses_to_one(): + """A host off for a week comes back owing one report, not a hundred and + sixty-eight. This is the whole reason `advance` is not just `next_after`.""" + clean = rule_service.validate( + {"start": "2026-08-01T00:00:00Z", "every": {"hours": 1}} + ) + due, following = rule_service.advance( + clean, + after=at("2026-08-01T00:00:00"), + now=at("2026-08-08T00:30:00"), + zone=PRAGUE, + ) + assert due is True + # The next one is measured from now, not from the slot that was missed. + assert following == at("2026-08-08T01:00:00") + + +def test_a_schedule_not_yet_due_is_left_alone(): + clean = rule_service.validate( + {"start": "2026-08-05T12:00:00Z", "every": {"hours": 1}} + ) + due, following = rule_service.advance( + clean, + after=at("2026-08-05T12:00:00"), + now=at("2026-08-05T12:30:00"), + zone=PRAGUE, + ) + assert due is False + assert following == at("2026-08-05T13:00:00") + + +def test_catching_up_the_last_of_a_count_leaves_nothing_behind(): + """The firing being caught up counts, so `count` is spent by what actually + ran rather than by what was scheduled.""" + clean = rule_service.validate( + {"start": "2026-08-01T00:00:00Z", "every": {"hours": 1}, "count": 3} + ) + due, following = rule_service.advance( + clean, after=at("2026-08-01T02:00:00"), now=at("2026-08-08T00:00:00"), + zone=PRAGUE, fired=2, + ) + assert due is True + assert following is None + + +def test_a_missed_one_shot_still_fires(): + """A reminder is not less wanted for being late.""" + clean = rule_service.validate({"start": "2026-08-01T09:00:00Z"}) + due, following = rule_service.advance( + clean, after=at("2026-07-31T00:00:00"), now=at("2026-08-08T00:00:00"), zone=PRAGUE + ) + assert due is True + assert following is None + + +# --- Saying it back -------------------------------------------------------------- +def test_describe_says_what_the_rule_actually_does(): + """A row reading "Every Monday at 3PM" over a rule that fires daily is the + same class of failure as three places disagreeing about a tool's name — and + this is the reader's only view of something that happens while they are not + looking.""" + cases = [ + ({"at": {"weekdays": [0], "times": ["15:00"]}}, "Every Monday at 15:00"), + ({"at": {"times": ["09:00"]}}, "Every day at 09:00"), + ({"at": {"times": ["09:00", "17:00"]}}, "Every day at 09:00 and 17:00"), + ({"start": "2026-08-05T12:00:00Z", "every": {"minutes": 10}}, "Every 10 minutes"), + ({"start": "2026-08-05T12:00:00Z", "every": {"hours": 6}}, "Every 6 hours"), + ( + {"start": "2026-08-05T12:00:00Z", "every": {"minutes": 10}, "count": 5}, + "Every 10 minutes, 5 times", + ), + ({"at": {"days": [1], "times": ["09:00"]}}, "Every the 1st at 09:00"), + ] + for raw, expected in cases: + assert rule_service.describe(rule_service.validate(raw), zone=PRAGUE) == expected + + +def test_describe_survives_an_empty_rule(): + assert rule_service.describe({}, zone=PRAGUE) == "Never" + assert rule_service.describe(rule_service.validate("nonsense"), zone=PRAGUE) == "Never" + + +def test_describe_names_the_one_shot_moment_in_the_readers_zone(): + clean = rule_service.validate({"start": "2026-08-05T13:00:00Z"}) + assert "15:00" in rule_service.describe(clean, zone=PRAGUE) + assert "13:00" in rule_service.describe(clean, zone=UTC_ZONE) diff --git a/tests/test_schedule_ticker.py b/tests/test_schedule_ticker.py new file mode 100644 index 0000000..c63fab5 --- /dev/null +++ b/tests/test_schedule_ticker.py @@ -0,0 +1,450 @@ +"""The ticker and the runner: claiming a firing, and not doing it twice. + +`rule.py` is tested on its own in `test_schedule_rule.py`. What is tested here +is everything that goes wrong *around* a correct rule — which is the half that +fails silently. Nothing in this file needs an endpoint: the firing itself is +stubbed, because what is being checked is the bookkeeping, not the reply. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import select + +from lembas.db.models import ( + ROLE_ASSISTANT, + TARGET_CHAT, + TARGET_REPORT, + Chat, + Message, + Report, + Schedule, + User, +) +from lembas.services import settings_store +from lembas.services.schedule import clock, runner, ticker +from lembas.services.schedule import rule as rule_service + + +@pytest.fixture(autouse=True) +def scheduling_on(db): + settings_store.update(db, {"enabled": True}, key=settings_store.SCHEDULES) + return None + + +def _user(db) -> User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _schedule(db, *, chat_id: str = "", **kwargs) -> Schedule: + fields = { + "user_id": _user(db).id, + "title": "A schedule", + "instruction": "Do the thing.", + "rule_json": rule_service.validate( + {"start": "2026-01-01T00:00:00Z", "every": {"hours": 1}} + ), + "target": TARGET_CHAT, + "chat_id": chat_id, + "enabled": True, + "next_fire_at": datetime(2026, 1, 1, tzinfo=UTC), + **kwargs, + } + schedule = Schedule(**fields) + db.add(schedule) + db.commit() + return schedule + + +# --- Claiming ------------------------------------------------------------------- +@pytest.mark.anyio +async def test_a_firing_that_raises_still_moves_the_schedule_on( + client: TestClient, db, registered, make_chat, monkeypatch +): + """The single most important property here. + + Claim, commit, *then* fire. The other order is a hot loop: a schedule whose + firing fails is retried every tick for ever, against whatever it was that + failed — and the only symptom is load. + """ + schedule = _schedule(db, chat_id=make_chat()) + + async def explode(schedule_id, **kwargs): + raise RuntimeError("the endpoint is down") + + monkeypatch.setattr(runner, "fire", explode) + + fired = await ticker.sweep(now=datetime(2026, 1, 1, 0, 30, tzinfo=UTC)) + assert fired == 1 + for task in list(ticker._FIRING): + with pytest.raises(RuntimeError): + await task + + db.refresh(schedule) + assert schedule.next_fire_at is not None + assert schedule.fired_count == 1 + + +@pytest.mark.anyio +async def test_two_overlapping_sweeps_fire_once( + client: TestClient, db, registered, make_chat, monkeypatch +): + """A sweep can take minutes — a firing awaits a model. The lock is what + stops the next tick claiming the same row again.""" + schedule = _schedule(db, chat_id=make_chat()) + calls: list[str] = [] + + async def record(schedule_id, **kwargs): + calls.append(schedule_id) + + monkeypatch.setattr(runner, "fire", record) + + now = datetime(2026, 1, 1, 0, 30, tzinfo=UTC) + await ticker.sweep(now=now) + await ticker.sweep(now=now) + for task in list(ticker._FIRING): + await task + + assert calls == [schedule.id] + + +@pytest.mark.anyio +async def test_exhaustion_disables_rather_than_looping( + client: TestClient, db, registered, make_chat, monkeypatch +): + """"Five times" meaning "for ever" is the failure. A rule with nothing left + switches the row off, so it stops being examined at all.""" + monkeypatch.setattr(runner, "fire", lambda *a, **k: _noop()) + schedule = _schedule( + db, + chat_id=make_chat(), + rule_json=rule_service.validate( + {"start": "2026-01-01T00:00:00Z", "every": {"hours": 1}, "count": 1} + ), + ) + + await ticker.sweep(now=datetime(2026, 1, 1, 0, 30, tzinfo=UTC)) + for task in list(ticker._FIRING): + await task + + db.refresh(schedule) + assert schedule.fired_count == 1 + assert schedule.enabled is False + assert schedule.next_fire_at is None + + +@pytest.mark.anyio +async def test_a_missed_week_fires_once( + client: TestClient, db, registered, make_chat, monkeypatch +): + """The host was off. It comes back owing one run, not a hundred and + sixty-eight — and the next one is measured from now.""" + calls: list[str] = [] + + async def record(schedule_id, **kwargs): + calls.append(schedule_id) + + monkeypatch.setattr(runner, "fire", record) + schedule = _schedule(db, chat_id=make_chat()) + + now = datetime(2026, 1, 8, 0, 30, tzinfo=UTC) + await ticker.sweep(now=now) + for task in list(ticker._FIRING): + await task + + assert len(calls) == 1 + db.refresh(schedule) + assert schedule.fired_count == 1 + assert schedule.next_fire_at.replace(tzinfo=UTC) > now + + +@pytest.mark.anyio +async def test_one_unreadable_row_does_not_stop_the_sweep( + client: TestClient, db, registered, make_chat, monkeypatch +): + """A ticker that dies on one bad row stops every schedule on the instance, + and nothing anywhere says so.""" + calls: list[str] = [] + + async def record(schedule_id, **kwargs): + calls.append(schedule_id) + + monkeypatch.setattr(runner, "fire", record) + + broken = _schedule(db, chat_id=make_chat(), title="Broken") + healthy = _schedule(db, chat_id=make_chat(), title="Healthy") + + real_advance = rule_service.advance + + def selective(rule, **kwargs): + if rule.get("_broken"): + raise ValueError("unreadable") + return real_advance(rule, **kwargs) + + broken.rule_json = {**broken.rule_json, "_broken": True} + db.commit() + monkeypatch.setattr(rule_service, "advance", selective) + + await ticker.sweep(now=datetime(2026, 1, 1, 0, 30, tzinfo=UTC)) + for task in list(ticker._FIRING): + await task + + assert calls == [healthy.id] + db.refresh(broken) + assert broken.enabled is False + assert broken.last_error + + +def test_a_deleted_account_takes_its_schedules_with_it( + client: TestClient, db, registered, make_chat +): + """`user_id` is a real ForeignKey with CASCADE — unlike `chat_id`, which is + a plain id because `migrations.py` cannot add a REFERENCES clause to a table + that already exists. So an orphaned schedule cannot be reached at all, and + the owner check in the sweep is a guard rather than a path. + + Worth pinning as the *reason* that guard looks unreachable: somebody + removing it would be right about today and wrong the moment the column + convention changes. + """ + _schedule(db, chat_id=make_chat()) + owner = _user(db) + + db.delete(owner) + db.commit() + # The cascade is enforced by SQLite, not by an ORM relationship, so the + # session's identity map still holds the row it was told about. Ask the + # database rather than the cache. + db.expunge_all() + + assert db.scalars(select(Schedule)).all() == [] + + +@pytest.mark.anyio +async def test_nothing_fires_while_scheduling_is_switched_off( + client: TestClient, db, registered, make_chat, monkeypatch +): + """The instance switch is a switch, not a suggestion. Checked in the sweep + rather than at the routes, so a row created while it was on does not go on + firing after it is turned off.""" + calls: list[str] = [] + monkeypatch.setattr(runner, "fire", lambda sid, **k: calls.append(sid) or _noop()) + _schedule(db, chat_id=make_chat()) + settings_store.update(db, {"enabled": False}, key=settings_store.SCHEDULES) + + assert await ticker.sweep(now=datetime(2026, 1, 1, 0, 30, tzinfo=UTC)) == 0 + assert calls == [] + + +# --- The runner ----------------------------------------------------------------- +async def _noop() -> None: + return None + + +@pytest.mark.anyio +async def test_a_firing_writes_a_machine_turn_and_starts_a_reply( + client: TestClient, db, registered, make_chat +): + """The role stays `user` — `build_messages` needs one there and `_inject` + sends a queued turn verbatim. `machine` is what stops the transcript + claiming the reader typed it.""" + chat_id = make_chat() + schedule = _schedule(db, chat_id=chat_id) + + await runner.fire(schedule.id) + + turns = list( + db.scalars(select(Message).where(Message.chat_id == chat_id).order_by(Message.created_at)) + ) + assert turns[0].role == "user" + assert turns[0].machine is True + assert "started by a schedule" in turns[0].content + assert "Do the thing." in turns[0].content + # And a reply was opened for it. + assert turns[-1].role == ROLE_ASSISTANT + assert turns[-1].complete is False + + +@pytest.mark.anyio +async def test_a_deleted_chat_switches_the_schedule_off( + client: TestClient, db, registered, make_chat +): + """Rather than firing into nothing on every tick from now on — which is a + schedule that looks alive and produces nothing.""" + schedule = _schedule(db, chat_id="nosuchchat") + + await runner.fire(schedule.id) + + db.refresh(schedule) + assert schedule.enabled is False + assert "no longer exists" in schedule.last_error + + +@pytest.mark.anyio +async def test_a_backlog_skips_rather_than_queues_for_ever( + client: TestClient, db, registered, make_chat +): + """`_drain` takes one queued turn per reply, so a schedule firing faster + than its chat can answer would build a backlog that outlives the day that + caused it.""" + chat_id = make_chat() + chat = db.get(Chat, chat_id) + for index in range(5): + db.add( + Message(chat_id=chat.id, role="user", content=f"waiting {index}", queued=True) + ) + db.commit() + schedule = _schedule(db, chat_id=chat_id) + + await runner.fire(schedule.id) + + db.refresh(schedule) + assert "previous run was still going" in schedule.last_error + assert schedule.claimed_at is None + + +@pytest.mark.anyio +async def test_run_now_does_not_consume_the_scheduled_run( + client: TestClient, db, registered, make_chat +): + """Testing a schedule must not skip the run it was testing. Advancing is the + ticker's job and nothing else's.""" + schedule = _schedule(db, chat_id=make_chat()) + before = clock.as_utc(schedule.next_fire_at) + fired_before = schedule.fired_count + + await runner.run_now(schedule.id) + + db.refresh(schedule) + # Through `as_utc` on both sides: a row read back from SQLite is naive while + # one still in the session keeps its tzinfo, and comparing the two raises. + assert clock.as_utc(schedule.next_fire_at) == before + assert schedule.fired_count == fired_before + + +def _finished(db, chat_id: str, text: str) -> Message: + """A reply that has already been written. + + `deliver` is tested against this rather than against the output of + `runner.fire`, because `fire` starts a real generation — which, with no + endpoint configured, races the test to write the same row. The delivery's + job begins at a finished message, so that is what it is handed. + """ + message = Message( + chat_id=chat_id, role=ROLE_ASSISTANT, content=text, complete=True, model_id="m" + ) + db.add(message) + db.commit() + return message + + +@pytest.mark.anyio +async def test_a_report_target_files_the_reply( + client: TestClient, db, registered, make_chat +): + chat_id = make_chat() + schedule = _schedule(db, chat_id=chat_id, target=TARGET_REPORT, title="Daily news") + assistant = _finished(db, chat_id, "Three things happened.") + + await runner.deliver( + schedule.id, assistant.id, since=datetime(2020, 1, 1, tzinfo=UTC) + ) + + filed = db.scalars(select(Report)).all() + assert [r.title for r in filed] == ["Daily news"] + assert filed[0].body == "Three things happened." + assert filed[0].schedule_id == schedule.id + assert filed[0].unread is True + + +@pytest.mark.anyio +async def test_a_run_that_produced_nothing_still_files_something( + client: TestClient, db, registered, make_chat +): + """A scheduled report that silently did not appear is indistinguishable + from a schedule that never fired. So the failure is filed as a report.""" + chat_id = make_chat() + schedule = _schedule(db, chat_id=chat_id, target=TARGET_REPORT, title="Daily news") + broken = Message(chat_id=chat_id, role=ROLE_ASSISTANT, content="", complete=True, + error="the endpoint refused") + db.add(broken) + db.commit() + + await runner.deliver(schedule.id, broken.id, since=datetime(2020, 1, 1, tzinfo=UTC)) + + filed = db.scalars(select(Report)).all() + assert len(filed) == 1 + assert filed[0].error + + +@pytest.mark.anyio +async def test_a_report_the_model_filed_itself_is_not_duplicated( + client: TestClient, db, registered, make_chat +): + """`report_write` during the run *is* the report. Filing the reply beside it + would put two of everything in the feed.""" + from lembas.services import reports as reports_service + + chat_id = make_chat() + schedule = _schedule(db, chat_id=chat_id, target=TARGET_REPORT) + began = datetime.now(tz=UTC) - timedelta(seconds=5) + reports_service.create( + db, owner=_user(db), title="Filed by the model", body="...", + source="chat", source_id=chat_id, + ) + assistant = _finished(db, chat_id, "Also this.") + + await runner.deliver(schedule.id, assistant.id, since=began) + + assert [r.title for r in db.scalars(select(Report))] == ["Filed by the model"] + + +@pytest.mark.anyio +async def test_a_report_from_an_earlier_run_does_not_suppress_this_one( + client: TestClient, db, registered, make_chat +): + """The other half of the dedup, and the one that would fail silently: a + daily report would be filed once and then never again, because last week's + is still sitting there with the same `source_id`.""" + from lembas.services import reports as reports_service + + chat_id = make_chat() + schedule = _schedule(db, chat_id=chat_id, target=TARGET_REPORT, title="Daily news") + reports_service.create( + db, owner=_user(db), title="Yesterday's", body="...", + source="schedule", source_id=chat_id, + ) + assistant = _finished(db, chat_id, "Today's news.") + + # This run began *after* yesterday's report was filed. + await runner.deliver(schedule.id, assistant.id, since=datetime.now(tz=UTC)) + + assert sorted(r.title for r in db.scalars(select(Report))) == [ + "Daily news", + "Yesterday's", + ] + + +# --- Lifecycle ------------------------------------------------------------------- +def test_starting_twice_makes_one_ticker(client: TestClient, registered): + """Two tickers in one process fires everything twice, which is the failure + two workers would cause and the reason this is idempotent.""" + ticker.start() + first = ticker._TICKER + ticker.start() + assert ticker._TICKER is first + + +def test_release_claims_clears_an_interrupted_run(client: TestClient, db, registered): + """A restart abandons a firing in flight. Without this the row keeps its + claim stamp for ever and reads as permanently running.""" + schedule = _schedule(db, claimed_at=datetime(2026, 1, 1, tzinfo=UTC)) + + assert ticker.release_claims() == 1 + + db.refresh(schedule) + assert schedule.claimed_at is None + assert "interrupted by a restart" in schedule.last_error diff --git a/tests/test_schedules_ui.py b/tests/test_schedules_ui.py new file mode 100644 index 0000000..b9732bc --- /dev/null +++ b/tests/test_schedules_ui.py @@ -0,0 +1,373 @@ +"""The Scheduled section: making one, changing it, and the strip on its chat. + +The failures worth pinning here are the ones that look like working software: +a control wired to a method its route does not serve, a form that quietly +creates something which can never fire, and a task chat that still has a way to +send a message into it. +""" + +from __future__ import annotations + +from datetime import UTC, datetime + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import select + +from lembas.db.models import KIND_TASK, TARGET_REPORT, Chat, Schedule, User +from lembas.security import permissions +from lembas.services import schedules as schedules_service +from lembas.services import settings_store +from lembas.services.schedule import clock + + +@pytest.fixture(autouse=True) +def scheduling_allowed(db, registered): + """`schedule.use` is off by default, deliberately. Granted here so the tests + are about the feature rather than about the gate — which has its own test. + + A model is configured too, and that is not scaffolding: with none, the chat + page renders its "no models yet" branch instead of the conversation, and + every assertion about what the composer area does or does not contain passes + for the wrong reason. This file caught exactly that. + """ + from lembas.db.models import Connection, Model + from lembas.services.crypto import encrypt + + settings_store.update(db, {"enabled": True}, key=settings_store.SCHEDULES) + settings_store.update( + db, {"default_permissions": {"schedule.use": True, "reports.use": True}} + ) + connection = Connection( + name="Test", base_url="http://127.0.0.1:1", api_key_encrypted=encrypt("") + ) + db.add(connection) + db.commit() + db.add(Model(connection_id=connection.id, model_id="test-model")) + db.commit() + return None + + +def _user(db) -> User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _make(client: TestClient, **overrides) -> None: + data = { + "title": "Monday build check", + "instruction": "Check the build and say what broke.", + "target": "chat", + "repeat": "calendar", + "weekdays": "0", + "times": "15:00", + "count": "0", + **overrides, + } + return client.post("/api/schedules", data=data, follow_redirects=False) + + +# --- Creating ------------------------------------------------------------------- +def test_creating_a_schedule_makes_its_chat_too(client: TestClient, db, registered): + """The one place "chats are created lazily" is bent, and on purpose: the + first firing may be days away with nobody present to make one.""" + response = _make(client) + assert response.status_code == 303 + + schedule = db.scalars(select(Schedule)).one() + chat = db.get(Chat, schedule.chat_id) + assert chat is not None + assert chat.kind == KIND_TASK + assert response.headers["location"] == f"/chat/{chat.id}" + + +def test_a_new_schedule_has_a_next_run(client: TestClient, db, registered): + """A schedule that can never fire looks exactly like a working one on the + list page. This is the invariant that stops one being written at all.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + + assert schedule.next_fire_at is not None + assert clock.as_utc(schedule.next_fire_at) > datetime.now(tz=UTC) + assert schedule.enabled is True + + +def test_a_rule_that_means_nothing_is_refused_with_a_reason( + client: TestClient, db, registered +): + """Not a 400 nobody can act on: back to the form, with the reason. A silent + refusal here would be a "Schedule it" button that appears to do nothing.""" + response = _make(client, repeat="once", start_date="", start_time="") + + assert response.status_code == 303 + assert "/scheduled/new?error=" in response.headers["location"] + assert db.scalars(select(Schedule)).all() == [] + + +def test_a_one_shot_in_the_past_is_refused(client: TestClient, db, registered): + response = _make(client, repeat="once", start_date="2020-01-01", start_time="09:00") + + assert "error=" in response.headers["location"] + assert db.scalars(select(Schedule)).all() == [] + + +def test_the_form_and_the_engine_agree_about_what_was_stored( + client: TestClient, db, registered +): + """The edit screen is derived from the *normalised* rule, so a form showing + something other than what runs is impossible rather than merely unlikely.""" + _make(client, repeat="every", every_amount="6", every_unit="hours") + schedule = db.scalars(select(Schedule)).one() + + page = client.get(f"/scheduled/{schedule.id}/edit").text + assert 'value="every"\n checked' in page or 'value="every" checked' in page + assert 'value="6"' in page + + +def test_a_per_user_ceiling_is_enforced(client: TestClient, db, registered): + settings_store.update(db, {"max_per_user": 1}, key=settings_store.SCHEDULES) + _make(client) + + response = _make(client, title="A second one") + + assert "error=" in response.headers["location"] + assert len(db.scalars(select(Schedule)).all()) == 1 + + +# --- The gate -------------------------------------------------------------------- +def test_scheduling_is_off_unless_granted(client: TestClient, db, registered): + """`schedule.use` defaults to False: this spends model time with nobody at + the keyboard, which is a capability chosen on purpose.""" + assert permissions.DEFAULT_PERMISSIONS["schedule.use"] is False + + +def _as_stranger(client: TestClient, db, *, role: str = "user") -> User: + """Sign in as somebody who is not the administrator. + + Necessary for any test about a permission: `permissions.resolve` gives an + admin everything, so asking Frodo whether a gate works answers a different + question and answers it yes. + """ + from lembas.security.passwords import hash_password + + stranger = User( + email="sam@shire.test", + name="Sam", + password_hash=hash_password("gardening-is-hard"), + role=role, + ) + db.add(stranger) + db.commit() + client.post("/auth/logout") + client.post( + "/auth/login", data={"email": "sam@shire.test", "password": "gardening-is-hard"} + ) + return stranger + + +def test_the_sidebar_entry_follows_the_permission(client: TestClient, db, registered): + _as_stranger(client, db) + assert 'href="/scheduled"' in client.get("/chat").text + + settings_store.update(db, {"default_permissions": {"schedule.use": False}}) + assert 'href="/scheduled"' not in client.get("/chat").text + + +def test_a_non_admin_without_the_permission_cannot_reach_it( + client: TestClient, db, registered +): + settings_store.update(db, {"default_permissions": {"schedule.use": False}}) + _as_stranger(client, db) + + assert client.get("/scheduled", follow_redirects=False).status_code in (302, 303, 403) + assert client.post("/api/schedules", data={}, follow_redirects=False).status_code in ( + 302, + 303, + 403, + ) + + +# --- The task chat ---------------------------------------------------------------- +def test_a_task_chat_has_no_composer(client: TestClient, db, registered): + """Suppressed by absence, not by hiding: `chat/_composer.html` is the only + thing that posts a message, so its absence is the guarantee. A hidden one + would still be a form anybody could post to.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + + body = client.get(f"/chat/{schedule.chat_id}").text + assert "composer__form" not in body + assert 'name="content"' not in body + # And the controls that do apply are there instead. + assert f"/api/schedules/{schedule.id}/run" in body + assert f"/api/schedules/{schedule.id}/toggle" in body + + +def test_the_strip_survives_its_schedule_being_removed( + client: TestClient, db, registered +): + """Removing a schedule keeps its chat by default. The chat becomes an + ordinary one, so it is reachable — a KIND_TASK chat with no schedule behind + it would be in no list at all.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + chat_id = schedule.chat_id + + client.post(f"/api/schedules/{schedule.id}/delete", data={"keep_chat": "1"}) + + chat = db.get(Chat, chat_id) + db.refresh(chat) + assert chat is not None + assert chat.kind == "chat" + assert client.get(f"/chat/{chat_id}").status_code == 200 + + +def test_removing_a_schedule_can_take_its_chat(client: TestClient, db, registered): + _make(client) + schedule = db.scalars(select(Schedule)).one() + chat_id = schedule.chat_id + + client.post(f"/api/schedules/{schedule.id}/delete", data={"keep_chat": "0"}) + + db.expunge_all() + assert db.get(Chat, chat_id) is None + + +# --- Controls that write ----------------------------------------------------------- +def test_pausing_and_resuming_move_the_row(client: TestClient, db, registered): + """Asserted on the row rather than on the response: a control wired to a + method its route does not serve returns 405 and looks exactly like working + software, which cost the agent-mode select an entire release.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + + client.post(f"/api/schedules/{schedule.id}/toggle", data={"enabled": "0"}) + db.refresh(schedule) + assert schedule.enabled is False + + client.post(f"/api/schedules/{schedule.id}/toggle", data={"enabled": "1"}) + db.refresh(schedule) + assert schedule.enabled is True + + +def test_resuming_recomputes_from_now(client: TestClient, db, registered): + """A schedule paused for a month must not come back owing a month of runs. + Without this it fires the instant it is switched on.""" + _make(client, repeat="every", every_amount="1", every_unit="hours") + schedule = db.scalars(select(Schedule)).one() + + schedule.enabled = False + schedule.next_fire_at = datetime(2020, 1, 1, tzinfo=UTC) + db.commit() + + schedules_service.set_enabled(db, schedule, owner=_user(db), enabled=True) + + assert clock.as_utc(schedule.next_fire_at) > datetime.now(tz=UTC) + + +def test_editing_the_rule_restarts_the_count(client: TestClient, db, registered): + """An edited schedule is a new intention. Carrying the old `fired_count` + into a new `count` would spend most of it before the first run.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + schedule.fired_count = 7 + db.commit() + + client.post( + f"/api/schedules/{schedule.id}", + data={ + "title": "Changed", + "instruction": "Something else.", + "target": "report", + "repeat": "every", + "every_amount": "2", + "every_unit": "hours", + "count": "3", + }, + ) + + db.refresh(schedule) + assert schedule.fired_count == 0 + assert schedule.title == "Changed" + assert schedule.target == TARGET_REPORT + + +def test_the_routes_refuse_the_wrong_verb(client: TestClient, db, registered): + """The other half of the agent-mode lesson: assert the wrong method is + *refused*, because only that half would have failed throughout.""" + _make(client) + schedule = db.scalars(select(Schedule)).one() + + assert client.get(f"/api/schedules/{schedule.id}/toggle").status_code == 405 + assert client.get(f"/api/schedules/{schedule.id}/run").status_code == 405 + assert client.patch(f"/api/schedules/{schedule.id}/delete").status_code == 405 + + +def test_one_persons_schedule_is_not_anothers(client: TestClient, db, registered): + from lembas.security.passwords import hash_password + + _make(client) + schedule = db.scalars(select(Schedule)).one() + + stranger = User( + email="sam@shire.test", + name="Sam", + password_hash=hash_password("gardening-is-hard"), + role="admin", + ) + db.add(stranger) + db.commit() + + assert schedules_service.get(db, schedule.id, stranger) is None + + +# --- The admin page ---------------------------------------------------------------- +def test_the_admin_page_saves_and_clamps(client: TestClient, db, registered): + """Clamped on read as well as here, for the reason `agents` and `images` + give: a value stored by an earlier release, or edited into the database by + hand, has to be survivable too. What this asserts is that neither half has + been quietly dropped.""" + response = client.post( + "/admin/schedules", + data={ + "enabled": "true", + "tick_seconds": "1", + "max_per_user": "9999", + "max_concurrent": "0", + "min_interval_seconds": "1", + "max_queued": "500", + }, + follow_redirects=False, + ) + assert response.status_code == 303 + + values = settings_store.schedules(db) + assert values["enabled"] is True + assert values["tick_seconds"] == 5 # floor: a busy loop otherwise + assert values["max_per_user"] == 200 # ceiling + # 0 falls back to the default rather than clamping to 1, because the + # accessor reads `int(stored or default)` -- the same shape `agents` and + # `images` use. There is no reading of "no runs at once" that anybody wants: + # it would be a ticker that claims work and never does it. + assert values["max_concurrent"] == 3 + assert values["min_interval_seconds"] == 60 + assert values["max_queued"] == 50 + + +def test_the_switch_actually_stops_firing(client: TestClient, db, registered): + """A switch that only greys something out is the failure. The sweep reads + it, so a schedule created while it was on stops when it is turned off.""" + import anyio + + from lembas.services.schedule import ticker + + _make(client) + client.post("/admin/schedules", data={"enabled": ""}, follow_redirects=False) + + assert anyio.run(ticker.sweep) == 0 + + +def test_the_admin_page_is_admin_only(client: TestClient, db, registered): + _as_stranger(client, db) + assert client.get("/admin/schedules", follow_redirects=False).status_code in ( + 302, 303, 403, 404, + ) diff --git a/tests/test_sidebar_sections.py b/tests/test_sidebar_sections.py new file mode 100644 index 0000000..015f0e2 --- /dev/null +++ b/tests/test_sidebar_sections.py @@ -0,0 +1,116 @@ +"""What may appear in the sidebar's chat tree, and what may not. + +The sidebar narrows on `Chat.kind`, and passes `""` whenever the Chat/Agent +switch is absent -- which is every instance with agent chats turned off. For as +long as there were exactly two kinds, "" meaning "no filter" and "" meaning +"both sides of the switch" were the same thing. They stopped being the same +thing the moment a third kind existed, and the difference is invisible until +somebody has a conversation that belongs to a section instead of to the tree. + +That is the shape this file exists for: correct code whose meaning changed +underneath it. It is pinned in both places that do the narrowing, because they +are two implementations of one rule and only one of them is SQL. +""" + +from __future__ import annotations + +from fastapi.testclient import TestClient +from sqlalchemy import select + +from lembas.db.models import ALL_KINDS, KIND_MESSAGES, KIND_TASK, KINDS, Chat, Folder, User + + +def _user(db) -> User: + return db.scalars(select(User).order_by(User.created_at)).first() + + +def _chat(db, *, kind: str, folder: Folder | None = None) -> Chat: + chat = Chat(user_id=_user(db).id, kind=kind, folder_id=folder.id if folder else None) + db.add(chat) + db.commit() + return chat + + +# --- The vocabulary ----------------------------------------------------------- +def test_kinds_stays_the_two_sided_switch(): + """`api/preferences.py:set_sidebar_kind` validates against KINDS, so a third + entry makes the tree filterable to a side with no button to leave it -- the + "one side of a fork nobody can move" failure `sidebar_split` already guards. + New kinds go in ALL_KINDS. + """ + assert KINDS == ("chat", "agent") + assert set(ALL_KINDS) > set(KINDS) + + +def test_the_sidebar_switch_refuses_a_kind_that_is_not_a_side( + client: TestClient, db, registered +): + client.post("/api/preferences/sidebar-kind", data={"kind": KIND_TASK}) + stored = (_user(db).settings_json or {}).get("sidebar_kind") + assert stored != KIND_TASK + + +# --- The two narrowings ------------------------------------------------------- +def test_unfiled_sections_chats_stay_out_of_the_tree(client: TestClient, db, registered): + """With no switch on screen the sidebar asks for "" -- and "" must not mean + "everything". This is the SQL half, in `sidebar_context`.""" + from lembas.api.pages import sidebar_context + + ordinary = _chat(db, kind="chat") + task = _chat(db, kind=KIND_TASK) + conversation = _chat(db, kind=KIND_MESSAGES) + + listed = {c.id for c in sidebar_context(db, _user(db))["unfiled_chats"]} + assert ordinary.id in listed + assert task.id not in listed + assert conversation.id not in listed + + +def test_foldered_sections_chats_stay_out_of_the_tree(client: TestClient, db, registered): + """And this is the Python half, in `Folder.visible_chats`. Two + implementations of one rule, so both are pinned: fixing only the query would + leave a task chat filed in a folder showing up anyway. + """ + folder = Folder(user_id=_user(db).id, name="Work") + db.add(folder) + db.commit() + + ordinary = _chat(db, kind="chat", folder=folder) + _chat(db, kind=KIND_TASK, folder=folder) + + listed = {c.id for c in folder.visible_chats()} + assert listed == {ordinary.id} + # And with the switch present it is still only the ordinary one. + assert {c.id for c in folder.visible_chats("chat")} == {ordinary.id} + + +def test_a_folder_holding_only_a_task_chat_reads_as_empty(client: TestClient, db, registered): + """`shown_in` keeps a folder that is empty of everything, because hiding a + container somebody just made means it can never be filed into. A folder + holding only a task chat has to count as that empty one -- otherwise it + shows on both sides claiming contents nobody can see. + """ + folder = Folder(user_id=_user(db).id, name="Scheduled work") + db.add(folder) + db.commit() + _chat(db, kind=KIND_TASK, folder=folder) + + assert folder.holds() is False + assert folder.shown_in("chat") is True + + +def test_the_composer_cannot_manufacture_a_section_chat(client: TestClient, db, registered): + """`_new_chat` collapses kind to agent-or-chat, so this is already true by + construction. Pinned so it stays true: the ordinary composer is a form + anybody can post to.""" + from lembas.db.models import Connection, Model + + connection = Connection(name="local", base_url="http://x.test/v1", enabled=True) + db.add(connection) + db.commit() + db.add(Model(connection_id=connection.id, model_id="m", display_name="M", enabled=True)) + db.commit() + + client.post("/api/chats/start", data={"content": "hello", "kind": KIND_TASK}) + kinds = {c.kind for c in db.scalars(select(Chat))} + assert KIND_TASK not in kinds