LLeMbas/PLAN.md

# LLeMbas — plan and status

Where the project is, what is deliberately not built yet, and the decisions
that would be expensive to revisit. Kept current as work lands; the detail of
*how* things work lives in [`CLAUDE.md`](CLAUDE.md).

**Status:** usable daily, and closing on 1.0.0. Streaming chat, attachments,
reasoning, tool calling with web search, custom HTTP tools and MCP servers,
agent chats that work on a machine over SSH, a knowledge library, notes, memory
and skills, speech in and out, image generation over ComfyUI, users and groups,
model administration, installable as an app, reports, messages, and scheduled
work that runs on its own. 2088 tests, `ruff` clean.

What remains before the first stable release is written out below, in phases,
under [The road to 1.0.0](#the-road-to-100).

---

## The shape of it

A self-hosted web UI for OpenAI-compatible endpoints, written in Python, themed
after Middle-earth.

| | |
|---|---|
| Stack | FastAPI + Jinja + htmx + a little Alpine |
| Build step | none — no Node, no npm, no CDN at runtime |
| Database | SQLite, schema synchronised additively at startup |
| Deployment | systemd unit + nginx vhost, one worker |

These are load-bearing. Dropping the no-build rule or moving off SQLite would
be a different project, not a refactor.

---

## Done

### Chat
- [x] Streaming replies over server-sent events
- [x] **Markdown renders progressively** — re-rendered whole every 100ms rather
      than appending tokens, because a list or code fence is only correct once
      its context exists
- [x] Syntax highlighting (Pygments), sanitised with nh3
- [x] **Generation runs in the background** — a task, not the request. Navigate
      away, open another chat, close the tab: the reply keeps being written and
      reattaching replays the whole state
- [x] **Stop** — the send button becomes Stop while writing; what arrived is kept
- [x] **Rewind** — edit one of your own turns and the conversation runs on from
      there. Truncates rather than branching
- [x] **Chat titles that fit the chat** — an ordinary chat is named by a model
      from the first exchange, an agent chat from its opening words alone, which
      are already an objective. Renameable from the heading and from the sidebar
      row; one response updates both
- [x] Chats created on first message, so an abandoned composer leaves nothing
- [x] **You are told when something arrives** — a dot and a toast for a reply,
      a report or a scheduled run; a count in the tab title while you are
      looking elsewhere; and a browser notification, opt-in per device, that
      reaches you with LLeMbas closed
- [x] **A reply that started without you asking still arrives** — the open chat
      page polls for turns it has not got, so a background job waking the model
      appears where you are looking instead of only after a reload. Quiet while
      a reply is streaming, since that reply delivers its own bubbles
- [x] **A turn nobody typed says so** — a background job's completion is a user
      turn on the wire, because the request needs one, and a machine event in
      the transcript: its own icon and name, no pencil, and no claim that you
      sent it
- [x] **Folders that carry something** — arbitrarily nested, with a name, a
      description, a system prompt inherited by the chats inside them, and seeds
      for the model, the kind and the agent target. Deleting one keeps the chats
- [x] **The sidebar splits Chat and Agent** — a switch below the pinned models,
      stored on the account, filtering the folder tree as well as the loose
      chats
- [x] **A reply reads as the sequence it was** — thinking, prose, a tool call,
      more prose, in the order they happened, rather than three stacked zones
      with every tool block in the middle. Marks on the row index the three
      stores; a reply written before them renders exactly as it always did
- [x] **Blocks open while the reply is still being written** — the ids are
      stable across every swap and across the final one, and opening a block
      stops the thread chasing the bottom until you scroll back down
- [x] Per-reply metrics — tokens, context used as a percentage, tokens/second,
      live while streaming and kept afterwards. Estimated with a `~` when the
      endpoint reports no usage. Two chips: what the reply **cost** and what the
      conversation now **occupies**, each labelled, both moving between one
      usage block and the next rather than once a round
- [x] Compaction — a button, and automatically at a configurable percentage of
      the model's context. Summarised turns are kept and collapsed, not deleted
- [x] Temporary chats — never listed, swept after a day, with a Keep button
- [x] An admin-only request inspector beside the thread
- [x] **Canvas** — a third side panel holding open files, in tabs. Project files
      over SFTP in an agent chat; notes, skills, knowledge documents, this
      chat's text attachments and its own scratch document everywhere. Read with
      syntax highlighting, edited in a plain textarea, saved with a conflict
      check. Files the model touches open themselves, without taking the screen

### Tools
- [x] **Tool calling** — one reply is a bounded loop of requests, not one
      request. Text produced before a call is kept
- [x] **Web search** as the first tool: DuckDuckGo (no setup), SearXNG or
      Firecrawl, chosen in the admin area
- [x] Only offered to models flagged `tools`, because an endpoint without
      support rejects the whole request rather than ignoring the array
- [x] Sources stay in the transcript; results are **not** replayed as context on
      the next turn, for the same reasons reasoning is not
- [x] A round's calls run together, and the reply says which tool is running —
      a remote tool taking seconds with nothing streaming looks like a hang
- [x] **A reply can stop and ask you something** — one or more questions on one
      card, with answers to pick from and a box to write your own, answered
      together. The same mechanism carries command approvals
- [x] **Custom HTTP tools** — an administrator describes one call: a JSON Schema,
      a URL template, headers, an encrypted secret and how to read the answer.
      Arguments may fill a hole but never move the target: the scheme and host
      are literal, values are escaped for where they land, and the origin is
      pinned afterwards
- [x] **MCP servers** over streamable HTTP — a hand-written client, so that
      `check_url` runs on every hop rather than being bypassed by somebody
      else's transport. Tools are discovered and cached by a button, namespaced
      per server, and a server's own descriptions are bounded before they reach
      a model as instructions
- [x] Both gated like the built-ins — a model capability, a permission — and
      restrictable to groups, with guidance of their own on `/admin/prompts`
- [x] Local MCP over stdio is deliberately absent: spawning a subprocess would
      run on this machine, which nothing here does

### Image generation
- [x] **Draws on a ComfyUI you are running**, as a tool the model chooses to
      call and as an `/image` command that makes it call one. Never on this
      machine, the same rule agent chats follow
- [x] **Multiple workflow templates** — a name, a description and a ComfyUI API
      export with `{{prompt}}` and ten other placeholders where the values go.
      The model picks between them by their descriptions, and by checkpoint,
      falling back to the chat's usual and then the instance default when it
      names neither
- [x] Model may set prompt, negative, seed, steps, cfg, width, height, sampler,
      scheduler, denoise, checkpoint and template; **only the prompt is
      required** and everything else has a default
- [x] **The result is checked before you see it** — optionally, a vision model
      is shown the picture and the request and says keep or retry, up to a
      configurable number of attempts. Only clearly wrong images are retried;
      the last attempt is kept whatever it says, so a request always produces
      something
- [x] **Preserve VRAM** — opt-in, for a machine that cannot hold both at once:
      unload the chat's own language model, generate, free ComfyUI, and let the
      next request load the model back. Per connection, so a box on the network
      is never touched
- [x] Instance-wide extra instructions, injected into the harness beside the
      tool's own guidance
- [x] **Failures say what actually happened** — out of memory, cancelled, or a
      node that raised, read out of ComfyUI's own record within a second rather
      than waiting out the timeout. A memory failure tells the model to retry at
      a named smaller size or a lighter checkpoint; a cancelled one tells it not
      to start again
- [x] Every parameter described by what it does to the picture and when to move
      it, because a model given "cfg: default 8" sends the prompt alone.
      `docs/image-generation-instructions.md` is a longer set to paste into the
      admin instructions box

### Agent chats
- [x] A chat is a **Chat** or an **Agent**, chosen when it starts and fixed
      thereafter — a transcript whose earlier turns ran somewhere else is not
      one conversation. Knowledge, memories and skills are shared across both
- [x] **Nothing runs on the LLeMbas host.** Commands go to a machine reached
      over SSH, so containment is somebody's considered choice of host — a
      container built for the job — rather than a sandbox built here. A local
      one was designed in detail and dropped; see CLAUDE.md for why
- [x] **SSH connections are user-owned**, like notes. An administrator decides
      only whether the feature exists at all
- [x] Trust on first use, made explicit: adding a host does not connect to it,
      **Check** shows its fingerprint with nothing sent, and only accepting
      pins it. A host that later answers with a different key is refused
- [x] Four modes as a table over what each tool does to the world —
      **Manual** asks about everything, **Edit** writes freely but asks before
      commands, **Auto** asks about nothing, **Plan** reads freely and changes
      nothing. Switchable at any time; read once per reply
- [x] Enforced in the generation loop, not in the prompt: a rule a model is
      merely told is one a poisoned file can argue with
- [x] A deny list beats **Auto** for any command it can match; an allow list
      cannot be matched at all by a command containing anything that joins two
      commands together. A deny pattern cannot either — so in Auto a compound
      line runs, which is the trade for Auto not asking about `cd build && make`.
      See CLAUDE.md; matching each segment would restore both and is not built
- [x] **The terminal and the canvas open before the chat exists** — on the
      new-chat screen, against the connection and directory being chosen there,
      and both re-point when that changes. The shell you opened and the files
      you left open are adopted into the chat when you send the first prompt
- [x] **Background jobs are visible** — a chip in the composer row counting what
      is still running, and a panel with each job's command, state, log tail,
      how long it took and a Stop button. The dot is coloured by outcome rather
      than by status, since `done` covers exit 0 and exit 2 alike. Survives a
      restart, because the job does
- [x] `shell_run`, `file_read`, `file_write`, `file_list` — files over SFTP,
      never through a shell, because the SSH exec protocol has no argv form
- [x] **Plan mode ends with a plan** you can carry out with one button, which
      switches to Edit and sends it back quoted rather than as an instruction
- [x] Per-reply budgets on steps, wall clock and output, with time spent
      waiting for you subtracted
- [x] **A terminal panel** beside the chat, holding a real shell on that chat's
      own connection. The modes govern the model; what a person types is theirs,
      since they hold the credential and could open the same shell with an ssh
      client. The model cannot see the panel — sending it output is a button
- [x] The shell outlives the panel and the page: closing it leaves a build
      running, and coming back reattaches with the scrollback. An idle timeout
      is what eventually ends one, and so does deleting the chat, or disabling,
      moving or deleting the connection
- [x] **The panel is resizable**, dragged from its edge or nudged with the
      arrow keys, and the width follows you to another browser
- [x] **It knows where one command ends and the next begins** — bash and zsh
      are given the markers VS Code and WezTerm use, so *Copy* and *Send* mean
      one command and its output rather than the last forty rows of the screen.
      An **Auto** toggle collects each one into the next message. Any other
      shell starts exactly as it did before, the buttons fall back to the
      screen and say so, and Auto is disabled rather than degraded
- [x] **The project directory is listed for the model** — one read-only
      command, `git ls-files` where that works so `.gitignore` is honoured for
      free, budgeted so a big directory becomes a count rather than a thousand
      filenames on every request
- [x] **A directory is chosen by browsing it** over SFTP, not by typing a path
      into an unlabelled box
- [x] The approval mode is chosen **before** the first message, beside the
      message box rather than in the header

### The library
- [x] **Knowledge bases** — documents, images and saved web pages, grouped into
      named collections and ingested through the same pipeline as chat
      attachments, searched with SQLite FTS5
- [x] A chat can be pointed at particular bases, so "answer from the contracts
      folder" is a different question from "answer from everything I have"
- [x] **Notes** — longer things the model writes down and searches later;
      editable by hand, because they are yours
- [x] **Memory** — short facts, injected on every turn to a budget rather than
      searched, and managed in your settings
- [x] **Skills** — saved procedures. Only the name and description are injected;
      the body is fetched when the model decides it applies
- [x] A model may write and revise its own notes, memories and skills. Every
      skill revision is kept, attributed and revertible — the safety story is a
      record and a way back, not a gate
- [x] **Sharing** — a knowledge base, a note or a skill can be shared with a
      group or with named people, read-only. One visibility rule, and
      administrators do not bypass it. Documents are shared through their base
- [x] **The harness** — an operational prompt assembled from what a model
      actually has, so the tools get used rather than ignored
- [x] Attach menu: file, image, a web page fetched on the spot, or a document
      from the library
- [x] **`@` to name one** — the library everywhere, and files in the project
      directory in an agent chat. The reference stays in the sentence and the
      contents come along, with the path and the machine, so the model knows
      exactly which file it was handed

### Scheduling
- [x] **Schedules** — work that runs because time passed rather than because
      somebody asked just now. Fire once or repeat; a fixed number of runs or
      until stopped; a timer ("every ten minutes") or a calendar ("every Monday
      at 3PM"), and the two compose into "every other Monday"
- [x] **Wall-clock and elapsed time are kept apart**, because they mean
      different things: a calendar time stays 15:00 across a daylight-saving
      change, while a six-hourly timer stays six hours. A time that does not
      exist on a spring-forward day fires at the first minute that does
- [x] **Per-user timezone**, so "every Monday" means the reader's Monday. The
      harness tells them their own time now, not the server's
- [x] **Scheduled** — one chat per task, replied into each time it comes round.
      No composer: run it now, pause it, edit it, remove it
- [x] A missed run **catches up once** and then resumes. A week of downtime owes
      one report, not a hundred and sixty-eight
- [x] Claim before firing, so a run that fails moves the schedule on rather than
      retrying every tick for ever; and "Run now" deliberately does *not* consume
      the run it was testing
- [x] **Say it in your own words** — a model turns "every Monday morning, check
      the build" into a recurrence and an instruction that reads on its own,
      and shows it back for approval before anything is saved. Anything it
      cannot work out lands in the same form, filled in as far as it got
- [x] A scheduled run knows nobody is watching: `ask_user` is **withdrawn**, not
      merely discouraged, because a question with no one to answer it holds the
      reply until it times out

### Messages
- [x] **Messages** — one conversation per person that is meant to run for
      years. It opens on the most recent turns and pages older ones in as you
      scroll up
- [x] **Bounded in the request, unbounded on disk.** Only the latest chunk is
      sent to the model; everything else stays exactly where it was written.
      Nothing is folded into text and nothing is deleted
- [x] Anything scheduled can post here, and the schedules that do are listed
      beside the conversation rather than two pages away

### Reports
- [x] **Reports** — a section of its own for finished work: an investigation
      written up, an account of what an agent chat changed, whatever a schedule
      leaves behind. Filed with `report_write`, searched with FTS5, read on its
      own page
- [x] **Nothing here can be replied to**, and that is the section rather than a
      restriction on it. No composer, no route that accepts a message, and
      nothing on either page that renders the streaming shell — so there is
      nothing that could start a generation
- [x] Its own family, permission and capability flag, so a model that keeps
      notes need not file reports and a model that files reports need not have
      a library at all

### Audio
- [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped
      `/v1/audio/transcriptions` endpoint. The recording never touches disk
- [x] **Read aloud** — any `/v1/audio/speech` endpoint, with the voice list
      discovered from the server where it offers one
- [x] Instance defaults in Admin, per-reader overrides in Settings — voice,
      speed, dictation language, and whether replies play automatically

### Models and reasoning
- [x] OpenAI-compatible connections with encrypted keys and model discovery
- [x] **Reasoning display** — `reasoning_content` and inline `<think>` tags,
      collapsed by default, labelled with how long it took, never replayed as
      context
- [x] Model admin as a list plus a page per model; scales to hundreds
- [x] Ordering, pinning (a sidebar shortcut, *not* a reordering), instance
      default, per-user default, images, capability flags
- [x] Custom model picker showing avatars, descriptions and capabilities

### Attachments
- [x] Drag, paste or pick images, PDFs and text files
- [x] Images downscaled and sent to vision models as content parts
- [x] PDF and text extracted at upload and placed in the prompt
- [x] Type decided by inspecting bytes, random names on disk, non-images served
      as downloads with `nosniff`
- [x] No OCR: a scanned PDF says so rather than silently contributing nothing

### People
- [x] Accounts, argon2, revocable server-side sessions, self-service password
      change
- [x] Users and groups with permissions that **union** rather than override
- [x] Model access restricted to chosen groups
- [x] Registration toggle, instance settings stored in the database

### Prompts
- [x] Three layers — instance, model, chat — with the most specific winning
      **outright** rather than being concatenated
- [x] Every injected fragment editable at `/admin/prompts`: the tool guidance,
      the memory and skill sections, the seam above the authored prompt, and the
      request that names a chat
- [x] `{{variables}}` with a legend, values shown as they currently resolve, and
      pass-through for anything that is not one
- [x] A preview of the whole assembled system message, including unsaved edits
- [x] Defaults in code and overrides in the database, so improving a default
      still reaches an instance that never edited it

### Suggestions
- [x] Admin-managed cards on the new-chat screen; three seeded once at startup

### Interface
- [x] **`/` for commands** — compact, usage, mode, model, title, the panels,
      the theme. Anything not in the table is sent as an ordinary message, and
      `//` starts one with a literal slash
- [x] **Keyboard shortcuts** for the same jobs, listed beside the commands in
      one table so `/help` cannot go stale
- [x] Mentions and recognised commands are marked as you type, and again in the
      transcript, so you can see what a message will do before sending it
- [x] **Reasoning effort** per chat, with a per-model default. Sent as both
      `reasoning_effort` and `chat_template_kwargs`, and only once chosen:
      OpenAI and vLLM read the first, llama.cpp silently drops it and reads
      only the second
- [x] **Installable** — manifest, generated PWA icons, a service worker for the
      shell and a themed offline page. The worker deliberately never touches
      `/api/`: a reply is an event stream and caching one breaks it
- [x] Two themes (`moria`, `shire`) from one set of design tokens
- [x] Every control sized from `--control-h`, so rows line up by construction
- [x] Toasts and dialogs of our own; no `window.confirm` anywhere, and
      `data-prompt` for asking one line before a request goes out
- [x] **An approval card's command can be corrected** before it is allowed, and
      the transcript says who wrote what ran
- [x] **Refusing can say why** — "Give reason" opens a box beside Don't, and what
      you write goes back as the instruction rather than as a rejection, so the
      model carries on from it instead of spending a round asking what you meant
- [x] Original SVG artwork generated from a single source

### Operations
- [x] Additive schema sync — new tables and columns applied at startup
- [x] `deploy/` — systemd unit and nginx templates, install and update scripts

---

## The road to 1.0.0

What is left is not another large feature. It is four kinds of work: gaps that
read as bugs, features still owed, two structural jobs, and making this
installable and updatable by somebody who is not its author.

Each phase ends the same way, and that is a requirement rather than a habit:
tests green, `ruff` clean, `__version__` bumped (the service worker cache is
keyed on it, so a release without a bump serves stale JavaScript), committed,
pushed, and `deploy/update.sh` run — so the next phase starts from something
seen working.

### Phase 0 — the known bugs, and the CSS (`0.8.x`)
- [ ] **One version, one homepage.** `pyproject.toml` reads `__version__`
      instead of carrying its own copy of it, which had drifted three minors
- [ ] **Canvas and Terminal appear only where they can work.** `hx-get=""` is an
      attribute htmx *finds*, so an empty one fetches the current document and
      swaps the whole site into the canvas panel. The buttons follow the
      composer's kind toggle and its connection, which only the browser knows
- [ ] **The two top borders come off.** The sidebar footer and the composer sat
      either side of one vertical edge and were held to the same height so their
      borders would meet. Content scrolling under an edge that is not drawn is
      better than an edge that has to be aligned
- [ ] **One scroll container per screen.** `.tabs` assumes it is a flex child of
      `.main`; under the admin layout it is not, so `.tabs__body` never scrolls,
      the outer container does, and switching to a shorter panel drops the
      reader at the bottom of the page
- [ ] Sidebar scroll no longer chains to the document
- [x] **A connection may not point at this machine** unless an administrator
      says so, in one of three positions — never, one named port, or anywhere.
      An SSH profile aimed at `127.0.0.1` walked past the sentence the whole
      security story rests on, looking from the SSH layer down exactly like a
      container on the network

### Phase 1 — the scheduling tools (`0.9.0`)
- [x] **A model can schedule.** There was no tool for it — the seam was left
      (`Schedule.origin` has defined `ORIGIN_MODEL` with no writer since
      scheduling landed) and the tool was never built, so a model asked to
      "remind me every Monday" wrote a note and said it had. `schedule_create`,
      `schedule_list`, `schedule_update` and `schedule_cancel` over the same
      `rule.validate` the form and the compile already share
- [x] **The reply says the timing back in words.** A schedule is invisible until
      it fires, so `rule.describe` in the answer is the only moment anybody can
      check that Monday was read as Monday
- [x] The Scheduled list badges the ones nobody typed
- [x] Guidance saying which target a run should reach, and that anything which
      happens later or repeatedly is a schedule rather than a note — said in
      `tool.notes` and `tool.memory` as well, because those are what the model
      actually reached for

### Notifications (`0.9.1`)
- [x] **Everything that arrives is announced**, not only chat replies. The dots
      covered Reports and Messages; the announcement did not, so a scheduled run
      lit a dot in a corner and said nothing
- [x] **A count in the tab title** while you are looking elsewhere, cleared when
      you come back
- [x] **Web push**, so a schedule firing at seven in the morning reaches a
      browser that is shut. Hand-rolled against RFC 8291 and 8292 with the
      `cryptography` already here. Opt-in per device, asked for once in a dialog
      of ours before the browser's own — and the one thing in LLeMbas that
      contacts an outside service, which `services/push.py` says plainly
- [x] One arrival never announced three times: the service worker stays quiet
      when a window of its own has focus

### Phase 2 — image generation admin (`0.9.2`)
- [x] **Defaults an administrator can set** — steps, cfg, size, sampler,
      scheduler, denoise, negative, checkpoint, batch. There were none: one
      hardcoded set from the SD1.5 era, and prose in a box as the only way to
      change it. An empty box means "no opinion" and falls through, so a floor
      improved in code still reaches everyone
- [x] The right control for each: samplers and schedulers as selects, from the
      lists ComfyUI has been discovering and nothing has been reading;
      checkpoints picked rather than typed; sizes as numbers with presets
- [x] **`batch` at last** — `batch_size` was a literal `1` in the template.
      Deliberately not something a model may set
- [x] **The tool's schema restates the defaults it quotes**, or it goes on
      telling the model "Default 512" beside an instance that draws at 1024
- [x] A legend on the workflow editor saying what each placeholder fills, what
      it lands as, and what it resolves to right now

### Phase 3 — subagents (`0.9.3`)
- [x] **A model can delegate.** `subagent_run` hands one self-contained piece of
      work to a helper carrying the parent's connection, directory, model and
      effort, and gives its answer back as the tool result. Built on the
      mechanism scheduled runs already use, so it gets tools, rounds, budgets,
      metrics and steps rather than a second loop
- [x] **Safe by resolution, not by instruction** — no `ask_user`, no recursion,
      nothing that writes unless the call asked and the parent's mode allowed
      it, and commands only from a fixed read-only list in every mode including
      Auto, because the task text can have come from a page the parent read
- [x] **An unattended chat refuses instead of waiting.** Withdrawing `ask_user`
      was only half: an approval still built a card nobody could see and parked
      the reply for fifteen minutes, which from every screen is the feature not
      working. The same flag now covers a scheduled task's chat, which had the
      same hole
- [x] Its own bounds — per reply on the parent's `Generation`, instance-wide in
      a set, and per helper in settings of its own, so one runs out of room long
      before the reply that asked does
- [x] Guidance for the two uses that differ: fanning out across a research
      question, and reading a codebase — plus what a helper reads about being
      one

### Phase 4 — rebranding and customization (`0.9.4`)
- [x] **An instance can be somebody else's.** Name, tagline, logo, favicon and
      launcher icons derived from the logo, and the Middle-earth strings as
      editable data — defaults in code and overrides in the database, so a later
      release still improves the wording nobody changed. Blanked rather than
      dropped, because the settings store merges and a dropped key means "leave
      what was there"
- [x] **One snapshot, reached from everywhere.** A Jinja global over a
      process-level cache, because `render()` has no session and four render
      paths never reach it — the sign-in page, the error pages, the offline page
      and the SSE fragments
- [x] **A custom theme is a set of tokens**, not a stylesheet, and inherits its
      base through `data-base` — one selector added to `tokens.css` is what makes
      a custom *light* theme land on parchment rather than on near-black
- [x] The theme list stops being a hard-coded pair in five places
- [x] Global CSS overrides, served as `/branding.css` — a route rather than an
      inline block, so an administrator's CSS has no markup to escape from, with
      a content hash in the link so a save is not left to the browser's cache

### Phase 5 — extraction, embeddings and hybrid search (`0.9.5`)
- [x] **Extraction has settings** — upload size, image edge, JPEG quality, PDF
      pages, extracted characters, orphan age, extra text extensions. Read
      through a process-level snapshot, because `prepare` is called from places
      with no session. The decompression-bomb guard stays a constant: it is a
      guard, not a preference
- [x] **A dedicated embedding model**, picked from the models flagged for it —
      and a model that lost its flag is *named* rather than silently dropped
      from the picker
- [x] **Search becomes hybrid** — FTS5 and vector recall fused by reciprocal
      rank fusion, behind the one call the stores already searched through.
      Ranks rather than scores, because bm25 and cosine are not comparable and
      normalising them means picking a constant nobody can tune
- [x] **No model chosen means exactly the keyword search there is today** — no
      rows, no requests, the same ids in the same order, asserted rather than
      claimed
- [x] Indexing is fired and forgotten and noticed by a session event, so no
      writer has to remember it — forgetting would be silent, since only
      semantic recall would go stale
- [x] Vectors from two models never meet: width and model are stored beside
      every vector and a mismatch is skipped, because scoring across two spaces
      is a confident wrong answer rather than a missing one
- [x] A rebuild that commits as it goes, reports itself, and stops polling when
      it finishes

### Phase 6 — permissions, quotas and sharing (`0.9.6`)
- [x] **"What can this user actually do?"** answered on screen, and *where each
      permission came from* — `explain()` is the resolution's working shown
      rather than thrown away, which is the simulation the union rule exists to
      make unnecessary
- [x] List plus detail for users and groups; membership edited from **one** side,
      since a full-form POST from either used to overwrite the other's view
- [x] Reading and writing split for the three gates where the difference is a
      real decision — checked on the tool's risk, after the gate, defaulting on
- [x] **Quotas on a group**, resolved by maximum with **zero meaning no limit
      and winning outright**, and enforced at the five places each is knowable:
      before a reply is built, before a second one starts, on an agent reply's
      clock, before a minute of GPU, and beside the helper cap
- [x] Usage recorded even for a reply that was stopped or failed, because an
      endpoint charges either way and a quota a Stop button walks past is not one
- [x] **Deleting a group or a user forgets its grants, which it never did** —
      both halves for an account, since their rows cascade and the shares of
      those rows have nothing to cascade from
- [x] Sharing as its own action with a search box — one grant per request, stored
      the moment it is made rather than when the resource happens to be saved
- [x] A "Shared with me" filter in all four listings, reports shareable, and
      `library.share` on by default. Sharing stays read-only

### Phase 7 — packaging and updating (`0.9.7`)
- [x] **Docker**, one stage, non-root, data on a volume — and baking neither a
      secret key nor a database nor `.git`, so a container correctly reports
      that it was not installed from a checkout. TLS in front is a constraint
      rather than a recommendation: the service worker and the microphone both
      require HTTPS or localhost
- [x] **An LXC bootstrap** that creates an unprivileged container and runs the
      existing installer inside it — a wrapper, not a second install path
- [x] **Updating without a shell**, and by **channel** rather than by commit:
      `stable` follows release tags and `edge` the branch tip, because a branch
      tip is not a release. `git describe` for what is running, notes out of the
      annotated tag, and the commits between. Checking reaches the remote;
      opening the page does not. Git plumbing throughout and never a forge API —
      no token on the deployment host, no forge lock-in, and the one this was
      checked against 500s on that endpoint
- [x] **The button writes a file and an opt-in systemd unit does the work.** The
      service runs unprivileged and cannot restart itself, and the request
      carries no branch and no ref — so pressing it is always "deploy the branch
      this host was configured with" and never "deploy something else". Without
      the helper the page says so and prints the manual command
- [x] `/healthz`, which opens the database rather than only proving the socket
      is listening, and says nothing about what is here

### Phase 8 — audit and finalization (`0.9.8` … `0.9.9`)
- [ ] Security review over the whole accumulated diff
- [ ] A sweep for the failure this codebase keeps cataloguing: a control that
      looks like it works — a verb against a route that does not serve it, a
      trigger bound where the event does not go
- [ ] Every harness fragment read as a model would read it
- [ ] Focus, contrast and narrow widths across the admin screens
- [ ] Documentation, a fresh install, and an upgrade from an 0.8.x database

### Phase 9 — 1.0.0
- [ ] A commit that changes the version, this file and the README, and tags it

---

## After 1.0.0

- **OCR** for scanned PDFs
- **Conversation branching** — `Message.parent_id` exists unused; needs a UI for
  choosing between versions, which is why rewind truncates for now
- **Chat export** (Markdown, JSON)
- **Archived chats** — the column exists, nothing surfaces it
- **Several workers** — see the first known limit below
- **Writable shares**, which need history and a merge story before they need a
  column

---

## Known limits

Worth knowing before they surprise someone.

**One worker.** The generation registry and the stop mechanism are in-process.
Running several workers needs that state in the database or a broker, because
the request following a reply would not necessarily land in the process writing
it.

The schedule ticker is now the strongest reason this is not merely a
convenience. It is in-process like the rest, so **two workers means two tickers
and every schedule firing twice**. The claim that prevents a double-fire is a
Python lock plus a write committed in the same transaction, not `SELECT ... FOR
UPDATE`, which SQLite does not have. Scheduling also makes downtime visible in a
way nothing else here does: a dropped reply is one somebody watched fail, while
a missed run is one nobody saw at all — which is what the catch-up in the sweep
is for, and why it lives there rather than in a startup hook (a suspended host
or a long stall reproduces it with no restart to hang one on).

**A restart abandons replies in flight.** Shutdown cancels them and keeps what
each had. There is no resume.

**Schema changes are additive only.** New tables and columns apply themselves;
renames, drops and retypes are manual against the SQLite file. `MANUAL_STEPS`
in `db/migrations.py` is where such a step gets recorded.

**Attachments live on disk, unreferenced files are swept at startup.** No
deduplication, no size quota.

**Unread is polled every 10 seconds.** A push channel would be more responsive
but means an always-on connection per tab for the sake of a green dot.

**Installing needs HTTPS or localhost.** Service workers are unavailable over
plain HTTP, so a LAN install without TLS is a normal browser tab. The
microphone is unavailable for the same reason.

**Tool calling needs a model that supports it.** The `tools` flag is an
administrator's assertion, not something endpoints reliably advertise. Set it on
a model that cannot, and its replies fail rather than degrade.

**Library search is keyword-only until an embedding model is chosen.** FTS5 ranks
well and needs no dependency, but "how do I get paid" will not find a document
that says "invoicing". Choosing a model on **Extraction** adds a vector ranking
fused with that one; choosing none is byte-for-byte the search that was always
there. What that costs is an index that has to be rebuilt when the model changes,
and stale vectors that are ignored until it is.

**A model can write its own skills, and they take effect at once.** Marked as
model-authored and fully revertible, but a model that has just read a hostile
page could save a skill that outlives the conversation. The mitigation is that
it is visible and undoable, not that it was prevented.

---

## Deliberate decisions

Recorded because each looks like an oversight until you know the reason.

- **No JavaScript build step.** Browser libraries are hash-pinned and committed.
  A self-hosted tool should work offline and not report page views to a CDN.
- **Permissions union, never deny**, and quotas resolved by maximum for the same
  reason -- with the corner that zero means *no limit* and therefore wins, or
  "unlimited" would count for less than a large number. With denies, "why can
  this user not do X"
  cannot be answered without simulating every group.
- **System prompts replace, never stack.** Two layers that disagree give the
  model contradictory instructions and nobody can tell which is losing.
- **Rewind truncates, does not branch.** Branching needs a UI for choosing
  between versions; "go back and try again from here" is what was asked for.
- **Pinning is a shortcut, not an ordering.** A picker whose order silently
  differs from the admin screen is confusing.
- **Images only reach models marked `vision`.** Not graceful degradation: most
  endpoints reject the entire request rather than ignoring an image part. Tools
  are gated the same way, for the same reason.
- **Sharing grants reading, never writing.** Two people editing one note with no
  history and no merge is worse than the inconvenience of copying it.
- **Memory is never shareable.** A record about a person is not content to hand
  round.
- **Knowledge attached to a message is copied, not referenced.** History must not
  change under a conversation because a document was edited later.
- **The harness is prepended to the authored prompt, not a fourth layer.** It
  describes the machinery; the authored layers describe the behaviour. Only one
  authored layer still wins.
- **Tool results are not replayed.** Like reasoning: the answer already contains
  what the model made of them, and replaying stale results into every later
  request wastes the window and sends small models into search loops.
- **The service worker caches the shell, never a page with a user in it.** A
  cached conversation would be a snapshot that silently went stale, belonging to
  whoever was signed in last.
- **Markdown rendered server-side.** One code path produces the streamed and
  the stored view, so they cannot disagree.
- **This repository is public.** Deployment hostnames, ports and paths stay out
  of it; `deploy/` is templates, and the real values live in private notes.