Jaroslav Beneš
4643d1b584
The terminal and the canvas both needed a Chat, so they were missing from the one screen where you are choosing which machine to work on. A draft is the smallest thing that fixes it: an id, and the three facts behind it. The trick is that a draft resolves to a *transient* Chat -- constructed, never added to a session. `canvas.agent_ready`, `_executor`, `_load_agent`, `_save_agent` and `agent_session.resolve` read exactly four attributes between them and none of them queries or writes the row, so all of it works unchanged and nothing had to learn what a draft is. Proven against a real sshd rather than a stub: a transient chat opens and saves a project file over the same SFTP path a real one uses, and the database stays empty throughout. Chats are still created lazily. A draft is not a chat and never becomes one; when the first prompt makes the real one, the shell is re-keyed into it and the open tabs are copied across. `terminal.rekey` moves the registry key *and* `session.chat_id`, because close_for_profile, close_for_owner and the reaper all pop by the field -- a stale one would leave a dead session that `get` keeps handing out. The shell is only adopted when its profile and directory match the chat as finally resolved, since `_new_chat` settles an empty directory to the connection's own; otherwise it is left alone rather than transplanted onto a chat that says it runs elsewhere. Two canvas sources are refused on a draft, by name, and one of them is a hole rather than an inconvenience. `_load_file` authorises with `attachment.chat_id != chat.id`, and an upload made on the new-chat screen is stored with `chat_id=None` -- so a draft whose chat carried no id would make that comparison `None != None`, which is False, and open every unclaimed attachment its owner has. `as_chat` does set an id, so it already fails; the refusal is stated anyway, because a guarantee that lives in an id-shaped coincidence is one the next change breaks without noticing. Adoption needed almost no JavaScript: start_chat already answers with HX-Redirect, so the page reloads and the canvas adopts by construction while the terminal reconnects to the re-keyed session and replays its scrollback -- the "a reload is indistinguishable from a second tab" property working for us. What re-points them mid-screen is a `lembas:agent-target` event, dispatched from `setDir` and the connection select because assigning to a hidden field's value fires nothing on its own. Driven under a DOM stub before committing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
373 lines
20 KiB
Markdown
373 lines
20 KiB
Markdown
# LLeMbas — plan and status
|
|
|
|
Where the project is, what is deliberately not built yet, and the decisions
|
|
that would be expensive to revisit. Kept current as work lands; the detail of
|
|
*how* things work lives in [`CLAUDE.md`](CLAUDE.md).
|
|
|
|
**Status:** usable daily. Streaming chat, attachments, reasoning, tool calling
|
|
with web search, custom HTTP tools and MCP servers, agent chats that work on a
|
|
machine over SSH, a knowledge library, notes, memory and skills, speech in and
|
|
out, users and groups, model administration, installable as an app. 1529 tests,
|
|
`ruff` clean.
|
|
|
|
---
|
|
|
|
## The shape of it
|
|
|
|
A self-hosted web UI for OpenAI-compatible endpoints, written in Python, themed
|
|
after Middle-earth.
|
|
|
|
| | |
|
|
|---|---|
|
|
| Stack | FastAPI + Jinja + htmx + a little Alpine |
|
|
| Build step | none — no Node, no npm, no CDN at runtime |
|
|
| Database | SQLite, schema synchronised additively at startup |
|
|
| Deployment | systemd unit + nginx vhost, one worker |
|
|
|
|
These are load-bearing. Dropping the no-build rule or moving off SQLite would
|
|
be a different project, not a refactor.
|
|
|
|
---
|
|
|
|
## Done
|
|
|
|
### Chat
|
|
- [x] Streaming replies over server-sent events
|
|
- [x] **Markdown renders progressively** — re-rendered whole every 100ms rather
|
|
than appending tokens, because a list or code fence is only correct once
|
|
its context exists
|
|
- [x] Syntax highlighting (Pygments), sanitised with nh3
|
|
- [x] **Generation runs in the background** — a task, not the request. Navigate
|
|
away, open another chat, close the tab: the reply keeps being written and
|
|
reattaching replays the whole state
|
|
- [x] **Stop** — the send button becomes Stop while writing; what arrived is kept
|
|
- [x] **Rewind** — edit one of your own turns and the conversation runs on from
|
|
there. Truncates rather than branching
|
|
- [x] **Chat titles that fit the chat** — an ordinary chat is named by a model
|
|
from the first exchange, an agent chat from its opening words alone, which
|
|
are already an objective. Renameable from the heading and from the sidebar
|
|
row; one response updates both
|
|
- [x] Chats created on first message, so an abandoned composer leaves nothing
|
|
- [x] **Unread indicator** — a green dot and a toast when a reply lands while
|
|
you were elsewhere
|
|
- [x] **Folders that carry something** — arbitrarily nested, with a name, a
|
|
description, a system prompt inherited by the chats inside them, and seeds
|
|
for the model, the kind and the agent target. Deleting one keeps the chats
|
|
- [x] **The sidebar splits Chat and Agent** — a switch below the pinned models,
|
|
stored on the account, filtering the folder tree as well as the loose
|
|
chats
|
|
- [x] **A reply reads as the sequence it was** — thinking, prose, a tool call,
|
|
more prose, in the order they happened, rather than three stacked zones
|
|
with every tool block in the middle. Marks on the row index the three
|
|
stores; a reply written before them renders exactly as it always did
|
|
- [x] **Blocks open while the reply is still being written** — the ids are
|
|
stable across every swap and across the final one, and opening a block
|
|
stops the thread chasing the bottom until you scroll back down
|
|
- [x] Per-reply metrics — tokens, context used as a percentage, tokens/second,
|
|
live while streaming and kept afterwards. Estimated with a `~` when the
|
|
endpoint reports no usage. Two chips: what the reply **cost** and what the
|
|
conversation now **occupies**, each labelled, both moving between one
|
|
usage block and the next rather than once a round
|
|
- [x] Compaction — a button, and automatically at a configurable percentage of
|
|
the model's context. Summarised turns are kept and collapsed, not deleted
|
|
- [x] Temporary chats — never listed, swept after a day, with a Keep button
|
|
- [x] An admin-only request inspector beside the thread
|
|
- [x] **Canvas** — a third side panel holding open files, in tabs. Project files
|
|
over SFTP in an agent chat; notes, skills, knowledge documents, this
|
|
chat's text attachments and its own scratch document everywhere. Read with
|
|
syntax highlighting, edited in a plain textarea, saved with a conflict
|
|
check. Files the model touches open themselves, without taking the screen
|
|
|
|
### Tools
|
|
- [x] **Tool calling** — one reply is a bounded loop of requests, not one
|
|
request. Text produced before a call is kept
|
|
- [x] **Web search** as the first tool: DuckDuckGo (no setup), SearXNG or
|
|
Firecrawl, chosen in the admin area
|
|
- [x] Only offered to models flagged `tools`, because an endpoint without
|
|
support rejects the whole request rather than ignoring the array
|
|
- [x] Sources stay in the transcript; results are **not** replayed as context on
|
|
the next turn, for the same reasons reasoning is not
|
|
- [x] A round's calls run together, and the reply says which tool is running —
|
|
a remote tool taking seconds with nothing streaming looks like a hang
|
|
- [x] **A reply can stop and ask you something** — one or more questions on one
|
|
card, with answers to pick from and a box to write your own, answered
|
|
together. The same mechanism carries command approvals
|
|
- [x] **Custom HTTP tools** — an administrator describes one call: a JSON Schema,
|
|
a URL template, headers, an encrypted secret and how to read the answer.
|
|
Arguments may fill a hole but never move the target: the scheme and host
|
|
are literal, values are escaped for where they land, and the origin is
|
|
pinned afterwards
|
|
- [x] **MCP servers** over streamable HTTP — a hand-written client, so that
|
|
`check_url` runs on every hop rather than being bypassed by somebody
|
|
else's transport. Tools are discovered and cached by a button, namespaced
|
|
per server, and a server's own descriptions are bounded before they reach
|
|
a model as instructions
|
|
- [x] Both gated like the built-ins — a model capability, a permission — and
|
|
restrictable to groups, with guidance of their own on `/admin/prompts`
|
|
- [x] Local MCP over stdio is deliberately absent: spawning a subprocess would
|
|
run on this machine, which nothing here does
|
|
|
|
### Agent chats
|
|
- [x] A chat is a **Chat** or an **Agent**, chosen when it starts and fixed
|
|
thereafter — a transcript whose earlier turns ran somewhere else is not
|
|
one conversation. Knowledge, memories and skills are shared across both
|
|
- [x] **Nothing runs on the LLeMbas host.** Commands go to a machine reached
|
|
over SSH, so containment is somebody's considered choice of host — a
|
|
container built for the job — rather than a sandbox built here. A local
|
|
one was designed in detail and dropped; see CLAUDE.md for why
|
|
- [x] **SSH connections are user-owned**, like notes. An administrator decides
|
|
only whether the feature exists at all
|
|
- [x] Trust on first use, made explicit: adding a host does not connect to it,
|
|
**Check** shows its fingerprint with nothing sent, and only accepting
|
|
pins it. A host that later answers with a different key is refused
|
|
- [x] Four modes as a table over what each tool does to the world —
|
|
**Manual** asks about everything, **Edit** writes freely but asks before
|
|
commands, **Auto** asks about nothing, **Plan** reads freely and changes
|
|
nothing. Switchable at any time; read once per reply
|
|
- [x] Enforced in the generation loop, not in the prompt: a rule a model is
|
|
merely told is one a poisoned file can argue with
|
|
- [x] A deny list beats **Auto** for any command it can match; an allow list
|
|
cannot be matched at all by a command containing anything that joins two
|
|
commands together. A deny pattern cannot either — so in Auto a compound
|
|
line runs, which is the trade for Auto not asking about `cd build && make`.
|
|
See CLAUDE.md; matching each segment would restore both and is not built
|
|
- [x] **The terminal and the canvas open before the chat exists** — on the
|
|
new-chat screen, against the connection and directory being chosen there,
|
|
and both re-point when that changes. The shell you opened and the files
|
|
you left open are adopted into the chat when you send the first prompt
|
|
- [x] **Background jobs are visible** — a chip in the composer row counting what
|
|
is still running, and a panel with each job's command, state, log tail and
|
|
a Stop button. Survives a restart, because the job does
|
|
- [x] `shell_run`, `file_read`, `file_write`, `file_list` — files over SFTP,
|
|
never through a shell, because the SSH exec protocol has no argv form
|
|
- [x] **Plan mode ends with a plan** you can carry out with one button, which
|
|
switches to Edit and sends it back quoted rather than as an instruction
|
|
- [x] Per-reply budgets on steps, wall clock and output, with time spent
|
|
waiting for you subtracted
|
|
- [x] **A terminal panel** beside the chat, holding a real shell on that chat's
|
|
own connection. The modes govern the model; what a person types is theirs,
|
|
since they hold the credential and could open the same shell with an ssh
|
|
client. The model cannot see the panel — sending it output is a button
|
|
- [x] The shell outlives the panel and the page: closing it leaves a build
|
|
running, and coming back reattaches with the scrollback. An idle timeout
|
|
is what eventually ends one, and so does deleting the chat, or disabling,
|
|
moving or deleting the connection
|
|
- [x] **The panel is resizable**, dragged from its edge or nudged with the
|
|
arrow keys, and the width follows you to another browser
|
|
- [x] **It knows where one command ends and the next begins** — bash and zsh
|
|
are given the markers VS Code and WezTerm use, so *Copy* and *Send* mean
|
|
one command and its output rather than the last forty rows of the screen.
|
|
An **Auto** toggle collects each one into the next message. Any other
|
|
shell starts exactly as it did before, the buttons fall back to the
|
|
screen and say so, and Auto is disabled rather than degraded
|
|
- [x] **The project directory is listed for the model** — one read-only
|
|
command, `git ls-files` where that works so `.gitignore` is honoured for
|
|
free, budgeted so a big directory becomes a count rather than a thousand
|
|
filenames on every request
|
|
- [x] **A directory is chosen by browsing it** over SFTP, not by typing a path
|
|
into an unlabelled box
|
|
- [x] The approval mode is chosen **before** the first message, beside the
|
|
message box rather than in the header
|
|
|
|
### The library
|
|
- [x] **Knowledge bases** — documents, images and saved web pages, grouped into
|
|
named collections and ingested through the same pipeline as chat
|
|
attachments, searched with SQLite FTS5
|
|
- [x] A chat can be pointed at particular bases, so "answer from the contracts
|
|
folder" is a different question from "answer from everything I have"
|
|
- [x] **Notes** — longer things the model writes down and searches later;
|
|
editable by hand, because they are yours
|
|
- [x] **Memory** — short facts, injected on every turn to a budget rather than
|
|
searched, and managed in your settings
|
|
- [x] **Skills** — saved procedures. Only the name and description are injected;
|
|
the body is fetched when the model decides it applies
|
|
- [x] A model may write and revise its own notes, memories and skills. Every
|
|
skill revision is kept, attributed and revertible — the safety story is a
|
|
record and a way back, not a gate
|
|
- [x] **Sharing** — a knowledge base, a note or a skill can be shared with a
|
|
group or with named people, read-only. One visibility rule, and
|
|
administrators do not bypass it. Documents are shared through their base
|
|
- [x] **The harness** — an operational prompt assembled from what a model
|
|
actually has, so the tools get used rather than ignored
|
|
- [x] Attach menu: file, image, a web page fetched on the spot, or a document
|
|
from the library
|
|
- [x] **`@` to name one** — the library everywhere, and files in the project
|
|
directory in an agent chat. The reference stays in the sentence and the
|
|
contents come along, with the path and the machine, so the model knows
|
|
exactly which file it was handed
|
|
|
|
### Audio
|
|
- [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped
|
|
`/v1/audio/transcriptions` endpoint. The recording never touches disk
|
|
- [x] **Read aloud** — any `/v1/audio/speech` endpoint, with the voice list
|
|
discovered from the server where it offers one
|
|
- [x] Instance defaults in Admin, per-reader overrides in Settings — voice,
|
|
speed, dictation language, and whether replies play automatically
|
|
|
|
### Models and reasoning
|
|
- [x] OpenAI-compatible connections with encrypted keys and model discovery
|
|
- [x] **Reasoning display** — `reasoning_content` and inline `<think>` tags,
|
|
collapsed by default, labelled with how long it took, never replayed as
|
|
context
|
|
- [x] Model admin as a list plus a page per model; scales to hundreds
|
|
- [x] Ordering, pinning (a sidebar shortcut, *not* a reordering), instance
|
|
default, per-user default, images, capability flags
|
|
- [x] Custom model picker showing avatars, descriptions and capabilities
|
|
|
|
### Attachments
|
|
- [x] Drag, paste or pick images, PDFs and text files
|
|
- [x] Images downscaled and sent to vision models as content parts
|
|
- [x] PDF and text extracted at upload and placed in the prompt
|
|
- [x] Type decided by inspecting bytes, random names on disk, non-images served
|
|
as downloads with `nosniff`
|
|
- [x] No OCR: a scanned PDF says so rather than silently contributing nothing
|
|
|
|
### People
|
|
- [x] Accounts, argon2, revocable server-side sessions, self-service password
|
|
change
|
|
- [x] Users and groups with permissions that **union** rather than override
|
|
- [x] Model access restricted to chosen groups
|
|
- [x] Registration toggle, instance settings stored in the database
|
|
|
|
### Prompts
|
|
- [x] Three layers — instance, model, chat — with the most specific winning
|
|
**outright** rather than being concatenated
|
|
- [x] Every injected fragment editable at `/admin/prompts`: the tool guidance,
|
|
the memory and skill sections, the seam above the authored prompt, and the
|
|
request that names a chat
|
|
- [x] `{{variables}}` with a legend, values shown as they currently resolve, and
|
|
pass-through for anything that is not one
|
|
- [x] A preview of the whole assembled system message, including unsaved edits
|
|
- [x] Defaults in code and overrides in the database, so improving a default
|
|
still reaches an instance that never edited it
|
|
|
|
### Suggestions
|
|
- [x] Admin-managed cards on the new-chat screen; three seeded once at startup
|
|
|
|
### Interface
|
|
- [x] **`/` for commands** — compact, usage, mode, model, title, the panels,
|
|
the theme. Anything not in the table is sent as an ordinary message, and
|
|
`//` starts one with a literal slash
|
|
- [x] **Keyboard shortcuts** for the same jobs, listed beside the commands in
|
|
one table so `/help` cannot go stale
|
|
- [x] Mentions and recognised commands are marked as you type, and again in the
|
|
transcript, so you can see what a message will do before sending it
|
|
- [x] **Reasoning effort** per chat, with a per-model default. Sent as both
|
|
`reasoning_effort` and `chat_template_kwargs`, and only once chosen:
|
|
OpenAI and vLLM read the first, llama.cpp silently drops it and reads
|
|
only the second
|
|
- [x] **Installable** — manifest, generated PWA icons, a service worker for the
|
|
shell and a themed offline page. The worker deliberately never touches
|
|
`/api/`: a reply is an event stream and caching one breaks it
|
|
- [x] Two themes (`moria`, `shire`) from one set of design tokens
|
|
- [x] Every control sized from `--control-h`, so rows line up by construction
|
|
- [x] Toasts and dialogs of our own; no `window.confirm` anywhere, and
|
|
`data-prompt` for asking one line before a request goes out
|
|
- [x] **An approval card's command can be corrected** before it is allowed, and
|
|
the transcript says who wrote what ran
|
|
- [x] Original SVG artwork generated from a single source
|
|
|
|
### Operations
|
|
- [x] Additive schema sync — new tables and columns applied at startup
|
|
- [x] `deploy/` — systemd unit and nginx templates, install and update scripts
|
|
|
|
---
|
|
|
|
## Not built yet
|
|
|
|
In the order they are likely to be worth doing.
|
|
|
|
### Image generation
|
|
Left until last from the start, as it needs heavy customisation. ComfyUI is
|
|
already running on this machine and is the obvious first target.
|
|
|
|
### Smaller things
|
|
- **OCR** for scanned PDFs
|
|
- **Conversation branching** — `Message.parent_id` exists unused; needs a UI for
|
|
choosing between versions, which is why rewind truncates for now
|
|
- **Chat export** (Markdown, JSON)
|
|
- **Semantic search** in the library — the retrieval service is one call, so an
|
|
embedding backend can go behind it without touching the tools or the UI
|
|
- **Archived chats** — the column exists, nothing surfaces it
|
|
- **Per-user quotas**
|
|
|
|
---
|
|
|
|
## Known limits
|
|
|
|
Worth knowing before they surprise someone.
|
|
|
|
**One worker.** The generation registry and the stop mechanism are in-process.
|
|
Running several workers needs that state in the database or a broker, because
|
|
the request following a reply would not necessarily land in the process writing
|
|
it.
|
|
|
|
**A restart abandons replies in flight.** Shutdown cancels them and keeps what
|
|
each had. There is no resume.
|
|
|
|
**Schema changes are additive only.** New tables and columns apply themselves;
|
|
renames, drops and retypes are manual against the SQLite file. `MANUAL_STEPS`
|
|
in `db/migrations.py` is where such a step gets recorded.
|
|
|
|
**Attachments live on disk, unreferenced files are swept at startup.** No
|
|
deduplication, no size quota.
|
|
|
|
**Unread is polled every 10 seconds.** A push channel would be more responsive
|
|
but means an always-on connection per tab for the sake of a green dot.
|
|
|
|
**Installing needs HTTPS or localhost.** Service workers are unavailable over
|
|
plain HTTP, so a LAN install without TLS is a normal browser tab. The
|
|
microphone is unavailable for the same reason.
|
|
|
|
**Tool calling needs a model that supports it.** The `tools` flag is an
|
|
administrator's assertion, not something endpoints reliably advertise. Set it on
|
|
a model that cannot, and its replies fail rather than degrade.
|
|
|
|
**Library search is keyword, not semantic.** FTS5 ranks well and needs no
|
|
dependency or embedding endpoint, but "how do I get paid" will not find a
|
|
document that says "invoicing".
|
|
|
|
**A model can write its own skills, and they take effect at once.** Marked as
|
|
model-authored and fully revertible, but a model that has just read a hostile
|
|
page could save a skill that outlives the conversation. The mitigation is that
|
|
it is visible and undoable, not that it was prevented.
|
|
|
|
---
|
|
|
|
## Deliberate decisions
|
|
|
|
Recorded because each looks like an oversight until you know the reason.
|
|
|
|
- **No JavaScript build step.** Browser libraries are hash-pinned and committed.
|
|
A self-hosted tool should work offline and not report page views to a CDN.
|
|
- **Permissions union, never deny.** With denies, "why can this user not do X"
|
|
cannot be answered without simulating every group.
|
|
- **System prompts replace, never stack.** Two layers that disagree give the
|
|
model contradictory instructions and nobody can tell which is losing.
|
|
- **Rewind truncates, does not branch.** Branching needs a UI for choosing
|
|
between versions; "go back and try again from here" is what was asked for.
|
|
- **Pinning is a shortcut, not an ordering.** A picker whose order silently
|
|
differs from the admin screen is confusing.
|
|
- **Images only reach models marked `vision`.** Not graceful degradation: most
|
|
endpoints reject the entire request rather than ignoring an image part. Tools
|
|
are gated the same way, for the same reason.
|
|
- **Sharing grants reading, never writing.** Two people editing one note with no
|
|
history and no merge is worse than the inconvenience of copying it.
|
|
- **Memory is never shareable.** A record about a person is not content to hand
|
|
round.
|
|
- **Knowledge attached to a message is copied, not referenced.** History must not
|
|
change under a conversation because a document was edited later.
|
|
- **The harness is prepended to the authored prompt, not a fourth layer.** It
|
|
describes the machinery; the authored layers describe the behaviour. Only one
|
|
authored layer still wins.
|
|
- **Tool results are not replayed.** Like reasoning: the answer already contains
|
|
what the model made of them, and replaying stale results into every later
|
|
request wastes the window and sends small models into search loops.
|
|
- **The service worker caches the shell, never a page with a user in it.** A
|
|
cached conversation would be a snapshot that silently went stale, belonging to
|
|
whoever was signed in last.
|
|
- **Markdown rendered server-side.** One code path produces the streamed and
|
|
the stored view, so they cannot disagree.
|
|
- **This repository is public.** Deployment hostnames, ports and paths stay out
|
|
of it; `deploy/` is templates, and the real values live in private notes.
|