Files

T

Jaroslav Beneš e9fab9d858 A job that finishes reaches the page you are looking at

Three complaints, all downstream of background commands.

A finished job woke the model and not the browser. `jobs.wake` writes the
completion and calls `generation.ensure`, and nothing tells the page: the only
stream here is per-message, opened by the `sse-connect` on an incomplete
assistant bubble -- which is a bubble this page has not got, because the reply
that created it began somewhere else. `_queue_frames` proves the swap works and
can only ride a stream already open. So the reader sat on the chat, watched the
sidebar dot light up for the chat in front of them, and had to click it or
reload to see a reply that had been there for minutes.

`GET /api/chats/{id}/tail?after=` and a five-second poller is the answer, polled
for the reason `/unread` is: a second always-on connection per tab is a lot of
machinery for something that happens a few times a day. A cursor it cannot place
-- absent, from another chat, naming a row a rewind deleted -- is answered with
204 and never with the transcript, which the page still holds every bubble of.
The cut is read from the row so `_inject`'s restamp moves it too, and compared in
SQL, a row read back from SQLite being naive where one still in the session is
aware; the `id >` tie-break is not decoration, since under a bare `>` a row
sharing the cut's microsecond is skipped for ever.

The cursor comes from the DOM, because the DOM is the honest answer to what the
page has -- the composer's POST, the `done` frame and the last poll all move it,
and a variable would have to be updated by each of them, correctly, for ever. On
`htmx:configRequest` rather than `hx-vals="js:…"`: two of the three things that
handler does are cancellations, which `hx-vals` cannot express. Not
`article.msg:last-of-type` either -- that is per-parent, so on a compacted chat
it answers with the last article inside the `<details>` and the poll re-appends
half the conversation. It is silent while a reply streams, since that reply
delivers its own bubbles in the one frame that can get the order right, and a
`htmx:beforeSwap` listener drops any answer holding a bubble already on the page:
the race `hx-sync` cannot reach, and a duplicate there is a second `sse-connect`
for one message rather than a cosmetic one. The route clears `unread` on every
tick including the 204, because `_persist` marks a reply unread whenever
`followers == 0` and that is true of a job-woken reply with somebody watching it.

The completion also claimed the reader had sent it. The role is load-bearing --
`_inject` sends a queued turn verbatim and `build_messages` must keep seeing a
user turn -- so `Message.machine` marks the bubble instead and the request is
untouched. Their initial, their name and a pencil offering to rewrite what a
machine reported: the route refuses the edit too, a hidden button being a
courtesy. `_completion_text` is deliberately unchanged, `tool.background` quoting
its opening sentence to the model, and there is now a test holding the two
together.

And the panel. `.jobs__row` had no horizontal padding while `.picker__menu` has
none either, so every row ran flush into the border under a header inset by
--sp-3. `jobs__row--open` had been emitted since the panel shipped with no rule
anywhere, so the row whose log was on screen looked like the ones that were not.
The dot was keyed on `status`, and `done` is exit 0 and exit 2 alike -- green
beside the row's own "Failed, exit 2" -- so `JobView.tone` answers the colour and
the template goes on answering the wording, which is the half a class name cannot
carry. `duration` is empty for a running job on purpose: this panel is fetched
when somebody opens it and never polled, so a live figure would freeze the
instant it painted. Its stamps are normalised before subtracting, a job started
before a restart and finished after it having one naive and one aware.

Driven under the DOM stub before committing, per the standing rule: two listeners
on document.body for events dispatched at a requesting element are exactly the
shape a regex cannot check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-05 12:19:07 +02:00

21 KiB

Raw Blame History

LLeMbas — plan and status

Where the project is, what is deliberately not built yet, and the decisions that would be expensive to revisit. Kept current as work lands; the detail of how things work lives in CLAUDE.md.

Status: usable daily. Streaming chat, attachments, reasoning, tool calling with web search, custom HTTP tools and MCP servers, agent chats that work on a machine over SSH, a knowledge library, notes, memory and skills, speech in and out, users and groups, model administration, installable as an app. 1529 tests, ruff clean.

The shape of it

A self-hosted web UI for OpenAI-compatible endpoints, written in Python, themed after Middle-earth.


Stack	FastAPI + Jinja + htmx + a little Alpine
Build step	none — no Node, no npm, no CDN at runtime
Database	SQLite, schema synchronised additively at startup
Deployment	systemd unit + nginx vhost, one worker

These are load-bearing. Dropping the no-build rule or moving off SQLite would be a different project, not a refactor.

Done

Chat

Streaming replies over server-sent events
Markdown renders progressively — re-rendered whole every 100ms rather than appending tokens, because a list or code fence is only correct once its context exists
Syntax highlighting (Pygments), sanitised with nh3
Generation runs in the background — a task, not the request. Navigate away, open another chat, close the tab: the reply keeps being written and reattaching replays the whole state
Stop — the send button becomes Stop while writing; what arrived is kept
Rewind — edit one of your own turns and the conversation runs on from there. Truncates rather than branching
Chat titles that fit the chat — an ordinary chat is named by a model from the first exchange, an agent chat from its opening words alone, which are already an objective. Renameable from the heading and from the sidebar row; one response updates both
Chats created on first message, so an abandoned composer leaves nothing
Unread indicator — a green dot and a toast when a reply lands while you were elsewhere
A reply that started without you asking still arrives — the open chat page polls for turns it has not got, so a background job waking the model appears where you are looking instead of only after a reload. Quiet while a reply is streaming, since that reply delivers its own bubbles
A turn nobody typed says so — a background job's completion is a user turn on the wire, because the request needs one, and a machine event in the transcript: its own icon and name, no pencil, and no claim that you sent it
Folders that carry something — arbitrarily nested, with a name, a description, a system prompt inherited by the chats inside them, and seeds for the model, the kind and the agent target. Deleting one keeps the chats
The sidebar splits Chat and Agent — a switch below the pinned models, stored on the account, filtering the folder tree as well as the loose chats
A reply reads as the sequence it was — thinking, prose, a tool call, more prose, in the order they happened, rather than three stacked zones with every tool block in the middle. Marks on the row index the three stores; a reply written before them renders exactly as it always did
Blocks open while the reply is still being written — the ids are stable across every swap and across the final one, and opening a block stops the thread chasing the bottom until you scroll back down
Per-reply metrics — tokens, context used as a percentage, tokens/second, live while streaming and kept afterwards. Estimated with a ~ when the endpoint reports no usage. Two chips: what the reply cost and what the conversation now occupies, each labelled, both moving between one usage block and the next rather than once a round
Compaction — a button, and automatically at a configurable percentage of the model's context. Summarised turns are kept and collapsed, not deleted
Temporary chats — never listed, swept after a day, with a Keep button
An admin-only request inspector beside the thread
Canvas — a third side panel holding open files, in tabs. Project files over SFTP in an agent chat; notes, skills, knowledge documents, this chat's text attachments and its own scratch document everywhere. Read with syntax highlighting, edited in a plain textarea, saved with a conflict check. Files the model touches open themselves, without taking the screen

Tools

Tool calling — one reply is a bounded loop of requests, not one request. Text produced before a call is kept
Web search as the first tool: DuckDuckGo (no setup), SearXNG or Firecrawl, chosen in the admin area
Only offered to models flagged tools, because an endpoint without support rejects the whole request rather than ignoring the array
Sources stay in the transcript; results are not replayed as context on the next turn, for the same reasons reasoning is not
A round's calls run together, and the reply says which tool is running — a remote tool taking seconds with nothing streaming looks like a hang
A reply can stop and ask you something — one or more questions on one card, with answers to pick from and a box to write your own, answered together. The same mechanism carries command approvals
Custom HTTP tools — an administrator describes one call: a JSON Schema, a URL template, headers, an encrypted secret and how to read the answer. Arguments may fill a hole but never move the target: the scheme and host are literal, values are escaped for where they land, and the origin is pinned afterwards
MCP servers over streamable HTTP — a hand-written client, so that check_url runs on every hop rather than being bypassed by somebody else's transport. Tools are discovered and cached by a button, namespaced per server, and a server's own descriptions are bounded before they reach a model as instructions
Both gated like the built-ins — a model capability, a permission — and restrictable to groups, with guidance of their own on /admin/prompts
Local MCP over stdio is deliberately absent: spawning a subprocess would run on this machine, which nothing here does

Agent chats

A chat is a Chat or an Agent, chosen when it starts and fixed thereafter — a transcript whose earlier turns ran somewhere else is not one conversation. Knowledge, memories and skills are shared across both
Nothing runs on the LLeMbas host. Commands go to a machine reached over SSH, so containment is somebody's considered choice of host — a container built for the job — rather than a sandbox built here. A local one was designed in detail and dropped; see CLAUDE.md for why
SSH connections are user-owned, like notes. An administrator decides only whether the feature exists at all
Trust on first use, made explicit: adding a host does not connect to it, Check shows its fingerprint with nothing sent, and only accepting pins it. A host that later answers with a different key is refused
Four modes as a table over what each tool does to the world — Manual asks about everything, Edit writes freely but asks before commands, Auto asks about nothing, Plan reads freely and changes nothing. Switchable at any time; read once per reply
Enforced in the generation loop, not in the prompt: a rule a model is merely told is one a poisoned file can argue with
A deny list beats Auto for any command it can match; an allow list cannot be matched at all by a command containing anything that joins two commands together. A deny pattern cannot either — so in Auto a compound line runs, which is the trade for Auto not asking about cd build && make. See CLAUDE.md; matching each segment would restore both and is not built
The terminal and the canvas open before the chat exists — on the new-chat screen, against the connection and directory being chosen there, and both re-point when that changes. The shell you opened and the files you left open are adopted into the chat when you send the first prompt
Background jobs are visible — a chip in the composer row counting what is still running, and a panel with each job's command, state, log tail, how long it took and a Stop button. The dot is coloured by outcome rather than by status, since done covers exit 0 and exit 2 alike. Survives a restart, because the job does
shell_run, file_read, file_write, file_list — files over SFTP, never through a shell, because the SSH exec protocol has no argv form
Plan mode ends with a plan you can carry out with one button, which switches to Edit and sends it back quoted rather than as an instruction
Per-reply budgets on steps, wall clock and output, with time spent waiting for you subtracted
A terminal panel beside the chat, holding a real shell on that chat's own connection. The modes govern the model; what a person types is theirs, since they hold the credential and could open the same shell with an ssh client. The model cannot see the panel — sending it output is a button
The shell outlives the panel and the page: closing it leaves a build running, and coming back reattaches with the scrollback. An idle timeout is what eventually ends one, and so does deleting the chat, or disabling, moving or deleting the connection
The panel is resizable, dragged from its edge or nudged with the arrow keys, and the width follows you to another browser
It knows where one command ends and the next begins — bash and zsh are given the markers VS Code and WezTerm use, so Copy and Send mean one command and its output rather than the last forty rows of the screen. An Auto toggle collects each one into the next message. Any other shell starts exactly as it did before, the buttons fall back to the screen and say so, and Auto is disabled rather than degraded
The project directory is listed for the model — one read-only command, git ls-files where that works so .gitignore is honoured for free, budgeted so a big directory becomes a count rather than a thousand filenames on every request
A directory is chosen by browsing it over SFTP, not by typing a path into an unlabelled box
The approval mode is chosen before the first message, beside the message box rather than in the header

The library

Knowledge bases — documents, images and saved web pages, grouped into named collections and ingested through the same pipeline as chat attachments, searched with SQLite FTS5
A chat can be pointed at particular bases, so "answer from the contracts folder" is a different question from "answer from everything I have"
Notes — longer things the model writes down and searches later; editable by hand, because they are yours
Memory — short facts, injected on every turn to a budget rather than searched, and managed in your settings
Skills — saved procedures. Only the name and description are injected; the body is fetched when the model decides it applies
A model may write and revise its own notes, memories and skills. Every skill revision is kept, attributed and revertible — the safety story is a record and a way back, not a gate
Sharing — a knowledge base, a note or a skill can be shared with a group or with named people, read-only. One visibility rule, and administrators do not bypass it. Documents are shared through their base
The harness — an operational prompt assembled from what a model actually has, so the tools get used rather than ignored
Attach menu: file, image, a web page fetched on the spot, or a document from the library
@ to name one — the library everywhere, and files in the project directory in an agent chat. The reference stays in the sentence and the contents come along, with the path and the machine, so the model knows exactly which file it was handed

Audio

Dictation — record in the composer, transcribed by any OpenAI-shaped /v1/audio/transcriptions endpoint. The recording never touches disk
Read aloud — any /v1/audio/speech endpoint, with the voice list discovered from the server where it offers one
Instance defaults in Admin, per-reader overrides in Settings — voice, speed, dictation language, and whether replies play automatically

Models and reasoning

OpenAI-compatible connections with encrypted keys and model discovery
Reasoning display — reasoning_content and inline <think> tags, collapsed by default, labelled with how long it took, never replayed as context
Model admin as a list plus a page per model; scales to hundreds
Ordering, pinning (a sidebar shortcut, not a reordering), instance default, per-user default, images, capability flags
Custom model picker showing avatars, descriptions and capabilities

Attachments

Drag, paste or pick images, PDFs and text files
Images downscaled and sent to vision models as content parts
PDF and text extracted at upload and placed in the prompt
Type decided by inspecting bytes, random names on disk, non-images served as downloads with nosniff
No OCR: a scanned PDF says so rather than silently contributing nothing

People

Accounts, argon2, revocable server-side sessions, self-service password change
Users and groups with permissions that union rather than override
Model access restricted to chosen groups
Registration toggle, instance settings stored in the database

Prompts

Three layers — instance, model, chat — with the most specific winning outright rather than being concatenated
Every injected fragment editable at /admin/prompts: the tool guidance, the memory and skill sections, the seam above the authored prompt, and the request that names a chat
{{variables}} with a legend, values shown as they currently resolve, and pass-through for anything that is not one
A preview of the whole assembled system message, including unsaved edits
Defaults in code and overrides in the database, so improving a default still reaches an instance that never edited it

Suggestions

Admin-managed cards on the new-chat screen; three seeded once at startup

Interface

/ for commands — compact, usage, mode, model, title, the panels, the theme. Anything not in the table is sent as an ordinary message, and // starts one with a literal slash
Keyboard shortcuts for the same jobs, listed beside the commands in one table so /help cannot go stale
Mentions and recognised commands are marked as you type, and again in the transcript, so you can see what a message will do before sending it
Reasoning effort per chat, with a per-model default. Sent as both reasoning_effort and chat_template_kwargs, and only once chosen: OpenAI and vLLM read the first, llama.cpp silently drops it and reads only the second
Installable — manifest, generated PWA icons, a service worker for the shell and a themed offline page. The worker deliberately never touches /api/: a reply is an event stream and caching one breaks it
Two themes (moria, shire) from one set of design tokens
Every control sized from --control-h, so rows line up by construction
Toasts and dialogs of our own; no window.confirm anywhere, and data-prompt for asking one line before a request goes out
An approval card's command can be corrected before it is allowed, and the transcript says who wrote what ran
Original SVG artwork generated from a single source

Operations

Additive schema sync — new tables and columns applied at startup
deploy/ — systemd unit and nginx templates, install and update scripts

Not built yet

In the order they are likely to be worth doing.

Image generation

Left until last from the start, as it needs heavy customisation. ComfyUI is already running on this machine and is the obvious first target.

Smaller things

OCR for scanned PDFs
Conversation branching — Message.parent_id exists unused; needs a UI for choosing between versions, which is why rewind truncates for now
Chat export (Markdown, JSON)
Semantic search in the library — the retrieval service is one call, so an embedding backend can go behind it without touching the tools or the UI
Archived chats — the column exists, nothing surfaces it
Per-user quotas

Known limits

Worth knowing before they surprise someone.

One worker. The generation registry and the stop mechanism are in-process. Running several workers needs that state in the database or a broker, because the request following a reply would not necessarily land in the process writing it.

A restart abandons replies in flight. Shutdown cancels them and keeps what each had. There is no resume.

Schema changes are additive only. New tables and columns apply themselves; renames, drops and retypes are manual against the SQLite file. MANUAL_STEPS in db/migrations.py is where such a step gets recorded.

Attachments live on disk, unreferenced files are swept at startup. No deduplication, no size quota.

Unread is polled every 10 seconds. A push channel would be more responsive but means an always-on connection per tab for the sake of a green dot.

Installing needs HTTPS or localhost. Service workers are unavailable over plain HTTP, so a LAN install without TLS is a normal browser tab. The microphone is unavailable for the same reason.

Tool calling needs a model that supports it. The tools flag is an administrator's assertion, not something endpoints reliably advertise. Set it on a model that cannot, and its replies fail rather than degrade.

Library search is keyword, not semantic. FTS5 ranks well and needs no dependency or embedding endpoint, but "how do I get paid" will not find a document that says "invoicing".

A model can write its own skills, and they take effect at once. Marked as model-authored and fully revertible, but a model that has just read a hostile page could save a skill that outlives the conversation. The mitigation is that it is visible and undoable, not that it was prevented.

Deliberate decisions

Recorded because each looks like an oversight until you know the reason.

No JavaScript build step. Browser libraries are hash-pinned and committed. A self-hosted tool should work offline and not report page views to a CDN.
Permissions union, never deny. With denies, "why can this user not do X" cannot be answered without simulating every group.
System prompts replace, never stack. Two layers that disagree give the model contradictory instructions and nobody can tell which is losing.
Rewind truncates, does not branch. Branching needs a UI for choosing between versions; "go back and try again from here" is what was asked for.
Pinning is a shortcut, not an ordering. A picker whose order silently differs from the admin screen is confusing.
Images only reach models marked vision. Not graceful degradation: most endpoints reject the entire request rather than ignoring an image part. Tools are gated the same way, for the same reason.
Sharing grants reading, never writing. Two people editing one note with no history and no merge is worse than the inconvenience of copying it.
Memory is never shareable. A record about a person is not content to hand round.
Knowledge attached to a message is copied, not referenced. History must not change under a conversation because a document was edited later.
The harness is prepended to the authored prompt, not a fourth layer. It describes the machinery; the authored layers describe the behaviour. Only one authored layer still wins.
Tool results are not replayed. Like reasoning: the answer already contains what the model made of them, and replaying stale results into every later request wastes the window and sends small models into search loops.
The service worker caches the shell, never a page with a user in it. A cached conversation would be a snapshot that silently went stale, belonging to whoever was signed in last.
Markdown rendered server-side. One code path produces the streamed and the stored view, so they cannot disagree.
This repository is public. Deployment hostnames, ports and paths stay out of it; deploy/ is templates, and the real values live in private notes.

21 KiB Raw Blame History