LLeMbas/PLAN.md

LLeMbas — plan and status

Where the project is, what is deliberately not built yet, and the decisions that would be expensive to revisit. Kept current as work lands; the detail of how things work lives in CLAUDE.md.

Status: usable daily. Streaming chat, attachments, reasoning, tool calling with web search, speech in and out, users and groups, model administration, installable as an app. 338 tests, ruff clean.

---

The shape of it

A self-hosted web UI for OpenAI-compatible endpoints, written in Python, themed after Middle-earth.

Stack	FastAPI + Jinja + htmx + a little Alpine
Build step	none — no Node, no npm, no CDN at runtime
Database	SQLite, schema synchronised additively at startup
Deployment	systemd unit + nginx vhost, one worker

These are load-bearing. Dropping the no-build rule or moving off SQLite would be a different project, not a refactor.

---

Done

Chat

• Streaming replies over server-sent events
• Markdown renders progressively — re-rendered whole every 100ms rather than appending tokens, because a list or code fence is only correct once its context exists
• Syntax highlighting (Pygments), sanitised with nh3
• Generation runs in the background — a task, not the request. Navigate away, open another chat, close the tab: the reply keeps being written and reattaching replays the whole state
• Stop — the send button becomes Stop while writing; what arrived is kept
• Rewind — edit one of your own turns and the conversation runs on from there. Truncates rather than branching
• Copy, regenerate, automatic chat titles
• Chats created on first message, so an abandoned composer leaves nothing
• Unread indicator — a green dot and a toast when a reply lands while you were elsewhere
• Folders, arbitrarily nested; deleting one keeps the chats inside it

Tools

• Tool calling — one reply is a bounded loop of requests, not one request. Text produced before a call is kept
• Web search as the first tool: DuckDuckGo (no setup), SearXNG or Firecrawl, chosen in the admin area
• Only offered to models flagged tools, because an endpoint without support rejects the whole request rather than ignoring the array
• Sources stay in the transcript; results are not replayed as context on the next turn, for the same reasons reasoning is not

Audio

• Dictation — record in the composer, transcribed by any OpenAI-shaped /v1/audio/transcriptions endpoint. The recording never touches disk
• Read aloud — any /v1/audio/speech endpoint, with the voice list discovered from the server where it offers one
• Instance defaults in Admin, per-reader overrides in Settings — voice, speed, dictation language, and whether replies play automatically

Models and reasoning

• OpenAI-compatible connections with encrypted keys and model discovery
• Reasoning display — reasoning_content and inline <think> tags, collapsed by default, labelled with how long it took, never replayed as context
• Model admin as a list plus a page per model; scales to hundreds
• Ordering, pinning (a sidebar shortcut, not a reordering), instance default, per-user default, images, capability flags
• Custom model picker showing avatars, descriptions and capabilities

Attachments

• Drag, paste or pick images, PDFs and text files
• Images downscaled and sent to vision models as content parts
• PDF and text extracted at upload and placed in the prompt
• Type decided by inspecting bytes, random names on disk, non-images served as downloads with nosniff
• No OCR: a scanned PDF says so rather than silently contributing nothing

People

• Accounts, argon2, revocable server-side sessions, self-service password change
• Users and groups with permissions that union rather than override
• Model access restricted to chosen groups
• Registration toggle, instance settings stored in the database

Prompts

• Three layers — instance, model, chat — with the most specific winning outright rather than being concatenated

Interface

• Installable — manifest, generated PWA icons, a service worker for the shell and a themed offline page. The worker deliberately never touches /api/: a reply is an event stream and caching one breaks it
• Two themes (moria, shire) from one set of design tokens
• Every control sized from --control-h, so rows line up by construction
• Toasts and dialogs of our own; no window.confirm anywhere
• Original SVG artwork generated from a single source

Operations

• Additive schema sync — new tables and columns applied at startup
• deploy/ — systemd unit and nginx templates, install and update scripts

---

Not built yet

In the order they are likely to be worth doing.

Custom tools and MCP servers

An MCP client managing configured servers, their tools surfaced alongside web_search. The loop they plug into exists now — services/tools.py is a registry and services/generation.py already runs bounded rounds — so this is a client and an admin screen rather than a change to how chat works.

Agentic execution

Two modes, as originally specified:

• local — subprocess on the machine LLeMbas runs on
• remote — SSH connection profiles, with shell.run / fs.read / fs.write

Needs a confirmation model before it does anything. Note that the systemd unit is deliberately only ProtectSystem=full rather than strict because of this — revisit the hardening when the real filesystem needs are known.

Image generation

Left until last from the start, as it needs heavy customisation. ComfyUI is already running on this machine and is the obvious first target.

Smaller things

• OCR for scanned PDFs
• Conversation branching — Message.parent_id exists unused; needs a UI for choosing between versions, which is why rewind truncates for now
• Chat export (Markdown, JSON)
• Archived chats — the column exists, nothing surfaces it
• Per-user quotas

---

Known limits

Worth knowing before they surprise someone.

One worker. The generation registry and the stop mechanism are in-process. Running several workers needs that state in the database or a broker, because the request following a reply would not necessarily land in the process writing it.

A restart abandons replies in flight. Shutdown cancels them and keeps what each had. There is no resume.

Schema changes are additive only. New tables and columns apply themselves; renames, drops and retypes are manual against the SQLite file. MANUAL_STEPS in db/migrations.py is where such a step gets recorded.

Attachments live on disk, unreferenced files are swept at startup. No deduplication, no size quota.

Unread is polled every 10 seconds. A push channel would be more responsive but means an always-on connection per tab for the sake of a green dot.

Installing needs HTTPS or localhost. Service workers are unavailable over plain HTTP, so a LAN install without TLS is a normal browser tab. The microphone is unavailable for the same reason.

Tool calling needs a model that supports it. The tools flag is an administrator's assertion, not something endpoints reliably advertise. Set it on a model that cannot, and its replies fail rather than degrade.

---

Deliberate decisions

Recorded because each looks like an oversight until you know the reason.

• No JavaScript build step. Browser libraries are hash-pinned and committed. A self-hosted tool should work offline and not report page views to a CDN.
• Permissions union, never deny. With denies, "why can this user not do X" cannot be answered without simulating every group.
• System prompts replace, never stack. Two layers that disagree give the model contradictory instructions and nobody can tell which is losing.
• Rewind truncates, does not branch. Branching needs a UI for choosing between versions; "go back and try again from here" is what was asked for.
• Pinning is a shortcut, not an ordering. A picker whose order silently differs from the admin screen is confusing.
• Images only reach models marked vision. Not graceful degradation: most endpoints reject the entire request rather than ignoring an image part. Tools are gated the same way, for the same reason.
• Tool results are not replayed. Like reasoning: the answer already contains what the model made of them, and replaying stale results into every later request wastes the window and sends small models into search loops.
• The service worker caches the shell, never a page with a user in it. A cached conversation would be a snapshot that silently went stale, belonging to whoever was signed in last.
• Markdown rendered server-side. One code path produces the streamed and the stored view, so they cannot disagree.
• This repository is public. Deployment hostnames, ports and paths stay out of it; deploy/ is templates, and the real values live in private notes.