Jaroslav Beneš
de178837b8
**Replies now run in the background.** Generation was driven by the SSE request, so navigating away or opening another chat cut the answer off mid-sentence. services/generation.py owns the work as its own task and the SSE endpoint merely follows it. Verified: attached briefly, closed the connection, went to another page -- the reply finished anyway, 832 characters, not marked stopped, auto-titled. Reattaching works because both `render` and `reasoning` frames now carry the whole block rather than a delta. A follower arriving late has no earlier fragments to append to, so deltas would leave it permanently missing the beginning. Verified: attached six seconds in and the first frame already contained 517 characters written while nobody watched. **Unread indicator.** A reply that lands with no follower attached marks its chat unread; the sidebar polls every 10s for out-of-band dot spans plus an HX-Trigger that raises a toast. Polled rather than pushed: a browser sitting on another chat has no connection to the one that finished, and an always-on channel per tab is a lot of machinery for a green dot. `unread_notified` stops the same arrival being announced every tick. Follower count is what decides "was anyone watching", so reading it as it arrives does not mark it unread -- verified both ways. **Stop is the send button.** While a reply is being written the send button becomes a red stop square, found via a MutationObserver on the thread since the composer and the streaming bubble are far apart in the document. The in-bubble Stop is gone. **Attachment border removed.** As asked -- an attachment is a picture, and the frame only ever drew at the wrong width. The anchor now shrink-wraps and the img's width/height attributes are overridden so a small image shows at its own size. Adds PLAN.md: what is built, what is not, known limits, and the decisions that look like oversights until you know the reason. 239 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7.2 KiB
LLeMbas — plan and status
Where the project is, what is deliberately not built yet, and the decisions
that would be expensive to revisit. Kept current as work lands; the detail of
how things work lives in CLAUDE.md.
Status: usable daily. Streaming chat, attachments, reasoning, users and
groups, model administration. 230 tests, ruff clean.
The shape of it
A self-hosted web UI for OpenAI-compatible endpoints, written in Python, themed after Middle-earth.
| Stack | FastAPI + Jinja + htmx + a little Alpine |
| Build step | none — no Node, no npm, no CDN at runtime |
| Database | SQLite, schema synchronised additively at startup |
| Deployment | systemd unit + nginx vhost, one worker |
These are load-bearing. Dropping the no-build rule or moving off SQLite would be a different project, not a refactor.
Done
Chat
- Streaming replies over server-sent events
- Markdown renders progressively — re-rendered whole every 100ms rather than appending tokens, because a list or code fence is only correct once its context exists
- Syntax highlighting (Pygments), sanitised with nh3
- Generation runs in the background — a task, not the request. Navigate away, open another chat, close the tab: the reply keeps being written and reattaching replays the whole state
- Stop — the send button becomes Stop while writing; what arrived is kept
- Rewind — edit one of your own turns and the conversation runs on from there. Truncates rather than branching
- Copy, regenerate, automatic chat titles
- Chats created on first message, so an abandoned composer leaves nothing
- Unread indicator — a green dot and a toast when a reply lands while you were elsewhere
- Folders, arbitrarily nested; deleting one keeps the chats inside it
Models and reasoning
- OpenAI-compatible connections with encrypted keys and model discovery
- Reasoning display —
reasoning_contentand inline<think>tags, collapsed by default, labelled with how long it took, never replayed as context - Model admin as a list plus a page per model; scales to hundreds
- Ordering, pinning (a sidebar shortcut, not a reordering), instance default, per-user default, images, capability flags
- Custom model picker showing avatars, descriptions and capabilities
Attachments
- Drag, paste or pick images, PDFs and text files
- Images downscaled and sent to vision models as content parts
- PDF and text extracted at upload and placed in the prompt
- Type decided by inspecting bytes, random names on disk, non-images served
as downloads with
nosniff - No OCR: a scanned PDF says so rather than silently contributing nothing
People
- Accounts, argon2, revocable server-side sessions, self-service password change
- Users and groups with permissions that union rather than override
- Model access restricted to chosen groups
- Registration toggle, instance settings stored in the database
Prompts
- Three layers — instance, model, chat — with the most specific winning outright rather than being concatenated
Interface
- Two themes (
moria,shire) from one set of design tokens - Every control sized from
--control-h, so rows line up by construction - Toasts and dialogs of our own; no
window.confirmanywhere - Original SVG artwork generated from a single source
Operations
- Additive schema sync — new tables and columns applied at startup
deploy/— systemd unit and nginx templates, install and update scripts
Not built yet
In the order they are likely to be worth doing.
Tools — built-in, with admin settings
A tool registry, per-tool settings in the admin area, and tool-calling wired
through the chat loop. Model.capabilities_json already carries a tools flag
that nothing reads. The chat loop currently assumes one request produces one
reply; tool calling makes it a loop, which is the real work.
Custom tools and MCP servers
An MCP client managing configured servers, their tools surfaced alongside the built-in ones. Depends on the tool loop above.
Agentic execution
Two modes, as originally specified:
- local — subprocess on the machine LLeMbas runs on
- remote — SSH connection profiles, with
shell.run/fs.read/fs.write
Needs a confirmation model before it does anything. Note that the systemd unit
is deliberately only ProtectSystem=full rather than strict because of
this — revisit the hardening when the real filesystem needs are known.
Image generation
Left until last from the start, as it needs heavy customisation. ComfyUI is already running on this machine and is the obvious first target.
Smaller things
- OCR for scanned PDFs
- Conversation branching —
Message.parent_idexists unused; needs a UI for choosing between versions, which is why rewind truncates for now - Web search as a built-in tool
- Chat export (Markdown, JSON)
- Archived chats — the column exists, nothing surfaces it
- Per-user quotas
Known limits
Worth knowing before they surprise someone.
One worker. The generation registry and the stop mechanism are in-process. Running several workers needs that state in the database or a broker, because the request following a reply would not necessarily land in the process writing it.
A restart abandons replies in flight. Shutdown cancels them and keeps what each had. There is no resume.
Schema changes are additive only. New tables and columns apply themselves;
renames, drops and retypes are manual against the SQLite file. MANUAL_STEPS
in db/migrations.py is where such a step gets recorded.
Attachments live on disk, unreferenced files are swept at startup. No deduplication, no size quota.
Unread is polled every 10 seconds. A push channel would be more responsive but means an always-on connection per tab for the sake of a green dot.
Deliberate decisions
Recorded because each looks like an oversight until you know the reason.
- No JavaScript build step. Browser libraries are hash-pinned and committed. A self-hosted tool should work offline and not report page views to a CDN.
- Permissions union, never deny. With denies, "why can this user not do X" cannot be answered without simulating every group.
- System prompts replace, never stack. Two layers that disagree give the model contradictory instructions and nobody can tell which is losing.
- Rewind truncates, does not branch. Branching needs a UI for choosing between versions; "go back and try again from here" is what was asked for.
- Pinning is a shortcut, not an ordering. A picker whose order silently differs from the admin screen is confusing.
- Images only reach models marked
vision. Not graceful degradation: most endpoints reject the entire request rather than ignoring an image part. - Markdown rendered server-side. One code path produces the streamed and the stored view, so they cannot disagree.
- This repository is public. Deployment hostnames, ports and paths stay out
of it;
deploy/is templates, and the real values live in private notes.