Jaroslav Beneš
7456525d19
Four pieces of work.
**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.
**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.
**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.
**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.
Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.
Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.
338 tests, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
376 lines
19 KiB
Markdown
376 lines
19 KiB
Markdown
# CLAUDE.md
|
||
|
||
Working notes for LLeMbas. Read this before changing anything.
|
||
|
||
## What it is
|
||
|
||
A self-hosted web UI for OpenAI-compatible LLM endpoints, written in Python and
|
||
themed after Middle-earth. Server-rendered FastAPI + Jinja + htmx; SQLite;
|
||
no JavaScript build step.
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
. .venv/bin/activate
|
||
pip install -e ".[dev,search]" # `search` adds ddgs for DuckDuckGo
|
||
|
||
lembas serve # http://127.0.0.1:8080
|
||
lembas info # paths + counts, useful when confused
|
||
lembas secret-key # generate LEMBAS_SECRET_KEY
|
||
lembas create-admin # create or promote an admin
|
||
|
||
pytest # 338 tests, ~16s
|
||
# PLAN.md tracks what is and is not built
|
||
ruff check . # lint (line length 100)
|
||
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
|
||
# needs fonttools and cairosvg)
|
||
python scripts/fetch_vendor.py # verify vendored JS against the lockfile
|
||
```
|
||
|
||
## Hard rules
|
||
|
||
These are the constraints the project is built around. Breaking one is a
|
||
redesign, not a tweak.
|
||
|
||
1. **No Node, no npm, no build step.** Browser libraries are downloaded once by
|
||
`scripts/fetch_vendor.py`, hash-pinned in `scripts/vendor.lock.json`, and
|
||
committed under `web/static/vendor/`.
|
||
2. **Nothing loads from a CDN at runtime.** A self-hosted tool must work
|
||
offline and must not report page views to a third party.
|
||
3. **No hard-coded values outside `tokens.css`.** Every colour, space, radius
|
||
and control height resolves through a CSS variable. `--control-h` is why
|
||
buttons, inputs and selects line up: they all take their height from it, so
|
||
a mixed row is flush by construction rather than by nudging.
|
||
4. **Additive-only schema changes.** SQLite only, no Alembic. `init_db()` runs
|
||
`db/migrations.py:sync_schema()`, which creates missing tables *and* adds
|
||
missing columns by diffing the models against the database. Renames, drops
|
||
and retypes are still manual. See "Changing the schema" below.
|
||
5. **Secrets never reach the browser.** API keys are Fernet-encrypted at rest
|
||
and only ever rendered masked.
|
||
6. **Model output is untrusted.** Everything from an endpoint goes through
|
||
`services/markdown.py` (markdown-it → nh3) or `escape_text()`. Never
|
||
`|safe` on anything that has not.
|
||
|
||
## The flavour rule
|
||
|
||
Middle-earth lives in the **artwork, theme names, empty states, loading lines
|
||
and error pages**. It does not live in the functional UI.
|
||
|
||
Chats are called *Chats*, not *Tales*. Folders are *Folders*, not *Chapters*.
|
||
Buttons say what they do. Someone who has never read the books must be able to
|
||
use this without a glossary. The two themes are named `moria` and `shire`, and
|
||
the 404 says "Not all those who wander are lost. This page, however, is." —
|
||
that is the right amount.
|
||
|
||
## Layout
|
||
|
||
```
|
||
src/lembas/
|
||
main.py app factory, lifespan, error handlers
|
||
config.py pydantic-settings, all LEMBAS_* variables
|
||
cli.py typer entry points
|
||
api/
|
||
deps.py Db / CurrentUser / RequiredUser / AdminUser
|
||
auth.py register, login, logout
|
||
pages.py full-page routes (chat shell, settings)
|
||
chats.py messaging + the SSE stream
|
||
folders.py folder CRUD
|
||
admin.py connections + instance settings
|
||
admin_models.py model ordering, defaults, images, access
|
||
admin_users.py users, groups, permissions
|
||
admin_audio.py speech-to-text and text-to-speech endpoints
|
||
admin_search.py web search provider and credentials
|
||
audio.py transcribe, speak, voice discovery
|
||
files.py upload, serve, remove attachments
|
||
preferences.py per-user theme, default model, password, audio
|
||
db/
|
||
base.py Base, UUID/Timestamp mixins
|
||
session.py engine, SQLite pragmas, init_db, session_scope
|
||
migrations.py additive schema sync (tables + columns)
|
||
models/ user, chat, connection, setting
|
||
security/ passwords (argon2), sessions, permissions
|
||
services/
|
||
llm/openai_client.py httpx streaming + model discovery
|
||
search/ ddgs, SearXNG and Firecrawl behind one shape
|
||
audio.py OpenAI-shaped /v1/audio/* client
|
||
tools.py tool registry, schemas, streamed-call reassembly
|
||
chat.py request building, endpoint resolution, titles
|
||
markdown.py markdown-it + pygments + nh3
|
||
crypto.py Fernet encrypt/decrypt/mask
|
||
files.py attachment validation, images, PDF/text extraction
|
||
reasoning.py splits thinking from the answer
|
||
settings_store.py runtime instance settings
|
||
uploads.py validated image storage
|
||
sse.py event framing
|
||
web/
|
||
templating.py render() -- always use this, not TemplateResponse
|
||
templates/ Jinja
|
||
static/ css, js, vendor, img, sw.js
|
||
assets/ SVG masters and PWA icons (generated)
|
||
deploy/ systemd unit, nginx vhost, install/update scripts
|
||
```
|
||
|
||
## Things that will bite you
|
||
|
||
**`render()`, not `TemplateResponse`.** `web/templating.py:render()` injects
|
||
`user`, `theme`, `version` and `allow_signup`. Templates assume they exist. If
|
||
you must call `templates.TemplateResponse` directly (the SSE path does, because
|
||
there is no `Request`), pass `user` explicitly — `chat/_message.html` renders
|
||
both roles and the user branch dereferences it.
|
||
|
||
**The message template is the state machine.** `chat/_message.html` renders an
|
||
incomplete assistant message as a streaming shell carrying `sse-connect`, and a
|
||
complete one as finished output. That is the *only* thing that starts a
|
||
generation. A consequence worth knowing: loading a page whose last reply is
|
||
unfinished restarts it, which is how a dropped connection recovers.
|
||
|
||
**SSE framing.** `services/sse.py:event()` splits payloads on newlines into
|
||
several `data:` lines. A raw newline in a single `data:` line truncates the
|
||
event — the failure shows up the first time a model emits a code block.
|
||
|
||
**Streaming opens its own database session.** `api/chats.py:_generate()` uses
|
||
`session_scope()`, not the request's session, because streaming outlives the
|
||
request handler.
|
||
|
||
**Escaping is chunk-safe on purpose.** `escape_text()` is `html.escape`, which
|
||
works character by character, so escaping stream chunks separately equals
|
||
escaping the whole string. `nh3.clean_text` would also be safe but escapes
|
||
spaces and slashes, tripling the size of every streamed token.
|
||
|
||
**The fence renderer is replaced, not configured.** markdown-it's `highlight`
|
||
option re-wraps output in `<pre><code>` unless the string starts with `<pre`,
|
||
which would nest a second `<pre>` inside our wrapper. `markdown.py` overrides
|
||
`renderer.rules["fence"]` instead. There is a regression test for this.
|
||
|
||
**SVG `<style>` is document-scoped.** Two text runs in one SVG sharing a class
|
||
name means the later rule recolours both. `build_artwork.py` takes class names
|
||
as parameters for exactly this reason.
|
||
|
||
**Gradient ids are document-global.** The `mark()` macro takes a `uid` because
|
||
two marks on one page with identical ids make the second silently reuse the
|
||
first one's gradients.
|
||
|
||
**Two kinds of settings.** `lembas.config` is deployment configuration read
|
||
from the environment at startup. `services/settings_store.py` is instance
|
||
settings an admin edits at runtime, stored in the `settings` table. Environment
|
||
variables seed the latter as an *initial* value only — once stored, the database
|
||
wins, or a toggle in the UI would silently revert on the next restart.
|
||
|
||
**Permissions are a union, and admins bypass them.** `security/permissions.py`
|
||
resolves a baseline (instance setting) widened by each group. A group grants;
|
||
it never denies — otherwise "why can this user not do X" needs a simulation of
|
||
every group to answer. Model *access* is separate: `models_visible_to()`.
|
||
|
||
**FastAPI cannot tell an empty form field from an absent one.** With
|
||
`x: str | None = Form(None)`, a submitted `x=` arrives as `None`, so "clear this
|
||
field" is indistinguishable from "leave it alone". `api/chats.py:update_chat`
|
||
reads `await request.form()` and checks key presence instead. Anything with a
|
||
clearable field must do the same.
|
||
|
||
**`Mapped[list]` without an element type is not a collection.** SQLAlchemy
|
||
treats a bare `Mapped[list]` as a scalar and hands back `None` instead of `[]`.
|
||
Always write `Mapped[list[Group]]`, with a `TYPE_CHECKING` import if the class
|
||
lives in another module.
|
||
|
||
**Reasoning arrives two ways.** A `reasoning_content` delta field (llama.cpp,
|
||
llama-swap, vLLM) or `<think>` tags inline in `content` (Ollama and friends).
|
||
`services/reasoning.py` handles the second with a streaming splitter, because
|
||
the tags arrive split across chunks. Reasoning is stored in `Message.reasoning`
|
||
and is deliberately **not** replayed as context on the next turn.
|
||
|
||
**Attachments are typed by their bytes, not their name.** `services/files.py`
|
||
sniffs magic numbers; a `.png` full of text is stored as text. Images are
|
||
downscaled and re-encoded (a phone photo is megabytes of base64), PDFs have
|
||
their text extracted **once at upload** — re-extracting per request would let a
|
||
reply change because a parser was upgraded.
|
||
|
||
**Images only go to models marked `vision`.** Sending content parts to an
|
||
endpoint without multimodal support is not graceful degradation; most reject
|
||
the whole request. `build_request()` checks the capability and falls back to a
|
||
plain string. A plain text turn must *stay* a plain string for the same reason.
|
||
|
||
**Attachments are served, never linked.** Images reach the model as base64 data
|
||
URIs: a local endpoint has no route back to LLeMbas and a hosted one has no
|
||
credentials. Non-images are served `Content-Disposition: attachment` with
|
||
`nosniff`, so an uploaded `.html` cannot execute in this origin.
|
||
|
||
**Uploads are unbound until the message is sent.** `Attachment.message_id` is
|
||
null in the composer; `files.claim()` binds them, and only unclaimed rows owned
|
||
by that user, so a forged id cannot pull in someone else's file. Abandoned ones
|
||
are swept at startup.
|
||
|
||
**Generation is a background task; the SSE endpoint only follows it.**
|
||
`services/generation.py` owns the work and the registry; `api/chats.py:_follow`
|
||
watches a `Generation` and streams what it sees. Closing the connection does
|
||
NOT stop the reply -- that was the old behaviour and it cut answers off when
|
||
the reader navigated away. Any route that creates an assistant placeholder must
|
||
also call `generation.ensure()`.
|
||
|
||
**Stream frames carry whole blocks, not deltas.** Both `render` and `reasoning`
|
||
send the complete text each time. That is what makes reattaching mid-reply
|
||
work: a follower arriving late has no earlier fragments to append to. It also
|
||
means Markdown is re-rendered whole, which is required anyway -- a list or code
|
||
fence is only correct once its context exists.
|
||
|
||
**Stopping sets a flag the producer checks.** `generation.request_stop()`;
|
||
whatever arrived is kept and the message is marked `stopped`, which is distinct
|
||
from `error`. In-process, so single-worker only.
|
||
|
||
**Unread is polled, not pushed.** A browser on another chat has no connection
|
||
to the one that finished. `/api/chats/unread` returns out-of-band dot spans and
|
||
an `HX-Trigger` for the toast; `unread_notified` stops the same arrival being
|
||
announced every tick. Re-rendering the whole sidebar instead would reset the
|
||
folder open/closed state every 10 seconds.
|
||
|
||
**Editing rewinds, it does not branch.** `POST .../messages/{id}/edit` rewrites
|
||
a user turn and **deletes everything after it**. Branching would need a UI for
|
||
choosing between versions; "go back and try again from here" is what was asked
|
||
for and what other clients do. `Message.parent_id` still exists unused.
|
||
|
||
**Dialogs and toasts are ours, not the browser's.** `static/js/ui.js` provides
|
||
`lembas.notify/confirm/prompt`, and intercepts htmx's `htmx:confirm` so every
|
||
existing `hx-confirm` gets the themed dialog with no change at the call site.
|
||
Plain forms opt in with `data-confirm`, lone submit buttons with
|
||
`data-confirm-button`. Never add a `window.confirm` back.
|
||
|
||
**The model picker is hand-built.** A `<select>` renders only text in an
|
||
`<option>` -- no avatar, no description, no badges. `chat/_model_picker.html`
|
||
plus the picker block in `ui.js`; the value lives in a hidden input so it still
|
||
behaves as a form field.
|
||
|
||
**Chats are created lazily.** There is no endpoint that makes an empty chat.
|
||
"New chat" is a link to `/chat`, which renders a composer with no row behind
|
||
it; `POST /api/chats/start` writes the chat together with its first message.
|
||
That is why an opened-and-abandoned chat never appears in the sidebar. Tests
|
||
that just need a chat use the `make_chat` fixture rather than the HTTP flow.
|
||
|
||
**Admin lists are list-plus-detail, never a form per row.** `/admin/models`
|
||
renders compact rows with search, filter tabs and pagination; the full form
|
||
lives at `/admin/models/{id}/edit`. A connection can advertise a hundred models,
|
||
and a page that renders a form for each is unusable. Any future admin list
|
||
(tools, agents) should follow the same shape.
|
||
|
||
**Route order matters for static path segments.** FastAPI matches in
|
||
registration order, so `/admin/models/bulk` must be registered *before*
|
||
`/admin/models/{model_id}` or "bulk" is parsed as a model id and 404s. This has
|
||
already been a bug once.
|
||
|
||
**Pinning is not ordering.** The model picker is always in the administrator's
|
||
`position` order. Pinned models get shortcuts in the chat sidebar and nothing
|
||
else -- a picker whose order differs from the admin screen is just confusing.
|
||
|
||
**System prompts are precedence, not concatenation.** chat > model > instance,
|
||
most specific wins outright (`services/chat.py:effective_system_prompt`).
|
||
Stacking them reads well in a settings screen and badly in practice: two layers
|
||
that disagree give the model contradictory instructions and nobody can tell
|
||
which is losing.
|
||
|
||
**JSON columns need reassignment.** `user.settings_json["theme"] = x` on a
|
||
plain dict is not detected. The columns use `MutableDict` (`db/types.py`), but
|
||
the safe habit is `obj.field = {**obj.field, "k": v}`.
|
||
|
||
**`[hidden]` needs `!important`.** The browser's rule is `[hidden] { display:
|
||
none }`, which any class setting `display` outranks — and `.btn` is
|
||
`display: inline-flex`. That is not theoretical: it is why the old Stop button,
|
||
created and then `hidden = true`, sat permanently beside Send. `app.css` forces
|
||
the attribute to win. Anything toggled with `hidden` depends on that line.
|
||
|
||
**Send and Stop are one button.** `chat/_composer.html` renders both icons and
|
||
`ui.js` flips `data-composer-action` plus `type` (`submit` ↔ `button`) when a
|
||
message in the thread is still streaming. Do not add a second button back.
|
||
|
||
**The tool loop is inside one generation.** `services/generation.py:_run()` runs
|
||
up to `tools_service.MAX_ROUNDS` request rounds for a single reply: stream,
|
||
accumulate tool calls, run them, append the results, ask again. `Generation`
|
||
accumulates content across all of them, so text emitted before a tool call
|
||
survives. Tools are only offered when search is enabled, the user has
|
||
`tools.web_search`, **and** the model is flagged `tools` — sending a `tools`
|
||
array to an endpoint without support fails the whole request, exactly as images
|
||
do without `vision`.
|
||
|
||
**Tool-call arguments arrive in fragments.** `delta.tool_calls` carries an
|
||
`index`, a name that appears once, and an `arguments` string split across
|
||
chunks. `tools.ToolCallAccumulator` rejoins them keyed on `index` — not on
|
||
name, which breaks the moment a model calls one tool twice in a turn.
|
||
|
||
**Tool results are not replayed.** Like reasoning, `Message.tool_calls_json` is
|
||
stored and rendered but never fed back as context. The answer already contains
|
||
what the model made of the results; replaying stale results and the schema into
|
||
every later request wastes the window and reliably sends a small model into a
|
||
search loop. The sources stay visible in the transcript.
|
||
|
||
**Search results are untrusted.** Hard rule 6 covers them as much as model
|
||
output. `chat/_tool_activity.html` escapes everything and only renders `http`
|
||
and `https` URLs as links — a result carrying a `javascript:` URL must never
|
||
become an anchor.
|
||
|
||
**A message bubble is rendered from four places.** `pages.py`,
|
||
`chats.post_message`, `chats.regenerate` and `chats._follow`. Each needs
|
||
`audio_service.template_flags(db, user)` or the speaker button's conditions are
|
||
undefined; the template uses `| default(false)` so a missed one degrades to no
|
||
button rather than an exception. `_follow` also passes `just_finished`, which is
|
||
what read-aloud-automatically keys off — without it, reopening a chat would
|
||
start reading its last reply out loud.
|
||
|
||
**Dictation audio never touches disk.** `api/audio.py` reads it into memory,
|
||
capped, and streams it upstream. It is not an attachment: it has no owner, no
|
||
row, and nothing would ever sweep it.
|
||
|
||
**The service worker must skip `/api/`.** A reply is an endless event stream and
|
||
passing one through a worker turns it into one delivery at the end, or nothing.
|
||
`static/js/sw.js` bails out on `/api/`, `/auth/`, `/admin/` and any request
|
||
accepting `text/event-stream`. It is served from `GET /sw.js` rather than the
|
||
static mount because a worker's scope is the path it came from.
|
||
|
||
## Changing the schema
|
||
|
||
There is no Alembic, but there *is* `db/migrations.py`. It compares the declared
|
||
models against the live database and issues `ALTER TABLE ... ADD COLUMN` for
|
||
anything missing, so adding a column to a model is free: restart and it appears,
|
||
with existing rows backfilled from a type-derived default.
|
||
|
||
It cannot rename, drop or retype a column, or add a UNIQUE/PRIMARY KEY to an
|
||
existing table — SQLite mostly cannot do those with ALTER TABLE either. Those
|
||
need the create-copy-swap dance by hand; record them in `MANUAL_STEPS` so a
|
||
failure has somewhere to point.
|
||
|
||
Because the runner exists, forward-looking columns are cheap now. `Message.parent_id`
|
||
and `content_parts_json` (branching, multimodal) predate it and are still unread.
|
||
|
||
## Artwork
|
||
|
||
Do not hand-edit files in `assets/` — they are generated. Change
|
||
`scripts/build_artwork.py` and re-run it. It also copies the few files the app
|
||
serves into `web/static/img/`.
|
||
|
||
The leaf geometry is defined once (`LEAF_BLADE`, `LEAF_MIDRIB`, …) and reused by
|
||
the icon, favicon, lockup and banner. The 64×64 mark must stay legible at 16px:
|
||
the favicon variant drops the score lines, rim and veins because they turn to
|
||
mud at that size. The icon sprite is a **template partial**
|
||
(`templates/partials/icons.html`), not an asset, because same-document
|
||
`<use href="#id">` is universally supported and the cross-document form is not.
|
||
|
||
The `mark()` macro in `_macros.html` duplicates the mark geometry so it can be
|
||
inlined and themed. If the mark changes, update both.
|
||
|
||
## Deployment
|
||
|
||
`deploy/` holds a systemd unit template, an nginx vhost template, and
|
||
install/update scripts. Both templates are parameterised (`__PREFIX__`,
|
||
`__SITE_HOST__`, …) and substituted at install time, so nothing host-specific is
|
||
committed here. See `deploy/README.md`.
|
||
|
||
This repository is **public**. Keep deployment-specific hostnames, ports and
|
||
internal infrastructure detail out of it — those belong in whatever private
|
||
notes describe the machine.
|
||
|
||
## Not built yet
|
||
|
||
Custom tools and MCP, agentic execution (local subprocess and SSH connection
|
||
profiles), image generation. Nav entries mark where each one goes. The tool
|
||
loop in `services/generation.py` is what they plug into — a second tool is a
|
||
registry entry, not a new code path.
|
||
|
||
No OCR: a scanned PDF is stored with an explanatory `extraction_error` rather
|
||
than silently contributing nothing.
|