# CLAUDE.md Working notes for LLeMbas. Read this before changing anything. ## What it is A self-hosted web UI for OpenAI-compatible LLM endpoints, written in Python and themed after Middle-earth. Server-rendered FastAPI + Jinja + htmx; SQLite; no JavaScript build step. ## Commands ```bash . .venv/bin/activate pip install -e ".[dev,search,ssh]" # `search` adds ddgs for DuckDuckGo, # `ssh` adds asyncssh for agent chats lembas serve # http://127.0.0.1:8080 lembas info # paths + counts, useful when confused lembas secret-key # generate LEMBAS_SECRET_KEY lembas create-admin # create or promote an admin pytest # 1897 tests, ~2min # PLAN.md tracks what is and is not built ruff check . # lint (line length 100) python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons; # needs fonttools and cairosvg) python scripts/fetch_vendor.py # verify vendored JS against the lockfile ``` ## Hard rules These are the constraints the project is built around. Breaking one is a redesign, not a tweak. 1. **No Node, no npm, no build step.** Browser libraries are downloaded once by `scripts/fetch_vendor.py`, hash-pinned in `scripts/vendor.lock.json`, and committed under `web/static/vendor/`. Adding one is a `--update`, same as bumping one: a name in `PACKAGES` with no lock entry has nothing to verify against, and the script refuses rather than writing it unpinned. xterm is the one heavy dependency — 280KB, more than everything else together — and is loaded only on a chat that can open a terminal. 2. **Nothing loads from a CDN at runtime.** A self-hosted tool must work offline and must not report page views to a third party. 3. **No hard-coded values outside `tokens.css`.** Every colour, space, radius and control height resolves through a CSS variable. `--control-h` is why buttons, inputs and selects line up: they all take their height from it, so a mixed row is flush by construction rather than by nudging. 4. **Additive-only schema changes.** SQLite only, no Alembic. `init_db()` runs `db/migrations.py:sync_schema()`, which creates missing tables *and* adds missing columns by diffing the models against the database. Renames, drops and retypes are still manual. See "Changing the schema" below. 5. **Secrets never reach the browser.** API keys are Fernet-encrypted at rest and only ever rendered masked. 6. **Model output is untrusted.** Everything from an endpoint goes through `services/markdown.py` (markdown-it → nh3) or `escape_text()`. Never `|safe` on anything that has not. ## The flavour rule Middle-earth lives in the **artwork, theme names, empty states, loading lines and error pages**. It does not live in the functional UI. Chats are called *Chats*, not *Tales*. Folders are *Folders*, not *Chapters*. Buttons say what they do. Someone who has never read the books must be able to use this without a glossary. The two themes are named `moria` and `shire`, and the 404 says "Not all those who wander are lost. This page, however, is." — that is the right amount. ## Layout ``` src/lembas/ main.py app factory, lifespan, error handlers config.py pydantic-settings, all LEMBAS_* variables cli.py typer entry points api/ deps.py Db / CurrentUser / RequiredUser / AdminUser auth.py register, login, logout pages.py full-page routes (chat shell, settings) chats.py messaging + the SSE stream folders.py folder CRUD admin.py connections + instance settings admin_models.py model ordering, defaults, images, access admin_users.py users, groups, permissions admin_audio.py speech-to-text and text-to-speech endpoints admin_search.py web search provider and credentials admin_images.py the ComfyUI, and the workflow templates on it admin_prompts.py the prompt fragment editor and its preview admin_suggestions.py the cards offered on the new-chat screen admin_tools.py custom HTTP tools and MCP servers admin_agents.py whether agent chats exist, and what they may spend agents.py SSH connections, kept by the people who own them terminal.py the terminal panel's WebSocket, and its two locks audio.py transcribe, speak, voice discovery library.py knowledge, notes, skills pages; memory CRUD messages.py the one long conversation, and paging back through it reports.py the Reports feed, and one report on its own page schedules.py the Scheduled list, the rule form, and a task's controls files.py upload, serve, remove attachments preferences.py per-user theme, default model, password, audio db/ base.py Base, UUID/Timestamp mixins session.py engine, SQLite pragmas, init_db, session_scope migrations.py additive schema sync (tables + columns) models/ user, chat, connection, setting security/ passwords (argon2), sessions, permissions services/ llm/openai_client.py httpx streaming + model discovery search/ ddgs, SearXNG and Firecrawl behind one shape images/ drawing on a ComfyUI: comfy.py speaks HTTP, workflow.py fills a template, tool.py ties them to a chat and decides whether to keep the result library/ documents, notes, memories, skills, FTS mcp/ remote MCP servers: framing, transport, rows to tools agent/ agent chats: the mode table, SSH, the six tools, terminal.py (shells held open behind the panel), shell_marks.py + capture.py (where one command ends), index.py (what is in the project directory), instructions.py (the project's own AGENTS.md), patch.py (applying a unified diff, and rendering one) audio.py OpenAI-shaped /v1/audio/* client push.py web push: the only thing here that talks to an outside service, and only because there is no other way to reach a browser that is closed fetch.py URL retrieval, HTML to text, the SSRF guard messages.py the one conversation per person: bounded in the request, unbounded on disk reports.py filing a finished piece of work, and finding it again schedules.py making, changing and stopping a schedule schedule/ work that happens because time passed: clock.py (whose "now"), rule.py (the recurrence, pure and total), ticker.py (the loop and the claim), runner.py (firing), compile.py (plain words into a rule), tool.py (the four a model calls) wake.py starting a reply from outside a request -- one lock discipline, shared by finished jobs and by schedules sharing.py one visibility rule for every library store prompts.py every injected prompt fragment, and {{variables}} metrics.py tokens, context percentage and tokens/second tokens.py the chars/4 estimate, for endpoints that report none compaction.py summarising the earlier turns of a long chat suggestions.py new-chat starting points, seeded once harness.py the operational prompt built from what a model has tools.py tool registry, schemas, streamed-call reassembly tool_labels.py what each tool is called and looks like, in one table plans.py a plan's shape, and keeping one current custom_tools.py the admin-defined HTTP tool runner tool_access.py who may be offered which admin-defined tool interaction.py pausing a reply to ask the reader something chat.py request building, endpoint resolution, titles markdown.py markdown-it + pygments + nh3 crypto.py Fernet encrypt/decrypt/mask files.py attachment validation, images, PDF/text extraction reasoning.py splits thinking from the answer settings_store.py runtime instance settings canvas.py what is open in the canvas panel, and where it comes from scratch.py a chat's own working document uploads.py validated image storage sse.py event framing web/ templating.py render() -- always use this, not TemplateResponse templates/ Jinja static/ css, js, vendor, img, sw.js js/commands.js the / table, and the keyboard that does the same jobs js/canvas.js the canvas panel's three behaviours js/composer.js the menu / and @ open, and the mirror that marks them assets/ SVG masters and PWA icons (generated) deploy/ systemd unit, nginx vhost, install/update scripts docs/notes/ the rest of "Things that will bite you", by topic ``` ## Things that will bite you **Three topics live in `docs/notes/`, and are not loaded with this file.** They were split out because this one is read in full on every session and each of them is only wanted while you are in that corner of the code. Read the file before touching the code it names -- these are the same notes, not a summary. - `docs/notes/agent-chats.md` -- the four modes and where they are enforced, how a round is authorised and how "always allow this" derives a pattern, the agent harness fragments, SSH, background jobs, the file tools and the patch matcher, the project listing and a project's own AGENTS.md, and the terminal panel. - `docs/notes/schedules-and-reports.md` -- claiming a schedule before firing it, the pure recurrence rule, the wake lock, task chats, Reports, why an empty `kind` means both sides of the sidebar switch and never "no filter", and the four scheduling tools a model calls (which did not exist, so a model asked to schedule something wrote a note and said it had). - `docs/notes/image-generation.md` -- the ComfyUI workflow with holes in it, what substitution walks, the review-and-retry loop, and how a failure reports itself. **`render()`, not `TemplateResponse`.** `web/templating.py:render()` injects `user`, `theme`, `layout`, `version` and `allow_signup`. Templates assume they exist. If you must call `templates.TemplateResponse` directly (the SSE path does, because there is no `Request`), pass `user` explicitly — `chat/_message.html` renders both roles and the user branch dereferences it. **The message template is the state machine.** `chat/_message.html` renders an incomplete assistant message as a streaming shell carrying `sse-connect`, and a complete one as finished output. That is the *only* thing that starts a generation. A consequence worth knowing: loading a page whose last reply is unfinished restarts it, which is how a dropped connection recovers. **SSE framing.** `services/sse.py:event()` splits payloads on newlines into several `data:` lines. A raw newline in a single `data:` line truncates the event — the failure shows up the first time a model emits a code block. **Streaming opens its own database session.** `api/chats.py:_generate()` uses `session_scope()`, not the request's session, because streaming outlives the request handler. **Escaping is chunk-safe on purpose.** `escape_text()` is `html.escape`, which works character by character, so escaping stream chunks separately equals escaping the whole string. `nh3.clean_text` would also be safe but escapes spaces and slashes, tripling the size of every streamed token. **The fence renderer is replaced, not configured.** markdown-it's `highlight` option re-wraps output in `
` unless the string starts with `` inside our wrapper. `markdown.py` overrides
`renderer.rules["fence"]` instead. There is a regression test for this.
**SVG `