Jaroslav Beneš
0f44e8d24c
LLeMbas now runs end to end. Register, add an OpenAI-compatible connection, and hold a real streaming conversation organised into folders. Verified against the local llama-swap instance. Streaming is the one genuinely tricky part. Sending a message returns two HTML fragments -- the user bubble and an empty assistant bubble carrying an sse-connect -- and that attribute is the ONLY thing that starts a generation. Rendering an incomplete assistant message as a streaming shell falls out of the same template, which means loading a page whose last reply never finished simply picks it up again. Details worth knowing about, each commented where it matters: - SSE payloads are split across several data: lines. A raw newline in one data: line truncates the event, which shows up the first time a model emits a code block. - Markdown is rendered server-side by the same helper for both the page and the final streamed frame, so the two cannot disagree. The fence renderer is replaced outright rather than using markdown-it's highlight option, which re-wraps output in a second <pre>. - escape_text is html.escape, not nh3.clean_text: it escapes character by character, so escaping stream chunks separately equals escaping the whole string. - The stream opens its own session via session_scope(); it outlives the request handler and the dependency-scoped session may be closed. - Deleting a folder keeps the chats inside it (FK is SET NULL). Losing a conversation to a mis-clicked folder delete is unforgivable. - Login failures use one message for "no such account" and "wrong password" so the form cannot enumerate registered addresses. Also adds deploy/ for the gamebox install at https://chat.lan: system unit, nginx vhost with buffering off (buffering on turns streaming into one lump at the end), and install/update scripts following the same service-user and /srv bind-mount conventions as llama-swap and comfyui. 70 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deployment
Installs LLeMbas as a system service behind nginx at https://chat.lan.
Written for gamebox (Arch), and follows the conventions already used there
for llama-swap and comfyui:
| Service user | lembas (system account, nologin) |
| Home | /home/lembas, bind-mounted to /srv/lembas |
| Checkout | /srv/lembas/app (git clone of the Gitea remote) |
| Virtualenv | /srv/lembas/venv |
| Database | /srv/lembas/data/lembas.db |
| Environment | /srv/lembas/lembas.env (mode 600) |
| Unit | /etc/systemd/system/lembas.service |
| Vhost | /etc/nginx/conf.d/chat.lan.conf, self-signed cert |
| Listens | 127.0.0.1:8080 — reachable only through nginx |
The home lives on /home rather than /var/lib because the root LV on that
box is only 50 GB; /srv/lembas is the same bind-mount trick as /srv/llama
and /srv/comfyui.
First install
./deploy/install.sh
Idempotent — safe to re-run. It creates the user and bind mount, clones the
repo, builds the venv, generates lembas.env with a fresh
LEMBAS_SECRET_KEY, installs the unit and vhost, issues a self-signed cert,
adds a /etc/hosts entry, and enables the service.
Then open https://chat.lan, accept the self-signed certificate warning, and create the first account — it becomes the administrator.
Deploying a change
git push # from the working copy
./deploy/update.sh
update.sh fetches, hard-resets /srv/lembas/app to origin/main, reinstalls
dependencies and restarts the service, then prints what changed. The hard reset
is deliberate: nothing is ever edited in place there, so there is no local work
to preserve and no merge conflicts to resolve.
Operating it
systemctl status lembas
journalctl -u lembas -f
sudo -u lembas /srv/lembas/venv/bin/lembas info # paths and counts
sudo systemctl restart lembas
Configuration lives in /srv/lembas/lembas.env. Edit it and restart.
Notes
The secret key is generated once. install.sh will not overwrite an
existing lembas.env. Rotating LEMBAS_SECRET_KEY signs every user out and
makes stored upstream API keys unreadable — they would have to be re-entered.
nginx buffering is off for a reason. Replies stream as server-sent events.
With proxy_buffering on (the default) nginx holds the entire reply and
delivers it in one lump at the end, which is indistinguishable from streaming
being broken. proxy_read_timeout is raised to an hour because a model can
think for minutes before the first token.
Name resolution. chat.lan is in Pi-hole, but this box queries the router
first and the router's dnsmasq is authoritative for .lan without forwarding
those queries on — the same reason comfy.lan needs one. install.sh adds a
/etc/hosts entry, which is harmless if DNS already answers.
Hardening is deliberately moderate. ProtectSystem=full, not strict: the
agentic features planned for later need to run commands, and a lockdown that
has to be torn out again is worse than one that was never applied.