Working chat: auth, connections, streaming, folders

LLeMbas now runs end to end. Register, add an OpenAI-compatible
connection, and hold a real streaming conversation organised into
folders. Verified against the local llama-swap instance.

Streaming is the one genuinely tricky part. Sending a message returns
two HTML fragments -- the user bubble and an empty assistant bubble
carrying an sse-connect -- and that attribute is the ONLY thing that
starts a generation. Rendering an incomplete assistant message as a
streaming shell falls out of the same template, which means loading a
page whose last reply never finished simply picks it up again.

Details worth knowing about, each commented where it matters:

- SSE payloads are split across several data: lines. A raw newline in
  one data: line truncates the event, which shows up the first time a
  model emits a code block.
- Markdown is rendered server-side by the same helper for both the page
  and the final streamed frame, so the two cannot disagree. The fence
  renderer is replaced outright rather than using markdown-it's
  highlight option, which re-wraps output in a second <pre>.
- escape_text is html.escape, not nh3.clean_text: it escapes character
  by character, so escaping stream chunks separately equals escaping
  the whole string.
- The stream opens its own session via session_scope(); it outlives the
  request handler and the dependency-scoped session may be closed.
- Deleting a folder keeps the chats inside it (FK is SET NULL). Losing
  a conversation to a mis-clicked folder delete is unforgivable.
- Login failures use one message for "no such account" and "wrong
  password" so the form cannot enumerate registered addresses.

Also adds deploy/ for the gamebox install at https://chat.lan: system
unit, nginx vhost with buffering off (buffering on turns streaming into
one lump at the end), and install/update scripts following the same
service-user and /srv bind-mount conventions as llama-swap and comfyui.

70 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 11:04:13 +02:00
parent 5ef2af6a9f
commit dd9e0e9440
59 changed files with 6273 additions and 12 deletions
+80
View File
@@ -0,0 +1,80 @@
# Deployment
Installs LLeMbas as a **system** service behind nginx at `https://chat.lan`.
Written for `gamebox` (Arch), and follows the conventions already used there
for llama-swap and comfyui:
| | |
|---|---|
| Service user | `lembas` (system account, `nologin`) |
| Home | `/home/lembas`, bind-mounted to `/srv/lembas` |
| Checkout | `/srv/lembas/app` (git clone of the Gitea remote) |
| Virtualenv | `/srv/lembas/venv` |
| Database | `/srv/lembas/data/lembas.db` |
| Environment | `/srv/lembas/lembas.env` (mode 600) |
| Unit | `/etc/systemd/system/lembas.service` |
| Vhost | `/etc/nginx/conf.d/chat.lan.conf`, self-signed cert |
| Listens | `127.0.0.1:8080` — reachable only through nginx |
The home lives on `/home` rather than `/var/lib` because the root LV on that
box is only 50 GB; `/srv/lembas` is the same bind-mount trick as `/srv/llama`
and `/srv/comfyui`.
## First install
```bash
./deploy/install.sh
```
Idempotent — safe to re-run. It creates the user and bind mount, clones the
repo, builds the venv, generates `lembas.env` with a fresh
`LEMBAS_SECRET_KEY`, installs the unit and vhost, issues a self-signed cert,
adds a `/etc/hosts` entry, and enables the service.
Then open <https://chat.lan>, accept the self-signed certificate warning, and
create the first account — it becomes the administrator.
## Deploying a change
```bash
git push # from the working copy
./deploy/update.sh
```
`update.sh` fetches, hard-resets `/srv/lembas/app` to `origin/main`, reinstalls
dependencies and restarts the service, then prints what changed. The hard reset
is deliberate: nothing is ever edited in place there, so there is no local work
to preserve and no merge conflicts to resolve.
## Operating it
```bash
systemctl status lembas
journalctl -u lembas -f
sudo -u lembas /srv/lembas/venv/bin/lembas info # paths and counts
sudo systemctl restart lembas
```
Configuration lives in `/srv/lembas/lembas.env`. Edit it and restart.
## Notes
**The secret key is generated once.** `install.sh` will not overwrite an
existing `lembas.env`. Rotating `LEMBAS_SECRET_KEY` signs every user out *and*
makes stored upstream API keys unreadable — they would have to be re-entered.
**nginx buffering is off for a reason.** Replies stream as server-sent events.
With `proxy_buffering on` (the default) nginx holds the entire reply and
delivers it in one lump at the end, which is indistinguishable from streaming
being broken. `proxy_read_timeout` is raised to an hour because a model can
think for minutes before the first token.
**Name resolution.** `chat.lan` is in Pi-hole, but this box queries the router
first and the router's dnsmasq is authoritative for `.lan` without forwarding
those queries on — the same reason `comfy.lan` needs one. `install.sh` adds a
`/etc/hosts` entry, which is harmless if DNS already answers.
**Hardening is deliberately moderate.** `ProtectSystem=full`, not `strict`: the
agentic features planned for later need to run commands, and a lockdown that
has to be torn out again is worse than one that was never applied.