Jaroslav Beneš
dd9e0e9440
LLeMbas now runs end to end. Register, add an OpenAI-compatible connection, and hold a real streaming conversation organised into folders. Verified against the local llama-swap instance. Streaming is the one genuinely tricky part. Sending a message returns two HTML fragments -- the user bubble and an empty assistant bubble carrying an sse-connect -- and that attribute is the ONLY thing that starts a generation. Rendering an incomplete assistant message as a streaming shell falls out of the same template, which means loading a page whose last reply never finished simply picks it up again. Details worth knowing about, each commented where it matters: - SSE payloads are split across several data: lines. A raw newline in one data: line truncates the event, which shows up the first time a model emits a code block. - Markdown is rendered server-side by the same helper for both the page and the final streamed frame, so the two cannot disagree. The fence renderer is replaced outright rather than using markdown-it's highlight option, which re-wraps output in a second <pre>. - escape_text is html.escape, not nh3.clean_text: it escapes character by character, so escaping stream chunks separately equals escaping the whole string. - The stream opens its own session via session_scope(); it outlives the request handler and the dependency-scoped session may be closed. - Deleting a folder keeps the chats inside it (FK is SET NULL). Losing a conversation to a mis-clicked folder delete is unforgivable. - Login failures use one message for "no such account" and "wrong password" so the form cannot enumerate registered addresses. Also adds deploy/ for the gamebox install at https://chat.lan: system unit, nginx vhost with buffering off (buffering on turns streaming into one lump at the end), and install/update scripts following the same service-user and /srv bind-mount conventions as llama-swap and comfyui. 70 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
81 lines
3.0 KiB
Markdown
81 lines
3.0 KiB
Markdown
# Deployment
|
|
|
|
Installs LLeMbas as a **system** service behind nginx at `https://chat.lan`.
|
|
|
|
Written for `gamebox` (Arch), and follows the conventions already used there
|
|
for llama-swap and comfyui:
|
|
|
|
| | |
|
|
|---|---|
|
|
| Service user | `lembas` (system account, `nologin`) |
|
|
| Home | `/home/lembas`, bind-mounted to `/srv/lembas` |
|
|
| Checkout | `/srv/lembas/app` (git clone of the Gitea remote) |
|
|
| Virtualenv | `/srv/lembas/venv` |
|
|
| Database | `/srv/lembas/data/lembas.db` |
|
|
| Environment | `/srv/lembas/lembas.env` (mode 600) |
|
|
| Unit | `/etc/systemd/system/lembas.service` |
|
|
| Vhost | `/etc/nginx/conf.d/chat.lan.conf`, self-signed cert |
|
|
| Listens | `127.0.0.1:8080` — reachable only through nginx |
|
|
|
|
The home lives on `/home` rather than `/var/lib` because the root LV on that
|
|
box is only 50 GB; `/srv/lembas` is the same bind-mount trick as `/srv/llama`
|
|
and `/srv/comfyui`.
|
|
|
|
## First install
|
|
|
|
```bash
|
|
./deploy/install.sh
|
|
```
|
|
|
|
Idempotent — safe to re-run. It creates the user and bind mount, clones the
|
|
repo, builds the venv, generates `lembas.env` with a fresh
|
|
`LEMBAS_SECRET_KEY`, installs the unit and vhost, issues a self-signed cert,
|
|
adds a `/etc/hosts` entry, and enables the service.
|
|
|
|
Then open <https://chat.lan>, accept the self-signed certificate warning, and
|
|
create the first account — it becomes the administrator.
|
|
|
|
## Deploying a change
|
|
|
|
```bash
|
|
git push # from the working copy
|
|
./deploy/update.sh
|
|
```
|
|
|
|
`update.sh` fetches, hard-resets `/srv/lembas/app` to `origin/main`, reinstalls
|
|
dependencies and restarts the service, then prints what changed. The hard reset
|
|
is deliberate: nothing is ever edited in place there, so there is no local work
|
|
to preserve and no merge conflicts to resolve.
|
|
|
|
## Operating it
|
|
|
|
```bash
|
|
systemctl status lembas
|
|
journalctl -u lembas -f
|
|
sudo -u lembas /srv/lembas/venv/bin/lembas info # paths and counts
|
|
sudo systemctl restart lembas
|
|
```
|
|
|
|
Configuration lives in `/srv/lembas/lembas.env`. Edit it and restart.
|
|
|
|
## Notes
|
|
|
|
**The secret key is generated once.** `install.sh` will not overwrite an
|
|
existing `lembas.env`. Rotating `LEMBAS_SECRET_KEY` signs every user out *and*
|
|
makes stored upstream API keys unreadable — they would have to be re-entered.
|
|
|
|
**nginx buffering is off for a reason.** Replies stream as server-sent events.
|
|
With `proxy_buffering on` (the default) nginx holds the entire reply and
|
|
delivers it in one lump at the end, which is indistinguishable from streaming
|
|
being broken. `proxy_read_timeout` is raised to an hour because a model can
|
|
think for minutes before the first token.
|
|
|
|
**Name resolution.** `chat.lan` is in Pi-hole, but this box queries the router
|
|
first and the router's dnsmasq is authoritative for `.lan` without forwarding
|
|
those queries on — the same reason `comfy.lan` needs one. `install.sh` adds a
|
|
`/etc/hosts` entry, which is harmless if DNS already answers.
|
|
|
|
**Hardening is deliberately moderate.** `ProtectSystem=full`, not `strict`: the
|
|
agentic features planned for later need to run commands, and a lockdown that
|
|
has to be torn out again is worse than one that was never applied.
|