Working chat: auth, connections, streaming, folders
LLeMbas now runs end to end. Register, add an OpenAI-compatible connection, and hold a real streaming conversation organised into folders. Verified against the local llama-swap instance. Streaming is the one genuinely tricky part. Sending a message returns two HTML fragments -- the user bubble and an empty assistant bubble carrying an sse-connect -- and that attribute is the ONLY thing that starts a generation. Rendering an incomplete assistant message as a streaming shell falls out of the same template, which means loading a page whose last reply never finished simply picks it up again. Details worth knowing about, each commented where it matters: - SSE payloads are split across several data: lines. A raw newline in one data: line truncates the event, which shows up the first time a model emits a code block. - Markdown is rendered server-side by the same helper for both the page and the final streamed frame, so the two cannot disagree. The fence renderer is replaced outright rather than using markdown-it's highlight option, which re-wraps output in a second <pre>. - escape_text is html.escape, not nh3.clean_text: it escapes character by character, so escaping stream chunks separately equals escaping the whole string. - The stream opens its own session via session_scope(); it outlives the request handler and the dependency-scoped session may be closed. - Deleting a folder keeps the chats inside it (FK is SET NULL). Losing a conversation to a mis-clicked folder delete is unforgivable. - Login failures use one message for "no such account" and "wrong password" so the form cannot enumerate registered addresses. Also adds deploy/ for the gamebox install at https://chat.lan: system unit, nginx vhost with buffering off (buffering on turns streaming into one lump at the end), and install/update scripts following the same service-user and /srv bind-mount conventions as llama-swap and comfyui. 70 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -0,0 +1,80 @@
|
||||
# Deployment
|
||||
|
||||
Installs LLeMbas as a **system** service behind nginx at `https://chat.lan`.
|
||||
|
||||
Written for `gamebox` (Arch), and follows the conventions already used there
|
||||
for llama-swap and comfyui:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Service user | `lembas` (system account, `nologin`) |
|
||||
| Home | `/home/lembas`, bind-mounted to `/srv/lembas` |
|
||||
| Checkout | `/srv/lembas/app` (git clone of the Gitea remote) |
|
||||
| Virtualenv | `/srv/lembas/venv` |
|
||||
| Database | `/srv/lembas/data/lembas.db` |
|
||||
| Environment | `/srv/lembas/lembas.env` (mode 600) |
|
||||
| Unit | `/etc/systemd/system/lembas.service` |
|
||||
| Vhost | `/etc/nginx/conf.d/chat.lan.conf`, self-signed cert |
|
||||
| Listens | `127.0.0.1:8080` — reachable only through nginx |
|
||||
|
||||
The home lives on `/home` rather than `/var/lib` because the root LV on that
|
||||
box is only 50 GB; `/srv/lembas` is the same bind-mount trick as `/srv/llama`
|
||||
and `/srv/comfyui`.
|
||||
|
||||
## First install
|
||||
|
||||
```bash
|
||||
./deploy/install.sh
|
||||
```
|
||||
|
||||
Idempotent — safe to re-run. It creates the user and bind mount, clones the
|
||||
repo, builds the venv, generates `lembas.env` with a fresh
|
||||
`LEMBAS_SECRET_KEY`, installs the unit and vhost, issues a self-signed cert,
|
||||
adds a `/etc/hosts` entry, and enables the service.
|
||||
|
||||
Then open <https://chat.lan>, accept the self-signed certificate warning, and
|
||||
create the first account — it becomes the administrator.
|
||||
|
||||
## Deploying a change
|
||||
|
||||
```bash
|
||||
git push # from the working copy
|
||||
./deploy/update.sh
|
||||
```
|
||||
|
||||
`update.sh` fetches, hard-resets `/srv/lembas/app` to `origin/main`, reinstalls
|
||||
dependencies and restarts the service, then prints what changed. The hard reset
|
||||
is deliberate: nothing is ever edited in place there, so there is no local work
|
||||
to preserve and no merge conflicts to resolve.
|
||||
|
||||
## Operating it
|
||||
|
||||
```bash
|
||||
systemctl status lembas
|
||||
journalctl -u lembas -f
|
||||
sudo -u lembas /srv/lembas/venv/bin/lembas info # paths and counts
|
||||
sudo systemctl restart lembas
|
||||
```
|
||||
|
||||
Configuration lives in `/srv/lembas/lembas.env`. Edit it and restart.
|
||||
|
||||
## Notes
|
||||
|
||||
**The secret key is generated once.** `install.sh` will not overwrite an
|
||||
existing `lembas.env`. Rotating `LEMBAS_SECRET_KEY` signs every user out *and*
|
||||
makes stored upstream API keys unreadable — they would have to be re-entered.
|
||||
|
||||
**nginx buffering is off for a reason.** Replies stream as server-sent events.
|
||||
With `proxy_buffering on` (the default) nginx holds the entire reply and
|
||||
delivers it in one lump at the end, which is indistinguishable from streaming
|
||||
being broken. `proxy_read_timeout` is raised to an hour because a model can
|
||||
think for minutes before the first token.
|
||||
|
||||
**Name resolution.** `chat.lan` is in Pi-hole, but this box queries the router
|
||||
first and the router's dnsmasq is authoritative for `.lan` without forwarding
|
||||
those queries on — the same reason `comfy.lan` needs one. `install.sh` adds a
|
||||
`/etc/hosts` entry, which is harmless if DNS already answers.
|
||||
|
||||
**Hardening is deliberately moderate.** `ProtectSystem=full`, not `strict`: the
|
||||
agentic features planned for later need to run commands, and a lockdown that
|
||||
has to be torn out again is worse than one that was never applied.
|
||||
@@ -0,0 +1,57 @@
|
||||
# chat.lan - HTTPS reverse proxy to LLeMbas (127.0.0.1:8080).
|
||||
# Deployed to /etc/nginx/conf.d/chat.lan.conf. Self-signed cert (chat.lan).
|
||||
#
|
||||
# Mirrors the comfy.lan and llama.lan vhosts on this box.
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name chat.lan;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
listen [::]:443 ssl;
|
||||
http2 on;
|
||||
server_name chat.lan;
|
||||
|
||||
ssl_certificate /etc/nginx/ssl/chat.lan.crt;
|
||||
ssl_certificate_key /etc/nginx/ssl/chat.lan.key;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
|
||||
# File uploads land here once that feature exists; 0 = no limit.
|
||||
client_max_body_size 0;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Streamed replies are server-sent events. Every one of these matters:
|
||||
# with buffering on, nginx holds the whole reply and delivers it in one
|
||||
# lump at the end, which looks exactly like streaming being broken.
|
||||
proxy_buffering off;
|
||||
proxy_request_buffering off;
|
||||
proxy_cache off;
|
||||
# SSE is plain HTTP/1.1 chunked, so the connection header must not be
|
||||
# the websocket upgrade dance -- it must simply stay open.
|
||||
proxy_set_header Connection "";
|
||||
|
||||
# A model can think for minutes before the first token. The default
|
||||
# 60s read timeout would cut long generations off mid-sentence.
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
}
|
||||
|
||||
# Static assets are immutable per release and never need revalidating.
|
||||
location /static/ {
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
proxy_set_header Host $host;
|
||||
expires 1h;
|
||||
add_header Cache-Control "public";
|
||||
}
|
||||
}
|
||||
Executable
+115
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env bash
|
||||
# Install LLeMbas as a system service behind nginx at https://chat.lan.
|
||||
#
|
||||
# Follows the conventions already used on this box for llama-swap and comfyui:
|
||||
# a dedicated service user whose home lives on /home (the root LV is only
|
||||
# 50 GB) and is bind-mounted to /srv/<name>, a system unit so it survives
|
||||
# logout, and an nginx vhost with a self-signed cert.
|
||||
#
|
||||
# Idempotent: safe to re-run. To deploy new code afterwards use update.sh,
|
||||
# which is what a `git push` should be followed by.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_URL="${LEMBAS_REPO_URL:-https://git.houmeres.sk/Houmeres/LLeMbas.git}"
|
||||
BRANCH="${LEMBAS_BRANCH:-main}"
|
||||
SERVICE_USER=lembas
|
||||
HOME_DIR=/home/lembas
|
||||
PREFIX=/srv/lembas
|
||||
APP="$PREFIX/app"
|
||||
VENV="$PREFIX/venv"
|
||||
ENV_FILE="$PREFIX/lembas.env"
|
||||
HERE="$(dirname "$(readlink -f "$0")")"
|
||||
|
||||
echo "== service user =="
|
||||
# --system: no ageing, no mail spool. Home under /home, not /var/lib, so the
|
||||
# venv and database sit on the big volume.
|
||||
if ! getent passwd "$SERVICE_USER" >/dev/null; then
|
||||
sudo useradd --system --create-home --home-dir "$HOME_DIR" \
|
||||
--shell /usr/bin/nologin --comment "LLeMbas" "$SERVICE_USER"
|
||||
else
|
||||
echo " user $SERVICE_USER already exists"
|
||||
fi
|
||||
sudo chmod 755 "$HOME_DIR"
|
||||
|
||||
echo "== /srv/lembas bind-mount onto /home =="
|
||||
sudo mkdir -p "$PREFIX"
|
||||
grep -q "^$HOME_DIR[[:space:]]" /etc/fstab \
|
||||
|| echo "$HOME_DIR $PREFIX none bind 0 0" | sudo tee -a /etc/fstab >/dev/null
|
||||
sudo systemctl daemon-reload
|
||||
mountpoint -q "$PREFIX" || sudo mount "$PREFIX"
|
||||
|
||||
echo "== checkout =="
|
||||
if [[ ! -d "$APP/.git" ]]; then
|
||||
sudo -u "$SERVICE_USER" git clone --branch "$BRANCH" "$REPO_URL" "$APP"
|
||||
else
|
||||
echo " already cloned; use update.sh to pull"
|
||||
fi
|
||||
|
||||
echo "== virtualenv =="
|
||||
if [[ ! -x "$VENV/bin/python" ]]; then
|
||||
sudo -u "$SERVICE_USER" python -m venv "$VENV"
|
||||
fi
|
||||
sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet --upgrade pip
|
||||
sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet -e "$APP"
|
||||
|
||||
echo "== environment =="
|
||||
# Generated once and never regenerated: rotating LEMBAS_SECRET_KEY would sign
|
||||
# every user out and make the stored API keys unreadable.
|
||||
if [[ ! -f "$ENV_FILE" ]]; then
|
||||
KEY=$("$VENV/bin/python" -c "import secrets; print(secrets.token_urlsafe(48))")
|
||||
sudo tee "$ENV_FILE" >/dev/null <<EOF
|
||||
# LLeMbas service environment. Generated by deploy/install.sh.
|
||||
# LEMBAS_SECRET_KEY signs sessions and encrypts stored API keys.
|
||||
# Changing it signs everyone out and makes stored API keys unreadable.
|
||||
LEMBAS_SECRET_KEY=$KEY
|
||||
LEMBAS_DATA_DIR=$PREFIX/data
|
||||
# Loopback only: reachable through the nginx chat.lan vhost, never direct.
|
||||
LEMBAS_HOST=127.0.0.1
|
||||
LEMBAS_PORT=8080
|
||||
LEMBAS_LOG_LEVEL=info
|
||||
LEMBAS_ALLOW_SIGNUP=true
|
||||
LEMBAS_DEFAULT_THEME=moria
|
||||
EOF
|
||||
sudo chown "$SERVICE_USER:$SERVICE_USER" "$ENV_FILE"
|
||||
sudo chmod 600 "$ENV_FILE"
|
||||
echo " generated $ENV_FILE"
|
||||
else
|
||||
echo " $ENV_FILE exists, keeping it (and its secret key)"
|
||||
fi
|
||||
|
||||
sudo install -d -o "$SERVICE_USER" -g "$SERVICE_USER" -m 750 "$PREFIX/data"
|
||||
|
||||
echo "== systemd unit =="
|
||||
sudo install -Dm644 "$HERE/lembas.service" /etc/systemd/system/lembas.service
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
echo "== self-signed cert for chat.lan =="
|
||||
sudo mkdir -p /etc/nginx/ssl
|
||||
if [[ ! -f /etc/nginx/ssl/chat.lan.crt ]]; then
|
||||
sudo openssl req -x509 -newkey rsa:2048 -nodes \
|
||||
-keyout /etc/nginx/ssl/chat.lan.key -out /etc/nginx/ssl/chat.lan.crt \
|
||||
-days 3650 -subj "/CN=chat.lan" -addext "subjectAltName=DNS:chat.lan"
|
||||
sudo chmod 600 /etc/nginx/ssl/chat.lan.key
|
||||
sudo chmod 644 /etc/nginx/ssl/chat.lan.crt
|
||||
fi
|
||||
|
||||
echo "== nginx vhost =="
|
||||
sudo install -Dm644 "$HERE/chat.lan.nginx.conf" /etc/nginx/conf.d/chat.lan.conf
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
|
||||
echo "== local name resolution =="
|
||||
# chat.lan is in Pi-hole, but this box asks the router first and the router's
|
||||
# dnsmasq is authoritative for .lan without forwarding it on -- same reason
|
||||
# comfy.lan needs a hosts entry. Harmless if DNS already resolves it.
|
||||
grep -q 'chat\.lan' /etc/hosts \
|
||||
|| printf '127.0.0.1\tchat.lan\n::1\t\tchat.lan\n' | sudo tee -a /etc/hosts >/dev/null
|
||||
|
||||
echo "== enable service =="
|
||||
sudo systemctl enable --now lembas
|
||||
sleep 2
|
||||
sudo systemctl --no-pager --lines=0 status lembas || true
|
||||
|
||||
echo
|
||||
echo "LLeMbas is up at https://chat.lan (self-signed cert; accept the warning)"
|
||||
echo "Create the first account -- it becomes the administrator."
|
||||
@@ -0,0 +1,49 @@
|
||||
# LLeMbas system service.
|
||||
#
|
||||
# Deployed to /etc/systemd/system/lembas.service by deploy/install.sh.
|
||||
#
|
||||
# A system unit, not a user unit, so it survives logout and comes up at boot
|
||||
# without anyone signing in -- matching llama-swap and comfyui on this box.
|
||||
#
|
||||
# /srv/lembas is a bind mount of /home/lembas: the root LV is only 50 GB and
|
||||
# the venv plus SQLite database belong on /home, same trick as /srv/llama.
|
||||
|
||||
[Unit]
|
||||
Description=LLeMbas - web UI for language models
|
||||
Documentation=https://git.houmeres.sk/Houmeres/LLeMbas
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
# The unit is useless without the bind mount: the venv and database live there.
|
||||
RequiresMountsFor=/srv/lembas
|
||||
# Not a hard dependency. LLeMbas starts fine with the endpoint down and shows a
|
||||
# readable error in the admin UI, which is better than refusing to boot.
|
||||
After=llama-swap.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=lembas
|
||||
Group=lembas
|
||||
WorkingDirectory=/srv/lembas/app
|
||||
EnvironmentFile=/srv/lembas/lembas.env
|
||||
ExecStart=/srv/lembas/venv/bin/lembas serve
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
# Reachable only through the nginx chat.lan vhost, never directly on the LAN.
|
||||
# The bind address is set by LEMBAS_HOST in the environment file.
|
||||
|
||||
# --- Hardening -------------------------------------------------------------
|
||||
# Modest rather than maximal: the agentic features planned for later will need
|
||||
# to run commands, so ProtectSystem=strict would only be torn out again.
|
||||
NoNewPrivileges=yes
|
||||
PrivateTmp=yes
|
||||
ProtectSystem=full
|
||||
ProtectKernelTunables=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
# Only /srv/lembas needs to be writable; /home/lembas is the same inode.
|
||||
ReadWritePaths=/srv/lembas
|
||||
LimitNOFILE=65535
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env bash
|
||||
# Pull the latest LLeMbas and restart the service.
|
||||
#
|
||||
# This is what to run after pushing: it fetches, hard-resets the deployment
|
||||
# checkout to the remote branch, reinstalls dependencies if they changed, and
|
||||
# restarts. Nothing is ever edited in place at /srv/lembas/app, so a hard reset
|
||||
# is safe and avoids merge conflicts from a dirty deployment tree.
|
||||
set -euo pipefail
|
||||
|
||||
SERVICE_USER=lembas
|
||||
PREFIX=/srv/lembas
|
||||
APP="$PREFIX/app"
|
||||
VENV="$PREFIX/venv"
|
||||
BRANCH="${LEMBAS_BRANCH:-main}"
|
||||
|
||||
if [[ ! -d "$APP/.git" ]]; then
|
||||
echo "No deployment at $APP. Run deploy/install.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
before=$(sudo -u "$SERVICE_USER" git -C "$APP" rev-parse HEAD)
|
||||
|
||||
echo "== fetching =="
|
||||
sudo -u "$SERVICE_USER" git -C "$APP" fetch --quiet origin "$BRANCH"
|
||||
sudo -u "$SERVICE_USER" git -C "$APP" reset --hard --quiet "origin/$BRANCH"
|
||||
|
||||
after=$(sudo -u "$SERVICE_USER" git -C "$APP" rev-parse HEAD)
|
||||
|
||||
if [[ "$before" == "$after" ]]; then
|
||||
echo " already at $(git -C "$APP" rev-parse --short HEAD), nothing to pull"
|
||||
else
|
||||
echo " $(echo "$before" | cut -c1-7) -> $(echo "$after" | cut -c1-7)"
|
||||
sudo -u "$SERVICE_USER" git -C "$APP" --no-pager log --oneline "$before..$after" | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
# Cheap and idempotent; catches a dependency added since the last deploy.
|
||||
echo "== dependencies =="
|
||||
sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet -e "$APP"
|
||||
|
||||
echo "== restart =="
|
||||
sudo systemctl restart lembas
|
||||
sleep 2
|
||||
|
||||
if systemctl is-active --quiet lembas; then
|
||||
echo " lembas is running at https://chat.lan"
|
||||
else
|
||||
echo " lembas FAILED to start:" >&2
|
||||
sudo journalctl -u lembas -n 30 --no-pager >&2
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user