The button cannot do the work, and that is the whole design. The service runs as an unprivileged account, cannot restart itself, and should not be able to: a web application that can restart its own service is one whose worst day is much worse. So /admin/updates writes a file, and an opt-in systemd .path unit runs deploy/update.sh as root. Three properties hold it up, and each is a thing that could have been got wrong. The request file carries nothing that reaches a command line -- no branch, no ref, no arguments -- because the branch is baked into the unit at install time, so pressing the button is always "deploy the branch this host was configured with" and can never be "deploy something else". It is off unless somebody passes INSTALL_UPDATE_HELPER=1, and re-running the installer without it removes both units and the marker. And without the helper the page says so and prints the manual command rather than writing a file nothing is watching, which would be a button that reports success and does nothing. The card that says all of this is rendered whether or not there is anything to apply. It was inside the "there is an update" branch first, so an administrator could not discover the helper was missing until the day they needed it, which is the worst possible moment. Opening the page makes no network request; Check is the one thing that fetches. And it shows the log between, not a count: "3 behind" is a number somebody has to go and look up, while the subjects are what decides whether this is worth restarting for right now. Docker is one stage, because there is nothing to build -- no Node, no compiled assets. It bakes no secret key (one in an image is one every copy shares, and rotating it makes stored API keys unreadable), no data, and no .git, so /admin/updates inside a container correctly reports that it was not installed from a checkout. Compose publishes on loopback and refuses to start without a key. TLS in front is a constraint rather than a recommendation: the service worker and the microphone both require HTTPS or localhost. The image was built and run before this was committed, which is how the missing COPY of LICENSE was found -- pyproject declares it and the build backend reads it, so the failure reads like a packaging problem and is one line. deploy/lxc-install.sh creates an unprivileged Debian container and runs the existing installer inside it. A wrapper, not a second install path: a parallel installer is two things to keep correct and one of them rots. /healthz opens the database rather than only proving the socket is listening -- a process that is up with a database it cannot open answers every page with a 500 -- and says nothing about what is here, being reachable without signing in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
9.1 KiB
Deployment
Installs LLeMbas as a system service behind nginx with a self-signed certificate. Written for a systemd + nginx host; tested on Arch.
| Default | |
|---|---|
| Service user | lembas (system account, nologin) |
| Home | /home/lembas |
| Install prefix | /srv/lembas (bind mount of the home) |
| Checkout | $PREFIX/app |
| Virtualenv | $PREFIX/venv |
| Database | $PREFIX/data/lembas.db |
| Environment | $PREFIX/lembas.env (mode 600) |
| Unit | /etc/systemd/system/lembas.service |
| Vhost | /etc/nginx/conf.d/<host>.conf |
| Listens on | 127.0.0.1:8080 — reachable only through nginx |
The prefix defaults to a bind mount of the service user's home because on many
machines the root filesystem is small while /home is not, and the virtualenv
plus database belong on the larger volume. Set PREFIX=$HOME_DIR to skip it.
First install
SITE_HOST=chat.example ./deploy/install.sh
Idempotent — safe to re-run. It creates the user and bind mount, clones the
repo, builds the venv, generates lembas.env with a fresh LEMBAS_SECRET_KEY,
installs the unit and vhost, issues a self-signed certificate, adds a
/etc/hosts entry if the name does not already resolve, and enables the
service.
Then open https://<SITE_HOST>, accept the certificate warning, and create the
first account — it becomes the administrator.
Everything is overridable from the environment:
| Variable | Default | |
|---|---|---|
SITE_HOST |
lembas.local |
nginx server_name and certificate CN |
APP_PORT |
8080 |
loopback port the service binds |
SERVICE_USER |
lembas |
system account to run as |
HOME_DIR |
/home/lembas |
that account's home |
PREFIX |
/srv/lembas |
install root (bind mount of HOME_DIR) |
REPO_URL |
this checkout's origin |
so a fork deploys itself |
LEMBAS_BRANCH |
main |
branch to deploy, and what /admin/updates compares against |
INSTALL_UPDATE_HELPER |
0 |
1 lets the web interface deploy that branch as root |
Updating from the web interface
/admin/updates says what is running, what is on the branch and what changed
between. Checking reaches the remote; opening the page does not.
The button is opt-in, and the reason is a boundary rather than caution:
INSTALL_UPDATE_HELPER=1 SITE_HOST=chat.example ./deploy/install.sh
That installs lembas-update.path and lembas-update.service. The web
interface writes $PREFIX/data/update-requested; the path unit notices and the
service runs update.sh as root.
What that grants. Anybody who can administer this web interface can then deploy whatever is on the configured branch and restart the service. That is the point of it, and it is why it is not the default.
What it deliberately does not grant. The request file carries nothing that
reaches a command line — no branch, no ref, no arguments. The branch is baked
into the unit at install time from LEMBAS_BRANCH, so the button is always
"deploy the branch this host was configured with" and never "deploy something
else". Re-running the installer without the flag removes both units and the
marker, and the page goes back to printing the manual command.
Without the helper the page says so and shows sudo …/deploy/update.sh, which is
the same honest degradation the SSH and search extras have.
Deploying a change
git push
./deploy/update.sh
update.sh fetches, hard-resets the deployment checkout to origin/main,
reinstalls dependencies and restarts, printing the commits it pulled. The hard
reset is deliberate: nothing is ever edited in place there, so there is no local
work to preserve and no conflicts to resolve.
In a container
A Dockerfile and a docker-compose.yml are in the repository root.
echo "LEMBAS_SECRET_KEY=$(python -c 'import secrets;print(secrets.token_urlsafe(48))')" > .env
docker compose up -d
It publishes on 127.0.0.1:8080 and expects a TLS reverse proxy in front.
That is a constraint, not a preference: a service worker and a microphone both
require HTTPS or localhost, so over plain http on a LAN address the app cannot be
installed and cannot dictate — and the session cookie is deliberately not marked
secure, so an attacker on that network could steal a session.
Three things about the image:
- No secret key is baked in, and compose refuses to start without one. A key in an image is a key every copy of that image shares, and rotating it signs everybody out and makes stored upstream API keys unreadable.
.gitis excluded, so/admin/updatesinside a container says it was not installed from a checkout and offers nothing. That is correct: a container is updated by pulling a new image.- One replica. The generation registry, the stop mechanism, the terminal sessions and the schedule ticker are all in-process — two would mean two tickers and every schedule firing twice.
On Proxmox
CTID=140 SITE_HOST=chat.example ./deploy/lxc-install.sh
Run on the Proxmox host. It creates an unprivileged Debian container,
installs the dependencies, and runs deploy/install.sh inside it — the same
installer, so a fix there reaches this without anybody remembering. Unprivileged
is not a default to change: nothing LLeMbas does needs privilege, because agent
chats run their commands over SSH on some other machine.
Operating it
systemctl status lembas
journalctl -u lembas -f
sudo -u lembas /srv/lembas/venv/bin/lembas info # paths and counts
Configuration lives in $PREFIX/lembas.env. Edit it and restart.
Notes
The secret key is generated once. install.sh will not overwrite an
existing lembas.env. Rotating LEMBAS_SECRET_KEY signs every user out and
makes stored upstream API keys unreadable — they would have to be re-entered.
nginx buffering is off for a reason. Replies stream as server-sent events.
With proxy_buffering on (the default) nginx holds the entire reply and
delivers it in one lump at the end, which is indistinguishable from streaming
being broken. proxy_read_timeout is raised to an hour because a model can
think for minutes before the first token.
The vhost passes WebSocket upgrades through, and must. The terminal panel
is the one WebSocket in LLeMbas. A location that sets Connection "" — which
is what SSE alone needs, and what this template used to say — fails every
handshake, and a failed handshake tells the browser nothing: no status, no
reason. The map $http_upgrade at the top of the vhost yields the empty string
when the client did not ask to upgrade, so streaming is unaffected. update.sh
warns when the installed vhost has drifted from the template, because this is
the failure most likely to be diagnosed as a bug in the application.
Every restart kills every open shell. A reply being written is persisted
with whatever it has; a terminal has nothing to persist, so a command still
running on the far side is cut off. update.sh restarts unconditionally, so a
deploy in the middle of somebody's apt-get dist-upgrade ends it. The panel is
told why rather than silently reconnecting to a new shell, which would have
lost the working directory and the half-typed command.
A terminal is not in the transcript, and is not logged. The open and the close are logged with the user, the chat and the connection; what was typed is not recorded anywhere. That follows from the design — the chat's mode governs the model, not the person at the keyboard — but everything else an agent chat does is in the transcript, so it is a difference in kind and worth knowing before somebody goes looking for the history.
Nothing an agent does runs on this machine. Agent chats execute their commands over SSH, on a host somebody added and prepared — a container, a VM, another machine. That is the whole isolation story, and it is why the unit can stay locked down instead of being opened up to make room for a sandbox.
ProtectSystem=full rather than strict only because the data directory must
be writable and strict would mean listing every path.
The practical consequence for whoever runs this: the security of an agent chat is the security of the host behind its SSH profile. A throwaway container with the one project mounted into it is a very different thing from a key to a production server, and LLeMbas cannot tell them apart.
Use a real certificate if this is exposed beyond a trusted LAN. The
self-signed cert exists so the install works with no external dependencies;
point ssl_certificate at a real one and nothing else needs to change.
The session cookie is deliberately not marked secure, so that a LAN install
over plain http can sign anybody in at all. That has always meant a network
attacker on http could steal a session; with the terminal it also means they
could open an interactive shell on the machine behind that chat. If the
terminal is switched on, run this over TLS.
One worker only. True of generations already — the registry is in-process — and sharper here: with two workers a browser reconnecting to its terminal could land in the process that has no shell for it, and silently open a second one on the same machine.