An update you can ask for, and a boundary that stays where it was

The button cannot do the work, and that is the whole design. The service runs as
an unprivileged account, cannot restart itself, and should not be able to: a web
application that can restart its own service is one whose worst day is much
worse. So /admin/updates writes a file, and an opt-in systemd .path unit runs
deploy/update.sh as root.

Three properties hold it up, and each is a thing that could have been got wrong.
The request file carries nothing that reaches a command line -- no branch, no
ref, no arguments -- because the branch is baked into the unit at install time,
so pressing the button is always "deploy the branch this host was configured
with" and can never be "deploy something else". It is off unless somebody passes
INSTALL_UPDATE_HELPER=1, and re-running the installer without it removes both
units and the marker. And without the helper the page says so and prints the
manual command rather than writing a file nothing is watching, which would be a
button that reports success and does nothing.

The card that says all of this is rendered whether or not there is anything to
apply. It was inside the "there is an update" branch first, so an administrator
could not discover the helper was missing until the day they needed it, which is
the worst possible moment.

Opening the page makes no network request; Check is the one thing that fetches.
And it shows the log between, not a count: "3 behind" is a number somebody has to
go and look up, while the subjects are what decides whether this is worth
restarting for right now.

Docker is one stage, because there is nothing to build -- no Node, no compiled
assets. It bakes no secret key (one in an image is one every copy shares, and
rotating it makes stored API keys unreadable), no data, and no .git, so
/admin/updates inside a container correctly reports that it was not installed
from a checkout. Compose publishes on loopback and refuses to start without a
key. TLS in front is a constraint rather than a recommendation: the service
worker and the microphone both require HTTPS or localhost.

The image was built and run before this was committed, which is how the missing
COPY of LICENSE was found -- pyproject declares it and the build backend reads
it, so the failure reads like a packaging problem and is one line.

deploy/lxc-install.sh creates an unprivileged Debian container and runs the
existing installer inside it. A wrapper, not a second install path: a parallel
installer is two things to keep correct and one of them rots.

/healthz opens the database rather than only proving the socket is listening -- a
process that is up with a database it cannot open answers every page with a 500
-- and says nothing about what is here, being reachable without signing in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/__init__.py

@@ -1,3 +1,3 @@
 """LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints."""
 
-__version__ = "0.9.6"
+__version__ = "0.9.7"

src/lembas/api/admin_updates.py

@@ -0,0 +1,80 @@
+"""What is running here, and getting to what is not.
+
+Read `services/updates.py` first — the reason the button writes a file rather
+than doing the work is there, and it is the whole design.
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Request, Response, status
+from fastapi.responses import RedirectResponse
+
+from lembas.api.deps import AdminUser, Db
+from lembas.services import updates as updates_service
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/updates", tags=["admin-updates"])
+
+
+def _page(request: Request, state, saved: str = "") -> Response:
+    return render(
+        request,
+        "admin/updates.html",
+        {
+            "state": state,
+            "command": updates_service.manual_command(),
+            "saved": saved,
+        },
+    )
+
+
+@router.get("")
+async def updates_page(request: Request, db: Db, user: AdminUser, saved: str = ""):
+    """No network on a page load.
+
+    `read(fetch=False)` compares against whatever the last fetch left behind, so
+    opening this is a few git reads off the local disk. A page that reached the
+    remote every time it was rendered would be one somebody stops opening.
+    """
+    return _page(request, updates_service.read(), saved)
+
+
+@router.post("/check")
+async def check(request: Request, db: Db, user: AdminUser) -> Response:
+    """Ask the remote what is there. The one place this touches the network."""
+    state = updates_service.read(fetch=True)
+    log.info("%s checked for updates", user.email)
+    return _page(request, state)
+
+
+@router.post("/apply")
+async def apply(db: Db, user: AdminUser) -> Response:
+    """Write the request the helper is watching for.
+
+    Refused when the helper is not installed rather than written and left to sit
+    there: a file nothing is watching is a button that reports success and does
+    nothing, which is the failure this codebase keeps cataloguing.
+    """
+    if not updates_service.helper_installed():
+        return RedirectResponse(
+            "/admin/updates?saved=The+update+helper+is+not+installed+on+this+host.",
+            status_code=status.HTTP_303_SEE_OTHER,
+        )
+    problem = updates_service.request_update(user.email)
+    message = problem or "Update requested. The service will restart in a moment."
+    return RedirectResponse(
+        f"/admin/updates?saved={message.replace(' ', '+')}",
+        status_code=status.HTTP_303_SEE_OTHER,
+    )
+
+
+@router.post("/cancel")
+async def cancel(db: Db, user: AdminUser) -> Response:
+    updates_service.clear_request()
+    return RedirectResponse(
+        "/admin/updates?saved=Request+withdrawn.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/pages.py

@@ -435,6 +435,32 @@ async def home(user: RequiredUser):
 # has by definition no server to ask who is looking at it.
 
 
+@router.get("/healthz", include_in_schema=False)
+async def healthz() -> Response:
+    """Is the process up and can it reach its database.
+
+    Unauthenticated, like the three below, and for a fourth reason: a
+    healthcheck that needed a session would be a healthcheck nothing could run.
+    It says nothing about *what* is here -- no version, no counts -- because it
+    is reachable without signing in and a health endpoint is a common place to
+    leak the first fact an attacker wants.
+
+    The query is what makes it worth having. A process that is up with a
+    database it cannot open answers every page with a 500, and a check that only
+    proved the socket was listening would call that healthy.
+    """
+    from sqlalchemy import text
+
+    from lembas.db.session import session_scope
+
+    try:
+        with session_scope() as db:
+            db.execute(text("SELECT 1"))
+    except Exception:  # noqa: BLE001 - the answer is the status code
+        return JSONResponse({"status": "error"}, status_code=503)
+    return JSONResponse({"status": "ok"})
+
+
 @router.get("/manifest.webmanifest", include_in_schema=False)
 async def manifest(db: Db) -> Response:
     """The web app manifest.

src/lembas/config.py

@@ -38,6 +38,13 @@ class Settings(BaseSettings):
     session_ttl: int = 60 * 60 * 24 * 30
     request_timeout: float = 300.0
 
+    # Which branch `/admin/updates` compares against and the helper deploys.
+    # Deployment configuration and deliberately not an instance setting: it
+    # decides what code runs on this machine, and a value a web administrator
+    # could edit would turn "you may deploy the branch" into "you may deploy
+    # anything". `deploy/install.sh` writes it beside the rest.
+    update_branch: str = "main"
+
     @model_validator(mode="after")
     def _generate_secret_if_absent(self) -> Settings:
         # A generated key lets `lembas serve` work with no configuration at all,

src/lembas/main.py

@@ -25,6 +25,7 @@ from lembas.api import (
     admin_search,
     admin_suggestions,
     admin_tools,
+    admin_updates,
     admin_users,
     agents,
     audio,
@@ -198,6 +199,7 @@ def create_app() -> FastAPI:
     app.include_router(sharing.router)
     app.include_router(admin.router)
     app.include_router(admin_users.router)
+    app.include_router(admin_updates.router)
     app.include_router(admin_models.router)
     app.include_router(admin_audio.router)
     app.include_router(admin_branding.router)

src/lembas/services/updates.py

@@ -0,0 +1,273 @@
+"""What is running here, what is available, and how to get from one to the other.
+
+## Why this exists
+
+Updating meant a shell on the box. That is fine for whoever built it and wrong
+for everybody else: an instance somebody installed from `deploy/install.sh` and
+handed over has an administrator who can configure every part of it and cannot
+tell whether it is three releases behind.
+
+So this answers three questions — what is running, what is on the branch, and
+what changed between — and offers a button for the fourth.
+
+## The button cannot do the work, and that is the design
+
+The service runs as an unprivileged account. It cannot restart itself, it cannot
+write outside its own prefix, and it should not be able to: a web application
+that can restart its own service is one whose worst day is much worse.
+
+So the button **writes a file**, and an opt-in systemd `.path` unit watching that
+file runs `deploy/update.sh` as root. Three things follow, and all three are
+deliberate:
+
+- **The request file carries nothing.** No branch, no ref, no arguments. The
+  helper runs the update script with the branch *it* was configured with, so an
+  administrator of the web UI cannot choose what gets deployed — only *that* the
+  configured branch gets deployed.
+- **It is opt-in.** Without the helper the page says so and shows the command,
+  which is the honest degradation and the same shape the SSH and search extras
+  already have. Installing it is a decision with a consequence worth stating:
+  anybody who can administer the web UI can then deploy whatever is on that
+  branch.
+- **Nothing here runs a shell.** Every subprocess is a fixed argv with
+  `shell=False`, and the only variable in any of them is the branch, which comes
+  from the environment rather than from a request.
+
+## On "nothing executes on this machine"
+
+That rule is about *agent chats* — a model's commands, on a host somebody chose.
+This is an administrator pressing a button to run `git`, with no model anywhere
+near it and no input from a request in the argv. Worth stating rather than
+leaving somebody to notice the tension.
+"""
+
+from __future__ import annotations
+
+import logging
+import subprocess
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from lembas import __version__
+from lembas.config import settings
+
+log = logging.getLogger(__name__)
+
+# Long enough for a fetch over a slow link, short enough that a page does not
+# appear to hang. A fetch that takes longer than this is a network problem, and
+# saying so is more use than waiting.
+GIT_TIMEOUT = 30.0
+
+# The file the button writes. Inside the data directory, which is the one place
+# the service account can write, and named so that finding it in a backup says
+# what it was.
+REQUEST_NAME = "update-requested"
+
+# Written by `install.sh` when the helper is installed. A marker rather than
+# asking systemd, because `systemctl is-enabled` means a subprocess on every
+# page render to answer a question that changes once.
+MARKER_NAME = ".update-helper"
+
+
+@dataclass(frozen=True)
+class Commit:
+    sha: str
+    subject: str
+    when: str = ""
+
+    @property
+    def short(self) -> str:
+        return self.sha[:7]
+
+
+@dataclass(frozen=True)
+class State:
+    """Everything the page shows, resolved in one go."""
+
+    version: str = __version__
+    checkout: str = ""
+    branch: str = "main"
+    head: Commit | None = None
+    remote: Commit | None = None
+    behind: list[Commit] = field(default_factory=list)
+    dirty: bool = False
+    helper: bool = False
+    requested: bool = False
+    error: str = ""
+
+    @property
+    def is_git(self) -> bool:
+        return bool(self.checkout)
+
+    @property
+    def up_to_date(self) -> bool:
+        return bool(self.head and self.remote and self.head.sha == self.remote.sha)
+
+
+def checkout_dir() -> Path | None:
+    """The git checkout this is running from, or None.
+
+    `src/lembas/__init__.py` upwards twice is the repository root for an editable
+    install, which is what every deployment here is. A wheel installed into
+    site-packages has no `.git` above it and answers None, which the page reads
+    as "this was not installed from a checkout" rather than as an error.
+    """
+    root = Path(__file__).resolve().parents[3]
+    return root if (root / ".git").exists() else None
+
+
+def _git(args: list[str], *, cwd: Path, timeout: float = GIT_TIMEOUT) -> tuple[int, str]:
+    """One git command, with no shell anywhere near it.
+
+    `shell=False` and a fixed argv, so nothing here can be made to run something
+    else by what it is given. Returns the code and the output together because
+    every caller wants both and none of them wants an exception: git failing is
+    a thing to report on the page, not a 500.
+    """
+    try:
+        done = subprocess.run(  # noqa: S603 - fixed argv, no shell, admin-only
+            ["git", *args],
+            cwd=str(cwd),
+            capture_output=True,
+            text=True,
+            timeout=timeout,
+            check=False,
+        )
+    except FileNotFoundError:
+        return 127, "git is not installed on this machine."
+    except subprocess.TimeoutExpired:
+        return 124, f"git took longer than {int(timeout)}s."
+    except Exception as exc:  # noqa: BLE001 - a broken git is a message, not a crash
+        log.debug("git %s failed", args, exc_info=True)
+        return 1, str(exc)
+    return done.returncode, (done.stdout or done.stderr or "").strip()
+
+
+def _commit(line: str) -> Commit | None:
+    """One line of `--format=%H\\x1f%s\\x1f%cI`.
+
+    Split on a unit separator rather than a space: a commit subject contains
+    spaces, and every other separator anybody reaches for is one a subject may
+    also contain.
+    """
+    parts = line.split("\x1f")
+    if len(parts) < 2 or not parts[0]:
+        return None
+    return Commit(sha=parts[0], subject=parts[1], when=parts[2] if len(parts) > 2 else "")
+
+
+FORMAT = "--format=%H\x1f%s\x1f%cI"
+
+
+def branch_name() -> str:
+    return settings.update_branch or "main"
+
+
+def helper_installed() -> bool:
+    return (settings.data_dir / MARKER_NAME).exists()
+
+
+def request_path() -> Path:
+    return settings.data_dir / REQUEST_NAME
+
+
+def pending() -> bool:
+    return request_path().exists()
+
+
+def read(*, fetch: bool = False) -> State:
+    """What is running and what is available.
+
+    `fetch` is the network half and is off by default: this is called to render a
+    page, and a page that made a network request on every load would be one
+    somebody stops opening. The Check button passes it.
+    """
+    branch = branch_name()
+    root = checkout_dir()
+    base = State(branch=branch, helper=helper_installed(), requested=pending())
+    if root is None:
+        return base
+
+    code, head_line = _git(["log", "-1", FORMAT], cwd=root)
+    if code != 0:
+        return State(**{**base.__dict__, "checkout": str(root), "error": head_line})
+
+    code, status = _git(["status", "--porcelain"], cwd=root)
+    dirty = bool(status.strip()) if code == 0 else False
+
+    error = ""
+    if fetch:
+        code, output = _git(["fetch", "--quiet", "origin", branch], cwd=root)
+        if code != 0:
+            error = f"Could not reach the remote: {output}"
+
+    remote_line = ""
+    code, output = _git(["log", "-1", FORMAT, f"origin/{branch}"], cwd=root)
+    if code == 0:
+        remote_line = output
+
+    behind: list[Commit] = []
+    if remote_line:
+        code, output = _git(["log", FORMAT, f"HEAD..origin/{branch}"], cwd=root)
+        if code == 0 and output:
+            behind = [c for line in output.splitlines() if (c := _commit(line))]
+
+    return State(
+        checkout=str(root),
+        branch=branch,
+        head=_commit(head_line),
+        remote=_commit(remote_line) if remote_line else None,
+        behind=behind,
+        dirty=dirty,
+        helper=helper_installed(),
+        requested=pending(),
+        error=error,
+    )
+
+
+def request_update(who: str = "") -> str:
+    """Ask the helper to update. Returns a message fit to show.
+
+    The file is written with the requester's email in it **for the log**, and
+    that is all it carries. Nothing in it reaches the helper's command line: the
+    helper runs the update script with its own configured branch, so being able
+    to press this button is not being able to choose what is deployed.
+    """
+    path = request_path()
+    try:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_text(f"requested by {who or 'an administrator'}\n", encoding="utf-8")
+    except OSError as exc:
+        log.warning("could not write the update request: %s", exc)
+        return f"Could not write {path}: {exc}"
+    log.info("update requested by %s", who or "an administrator")
+    return ""
+
+
+def clear_request() -> None:
+    """Forget a request. For the page's Cancel, and for tests."""
+    request_path().unlink(missing_ok=True)
+
+
+def manual_command() -> str:
+    """What to run by hand when the helper is not installed."""
+    root = checkout_dir()
+    return f"sudo {root}/deploy/update.sh" if root else "sudo ./deploy/update.sh"
+
+
+__all__ = [
+    "GIT_TIMEOUT",
+    "MARKER_NAME",
+    "REQUEST_NAME",
+    "Commit",
+    "State",
+    "branch_name",
+    "checkout_dir",
+    "clear_request",
+    "helper_installed",
+    "manual_command",
+    "pending",
+    "read",
+    "request_path",
+    "request_update",
+]

src/lembas/web/templates/admin/_layout.html

@@ -86,6 +86,10 @@
           {{ icon("user", "icon--sm") }}
           <span class="nav-item__label">Users</span>
         </a>
+        <a class="nav-item {{ 'is-active' if section == 'updates' }}" href="/admin/updates">
+          {{ icon("refresh", "icon--sm") }}
+          <span class="nav-item__label">Updates</span>
+        </a>
         <a class="nav-item {{ 'is-active' if section == 'groups' }}" href="/admin/groups">
           {{ icon("users", "icon--sm") }}
           <span class="nav-item__label">Groups &amp; permissions</span>

src/lembas/web/templates/admin/updates.html

@@ -0,0 +1,175 @@
+{% extends "admin/_layout.html" %}
+{% from "_macros.html" import icon %}
+{% set section = "updates" %}
+
+{% block title %}Updates - {{ brand.name }}{% endblock %}
+{% block heading %}Updates{% endblock %}
+
+{% block admin_content %}
+<p class="admin-lede">
+  What is running here, what is on <code>{{ state.branch }}</code>, and what
+  changed between. Checking reaches the remote; opening this page does not.
+</p>
+
+{% if saved %}
+<div class="alert alert--success">{{ icon("check", "icon--sm") }} <span>{{ saved }}</span></div>
+{% endif %}
+{% if state.error %}
+<div class="alert alert--error">{{ icon("warning", "icon--sm") }} <span>{{ state.error }}</span></div>
+{% endif %}
+
+<section class="card">
+  <h2 class="card__title">Running</h2>
+  <dl class="mode-list">
+    <div class="mode-list__row">
+      <dt><strong>Version</strong></dt>
+      <dd>{{ state.version }}</dd>
+    </div>
+    {% if state.head %}
+    <div class="mode-list__row">
+      <dt><strong>Commit</strong></dt>
+      <dd>
+        <code>{{ state.head.short }}</code> {{ state.head.subject }}
+        {% if state.dirty %}
+        <span class="badge">uncommitted changes</span>
+        {% endif %}
+      </dd>
+    </div>
+    {% endif %}
+    {% if state.checkout %}
+    <div class="mode-list__row">
+      <dt><strong>Checkout</strong></dt>
+      <dd><code>{{ state.checkout }}</code></dd>
+    </div>
+    {% endif %}
+  </dl>
+
+  {% if not state.is_git %}
+  <p class="field__hint">
+    This was not installed from a git checkout — a container image, or a wheel —
+    so there is nothing here to compare or update. Update the image instead.
+  </p>
+  {% else %}
+  <form method="post" action="/admin/updates/check" class="btn-row">
+    <button class="btn" type="submit">
+      {{ icon("globe", "icon--sm") }} Check the remote
+    </button>
+  </form>
+  {% endif %}
+</section>
+
+{% if state.is_git %}
+<section class="card">
+  <h2 class="card__title">
+    Available
+    {% if state.up_to_date %}<span class="badge badge--leaf">up to date</span>
+    {% elif state.behind %}<span class="badge">{{ state.behind | length }} behind</span>
+    {% endif %}
+  </h2>
+
+  {% if not state.remote %}
+  <p class="field__hint">
+    Nothing known about <code>origin/{{ state.branch }}</code> yet. Check the
+    remote above.
+  </p>
+  {% elif state.up_to_date %}
+  <p class="field__hint">
+    <code>{{ state.remote.short }}</code> is what is running.
+  </p>
+  {% else %}
+  <p class="card__lede">
+    <code>{{ state.remote.short }}</code> {{ state.remote.subject }}
+  </p>
+
+  {#
+    The log between, not just a count. "3 behind" is a number somebody has to go
+    and look up; the subjects are the thing that decides whether this is worth
+    restarting for right now.
+  #}
+  <ul class="model-list">
+    {% for commit in state.behind %}
+    <li class="model-list__item">
+      <div style="min-width: 0">
+        <code class="text-xs">{{ commit.short }}</code> {{ commit.subject }}
+      </div>
+    </li>
+    {% endfor %}
+  </ul>
+
+  {% if state.dirty %}
+  <div class="alert alert--warning">
+    {{ icon("warning", "icon--sm") }}
+    <span>
+      This checkout has uncommitted changes. The update does a hard reset, which
+      would throw them away — nothing here is meant to be edited in place, so
+      this usually means somebody was debugging on the box.
+    </span>
+  </div>
+  {% endif %}
+
+  {% endif %}
+</section>
+
+{#
+  Its own card, and rendered whether or not there is anything to apply. It used
+  to live inside the "there is an update" branch, so an administrator could not
+  find out that the helper was missing until the day they needed it -- which is
+  the worst moment to discover a thing has to be installed from a shell.
+#}
+<section class="card">
+  <h2 class="card__title">Applying an update</h2>
+
+  {% if state.requested %}
+  <div class="alert">
+    {{ icon("clock", "icon--sm") }}
+    <span>
+      An update has been requested and is waiting for the helper to pick it up.
+      The service restarts when it does.
+    </span>
+  </div>
+  <form method="post" action="/admin/updates/cancel" class="btn-row">
+    <button class="btn btn--sm" type="submit">Withdraw the request</button>
+  </form>
+
+  {% elif state.helper %}
+  <p class="card__lede">
+    This host has the update helper installed, so the button below writes a
+    request that a systemd unit picks up and runs as root. It always deploys
+    <code>{{ state.branch }}</code> — the request carries no branch and no ref,
+    so pressing it is never “deploy something else”.
+  </p>
+  <form method="post" action="/admin/updates/apply" class="btn-row"
+        data-confirm="Update and restart? Replies being written are saved; open terminals are cut off.">
+    <button class="btn btn--primary" type="submit" {{ 'disabled' if state.up_to_date }}>
+      {{ icon("sparkle", "icon--sm") }}
+      {{ "Nothing to apply" if state.up_to_date else "Update and restart" }}
+    </button>
+  </form>
+  <p class="field__hint">
+    Every restart ends every open terminal session — a command still running on
+    the far side is cut off. A reply being written is saved with whatever it has.
+  </p>
+
+  {% else %}
+  <div class="alert">
+    {{ icon("shield", "icon--sm") }}
+    <span>
+      The update helper is <strong>not installed on this host</strong>, so there
+      is no button. That is the honest default: the service runs as an
+      unprivileged account and cannot restart itself, and a web application that
+      <em>can</em> is one whose worst day is much worse. Run this instead:
+      <code>{{ command }}</code>
+    </span>
+  </div>
+  <p class="field__hint">
+    To install it, re-run the installer with
+    <code>INSTALL_UPDATE_HELPER=1</code>. Doing so means anybody who can
+    administer this web interface can deploy whatever is on
+    <code>{{ state.branch }}</code> — which is the point, and worth deciding on
+    purpose rather than arriving at.
+  </p>
+  {% endif %}
+</section>
+{% endif %}
+{% endblock %}
+