News that finds you, including when nothing of ours is open

The dots covered Reports and Messages from the day those sections existed. The
announcement did not: only a chat reply produced an HX-Trigger, so a scheduled
run that filed a report or posted into Messages lit a green dot in a corner and
said nothing at all. That is precisely the arrival nobody is watching for -- a
chat reply is one you asked for a moment ago and are probably looking at.

So every kind announces, each with its own once-only flag, and the payload is a
list of items rather than of titles, because a notification is a thing you click
and a title cannot say where.

One arrival, three channels, and they must not all fire. A toast for somebody
looking at the page; a count in the tab title while it is hidden, cleared on
focus; a system notification for somebody elsewhere entirely. The service worker
is the only place that can tell them apart -- the server cannot see whether a
window is focused and the page cannot see a push it did not receive -- so it
stays quiet when one of its own windows has focus.

And web push, hand-rolled against RFC 8291 and RFC 8292 with the cryptography
already here for Fernet. It exists because everything else is polled by an open
page, and the arrival worth interrupting somebody for is a schedule firing at
seven in the morning with the laptop shut.

The trade is real and is written down rather than glossed: the POST goes to
Google's or Mozilla's push service, the payload is sealed end to end so they
cannot read it, and what they do learn is that this server sent something and
when. Opt-in per device, off until asked for, and the rest of the system works
without it. Nothing else in LLeMbas contacts an outside service on its own.

The encryption is tested by decrypting it back with an independent
implementation of the specification's other half. There is no other way to know:
a push service accepts the POST and forwards bytes it cannot read, so a wrong
derivation is a notification that never appears, with a 201 in the log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/services/generation.py

@@ -34,6 +34,7 @@ from lembas.services import compaction as compaction_service
 from lembas.services import interaction, settings_store, tokens, tool_labels
 from lembas.services import metrics as metrics_service
 from lembas.services import prompts as prompts_service
+from lembas.services import push as push_service
 from lembas.services import tools as tools_service
 from lembas.services.agent import policy as agent_policy
 from lembas.services.agent import session as agent_session
@@ -2018,6 +2019,19 @@ def _persist(generation: Generation, title: str, elapsed: float) -> None:
             if generation.followers == 0 and not chat.temporary:
                 chat.unread = True
                 chat.unread_notified = False
+                # And out to any browser that asked to be told, which is the
+                # only channel that reaches somebody with nothing of ours open.
+                # Here rather than in the unread poll because the poll needs a
+                # page, and this is exactly the case where there is not one:
+                # `followers == 0` says so. Fire and forget -- the reply is
+                # finished and nothing about it should wait on a push service.
+                push_service.announce_later(
+                    chat.user_id,
+                    title=chat.title or "New reply",
+                    body="Your reply is ready.",
+                    url=f"/chat/{chat.id}",
+                    kind="chat",
+                )
 
             db.commit()
             log.debug(

src/lembas/services/push.py

@@ -0,0 +1,401 @@
+"""Web Push: a notification that arrives with nothing of ours running.
+
+Everything else here is polled. `/api/chats/unread` runs in an open page, which
+is enough for "a reply landed while you were on another chat" and is nothing at
+all for the case this exists for -- a schedule firing at 07:00 on a laptop whose
+browser is shut. There is no way to close that gap from inside a page, because
+there is no page.
+
+## The trade, stated plainly
+
+A push goes to the **browser vendor's** push service: Google's for Chrome,
+Mozilla's for Firefox, Apple's for Safari. The endpoint is chosen by the
+browser and there is no version of this feature that avoids it. That sits
+against "a self-hosted tool must not report page views to a third party", and
+the answer is not that the tension is imaginary:
+
+- The payload is encrypted end to end (RFC 8291) with a key derived from a
+  secret only the browser and this server hold, so the push service carries
+  bytes it cannot read.
+- What it *does* learn is that this server sent something to that subscription,
+  and when. On a personal instance that is a timing channel over your own
+  activity, and it is real.
+- So it is **opt-in per device**, off until somebody presses the button, and
+  the rest of the notification system works without it.
+
+Nothing else in LLeMbas contacts an outside service on its own.
+
+## Hand-rolled, and why
+
+`pywebpush` would do this in three lines and bring `http-ece` and `py-vapid`
+with it. The encryption below is one ECDH, two HKDFs and one AES-GCM seal, all
+from `cryptography`, which is already a dependency because API keys are
+Fernet-encrypted. That is the same call the MCP client makes: a hand-written
+client, so what actually goes on the wire is in this repository.
+
+## The two specifications
+
+**RFC 8291** is the payload: `aes128gcm`, one record, the salt and the server's
+ephemeral public key carried in the body's own header block.
+
+**RFC 8292** is the authorisation: a JWT signed with a P-256 key whose public
+half identifies this server. The keypair is generated once and kept in the
+settings table with the private half Fernet-encrypted. It must be **stable** --
+the public key is baked into every subscription a browser has made, so
+regenerating it silently invalidates all of them.
+
+## Failure is a subscription being dropped, not an error
+
+A push service answers 404 or 410 for a subscription that no longer exists --
+the browser was uninstalled, the site's data cleared, the permission revoked.
+That is the normal end of a subscription's life and not a fault, so those two
+delete the row. Everything else is logged and left, because a push service
+having a bad hour is not a reason to lose somebody's registration.
+"""
+
+from __future__ import annotations
+
+import base64
+import hmac
+import json
+import logging
+import os
+import time
+from dataclasses import dataclass
+from hashlib import sha256
+from typing import Any
+from urllib.parse import urlsplit
+
+import httpx
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.hazmat.primitives.asymmetric import utils as asym_utils
+from cryptography.hazmat.primitives.ciphers.aead import AESGCM
+from cryptography.hazmat.primitives.kdf.hkdf import HKDFExpand
+from sqlalchemy import select
+from sqlalchemy.orm import Session as DBSession
+
+from lembas.db.models import PushSubscription, User
+from lembas.services import settings_store
+from lembas.services.crypto import decrypt, encrypt
+
+log = logging.getLogger(__name__)
+
+# How long a push service should hold an undelivered message. Four hours: long
+# enough that a laptop opened after lunch still gets the morning's report, short
+# enough that nothing arrives claiming to be news when it is a day old.
+TTL_SECONDS = 4 * 3600
+
+# The JWT's life. Twelve hours is the maximum RFC 8292 allows, and a short one
+# buys nothing here -- it is minted per request.
+JWT_SECONDS = 12 * 3600
+
+# Bigger than any payload we send; the record size field must still be present
+# and must exceed the ciphertext.
+RECORD_SIZE = 4096
+
+# What a payload may carry. A push service will refuse a large body outright,
+# and there is nothing here worth more than a title and a line.
+MAX_PAYLOAD_BYTES = 3000
+
+SETTING_PRIVATE = "push_private_key"
+SETTING_PUBLIC = "push_public_key"
+
+
+# --- base64url, without padding, everywhere ------------------------------------
+def b64(raw: bytes) -> str:
+    return base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii")
+
+
+def unb64(text: str) -> bytes:
+    padded = text + "=" * (-len(text) % 4)
+    return base64.urlsafe_b64decode(padded.encode("ascii"))
+
+
+# --- The server's identity -----------------------------------------------------
+@dataclass(frozen=True)
+class Keys:
+    private: ec.EllipticCurvePrivateKey
+    public_b64: str
+
+
+def keys(db: DBSession) -> Keys:
+    """This instance's VAPID keypair, generated once and kept.
+
+    Generated on first use rather than by a setup step, because a feature that
+    needs somebody to run a command before it works is one that is off on every
+    instance that did not read the release notes. The private half is Fernet-
+    encrypted at rest, like every other secret here.
+
+    **Never regenerated.** The public key is inside every subscription a browser
+    holds, so a new one silently invalidates all of them -- notifications simply
+    stop, with nothing anywhere saying why.
+    """
+    stored = settings_store.get(db, SETTING_PRIVATE)
+    if stored:
+        private = serialization.load_pem_private_key(decrypt(str(stored)).encode(), password=None)
+        return Keys(private=private, public_b64=str(settings_store.get(db, SETTING_PUBLIC) or ""))
+
+    private = ec.generate_private_key(ec.SECP256R1())
+    pem = private.private_bytes(
+        encoding=serialization.Encoding.PEM,
+        format=serialization.PrivateFormat.PKCS8,
+        encryption_algorithm=serialization.NoEncryption(),
+    ).decode()
+    public_b64 = b64(_raw_public(private.public_key()))
+    settings_store.update(db, {SETTING_PRIVATE: encrypt(pem), SETTING_PUBLIC: public_b64})
+    log.info("generated a VAPID keypair for web push")
+    return Keys(private=private, public_b64=public_b64)
+
+
+def public_key(db: DBSession) -> str:
+    """What a browser needs in order to subscribe."""
+    return keys(db).public_b64
+
+
+def _raw_public(key: ec.EllipticCurvePublicKey) -> bytes:
+    """The uncompressed 65-byte point, which is the only form either spec uses."""
+    return key.public_bytes(
+        encoding=serialization.Encoding.X962,
+        format=serialization.PublicFormat.UncompressedPoint,
+    )
+
+
+# --- RFC 8292: proving who is asking -------------------------------------------
+def _jwt(private: ec.EllipticCurvePrivateKey, audience: str, subject: str) -> str:
+    header = b64(json.dumps({"typ": "JWT", "alg": "ES256"}, separators=(",", ":")).encode())
+    claims = b64(
+        json.dumps(
+            {"aud": audience, "exp": int(time.time()) + JWT_SECONDS, "sub": subject},
+            separators=(",", ":"),
+        ).encode()
+    )
+    signing_input = f"{header}.{claims}".encode()
+
+    der = private.sign(signing_input, ec.ECDSA(hashes.SHA256()))
+    # JWS wants the raw pair, not DER. `cryptography` only signs to DER, so it
+    # is decoded and re-emitted fixed-width -- a leading zero dropped here is a
+    # signature every push service rejects, and the error it gives is 401.
+    r, s = asym_utils.decode_dss_signature(der)
+    raw = r.to_bytes(32, "big") + s.to_bytes(32, "big")
+    return f"{header}.{claims}.{b64(raw)}"
+
+
+def _audience(endpoint: str) -> str:
+    parts = urlsplit(endpoint)
+    return f"{parts.scheme}://{parts.netloc}"
+
+
+# --- RFC 8291: the payload ------------------------------------------------------
+def _hkdf(salt: bytes, ikm: bytes, info: bytes, length: int) -> bytes:
+    """Extract-then-expand, written out because the two halves take different
+    salts here and `HKDF` in one call cannot express that."""
+    prk = hmac.new(salt, ikm, sha256).digest()
+    return HKDFExpand(algorithm=hashes.SHA256(), length=length, info=info).derive(prk)
+
+
+def encrypt_payload(payload: bytes, *, p256dh: str, auth: str) -> bytes:
+    """One `aes128gcm` record, ready to be the body of the POST.
+
+    The layout is the specification's, and the order matters to a parser that
+    has never seen our code:
+
+        salt (16) | record size (4) | key id length (1) | server key (65) | ct
+    """
+    client_public = ec.EllipticCurvePublicKey.from_encoded_point(
+        ec.SECP256R1(), unb64(p256dh)
+    )
+    auth_secret = unb64(auth)
+
+    server_private = ec.generate_private_key(ec.SECP256R1())
+    server_public = _raw_public(server_private.public_key())
+    shared = server_private.exchange(ec.ECDH(), client_public)
+
+    # The first HKDF is salted with the subscription's own auth secret and its
+    # info binds both public keys, which is what stops a captured record being
+    # replayed at a different subscriber.
+    key_info = b"WebPush: info\x00" + unb64(p256dh) + server_public
+    ikm = _hkdf(auth_secret, shared, key_info, 32)
+
+    salt = os.urandom(16)
+    content_key = _hkdf(salt, ikm, b"Content-Encoding: aes128gcm\x00", 16)
+    nonce = _hkdf(salt, ikm, b"Content-Encoding: nonce\x00", 12)
+
+    # 0x02 is the padding delimiter for the last (here, only) record. 0x01 would
+    # say another follows, and a receiver would wait for it.
+    ciphertext = AESGCM(content_key).encrypt(nonce, payload + b"\x02", None)
+
+    return (
+        salt
+        + RECORD_SIZE.to_bytes(4, "big")
+        + len(server_public).to_bytes(1, "big")
+        + server_public
+        + ciphertext
+    )
+
+
+# --- Sending --------------------------------------------------------------------
+def subject_for(db: DBSession) -> str:
+    """The `sub` claim: who to contact about this server's pushes.
+
+    A URL is as acceptable as a mailto and needs nothing configured, so the
+    instance's own base URL is used when there is one. Push services require the
+    claim to be present; none of them checks that it resolves.
+    """
+    configured = str(settings_store.get(db, "public_url") or "").strip()
+    return configured or "https://lembas.invalid"
+
+
+async def send_one(db: DBSession, subscription: PushSubscription, payload: dict[str, Any]) -> bool:
+    """Deliver to one registration. True if it was accepted.
+
+    Never raises: this runs from arrival paths that must not fail because a push
+    service is having a bad hour.
+    """
+    body = json.dumps(payload, separators=(",", ":")).encode()
+    if len(body) > MAX_PAYLOAD_BYTES:  # pragma: no cover - titles are bounded already
+        body = json.dumps({"title": payload.get("title", "LLeMbas")}).encode()
+
+    try:
+        encrypted = encrypt_payload(
+            body, p256dh=subscription.p256dh, auth=subscription.auth_secret
+        )
+        token = _jwt(keys(db).private, _audience(subscription.endpoint), subject_for(db))
+    except Exception:  # noqa: BLE001 - a malformed stored key must not kill a reply
+        log.exception("could not build a push for %s", subscription.id)
+        return False
+
+    headers = {
+        "TTL": str(TTL_SECONDS),
+        "Content-Encoding": "aes128gcm",
+        "Content-Type": "application/octet-stream",
+        # "high" would let a phone wake for it; this is news, not an alarm.
+        "Urgency": "normal",
+        "Authorization": f"vapid t={token}, k={keys(db).public_b64}",
+    }
+
+    try:
+        async with httpx.AsyncClient(timeout=10.0) as client:
+            response = await client.post(subscription.endpoint, content=encrypted, headers=headers)
+    except httpx.RequestError as exc:
+        log.warning("push to %s failed: %s", _audience(subscription.endpoint), exc)
+        return False
+
+    if response.status_code in (404, 410):
+        # The normal end of a subscription's life: uninstalled, cleared, or the
+        # permission revoked. Deleting it is the correct response and not an
+        # error -- keeping it would mean retrying forever against a dead address.
+        log.info("push subscription %s is gone; removing it", subscription.id)
+        db.delete(subscription)
+        db.commit()
+        return False
+    if response.status_code >= 400:
+        subscription.last_error = f"{response.status_code}: {response.text[:200]}"
+        db.commit()
+        log.warning(
+            "push to %s refused: %s %s",
+            _audience(subscription.endpoint),
+            response.status_code,
+            response.text[:200],
+        )
+        return False
+
+    if subscription.last_error:
+        subscription.last_error = ""
+        db.commit()
+    return True
+
+
+def subscriptions_for(db: DBSession, user: User | None) -> list[PushSubscription]:
+    if user is None:
+        return []
+    return list(
+        db.scalars(select(PushSubscription).where(PushSubscription.user_id == user.id))
+    )
+
+
+async def announce(
+    db: DBSession, user: User | None, *, title: str, body: str, url: str = "", kind: str = ""
+) -> int:
+    """Tell every device this person has registered. Returns how many took it.
+
+    Called at the moment something arrives rather than from the poll, because
+    the whole point is the case where no page is open to poll. A device with a
+    page open gets this *and* the in-page toast -- the service worker resolves
+    that by not showing a notification when one of its own windows is focused,
+    which is the only place that can be known.
+    """
+    subscriptions = subscriptions_for(db, user)
+    if not subscriptions:
+        return 0
+
+    payload = {"title": title, "body": body, "url": url, "kind": kind}
+    delivered = 0
+    for subscription in list(subscriptions):
+        if await send_one(db, subscription, payload):
+            delivered += 1
+    return delivered
+
+
+# Fire-and-forget tasks, held so the event loop does not collect one mid-flight.
+# asyncio keeps only a weak reference to a task nobody awaits, and a push that
+# vanishes halfway is the kind of intermittent nobody reproduces.
+_TASKS: set[Any] = set()
+
+
+def announce_later(user_id: str, *, title: str, body: str, url: str = "", kind: str = "") -> None:
+    """Announce from a path that must not wait for it, and must not fail with it.
+
+    Called where something *arrives* -- a reply finishing with nobody watching,
+    a report being filed, a run posting into Messages -- rather than from the
+    poll. That is the whole point: the poll needs an open page, and the case
+    worth a notification is the one where there is none.
+
+    Each of those sites runs exactly once per arrival, which is what makes this
+    fire once with no "already notified" flag of its own. `unread_notified` is
+    the *page's* record of having toasted; borrowing it here would mean whichever
+    channel got there first silenced the other.
+
+    Its own session, opened inside the task: the caller's is usually about to be
+    committed and closed, and holding one open across a POST to somebody else's
+    server is how a request comes to wait on a push service having a bad day.
+    """
+    import asyncio
+
+    from lembas.db.session import session_scope
+
+    # The loop is checked *before* the coroutine is built, not by catching what
+    # `create_task` raises without one. A coroutine made and then dropped is a
+    # "never awaited" RuntimeWarning from wherever it was created -- which here
+    # is every synchronous caller in the suite and every CLI command that files
+    # a report. The warning would be the only symptom, and it would be
+    # attributed to the caller rather than to this.
+    try:
+        asyncio.get_running_loop()
+    except RuntimeError:
+        return
+
+    async def run() -> None:
+        try:
+            with session_scope() as db:
+                user = db.get(User, user_id)
+                if user is None:
+                    return
+                await announce(db, user, title=title, body=body, url=url, kind=kind)
+        except Exception:  # noqa: BLE001 - nothing upstream can act on this
+            log.exception("could not announce to %s", user_id)
+
+    task = asyncio.create_task(run())
+    _TASKS.add(task)
+    task.add_done_callback(_TASKS.discard)
+
+
+__all__ = [
+    "announce",
+    "announce_later",
+    "encrypt_payload",
+    "keys",
+    "public_key",
+    "send_one",
+    "subscriptions_for",
+]

src/lembas/services/reports.py

@@ -85,6 +85,32 @@ def unread_count(db: DBSession, user: User | None) -> int:
     )
 
 
+def unannounced(db: DBSession, user: User | None) -> list[Report]:
+    """Reports that have arrived and have not been announced yet.
+
+    Separate from `unread_count`, which drives the dot: the dot may be shown for
+    as long as something is unread, while an announcement fires once. Reading
+    them apart is what stops the poll interrupting somebody every ten seconds
+    with the same report until they open it.
+
+    Ordered oldest first, so several arriving between two ticks are announced in
+    the order they were filed.
+    """
+    if user is None:
+        return []
+    return list(
+        db.scalars(
+            select(Report)
+            .where(
+                Report.owner_id == user.id,
+                Report.unread.is_(True),
+                Report.unread_notified.is_(False),
+            )
+            .order_by(Report.created_at)
+        )
+    )
+
+
 def _first_line(body: str) -> str:
     """A summary for a model that did not write one.
 
@@ -137,6 +163,22 @@ def create(
     )
     db.add(report)
     db.commit()
+
+    # Here rather than at the scheduled-run site, because a report is filed from
+    # two places -- a schedule delivering one, and a model calling `report_write`
+    # in a chat nobody stayed on -- and both are arrivals somebody would want to
+    # know about. `unread` is what says it is news; a report filed with it off
+    # was made by the person reading this screen.
+    if report.unread:
+        from lembas.services import push as push_service
+
+        push_service.announce_later(
+            report.owner_id,
+            title=report.title or "Report filed",
+            body=report.summary or "A report is waiting for you.",
+            url=f"/reports/{report.id}",
+            kind="report",
+        )
     return report
 
 

src/lembas/services/schedule/runner.py

@@ -122,6 +122,18 @@ def _finished_reply(db, chat_id: str, message_id: str) -> Message | None:
     return message
 
 
+def _preview(text: str, limit: int = 160) -> str:
+    """The opening of a reply, as the body of a notification.
+
+    A notification saying "a scheduled run finished" is one somebody has to open
+    something to understand, which is most of the reason notifications get
+    ignored. Flattened to one line because a push service and an operating
+    system will each do their own thing with newlines.
+    """
+    flat = " ".join((text or "").split())
+    return flat[: limit - 1] + "…" if len(flat) > limit else flat
+
+
 async def deliver(schedule_id: str, message_id: str, *, since: datetime) -> None:
     """Put a finished reply where the schedule said it should go.
 
@@ -215,6 +227,20 @@ async def deliver(schedule_id: str, message_id: str, *, since: datetime) -> None
             conversation.unread_notified = False
             db.commit()
 
+            # The arrival this whole channel exists for: a run that fired while
+            # nobody was here, landing somewhere they are not looking. The first
+            # line of the reply is the body, because "a scheduled run finished"
+            # is a notification you have to open something to understand.
+            from lembas.services import push as push_service
+
+            push_service.announce_later(
+                owner.id,
+                title=schedule.title or "Scheduled run",
+                body=_preview(message.content or ""),
+                url="/messages",
+                kind="message",
+            )
+
 
 async def fire(schedule_id: str, *, due_at: datetime | None = None) -> None:
     """Run one schedule now.