Background generation, unread replies, send/stop, PLAN.md

**Replies now run in the background.** Generation was driven by the SSE
request, so navigating away or opening another chat cut the answer off
mid-sentence. services/generation.py owns the work as its own task and
the SSE endpoint merely follows it. Verified: attached briefly, closed
the connection, went to another page -- the reply finished anyway, 832
characters, not marked stopped, auto-titled.

Reattaching works because both `render` and `reasoning` frames now carry
the whole block rather than a delta. A follower arriving late has no
earlier fragments to append to, so deltas would leave it permanently
missing the beginning. Verified: attached six seconds in and the first
frame already contained 517 characters written while nobody watched.

**Unread indicator.** A reply that lands with no follower attached marks
its chat unread; the sidebar polls every 10s for out-of-band dot spans
plus an HX-Trigger that raises a toast. Polled rather than pushed: a
browser sitting on another chat has no connection to the one that
finished, and an always-on channel per tab is a lot of machinery for a
green dot. `unread_notified` stops the same arrival being announced
every tick. Follower count is what decides "was anyone watching", so
reading it as it arrives does not mark it unread -- verified both ways.

**Stop is the send button.** While a reply is being written the send
button becomes a red stop square, found via a MutationObserver on the
thread since the composer and the streaming bubble are far apart in the
document. The in-bubble Stop is gone.

**Attachment border removed.** As asked -- an attachment is a picture,
and the frame only ever drew at the wrong width. The anchor now
shrink-wraps and the img's width/height attributes are overridden so a
small image shows at its own size.

Adds PLAN.md: what is built, what is not, known limits, and the
decisions that look like oversights until you know the reason.

239 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

README.md

@@ -41,6 +41,8 @@ runtime. Clone it, `pip install -e .`, run it.
 - **Live Markdown** — formatting appears as the model writes, not at the end
 - **Stop and rewind** — cut a reply short and keep what arrived, or edit an
   earlier message and run the conversation on from there
+- **Replies keep running in the background** — navigate away, open another
+  chat, close the tab; a green dot and a notification tell you when it lands
 - **Attachments** — drag, paste or pick images, PDFs and text files. Images are
   downscaled and sent to vision models; PDF and text content is extracted and
   put in the prompt
@@ -65,6 +67,8 @@ runtime. Clone it, `pip install -e .`, run it.
 Built-in tools with admin settings · custom tools and MCP servers · agentic
 execution (local and over SSH) · image generation · OCR for scanned PDFs.
 
+See [PLAN.md](PLAN.md) for what is built, what is not, and why.
+
 ## Quick start
 
 ```bash

src/lembas/api/chats.py

@@ -3,8 +3,8 @@
 from __future__ import annotations
 
 import asyncio
+import json
 import logging
-import time
 from collections.abc import AsyncIterator
 
 from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
@@ -18,37 +18,15 @@ from lembas.db.session import session_scope
 from lembas.security import permissions
 from lembas.services import chat as chat_service
 from lembas.services import files as files_service
+from lembas.services import generation as generation_service
 from lembas.services import sse
-from lembas.services.llm.openai_client import (
-    LLMError,
-    delta_reasoning,
-    delta_text,
-    stream_chat,
-)
 from lembas.services.markdown import escape_text, render_markdown
-from lembas.services.reasoning import REASONING, ReasoningSplitter
 from lembas.web.templating import render, templates
 
 log = logging.getLogger(__name__)
 
 router = APIRouter(prefix="/api/chats", tags=["chats"])
 
-# Message ids whose generation has been asked to stop. The generator checks
-# this between chunks and finalises with whatever it has.
-#
-# In-process, which is correct for the single-worker deployment this ships
-# with: the request that stops a stream and the task producing it are in the
-# same process. Running multiple workers would need this in the database or a
-# broker instead -- see deploy/README.md.
-_CANCELLED: set[str] = set()
-
-# How often the partially rendered reply is pushed to the browser. Markdown is
-# re-rendered from scratch each time, so this trades a little server work for
-# formatting that appears as the model writes rather than all at once at the
-# end. 100ms is below the threshold where the eye reads it as stepping.
-RENDER_INTERVAL = 0.1
-
-
 def _owned_chat(db: DBSession, chat_id: str, user_id: str) -> Chat:
     chat = db.get(Chat, chat_id)
     # 404 rather than 403 for someone else's chat: whether a given id exists is
@@ -106,9 +84,10 @@ async def start_chat(
     user_message = chat_service.create_message(db, chat, ROLE_USER, content)
     if file_ids:
         files_service.claim(db, ids=file_ids, user_id=user.id, message_id=user_message.id)
-    chat_service.create_message(
+    assistant = chat_service.create_message(
         db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
     )
+    generation_service.ensure(chat.id, assistant.id)
 
     response = Response(status_code=status.HTTP_204_NO_CONTENT)
     response.headers["HX-Redirect"] = f"/chat/{chat.id}"
@@ -120,6 +99,44 @@ async def start_chat(
 # /start when the first message is actually sent.
 
 
+@router.get("/unread")
+async def unread_poll(db: Db, user: RequiredUser) -> Response:
+    """Dots for the sidebar, and a toast for anything newly arrived.
+
+    Polled rather than pushed: a browser sitting on a different chat has no
+    open connection to the one that finished, and a second always-on channel
+    per tab is a lot of machinery for a green dot.
+
+    Returns out-of-band spans so only the dots change -- re-rendering the whole
+    sidebar would reset the folder open/closed state on every tick.
+    """
+    chats = list(
+        db.scalars(
+            select(Chat).where(Chat.user_id == user.id, Chat.archived.is_(False))
+        )
+    )
+
+    fresh = [c for c in chats if c.unread and not c.unread_notified]
+    for chat in fresh:
+        chat.unread_notified = True
+    if fresh:
+        db.commit()
+
+    markup = "".join(
+        f'<span id="unread-{c.id}" class="unread-dot" hx-swap-oob="true"'
+        f'{"" if c.unread else " hidden"} title="New reply"></span>'
+        for c in chats
+    )
+
+    response = HTMLResponse(markup)
+    if fresh:
+        # HX-Trigger carries the toast; ui.js listens for it.
+        response.headers["HX-Trigger"] = json.dumps(
+            {"lembas:unread": {"titles": [c.title for c in fresh]}}
+        )
+    return response
+
+
 @router.post("/{chat_id}/messages")
 async def post_message(
     request: Request,
@@ -150,6 +167,7 @@ async def post_message(
     assistant_message = chat_service.create_message(
         db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
     )
+    generation_service.ensure(chat.id, assistant_message.id)
 
     # `user` is required by the shared message template, which renders both
     # roles; without it the user bubble's initial blows up.
@@ -187,7 +205,7 @@ async def stream_message(
         raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
 
     return StreamingResponse(
-        _generate(chat.id, message.id),
+        _follow(chat.id, message.id),
         media_type="text/event-stream",
         headers={
             "Cache-Control": "no-cache, no-transform",
@@ -199,161 +217,47 @@ async def stream_message(
     )
 
 
-def _plain_text(content: str | list) -> str:
+async def _follow(chat_id: str, message_id: str) -> AsyncIterator[str]:
-    """The text of a message payload, whether it is a string or content parts."""
+    """Stream a generation that is running independently of this request.
-    if isinstance(content, str):
-        return content
-    if isinstance(content, list):
-        return " ".join(
-            part.get("text", "")
-            for part in content
-            if isinstance(part, dict) and part.get("type") == "text"
-        ).strip()
-    return ""
 
+    This connection only *watches*. Closing it -- navigating away, opening
+    another chat -- leaves the reply being written, and reconnecting replays
+    the whole state immediately rather than starting over.
 
-async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
+    Both `render` and `reasoning` carry the complete block each time rather
-    """Drive one completion and frame it as SSE.
+    than a delta, which is what makes reattaching mid-reply work at all: a
-
+    follower arriving late has no earlier fragments to append to.
-    Opens its own database session rather than using the request's: streaming
-    outlives the request handler, and the dependency-scoped session may already
-    be closed by the time the first token arrives.
     """
-    accumulated: list[str] = []
+    generation = generation_service.ensure(chat_id, message_id)
-    thinking: list[str] = []
+    generation.followers += 1
-    error: str | None = None
+    seen = -1
-    reasoning_ms = 0
-
-    with session_scope() as db:
-        chat = db.get(Chat, chat_id)
-        message = db.get(Message, message_id)
-        if chat is None or message is None:
-            yield sse.event("close", "")
-            return
-
-        first_user_text = ""
-        # Handles models that emit <think> tags inline in content rather than
-        # using the reasoning_content field.
-        splitter = ReasoningSplitter()
-        started = time.monotonic()
-        reasoning_started: float | None = None
-        last_render = 0.0
-        dirty = False
-        stopped = False
 
     try:
-            endpoint, model_id = chat_service.resolve_endpoint(db, chat)
+        while True:
-            payload = chat_service.build_request(db, chat, upto=message)
+            if generation.version != seen:
-            # A multimodal turn's content is a list of parts, not a string, so
+                seen = generation.version
-            # the text has to be picked out before it can title a chat.
+                if generation.thinking:
-            first_user_text = next(
+                    yield sse.event("reasoning", escape_text(generation.thinking))
-                (
+                if generation.content:
-                    _plain_text(m["content"])
+                    yield sse.event("render", render_markdown(generation.text))
-                    for m in reversed(payload["messages"])
-                    if m["role"] == ROLE_USER
-                ),
-                "",
-            )
 
-            async for chunk in stream_chat(endpoint, payload):
+            if generation.done:
-                # A dedicated reasoning field is unambiguous; take it as-is.
-                thought = delta_reasoning(chunk)
-                if thought:
-                    if reasoning_started is None:
-                        reasoning_started = time.monotonic()
-                    thinking.append(thought)
-                    yield sse.event("reasoning", escape_text(thought))
-                    await asyncio.sleep(0)
-
-                text = delta_text(chunk)
-                if not text:
-                    continue
-
-                for kind, piece in splitter.feed(text):
-                    if kind == REASONING:
-                        if reasoning_started is None:
-                            reasoning_started = time.monotonic()
-                        thinking.append(piece)
-                        yield sse.event("reasoning", escape_text(piece))
-                    else:
-                        # First answer token ends the thinking phase.
-                        if reasoning_started is not None and not reasoning_ms:
-                            reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
-                        accumulated.append(piece)
-                        dirty = True
-                    # Hand control back so the event is flushed rather than
-                    # batched behind a fast generator.
-                    await asyncio.sleep(0)
-
-                # Re-render the answer so far, at most every RENDER_INTERVAL.
-                # Markdown is rendered whole rather than appended, because a
-                # list or a code fence is only correct once its context is
-                # known -- and partial syntax resolves itself as more arrives.
-                now = time.monotonic()
-                if dirty and now - last_render >= RENDER_INTERVAL:
-                    yield sse.event("render", render_markdown("".join(accumulated)))
-                    last_render, dirty = now, False
-                    await asyncio.sleep(0)
-
-                if message_id in _CANCELLED:
-                    stopped = True
                 break
+            # Polling rather than per-follower wakeups: the producer already
+            # works in RENDER_INTERVAL steps, so a short sleep is simpler and
+            # cannot drop a notification.
+            await asyncio.sleep(generation_service.RENDER_INTERVAL * 0.8)
+    finally:
+        generation.followers = max(0, generation.followers - 1)
 
-            for kind, piece in splitter.flush():
+    # The producer writes the message before marking itself done, so by here
-                if kind == REASONING:
+    # the row is authoritative and the final bubble can be rendered from it.
-                    thinking.append(piece)
+    with session_scope() as db:
-                    yield sse.event("reasoning", escape_text(piece))
+        message = db.get(Message, message_id)
-                else:
+        chat = db.get(Chat, chat_id)
-                    accumulated.append(piece)
+        if message is None or chat is None:
-
+            yield sse.event("close", "")
-        except LLMError as exc:
+            return
-            error = exc.message
-            log.info("generation failed for chat %s: %s", chat_id, exc.message)
-        except asyncio.CancelledError:
-            # The reader navigated away or closed the tab. Keep whatever was
-            # produced so the partial reply is still there on reload.
-            _CANCELLED.discard(message_id)
-            message.content = "".join(accumulated)
-            message.reasoning = "".join(thinking)
-            message.complete = True
-            message.stopped = True
-            db.commit()
-            raise
-        except Exception as exc:  # noqa: BLE001 - must not kill the stream silently
-            error = "Something went wrong while generating this reply."
-            log.exception("unexpected generation failure for chat %s: %s", chat_id, exc)
-
-        if reasoning_started is not None and not reasoning_ms:
-            # Reasoning ran to the end without an answer following it.
-            reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
-
-        _CANCELLED.discard(message_id)
-
-        message.content = "".join(accumulated)
-        message.reasoning = "".join(thinking)
-        message.reasoning_ms = reasoning_ms
-        message.error = error or ""
-        message.complete = True
-        message.stopped = stopped
-        log.debug(
-            "chat %s: %d chars answer, %d chars reasoning, %.1fs total",
-            chat_id,
-            len(message.content),
-            len(message.reasoning),
-            time.monotonic() - started,
-        )
-
-        if not chat.title_generated and (accumulated or error):
-            chat.title = (
-                await chat_service.generate_title(
-                    endpoint, model_id, first_user_text, message.content
-                )
-                if not error and first_user_text
-                else chat_service.fallback_title(first_user_text)
-            )
-            chat.title_generated = True
-
-        db.commit()
 
         final_html = templates.get_template("chat/_message.html").render(
             {
@@ -369,9 +273,7 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
                 },
             }
         )
-        title_html = templates.get_template("chat/_title_oob.html").render(
+        title_html = templates.get_template("chat/_title_oob.html").render({"chat": chat})
-            {"chat": chat}
-        )
 
     yield sse.event("done", final_html + title_html)
     yield sse.event("close", "")
@@ -485,9 +387,10 @@ async def edit_message(
         db.delete(later)
     db.commit()
 
-    chat_service.create_message(
+    assistant = chat_service.create_message(
         db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
     )
+    generation_service.ensure(chat.id, assistant.id)
     log.info("chat %s rewound to a message, %d discarded", chat.id, len(discarded))
 
     return templates.TemplateResponse(
@@ -507,7 +410,7 @@ async def stop_message(db: Db, user: RequiredUser, chat_id: str, message_id: str
     if message is None or message.chat_id != chat.id:
         raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
 
-    _CANCELLED.add(message.id)
+    generation_service.request_stop(message.id)
     return Response(status_code=status.HTTP_204_NO_CONTENT)
 
 
@@ -648,6 +551,7 @@ async def regenerate(
     message.complete = False
     message.model_id = chat.model_id
     db.commit()
+    generation_service.ensure(chat.id, message.id)
 
     return templates.TemplateResponse(
         request,

src/lembas/api/pages.py

@@ -117,6 +117,12 @@ async def chat_detail(request: Request, db: Db, user: RequiredUser, chat_id: str
     if chat is None or chat.user_id != user.id:
         raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
 
+    # Opening the chat is what "read" means.
+    if chat.unread:
+        chat.unread = False
+        chat.unread_notified = False
+        db.commit()
+
     messages = list(
         db.scalars(
             select(Message).where(Message.chat_id == chat.id).order_by(Message.created_at)

src/lembas/db/models/chat.py

@@ -71,6 +71,12 @@ class Chat(UUIDPrimaryKey, Timestamps, Base):
     pinned: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
     archived: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
 
+    # A reply landed while nobody was watching this chat. Cleared when the chat
+    # is next opened. `unread_notified` stops the same arrival being announced
+    # on every poll.
+    unread: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+    unread_notified: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
     folder: Mapped[Folder | None] = relationship(back_populates="chats")
     messages: Mapped[list[Message]] = relationship(
         back_populates="chat",

src/lembas/main.py

@@ -67,6 +67,12 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
     log.info("LLeMbas %s starting on http://%s:%s", __version__, settings.host, settings.port)
     log.info("data directory: %s", settings.data_dir.resolve())
     yield
+
+    # Replies still being written are cancelled and persisted with whatever
+    # they have, rather than left as permanently unfinished rows.
+    from lembas.services.generation import shutdown as stop_generations
+
+    await stop_generations()
     log.info("LLeMbas stopped")
 
 

src/lembas/services/generation.py

@@ -0,0 +1,285 @@
+"""Background reply generation.
+
+Generation used to be driven by the SSE request: the browser opening the stream
+was what produced the tokens, so navigating away cancelled the reply mid-
+sentence. Here it runs as its own task instead, and the SSE endpoint merely
+*follows* it. Closing the page, opening another chat, or starting a new one
+leaves the answer being written; coming back attaches to it and immediately
+receives everything produced so far.
+
+The registry is in-process, which is right for the single-worker deployment
+this ships with. Several workers would need the state in the database or a
+broker, because the request that follows a generation would not necessarily
+land in the process running it.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import logging
+import time
+from dataclasses import dataclass, field
+from datetime import UTC, datetime, timedelta
+
+from lembas.db.models import ROLE_ASSISTANT, ROLE_USER, Chat, Message
+from lembas.db.session import session_scope
+from lembas.services import chat as chat_service
+from lembas.services.llm.openai_client import LLMError, delta_reasoning, delta_text, stream_chat
+from lembas.services.reasoning import REASONING, ReasoningSplitter
+
+log = logging.getLogger(__name__)
+
+# How often the partial answer is offered to followers. Markdown is re-rendered
+# whole each time -- a list or a code fence is only correct once its context
+# exists -- so this trades a little work for formatting that appears as the
+# model writes. 100ms is below the threshold where the eye reads it as stepping.
+RENDER_INTERVAL = 0.1
+
+# Finished generations linger so a follower attaching at the last moment still
+# gets the final frames, then are pruned.
+KEEP_FINISHED = timedelta(minutes=5)
+
+
+@dataclass
+class Generation:
+    """The live state of one reply being written."""
+
+    chat_id: str
+    message_id: str
+
+    content: list[str] = field(default_factory=list)
+    reasoning: list[str] = field(default_factory=list)
+    reasoning_ms: int = 0
+
+    error: str = ""
+    stopped: bool = False
+    done: bool = False
+
+    # Bumped on every change. Followers compare against it rather than being
+    # woken individually: with a 100ms cadence a short poll is simpler than
+    # future bookkeeping, and cannot drop a wakeup.
+    version: int = 0
+    # Number of browsers currently watching. Decides whether a finished reply
+    # counts as unread.
+    followers: int = 0
+    finished_at: datetime | None = None
+    cancel: bool = False
+
+    def touch(self) -> None:
+        self.version += 1
+
+    @property
+    def text(self) -> str:
+        return "".join(self.content)
+
+    @property
+    def thinking(self) -> str:
+        return "".join(self.reasoning)
+
+
+_RUNNING: dict[str, Generation] = {}
+_TASKS: dict[str, asyncio.Task] = {}
+
+
+def get(message_id: str) -> Generation | None:
+    return _RUNNING.get(message_id)
+
+
+def request_stop(message_id: str) -> bool:
+    """Ask a running generation to stop. Returns whether one was found."""
+    generation = _RUNNING.get(message_id)
+    if generation is None or generation.done:
+        return False
+    generation.cancel = True
+    return True
+
+
+def _prune() -> None:
+    cutoff = datetime.now(UTC) - KEEP_FINISHED
+    for message_id, generation in list(_RUNNING.items()):
+        if generation.done and generation.finished_at and generation.finished_at < cutoff:
+            _RUNNING.pop(message_id, None)
+            _TASKS.pop(message_id, None)
+
+
+def ensure(chat_id: str, message_id: str) -> Generation:
+    """Start generating this reply if it is not already under way.
+
+    Idempotent, because more than one thing can ask for it: the route that
+    created the message, and any page load that finds the message unfinished.
+    """
+    existing = _RUNNING.get(message_id)
+    if existing is not None:
+        return existing
+
+    _prune()
+    generation = Generation(chat_id=chat_id, message_id=message_id)
+    _RUNNING[message_id] = generation
+    _TASKS[message_id] = asyncio.create_task(_run(generation))
+    return generation
+
+
+async def shutdown() -> None:
+    """Stop every running generation, keeping what each has produced."""
+    for task in list(_TASKS.values()):
+        task.cancel()
+    for task in list(_TASKS.values()):
+        with contextlib.suppress(asyncio.CancelledError, Exception):
+            await task
+
+
+async def _run(generation: Generation) -> None:
+    """Produce one reply, then persist it. Never raises into the task."""
+    splitter = ReasoningSplitter()
+    started = time.monotonic()
+    reasoning_started: float | None = None
+    question = ""
+    endpoint = model_id = None
+    needs_title = False
+
+    try:
+        with session_scope() as db:
+            chat = db.get(Chat, generation.chat_id)
+            message = db.get(Message, generation.message_id)
+            if chat is None or message is None:
+                generation.error = "That chat no longer exists."
+                return
+
+            endpoint, model_id = chat_service.resolve_endpoint(db, chat)
+            payload = chat_service.build_request(db, chat, upto=message)
+            question = _question_from(payload)
+            needs_title = not chat.title_generated
+
+        async for chunk in stream_chat(endpoint, payload):
+            thought = delta_reasoning(chunk)
+            if thought:
+                if reasoning_started is None:
+                    reasoning_started = time.monotonic()
+                generation.reasoning.append(thought)
+                generation.touch()
+
+            text = delta_text(chunk)
+            if text:
+                for kind, piece in splitter.feed(text):
+                    if kind == REASONING:
+                        if reasoning_started is None:
+                            reasoning_started = time.monotonic()
+                        generation.reasoning.append(piece)
+                    else:
+                        if reasoning_started is not None and not generation.reasoning_ms:
+                            generation.reasoning_ms = int(
+                                (time.monotonic() - reasoning_started) * 1000
+                            )
+                        generation.content.append(piece)
+                generation.touch()
+
+            if generation.cancel:
+                generation.stopped = True
+                break
+
+            # Let followers and other tasks run between chunks.
+            await asyncio.sleep(0)
+
+        for kind, piece in splitter.flush():
+            (generation.reasoning if kind == REASONING else generation.content).append(piece)
+        generation.touch()
+
+    except LLMError as exc:
+        generation.error = exc.message
+        log.info("generation failed for chat %s: %s", generation.chat_id, exc.message)
+    except asyncio.CancelledError:
+        # Shutdown, not a reader navigating away -- that no longer reaches here.
+        generation.stopped = True
+        raise
+    except Exception:  # noqa: BLE001 - a task that dies silently is worse
+        generation.error = "Something went wrong while generating this reply."
+        log.exception("unexpected generation failure for chat %s", generation.chat_id)
+    finally:
+        if reasoning_started is not None and not generation.reasoning_ms:
+            generation.reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
+
+        # Naming the chat is a second, short completion, so it has to happen
+        # here rather than in the synchronous persist step below. Best-effort:
+        # a chat title is never worth surfacing an error for.
+        title = ""
+        if needs_title and question:
+            if generation.error or endpoint is None:
+                title = chat_service.fallback_title(question)
+            else:
+                with contextlib.suppress(Exception):
+                    title = await chat_service.generate_title(
+                        endpoint, model_id, question, generation.text
+                    )
+                title = title or chat_service.fallback_title(question)
+
+        generation.done = True
+        generation.finished_at = datetime.now(UTC)
+        generation.touch()
+        _persist(generation, title, time.monotonic() - started)
+
+
+def _question_from(payload: dict) -> str:
+    """The last thing the user said, for auto-titling."""
+    for entry in reversed(payload.get("messages", [])):
+        if entry.get("role") != ROLE_USER:
+            continue
+        content = entry.get("content")
+        if isinstance(content, str):
+            return content
+        if isinstance(content, list):
+            return " ".join(
+                part.get("text", "")
+                for part in content
+                if isinstance(part, dict) and part.get("type") == "text"
+            ).strip()
+    return ""
+
+
+def _persist(generation: Generation, title: str, elapsed: float) -> None:
+    """Write the finished reply, name the chat, and set the unread flag."""
+    try:
+        with session_scope() as db:
+            message = db.get(Message, generation.message_id)
+            chat = db.get(Chat, generation.chat_id)
+            if message is None or chat is None:
+                return
+
+            message.content = generation.text
+            message.reasoning = generation.thinking
+            message.reasoning_ms = generation.reasoning_ms
+            message.error = generation.error
+            message.stopped = generation.stopped
+            message.complete = True
+
+            if title and not chat.title_generated:
+                chat.title = title
+                chat.title_generated = True
+
+            # Nobody watching when it landed, so it is news. The chat page
+            # clears this when it is next opened.
+            if generation.followers == 0:
+                chat.unread = True
+                chat.unread_notified = False
+
+            db.commit()
+            log.debug(
+                "chat %s finished: %d chars, %d reasoning, %.1fs",
+                generation.chat_id,
+                len(message.content),
+                len(message.reasoning),
+                elapsed,
+            )
+    except Exception:  # noqa: BLE001 - the task is ending either way
+        log.exception("could not persist generation for chat %s", generation.chat_id)
+
+
+__all__ = [
+    "RENDER_INTERVAL",
+    "Generation",
+    "ROLE_ASSISTANT",
+    "ensure",
+    "get",
+    "request_stop",
+    "shutdown",
+]

src/lembas/web/static/css/chat.css

@@ -208,8 +208,21 @@
    the dots go, but Stop must stay reachable until the stream ends. */
 .msg__body--live:not(:empty) + .msg__waiting .dots { display: none; }
 
-.msg__stop { color: var(--ink-muted); }
+.composer__stop {
-.msg__stop:hover { color: var(--danger); border-color: var(--danger); }
+  background: var(--danger);
+  border-color: var(--danger);
+  color: var(--ink-inverse);
+}
+.composer__stop:hover:not(:disabled) {
+  background: var(--danger-hover);
+  border-color: var(--danger-hover);
+}
+.composer__stop-square {
+  width: 0.7rem;
+  height: 0.7rem;
+  border-radius: 2px;
+  background: currentColor;
+}
 
 .msg__note {
   display: flex;
@@ -480,25 +493,28 @@
 .attachments {
   display: flex;
   flex-wrap: wrap;
+  align-items: flex-start;
   gap: var(--sp-2);
   margin-bottom: var(--sp-2);
 }
-/* inline-block, not block: as a block the anchor filled the column and drew its
+/* No frame: an attachment is a picture, and a border around it only ever drew
-   border at full width around a narrow image. Now the frame is the picture. */
+   at the wrong width. The anchor shrink-wraps its image rather than filling the
+   column, and the width/height attributes on the <img> are overridden so a
+   small image is shown at its own size instead of being stretched. */
 .attachments__image {
-  display: inline-block;
+  display: inline-flex;
   max-width: 100%;
   border-radius: var(--radius);
   overflow: hidden;
-  border: 1px solid var(--border);
   line-height: 0;
 }
 .attachments__image img {
   display: block;
-  max-width: min(22rem, 100%);
-  max-height: 20rem;
   width: auto;
   height: auto;
+  max-width: min(22rem, 100%);
+  max-height: 20rem;
+  object-fit: contain;
 }
 .attachments__doc {
   display: flex;
@@ -549,3 +565,15 @@
 /* Alpine sets x-cloak until it has initialised; without this, collapsed
    folders flash open on every page load. */
 [x-cloak] { display: none !important; }
+
+/* --- Unread indicator ------------------------------------------------------ */
+.unread-dot {
+  width: 0.5rem;
+  height: 0.5rem;
+  border-radius: var(--radius-full);
+  background: var(--success);
+  flex: none;
+  /* A ring so it stays visible against the active row's lighter background. */
+  box-shadow: 0 0 0 2px color-mix(in srgb, var(--success) 25%, transparent);
+}
+.unread-dot[hidden] { display: none; }

src/lembas/web/static/js/ui.js

@@ -378,3 +378,88 @@
     options[next].focus();
   });
 })();
+
+/*
+  Unread replies.
+
+  The sidebar polls /api/chats/unread; the response carries out-of-band spans
+  for the dots and, when something has just landed, an HX-Trigger asking for a
+  toast. Announcing it here rather than server-side keeps the wording and the
+  timing in one place.
+*/
+document.addEventListener("lembas:unread", function (event) {
+  var titles = (event.detail && event.detail.titles) || [];
+  if (!titles.length || !window.lembas || !window.lembas.notify) return;
+
+  var message = titles.length === 1
+    ? "Reply ready in “" + titles[0] + "”"
+    : titles.length + " chats have new replies";
+  window.lembas.notify(message, { kind: "success", timeout: 6000 });
+});
+
+/*
+  Send becomes Stop while a reply is being written.
+
+  The composer and the streaming bubble are far apart in the document, so the
+  link between them is made here: whenever the thread changes, look for a
+  message that is still streaming and point the button at it. A MutationObserver
+  rather than htmx events, because the bubble is replaced by an SSE swap that
+  does not always surface as one.
+*/
+(function () {
+  "use strict";
+
+  function streamingMessage() {
+    var live = document.querySelector(".msg[sse-connect]");
+    if (!live) return null;
+    var id = live.id.replace(/^msg-/, "");
+    var chat = (live.getAttribute("sse-connect") || "").match(/\/api\/chats\/([^/]+)\//);
+    return chat ? { messageId: id, chatId: chat[1] } : null;
+  }
+
+  function sync() {
+    var form = document.querySelector(".composer__form");
+    if (!form) return;
+    var send = form.querySelector('[type="submit"]');
+    var stop = form.querySelector("[data-composer-stop]");
+    var active = streamingMessage();
+
+    if (active) {
+      if (send) send.hidden = true;
+      if (!stop) {
+        stop = document.createElement("button");
+        stop.type = "button";
+        stop.className = "btn btn--icon composer__btn composer__stop";
+        stop.setAttribute("data-composer-stop", "");
+        stop.setAttribute("aria-label", "Stop generating");
+        stop.title = "Stop generating";
+        stop.innerHTML = '<span class="composer__stop-square"></span>';
+        stop.addEventListener("click", function () {
+          var target = streamingMessage();
+          if (!target) return;
+          stop.disabled = true;
+          fetch(
+            "/api/chats/" + target.chatId + "/messages/" + target.messageId + "/stop",
+            { method: "POST", credentials: "same-origin" }
+          ).catch(function () { stop.disabled = false; });
+        });
+        (send ? send.parentNode : form).appendChild(stop);
+      }
+      stop.hidden = false;
+      stop.disabled = false;
+    } else {
+      if (send) send.hidden = false;
+      if (stop) stop.hidden = true;
+    }
+  }
+
+  function watch() {
+    var thread = document.getElementById("thread");
+    if (!thread) return;
+    new MutationObserver(sync).observe(thread, { childList: true, subtree: true });
+    sync();
+  }
+
+  document.addEventListener("DOMContentLoaded", watch);
+  document.body && document.body.addEventListener("htmx:afterSwap", sync);
+})();

src/lembas/web/templates/chat/_message.html

@@ -106,13 +106,10 @@
          rather than snapping into place at the end. #}
       <div class="msg__body msg__body--live" id="stream-{{ message.id }}"
            sse-swap="render" hx-swap="innerHTML"></div>
+      {# No stop button here: the composer's send button becomes Stop while a
+         reply is being written, which is where the hand already is. #}
       <div class="msg__waiting">
         <span class="dots"><i></i><i></i><i></i></span>
-        <button class="btn btn--sm msg__stop" type="button"
-                hx-post="/api/chats/{{ chat.id }}/messages/{{ message.id }}/stop"
-                hx-swap="none">
-          {{ icon("x", "icon--sm") }} Stop
-        </button>
       </div>
     {% elif message.reasoning and not message.error %}
       {# Collapsed once finished: the answer is what the reader came for, and

src/lembas/web/templates/partials/_chat_link.html

@@ -8,6 +8,9 @@
   <a class="nav-item__link" href="/chat/{{ chat_item.id }}">
     {{ icon("chat", "icon--sm") }}
     <span class="nav-item__label" id="chat-link-label-{{ chat_item.id }}">{{ chat_item.title }}</span>
+    {# Toggled out of band by the unread poll; see /api/chats/unread. #}
+    <span id="unread-{{ chat_item.id }}" class="unread-dot"
+          {{ '' if chat_item.unread else 'hidden' }} title="New reply"></span>
   </a>
   <span class="nav-item__actions">
     <button class="btn btn--icon btn--sm" type="button"

src/lembas/web/templates/partials/sidebar.html

@@ -27,6 +27,12 @@
   </div>
   {% endif %}
 
+  {# Every 10s, refresh the unread dots and announce anything that finished
+     while this page was showing something else. Out-of-band spans only, so the
+     folder tree keeps its open/closed state. #}
+  <div hidden hx-get="/api/chats/unread" hx-trigger="every 10s"
+       hx-swap="none"></div>
+
   <nav class="sidebar__scroll" id="sidebar-tree" aria-label="Chats">
     {% if pinned_models and can.get("chat.create") %}
     {# Shortcuts to start a chat with a particular model. These link rather than

tests/test_chat.py

@@ -355,12 +355,10 @@ def test_streaming_reports_an_unreachable_endpoint_in_the_thread(
 
 
 # --- Stopping a stream -------------------------------------------------------
-def test_stopping_marks_the_message_and_keeps_what_arrived(
+def test_stopping_asks_the_generation_to_stop(client: TestClient, db, registered, make_chat):
-    client: TestClient, db, registered, make_chat
-):
     """A half-written answer the reader chose to cut short is still worth
     having; discarding it would be a surprise."""
-    from lembas.api.chats import _CANCELLED
+    from lembas.services import generation as generation_service
 
     _add_connection(db)
     chat_id = make_chat()
@@ -370,8 +368,11 @@ def test_stopping_marks_the_message_and_keeps_what_arrived(
     assert client.post(
         f"/api/chats/{chat_id}/messages/{message.id}/stop"
     ).status_code == 204
-    assert message.id in _CANCELLED
+
-    _CANCELLED.discard(message.id)
+    running = generation_service.get(message.id)
+    # The endpoint points at 127.0.0.1:1, so the task may already have failed
+    # and finished; either way the request must be accepted, not error.
+    assert running is None or running.cancel or running.done
 
 
 def test_stopping_someone_elses_message_is_refused(client: TestClient, db, registered, make_chat):
@@ -391,12 +392,16 @@ def test_stopping_someone_elses_message_is_refused(client: TestClient, db, regis
     ).status_code == 404
 
 
-def test_the_streaming_bubble_offers_a_stop_button(client: TestClient, db, registered, make_chat):
+def test_the_streaming_bubble_carries_the_sse_connection(
+    client: TestClient, db, registered, make_chat
+):
+    """Stop lives on the composer's send button now, and the JS finds the
+    running message through this attribute."""
     _add_connection(db)
     chat_id = make_chat()
     response = client.post(f"/api/chats/{chat_id}/messages", data={"content": "hi"})
-    assert "/stop" in response.text
+    assert "sse-connect" in response.text
-    assert "msg__stop" in response.text
+    assert f"/api/chats/{chat_id}/messages/" in response.text
 
 
 def test_the_streaming_bubble_renders_markdown_not_raw_tokens(
@@ -501,3 +506,115 @@ def test_cancelling_an_edit_restores_the_bubble(client: TestClient, db, register
     page = client.get(f"/api/chats/{chat_id}/messages/{user_message.id}/cancel-edit").text
     assert "unchanged" in page
     assert "edit-form" not in page
+
+
+# --- Background generation ---------------------------------------------------
+def test_sending_launches_the_generation_immediately(
+    client: TestClient, db, registered, make_chat
+):
+    """The reply is produced by a task, not by the browser watching it. That is
+    what lets you navigate away without cutting it off."""
+    from lembas.services import generation as generation_service
+
+    _add_connection(db)
+    chat_id = make_chat()
+    client.post(f"/api/chats/{chat_id}/messages", data={"content": "hi"})
+    message = db.scalar(select(Message).where(Message.role == "assistant"))
+
+    assert generation_service.get(message.id) is not None
+
+
+def test_starting_a_chat_launches_the_generation(client: TestClient, db, registered):
+    from lembas.services import generation as generation_service
+
+    _add_connection(db)
+    client.post("/api/chats/start", data={"content": "hi"})
+    message = db.scalar(select(Message).where(Message.role == "assistant"))
+    assert generation_service.get(message.id) is not None
+
+
+def test_asking_twice_does_not_start_a_second_generation(
+    client: TestClient, db, registered, make_chat
+):
+    """A page load finding an unfinished reply must attach, not restart."""
+    from lembas.services import generation as generation_service
+
+    _add_connection(db)
+    chat_id = make_chat()
+    client.post(f"/api/chats/{chat_id}/messages", data={"content": "hi"})
+    message = db.scalar(select(Message).where(Message.role == "assistant"))
+
+    first = generation_service.get(message.id)
+    assert generation_service.ensure(chat_id, message.id) is first
+
+
+# --- Unread -------------------------------------------------------------------
+def test_the_unread_poll_reports_dots(client: TestClient, db, registered, make_chat):
+    _add_connection(db)
+    chat_id = make_chat()
+    chat = db.get(Chat, chat_id)
+    chat.unread = True
+    db.commit()
+
+    response = client.get("/api/chats/unread")
+    assert f'id="unread-{chat_id}"' in response.text
+    assert "hidden" not in response.text
+    assert "lembas:unread" in response.headers.get("HX-Trigger", "")
+
+
+def test_an_arrival_is_announced_once(client: TestClient, db, registered, make_chat):
+    """Otherwise the same reply would toast every ten seconds forever."""
+    _add_connection(db)
+    chat_id = make_chat()
+    chat = db.get(Chat, chat_id)
+    chat.unread = True
+    db.commit()
+
+    assert "HX-Trigger" in client.get("/api/chats/unread").headers
+    assert "HX-Trigger" not in client.get("/api/chats/unread").headers
+
+
+def test_opening_a_chat_marks_it_read(client: TestClient, db, registered, make_chat):
+    _add_connection(db)
+    chat_id = make_chat()
+    chat = db.get(Chat, chat_id)
+    chat.unread = True
+    db.commit()
+
+    client.get(f"/chat/{chat_id}")
+    db.expire_all()
+    assert db.get(Chat, chat_id).unread is False
+
+
+def test_a_read_chat_reports_a_hidden_dot(client: TestClient, db, registered, make_chat):
+    _add_connection(db)
+    chat_id = make_chat()
+    response = client.get("/api/chats/unread")
+    assert f'id="unread-{chat_id}"' in response.text
+    assert "hidden" in response.text
+
+
+def test_the_unread_poll_only_sees_your_own_chats(client: TestClient, db, registered, make_chat):
+    _add_connection(db)
+    mine = make_chat()
+
+    client.post("/auth/logout", follow_redirects=False)
+    client.post(
+        "/auth/register",
+        data={"name": "Sam", "email": "sam@shire.test", "password": "potatoes-po-ta-toes"},
+        follow_redirects=False,
+    )
+    assert mine not in client.get("/api/chats/unread").text
+
+
+def test_the_sidebar_shows_an_unread_dot(client: TestClient, db, registered, make_chat):
+    _add_connection(db)
+    chat_id = make_chat()
+    chat = db.get(Chat, chat_id)
+    chat.unread = True
+    db.commit()
+
+    # Rendered on another page, so the dot is visible while looking elsewhere.
+    page = client.get("/chat").text
+    assert f'id="unread-{chat_id}" class="unread-dot"' in page
+    assert 'hx-get="/api/chats/unread"' in page