Two selects that never wrote anything, and a queue

The approval card in Auto mode and the missing /effort were one bug. Both
selects hung their hx-patch on an empty sibling form reached by form="…",
and htmx binds a trigger to the annotated element: change fires on the
select and bubbles to its ancestors, which a sibling is not. The live rows
read agent_mode=manual and params_json={} while the browser showed Auto and
Effort: high. policy.py was never involved.

The verb moves onto the control; the empty form stays as value scoping,
which is the half of the CLAUDE.md note that was right. conftest gains
control_named so a test asserts the element carrying the name carries the
verb, rather than asserting the markup that was there throughout.

The composer's highlight was a third instance of the same carelessness in
CSS: .tok-mention is written for the transcript and scoped to nothing, so
the mirror painted its token in accent-coloured monospace over the
textarea's own text. Scoped under .msg; the mirror restates transparency
and font rather than inheriting them, and bleeds by box-shadow.

/effort is now offered before the first prompt and _new_chat reads it.
/index re-walks the project directory on demand, file_write drops the
listing it just invalidated, and the index ladder falls through to SFTP on
a host that refuses exec instead of returning nothing.

A second message during a reply is queued rather than starting a second
concurrent generation: a real Message row with queued set, so it survives a
restart and can be withdrawn. _drain hands one on at the end of a reply,
_inject takes one in at a tool-round boundary so an agent can be steered
mid-task. Stop leaves the queue undelivered. The terminal's Auto toggle
becomes off/copy/send, and send posts straight to the chat without touching
the composer.

@ now offers notes, skills, this chat's attachments and a URL to fetch; a
knowledge base attaches as a reference rather than a copy. copy_document
carries provenance, which was the one attach path that dropped it.

Also fixes an unrelated live bug: the round loop compared against the
global MAX_ROUNDS of 3 while sizing itself from the agent budget of 40, so
agent replies stopped after three rounds and reported forty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/services/agent/index.py

@@ -223,12 +223,21 @@ async def build(executor: Executor, project_dir: str) -> ProjectIndex:
     """Walk the directory, by whichever means works first."""
     started = time.monotonic()
     try:
+        found = None
         for attempt in (_from_git, _from_find):
-            found = await attempt(executor, project_dir)
+            try:
+                found = await attempt(executor, project_dir)
+            except ExecError as exc:
+                # A rung that cannot run at all is a rung that did not answer,
+                # not the end of the ladder. A host that refuses exec entirely
+                # -- an SFTP-only account, a forced command -- is the exact case
+                # the SFTP rung below exists for, and letting this out skipped
+                # straight past it to an empty listing.
+                log.debug("indexing %s: %s did not run: %s", project_dir, attempt.__name__,
+                          exc.message)
+                found = None
             if found is not None:
                 break
-        else:
-            found = None
         if found is None:
             found = await _from_sftp(executor, project_dir)
     except ExecError as exc:
@@ -491,6 +500,17 @@ def forget(profile_id: str) -> int:
     return len(doomed)
 
 
+def forget_dir(profile_id: str, project_dir: str) -> None:
+    """Drop one tree's listing, because something just changed it.
+
+    The TTL exists for drift nobody can see coming. A write through `file_write`
+    is not that: it is this process changing the tree it has just described, and
+    leaving five minutes of a listing that is known to be wrong is worse than
+    having none -- a model reading it concludes the file it created is missing.
+    """
+    _CACHE.pop((profile_id, project_dir), None)
+
+
 def clear() -> None:
     _CACHE.clear()
 
@@ -503,5 +523,6 @@ __all__ = [
     "clear",
     "ensure",
     "forget",
+    "forget_dir",
     "render",
 ]

src/lembas/services/agent/session.py

@@ -36,6 +36,10 @@ class AgentContext:
     chat_id: str
     label: str
     project_dir: str
+    # The connection's id, carried so a runner can drop the project listing it
+    # has just invalidated. `index` is keyed on the connection and the
+    # directory, not on the chat -- two chats on one tree share a listing.
+    profile_id: str = ""
     mode: str = policy.MODE_MANUAL
     allow: tuple[str, ...] = ()
     deny: tuple[str, ...] = ()
@@ -112,6 +116,7 @@ def resolve(db: DBSession, chat: Chat, user: User | None) -> AgentContext | None
         chat_id=chat.id,
         label=profile.label,
         project_dir=chat.project_dir or profile.default_dir or "",
+        profile_id=profile.id,
         mode=chat.agent_mode if chat.agent_mode in policy.MODES else policy.MODE_MANUAL,
         allow=tuple(values.get("allow_default") or ()),
         deny=tuple(values.get("deny_default") or ()),

src/lembas/services/agent/tools.py

@@ -21,7 +21,7 @@ import json
 import logging
 from typing import Any
 
-from lembas.services.agent import policy
+from lembas.services.agent import index, policy
 from lembas.services.agent.base import ExecError, ExecRequest
 from lembas.services.agent.session import AgentContext
 from lembas.services.tools import (
@@ -198,6 +198,13 @@ async def _run_write(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
             exc.message, _event("file_write", agent, path, status="error", error=exc.message)
         )
 
+    # The tree just changed, and this process is what changed it. The listing's
+    # TTL is for drift nobody can see coming; leaving five more minutes of a
+    # listing known to be wrong makes a model conclude the file it has just
+    # written does not exist.
+    if agent.profile_id:
+        index.forget_dir(agent.profile_id, agent.project_dir)
+
     return ToolOutcome(
         f"Wrote {written} bytes to {path}.",
         _event("file_write", agent, path, status="ok", text=f"{written} bytes"),

src/lembas/services/chat.py

@@ -226,6 +226,11 @@ def build_messages(
             message
         ) <= compaction_service.moment(cutoff):
             continue
+        # Typed while the previous reply was still being written, and not yet
+        # handed to a model. It is in the transcript and it is not in the
+        # request; delivery is what moves it from one to the other.
+        if message.queued:
+            continue
         # Skip turns that failed or produced nothing -- but a message carrying
         # only an attachment has no text and must still be sent.
         if message.error:
@@ -447,6 +452,7 @@ def create_message(
     *,
     complete_: bool = True,
     model_id: str = "",
+    queued: bool = False,
 ) -> Message:
     message = Message(
         chat_id=chat.id,
@@ -454,6 +460,7 @@ def create_message(
         content=content,
         complete=complete_,
         model_id=model_id,
+        queued=queued,
     )
     db.add(message)
     db.commit()

src/lembas/services/compaction.py

@@ -98,8 +98,12 @@ def split(
         return [], list(messages)
     boundary = moment(cutoff)
     return (
-        [m for m in messages if moment(m) <= boundary],
-        [m for m in messages if moment(m) > boundary],
+        # A prompt still waiting to be sent stays on the live side whatever its
+        # timestamp says. Folding one into the "earlier messages" details would
+        # hide the only place its Send now and Discard exist, and it has not
+        # been part of any request to summarise.
+        [m for m in messages if not m.queued and moment(m) <= boundary],
+        [m for m in messages if m.queued or moment(m) > boundary],
     )
 
 
@@ -133,6 +137,9 @@ def transcript(db: DBSession, chat: Chat, *, upto: Message) -> str:
         Message.chat_id == chat.id,
         Message.created_at <= upto.created_at,
         Message.error == "",
+        # Not yet sent to anything. Summarising it would fold words the model
+        # has never seen into the record, and then deliver them again later.
+        Message.queued.is_(False),
     )
     if previous is not None:
         query = query.where(Message.created_at > previous.created_at)

src/lembas/services/files.py

@@ -375,6 +375,43 @@ def store_text(
     return attachment
 
 
+def copy_attachment(
+    db: DBSession, *, user_id: str, chat_id: str | None, attachment: Attachment
+) -> Attachment:
+    """Duplicate something already sent, so it can ride along with a new message.
+
+    A copy and not a second reference to one row: an attachment belongs to the
+    message it was sent with, and sharing one between two would make deleting
+    either of them a question rather than an answer.
+    """
+    stored_name = ""
+    source = attachments_dir() / attachment.stored_name if attachment.stored_name else None
+    if source is not None and source.exists():
+        stored_name = f"{secrets.token_hex(16)}{Path(source.name).suffix}"
+        (attachments_dir() / stored_name).write_bytes(source.read_bytes())
+
+    copy = Attachment(
+        user_id=user_id,
+        chat_id=chat_id,
+        filename=attachment.filename,
+        stored_name=stored_name,
+        media_type=attachment.media_type,
+        size_bytes=attachment.size_bytes,
+        kind=attachment.kind,
+        width=attachment.width,
+        height=attachment.height,
+        extracted_text=attachment.extracted_text,
+        pages=attachment.pages,
+        truncated=attachment.truncated,
+        extraction_error=attachment.extraction_error,
+        source_path=attachment.source_path,
+        source_label=attachment.source_label,
+    )
+    db.add(copy)
+    db.commit()
+    return copy
+
+
 def copy_document(
     db: DBSession, *, user_id: str, chat_id: str | None, document
 ) -> Attachment:
@@ -407,6 +444,12 @@ def copy_document(
         pages=document.pages,
         truncated=document.truncated,
         extraction_error=document.extraction_error,
+        # Where it came from, for the same reason a project file carries it: a
+        # model handed four documents cannot tell which is which, and cannot
+        # name one back when asked to work on it. This was the one attach path
+        # that dropped provenance.
+        source_path=(document.title or "")[:1000],
+        source_label=(document.base.name if document.base else "Knowledge")[:200],
     )
     db.add(attachment)
     db.commit()

src/lembas/services/generation.py

@@ -129,6 +129,13 @@ class Generation:
     # the reply and is written onto the message, so the Execute button sends
     # exactly what was proposed rather than something parsed back out of prose.
     plan: dict | None = None
+    # The queue, seen from the reply's side. `drained` says this reply's ending
+    # handed the next waiting prompt to a fresh one; `injected_ids` names the
+    # prompts taken into *this* reply between two rounds of tool calls. Both are
+    # read only by `_follow`, which turns them into bubbles on the `done` frame
+    # -- the one frame that reaches a browser after a reply is over.
+    drained: bool = False
+    injected_ids: list[str] = field(default_factory=list)
 
     def touch(self) -> None:
         self.version += 1
@@ -187,6 +194,22 @@ def answer(
     return False
 
 
+def running_for(chat_id: str) -> Generation | None:
+    """The reply being written in this chat, if there is one.
+
+    A linear scan for the reason `answer` gives above: one entry per reply in
+    flight, consulted at human speed. `_prune` first, because a finished
+    generation lingers `KEEP_FINISHED` so that late followers still get the
+    final frames -- and without the sweep those five minutes would look like a
+    chat that is permanently busy, and queue everything typed into it.
+    """
+    _prune()
+    for generation in _RUNNING.values():
+        if generation.chat_id == chat_id and not generation.done:
+            return generation
+    return None
+
+
 _VERDICTS = (interaction.ALLOW, interaction.ALLOW_ALWAYS, interaction.DENY)
 
 
@@ -328,6 +351,12 @@ async def _run(generation: Generation) -> None:
 
             model = chat_service.model_for(db, chat)
             generation.context_limit = model.context_length if model is not None else 0
+            # Kept for `_inject`, which builds a user turn after this session
+            # has closed. A turn taken in mid-reply has to be shaped exactly as
+            # the same words typed a moment later would have been -- images to a
+            # vision model, a plain string to anything else, or the endpoint
+            # rejects the whole request.
+            vision = chat_service.model_supports(db, chat, "vision")
 
         generation.prompt_estimate = tokens.estimate_request(payload)
 
@@ -406,10 +435,17 @@ async def _run(generation: Generation) -> None:
             if generation.stopped or not calls:
                 break
 
-            if round_number == tools_service.MAX_ROUNDS:
+            if round_number == budget:
                 # Out of rounds with the model still asking for tools. Recorded
                 # rather than silently dropped: an answer that stops here needs
                 # to be explicable.
+                #
+                # `budget`, not `MAX_ROUNDS`. The loop is sized by the budget on
+                # the line above and the message below has always reported it,
+                # but the comparison was against the global 3 -- so an agent
+                # chat allowed forty steps stopped after three and said it had
+                # taken forty. Two numbers, one of them wrong, in code whose
+                # whole job is to say what happened.
                 generation.tool_events.append(
                     {
                         "name": calls[0]["name"],
@@ -454,6 +490,22 @@ async def _run(generation: Generation) -> None:
                     generation.plan = outcome.event["plan"]
             generation.touch()
 
+            # Something typed while this reply was working. Taken in here, at a
+            # round boundary, rather than made to wait for the whole reply: an
+            # agent that has just finished one loop and is about to start
+            # another is exactly when "actually, do it the other way" is worth
+            # having.
+            #
+            # Only while there is a round left to answer in. Injecting into the
+            # last one would deliver the prompt into a reply that then runs out
+            # of budget without addressing it -- and it is marked delivered, so
+            # nothing would ever send it again. Below that line it waits for
+            # `_drain`, which always gives it a reply of its own.
+            if round_number + 1 < budget and (
+                added := _inject(generation, generation.chat_id, vision)
+            ):
+                messages.append(added)
+
             payload = {**payload, "messages": messages}
 
             # A plan ends the turn. One more request so the model can say what
@@ -525,6 +577,11 @@ async def _run(generation: Generation) -> None:
         # the row. The other order left a window in which the finished frame
         # showed the previous turn's stored values.
         _persist(generation, title, time.monotonic() - started)
+        # After the row is authoritative and before `done`, for the same reason
+        # `_persist` is: `_follow` breaks the instant it sees that flag, and the
+        # frame it then sends is the one that has to carry the next turn's
+        # bubbles. There is no push channel that outlives a single reply.
+        _drain(generation)
         generation.done = True
         generation.finished_at = datetime.now(UTC)
         generation.touch()
@@ -1006,6 +1063,117 @@ def _question_from(payload: dict) -> str:
     return ""
 
 
+def _next_waiting(db, chat_id: str) -> Message | None:
+    """The oldest prompt in this chat that has not been sent."""
+    return db.scalars(
+        select(Message)
+        .where(
+            Message.chat_id == chat_id,
+            Message.role == ROLE_USER,
+            Message.queued.is_(True),
+        )
+        .order_by(Message.created_at)
+        .limit(1)
+    ).first()
+
+
+def _drain(generation: Generation) -> None:
+    """Hand the next waiting prompt to a reply of its own, if there is one.
+
+    Exactly one, not all of them. Draining the lot would put two consecutive
+    user turns into the next request, which several local chat templates refuse
+    outright -- `build_messages` already goes to some trouble over that around
+    the compaction lead. "One after another" is also what was asked for: the
+    second waiting prompt is drained by the reply the first one starts, and so
+    on down the chain.
+
+    Three refusals, and none of them is a special case:
+
+    - **Superseded.** The same test `_persist` makes, for the same reason: a
+      regeneration cancels its predecessor and the predecessor's `finally:`
+      still runs. Without this, regenerating would drain the queue *and* leave
+      a third generation running.
+    - **Stopped.** Stop means stop, and the queue stays visible and
+      undelivered with Send now beside it. This is also what makes shutdown
+      safe -- cancellation sets `stopped`, so a restart never fires off a reply
+      with nobody watching.
+    - **Errored.** The endpoint has just failed. Feeding the next prompt into it
+      produces a second failure and spends somebody's words to do it.
+    """
+    owner = _RUNNING.get(generation.message_id)
+    if owner is not None and owner is not generation:
+        return
+    if generation.stopped or generation.error:
+        return
+
+    try:
+        with session_scope() as db:
+            chat = db.get(Chat, generation.chat_id)
+            if chat is None:
+                return
+            waiting = _next_waiting(db, chat.id)
+            if waiting is None:
+                return
+
+            waiting.queued = False
+            assistant = chat_service.create_message(
+                db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
+            )
+            chat_id, assistant_id = chat.id, assistant.id
+    except Exception:  # noqa: BLE001 - the reply is over either way
+        log.exception("could not drain the queue for chat %s", generation.chat_id)
+        return
+
+    # Outside the session: this starts a task, and a task is not something to
+    # hold a database session open across.
+    ensure(chat_id, assistant_id)
+    generation.drained = True
+
+
+def _inject(generation: Generation, chat_id: str, vision: bool) -> dict | None:
+    """Take the oldest waiting prompt into this reply, between two rounds.
+
+    Marked delivered and committed *before* the request goes out, so this is
+    at-most-once. A crash in between loses the turn, which is recoverable --
+    the words are still in the transcript with Send now beside them. The other
+    way round would ask the same question twice and let an agent act on it
+    twice, which is not.
+
+    Sent verbatim, in the user role, with no framing. Everything else this
+    codebase injects is quoted and attributed because it came out of a file, a
+    page or a machine; this one genuinely *is* the person at the keyboard,
+    authenticated by the session cookie and stored as a `Message` whose role
+    says so. Wrapping it would teach a model that a user turn can be a
+    quotation, which is the exact distinction the other two rely on. What the
+    model needs -- that this can happen at all -- is one sentence in the
+    harness, where authored wording lives.
+    """
+    try:
+        with session_scope() as db:
+            waiting = _next_waiting(db, chat_id)
+            if waiting is None:
+                return None
+
+            waiting.queued = False
+            entry = chat_service.message_payload(waiting, vision=vision)
+            # The reply that answers it must sort *before* it, or the next
+            # turn's transcript reads "answer, then the question it answered"
+            # and a small model dutifully answers again. Moving the placeholder
+            # rather than the prompt keeps several interjections in the order
+            # they were typed.
+            placeholder = db.get(Message, generation.message_id)
+            if placeholder is not None:
+                placeholder.created_at = datetime.now(UTC)
+            generation.injected_ids.append(waiting.id)
+    except Exception:  # noqa: BLE001 - a lost interjection is not a failed reply
+        log.exception("could not take a queued prompt into chat %s", chat_id)
+        return None
+
+    generation.status = "Taking in what you just added…"
+    generation.touch()
+    return entry
+
+
 def _persist(generation: Generation, title: str, elapsed: float) -> None:
     """Write the finished reply, name the chat, and set the unread flag.
 

src/lembas/services/prompts.py

@@ -587,6 +587,22 @@ BUILTIN: tuple[Fragment, ...] = (
             "within that budget: two careful searches beat six that run out halfway."
         ),
     ),
+    Fragment(
+        key="core.interjection",
+        label="Being interrupted",
+        group=GROUP_CORE,
+        order=115,
+        when_tools=True,
+        hint="A message typed while you are working is handed to you between two "
+        "rounds of tool calls. Without this a model reads it as a fresh "
+        "conversation and starts the whole task again.",
+        default=(
+            "A new message from the person you are working for can arrive between "
+            "rounds of tool calls, while you are still working. Take it into account "
+            "from that point on. You do not need to start again or to re-explain what "
+            "you have already done — carry on, adjusted."
+        ),
+    ),
     Fragment(
         key="core.no_replay",
         label="Results are not kept",