Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

The terminal learns where one command ends, and can be dragged wider

Browse Source 
"The last command and its output" was not something the panel could honestly
offer. sendToChat took the last forty rows of the screen buffer, hard-wrapped at
the terminal's width with no way to tell a wrap from a newline -- its own comment
said so. So bash and zsh are given the OSC 133 markers VS Code and WezTerm use,
and Copy, Send and an Auto toggle are built on those.

The integration is written by the PTY command string itself, with printf. sshd
runs that string through $SHELL -c, so it can case on the shell's own name and
needs no probe, no second channel and no writable home. Passing it through the
environment does not work -- every distribution ships AcceptEnv LANG LC_*, so
anything else is dropped silently -- and feeding `source ...` in as keystrokes
races a slow .zshrc, echoes into the scrollback and lands in shell history.

Nothing needs hiding, which is the point of choosing it: the setup runs before
the shell exists and never writes to the PTY's input side, so there is nothing
to echo and no fan-out gate to build.

Two things were wrong in the first version and both were found by running it
against real shells rather than the fake one. bash: the DEBUG trap fires before
every simple command *including each one inside PROMPT_COMMAND*, so $? read from
there is whatever ran a moment ago -- every command reported success. The status
is captured in the trap now, which also removes the two-entry PROMPT_COMMAND
dance entirely. zsh: $ZDOTDIR is already ours by the time .zshenv runs, so the
shims were sourcing themselves and none of the user's configuration loaded; the
original is passed on the exec line.

Parsing is server-side. The `behind` path resets the terminal and replays a
truncated scrollback, so a client parser routinely sees a finish with no start;
two tabs share one shell and can disagree; and what comes out of this ends up
inside a prompt, so deriving it here leaves nothing to disbelieve. The bytes are
fanned out unchanged -- xterm consumes an OSC it has no handler for.

Output is bounded head and tail, 48KB and 16KB: a build that fails ten megabytes
in has the invocation at the top and the error at the bottom. Carriage returns
collapse to the last state of each line, which is the difference between a
usable prompt and two megabytes of spinner. The fence is sized to its content,
because output containing three backticks would otherwise break out and read as
prose.

Any shell that is not bash or zsh starts exactly as it did before. The buttons
then scrape the screen and say so, and Auto is disabled rather than degraded:
forty arbitrary lines on every message is worse than nothing.

Also a generic [data-resize] handle, keyboard included, persisted the way the
theme is. The inspector and sidebar can have it whenever they want it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-02 17:28:38 +02:00
parent b6cea42631
commit 131a4083f8
19 changed files with 1854 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+94 -10
View File
@@ -107,7 +107,9 @@ src/lembas/
library/ documents, notes, memories, skills, FTS library/ documents, notes, memories, skills, FTS
mcp/ remote MCP servers: framing, transport, rows to tools mcp/ remote MCP servers: framing, transport, rows to tools
agent/ agent chats: the mode table, SSH, the four tools, agent/ agent chats: the mode table, SSH, the four tools,
and terminal.py, the shells held open behind the panel terminal.py (shells held open behind the panel),
shell_marks.py + capture.py (where one command ends),
index.py (what is in the project directory)
audio.py OpenAI-shaped /v1/audio/* client audio.py OpenAI-shaped /v1/audio/* client
fetch.py URL retrieval, HTML to text, the SSRF guard fetch.py URL retrieval, HTML to text, the SSRF guard
sharing.py one visibility rule for every library store sharing.py one visibility rule for every library store
@@ -133,6 +135,8 @@ src/lembas/
templating.py render() -- always use this, not TemplateResponse templating.py render() -- always use this, not TemplateResponse
templates/ Jinja templates/ Jinja
static/ css, js, vendor, img, sw.js static/ css, js, vendor, img, sw.js
js/commands.js the / table, and the keyboard that does the same jobs
js/composer.js the menu that / and @ open, over the message box
assets/ SVG masters and PWA icons (generated) assets/ SVG masters and PWA icons (generated)
deploy/ systemd unit, nginx vhost, install/update scripts deploy/ systemd unit, nginx vhost, install/update scripts
``` ```
@@ -140,8 +144,9 @@ deploy/ systemd unit, nginx vhost, install/update scripts
## Things that will bite you ## Things that will bite you
**`render()`, not `TemplateResponse`.** `web/templating.py:render()` injects **`render()`, not `TemplateResponse`.** `web/templating.py:render()` injects
`user`, `theme`, `version` and `allow_signup`. Templates assume they exist. If `user`, `theme`, `layout`, `version` and `allow_signup`. Templates assume they
you must call `templates.TemplateResponse` directly (the SSE path does, because exist. If you must call `templates.TemplateResponse` directly (the SSE path does,
because
there is no `Request`), pass `user` explicitly — `chat/_message.html` renders there is no `Request`), pass `user` explicitly — `chat/_message.html` renders
both roles and the user branch dereferences it. both roles and the user branch dereferences it.
@@ -377,6 +382,41 @@ no longer in the transcript are still there. Nothing tries to undo them: the
project directory is somebody's real working tree, and deleting their work to project directory is somebody's real working tree, and deleting their work to
match would be far worse than the inconsistency. match would be far worse than the inconsistency.
**The project listing is read from a cache and never fetched.**
`harness.context_variables` runs synchronously on the request path, so
`agent/index.py:cached()` is all it may call — an SFTP round trip from there
would hold a request open while somebody's box thought about it. The walk
happens in `generation._warm_index`, which is async and already doing network
work, with a short wait. A chat whose first reply outruns its first walk simply
has no listing that turn, and the fragment's `requires` makes it vanish rather
than appear as an empty heading. Anything else wanting the listing gets the same
deal: the `@` picker offers no files until one exists, because a keystroke must
never wait on a machine.
**A listing is budgeted, not dumped.** A tree of a thousand files costs the
window on every request forever and buries the four names that mattered.
`index.render` collapses what will not fit to `src/vendor/ (412 files)` and says
so. Collapsing picks the **deepest and largest first**: by saving alone it would
take `src/` before `src/web/static/vendor/`, because it contains it, and lose
every name worth having. Watch the double-count — collapsing a parent subsumes a
child already collapsed, and adding both savings stops the loop early believing
it has made room it has not.
**A slash command must never swallow a message.** `static/js/commands.js`
intercepts only an exact match against its table; `//` escapes, and anything
unrecognised is sent as written. Eating somebody's message because it began with
a slash is a far worse failure than an unknown command, and it is the one the
implementation has to be arranged around rather than patched for afterwards.
**`@` inserts a reference *and* attaches the contents.** The token stays in the
sentence so "change the thing in @main.py" reads as one, and the file arrives as
an attachment chip — the same component every other attach path returns, so the
composer learns nothing new. `Attachment.source_path` and `source_label` carry
where it came from into `chat.document_context`'s tag, because a model handed
`main.py` cannot tell which of four it is looking at and cannot name it back when
asked to change something. Those two are attribute values in a tag we write, so
`_attr` strips quotes and angle brackets rather than escaping them.
**Unread is polled, not pushed.** A browser on another chat has no connection **Unread is polled, not pushed.** A browser on another chat has no connection
to the one that finished. `/api/chats/unread` returns out-of-band dot spans and to the one that finished. `/api/chats/unread` returns out-of-band dot spans and
an `HX-Trigger` for the toast; `unread_notified` stops the same arrival being an `HX-Trigger` for the toast; `unread_notified` stops the same arrival being
@@ -700,13 +740,57 @@ construction, while decoding each frame server-side would corrupt every
boundary. Only `resize`, `ready`, `closed` and `error` are text, and they are boundary. Only `resize`, `ready`, `closed` and `error` are text, and they are
JSON. JSON.
**The modes do not govern the keyboard.** `agent/policy.py` exists because a **The modes do not govern the keyboard, and now there are three exceptions, not
model reads pages, files and command output it did not write and can be talked one.** `agent/policy.py` exists because a model reads pages, files and command
into things. A person typing into the panel holds the credential already and output it did not write and can be talked into things. A person typing into the
could open the same shell with an ssh client, so nothing they type is checked terminal panel holds the credential already and could open the same shell with
against the mode or the two lists. There is a test named after this, because it an ssh client, so nothing they type is checked against the mode or the two
reads like a bug next to `policy.py` and "fixing" it would make the panel lists. The directory browser (`GET /api/agents/{id}/browse`) and the project
useless in the mode people spend the most time in. listing (`agent/index.py`) are the same argument again: both are read-only, both
are LLeMbas acting on somebody's instruction rather than a model choosing to,
and both would be pointless if they asked. But it does mean **Manual** mode's
"everything is shown to you before it happens" is now true of the *model* and
not of the interface, and that is worth saying out loud rather than discovering.
There is a test named after the first one, because it reads like a bug next to
`policy.py` and "fixing" it would make the panel useless in the mode people
spend the most time in.
**A control wired to a method its route does not serve fails silently.** The
agent-mode select posted with `hx-post` against a route that only answers
`PATCH`, so every change returned 405 and the mode never moved — for the whole
life of the feature. htmx surfaces nothing on a failed request, so the select
stayed where it was put and the server ignored it, which looks exactly like
working. `tests/test_agent_mode.py` asserts the method is *refused* as well as
that the right one works, because only the second half would have passed
throughout. When adding a control that writes, check the verb against the route,
and assert on the row rather than on the response.
**Shell integration is best-effort, and the fallback is the point.**
`agent/shell_marks.py` gives bash and zsh hooks that emit OSC 133 around the
prompt, the command and its result, so the panel can say what "the last command
and its output" means. Three things about it:
- **It is written by the PTY command string itself**, with `printf`. sshd runs
that string through `$SHELL -c`, so it can `case` on the shell's own name and
needs no probe, no second channel and no writable `$HOME`. Environment
variables do not work — every distribution ships `AcceptEnv LANG LC_*`, so
anything else is dropped silently — and feeding `source …` in as keystrokes
races a slow `.zshrc`, echoes, and lands in shell history.
- **Nothing needs hiding.** The setup runs before the shell exists and never
writes to the PTY's *input* side, so there is nothing to echo and no fan-out
gate. That is why this mechanism was chosen over the one that looks obvious.
- **The exit status is captured in the `DEBUG` trap, not in `PROMPT_COMMAND`.**
DEBUG fires before every simple command *including each one inside
`PROMPT_COMMAND`*, so `$?` read from there is whatever ran a moment ago. This
was wrong in the first version and every command reported success. zsh has the
mirror-image trap: `$ZDOTDIR` is already ours by the time `.zshenv` runs, so
the user's own must be passed on the exec line or the shims source themselves
and none of somebody's configuration loads.
Any shell that is not bash or zsh gets exactly the command that ran before, and
therefore no markers — at which point Copy and Send fall back to scraping the
screen and say so, and the automatic toggle is **disabled rather than degraded**.
Forty arbitrary lines attached to every message is worse than nothing attached.
**The nginx vhost must pass upgrades through.** `deploy/nginx-vhost.conf` used **The nginx vhost must pass upgrades through.** `deploy/nginx-vhost.conf` used
to set `Connection ""`, which is right for SSE and fails every WebSocket to set `Connection ""`, which is right for SSE and fails every WebSocket
PLAN.md
+26 -1
View File
@@ -7,7 +7,7 @@ that would be expensive to revisit. Kept current as work lands; the detail of
**Status:** usable daily. Streaming chat, attachments, reasoning, tool calling **Status:** usable daily. Streaming chat, attachments, reasoning, tool calling
with web search, custom HTTP tools and MCP servers, agent chats that work on a with web search, custom HTTP tools and MCP servers, agent chats that work on a
machine over SSH, a knowledge library, notes, memory and skills, speech in and machine over SSH, a knowledge library, notes, memory and skills, speech in and
out, users and groups, model administration, installable as an app. 837 tests, out, users and groups, model administration, installable as an app. 981 tests,
`ruff` clean. `ruff` clean.
--- ---
@@ -120,6 +120,22 @@ be a different project, not a refactor.
running, and coming back reattaches with the scrollback. An idle timeout running, and coming back reattaches with the scrollback. An idle timeout
is what eventually ends one, and so does deleting the chat, or disabling, is what eventually ends one, and so does deleting the chat, or disabling,
moving or deleting the connection moving or deleting the connection
- [x] **The panel is resizable**, dragged from its edge or nudged with the
arrow keys, and the width follows you to another browser
- [x] **It knows where one command ends and the next begins** — bash and zsh
are given the markers VS Code and WezTerm use, so *Copy* and *Send* mean
one command and its output rather than the last forty rows of the screen.
An **Auto** toggle collects each one into the next message. Any other
shell starts exactly as it did before, the buttons fall back to the
screen and say so, and Auto is disabled rather than degraded
- [x] **The project directory is listed for the model** — one read-only
command, `git ls-files` where that works so `.gitignore` is honoured for
free, budgeted so a big directory becomes a count rather than a thousand
filenames on every request
- [x] **A directory is chosen by browsing it** over SFTP, not by typing a path
into an unlabelled box
- [x] The approval mode is chosen **before** the first message, beside the
message box rather than in the header
### The library ### The library
- [x] **Knowledge bases** — documents, images and saved web pages, grouped into - [x] **Knowledge bases** — documents, images and saved web pages, grouped into
@@ -143,6 +159,10 @@ be a different project, not a refactor.
actually has, so the tools get used rather than ignored actually has, so the tools get used rather than ignored
- [x] Attach menu: file, image, a web page fetched on the spot, or a document - [x] Attach menu: file, image, a web page fetched on the spot, or a document
from the library from the library
- [x] **`@` to name one** — the library everywhere, and files in the project
directory in an agent chat. The reference stays in the sentence and the
contents come along, with the path and the machine, so the model knows
exactly which file it was handed
### Audio ### Audio
- [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped - [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped
@@ -193,6 +213,11 @@ be a different project, not a refactor.
- [x] Admin-managed cards on the new-chat screen; three seeded once at startup - [x] Admin-managed cards on the new-chat screen; three seeded once at startup
### Interface ### Interface
- [x] **`/` for commands** — compact, usage, mode, model, title, the panels,
the theme. Anything not in the table is sent as an ordinary message, and
`//` starts one with a literal slash
- [x] **Keyboard shortcuts** for the same jobs, listed beside the commands in
one table so `/help` cannot go stale
- [x] **Installable** — manifest, generated PWA icons, a service worker for the - [x] **Installable** — manifest, generated PWA icons, a service worker for the
shell and a themed offline page. The worker deliberately never touches shell and a themed offline page. The worker deliberately never touches
`/api/`: a reply is an event stream and caching one breaks it `/api/`: a reply is an event stream and caching one breaks it
README.md
+45 -6
View File
@@ -46,6 +46,13 @@ runtime. Clone it, `pip install -e .`, run it.
- **Attachments** — drag, paste or pick images, PDFs and text files. Images are - **Attachments** — drag, paste or pick images, PDFs and text files. Images are
downscaled and sent to vision models; PDF and text content is extracted and downscaled and sent to vision models; PDF and text content is extracted and
put in the prompt put in the prompt
- **`@` to name something** — a document from your library, or in an agent chat
a file in the project directory. The reference stays in the sentence you are
writing and the contents come with it
- **`/` for commands** — `/compact`, `/usage`, `/mode plan`, `/model`,
`/title`, `/terminal`, `/theme`. `/help` lists them and the keyboard
shortcuts beside them. A message that merely starts with a slash is still
sent as written
- **Folders** — arbitrarily nested, delete a folder without losing the chats - **Folders** — arbitrarily nested, delete a folder without losing the chats
inside it inside it
- **Web search** — offered to the model as a tool it calls when a question needs - **Web search** — offered to the model as a tool it calls when a question needs
@@ -178,9 +185,10 @@ fingerprint with nothing sent — not your username, not your key — and only
accepting pins it. If that host later answers with a different key, it is accepting pins it. If that host later answers with a different key, it is
refused rather than quietly trusted. refused rather than quietly trusted.
Then start a chat with the **Agent** toggle, pick the connection and a Then start a chat with the **Agent** toggle, pick the connection, browse to a
directory, and choose a mode. The mode is in the chat header and changes at any directory, and choose a mode — all of it under the message box, before you send
time: anything. The connection and the directory are fixed once the chat exists; the
mode changes at any time and stays where you chose it:
| | Reads | Writes files | Runs commands | | | Reads | Writes files | Runs commands |
|---|---|---|---| |---|---|---|---|
@@ -198,6 +206,23 @@ with. In **Auto**, nothing stands between that and a command running.
switches to *Edit*, never *Auto*, because the plan was written under a mode switches to *Edit*, never *Auto*, because the plan was written under a mode
where every command still asked. where every command still asked.
#### What the model knows about the directory
An agent chat starts by listing the project directory, so a reply does not spend
its first rounds finding out what is there. It is one read-only command —
`git ls-files` in a repository, so `.gitignore` is honoured for free, otherwise
`find` with the usual noise pruned — and it is cached and shared by every chat
pointed at the same place.
What reaches the model is budgeted rather than dumped: a directory that will not
fit is shown as `node_modules/ (4,102 files)` and the model is told to open it
itself if it needs to. **Admin → Agents** sets the budget, and `0` keeps the
listing for the `@` picker while putting none of it in the prompt.
Listing a directory and browsing one are things *you* asked for, not things a
model chose, so neither goes through the modes above. Worth knowing if you read
**Manual** as "nothing happens without me": it means nothing the *model* does.
#### The terminal #### The terminal
An agent chat has a **Terminal** button in its header, which opens a real shell An agent chat has a **Terminal** button in its header, which opens a real shell
@@ -207,9 +232,23 @@ the *Open a terminal* permission, which is off by default.
The modes above do not apply to it. They exist because a model reads pages, The modes above do not apply to it. They exist because a model reads pages,
files and command output it did not write; you hold the credential and could files and command output it did not write; you hold the credential and could
open the same shell with an ssh client, so nothing you type is queued for your open the same shell with an ssh client, so nothing you type is queued for your
own approval. The model cannot see the panel either — the button in its header own approval. The model cannot see the panel either — three buttons in its
puts the selection, or the last of the output, into the message box, where you header decide what it sees: **Copy** takes the last command and its output to
read it before it goes anywhere. the clipboard, **Send** puts the same into the message box, and **Auto**
collects every command you run into your next message. Nothing is ever sent on
its own; the box is where you read it first.
Knowing what "the last command" means takes a little help from the shell.
LLeMbas gives bash and zsh the same invisible markers VS Code and WezTerm use,
written into a temporary file the shell deletes itself, so it can tell one
command's output from the next and record the exit status and the directory.
Your own dotfiles are loaded first and nothing of yours is skipped. Any other
shell starts exactly as it would have; the two buttons then copy the last of the
screen as it appeared, say so, and Auto is switched off rather than guessing.
Drag the panel's left edge to make it wider — a terminal narrower than eighty
columns re-wraps everything a program prints — and the width follows you to
another browser.
The shell is not tied to the panel. Close it and a build carries on; come back, The shell is not tied to the panel. Close it and a build carries on; come back,
or reload, and you reattach with the scrollback. Two tabs share one shell, and or reload, and you reattach with the scrollback. Two tabs share one shell, and
src/lembas/api/admin_agents.py
+2
View File
@@ -72,6 +72,7 @@ async def save_agents(
terminal_idle_timeout: int = Form(1800), terminal_idle_timeout: int = Form(1800),
terminal_max_sessions: int = Form(20), terminal_max_sessions: int = Form(20),
terminal_max_per_user: int = Form(3), terminal_max_per_user: int = Form(3),
terminal_integration: bool = Form(False),
index_enabled: bool = Form(False), index_enabled: bool = Form(False),
index_chars: int = Form(2000), index_chars: int = Form(2000),
) -> Response: ) -> Response:
@@ -96,6 +97,7 @@ async def save_agents(
"terminal_idle_timeout": min(max(terminal_idle_timeout, 60), 86400), "terminal_idle_timeout": min(max(terminal_idle_timeout, 60), 86400),
"terminal_max_sessions": min(max(terminal_max_sessions, 1), 500), "terminal_max_sessions": min(max(terminal_max_sessions, 1), 500),
"terminal_max_per_user": min(max(terminal_max_per_user, 1), 50), "terminal_max_per_user": min(max(terminal_max_per_user, 1), 50),
"terminal_integration": terminal_integration,
"index_enabled": index_enabled, "index_enabled": index_enabled,
# Zero is kept rather than clamped up: it means "list the # Zero is kept rather than clamped up: it means "list the
# directory for the file picker but put none of it in the # directory for the file picker but put none of it in the
src/lembas/api/preferences.py
+38
View File
@@ -38,6 +38,44 @@ async def set_theme(db: Db, user: RequiredUser, theme: str = Body(..., embed=Tru
return {"ok": True, "theme": theme} return {"ok": True, "theme": theme}
# Which CSS variables a browser is allowed to set from here, and how far. An
# open dict would let a page store anything under somebody's account and have
# it read back on every load; a width outside these bounds would hand them a
# panel they cannot see to drag back.
LAYOUT_BOUNDS = {
"--terminal-width": (384, 2400),
"--inspector-width": (280, 2400),
"--sidebar-width": (200, 800),
}
@router.post("/layout")
async def set_layout(db: Db, user: RequiredUser, widths: dict = Body(...)) -> dict:
"""Remember how wide somebody dragged the panels.
Same two tiers as the theme: `localStorage` is the truth for the tab that
did the dragging, and this is what carries it to another browser. Unknown
names are dropped rather than refused -- an older browser sending a key a
newer release removed should not fail the request.
"""
kept: dict[str, int] = {}
for name, raw in (widths or {}).items():
bounds = LAYOUT_BOUNDS.get(str(name))
if bounds is None:
continue
try:
value = int(float(raw))
except (TypeError, ValueError):
continue
kept[str(name)] = min(max(value, bounds[0]), bounds[1])
settings = {**(user.settings_json or {})}
settings["layout"] = {**(settings.get("layout") or {}), **kept}
user.settings_json = settings
db.commit()
return {"ok": True, "layout": kept}
@router.post("/default-model") @router.post("/default-model")
async def set_default_model( async def set_default_model(
db: Db, user: RequiredUser, model_id: str = Form("") db: Db, user: RequiredUser, model_id: str = Form("")
src/lembas/api/terminal.py
+76 -1
View File
@@ -27,8 +27,9 @@ import json
import logging import logging
from urllib.parse import urlsplit from urllib.parse import urlsplit
from fastapi import APIRouter, WebSocket, WebSocketDisconnect from fastapi import APIRouter, HTTPException, WebSocket, WebSocketDisconnect, status
from lembas.api.deps import Db, RequiredUser
from lembas.db.models import KIND_AGENT, Chat from lembas.db.models import KIND_AGENT, Chat
from lembas.db.session import session_scope from lembas.db.session import session_scope
from lembas.security import permissions from lembas.security import permissions
@@ -111,6 +112,7 @@ def _prepare(db, user, chat_id: str) -> tuple[str, dict]:
"idle_timeout": float(values["terminal_idle_timeout"]), "idle_timeout": float(values["terminal_idle_timeout"]),
"max_sessions": int(values["terminal_max_sessions"]), "max_sessions": int(values["terminal_max_sessions"]),
"max_per_user": int(values["terminal_max_per_user"]), "max_per_user": int(values["terminal_max_per_user"]),
"integrate": bool(values.get("terminal_integration", True)),
} }
@@ -148,6 +150,13 @@ async def terminal_socket(
await _refuse(websocket, "The shell could not be started.") await _refuse(websocket, "The shell could not be started.")
return return
# Shaping a frame is this layer's job, not the session's; the session only
# knows it finished something. Reassigned per socket and harmless: every
# socket on this session would build the identical frame.
session.on_command = lambda found: session.announce(
json.dumps({"t": "command", "command": _command_frame(found)})
)
viewer = session.attach(cols, rows) viewer = session.attach(cols, rows)
await websocket.send_text( await websocket.send_text(
json.dumps( json.dumps(
@@ -160,6 +169,10 @@ async def terminal_socket(
# Two tabs share one shell, and a size neither of them chose is # Two tabs share one shell, and a size neither of them chose is
# otherwise a mystery. # otherwise a mystery.
"shared": len(session.viewers) > 1, "shared": len(session.viewers) > 1,
# Whether this shell will tell us where commands begin and end,
# which is what the Copy and Send buttons are made of.
"integration": session.integration,
"last": _command_frame(session.latest()),
} }
) )
) )
@@ -182,6 +195,63 @@ async def terminal_socket(
session.detach(viewer) session.detach(viewer)
@router.get("/{chat_id}/terminal/last")
async def last_command(db: Db, user: RequiredUser, chat_id: str) -> dict:
"""The last command and its output, rendered ready to paste.
The *server* renders the text, so Copy and Send are a fetch and a
clipboard write with no formatting logic in the browser -- and the block a
model eventually reads exists in exactly one place. The panel's own screen
buffer could not produce it anyway: it holds what is on screen, hard-wrapped
at the terminal's width, with no way to tell a wrap from a newline.
"""
chat = db.get(Chat, chat_id)
if chat is None or chat.user_id != user.id:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
if not permissions.has(db, user, "agent.terminal"):
raise HTTPException(status.HTTP_403_FORBIDDEN, "You cannot open a terminal.")
session = terminal_service.get(chat_id)
found = session.latest() if session is not None else None
if session is None or found is None:
return {
"ok": False,
"message": "Nothing has been run in this shell yet."
if session is not None
else "This terminal is not open.",
}
return {
"ok": True,
"command": found.command,
"cwd": found.cwd,
"exit": found.exit_status,
"running": found.running,
"summary": found.summary(),
"text": found.as_text(label=session.label),
}
def _command_frame(found) -> dict | None:
"""A finished command, small enough to push at every viewer.
Tens of bytes, and deliberately *not* the output: a 64KB text frame would
compete with PTY bytes on the one path that has to stay responsive, and the
two buttons are pressed by a person, where a request is the natural shape.
"""
if found is None:
return None
return {
"seq": found.seq,
"command": found.command,
"cwd": found.cwd,
"exit": found.exit_status,
"running": found.running,
"ms": found.duration_ms,
"summary": found.summary(),
}
async def _to_browser(websocket: WebSocket, session, viewer) -> None: async def _to_browser(websocket: WebSocket, session, viewer) -> None:
"""Everything the shell says, plus the one frame that says it stopped.""" """Everything the shell says, plus the one frame that says it stopped."""
while True: while True:
@@ -198,6 +268,11 @@ async def _to_browser(websocket: WebSocket, session, viewer) -> None:
with contextlib.suppress(Exception): with contextlib.suppress(Exception):
await websocket.send_text(json.dumps(payload)) await websocket.send_text(json.dumps(payload))
return return
# A string in the queue is a control frame that had to keep its place
# in the stream -- see `Session.announce`.
if isinstance(chunk, str):
await websocket.send_text(chunk)
continue
await websocket.send_bytes(chunk) await websocket.send_bytes(chunk)
src/lembas/services/agent/capture.py
+164
View File
@@ -0,0 +1,164 @@
"""One command and its output, kept so it can be handed to a model.
Bounded at both ends rather than only the front. A build that fails ten
megabytes in has the invocation and the configuration at the top and the error
at the bottom, and either half alone is the wrong half.
Raw bytes are kept and decoded only when somebody asks. Head/tail slicing
splits UTF-8 characters at will, and `base.clean_output` decodes with
`errors="replace"`, which is exactly the right handling -- decoding eagerly per
chunk would be the same mistake the terminal pump already avoids.
"""
from __future__ import annotations
import re
import time
from collections import deque
from dataclasses import dataclass, field
from lembas.services.agent.base import clean_output
# What one command's output may keep, at each end.
CAPTURE_HEAD_BYTES = 48 * 1024
CAPTURE_TAIL_BYTES = 16 * 1024
# The command line itself. Longer than any command and shorter than a paste.
CAPTURE_COMMAND_BYTES = 4 * 1024
# One line of output. A minified bundle on one line is not worth keeping whole.
MAX_LINE_CHARS = 2000
# C0 except tab and newline, and the C1 block. Not in `clean_output`, which
# `shell_run` shares: there a control character inside a file's contents is
# data. Here it is a terminal being driven.
_CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
def flatten(text: str) -> str:
"""What the screen would have shown, from what the wire carried.
The highest-value transform here by a distance. A progress bar redraws
itself by returning to the start of the line and writing again; keeping
every state turns two megabytes of `pip install` into two megabytes of
spinner in somebody's prompt. Only the last state of a line was ever
visible, so only the last state is kept.
"""
lines = []
for line in text.replace("\r\n", "\n").split("\n"):
if "\r" in line:
line = line.rsplit("\r", 1)[-1]
lines.append(_CONTROLS.sub("", line)[:MAX_LINE_CHARS])
return "\n".join(lines).strip("\n")
def fenced(text: str) -> str:
"""A fence long enough that the content cannot end it early.
Output containing three backticks would otherwise break out, and everything
after it would read to the model as prose rather than as what a machine
printed. That is a real injection route and it costs one line to close.
"""
longest = max((len(run) for run in re.findall(r"`+", text)), default=0)
ticks = "`" * max(3, longest + 1)
return f"{ticks}console\n{text}\n{ticks}"
@dataclass
class Capture:
"""A command, and as much of its output as is worth keeping."""
seq: int = 0
command: str = ""
cwd: str = ""
started: float = field(default_factory=time.monotonic)
ended: float = 0.0
exit_status: int | None = None # None while it is still running
head: bytearray = field(default_factory=bytearray)
tail: deque[bytes] = field(default_factory=deque)
tail_bytes: int = 0
dropped: int = 0
total: int = 0
@property
def running(self) -> bool:
return self.exit_status is None
@property
def duration_ms(self) -> int:
end = self.ended or time.monotonic()
return int((end - self.started) * 1000)
def absorb(self, chunk: bytes) -> None:
"""Keep the front, keep the back, count what fell out of the middle."""
self.total += len(chunk)
if len(self.head) < CAPTURE_HEAD_BYTES:
take = CAPTURE_HEAD_BYTES - len(self.head)
self.head += chunk[:take]
chunk = chunk[take:]
if not chunk:
return
self.tail.append(chunk)
self.tail_bytes += len(chunk)
while self.tail_bytes > CAPTURE_TAIL_BYTES and len(self.tail) > 1:
gone = self.tail.popleft()
self.tail_bytes -= len(gone)
self.dropped += len(gone)
def output(self) -> str:
"""The kept output as text, with the gap marked if there is one."""
head = flatten(clean_output(bytes(self.head), limit=CAPTURE_HEAD_BYTES * 2)[0])
if not self.dropped and not self.tail:
return head
tail = flatten(clean_output(b"".join(self.tail), limit=CAPTURE_TAIL_BYTES * 2)[0])
if not self.dropped:
return f"{head}\n{tail}" if tail else head
gap = f"\n\n… {self.dropped / 1024:,.0f} KB dropped …\n\n"
return f"{head}{gap}{tail}"
def as_text(self, *, label: str) -> str:
"""The block that goes into a message, attribution and all.
The sentence sits **outside** the fence and is written here, so nothing
the far side printed can forge it, and the `$ ` line is synthesised
rather than lifted from the shell -- what the shell echoed carries
readline's editing escapes and is not the command.
"""
where = f", in {self.cwd}" if self.cwd else ""
if self.running:
how = "still running"
elif self.exit_status:
how = f"exit {self.exit_status}"
else:
how = "succeeded"
seconds = self.duration_ms / 1000
took = f" after {seconds:.0f}s" if seconds >= 1 else ""
body = f"$ {self.command}\n{self.output()}".rstrip()
return (
f"Ran in the terminal on {label}{where} — {how}{took}:\n\n{fenced(body)}"
)
def summary(self) -> str:
"""A short label for a chip, never rendered as markup."""
command = self.command or "(no command)"
if len(command) > 60:
command = command[:57] + "…"
if self.running:
return f"{command} · running"
return f"{command} · exit {self.exit_status}"
def trim_command(raw: str) -> str:
text, _ = clean_output(raw, limit=CAPTURE_COMMAND_BYTES)
return _CONTROLS.sub("", text).strip()
__all__ = [
"CAPTURE_COMMAND_BYTES",
"CAPTURE_HEAD_BYTES",
"CAPTURE_TAIL_BYTES",
"Capture",
"fenced",
"flatten",
"trim_command",
]
src/lembas/services/agent/shell_marks.py
+401
View File
@@ -0,0 +1,401 @@
"""Knowing where one command ends and the next begins, in the terminal panel.
Without this the panel can offer "the last forty rows of the screen", which is
hard-wrapped at the terminal's width with no way to tell a wrap from a newline.
That is not something to hand a model and call it the output of a command.
So the shell is given hooks that emit invisible markers around the prompt, the
command and its result -- OSC 133, which is what VS Code, WezTerm and Ghostty
all use, plus two of VS Code's private codes for the things 133 has no room
for. A real terminal that understands 133 is not confused by ours, and one that
does not consumes and discards them, which is why the bytes are fanned out to
the browser unchanged.
**Three things about the mechanism.**
The integration is written by the PTY command string itself, with `printf`.
sshd runs that string through `$SHELL -c`, so it can branch on the shell's own
name and needs no probe, no second channel and no writable `$HOME`. Passing it
through the environment does not work: sshd's `AcceptEnv` is `LANG LC_*` on
every distribution anybody runs, so the variable is dropped silently -- and
bash reads `$BASH_ENV` only when non-interactive anyway. Feeding `source …` in
as keystrokes does work, and races a slow `.zshrc`, echoes into the scrollback,
and lands in shell history with no portable way to remove it.
Nothing needs hiding, and that is the point of choosing this mechanism. The
setup runs before the shell exists and never writes to the PTY's *input* side,
so there is nothing for the tty to echo. The first byte a viewer sees is the
first byte of their own prompt.
There is deliberately no `133;B`. It has to live at the end of `PS1`, and any
theme that rebuilds the prompt in a hook drops it silently every time. Every
marker here comes from a shell hook instead, so none of them depends on a
prompt string surviving somebody's dotfiles.
**Markers are advisory.** A program can print `\\e]133;D;0\\a` and move a
boundary. That is not a security problem -- the captured text is sanitised and
fenced either way, and a program could already print anything on screen -- but
nobody should later try to "validate" them.
"""
from __future__ import annotations
import re
from collections.abc import Callable
# Everything ours is under these two. 133 is FinalTerm's de-facto convention;
# 633 is VS Code's private space, borrowed because the raw stream carries the
# *echoed* command with readline's editing escapes in it and is not
# recoverable, and there is no standard code for "here is the command line".
MARK_PROMPT = "A" # 133;A -- a prompt is about to be drawn
MARK_OUTPUT = "C" # 133;C -- output starts here
MARK_DONE = "D" # 133;D;<exit>
MARK_COMMAND = "E" # 633;E;<escaped command>
MARK_CWD = "P" # 633;P;Cwd=<escaped path>
MARK_READY = "LEMBAS" # 633;LEMBAS;<shell>;1
# A marker longer than this is not one of ours. `cat` of a binary file produces
# stray ESC ] regularly, and without a bound one of them would swallow the rest
# of the session into a buffer that never emptied.
MAX_MARKER_BYTES = 8 * 1024
_ESC = 0x1B
_BEL = 0x07
# --- The snippets ------------------------------------------------------------
# `__lembas_esc` exists because an OSC payload may contain no BEL and no ESC
# (either would end it ambiguously) and no ';' (which would split the fields).
# Everything else rides through. It is also what lets `base.clean_output`'s
# existing OSC pattern swallow a whole marker in one bite.
BASH_RC = r"""
# LLeMbas shell integration.
#
# --rcfile replaces bash's normal startup files rather than adding to them, so
# bash's login sequence is reproduced here, in bash's own order, and nothing
# anybody has in a dotfile is skipped.
if [ -r /etc/profile ]; then . /etc/profile; fi
for __lembas_rc in "$HOME/.bash_profile" "$HOME/.bash_login" "$HOME/.profile"; do
if [ -r "$__lembas_rc" ]; then . "$__lembas_rc"; break; fi
done
unset __lembas_rc
if [ -r "$HOME/.bashrc" ]; then . "$HOME/.bashrc"; fi
__lembas_esc() {
local s=${1//\\/\\\\}
s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
builtin printf '%s' "$s"
}
__lembas_emit() {
if [ -n "$__lembas_running" ]; then
builtin printf '\e]133;D;%s\a' "${__lembas_last:-0}"
__lembas_running=
__lembas_have=
fi
builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc "$PWD")"
builtin printf '\e]133;A\a'
__lembas_armed=1
}
# $BASH_COMMAND is the current *simple* command, so `a | b` would give "a".
# `history 1` is the whole line as typed, which is what somebody would
# recognise; it falls back when history is off.
__lembas_line() {
local h
h=$(HISTTIMEFORMAT='' builtin history 1 2>/dev/null) || {
builtin printf '%s' "$BASH_COMMAND"; return; }
if [[ $h =~ ^[[:space:]]*[0-9]+[[:space:]]+(.*)$ ]]; then
builtin printf '%s' "${BASH_REMATCH[1]}"
else
builtin printf '%s' "$BASH_COMMAND"
fi
}
# The exit status is captured *in the DEBUG trap*, not in PROMPT_COMMAND.
#
# This is the one genuinely subtle thing in the file. DEBUG fires before every
# simple command -- including each command inside PROMPT_COMMAND -- so anything
# reading $? from there has already had it overwritten by whatever ran a moment
# earlier, and by this trap's own command substitution. The first DEBUG firing
# after the user's command is the last place the real status exists, so it is
# taken there and held until the prompt emits it.
#
# `__lembas_have` is what stops the later firings (the ones inside
# PROMPT_COMMAND) overwriting it, and `return $__s` puts $? back so nothing
# downstream sees a status this trap invented.
__lembas_debug() {
local __s=$?
if [ -n "$__lembas_running" ] && [ -z "$__lembas_have" ]; then
__lembas_last=$__s
__lembas_have=1
fi
if [ -n "$__lembas_armed" ]; then
__lembas_armed=
builtin printf '\e]633;E;%s\a' "$(__lembas_esc "$(__lembas_line)")"
builtin printf '\e]133;C\a'
__lembas_running=1
fi
return $__s
}
trap '__lembas_debug' DEBUG
# Appended, so anything already there still runs and runs first.
if [ -n "${BASH_VERSINFO[0]}" ] && [ "${BASH_VERSINFO[0]}" -ge 5 ] \
&& [ "${PROMPT_COMMAND@a}" = "a" ]; then
PROMPT_COMMAND=("${PROMPT_COMMAND[@]}" __lembas_emit)
else
PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND; }__lembas_emit"
fi
builtin printf '\e]633;LEMBAS;bash;1\a'
# Self-deleting: bash has read the whole file by the time this runs, and one
# left in /tmp that a later shell might source is worse than any benefit.
rm -rf -- "$(dirname -- "${BASH_SOURCE[0]}")" 2>/dev/null
"""
# zsh re-reads $ZDOTDIR before *each* startup file, so every shim points it back
# at the user's directory, sources their file, and takes it again -- otherwise
# zsh finds the user's .zshrc and ours never loads.
#
# `LEMBAS_USER_ZDOTDIR` is passed in on the exec line and must not be guessed
# here. By the time .zshenv runs, `$ZDOTDIR` is already *our* directory, so a
# `${ZDOTDIR:-$HOME}` fallback in this file captures the wrong path and the
# shim ends up sourcing itself -- which looks like it works, right up to the
# point somebody notices none of their own configuration is loaded.
_ZSH_SHIM = r"""
: ${LEMBAS_USER_ZDOTDIR:=$HOME}
ZDOTDIR=$LEMBAS_USER_ZDOTDIR
[[ -r $ZDOTDIR/%(file)s ]] && source $ZDOTDIR/%(file)s
LEMBAS_USER_ZDOTDIR=$ZDOTDIR
ZDOTDIR=$LEMBAS_DIR
"""
ZSH_ENV = _ZSH_SHIM % {"file": ".zshenv"}
ZSH_PROFILE = _ZSH_SHIM % {"file": ".zprofile"}
ZSH_RC = (
_ZSH_SHIM % {"file": ".zshrc"}
+ r"""
__lembas_esc() {
local s=${1//\\/\\\\}
s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
builtin print -rn -- $s
}
__lembas_precmd() {
local __s=$?
if [[ -n $__lembas_running ]]; then
builtin printf '\e]133;D;%s\a' $__s
__lembas_running=
fi
builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc $PWD)"
builtin printf '\e]133;A\a'
}
# $1 is the line as typed, before alias and glob expansion -- what somebody
# would recognise. $2 and $3 are progressively more expanded and less useful.
__lembas_preexec() {
builtin printf '\e]633;E;%s\a' "$(__lembas_esc $1)"
builtin printf '\e]133;C\a'
__lembas_running=1
}
# Prepended, not appended: whichever precmd runs first is the only one that
# sees the real $?, and a theme's hook will have clobbered it by the time a
# later one runs.
precmd_functions=(__lembas_precmd $precmd_functions)
preexec_functions=(__lembas_preexec $preexec_functions)
builtin printf '\e]633;LEMBAS;zsh;1\a'
"""
)
ZSH_LOGIN = (
_ZSH_SHIM % {"file": ".zlogin"}
+ r"""
# The last file zsh reads, so this is where ZDOTDIR goes back for good.
ZDOTDIR=$LEMBAS_USER_ZDOTDIR
[[ -n $LEMBAS_DIR ]] && rm -rf -- $LEMBAS_DIR
unset LEMBAS_DIR LEMBAS_USER_ZDOTDIR
"""
)
def quote(value: str) -> str:
"""A single-quoted shell word, with embedded quotes escaped."""
return "'" + value.replace("'", "'\\''") + "'"
def command_for(project_dir: str, *, integrate: bool = True) -> str | None:
"""What the PTY runs.
Returns None for the account's plain login shell in the one case that has
always returned it -- no project directory and no integration -- so the
existing behaviour is byte for byte what it was.
`$SHELL -c` is what sshd puts this through, so it is POSIX sh and branches
on the shell's own name. Anything that is not bash or zsh falls out of the
`case` into exactly the line that was here before, because a terminal that
works without markers is worth more than markers that break a terminal.
Every step is `2>/dev/null`, so a full `/tmp` or a read-only home costs the
markers and nothing else.
"""
cd = f"cd {quote(project_dir)} 2>/dev/null; " if project_dir else ""
if not integrate:
if not project_dir:
return None
return f"{cd}exec ${{SHELL:-/bin/sh}} -l"
return (
f"{cd}umask 077; "
# mkdir -m 700 fails on a path that already exists, which is what makes
# the fallback safe when mktemp is missing.
'__L=$(mktemp -d 2>/dev/null) || { '
'__L=${TMPDIR:-/tmp}/.lembas-$$-$RANDOM; mkdir -m 700 "$__L"; }; '
"case ${SHELL##*/} in "
f' bash) printf %s {quote(BASH_RC)} > "$__L/rc" 2>/dev/null && '
' exec bash --rcfile "$__L/rc" -i ;; '
f' zsh) printf %s {quote(ZSH_ENV)} > "$__L/.zshenv" 2>/dev/null && '
f' printf %s {quote(ZSH_PROFILE)} > "$__L/.zprofile" 2>/dev/null && '
f' printf %s {quote(ZSH_RC)} > "$__L/.zshrc" 2>/dev/null && '
f' printf %s {quote(ZSH_LOGIN)} > "$__L/.zlogin" 2>/dev/null && '
# The user's own ZDOTDIR is captured *here*, before it is replaced.
' LEMBAS_DIR="$__L" LEMBAS_USER_ZDOTDIR="${ZDOTDIR:-$HOME}" '
' ZDOTDIR="$__L" exec zsh -l ;; '
"esac; "
# Everything that did not exec lands here: an unknown shell, a failed
# mkdir, a full /tmp. You still get a shell.
'rm -rf -- "$__L" 2>/dev/null; exec ${SHELL:-/bin/sh} -l'
)
# --- Reading them back -------------------------------------------------------
_UNESCAPE = re.compile(r"\\x([0-9a-fA-F]{2})")
def unescape(value: str) -> str:
"""Undo `__lembas_esc`."""
return _UNESCAPE.sub(lambda m: chr(int(m.group(1), 16)), value).replace("\\\\", "\\")
class Marks:
"""Pulls the markers out of a stream that arrives in any pieces.
Deliberately not a regex over the scrollback. The pump is handed 64 KB at a
time on no particular boundary, so the ESC and the `]` land in different
frames often enough to matter -- which is the same reason nothing here
decodes the bytes. This holds at most one partial marker and nothing else.
"""
def __init__(
self,
on_mark: Callable[[str, str], None],
on_text: Callable[[bytes], None] | None = None,
) -> None:
self._on_mark = on_mark
self._on_text = on_text
self._buffer = bytearray()
self._in_marker = False
self._plain = bytearray()
def feed(self, data: bytes) -> None:
"""Split the stream into markers and everything else.
Both come back *in order* and as they are found, not after the whole
chunk has been scanned. That matters: a shell frequently writes the
command marker, the output and the finished marker in one 64KB read, so
anything that scanned first and absorbed afterwards would find the
capture already closed and keep nothing.
"""
for byte in data:
if not self._in_marker:
# A lone ESC is held: the ']' may be in the next frame.
if byte == _ESC:
self._flush()
self._buffer = bytearray([byte])
self._in_marker = True
else:
self._plain.append(byte)
continue
if len(self._buffer) == 1:
if byte != 0x5D: # ']' -- some other escape sequence
# Not ours, so it is output like anything else. Emitted
# rather than dropped: `clean_output` strips it later, and
# swallowing it here would silently eat a colour change.
self._in_marker = False
self._plain += self._buffer
self._plain.append(byte)
self._buffer.clear()
continue
self._buffer.append(byte)
continue
# Two legal terminators, and shells in the wild use both: BEL, and
# ESC \ (ST). The ESC has to be *appended* rather than treated as
# the start of something new, or the ST branch below can never fire.
if byte == _BEL:
self._finish()
continue
self._buffer.append(byte)
if self._buffer.endswith(b"\x1b\\"):
del self._buffer[-2:]
self._finish()
continue
# An `ESC ]` inside a marker that was never terminated starts a new
# one. Without this a truncated marker would eat the next.
if self._buffer.endswith(b"\x1b]"):
self._buffer = bytearray(b"\x1b]")
continue
if len(self._buffer) > MAX_MARKER_BYTES:
# Not one of ours. Output is worth more than a marker.
self._in_marker = False
self._plain += self._buffer
self._buffer.clear()
self._flush()
def _flush(self) -> None:
if not self._plain:
return
if self._on_text is not None:
self._on_text(bytes(self._plain))
self._plain.clear()
def _finish(self) -> None:
payload = bytes(self._buffer[2:]).decode("utf-8", "replace")
self._in_marker = False
self._buffer.clear()
code, _, rest = payload.partition(";")
if code not in ("133", "633") or not rest:
return
kind, _, value = rest.partition(";")
self._on_mark(kind, value)
__all__ = [
"BASH_RC",
"MARK_COMMAND",
"MARK_CWD",
"MARK_DONE",
"MARK_OUTPUT",
"MARK_PROMPT",
"MARK_READY",
"MAX_MARKER_BYTES",
"Marks",
"ZSH_ENV",
"ZSH_LOGIN",
"ZSH_PROFILE",
"ZSH_RC",
"command_for",
"quote",
"unescape",
]
src/lembas/services/agent/terminal.py
+155 -4
View File
@@ -44,6 +44,7 @@ from collections import deque
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any from typing import Any
from lembas.services.agent import capture, shell_marks
from lembas.services.agent.base import ExecError from lembas.services.agent.base import ExecError
from lembas.services.agent.ssh import available, connect_kwargs from lembas.services.agent.ssh import available, connect_kwargs
@@ -90,6 +91,20 @@ CLOSED_SHUTDOWN = "shutdown"
CLOSED_REVOKED = "revoked" CLOSED_REVOKED = "revoked"
CLOSED_ERROR = "error" CLOSED_ERROR = "error"
# Whether this shell tells us where its commands begin and end.
# live -- it does
# loading -- the hooks went in; the first prompt has not arrived yet
# none -- it never will: an unknown shell, or a dotfile that replaced it
INTEGRATION_LIVE = "live"
INTEGRATION_LOADING = "loading"
INTEGRATION_NONE = "none"
# How long a shell may produce output without ever marking a prompt before we
# conclude it is not going to. This is what catches a `.bashrc` ending in `exec
# tmux`: the hooks were installed and then the shell replaced itself. Without
# it the buttons stay greyed out forever with no explanation.
INTEGRATION_GRACE = 10.0
@dataclass @dataclass
class Viewer: class Viewer:
@@ -122,6 +137,7 @@ class Session:
idle_timeout: float = 1800.0, idle_timeout: float = 1800.0,
cols: int = 80, cols: int = 80,
rows: int = 24, rows: int = 24,
integrate: bool = True,
) -> None: ) -> None:
self.chat_id = chat_id self.chat_id = chat_id
self.owner_id = owner_id self.owner_id = owner_id
@@ -134,6 +150,28 @@ class Session:
self._scrollback: deque[bytes] = deque() self._scrollback: deque[bytes] = deque()
self._scrollback_bytes = 0 self._scrollback_bytes = 0
# --- Command boundaries ---------------------------------------------
# Three states, and the middle one matters: INTEGRATION_LOADING means
# the hooks were installed and no marker has arrived yet, which is a
# different thing to tell somebody than "this shell will never mark".
self.integrate = integrate
self.integration = INTEGRATION_LOADING if integrate else INTEGRATION_NONE
self.shell = ""
self._marks = shell_marks.Marks(self._on_mark, self._on_text)
# At most two, ever. The one being written and the last finished one --
# a history would be a second scrollback with none of the bounding.
self.current: capture.Capture | None = None
self.last: capture.Capture | None = None
self._captures = 0
# Where the shell says it is, which the panel header shows live and a
# capture records. Seeded from the chat so it says something sensible
# before the first prompt.
self.cwd = project_dir
# Set by the socket layer, which knows how to shape a frame. Called
# when a command finishes so a panel can enable its buttons without
# polling for something that happens a few times a minute.
self.on_command: Any = None
self._conn: Any = None self._conn: Any = None
self._process: Any = None self._process: Any = None
self._pump: asyncio.Task | None = None self._pump: asyncio.Task | None = None
@@ -213,11 +251,79 @@ class Session:
command nobody entered. It is single-quoted, and a failure is ignored: command nobody entered. It is single-quoted, and a failure is ignored:
a directory that has been deleted should leave somebody at a shell to a directory that has been deleted should leave somebody at a shell to
find out why, not with a connection that closes as it opens. find out why, not with a connection that closes as it opens.
With integration on, the same string also writes the shell-integration
files and execs through them; see `shell_marks` for why it is done in
the command rather than over SFTP or through the environment.
""" """
if not self.project_dir: return shell_marks.command_for(self.project_dir, integrate=self.integrate)
return None
quoted = "'" + self.project_dir.replace("'", "'\\''") + "'" # --- Where one command ends and the next begins --------------------------
return f"cd {quoted} 2>/dev/null; exec ${{SHELL:-/bin/sh}} -l"
def _on_mark(self, kind: str, value: str) -> None:
"""One marker, from the scanner in the pump.
Advisory, never trusted: a program can print these itself and move a
boundary. It is not a way in -- the text is sanitised and fenced either
way, and a program could already print anything on screen -- but that is
why nothing here validates them, and why none of it decides anything a
person could not already do at the keyboard.
"""
if kind == shell_marks.MARK_READY:
self.integration = INTEGRATION_LIVE
self.shell = value.split(";")[0][:32]
return
if self.integration != INTEGRATION_LIVE and kind in (
shell_marks.MARK_PROMPT,
shell_marks.MARK_OUTPUT,
):
self.integration = INTEGRATION_LIVE
if kind == shell_marks.MARK_CWD:
path = shell_marks.unescape(value.partition("=")[2])[:1000]
self.cwd = path
return
if kind == shell_marks.MARK_COMMAND:
self._captures += 1
self.current = capture.Capture(
seq=self._captures,
command=capture.trim_command(shell_marks.unescape(value)),
cwd=self.cwd,
)
return
if kind == shell_marks.MARK_DONE and self.current is not None:
try:
self.current.exit_status = int(value.strip() or 0)
except ValueError:
self.current.exit_status = 0
self.current.ended = time.monotonic()
self.last = self.current
self.current = None
if self.on_command is not None:
self.on_command(self.last)
def _on_text(self, data: bytes) -> None:
"""Everything that was not a marker, while a command is running.
Interleaved with `_on_mark` rather than applied to the whole chunk
afterwards: a shell often writes the command marker, the output and the
finished marker in one read, and absorbing after the scan would find
the capture already closed and keep nothing at all.
"""
if self.current is not None:
self.current.absorb(data)
def latest(self) -> capture.Capture | None:
"""The command to act on: the one still running, else the last one.
In-flight counts. "Copy the last command and its output" while `make` is
still going should give what has been printed so far, marked as still
running -- not "nothing yet".
"""
return self.current or self.last
# --- Following ----------------------------------------------------------- # --- Following -----------------------------------------------------------
@@ -291,6 +397,9 @@ class Session:
if not data: if not data:
break break
self._remember(data) self._remember(data)
# Before the fan-out, so a "this command finished" frame can
# never reach a browser after the output it describes.
self._observe(data)
self._fan_out(data) self._fan_out(data)
except asyncio.CancelledError: except asyncio.CancelledError:
raise raise
@@ -300,12 +409,52 @@ class Session:
return return
await self._finish(CLOSED_EXITED) await self._finish(CLOSED_EXITED)
def _observe(self, data: bytes) -> None:
"""Watch the stream for markers, and feed the command being captured.
Server-side rather than in the browser, for five reasons. The `behind`
path calls `term.reset()` and replays a *truncated* scrollback, so a
client parser routinely sees a "finished" with no matching "started".
Two tabs share one shell and two parsers can disagree about what "the
last command" is. The server sees the stream once however many are
watching. And what comes out of this ends up inside a prompt -- deriving
it here means there is nothing to disbelieve later.
The bytes are still fanned out unchanged, markers and all: xterm
consumes an OSC it has no handler for and never draws it, and rewriting
frames on the hot path would break the "nothing decodes, so nothing can
split" property the pump depends on.
"""
self._marks.feed(data)
if self.current is None and (
self.integration == INTEGRATION_LOADING
and time.monotonic() - self.started_at > INTEGRATION_GRACE
):
# Output arrived, the grace period passed, and no marker ever came.
# Output arrived, the grace period passed, and no marker ever came.
# Something replaced the shell -- a dotfile ending in `exec tmux` is
# the usual one. Say so rather than leaving the buttons greyed.
self.integration = INTEGRATION_NONE
def _remember(self, data: bytes) -> None: def _remember(self, data: bytes) -> None:
self._scrollback.append(data) self._scrollback.append(data)
self._scrollback_bytes += len(data) self._scrollback_bytes += len(data)
while self._scrollback_bytes > SCROLLBACK_BYTES and len(self._scrollback) > 1: while self._scrollback_bytes > SCROLLBACK_BYTES and len(self._scrollback) > 1:
self._scrollback_bytes -= len(self._scrollback.popleft()) self._scrollback_bytes -= len(self._scrollback.popleft())
def announce(self, text: str) -> None:
"""Put one text frame in front of every viewer.
Through the same queues as the output so ordering is preserved: a
"finished" that overtook the last of the output it describes would have
a panel offering a capture the screen has not caught up with. Dropped
rather than blocking on a full queue -- that viewer is already being
disconnected and will be told again on reattach.
"""
for viewer in list(self.viewers.values()):
with contextlib.suppress(asyncio.QueueFull):
viewer.queue.put_nowait(text)
def _fan_out(self, data: bytes) -> None: def _fan_out(self, data: bytes) -> None:
for viewer in list(self.viewers.values()): for viewer in list(self.viewers.values()):
try: try:
@@ -414,6 +563,7 @@ async def open_session(
max_per_user: int = 3, max_per_user: int = 3,
cols: int = 80, cols: int = 80,
rows: int = 24, rows: int = 24,
integrate: bool = True,
) -> Session: ) -> Session:
"""The shell for this chat, opening one if it is not already there. """The shell for this chat, opening one if it is not already there.
@@ -442,6 +592,7 @@ async def open_session(
owner_id=owner_id, owner_id=owner_id,
profile_id=profile_id, profile_id=profile_id,
label=label, label=label,
integrate=integrate,
project_dir=project_dir, project_dir=project_dir,
idle_timeout=idle_timeout, idle_timeout=idle_timeout,
cols=cols, cols=cols,
src/lembas/services/settings_store.py
+5
View File
@@ -88,6 +88,11 @@ def _agents_defaults() -> dict[str, Any]:
# SSH connection held open, so this is a real resource, not a scruple. # SSH connection held open, so this is a real resource, not a scruple.
"terminal_max_sessions": 20, "terminal_max_sessions": 20,
"terminal_max_per_user": 3, "terminal_max_per_user": 3,
# Whether the panel's shell is given hooks that mark where one
# command ends and the next begins. Off means the Copy and Send
# buttons fall back to scraping the screen, and Auto is
# unavailable -- there is nothing to key it on.
"terminal_integration": True,
# A listing of the project directory, put in front of the model so the # A listing of the project directory, put in front of the model so the
# first rounds of a reply are not spent discovering what is there. It # first rounds of a reply are not spent discovering what is there. It
# costs its budget on *every* request in an agent chat, forever, which # costs its budget on *every* request in an agent chat, forever, which
src/lembas/web/static/css/app.css
+56
View File
@@ -544,14 +544,57 @@ button, input, textarea, select {
`hidden` attribute. */ `hidden` attribute. */
.terminal { .terminal {
width: var(--terminal-width); width: var(--terminal-width);
min-width: var(--terminal-width-min);
max-width: 80vw;
flex: none; flex: none;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
min-height: 0; min-height: 0;
/* So the resize handle can sit on the edge. */
position: relative;
background: var(--bg-sunken); background: var(--bg-sunken);
border-left: 1px solid var(--border); border-left: 1px solid var(--border);
} }
/* The drag handle on a panel's left edge. Wider than it looks -- a one-pixel
border is a target nobody can hit -- and it sits *outside* the panel's own
padding so it never overlaps what is being resized. */
.panel-resize {
position: absolute;
top: 0;
bottom: 0;
left: -3px;
width: 9px;
z-index: var(--z-handle);
display: flex;
align-items: center;
justify-content: center;
cursor: col-resize;
color: transparent;
touch-action: none;
transition: background var(--transition-fast), color var(--transition-fast);
}
.panel-resize:hover,
.panel-resize:focus-visible,
body.is-resizing .panel-resize {
background: var(--accent-soft);
color: var(--accent);
outline: none;
}
/* While dragging, nothing else may take the pointer -- a text selection
starting mid-drag makes the whole page flicker blue, and the iframe-shaped
hazard is the terminal itself swallowing pointermove. */
body.is-resizing {
cursor: col-resize;
user-select: none;
}
body.is-resizing .terminal__screen { pointer-events: none; }
@media (max-width: 64rem) {
/* A full-height overlay has no edge to drag, and no room to spare. */
.panel-resize { display: none; }
}
.terminal__where { .terminal__where {
font-weight: 400; font-weight: 400;
font-family: var(--font-mono); font-family: var(--font-mono);
@@ -586,6 +629,19 @@ button, input, textarea, select {
} }
.terminal__status strong { color: var(--ink-muted); font-weight: 600; } .terminal__status strong { color: var(--ink-muted); font-weight: 600; }
.terminal__message { flex: 1; min-width: 0; overflow-wrap: anywhere; } .terminal__message { flex: 1; min-width: 0; overflow-wrap: anywhere; }
/* What the last command was. Monospace and clipped: it is a command line, and
one long enough to wrap would push the status bar into two rows. */
.terminal__last {
flex: 0 1 auto;
min-width: 0;
max-width: 16rem;
font-family: var(--font-mono);
color: var(--ink-muted);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.terminal__last:empty { display: none; }
.terminal__message--error { color: var(--danger); } .terminal__message--error { color: var(--danger); }
@media (max-width: 64rem) { @media (max-width: 64rem) {
src/lembas/web/static/js/app.js
+112
View File
@@ -493,6 +493,113 @@
); );
} }
/* --- Dragging a panel wider ---------------------------------------------
Generic rather than terminal-specific: the inspector and the sidebar want
the same handle, and a second copy of this is how two panels end up
resizing differently.
The width lands on a CSS variable on <html> rather than on the panel, so
the ≤64rem overlay rule -- which clamps it with min() -- keeps working
without knowing anything about dragging. Persisted the way the theme is:
localStorage for this tab, best-effort POST for the next device. */
var RESIZE_KEY = "lembas-panel-widths";
function storedWidths() {
try {
return JSON.parse(localStorage.getItem(RESIZE_KEY) || "{}") || {};
} catch (e) {
return {};
}
}
function applyWidths() {
var widths = storedWidths();
Object.keys(widths).forEach(function (name) {
document.documentElement.style.setProperty(name, widths[name] + "px");
});
}
function rememberWidth(name, pixels) {
var widths = storedWidths();
widths[name] = Math.round(pixels);
try {
localStorage.setItem(RESIZE_KEY, JSON.stringify(widths));
} catch (e) { /* private mode */ }
if (document.body.dataset.authenticated === "true") {
fetch("/api/preferences/layout", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(widths)
}).catch(function () { /* already applied locally */ });
}
}
function setupResize() {
document.addEventListener("pointerdown", function (event) {
var handle = event.target.closest("[data-resize]");
if (!handle || event.button !== 0) return;
var panel = handle.closest("[data-resize-target]") || handle.parentElement;
if (!panel) return;
var name = handle.dataset.resize;
var min = parseFloat(handle.dataset.resizeMin || "320");
var startX = event.clientX;
var startWidth = panel.getBoundingClientRect().width;
var frame = null;
var pending = startWidth;
event.preventDefault();
handle.setPointerCapture(event.pointerId);
document.body.classList.add("is-resizing");
function move(moveEvent) {
/* The handle is on the panel's *left* edge and the panel is on the
right of the shell, so dragging left makes it wider. */
var max = Math.max(min, window.innerWidth - 360);
pending = Math.min(Math.max(startWidth - (moveEvent.clientX - startX), min), max);
/* Coalesced to a frame: the ResizeObserver on the panel calls xterm's
fit() and sends a resize frame up the socket, and doing that once
per pointermove is a frame per pixel of drag. */
if (frame) return;
frame = requestAnimationFrame(function () {
frame = null;
document.documentElement.style.setProperty(name, pending + "px");
});
}
function stop() {
handle.removeEventListener("pointermove", move);
handle.removeEventListener("pointerup", stop);
handle.removeEventListener("pointercancel", stop);
document.body.classList.remove("is-resizing");
if (frame) cancelAnimationFrame(frame);
document.documentElement.style.setProperty(name, pending + "px");
rememberWidth(name, pending);
}
handle.addEventListener("pointermove", move);
handle.addEventListener("pointerup", stop);
handle.addEventListener("pointercancel", stop);
});
/* A keyboard has to be able to do this too, or the panel is only resizable
with a mouse and the handle is a focus trap that does nothing. */
document.addEventListener("keydown", function (event) {
var handle = event.target.closest("[data-resize]");
if (!handle) return;
var step = event.key === "ArrowLeft" ? 32 : event.key === "ArrowRight" ? -32 : 0;
if (!step) return;
event.preventDefault();
var panel = handle.closest("[data-resize-target]") || handle.parentElement;
var name = handle.dataset.resize;
var min = parseFloat(handle.dataset.resizeMin || "320");
var max = Math.max(min, window.innerWidth - 360);
var width = Math.min(Math.max(panel.getBoundingClientRect().width + step, min), max);
document.documentElement.style.setProperty(name, width + "px");
rememberWidth(name, width);
});
}
window.lembas = { window.lembas = {
setPanel: setPanel, setPanel: setPanel,
applyTheme: applyTheme, applyTheme: applyTheme,
@@ -590,8 +697,13 @@
scrollThread(true); scrollThread(true);
applyTheme(currentTheme()); applyTheme(currentTheme());
setupDropzone(); setupDropzone();
setupResize();
}); });
/* Before first paint rather than on DOMContentLoaded, so a panel that was
dragged wider does not open at its default and jump. */
applyWidths();
/* After any htmx swap: re-measure the composer and follow new content. */ /* After any htmx swap: re-measure the composer and follow new content. */
document.body.addEventListener("htmx:afterSwap", function () { document.body.addEventListener("htmx:afterSwap", function () {
document.querySelectorAll("[data-autosize]").forEach(autosize); document.querySelectorAll("[data-autosize]").forEach(autosize);
src/lembas/web/static/js/terminal.js
+144 -21
View File
@@ -32,6 +32,11 @@
var messageEl = null; var messageEl = null;
var observer = null; var observer = null;
var closedOnPurpose = false; var closedOnPurpose = false;
/* Whether this shell tells us where commands begin and end -- "live",
"loading" or "none". Everything the three buttons do keys off it. */
var integration = "loading";
var autoSend = false;
var lastCommand = null;
function say(text, isError) { function say(text, isError) {
if (!messageEl) return; if (!messageEl) return;
@@ -148,6 +153,9 @@
var where = panel.querySelector("[data-terminal-where]"); var where = panel.querySelector("[data-terminal-where]");
if (where) where.textContent = payload.dir; if (where) where.textContent = payload.dir;
} }
integration = payload.integration || "none";
showLast(payload.last);
applyIntegration();
/* The server may have opened the shell at a size chosen by whoever got /* The server may have opened the shell at a size chosen by whoever got
here first, so ask for ours now that there is something to ask. */ here first, so ask for ours now that there is something to ask. */
refit(); refit();
@@ -155,6 +163,18 @@
return; return;
} }
if (payload.t === "command") {
/* A command finished. Tens of bytes, not the output: a 64KB text frame
would compete with PTY bytes on the one path that has to stay quick,
and the buttons fetch what they need when they are pressed. */
if (integration !== "live") { integration = "live"; applyIntegration(); }
showLast(payload.command);
if (autoSend) {
capture(true).then(function (text) { intoComposer(text, true); });
}
return;
}
if (payload.t === "behind") { if (payload.t === "behind") {
/* This window stopped reading and was disconnected so the others kept /* This window stopped reading and was disconnected so the others kept
up. Reconnecting costs nothing: the scrollback is the state. */ up. Reconnecting costs nothing: the scrollback is the state. */
@@ -253,14 +273,54 @@
}); });
} }
/* --- Send to chat ------------------------------------------------------- */ /* --- Handing a command to the chat --------------------------------------
/* Into the composer, never sent. What a machine printed is exactly the sort Three buttons over one idea: the last command, its output, and where it
of text somebody should read before a model does, and the box is where ran. The server renders the block, so the text a model eventually reads
that happens. */ exists in exactly one place -- and the screen buffer could not produce it
function sendToChat() { anyway, holding as it does what is *on screen*, hard-wrapped at the
if (!term) return; terminal's width with no way to tell a wrap from a newline. */
var text = term.getSelection();
if (!text) { function applyIntegration() {
if (!panel) return;
var auto = panel.querySelector("[data-terminal-auto]");
if (!auto) return;
var usable = integration === "live";
auto.disabled = !usable;
auto.title = usable
? "Attach every command you run to your next message"
: "This shell did not load LLeMbas's command markers, so there is no way " +
"to tell where one command's output ends.";
if (!usable && autoSend) setAuto(false);
}
function showLast(command) {
lastCommand = command || null;
var slot = panel && panel.querySelector("[data-terminal-last]");
if (!slot) return;
// textContent, always: this is a command line off somebody's machine.
slot.textContent = lastCommand ? lastCommand.summary : "";
}
function setAuto(on) {
autoSend = !!on;
var button = panel.querySelector("[data-terminal-auto]");
if (button) {
button.setAttribute("aria-pressed", autoSend ? "true" : "false");
button.classList.toggle("is-active", autoSend);
}
say(autoSend
? "Every command you run will be attached to your next message."
: "Commands are no longer attached automatically.");
}
/* A selection always wins, in every state. People rely on it, and it is the
only way to send part of something. */
function selected() {
var text = term ? term.getSelection() : "";
return text && text.trim() ? text : "";
}
function scraped() {
var lines = []; var lines = [];
var buffer = term.buffer.active; var buffer = term.buffer.active;
var last = buffer.baseY + buffer.cursorY; var last = buffer.baseY + buffer.cursorY;
@@ -268,23 +328,78 @@
var line = buffer.getLine(y); var line = buffer.getLine(y);
if (line) lines.push(line.translateToString(true)); if (line) lines.push(line.translateToString(true));
} }
text = lines.join("\n").replace(/\n+$/, ""); return lines.join("\n").replace(/\n+$/, "");
}
if (!text.trim()) {
say("Nothing to send: select some output first.");
return;
} }
/* Fetches the rendered block, or falls back to the screen. `quiet` is the
auto path, which must not narrate every command it collects. */
function capture(quiet) {
var chosen = selected();
if (chosen && !quiet) return Promise.resolve("```\n" + chosen + "\n```\n");
if (integration !== "live") {
if (quiet) return Promise.resolve("");
var text = scraped();
if (!text.trim()) {
say("Nothing to send: select some output first.");
return Promise.resolve("");
}
/* Said plainly rather than dressed up. Without markers this is the last
forty rows as they appeared, wraps and all, and pretending otherwise
would put a precise-looking block in front of a model that is not. */
say("Copied the last of the screen, as it appeared. This shell does not " +
"mark where commands begin.");
return Promise.resolve("```\n" + text + "\n```\n");
}
return fetch(panel.dataset.url.replace(/\/ws$/, "/last"), { credentials: "same-origin" })
.then(function (response) { return response.json(); })
.then(function (body) {
if (!body.ok) {
if (!quiet) say(body.message || "Nothing to send yet.");
return "";
}
return body.text + "\n";
})
.catch(function () {
if (!quiet) say("Could not read the last command.", true);
return "";
});
}
function intoComposer(text, quiet) {
if (!text) return;
var input = document.querySelector("[data-composer-input]"); var input = document.querySelector("[data-composer-input]");
if (!input) return; if (!input) return;
var fence = "```\n" + text + "\n```\n"; input.value = input.value ? input.value.replace(/\s*$/, "\n\n") + text : text;
input.value = input.value ? input.value.replace(/\s*$/, "\n\n") + fence : fence;
if (window.lembas && window.lembas.autosize) window.lembas.autosize(input); if (window.lembas && window.lembas.autosize) window.lembas.autosize(input);
input.focus(); /* Auto-send never steals focus: it fires while somebody is typing in the
/* "As it appeared" and not "as it was written": the buffer holds what is on terminal, and yanking the caret out of a shell mid-command is the sort
screen, hard-wrapped at the terminal's width, with no way to tell a wrap of thing that gets a feature switched off for good. */
from a newline. */ if (!quiet) input.focus();
say("Copied into the message box as it appeared on screen."); }
function sendToChat() {
if (!term) return;
capture(false).then(function (text) {
if (!text) return;
intoComposer(text, false);
/* Into the composer, never sent. What a machine printed is exactly the
sort of text somebody should read before a model does, and the box is
where that happens. */
if (integration === "live") say("Put into the message box. It is not sent yet.");
});
}
function copyToClipboard() {
if (!term) return;
capture(false).then(function (text) {
if (!text) return;
if (window.lembas && window.lembas.copyText) {
window.lembas.copyText(text);
say("Copied.");
}
});
} }
/* --- Wiring ------------------------------------------------------------- */ /* --- Wiring ------------------------------------------------------------- */
@@ -305,7 +420,15 @@
panel.addEventListener("click", function (event) { panel.addEventListener("click", function (event) {
if (event.target.closest("[data-terminal-send]")) { if (event.target.closest("[data-terminal-send]")) {
event.preventDefault(); event.preventDefault();
sendToChat(); return sendToChat();
}
if (event.target.closest("[data-terminal-copy]")) {
event.preventDefault();
return copyToClipboard();
}
if (event.target.closest("[data-terminal-auto]")) {
event.preventDefault();
return setAuto(!autoSend);
} }
}); });
src/lembas/web/templates/admin/agents.html
+16
View File
@@ -222,6 +222,22 @@
One per chat. Each holds an SSH connection open on the far machine. One per chat. Each holds an SSH connection open on the far machine.
</p> </p>
</div> </div>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="terminal_integration"
{{ 'checked' if values.terminal_integration }}>
<span>Mark where commands begin and end</span>
</label>
<p class="field__hint">
Gives bash and zsh the same invisible markers VS Code and WezTerm use,
so <strong>Copy</strong>, <strong>Send</strong> and the automatic
toggle know which output belongs to which command. Written by the shell
into a temporary file it deletes itself, and any other shell is started
exactly as it was before. Off means those buttons fall back to copying
the last of the screen as it appeared, wraps and all.
</p>
</div>
</section> </section>
<section class="card"> <section class="card">
src/lembas/web/templates/base.html
+13 -1
View File
@@ -1,5 +1,5 @@
<!doctype html> <!doctype html>
<html lang="en" data-theme="{{ theme }}"> <html lang="en" data-theme="{{ theme }}"{% if layout %} style="{{ layout }}"{% endif %}>
<head> <head>
<meta charset="utf-8"> <meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
@@ -41,6 +41,18 @@
document.documentElement.dataset.theme = stored; document.documentElement.dataset.theme = stored;
} }
} catch (e) { /* private mode: the server-rendered theme stands */ } } catch (e) { /* private mode: the server-rendered theme stands */ }
/* Panel widths, for the same reason and in the same breath. app.js also
applies these, but it is deferred -- so without this a panel dragged
wider opens at its default and jumps once the script runs. */
try {
var widths = JSON.parse(localStorage.getItem("lembas-panel-widths") || "{}");
Object.keys(widths).forEach(function (name) {
if (name.indexOf("--") === 0) {
document.documentElement.style.setProperty(name, widths[name] + "px");
}
});
} catch (e) { /* the stylesheet's defaults stand */ }
})(); })();
</script> </script>
</head> </head>
src/lembas/web/templates/chat/_terminal.html
+33 -2
View File
@@ -14,18 +14,48 @@
data-terminal data-terminal
data-url="/api/chats/{{ chat.id }}/terminal/ws" data-url="/api/chats/{{ chat.id }}/terminal/ws"
data-label="{{ agent_profile.name if agent_profile else 'this connection' }}" data-label="{{ agent_profile.name if agent_profile else 'this connection' }}"
data-dir="{{ chat.project_dir }}"> data-dir="{{ chat.project_dir }}"
data-resize-target>
{#
The left edge, dragged. A separator rather than a decoration: it takes
focus and answers the arrow keys, or the panel is only resizable with a
mouse and the grip is a focus trap that does nothing.
#}
<div class="panel-resize" data-resize="--terminal-width" data-resize-min="384"
role="separator" aria-orientation="vertical" tabindex="0"
aria-label="Resize the terminal">
{{ icon("grip", "icon--sm") }}
</div>
<div class="panel-head"> <div class="panel-head">
<h2 class="panel-head__title"> <h2 class="panel-head__title">
{{ icon("terminal", "icon--sm") }} {{ icon("terminal", "icon--sm") }}
<span>{{ agent_profile.name if agent_profile else "Terminal" }}</span> <span>{{ agent_profile.name if agent_profile else "Terminal" }}</span>
<span class="terminal__where" data-terminal-where>{{ chat.project_dir }}</span> <span class="terminal__where" data-terminal-where>{{ chat.project_dir }}</span>
</h2> </h2>
{#
Copy, Send and Auto. All three need to know where one command ends and
the next begins, which is what the shell integration provides; without it
the first two fall back to scraping the screen and say so, and Auto is
disabled rather than degraded. Forty arbitrary lines attached to every
message is worse than nothing attached at all.
#}
<button class="btn btn--icon btn--sm" type="button" data-terminal-copy
title="Copy the last command and its output"
aria-label="Copy the last command and its output">
{{ icon("copy", "icon--sm") }}
</button>
<button class="btn btn--icon btn--sm" type="button" data-terminal-send <button class="btn btn--icon btn--sm" type="button" data-terminal-send
title="Put the selection, or the last of the output, into the message box" title="Put the last command and its output into the message box"
aria-label="Send to chat"> aria-label="Send to chat">
{{ icon("arrow-up", "icon--sm") }} {{ icon("arrow-up", "icon--sm") }}
</button> </button>
<button class="btn btn--icon btn--sm" type="button" data-terminal-auto
aria-pressed="false"
title="Attach every command you run to your next message"
aria-label="Send every command automatically">
{{ icon("sparkle", "icon--sm") }}
</button>
<button class="btn btn--icon btn--sm" type="button" data-toggle="#terminal" <button class="btn btn--icon btn--sm" type="button" data-toggle="#terminal"
aria-label="Close terminal"> aria-label="Close terminal">
{{ icon("x", "icon--sm") }} {{ icon("x", "icon--sm") }}
@@ -36,6 +66,7 @@
<div class="terminal__status"> <div class="terminal__status">
<span class="terminal__message" data-terminal-message>Connecting…</span> <span class="terminal__message" data-terminal-message>Connecting…</span>
<span class="terminal__last" data-terminal-last></span>
<span>Ctrl+Shift+C / V</span> <span>Ctrl+Shift+C / V</span>
</div> </div>
</aside> </aside>
src/lembas/web/templating.py
+29
View File
@@ -59,6 +59,34 @@ def resolve_theme(user: User | None) -> str:
return settings.default_theme return settings.default_theme
def resolve_layout(user: User | None) -> str:
"""Stored panel widths as a `style` value for <html>, or "".
Same shape as the theme and for the same reason: a first guess, corrected
from localStorage before first paint. This is what carries a dragged width
to a second browser, where localStorage has nothing to say.
Re-clamped on the way out rather than trusted from the column. The bounds
could have tightened since it was stored, and a width outside them is a
panel somebody cannot see well enough to drag back.
"""
if user is None:
return ""
from lembas.api.preferences import LAYOUT_BOUNDS
parts = []
for name, raw in ((user.settings_json or {}).get("layout") or {}).items():
bounds = LAYOUT_BOUNDS.get(str(name))
if bounds is None:
continue
try:
value = min(max(int(float(raw)), bounds[0]), bounds[1])
except (TypeError, ValueError):
continue
parts.append(f"{name}:{value}px")
return ";".join(parts)
def render( def render(
request: Request, request: Request,
template: str, template: str,
@@ -76,6 +104,7 @@ def render(
"request": request, "request": request,
"user": user, "user": user,
"theme": resolve_theme(user), "theme": resolve_theme(user),
"layout": resolve_layout(user),
"version": __version__, "version": __version__,
"allow_signup": settings.allow_signup, "allow_signup": settings.allow_signup,
} }
tests/test_terminal.py
+26 -2
View File
@@ -142,8 +142,14 @@ async def test_a_project_directory_becomes_a_cd_before_the_shell(shell_host):
await session.close() await session.close()
async def test_no_project_directory_means_the_plain_login_shell(shell_host): async def test_no_project_directory_and_no_integration_is_the_plain_login_shell(shell_host):
session = await _open(shell_host) """The original behaviour, kept reachable and kept tested.
Switching integration off has to give back *exactly* what was there before,
byte for byte -- including the None that means "whatever the account logs
in with". A fallback that is nearly the old behaviour is not a fallback.
"""
session = await _open(shell_host, integrate=False)
viewer = session.attach() viewer = session.attach()
await _read_until(viewer, b"READY") await _read_until(viewer, b"READY")
@@ -151,6 +157,24 @@ async def test_no_project_directory_means_the_plain_login_shell(shell_host):
await session.close() await session.close()
async def test_an_unknown_shell_falls_through_to_the_plain_login_shell(shell_host):
"""With integration on the command is no longer None -- but every branch
that does not recognise the shell ends in the same `exec` that was there
before, because a terminal that works without markers is worth more than
markers that break a terminal."""
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
command = shell_host["seen"]["command"]
assert "case ${SHELL##*/} in" in command
assert command.rstrip().endswith("exec ${SHELL:-/bin/sh} -l")
# Every step is silenced, so a full /tmp or a read-only home costs the
# markers and nothing else.
assert "2>/dev/null" in command
await session.close()
async def test_a_single_quote_in_the_directory_cannot_end_the_quoting(shell_host): async def test_a_single_quote_in_the_directory_cannot_end_the_quoting(shell_host):
session = await _open(shell_host, project_dir="/tmp/it's here; rm -rf /") session = await _open(shell_host, project_dir="/tmp/it's here; rm -rf /")
viewer = session.attach() viewer = session.attach()
tests/test_terminal_capture.py
+413
View File
@@ -0,0 +1,413 @@
"""Where one command ends and the next begins, and what is kept of it.
Three layers, tested separately because they fail separately: the byte scanner,
the bounding, and a real PTY emitting real markers.
The scanner tests need no server and no event loop -- it is a byte state
machine. That is most of the argument for parsing server-side rather than in
xterm: the thing that has to be right is testable without a browser.
"""
from __future__ import annotations
import pytest
from lembas.services.agent import capture, shell_marks
asyncssh = pytest.importorskip("asyncssh")
def _osc(body: str) -> bytes:
return f"\033]{body}\007".encode()
def _seen(chunks) -> list[tuple[str, str]]:
marks: list[tuple[str, str]] = []
scanner = shell_marks.Marks(lambda kind, value: marks.append((kind, value)))
for chunk in chunks:
scanner.feed(chunk)
return marks
# --- The scanner -------------------------------------------------------------
def test_a_marker_is_read_out_of_a_stream():
marks = _seen([b"hello" + _osc("133;A") + b"$ "])
assert marks == [("A", "")]
def test_a_marker_split_across_writes_is_still_seen():
"""The test that justifies parsing server-side being safe at all.
The pump is handed 64KB at a time on no particular boundary, so the ESC and
the `]` land in different frames often enough to matter. Fed one byte at a
time, which is the worst case and the cheapest way to prove it.
"""
stream = b"before" + _osc("633;E;make -j8") + b"after"
marks = _seen([stream[i : i + 1] for i in range(len(stream))])
assert marks == [("E", "make -j8")]
def test_a_marker_terminated_with_st_rather_than_bel_is_seen():
"""Both terminators are legal and shells in the wild use both."""
marks = _seen([b"\033]133;D;2\033\\"])
assert marks == [("D", "2")]
def test_a_lone_escape_bracket_in_binary_output_does_not_swallow_the_session():
"""`cat` of a binary file produces stray ESC ] regularly. Without the bound
one of them would buffer the rest of the session and never emit."""
junk = b"\033]" + b"x" * (shell_marks.MAX_MARKER_BYTES + 10)
marks = _seen([junk, _osc("133;A")])
assert marks == [("A", "")]
def test_an_unterminated_marker_does_not_eat_the_next_one():
marks = _seen([b"\033]133;A" + _osc("133;C")])
assert ("C", "") in marks
def test_an_escape_that_is_not_an_osc_is_ignored():
"""A colour change is `ESC [`, not `ESC ]`, and there is a lot of it."""
assert _seen([b"\033[31mred\033[0m"]) == []
def test_a_semicolon_in_a_command_survives_the_escaping():
"""The payload may contain no raw `;` or the fields split, so the shell
escapes it and this puts it back."""
marks = _seen([_osc("633;E;cd /tmp \\x3b ls")])
assert marks == [("E", "cd /tmp \\x3b ls")]
assert shell_marks.unescape(marks[0][1]) == "cd /tmp ; ls"
def test_an_unrelated_osc_is_not_ours():
"""Setting the window title is OSC 0 and happens constantly."""
assert _seen([_osc("0;some title")]) == []
# --- The bounding ------------------------------------------------------------
def test_a_flood_keeps_the_head_and_the_tail_and_says_what_it_dropped():
"""Either half alone is the wrong half: a build that fails ten megabytes in
has the invocation at the top and the error at the bottom."""
found = capture.Capture(command="make")
found.absorb(b"START\n")
found.absorb(b"x" * (capture.CAPTURE_HEAD_BYTES * 2))
found.absorb(b"END\n")
output = found.output()
assert output.startswith("START")
assert output.rstrip().endswith("END")
assert "dropped" in output
def test_short_output_is_kept_whole():
found = capture.Capture(command="ls")
found.absorb(b"one\ntwo\n")
assert found.output() == "one\ntwo"
def test_a_progress_bar_collapses_to_its_last_state():
"""The highest-value transform here. Only the last state of a line was ever
on screen, and keeping every one turns two megabytes of `pip install` into
two megabytes of spinner in somebody's prompt."""
found = capture.Capture(command="pip install")
found.absorb(b"".join(f"\r{n}%".encode() for n in range(500)) + b"\ndone\n")
output = found.output()
assert output == "499%\ndone"
def test_escape_sequences_are_stripped():
found = capture.Capture(command="ls --color")
found.absorb(b"\033[31mred\033[0m\n")
assert found.output() == "red"
def test_a_split_multibyte_character_degrades_rather_than_raising():
"""Head/tail slicing splits UTF-8 at will, so the decode has to replace."""
found = capture.Capture(command="cat")
found.absorb("héllo".encode()[:3])
assert isinstance(found.output(), str)
def test_a_fence_cannot_be_ended_early_by_the_output():
"""A real injection route: output containing three backticks would close
the block, and everything after it would read to the model as prose rather
than as what a machine printed."""
block = capture.fenced("here are ``` three")
assert block.startswith("````")
assert block.rstrip().endswith("````")
def test_the_attribution_sits_outside_the_fence():
"""So nothing the far side printed can forge it. The `$ ` line is
synthesised here too -- what the shell echoed carries readline's editing
escapes and is not the command."""
found = capture.Capture(command="pytest -q", cwd="/srv/work", exit_status=1, ended=1.0)
found.absorb(b"1 failed\n")
text = found.as_text(label="Container")
assert text.startswith("Ran in the terminal on Container, in /srv/work — exit 1")
assert "```console\n$ pytest -q" in text
def test_output_still_running_is_capturable():
""""Copy the last command" while `make` is going should give what has been
printed so far, marked as running -- not "nothing yet"."""
found = capture.Capture(command="make")
found.absorb(b"compiling\n")
assert found.running
assert "still running" in found.as_text(label="Box")
assert "compiling" in found.as_text(label="Box")
def test_a_very_long_line_is_cut():
found = capture.Capture(command="cat bundle.min.js")
found.absorb(b"z" * (capture.MAX_LINE_CHARS * 3) + b"\n")
assert len(found.output()) <= capture.MAX_LINE_CHARS + 1
# --- The command string ------------------------------------------------------
def test_the_command_branches_on_the_shell_name():
command = shell_marks.command_for("/srv/work")
assert "case ${SHELL##*/} in" in command
assert "bash --rcfile" in command
assert "ZDOTDIR=" in command
def test_a_single_quote_in_the_directory_cannot_end_the_quoting():
command = shell_marks.command_for("/tmp/it's here; rm -rf /")
assert command.startswith("cd '/tmp/it'\\''s here; rm -rf /'")
def test_switching_integration_off_gives_back_exactly_what_was_there_before():
assert shell_marks.command_for("", integrate=False) is None
assert (
shell_marks.command_for("/srv/work", integrate=False)
== "cd '/srv/work' 2>/dev/null; exec ${SHELL:-/bin/sh} -l"
)
def test_the_integration_writes_nothing_to_the_terminals_input_side():
"""Which is the whole reason this mechanism was chosen over feeding
`source …` in as keystrokes. Nothing is echoed because nothing is typed, so
there is no setup to hide from the scrollback and no fan-out gate."""
command = shell_marks.command_for("/srv/work")
assert "printf %s" in command # written to a file
assert "$__L/rc" in command
# --- Against a real PTY ------------------------------------------------------
# A fake shell that speaks the markers rather than a real bash: what is under
# test here is the session's bookkeeping, and a real bash would make every
# assertion depend on whoever's dotfiles the machine happens to carry. The rc
# files themselves are verified by hand, once, per shell.
import asyncio # noqa: E402
from lembas.services.agent import terminal as terminal_service # noqa: E402
class _Server(asyncssh.SSHServer):
def begin_auth(self, username: str) -> bool:
return False
@pytest.fixture
async def marking_host():
seen: dict = {}
def osc(body: str) -> str:
return f"\033]{body}\007"
async def handler(process):
seen["command"] = process.command
process.stdout.write(osc("633;LEMBAS;bash;1"))
process.stdout.write(osc("633;P;Cwd=/srv/work") + osc("133;A") + "$ ")
while True:
try:
line = (await process.stdin.readline()).rstrip("\n")
except asyncssh.TerminalSizeChanged:
continue
except Exception: # noqa: BLE001
break
if not line or line == "exit":
break
process.stdout.write(osc("633;E;" + line.replace(";", "\\x3b")))
process.stdout.write(osc("133;C"))
status = 0
if line.startswith("fail"):
process.stdout.write("boom\n")
status = 3
elif line.startswith("spin"):
for n in range(200):
process.stdout.write(f"\r{n}%")
process.stdout.write("\ndone\n")
else:
process.stdout.write(f"out:{line}\n")
process.stdout.write(osc(f"133;D;{status}"))
process.stdout.write(osc("633;P;Cwd=/srv/work") + osc("133;A") + "$ ")
process.exit(0)
server = await asyncssh.create_server(
_Server,
"127.0.0.1",
0,
server_host_keys=[asyncssh.generate_private_key("ssh-ed25519")],
process_factory=handler,
)
port = next(iter(server.sockets)).getsockname()[1]
from lembas.services.agent import ssh as ssh_service
line, _fingerprint = await ssh_service.capture_host_key("127.0.0.1", port)
try:
yield {"port": port, "host_key": line, "seen": seen}
finally:
await terminal_service.shutdown()
server.close()
await server.wait_closed()
def _spec(host) -> dict:
return {
"host": "127.0.0.1",
"port": host["port"],
"username": "tester",
"auth": "password",
"password": "",
"private_key": "",
"key_passphrase": "",
"host_key": host["host_key"],
"connect_timeout": 10,
}
async def _session(host, **kwargs):
return await terminal_service.open_session(
"chat-1",
owner_id="user-1",
profile_id="profile-1",
label="Container",
spec=_spec(host),
**kwargs,
)
async def _settle(session, want, *, timeout=5.0):
"""Wait until `want(session)` holds, or give up."""
async with asyncio.timeout(timeout):
while not want(session):
await asyncio.sleep(0.02)
async def test_the_shell_says_it_loaded_the_integration(marking_host):
session = await _session(marking_host)
session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
assert session.shell == "bash"
await session.close()
async def test_a_command_and_its_output_are_captured_between_the_markers(marking_host):
session = await _session(marking_host)
session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
await session.send(b"ls -la\n")
await _settle(session, lambda s: s.last is not None)
assert session.last.command == "ls -la"
assert "out:ls -la" in session.last.output()
assert session.last.cwd == "/srv/work"
await session.close()
async def test_the_exit_code_comes_back_with_the_command(marking_host):
session = await _session(marking_host)
session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
await session.send(b"fail now\n")
await _settle(session, lambda s: s.last is not None)
assert session.last.exit_status == 3
assert not session.last.running
await session.close()
async def test_a_semicolon_in_the_command_survives_the_round_trip(marking_host):
session = await _session(marking_host)
session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
await session.send(b"cd /tmp ; ls\n")
await _settle(session, lambda s: s.last is not None)
assert session.last.command == "cd /tmp ; ls"
await session.close()
async def test_a_progress_bar_is_collapsed_before_it_reaches_a_prompt(marking_host):
session = await _session(marking_host)
session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
await session.send(b"spin\n")
await _settle(session, lambda s: s.last is not None)
output = session.last.output()
assert output.count("%") == 1
assert output.rstrip().endswith("done")
await session.close()
async def test_the_markers_still_reach_the_browser(marking_host):
"""Fanned out unchanged. xterm consumes an OSC it has no handler for and
never draws it, and rewriting frames on the hot path would break the
"nothing decodes, so nothing splits" property the pump depends on."""
session = await _session(marking_host)
viewer = session.attach()
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_LIVE)
seen = b""
while not viewer.queue.empty():
chunk = viewer.queue.get_nowait()
if isinstance(chunk, bytes):
seen += chunk
assert b"\033]133;A" in seen
await session.close()
async def test_a_shell_that_never_marks_ends_up_reported_as_none(marking_host, monkeypatch):
"""The fallback path's own test, and the one that matters most: without it
the buttons would sit greyed out forever with no explanation. This is what
catches a `.bashrc` that ends in `exec tmux`."""
monkeypatch.setattr(terminal_service, "INTEGRATION_GRACE", 0.0)
session = await _session(marking_host)
session.integration = terminal_service.INTEGRATION_LOADING
session._marks = shell_marks.Marks(lambda kind, value: None) # deaf on purpose
session.attach()
await session.send(b"anything\n")
await _settle(session, lambda s: s.integration == terminal_service.INTEGRATION_NONE)
await session.close()