A terminal panel beside an agent chat

A real shell on the chat's own connection, opened and closed like the
inspector and never beside it. The modes govern the model; what a person
types is theirs, since they hold the credential and could open the same
shell with an ssh client. The model cannot see the panel -- a button
copies the output you choose into the composer.

The session outlives the socket: closing the panel leaves a build
running, and coming back reattaches with the scrollback. Two tabs share
one shell and the smaller window decides the size. It ends on an idle
timeout, on deleting the chat, on disabling, moving or deleting the
connection, and on a restart -- which says why rather than quietly
opening a fresh shell that has lost the working directory.

The nginx template's `Connection ""` is right for SSE and fails every
WebSocket handshake, so `location /` now uses a `map $http_upgrade`;
update.sh grows a drift check for it, because the only symptom on a
stale vhost is a panel that cannot connect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-02 01:44:07 +02:00
parent 246be1fa8e
commit 5117168454
39 changed files with 2981 additions and 34 deletions
+84 -3
View File
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 837 tests, ~50s
pytest # 891 tests, ~55s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
@@ -35,7 +35,11 @@ redesign, not a tweak.
1. **No Node, no npm, no build step.** Browser libraries are downloaded once by
`scripts/fetch_vendor.py`, hash-pinned in `scripts/vendor.lock.json`, and
committed under `web/static/vendor/`.
committed under `web/static/vendor/`. Adding one is a `--update`, same as
bumping one: a name in `PACKAGES` with no lock entry has nothing to verify
against, and the script refuses rather than writing it unpinned. xterm is
the one heavy dependency — 280KB, more than everything else together — and
is loaded only on a chat that can open a terminal.
2. **Nothing loads from a CDN at runtime.** A self-hosted tool must work
offline and must not report page views to a third party.
3. **No hard-coded values outside `tokens.css`.** Every colour, space, radius
@@ -86,6 +90,7 @@ src/lembas/
admin_tools.py custom HTTP tools and MCP servers
admin_agents.py whether agent chats exist, and what they may spend
agents.py SSH connections, kept by the people who own them
terminal.py the terminal panel's WebSocket, and its two locks
audio.py transcribe, speak, voice discovery
library.py knowledge, notes, skills pages; memory CRUD
files.py upload, serve, remove attachments
@@ -101,7 +106,8 @@ src/lembas/
search/ ddgs, SearXNG and Firecrawl behind one shape
library/ documents, notes, memories, skills, FTS
mcp/ remote MCP servers: framing, transport, rows to tools
agent/ agent chats: the mode table, SSH, and the four tools
agent/ agent chats: the mode table, SSH, the four tools,
and terminal.py, the shells held open behind the panel
audio.py OpenAI-shaped /v1/audio/* client
fetch.py URL retrieval, HTML to text, the SSRF guard
sharing.py one visibility rule for every library store
@@ -650,6 +656,81 @@ passing one through a worker turns it into one delivery at the end, or nothing.
accepting `text/event-stream`. It is served from `GET /sw.js` rather than the
static mount because a worker's scope is the path it came from.
**XSS is now a root shell, not a leaked chat.** `api/terminal.py` is the one
WebSocket here, it is same-origin, the cookie rides along automatically, and
what it opens is an interactive shell. Every other route a script could reach
gives up a conversation; this one gives up the machine. Nothing about hard rule
6 changes — it was already absolute — but the *price* of getting it wrong did,
and so did the price of a stray `|safe`. The two locks are: the session cookie
is SameSite Lax, so a foreign page's handshake carries no cookie, and the
endpoint additionally **requires** an Origin header matching Host rather than
checking one when it happens to be present.
**A WebSocket dependency must be typed `HTTPConnection`.** `api/deps.py:
get_current_user` used to take a `Request`; FastAPI injects a `WebSocket` on a
websocket route, so the annotation fails at *connect* time rather than at
import. That is a failure which passes every test that does not open a socket
and breaks in a browser. `HTTPConnection` is the shared base and carries both
the cookies and `.state`.
**Terminal sessions are keyed on the chat, and outlive the socket.** A reload is
indistinguishable from a second tab, so anything finer needs an id in the
browser's storage — and then an abandoned tab leaks a PTY nothing in the UI can
find. One chat, one shell; two tabs share it and the smaller window decides the
size. Closing the panel calls `detach`, never `close`: a build running behind a
shut panel is the case the whole lifetime exists for. What ends one is the idle
timeout (nobody attached *and* nothing typed), deleting the chat, disabling,
moving or deleting the connection, forgetting its host key, or a restart.
**Unlike generations, nothing here ends by itself.** `generation.ensure` can
prune inside itself because a reply finishes and something calls in again. A
shell sits at a prompt forever, so `agent/terminal.py` runs a reaper task
instead. Copying the generation shape would mean nothing was ever swept.
**A slow viewer is dropped, not buffered.** Each viewer has a bounded queue; one
that fills is disconnected and reconnects with the scrollback, which costs it
nothing because the scrollback *is* the state. Blocking the pump instead would
stall every other viewer and buffer without bound — and `yes` is one word to
type. The reflex fix is an unbounded queue; it is the wrong one.
**Terminal traffic is bytes in both directions, and nothing decodes it.** A read
on the far side lands mid-character often enough to matter. xterm's decoder is
stateful across `write()` calls, so passing raw bytes through is correct by
construction, while decoding each frame server-side would corrupt every
boundary. Only `resize`, `ready`, `closed` and `error` are text, and they are
JSON.
**The modes do not govern the keyboard.** `agent/policy.py` exists because a
model reads pages, files and command output it did not write and can be talked
into things. A person typing into the panel holds the credential already and
could open the same shell with an ssh client, so nothing they type is checked
against the mode or the two lists. There is a test named after this, because it
reads like a bug next to `policy.py` and "fixing" it would make the panel
useless in the mode people spend the most time in.
**The nginx vhost must pass upgrades through.** `deploy/nginx-vhost.conf` used
to set `Connection ""`, which is right for SSE and fails every WebSocket
handshake — and a failed handshake tells the browser nothing: no status, no
reason. It now uses `map $http_upgrade`, which yields the empty string when
nothing asked to upgrade, so one `location` serves both. `update.sh` has a drift
check for exactly this.
**`data-toggle` syncs every toggle, not the one that was clicked.** A panel can
be opened by the topbar button and closed by its own Close, and now also closed
by nothing at all: `data-toggle-group="side"` makes the terminal and the
inspector mutually exclusive, because at 1280px both plus the sidebar leave the
conversation about seventy pixels wide. `app.js:setPanel` applies the state and
then brings every `[data-toggle]` pointing at that panel in line, and fires
`lembas:toggle` — which is how `terminal.js` learns it is visible and may
measure itself. xterm's `fit()` reads `offsetWidth`, which is 0 inside a
`[hidden]` ancestor, so fitting early is a silent no-op that leaves an
80-column terminal in a 34rem panel.
**xterm holds colours as values, so the theme has to be pushed at it.**
`applyTheme` dispatches `lembas:theme`; without it, switching to `shire` leaves
a black rectangle in a light interface. Same reason a `ResizeObserver` is on the
panel: a window `resize` never fires when the sidebar is toggled beside it.
## Changing the schema
There is no Alembic, but there *is* `db/migrations.py`. It compares the declared
+8
View File
@@ -112,6 +112,14 @@ be a different project, not a refactor.
switches to Edit and sends it back quoted rather than as an instruction
- [x] Per-reply budgets on steps, wall clock and output, with time spent
waiting for you subtracted
- [x] **A terminal panel** beside the chat, holding a real shell on that chat's
own connection. The modes govern the model; what a person types is theirs,
since they hold the credential and could open the same shell with an ssh
client. The model cannot see the panel — sending it output is a button
- [x] The shell outlives the panel and the page: closing it leaves a build
running, and coming back reattaches with the scrollback. An idle timeout
is what eventually ends one, and so does deleting the chat, or disabling,
moving or deleting the connection
### The library
- [x] **Knowledge bases** — documents, images and saved web pages, grouped into
+30
View File
@@ -63,6 +63,10 @@ runtime. Clone it, `pip install -e .`, run it.
but asks before commands, *Auto* asks about nothing, and *Plan* reads freely,
changes nothing, and finishes by proposing steps you can carry out with one
button. Adding a host shows you its fingerprint before anything is sent to it
- **A terminal beside the chat** — the same connection, a real shell, opened and
closed like any panel. It survives closing the panel and reloading the page,
so a build keeps running; the model cannot see it, and a button hands it the
output you choose
- **It can ask you things** — a model that needs a decision can stop and put a
few questions on one card, with answers to pick from and a box to write your
own. In any chat, not only an agent one
@@ -194,6 +198,32 @@ with. In **Auto**, nothing stands between that and a command running.
switches to *Edit*, never *Auto*, because the plan was written under a mode
where every command still asked.
#### The terminal
An agent chat has a **Terminal** button in its header, which opens a real shell
on that chat's connection, in its directory, beside the conversation. It needs
the *Open a terminal* permission, which is off by default.
The modes above do not apply to it. They exist because a model reads pages,
files and command output it did not write; you hold the credential and could
open the same shell with an ssh client, so nothing you type is queued for your
own approval. The model cannot see the panel either — the button in its header
puts the selection, or the last of the output, into the message box, where you
read it before it goes anywhere.
The shell is not tied to the panel. Close it and a build carries on; come back,
or reload, and you reattach with the scrollback. Two tabs share one shell, and
the smaller window decides the size. It ends when nobody has watched it and
nothing has been typed for a while, when the chat is deleted, when the
connection is disabled or deleted, or when LLeMbas restarts — a deploy cuts off
whatever was running, and the panel says so rather than quietly opening a fresh
shell that has lost your working directory.
> Nothing typed here is in the transcript and nothing is logged but the opening
> and the closing. If you are running this over plain http, note that the
> session cookie is not marked `secure` so a LAN install works at all — with a
> terminal switched on, that is worth a certificate.
### The library
**Sidebar → Library**, and **Settings → Memory**. Nothing is on by default for a
+34
View File
@@ -81,6 +81,29 @@ delivers it in one lump at the end, which is indistinguishable from streaming
being broken. `proxy_read_timeout` is raised to an hour because a model can
think for minutes before the first token.
**The vhost passes WebSocket upgrades through, and must.** The terminal panel
is the one WebSocket in LLeMbas. A `location` that sets `Connection ""` — which
is what SSE alone needs, and what this template used to say — fails every
handshake, and a failed handshake tells the browser nothing: no status, no
reason. The `map $http_upgrade` at the top of the vhost yields the empty string
when the client did not ask to upgrade, so streaming is unaffected. `update.sh`
warns when the installed vhost has drifted from the template, because this is
the failure most likely to be diagnosed as a bug in the application.
**Every restart kills every open shell.** A reply being written is persisted
with whatever it has; a terminal has nothing to persist, so a command still
running on the far side is cut off. `update.sh` restarts unconditionally, so a
deploy in the middle of somebody's `apt-get dist-upgrade` ends it. The panel is
told why rather than silently reconnecting to a new shell, which would have
lost the working directory and the half-typed command.
**A terminal is not in the transcript, and is not logged.** The open and the
close are logged with the user, the chat and the connection; what was typed is
not recorded anywhere. That follows from the design — the chat's mode governs
the model, not the person at the keyboard — but everything else an agent chat
does *is* in the transcript, so it is a difference in kind and worth knowing
before somebody goes looking for the history.
**Nothing an agent does runs on this machine.** Agent chats execute their
commands over SSH, on a host somebody added and prepared — a container, a VM,
another machine. That is the whole isolation story, and it is why the unit can
@@ -97,3 +120,14 @@ key to a production server, and LLeMbas cannot tell them apart.
**Use a real certificate if this is exposed beyond a trusted LAN.** The
self-signed cert exists so the install works with no external dependencies;
point `ssl_certificate` at a real one and nothing else needs to change.
The session cookie is deliberately not marked `secure`, so that a LAN install
over plain http can sign anybody in at all. That has always meant a network
attacker on http could steal a session; with the terminal it also means they
could open an interactive shell on the machine behind that chat. If the
terminal is switched on, run this over TLS.
**One worker only.** True of generations already — the registry is in-process —
and sharper here: with two workers a browser reconnecting to its terminal could
land in the process that has no shell for it, and silently open a second one on
the same machine.
+8
View File
@@ -136,6 +136,14 @@ sed -e "s|__SITE_HOST__|$SITE_HOST|g" -e "s|__APP_PORT__|$APP_PORT|g" \
sudo nginx -t
sudo systemctl reload nginx
# What this host was installed with, so update.sh can name the vhost it should
# be comparing against and print a command that actually runs. Without it the
# drift check below could only say "something changed somewhere".
printf 'SITE_HOST=%s\nAPP_PORT=%s\n' "$SITE_HOST" "$APP_PORT" \
| sudo tee "$PREFIX/.deploy-env" >/dev/null
sha256sum "$HERE/nginx-vhost.conf" | cut -d' ' -f1 \
| sudo tee "$PREFIX/.vhost-applied" >/dev/null
echo "== local name resolution =="
# Only useful when the LAN's DNS does not already answer for this name.
if ! getent hosts "$SITE_HOST" >/dev/null; then
+20 -3
View File
@@ -7,6 +7,19 @@
# install.sh generates. To use a real certificate, point ssl_certificate at it;
# nothing else here needs to change.
# The terminal panel is a WebSocket, and a proxy that does not pass an upgrade
# through breaks it with no error either side can report -- the browser sees a
# failed handshake, which carries no status and no reason. This map yields
# "upgrade" only when the client asked for one and the empty string otherwise,
# which is exactly what the streamed-reply case below needs, so one `location`
# serves both. `conf.d/*.conf` is included inside `http {}`, where `map` is
# legal; the name is prefixed because two vhosts from this template would
# otherwise collide.
map $http_upgrade $lembas_connection_upgrade {
default upgrade;
'' '';
}
server {
listen 80;
listen [::]:80;
@@ -42,9 +55,13 @@ server {
proxy_buffering off;
proxy_request_buffering off;
proxy_cache off;
# SSE is plain HTTP/1.1 chunked, not a websocket upgrade, so the
# connection header must simply be left to keep-alive.
proxy_set_header Connection "";
# SSE is plain HTTP/1.1 chunked and needs Connection left empty; the
# terminal is a real upgrade and needs it set. The map at the top of
# this file is what lets one location do both -- a hard-coded
# `Connection ""` here, which is what was here before, works for every
# streamed reply and silently breaks every terminal.
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $lembas_connection_upgrade;
# A model can think for minutes before the first token. The default
# 60s read timeout would cut long generations off mid-sentence.
+44 -3
View File
@@ -56,9 +56,8 @@ sudo -u "$SERVICE_USER" "$VENV/bin/pip" install --quiet -e "$APP[$LEMBAS_EXTRAS]
# about the local lines, and a warning that always fires is one nobody reads.
#
# The drift is worth catching: a change in the unit can be what makes a release
# work at all. Dropping ProtectKernelTunables is why an agent chat can start a
# sandbox, and a host that pulled the code without it would run the new version
# under the old confinement and fail confusingly.
# work at all, and a host that pulled the code without it would run the new
# version under the old settings and fail confusingly.
STAMP="$PREFIX/.unit-applied"
current=$(sha256sum "$APP/deploy/lembas.service" | cut -d' ' -f1)
if [[ -f "$STAMP" && "$(cat "$STAMP")" != "$current" ]]; then
@@ -77,6 +76,48 @@ elif [[ ! -f "$STAMP" ]]; then
echo "$current" | sudo tee "$STAMP" >/dev/null
fi
# The same argument for the vhost, and the failure is worse. A stale unit at
# least says something in the journal; a stale vhost breaks a feature two layers
# away, and the only symptom is a panel that says it could not connect. The
# terminal is a WebSocket, and a `location` that does not pass an upgrade
# through fails every handshake while every test in the suite still passes.
VHOST_STAMP="$PREFIX/.vhost-applied"
vhost_now=$(sha256sum "$APP/deploy/nginx-vhost.conf" | cut -d' ' -f1)
site_host=""; app_port="8080"
# Written by install.sh. Absent on a deployment that predates it, in which case
# the commands below are a template rather than a copy-paste.
if [[ -f "$PREFIX/.deploy-env" ]]; then
. "$PREFIX/.deploy-env"
site_host="$SITE_HOST"; app_port="$APP_PORT"
fi
installed_vhost="/etc/nginx/conf.d/${site_host:-your-host}.conf"
vhost_stale=""
if [[ -f "$VHOST_STAMP" ]]; then
[[ "$(cat "$VHOST_STAMP")" != "$vhost_now" ]] && vhost_stale="the template has changed"
elif [[ -n "$site_host" && -f "$installed_vhost" ]]; then
# First run with this check, so there is no stamp to compare against. Rather
# than assume what is installed is current -- which is what the unit check
# does, and would hide exactly the change this was added for -- look for the
# one thing that must be there. Everything else is left to the stamp.
grep -q 'lembas_connection_upgrade' "$installed_vhost" \
|| vhost_stale="the installed vhost does not pass WebSocket upgrades through, so the terminal cannot connect"
fi
if [[ -n "$vhost_stale" ]]; then
echo "== nginx vhost ==" >&2
echo " $vhost_stale." >&2
echo " Review and reinstall it:" >&2
echo " diff $installed_vhost <(sed \\" >&2
echo " -e 's|__SITE_HOST__|${site_host:-your-host}|g' -e 's|__APP_PORT__|$app_port|g' \\" >&2
echo " $APP/deploy/nginx-vhost.conf)" >&2
echo " Then: sudo nginx -t && sudo systemctl reload nginx" >&2
echo " And record it as applied: echo $vhost_now | sudo tee $VHOST_STAMP" >&2
elif [[ ! -f "$VHOST_STAMP" ]]; then
echo "$vhost_now" | sudo tee "$VHOST_STAMP" >/dev/null
fi
echo "== restart =="
sudo systemctl restart lembas
sleep 2
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "lembas"
version = "0.4.0"
version = "0.5.0"
description = "LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints"
readme = "README.md"
requires-python = ">=3.11"
+32 -2
View File
@@ -3,11 +3,12 @@
LLeMbas has no Node toolchain and loads nothing from a CDN at runtime -- a
self-hosted tool should keep working without internet access, and should not
report every user's page view to a third party. The three libraries it does use
report every user's page view to a third party. The few libraries it does use
are fetched once, here, and committed.
Integrity is enforced with vendor.lock.json. A mismatched hash aborts rather
than overwriting: that is the whole point of pinning.
than overwriting, and so does a name that is not in the lock at all: that is
the whole point of pinning.
python scripts/fetch_vendor.py # fetch and verify against the lock
python scripts/fetch_vendor.py --update # re-pin after a version bump
@@ -44,6 +45,21 @@ PACKAGES = {
"url": "https://unpkg.com/alpinejs@3.15.12/dist/cdn.min.js",
"why": "Small client-only state: menus, theme toggle, composer autosize.",
},
"xterm.js": {
"version": "5.5.0",
"url": "https://unpkg.com/@xterm/xterm@5.5.0/lib/xterm.js",
"why": "The terminal panel. Loaded only on a chat that has an SSH connection.",
},
"xterm.css": {
"version": "5.5.0",
"url": "https://unpkg.com/@xterm/xterm@5.5.0/css/xterm.css",
"why": "Terminal layout. Its colours are overridden from tokens.css at runtime.",
},
"xterm-addon-fit.js": {
"version": "0.10.0",
"url": "https://unpkg.com/@xterm/addon-fit@0.10.0/lib/addon-fit.js",
"why": "Sizes the terminal to the panel; without it a resize is 80x24 forever.",
},
}
@@ -83,6 +99,20 @@ def main() -> int:
digest = sha256(payload)
expected = lock.get(filename, {}).get("sha256")
if lock and not expected and not args.update:
# A name added to PACKAGES but absent from the lock has nothing to
# compare against, so the mismatch branch below never fires and the
# file lands unpinned -- which is the one thing this script exists
# to prevent. Adding a library is a --update, like bumping one.
print(
f" FAIL {filename}: not in {LOCKFILE.name}\n"
f" Nothing to verify this download against. If the "
f"library was added deliberately, re-run with --update.",
file=sys.stderr,
)
failed = True
continue
if expected and digest != expected and not args.update:
print(
f" FAIL {filename}: hash mismatch\n"
+15
View File
@@ -13,5 +13,20 @@
"sha256": "71ea67185bfa8c98c39d31717c6fce5d852370fcdfd129db4543774d3145c0de",
"url": "https://unpkg.com/htmx.org@2.0.10/dist/htmx.min.js",
"version": "2.0.10"
},
"xterm-addon-fit.js": {
"sha256": "bdaefa370b1bfc42ee88d46fe6072400902a4d4b2d45cd93438dda9b23c97089",
"url": "https://unpkg.com/@xterm/addon-fit@0.10.0/lib/addon-fit.js",
"version": "0.10.0"
},
"xterm.css": {
"sha256": "ba8e6985669488981ccf40c0cefe3aba80722cb6c92de7ad628b0bd717faf2b6",
"url": "https://unpkg.com/@xterm/xterm@5.5.0/css/xterm.css",
"version": "5.5.0"
},
"xterm.js": {
"sha256": "1f991ac3b4b283ebf96e60ae23a00a52765dd3a2e46fa6fdda9f1aab032f7495",
"url": "https://unpkg.com/@xterm/xterm@5.5.0/lib/xterm.js",
"version": "5.5.0"
}
}
+1 -1
View File
@@ -1,3 +1,3 @@
"""LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints."""
__version__ = "0.4.0"
__version__ = "0.5.0"
+10
View File
@@ -21,6 +21,7 @@ from lembas.db.models import SshProfile
from lembas.services import settings_store
from lembas.services.agent import policy
from lembas.services.agent import ssh as ssh_service
from lembas.services.agent import terminal as terminal_service
from lembas.web.templating import render
log = logging.getLogger(__name__)
@@ -45,6 +46,7 @@ async def agents_page(request: Request, db: Db, user: AdminUser, saved: bool = F
"deny_text": "\n".join(values.get("deny_default") or []),
"problem": ssh_service.available(),
"profile_count": db.scalar(select(func.count()).select_from(SshProfile)) or 0,
"terminal_count": terminal_service.count(),
"modes": [(m, policy.MODE_LABELS[m], policy.MODE_HINTS[m]) for m in policy.MODES],
"saved": saved,
},
@@ -66,6 +68,10 @@ async def save_agents(
allow_default: str = Form(""),
deny_default: str = Form(""),
ask_free_text: bool = Form(False),
terminal_enabled: bool = Form(False),
terminal_idle_timeout: int = Form(1800),
terminal_max_sessions: int = Form(20),
terminal_max_per_user: int = Form(3),
) -> Response:
settings_store.update(
db,
@@ -84,6 +90,10 @@ async def save_agents(
"allow_default": _lines(allow_default),
"deny_default": _lines(deny_default),
"ask_free_text": ask_free_text,
"terminal_enabled": terminal_enabled,
"terminal_idle_timeout": min(max(terminal_idle_timeout, 60), 86400),
"terminal_max_sessions": min(max(terminal_max_sessions, 1), 500),
"terminal_max_per_user": min(max(terminal_max_per_user, 1), 50),
},
key=settings_store.AGENTS,
)
+12
View File
@@ -25,6 +25,7 @@ from lembas.api.pages import sidebar_context
from lembas.db.models import AUTH_METHODS, AUTH_PASSWORD, SshProfile
from lembas.services import settings_store
from lembas.services.agent import ssh as ssh_service
from lembas.services.agent import terminal as terminal_service
from lembas.services.agent.base import ExecError
from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
from lembas.web.templating import render
@@ -293,6 +294,9 @@ async def forget_host_key(request: Request, db: Db, user: RequiredUser, profile_
profile = _profile(db, user, profile_id)
profile.host_key = ""
profile.host_fingerprint = ""
# Un-trusting a host has to reach the shell already open on it, or the one
# connection that matters is the one this does not touch.
await terminal_service.close_for_profile(profile.id)
db.commit()
return render(request, "agents/_check.html", {"profile": profile, "forgotten": True})
@@ -301,6 +305,7 @@ async def forget_host_key(request: Request, db: Db, user: RequiredUser, profile_
async def delete_profile(db: Db, user: RequiredUser, profile_id: str) -> Response:
profile = _profile(db, user, profile_id)
name = profile.name
await terminal_service.close_for_profile(profile.id)
db.delete(profile)
db.commit()
log.info("%s deleted ssh profile %s", user.email, name)
@@ -342,6 +347,13 @@ async def update_profile(request: Request, db: Db, user: RequiredUser, profile_i
profile.host_fingerprint = ""
log.info("%s moved ssh profile %s; its host key was forgotten", user.email, profile.name)
# A shell already open holds its own connection and would not notice any of
# this. `session.profile_for` re-checks the profile on every reply, so the
# model stops at once; without the line below, "I disabled that connection"
# would simply not be true of the terminal on screen.
if not profile.enabled or not profile.host_key or (profile.host, profile.port) != before:
await terminal_service.close_for_profile(profile.id)
db.commit()
return RedirectResponse(
f"/agents/{profile.id}?saved=Saved.", status_code=status.HTTP_303_SEE_OTHER
+10
View File
@@ -34,9 +34,19 @@ def _set_session_cookie(response: Response, token: str) -> None:
# Lax is what makes this application CSRF-safe without tokens: the
# cookie is not sent on cross-site POSTs, and every mutating route here
# is a POST. Do not relax to "none".
#
# One route is no longer a POST: the terminal WebSocket is a GET, and
# what it opens is a shell. Lax still withholds the cookie from a
# handshake a foreign page starts, so the attack is blocked -- but the
# sentence above is no longer the whole story, which is why
# `api/terminal.py` also *requires* a same-origin Origin header rather
# than merely checking one when it happens to be there.
samesite="lax",
# Only over HTTPS when the deployment is not plain local http. Marking
# it secure on http would silently break sign-in for a LAN install.
# It has always meant "a network attacker on plain http can steal a
# session"; with the terminal it also means they get a shell on the
# machine behind that chat. See deploy/README.md.
secure=False,
path="/",
)
+5
View File
@@ -36,6 +36,7 @@ from lembas.services import prompts as prompts_service
from lembas.services import settings_store, sse
from lembas.services import tools as tools_service
from lembas.services.agent import policy as agent_policy
from lembas.services.agent import terminal as terminal_service
from lembas.services.markdown import escape_text, render_markdown
from lembas.web.templating import render, templates
@@ -997,6 +998,10 @@ def _clean_params(**submitted: str | None) -> dict[str, float | int | None]:
@router.delete("/{chat_id}", dependencies=[Depends(require_permission("chat.delete"))])
async def delete_chat(db: Db, user: RequiredUser, chat_id: str) -> Response:
chat = _owned_chat(db, chat_id, user.id)
# Before the row goes: a terminal is keyed on the chat id, so afterwards
# there would be nothing left to find it by and a shell would sit open on
# somebody's machine until the idle timeout noticed.
await terminal_service.close_chat(chat_id)
db.delete(chat)
db.commit()
+13 -6
View File
@@ -8,6 +8,7 @@ from typing import Annotated
from fastapi import Depends, HTTPException, Request, status
from fastapi.responses import RedirectResponse
from sqlalchemy.orm import Session as DBSession
from starlette.requests import HTTPConnection
from lembas.db.models import User
from lembas.db.session import get_session_factory
@@ -26,17 +27,23 @@ def get_db() -> Iterator[DBSession]:
Db = Annotated[DBSession, Depends(get_db)]
def get_current_user(request: Request, db: Db) -> User | None:
def get_current_user(conn: HTTPConnection, db: Db) -> User | None:
"""Resolve the session cookie to a user, or None when signed out.
Cached on request.state so several dependencies in one request do not each
hit the sessions table.
Cached on the connection's state so several dependencies in one request do
not each hit the sessions table.
`HTTPConnection` rather than `Request` because the terminal panel is a
WebSocket, and FastAPI injects a `WebSocket` there -- annotating this
`Request` fails at *connect* time rather than at import, so it would pass
every smoke test and break in a browser. `HTTPConnection` is the base of
both and carries the cookies and the state either way.
"""
cached = getattr(request.state, "user", None)
cached = getattr(conn.state, "user", None)
if cached is not None:
return cached
user = resolve_session(db, request.cookies.get(COOKIE_NAME))
request.state.user = user
user = resolve_session(db, conn.cookies.get(COOKIE_NAME))
conn.state.user = user
return user
+22
View File
@@ -97,9 +97,31 @@ def _agent_context(db: DBSession, user: User, chat: Chat | None) -> dict:
(m, agent_policy.MODE_LABELS[m], agent_policy.MODE_HINTS[m])
for m in agent_policy.MODES
],
"terminal_enabled": _terminal_enabled(db, user, chat, current),
}
def _terminal_enabled(db: DBSession, user: User, chat: Chat | None, profile) -> bool:
"""Whether this chat can offer a shell of its own.
Every condition, not a subset: the button loads 280KB of terminal and opens
a socket, so one that cannot work is worse than none. `ssh.available()` is
in here because an instance that installed LLeMbas without the `ssh` extra
would otherwise render a button whose only outcome is an error frame.
"""
from lembas.db.models import KIND_AGENT
from lembas.services.agent import ssh as ssh_service
if chat is None or chat.kind != KIND_AGENT or profile is None:
return False
if not permissions.has(db, user, "agent.terminal"):
return False
values = settings_store.agents(db)
if not values.get("enabled") or not values.get("terminal_enabled", True):
return False
return ssh_service.available() == ""
def sidebar_context(db: DBSession, user: User) -> dict:
"""Folder tree plus the chats that belong to no folder.
+251
View File
@@ -0,0 +1,251 @@
"""The socket behind the terminal panel.
A WebSocket rather than SSE, because SSE is one-directional and a terminal is
not: keystrokes have to go up, and an HTTP round trip per keypress is not a
terminal. It is the only WebSocket in LLeMbas, and it is worth saying what that
costs -- a cross-site page that could reach this endpoint would have a shell on
somebody's machine, not merely a copy of their chat. So there are two locks on
the door, and this module is mostly them.
**Where a refusal happens is load-bearing.** A browser tells a page nothing
about a handshake that *failed*: `new WebSocket()` fires `error` with no status
and no reason. So the socket is accepted first and the reason sent as a frame
for everything a person could act on -- no permission, the connection is
disabled, its host key was never confirmed -- and refused before accepting only
for the two cases where accepting is itself the risk.
It holds no database session. A dependency would keep one open for the hour a
shell sits at a prompt; `session_scope()` opens one for the authorisation and
closes it, exactly as `generation._run` does.
"""
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
from urllib.parse import urlsplit
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
from lembas.db.models import KIND_AGENT, Chat
from lembas.db.session import session_scope
from lembas.security import permissions
from lembas.security.sessions import COOKIE_NAME, resolve_session
from lembas.services import settings_store
from lembas.services.agent import session as agent_session
from lembas.services.agent import terminal as terminal_service
from lembas.services.agent.base import ExecError
log = logging.getLogger(__name__)
router = APIRouter(prefix="/api/chats", tags=["terminal"])
# Nothing a keyboard produces is anywhere near this. Paste is the only thing
# that comes close, and a megabyte pasted into a shell is a mistake either way.
MAX_INPUT_BYTES = 256 * 1024
# 1008 is "policy violation", the closest thing the protocol has to "no".
CLOSE_POLICY = 1008
# What the far side is told when a shell ends, in words rather than a code.
CLOSED_WORDS = {
terminal_service.CLOSED_EXITED: "The shell exited.",
terminal_service.CLOSED_IDLE: "This terminal was closed after sitting idle.",
terminal_service.CLOSED_SHUTDOWN: "LLeMbas restarted, so this shell was closed.",
terminal_service.CLOSED_REVOKED: "The connection behind this terminal was closed.",
terminal_service.CLOSED_ERROR: "The connection to the machine was lost.",
}
def _same_origin(websocket: WebSocket) -> bool:
"""Whether this handshake came from a page served by this site.
Required, not merely checked when present. The session cookie is SameSite
Lax, which already withholds it from a handshake a foreign page starts, and
this is the belt to that brace -- an absent Origin is not a browser, and a
non-browser client has no business here.
"""
origin = websocket.headers.get("origin")
host = websocket.headers.get("host")
if not origin or not host:
return False
return urlsplit(origin).netloc.lower() == host.lower()
def _prepare(db, user, chat_id: str) -> tuple[str, dict]:
"""Everything that has to be true, and what opening needs. One or the other.
Returns a message to show, or the arguments for `open_session`. The order is
the order somebody would ask the questions in, and every "no" is a sentence
rather than a silence.
"""
if not permissions.has(db, user, "agent.terminal"):
return "You do not have permission to open a terminal.", {}
chat = db.get(Chat, chat_id)
if chat is None or chat.user_id != user.id:
return "That chat no longer exists.", {}
if chat.kind != KIND_AGENT:
return "This is an ordinary chat, so it has no machine to open a shell on.", {}
values = settings_store.agents(db)
if not values.get("terminal_enabled", True):
return "The terminal is switched off on this instance.", {}
context = agent_session.resolve(db, chat, user)
if context is None:
return (
"This chat's connection is not usable: it may have been deleted, "
"disabled, or agent chats may be switched off here.",
{},
)
return "", {
"owner_id": user.id,
"profile_id": chat.ssh_profile_id or "",
"label": context.label,
"spec": context.spec,
"project_dir": context.project_dir,
"idle_timeout": float(values["terminal_idle_timeout"]),
"max_sessions": int(values["terminal_max_sessions"]),
"max_per_user": int(values["terminal_max_per_user"]),
}
@router.websocket("/{chat_id}/terminal/ws")
async def terminal_socket(
websocket: WebSocket,
chat_id: str,
cols: int = 80,
rows: int = 24,
) -> None:
if not _same_origin(websocket):
await websocket.close(code=CLOSE_POLICY)
return
with session_scope() as db:
user = resolve_session(db, websocket.cookies.get(COOKIE_NAME))
if user is None:
await websocket.close(code=CLOSE_POLICY)
return
problem, opening = _prepare(db, user, chat_id)
owner_email = user.email
await websocket.accept()
if problem:
await _refuse(websocket, problem)
return
try:
session = await terminal_service.open_session(chat_id, cols=cols, rows=rows, **opening)
except ExecError as exc:
await _refuse(websocket, str(exc))
return
except Exception: # noqa: BLE001 - a failure here is one socket, not the app
log.exception("could not open a terminal for %s", owner_email)
await _refuse(websocket, "The shell could not be started.")
return
viewer = session.attach(cols, rows)
await websocket.send_text(
json.dumps(
{
"t": "ready",
"label": session.label,
"dir": session.project_dir,
"cols": session.cols,
"rows": session.rows,
# Two tabs share one shell, and a size neither of them chose is
# otherwise a mystery.
"shared": len(session.viewers) > 1,
}
)
)
if viewer.snapshot:
await websocket.send_bytes(viewer.snapshot)
downward = asyncio.create_task(_to_browser(websocket, session, viewer))
upward = asyncio.create_task(_from_browser(websocket, session, viewer))
try:
await asyncio.wait({downward, upward}, return_when=asyncio.FIRST_COMPLETED)
finally:
for task in (downward, upward):
task.cancel()
with contextlib.suppress(asyncio.CancelledError, Exception):
await task
# The session is deliberately left running. Closing the panel, or
# navigating away, is not "I am finished with this machine" -- a build
# carries on and the scrollback is still there on the way back. The
# idle timeout is what eventually ends it.
session.detach(viewer)
async def _to_browser(websocket: WebSocket, session, viewer) -> None:
"""Everything the shell says, plus the one frame that says it stopped."""
while True:
chunk = await viewer.queue.get()
if chunk is None:
reason = terminal_service.CLOSED_EXITED if viewer.dropped else session.closed_reason
payload = {"t": "closed", "reason": reason, "message": _words(reason)}
if viewer.dropped:
# Not the session's doing: this browser stopped reading and was
# disconnected so the others kept up. Reconnecting costs it
# nothing, because the scrollback is the state.
payload = {"t": "behind", "message": "Reconnecting: output arrived faster than "
"this window could draw it."}
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps(payload))
return
await websocket.send_bytes(chunk)
async def _from_browser(websocket: WebSocket, session, viewer) -> None:
"""Keystrokes as binary, everything else as JSON.
Binary for the hot path is what makes multi-byte characters safe: a read on
the far side lands mid-sequence often enough to matter, and decoding each
frame here would corrupt every boundary. Nothing decodes, so nothing splits.
"""
while True:
try:
message = await websocket.receive()
except WebSocketDisconnect:
return
if message["type"] == "websocket.disconnect":
return
data = message.get("bytes")
if data is not None:
if len(data) > MAX_INPUT_BYTES:
continue
await session.send(data)
continue
text = message.get("text")
if text:
_control(session, viewer, text)
def _control(session, viewer, text: str) -> None:
try:
payload = json.loads(text)
except ValueError:
return
if not isinstance(payload, dict) or payload.get("t") != "resize":
return
session.resize(viewer, payload.get("cols", 80), payload.get("rows", 24))
def _words(reason: str) -> str:
return CLOSED_WORDS.get(reason, "This terminal closed.")
async def _refuse(websocket: WebSocket, message: str) -> None:
"""Say why, then close. Sent as a frame because a browser cannot read a
rejected handshake -- the reason would be lost exactly when it is needed."""
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"t": "error", "message": message}))
with contextlib.suppress(Exception):
await websocket.close()
+7
View File
@@ -31,6 +31,7 @@ from lembas.api import (
library,
pages,
preferences,
terminal,
)
from lembas.api.deps import RedirectToLogin, is_htmx, login_redirect
from lembas.config import settings
@@ -90,9 +91,14 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
# Replies still being written are cancelled and persisted with whatever
# they have, rather than left as permanently unfinished rows.
from lembas.services.agent.terminal import shutdown as stop_terminals
from lembas.services.generation import shutdown as stop_generations
await stop_generations()
# Open shells have nothing to persist: whatever was running on the far side
# is cut off mid-command. Every deploy does this, and the panel is told why
# rather than left to guess -- see deploy/README.md.
await stop_terminals()
log.info("LLeMbas stopped")
@@ -112,6 +118,7 @@ def create_app() -> FastAPI:
app.include_router(auth.router)
app.include_router(preferences.router)
app.include_router(chats.router)
app.include_router(terminal.router)
app.include_router(audio.router)
app.include_router(files.router)
app.include_router(folders.router)
+9
View File
@@ -119,6 +119,15 @@ PERMISSION_DEFS: tuple[PermissionDef, ...] = (
False,
"Agent",
),
PermissionDef(
"agent.terminal",
"Open a terminal",
"Open an interactive shell on one of their own SSH connections, from "
"inside the chat. What they type there is theirs: the chat's mode "
"governs the model, not the person at the keyboard.",
False,
"Agent",
),
PermissionDef(
"tools.ask",
"Be asked questions",
+3 -2
View File
@@ -90,7 +90,7 @@ def spec_from(profile: SshProfile) -> dict[str, Any]:
}
def _connect_kwargs(spec: dict[str, Any]) -> dict[str, Any]:
def connect_kwargs(spec: dict[str, Any]) -> dict[str, Any]:
"""Everything asyncssh must be told rather than left to discover.
See the module docstring: every one of these has a default that is wrong
@@ -175,7 +175,7 @@ class SshExecutor:
raise ExecError(problem)
import asyncssh
return asyncssh.connect(self.spec["host"], **_connect_kwargs(self.spec))
return asyncssh.connect(self.spec["host"], **connect_kwargs(self.spec))
def _wrap(self, exc: Exception) -> ExecError:
import asyncssh
@@ -345,5 +345,6 @@ __all__ = [
"available",
"capture_host_key",
"check",
"connect_kwargs",
"spec_from",
]
+576
View File
@@ -0,0 +1,576 @@
"""Interactive shells, one per agent chat, held open behind the panel.
The other half of `ssh.py`. There a connection lives for one command, because a
runner is a request-and-answer and holding state would be the wrong shape. Here
the connection *is* the state: a PTY with a shell on the far side, its scrollback,
and whoever is currently watching it.
Shaped after `services/generation.py` -- a registry, a background task that owns
the work, and a socket that merely follows it -- and it differs in three ways
worth knowing:
* **Keyed on the chat, not on a session of its own.** A reload is
indistinguishable from a second tab, so anything finer needs an id in the
browser's storage, and then an abandoned tab leaks a shell nothing in the UI
can find. One chat, one shell. Two tabs share it, like `tmux attach` twice,
which is the only reading under which "it is still there when you come back"
means anything. They also share a size, and the smaller one wins.
* **Nothing here ends by itself.** A generation finishes, so `generation.ensure`
can prune inside itself. A shell sits at a prompt forever and nothing calls in
again, so there is a reaper task instead. Copying the generation shape here
would mean nothing was ever swept.
* **A slow reader is dropped, not buffered.** Every viewer has a bounded queue;
one that fills is disconnected and reattaches with the scrollback. Blocking
the pump instead would stall every other viewer and buffer without bound
inside the server -- and `yes` is one word to type.
What a person types here is deliberately not run past `agent/policy.py`. The
modes and the two lists govern a *model*, which reads untrusted pages and files
and can be talked into things. Somebody at a keyboard holds the credential
already and could open the same shell with an ssh client; asking them to approve
their own keystrokes would be theatre.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import time
import uuid
from collections import deque
from dataclasses import dataclass, field
from typing import Any
from lembas.services.agent.base import ExecError
from lembas.services.agent.ssh import available, connect_kwargs
log = logging.getLogger(__name__)
# What one shell keeps to hand a returning viewer. Bytes rather than lines: a
# line budget is dishonest about a program that writes one very long line.
SCROLLBACK_BYTES = 256 * 1024
# How much output one viewer may fall behind by before it is dropped. Frames
# are whatever the far side wrote, so this is generous in wall-clock terms and
# only reached by a browser that has genuinely stopped reading.
VIEWER_QUEUE = 512
# Read size. Large enough that `cat` of a big file is not a million wakeups,
# small enough that a prompt appears the instant it is written.
READ_BYTES = 64 * 1024
# A terminal nobody has ever heard of gets no colours; this one every shell
# knows and it is what an ordinary ssh client announces.
TERM_TYPE = "xterm-256color"
# A size is a number the browser sends. `change_terminal_size(100000, 100000)`
# is a way to ask the far side to allocate.
MAX_COLS = 500
MAX_ROWS = 300
MIN_COLS = 20
MIN_ROWS = 5
# How long a closed session stays in the registry. A tab attaching a second
# after the shell exited should be told what happened rather than silently
# handed a fresh one.
KEEP_CLOSED = 60.0
# How often the reaper looks. Nothing here is urgent: the idle timeout is
# measured in minutes.
REAP_INTERVAL = 30.0
# Why a session ended. The browser is told, and the wording differs enough to be
# worth the constants.
CLOSED_EXITED = "exited"
CLOSED_IDLE = "idle"
CLOSED_SHUTDOWN = "shutdown"
CLOSED_REVOKED = "revoked"
CLOSED_ERROR = "error"
@dataclass
class Viewer:
"""One browser watching one shell."""
cols: int = 80
rows: int = 24
id: str = field(default_factory=lambda: uuid.uuid4().hex)
queue: asyncio.Queue = field(default_factory=lambda: asyncio.Queue(maxsize=VIEWER_QUEUE))
# Everything the shell has said so far, handed over in the same synchronous
# call that subscribes. Reading the buffer and subscribing as two awaits
# loses whatever arrives between them.
snapshot: bytes = b""
# Set when this viewer fell behind. It is woken with the sentinel below and
# told to reconnect, which costs it nothing: the scrollback is the state.
dropped: bool = False
class Session:
"""A shell on the far side of one chat's connection."""
def __init__(
self,
chat_id: str,
*,
owner_id: str,
profile_id: str,
label: str,
project_dir: str = "",
idle_timeout: float = 1800.0,
cols: int = 80,
rows: int = 24,
) -> None:
self.chat_id = chat_id
self.owner_id = owner_id
self.profile_id = profile_id
self.label = label
self.project_dir = project_dir
self.idle_timeout = idle_timeout
self.viewers: dict[str, Viewer] = {}
self._scrollback: deque[bytes] = deque()
self._scrollback_bytes = 0
self._conn: Any = None
self._process: Any = None
self._pump: asyncio.Task | None = None
self.closed = False
self.closed_reason = ""
self.closed_at = 0.0
self.started_at = time.monotonic()
# Bumped by a keystroke and by a viewer coming or going. Idle is this
# going quiet *with nobody attached*: a build running behind a closed
# panel is the case this whole lifetime exists for.
self.last_active = time.monotonic()
# The size the PTY is *created* with, which matters: a shell prints its
# prompt before anything could resize it, and a prompt drawn at 80
# columns inside a 140-column window stays wrong until the next one.
self.cols = _clamp(cols, MIN_COLS, MAX_COLS)
self.rows = _clamp(rows, MIN_ROWS, MAX_ROWS)
# --- Opening -------------------------------------------------------------
async def start(self, spec: dict[str, Any]) -> None:
"""Connect, ask for a PTY, and start pumping what it says.
The credential is used here and not kept. `spec` is a decrypted snapshot
of a profile and the connection outlives the request that made it, so
holding a private key in memory for the hour a shell sits at a prompt
buys nothing.
"""
if problem := available():
raise ExecError(problem)
import asyncssh
try:
self._conn = await asyncssh.connect(
spec["host"],
**connect_kwargs(spec),
# asyncssh sends no keepalives by default. `SshExecutor` never
# needed them because its connections live for one command; a
# shell held open behind NAT otherwise gets dropped with no FIN
# and no exception, and the pump simply never returns -- the
# panel looks alive and answers nothing.
keepalive_interval=30,
keepalive_count_max=3,
)
self._process = await self._conn.create_process(
self._command(),
term_type=TERM_TYPE,
term_size=(self.cols, self.rows),
# Bytes in both directions. A read lands mid-character often
# enough to matter, and the browser's decoder is stateful across
# writes while a per-frame decode here is not: it would corrupt
# every boundary. Nothing decodes, so nothing can split.
encoding=None,
stderr=asyncssh.STDOUT,
)
except asyncssh.HostKeyNotVerifiable as exc:
await self._teardown()
raise ExecError(
f"{self.label} presented a different host key than the one that "
"was confirmed. Nothing was sent."
) from exc
except asyncssh.PermissionDenied as exc:
await self._teardown()
raise ExecError(f"{self.label} refused the credential.") from exc
except (OSError, asyncssh.Error) as exc:
await self._teardown()
raise ExecError(f"Could not reach {self.label}: {exc}") from exc
self._pump = asyncio.create_task(self._read_forever())
def _command(self) -> str | None:
"""What the PTY runs, or None for the account's plain login shell.
A shell has no notion of "start here" that SSH can carry, so the chat's
project directory has to be a `cd` -- run before the shell rather than
typed into it, so the scrollback opens on a prompt instead of on a
command nobody entered. It is single-quoted, and a failure is ignored:
a directory that has been deleted should leave somebody at a shell to
find out why, not with a connection that closes as it opens.
"""
if not self.project_dir:
return None
quoted = "'" + self.project_dir.replace("'", "'\\''") + "'"
return f"cd {quoted} 2>/dev/null; exec ${{SHELL:-/bin/sh}} -l"
# --- Following -----------------------------------------------------------
def attach(self, cols: int = 80, rows: int = 24) -> Viewer:
"""Subscribe, and take the scrollback, in one synchronous step.
One step on purpose: reading the buffer and subscribing as two awaits
loses whatever the shell says between them, which is exactly the moment
somebody reattaches to a build that is still writing.
"""
viewer = Viewer(
cols=_clamp(cols, MIN_COLS, MAX_COLS),
rows=_clamp(rows, MIN_ROWS, MAX_ROWS),
)
viewer.snapshot = b"".join(self._scrollback)
self.viewers[viewer.id] = viewer
self.last_active = time.monotonic()
self.apply_size()
return viewer
def detach(self, viewer: Viewer) -> None:
self.viewers.pop(viewer.id, None)
# Counts as activity: the timeout is "nobody has been here and nothing
# has happened for a while", so it starts when the last viewer leaves.
self.last_active = time.monotonic()
self.apply_size()
async def send(self, data: bytes) -> None:
"""Type into the shell."""
if self.closed or self._process is None:
return
self.last_active = time.monotonic()
try:
self._process.stdin.write(data)
except (BrokenPipeError, ConnectionResetError, OSError):
await self._finish(CLOSED_EXITED)
def resize(self, viewer: Viewer, cols: int, rows: int) -> None:
"""Record this viewer's size and give the far side the smallest.
Two tabs on one PTY cannot each have their own geometry. The smaller
wins in both directions, so nothing is drawn off the edge of the smaller
window -- the larger one gets an unused margin, which is the harmless
half of the trade.
"""
viewer.cols = _clamp(cols, MIN_COLS, MAX_COLS)
viewer.rows = _clamp(rows, MIN_ROWS, MAX_ROWS)
self.apply_size()
def apply_size(self) -> None:
"""Synchronous: `change_terminal_size` only queues a window-change
message, so there is nothing to await and no reason to make every
caller a coroutine."""
if self.closed or self._process is None or not self.viewers:
return
cols = min(v.cols for v in self.viewers.values())
rows = min(v.rows for v in self.viewers.values())
if (cols, rows) == (self.cols, self.rows):
return
self.cols, self.rows = cols, rows
with contextlib.suppress(Exception):
self._process.change_terminal_size(cols, rows)
# --- The pump ------------------------------------------------------------
async def _read_forever(self) -> None:
assert self._process is not None
try:
while True:
data = await self._process.stdout.read(READ_BYTES)
if not data:
break
self._remember(data)
self._fan_out(data)
except asyncio.CancelledError:
raise
except Exception as exc: # noqa: BLE001 - one shell dying is not a crash
log.info("terminal %s ended: %s", self.chat_id, exc)
await self._finish(CLOSED_ERROR)
return
await self._finish(CLOSED_EXITED)
def _remember(self, data: bytes) -> None:
self._scrollback.append(data)
self._scrollback_bytes += len(data)
while self._scrollback_bytes > SCROLLBACK_BYTES and len(self._scrollback) > 1:
self._scrollback_bytes -= len(self._scrollback.popleft())
def _fan_out(self, data: bytes) -> None:
for viewer in list(self.viewers.values()):
try:
viewer.queue.put_nowait(data)
except asyncio.QueueFull:
# Emptied first so the sentinel fits and so the socket does not
# spend its last moments writing frames nobody will see.
_drain(viewer.queue)
viewer.dropped = True
with contextlib.suppress(asyncio.QueueFull):
viewer.queue.put_nowait(None)
self.viewers.pop(viewer.id, None)
# --- Closing -------------------------------------------------------------
async def close(self, reason: str = CLOSED_SHUTDOWN) -> None:
pump, self._pump = self._pump, None
if pump is not None:
pump.cancel()
with contextlib.suppress(asyncio.CancelledError, Exception):
await pump
await self._finish(reason)
async def _finish(self, reason: str) -> None:
"""Mark this session over and wake everybody watching.
Called from the pump when the shell exits, and from `close` after the
pump has been cancelled -- which is why it does not cancel the pump
itself. The entry stays in the registry for KEEP_CLOSED so a late
attachment gets an explanation.
"""
if self.closed:
return
self.closed = True
self.closed_reason = reason
self.closed_at = time.monotonic()
for viewer in list(self.viewers.values()):
with contextlib.suppress(asyncio.QueueFull):
viewer.queue.put_nowait(None)
await self._teardown()
log.info(
"terminal closed chat=%s owner=%s profile=%s reason=%s after=%.0fs",
self.chat_id,
self.owner_id,
self.profile_id,
reason,
time.monotonic() - self.started_at,
)
async def _teardown(self) -> None:
process, self._process = self._process, None
conn, self._conn = self._conn, None
if process is not None:
with contextlib.suppress(Exception):
process.terminate()
if conn is not None:
with contextlib.suppress(Exception):
conn.close()
with contextlib.suppress(Exception):
await conn.wait_closed()
@property
def idle_for(self) -> float:
if self.viewers:
return 0.0
return time.monotonic() - self.last_active
# --- The registry ------------------------------------------------------------
_SESSIONS: dict[str, Session] = {}
_REAPER: asyncio.Task | None = None
def get(chat_id: str) -> Session | None:
"""The live session for a chat, if there is one. Closed ones do not count."""
session = _SESSIONS.get(chat_id)
if session is None or session.closed:
return None
return session
def peek(chat_id: str) -> Session | None:
"""As `get`, but a recently closed session too -- it carries the reason."""
return _SESSIONS.get(chat_id)
def count() -> int:
return sum(1 for s in _SESSIONS.values() if not s.closed)
def count_for(owner_id: str) -> int:
return sum(1 for s in _SESSIONS.values() if not s.closed and s.owner_id == owner_id)
async def open_session(
chat_id: str,
*,
owner_id: str,
profile_id: str,
label: str,
spec: dict[str, Any],
project_dir: str = "",
idle_timeout: float = 1800.0,
max_sessions: int = 20,
max_per_user: int = 3,
cols: int = 80,
rows: int = 24,
) -> Session:
"""The shell for this chat, opening one if it is not already there.
Idempotent for the same reason `generation.ensure` is: a second tab, or the
same tab after a reload, must attach to what is running rather than start a
second shell on the same machine.
"""
existing = get(chat_id)
if existing is not None:
return existing
_reap()
if count() >= max_sessions:
raise ExecError(
"This instance already has as many terminals open as it allows. "
"Close one, or ask an administrator to raise the limit."
)
if count_for(owner_id) >= max_per_user:
raise ExecError(
f"You already have {max_per_user} terminal"
f"{'' if max_per_user == 1 else 's'} open. Close one first."
)
session = Session(
chat_id,
owner_id=owner_id,
profile_id=profile_id,
label=label,
project_dir=project_dir,
idle_timeout=idle_timeout,
cols=cols,
rows=rows,
)
await session.start(spec)
_SESSIONS[chat_id] = session
_ensure_reaper()
log.info(
"terminal opened chat=%s owner=%s profile=%s host=%s dir=%s",
chat_id,
owner_id,
profile_id,
label,
project_dir or "~",
)
return session
async def close_chat(chat_id: str, reason: str = CLOSED_REVOKED) -> bool:
session = _SESSIONS.pop(chat_id, None)
if session is None:
return False
await session.close(reason)
return True
async def close_for_profile(profile_id: str) -> int:
"""End every shell opened on one connection.
`session.profile_for` re-checks the profile on every reply, so deleting or
disabling one stops the model at once. A terminal resolves the profile
when it opens and then holds the connection, so without this "I disabled
that connection" would simply not be true of the shell already on screen.
"""
doomed = [s for s in _SESSIONS.values() if s.profile_id == profile_id and not s.closed]
for session in doomed:
_SESSIONS.pop(session.chat_id, None)
await session.close(CLOSED_REVOKED)
return len(doomed)
async def close_for_owner(owner_id: str) -> int:
doomed = [s for s in _SESSIONS.values() if s.owner_id == owner_id and not s.closed]
for session in doomed:
_SESSIONS.pop(session.chat_id, None)
await session.close(CLOSED_REVOKED)
return len(doomed)
async def shutdown() -> None:
"""End every shell. Called from the lifespan, beside stop_generations."""
global _REAPER
reaper, _REAPER = _REAPER, None
if reaper is not None:
reaper.cancel()
with contextlib.suppress(asyncio.CancelledError, Exception):
await reaper
for session in list(_SESSIONS.values()):
await session.close(CLOSED_SHUTDOWN)
_SESSIONS.clear()
def _reap() -> None:
"""Drop sessions that have been closed long enough to stop explaining."""
now = time.monotonic()
for chat_id, session in list(_SESSIONS.items()):
if session.closed and now - session.closed_at > KEEP_CLOSED:
_SESSIONS.pop(chat_id, None)
def _ensure_reaper() -> None:
global _REAPER
if _REAPER is None or _REAPER.done():
_REAPER = asyncio.create_task(_reaper_loop())
async def _reaper_loop() -> None:
"""Close idle shells, then forget closed ones.
A task rather than a sweep inside `open_session`, which is the shape
`generation` uses. That works there because a generation ends on its own and
something calls in again; a shell at a prompt does neither, so a lazy sweep
would run only when somebody opened the *next* terminal.
"""
while True:
try:
await asyncio.sleep(REAP_INTERVAL)
for session in list(_SESSIONS.values()):
if not session.closed and session.idle_for > session.idle_timeout:
_SESSIONS.pop(session.chat_id, None)
await session.close(CLOSED_IDLE)
_reap()
except asyncio.CancelledError:
raise
except Exception: # noqa: BLE001 - the reaper must outlive one bad sweep
log.exception("the terminal reaper raised")
def _drain(queue: asyncio.Queue) -> None:
while True:
try:
queue.get_nowait()
except asyncio.QueueEmpty:
return
def _clamp(value: int, low: int, high: int) -> int:
try:
number = int(value)
except (TypeError, ValueError):
return low
return min(max(number, low), high)
__all__ = [
"CLOSED_EXITED",
"CLOSED_IDLE",
"CLOSED_REVOKED",
"CLOSED_SHUTDOWN",
"Session",
"Viewer",
"close_chat",
"close_for_owner",
"close_for_profile",
"count",
"count_for",
"get",
"open_session",
"peek",
"shutdown",
]
+24 -3
View File
@@ -75,6 +75,19 @@ def _agents_defaults() -> dict[str, Any]:
"allow_default": ["file_read", "file_list", "ls *", "pwd", "git status"],
"deny_default": ["shutdown *", "reboot *", "mkfs*"],
"ask_free_text": True,
# The terminal panel: a person's own shell on their own connection.
# Separate from `enabled` because the two are different capabilities --
# one lets a model run commands, the other lets a human do what they
# could already do with an ssh client. Neither implies the other.
"terminal_enabled": True,
# Seconds with nobody watching *and* nothing typed before the session is
# closed. A build running with the panel shut is not idle. Clamped on
# read: zero would leave a shell open until the next restart.
"terminal_idle_timeout": 1800,
# Open shells across the instance, and per person. Each is a PTY and an
# SSH connection held open, so this is a real resource, not a scruple.
"terminal_max_sessions": 20,
"terminal_max_per_user": 3,
}
@@ -213,14 +226,22 @@ def search(db: DBSession) -> dict[str, Any]:
def agents(db: DBSession) -> dict[str, Any]:
"""Agent settings, with the two numbers that must not be zero clamped.
"""Agent settings, with the numbers that must not be zero clamped.
`approval_timeout` of 0 would park a background task on a question nobody
is going to answer, and nothing else prunes a generation that is not
finished. Clamped on read rather than on save, so a value already stored by
an earlier version cannot bite either.
finished. `terminal_idle_timeout` of 0 would keep a PTY and an SSH
connection open until the next restart. Clamped on read rather than on save,
so a value already stored by an earlier version cannot bite either.
"""
values = get_group(db, AGENTS)
values["approval_timeout"] = min(max(int(values.get("approval_timeout") or 0), 60), 3600)
values["max_timeout"] = min(max(int(values.get("max_timeout") or 0), 1), 3600)
values["terminal_idle_timeout"] = min(
max(int(values.get("terminal_idle_timeout") or 0), 60), 86400
)
values["terminal_max_sessions"] = min(
max(int(values.get("terminal_max_sessions") or 0), 1), 500
)
values["terminal_max_per_user"] = min(max(int(values.get("terminal_max_per_user") or 0), 1), 50)
return values
+79
View File
@@ -510,6 +510,85 @@ button, input, textarea, select {
}
}
/* The terminal panel: a fourth child of .shell, to the left of the inspector.
Built beside it rather than in chat.css because the shell layout lives here,
and the two are the same shape -- a fixed-width column that hides with the
`hidden` attribute. */
.terminal {
width: var(--terminal-width);
flex: none;
display: flex;
flex-direction: column;
min-height: 0;
background: var(--bg-sunken);
border-left: 1px solid var(--border);
}
.terminal__header {
display: flex;
align-items: center;
gap: var(--sp-2);
height: var(--header-height);
flex: none;
padding: 0 var(--sp-3);
border-bottom: 1px solid var(--border);
}
.terminal__title {
display: flex;
align-items: center;
gap: var(--sp-2);
flex: 1;
min-width: 0;
font-size: var(--text-sm);
font-weight: 600;
color: var(--ink-muted);
}
.terminal__where {
font-weight: 400;
font-family: var(--font-mono);
font-size: var(--text-xs);
color: var(--ink-faint);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
/* The element xterm renders into. It measures itself from this box, so it must
have a size of its own -- min-height: 0 on a flex child is what stops the
terminal growing the panel instead of scrolling inside it. */
.terminal__screen {
flex: 1;
min-height: 0;
padding: var(--sp-2);
background: var(--code-bg);
}
.terminal__screen .xterm { height: 100%; }
.terminal__status {
flex: none;
display: flex;
align-items: center;
gap: var(--sp-2);
padding: var(--sp-2) var(--sp-3);
border-top: 1px solid var(--border);
font-size: var(--text-xs);
color: var(--ink-faint);
line-height: var(--leading-normal);
}
.terminal__status strong { color: var(--ink-muted); font-weight: 600; }
.terminal__message { flex: 1; min-width: 0; overflow-wrap: anywhere; }
.terminal__message--error { color: var(--danger); }
@media (max-width: 64rem) {
.terminal {
position: fixed;
inset: 0 0 0 auto;
width: min(var(--terminal-width), 100vw);
z-index: 40;
box-shadow: var(--shadow-lg);
}
}
.topbar {
display: flex;
align-items: center;
+4
View File
@@ -66,6 +66,10 @@
/* --- Layout ----------------------------------------------------------- */
--sidebar-width: 17.5rem;
--inspector-width: 24rem;
/* Wider than the inspector because the content is not prose: eighty columns
of --font-mono do not fit in 24rem, and a terminal narrower than eighty
re-wraps everything a program prints. */
--terminal-width: 34rem;
--thread-max-width: 48rem;
--header-height: 3.5rem;
+48 -4
View File
@@ -41,6 +41,12 @@
: "Switch to Moria (dark)");
});
/* For anything holding colours as values rather than reading them from a
variable. The terminal is the only such thing: xterm copies its palette
at construction, so switching to Shire would otherwise leave a black
rectangle in a light interface. */
document.dispatchEvent(new CustomEvent("lembas:theme", { detail: { theme: name } }));
if (document.body.dataset.authenticated === "true") {
fetch("/api/preferences/theme", {
method: "POST",
@@ -357,7 +363,48 @@
revealInstall(false);
});
/* --- Panels ------------------------------------------------------------- */
/* A panel can be opened or closed by more than one control -- the button in
the topbar and the panel's own Close -- and it can now also be closed by
something nobody clicked, because two panels sharing the right-hand side
of the screen must not both be open. So the state is applied to the panel
and then *every* toggle pointing at it is brought in line. Setting
aria-expanded on the clicked button alone left the other one lying. */
function syncToggles(selector, open) {
var toggles = document.querySelectorAll('[data-toggle="' + selector + '"]');
for (var i = 0; i < toggles.length; i++) {
toggles[i].setAttribute("aria-expanded", open ? "true" : "false");
toggles[i].classList.toggle("is-active", open);
}
}
function setPanel(selector, open, group) {
var panel = document.querySelector(selector);
if (!panel) return;
/* One at a time down the right-hand side. Not only a narrow-screen
concern: a 1280px window with the sidebar, the inspector and the
terminal all open leaves the conversation about seventy pixels wide. */
if (open && group) {
var others = document.querySelectorAll('[data-toggle-group="' + group + '"]');
for (var i = 0; i < others.length; i++) {
var other = others[i].dataset.toggle;
if (other && other !== selector) setPanel(other, false);
}
}
panel.toggleAttribute("hidden", !open);
syncToggles(selector, open);
/* What a panel needs to know it is visible. The terminal listens for this:
xterm cannot measure itself inside a hidden element, so it has to be
told rather than left to discover. */
panel.dispatchEvent(
new CustomEvent("lembas:toggle", { bubbles: true, detail: { open: open } })
);
}
window.lembas = {
setPanel: setPanel,
applyTheme: applyTheme,
toggleTheme: toggleTheme,
copyText: copyText,
@@ -410,10 +457,7 @@
event.preventDefault();
var panel = document.querySelector(toggle.dataset.toggle);
if (!panel) return;
var nowOpen = panel.hasAttribute("hidden");
panel.toggleAttribute("hidden");
toggle.setAttribute("aria-expanded", nowOpen ? "true" : "false");
toggle.classList.toggle("is-active", nowOpen);
setPanel(toggle.dataset.toggle, panel.hasAttribute("hidden"), toggle.dataset.toggleGroup);
}
});
+4
View File
@@ -29,6 +29,10 @@ var SHELL = [
"/static/js/app.js",
"/static/js/ui.js",
"/static/js/audio.js",
"/static/js/terminal.js",
// Deliberately not the three xterm files below it: ~300KB precached on every
// install, for a panel most people never open, to spare one fetch from the
// people who do. The runtime branch caches them the first time it is opened.
"/static/vendor/htmx.min.js",
"/static/vendor/htmx-ext-sse.js",
"/static/vendor/alpine.min.js",
+300
View File
@@ -0,0 +1,300 @@
/*
The terminal panel.
Loaded only on a chat that can actually open a shell -- see the head and
scripts blocks in chat/index.html -- because xterm is nearly three times the
size of everything else vendored here. The Terminal object itself is built on
the first *open* rather than on load, so even here nothing is parsed for
somebody who never presses the button.
Three things about xterm that are easy to get wrong, and cost an afternoon
each:
* `fit()` measures `offsetWidth`, which is 0 inside a `[hidden]` ancestor, so
fitting while closed silently does nothing and leaves an 80-column terminal
in a 34rem panel. Everything below is arranged so a fit only ever happens
after the panel is visible.
* A window `resize` event does not fire when the sidebar is toggled or a panel
opens beside this one, which is by far the commonest way the panel changes
size. Hence the ResizeObserver.
* xterm does not read CSS variables. The theme is built from the computed
style at open time and rebuilt when the theme changes, or switching to
`shire` leaves a black rectangle in a light interface.
*/
(function () {
"use strict";
var panel = null;
var term = null;
var fit = null;
var socket = null;
var screen = null;
var messageEl = null;
var observer = null;
var closedOnPurpose = false;
function say(text, isError) {
if (!messageEl) return;
messageEl.textContent = text;
messageEl.classList.toggle("terminal__message--error", !!isError);
}
/* --- Theme -------------------------------------------------------------- */
function readTheme() {
var style = getComputedStyle(document.documentElement);
function token(name, fallback) {
return (style.getPropertyValue(name) || "").trim() || fallback;
}
return {
background: token("--code-bg", "#0C0F13"),
foreground: token("--ink", "#E8E2D4"),
cursor: token("--accent", "#C9A227"),
cursorAccent: token("--code-bg", "#0C0F13"),
selectionBackground: token("--accent-soft", "rgba(201, 162, 39, 0.3)")
};
}
/* --- Sizing ------------------------------------------------------------- */
function visible() {
return panel && !panel.hasAttribute("hidden") && panel.offsetWidth > 0;
}
function refit() {
if (!term || !fit || !visible()) return;
try {
fit.fit();
} catch (error) {
return;
}
send({ t: "resize", cols: term.cols, rows: term.rows });
}
/* --- The socket --------------------------------------------------------- */
function send(payload) {
if (socket && socket.readyState === WebSocket.OPEN) {
socket.send(JSON.stringify(payload));
}
}
function connect() {
if (socket) return;
closedOnPurpose = false;
var base = location.protocol === "https:" ? "wss://" : "ws://";
var url =
base + location.host + panel.dataset.url +
"?cols=" + (term.cols || 80) + "&rows=" + (term.rows || 24);
say("Connecting…");
socket = new WebSocket(url);
socket.binaryType = "arraybuffer";
socket.onmessage = function (event) {
if (typeof event.data === "string") return control(event.data);
/* Written straight through as bytes. xterm's decoder is stateful across
calls, so a multi-byte character split across two frames still lands
correctly -- which is exactly why the server never decodes either. */
term.write(new Uint8Array(event.data));
};
socket.onclose = function () {
socket = null;
if (!closedOnPurpose) say("Disconnected. Close and reopen to reconnect.");
};
socket.onerror = function () {
/* A failed handshake gives the page nothing: no status, no reason. So
this is a guess, and it names the likeliest cause rather than
pretending to know. */
say("Could not connect. If this instance is behind a proxy, it may not " +
"be passing WebSocket upgrades through.", true);
};
}
function control(raw) {
var payload;
try {
payload = JSON.parse(raw);
} catch (error) {
return;
}
if (payload.t === "ready") {
say(payload.shared
? "Connected. This shell is also open in another tab, and they share a size."
: "Connected.");
if (payload.dir) {
var where = panel.querySelector("[data-terminal-where]");
if (where) where.textContent = payload.dir;
}
/* The server may have opened the shell at a size chosen by whoever got
here first, so ask for ours now that there is something to ask. */
refit();
term.focus();
return;
}
if (payload.t === "behind") {
/* This window stopped reading and was disconnected so the others kept
up. Reconnecting costs nothing: the scrollback is the state. */
say(payload.message || "Reconnecting…");
closedOnPurpose = true;
if (socket) socket.close();
socket = null;
term.reset();
connect();
return;
}
if (payload.t === "closed" || payload.t === "error") {
say(payload.message || "This terminal closed.", payload.t === "error");
closedOnPurpose = true;
/* Deliberately no reconnect. A new shell has lost the working directory,
the environment and the half-typed command, and quietly substituting
one is worse than saying the connection went. */
}
}
/* --- Building it -------------------------------------------------------- */
function build() {
if (term) return true;
if (typeof Terminal === "undefined" || typeof FitAddon === "undefined") {
say("The terminal could not be loaded.", true);
return false;
}
term = new Terminal({
allowProposedApi: true,
convertEol: false,
cursorBlink: true,
fontFamily: getComputedStyle(document.documentElement)
.getPropertyValue("--font-mono").trim() || "monospace",
fontSize: 13,
scrollback: 5000,
theme: readTheme()
});
/* The module namespace is the UMD global, so the class is a property of
it. `new FitAddon()` is the mistake that reads correctly. */
fit = new FitAddon.FitAddon();
term.loadAddon(fit);
term.open(screen);
term.onData(function (data) {
if (socket && socket.readyState === WebSocket.OPEN) {
socket.send(new TextEncoder().encode(data));
}
});
/* Ctrl+C is interrupt here, which is correct and will still surprise
somebody. Copy and paste are the shifted pair, as in every terminal. */
term.attachCustomKeyEventHandler(function (event) {
if (!event.ctrlKey || !event.shiftKey || event.type !== "keydown") return true;
var key = event.key.toLowerCase();
if (key === "c") {
var selection = term.getSelection();
if (selection) navigator.clipboard.writeText(selection);
return false;
}
if (key === "v") {
navigator.clipboard.readText().then(function (text) {
if (socket && socket.readyState === WebSocket.OPEN) {
socket.send(new TextEncoder().encode(text));
}
});
return false;
}
return true;
});
/* The panel changes size when the sidebar is toggled or the window is
resized, and only the second of those fires a `resize` event. */
if (window.ResizeObserver) {
observer = new ResizeObserver(function () {
refit();
});
observer.observe(panel);
}
return true;
}
function open() {
if (!build()) return;
/* Next frame: the panel has just had `hidden` removed and has no measured
width yet, so fitting now would be the silent no-op this file exists to
avoid. */
requestAnimationFrame(function () {
refit();
connect();
term.focus();
});
}
/* --- Send to chat ------------------------------------------------------- */
/* Into the composer, never sent. What a machine printed is exactly the sort
of text somebody should read before a model does, and the box is where
that happens. */
function sendToChat() {
if (!term) return;
var text = term.getSelection();
if (!text) {
var lines = [];
var buffer = term.buffer.active;
var last = buffer.baseY + buffer.cursorY;
for (var y = Math.max(0, last - 40); y <= last; y++) {
var line = buffer.getLine(y);
if (line) lines.push(line.translateToString(true));
}
text = lines.join("\n").replace(/\n+$/, "");
}
if (!text.trim()) {
say("Nothing to send: select some output first.");
return;
}
var input = document.querySelector("[data-composer-input]");
if (!input) return;
var fence = "```\n" + text + "\n```\n";
input.value = input.value ? input.value.replace(/\s*$/, "\n\n") + fence : fence;
if (window.lembas && window.lembas.autosize) window.lembas.autosize(input);
input.focus();
/* "As it appeared" and not "as it was written": the buffer holds what is on
screen, hard-wrapped at the terminal's width, with no way to tell a wrap
from a newline. */
say("Copied into the message box as it appeared on screen.");
}
/* --- Wiring ------------------------------------------------------------- */
function start() {
panel = document.querySelector("[data-terminal]");
if (!panel) return;
screen = panel.querySelector("[data-terminal-screen]");
messageEl = panel.querySelector("[data-terminal-message]");
panel.addEventListener("lembas:toggle", function (event) {
if (event.detail && event.detail.open) open();
/* Closing leaves the Terminal object and the socket alone. `write()` is
internally queued, so disposing mid-output drops it, and keeping the
object is what makes reopening instant. The session on the far side
outlives this panel by design. */
});
panel.addEventListener("click", function (event) {
if (event.target.closest("[data-terminal-send]")) {
event.preventDefault();
sendToChat();
}
});
/* xterm holds colours as values, not as variables, so a theme change has
to be pushed into it. */
document.addEventListener("lembas:theme", function () {
if (term) term.options.theme = readTheme();
});
}
if (document.readyState === "loading") {
document.addEventListener("DOMContentLoaded", start);
} else {
start();
}
})();
+2
View File
@@ -0,0 +1,2 @@
!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.FitAddon=t():e.FitAddon=t()}(self,(()=>(()=>{"use strict";var e={};return(()=>{var t=e;Object.defineProperty(t,"__esModule",{value:!0}),t.FitAddon=void 0,t.FitAddon=class{activate(e){this._terminal=e}dispose(){}fit(){const e=this.proposeDimensions();if(!e||!this._terminal||isNaN(e.cols)||isNaN(e.rows))return;const t=this._terminal._core;this._terminal.rows===e.rows&&this._terminal.cols===e.cols||(t._renderService.clear(),this._terminal.resize(e.cols,e.rows))}proposeDimensions(){if(!this._terminal)return;if(!this._terminal.element||!this._terminal.element.parentElement)return;const e=this._terminal._core,t=e._renderService.dimensions;if(0===t.css.cell.width||0===t.css.cell.height)return;const r=0===this._terminal.options.scrollback?0:e.viewport.scrollBarWidth,i=window.getComputedStyle(this._terminal.element.parentElement),o=parseInt(i.getPropertyValue("height")),s=Math.max(0,parseInt(i.getPropertyValue("width"))),n=window.getComputedStyle(this._terminal.element),l=o-(parseInt(n.getPropertyValue("padding-top"))+parseInt(n.getPropertyValue("padding-bottom"))),a=s-(parseInt(n.getPropertyValue("padding-right"))+parseInt(n.getPropertyValue("padding-left")))-r;return{cols:Math.max(2,Math.floor(a/t.css.cell.width)),rows:Math.max(1,Math.floor(l/t.css.cell.height))}}}})(),e})()));
//# sourceMappingURL=addon-fit.js.map
+218
View File
@@ -0,0 +1,218 @@
/**
* Copyright (c) 2014 The xterm.js authors. All rights reserved.
* Copyright (c) 2012-2013, Christopher Jeffrey (MIT License)
* https://github.com/chjj/term.js
* @license MIT
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
* Originally forked from (with the author's permission):
* Fabrice Bellard's javascript vt100 for jslinux:
* http://bellard.org/jslinux/
* Copyright (c) 2011 Fabrice Bellard
* The original design remains. The terminal itself
* has been extended to include xterm CSI codes, among
* other features.
*/
/**
* Default styles for xterm.js
*/
.xterm {
cursor: text;
position: relative;
user-select: none;
-ms-user-select: none;
-webkit-user-select: none;
}
.xterm.focus,
.xterm:focus {
outline: none;
}
.xterm .xterm-helpers {
position: absolute;
top: 0;
/**
* The z-index of the helpers must be higher than the canvases in order for
* IMEs to appear on top.
*/
z-index: 5;
}
.xterm .xterm-helper-textarea {
padding: 0;
border: 0;
margin: 0;
/* Move textarea out of the screen to the far left, so that the cursor is not visible */
position: absolute;
opacity: 0;
left: -9999em;
top: 0;
width: 0;
height: 0;
z-index: -5;
/** Prevent wrapping so the IME appears against the textarea at the correct position */
white-space: nowrap;
overflow: hidden;
resize: none;
}
.xterm .composition-view {
/* TODO: Composition position got messed up somewhere */
background: #000;
color: #FFF;
display: none;
position: absolute;
white-space: nowrap;
z-index: 1;
}
.xterm .composition-view.active {
display: block;
}
.xterm .xterm-viewport {
/* On OS X this is required in order for the scroll bar to appear fully opaque */
background-color: #000;
overflow-y: scroll;
cursor: default;
position: absolute;
right: 0;
left: 0;
top: 0;
bottom: 0;
}
.xterm .xterm-screen {
position: relative;
}
.xterm .xterm-screen canvas {
position: absolute;
left: 0;
top: 0;
}
.xterm .xterm-scroll-area {
visibility: hidden;
}
.xterm-char-measure-element {
display: inline-block;
visibility: hidden;
position: absolute;
top: 0;
left: -9999em;
line-height: normal;
}
.xterm.enable-mouse-events {
/* When mouse events are enabled (eg. tmux), revert to the standard pointer cursor */
cursor: default;
}
.xterm.xterm-cursor-pointer,
.xterm .xterm-cursor-pointer {
cursor: pointer;
}
.xterm.column-select.focus {
/* Column selection mode */
cursor: crosshair;
}
.xterm .xterm-accessibility:not(.debug),
.xterm .xterm-message {
position: absolute;
left: 0;
top: 0;
bottom: 0;
right: 0;
z-index: 10;
color: transparent;
pointer-events: none;
}
.xterm .xterm-accessibility-tree:not(.debug) *::selection {
color: transparent;
}
.xterm .xterm-accessibility-tree {
user-select: text;
white-space: pre;
}
.xterm .live-region {
position: absolute;
left: -9999px;
width: 1px;
height: 1px;
overflow: hidden;
}
.xterm-dim {
/* Dim should not apply to background, so the opacity of the foreground color is applied
* explicitly in the generated class and reset to 1 here */
opacity: 1 !important;
}
.xterm-underline-1 { text-decoration: underline; }
.xterm-underline-2 { text-decoration: double underline; }
.xterm-underline-3 { text-decoration: wavy underline; }
.xterm-underline-4 { text-decoration: dotted underline; }
.xterm-underline-5 { text-decoration: dashed underline; }
.xterm-overline {
text-decoration: overline;
}
.xterm-overline.xterm-underline-1 { text-decoration: overline underline; }
.xterm-overline.xterm-underline-2 { text-decoration: overline double underline; }
.xterm-overline.xterm-underline-3 { text-decoration: overline wavy underline; }
.xterm-overline.xterm-underline-4 { text-decoration: overline dotted underline; }
.xterm-overline.xterm-underline-5 { text-decoration: overline dashed underline; }
.xterm-strikethrough {
text-decoration: line-through;
}
.xterm-screen .xterm-decoration-container .xterm-decoration {
z-index: 6;
position: absolute;
}
.xterm-screen .xterm-decoration-container .xterm-decoration.xterm-decoration-top-layer {
z-index: 7;
}
.xterm-decoration-overview-ruler {
z-index: 8;
position: absolute;
top: 0;
right: 0;
pointer-events: none;
}
.xterm-decoration-top {
z-index: 2;
position: relative;
}
File diff suppressed because one or more lines are too long
@@ -177,6 +177,53 @@
</div>
</section>
<section class="card">
<h2 class="card__title">The terminal</h2>
<p class="field__hint">
A panel beside an agent chat holding an interactive shell on that chat's
own connection. What somebody types there is <em>theirs</em>: the modes and
the two lists above govern the model, not the person at the keyboard, who
could open the same shell with an ssh client. The model cannot see the
panel; sending it something is a button they press.
</p>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="terminal_enabled" value="true"
{{ 'checked' if values.terminal_enabled }}>
<span>Allow the terminal panel</span>
</label>
<p class="field__hint">
People also need the <strong>Open a terminal</strong> permission.
{{ terminal_count }} shell{{ '' if terminal_count == 1 else 's' }} open right now.
</p>
</div>
<div class="field">
<label class="field__label" for="terminal_idle_timeout">Close a shell after</label>
<input class="input" id="terminal_idle_timeout" name="terminal_idle_timeout"
value="{{ values.terminal_idle_timeout }}" inputmode="numeric">
<p class="field__hint">
Seconds with nobody watching <em>and</em> nothing typed. Closing the
panel does not end the session — a build carries on and is still there
on the way back — so this is what eventually ends one.
</p>
</div>
<div class="field">
<label class="field__label" for="terminal_max_sessions">Most shells at once</label>
<input class="input" id="terminal_max_sessions" name="terminal_max_sessions"
value="{{ values.terminal_max_sessions }}" inputmode="numeric">
</div>
<div class="field">
<label class="field__label" for="terminal_max_per_user">Most shells per person</label>
<input class="input" id="terminal_max_per_user" name="terminal_max_per_user"
value="{{ values.terminal_max_per_user }}" inputmode="numeric">
<p class="field__hint">
One per chat. Each holds an SSH connection open on the far machine.
</p>
</div>
</section>
<div class="btn-row">
<button class="btn btn--primary" type="submit">Save changes</button>
</div>
@@ -0,0 +1,41 @@
{% from "_macros.html" import icon %}
{#
The terminal panel: a fourth child of .shell, to the left of the inspector and
never open beside it. Empty on a page load -- xterm is created the first time
the panel is opened, so the 280KB it costs is paid by somebody who asked for a
shell rather than by everyone who opened a chat.
What is typed here is not run past the chat's mode or its allow and deny
lists. Those govern the model, which reads pages and files it did not write;
the person at the keyboard holds the credential and could open the same shell
with an ssh client.
#}
<aside class="terminal" id="terminal" hidden aria-label="Terminal"
data-terminal
data-url="/api/chats/{{ chat.id }}/terminal/ws"
data-label="{{ agent_profile.name if agent_profile else 'this connection' }}"
data-dir="{{ chat.project_dir }}">
<div class="terminal__header">
<h2 class="terminal__title">
{{ icon("terminal", "icon--sm") }}
<span>{{ agent_profile.name if agent_profile else "Terminal" }}</span>
<span class="terminal__where" data-terminal-where>{{ chat.project_dir }}</span>
</h2>
<button class="btn btn--icon btn--sm" type="button" data-terminal-send
title="Put the selection, or the last of the output, into the message box"
aria-label="Send to chat">
{{ icon("arrow-up", "icon--sm") }}
</button>
<button class="btn btn--icon btn--sm" type="button" data-toggle="#terminal"
aria-label="Close terminal">
{{ icon("x", "icon--sm") }}
</button>
</div>
<div class="terminal__screen" data-terminal-screen></div>
<div class="terminal__status">
<span class="terminal__message" data-terminal-message>Connecting…</span>
<span>Ctrl+Shift+C / V</span>
</div>
</aside>
+30 -2
View File
@@ -5,6 +5,9 @@
{% block head %}
<link rel="stylesheet" href="{{ url_for('static', path='css/chat.css') }}">
{% if terminal_enabled %}
<link rel="stylesheet" href="{{ url_for('static', path='vendor/xterm.css') }}">
{% endif %}
{% endblock %}
{% block body_attrs %} data-authenticated="true"{% endblock %}
@@ -98,9 +101,20 @@
</button>
{% endif %}
{% if terminal_enabled %}
{# To the left of the inspector, and never open beside it: see the
toggle group in app.js. #}
<button class="btn btn--icon" type="button" aria-label="Terminal"
title="Open a shell on {{ agent_profile.name if agent_profile else 'this connection' }}"
aria-expanded="false" data-toggle="#terminal" data-toggle-group="side">
{{ icon("terminal") }}
</button>
{% endif %}
{% if chat and user.is_admin %}
<button class="btn btn--icon" type="button" aria-label="Inspect this chat"
title="Inspect this chat" aria-expanded="false" data-toggle="#inspector">
title="Inspect this chat" aria-expanded="false" data-toggle="#inspector"
data-toggle-group="side">
{{ icon("search") }}
</button>
{% endif %}
@@ -253,9 +267,23 @@
{% endif %}
</main>
{# A third child of .shell, mirroring the sidebar opposite it. #}
{# Third and fourth children of .shell, mirroring the sidebar opposite. The
terminal comes first so it sits to the left of the inspector. #}
{% if terminal_enabled %}
{% include "chat/_terminal.html" %}
{% endif %}
{% if chat and user.is_admin %}
{% include "chat/_inspector.html" %}
{% endif %}
</div>
{% endblock %}
{% block scripts %}
{% if terminal_enabled %}
{# Only where it can be used. xterm is nearly three times everything else
vendored, so a plain chat must never load it. #}
<script src="{{ url_for('static', path='vendor/xterm.js') }}" defer></script>
<script src="{{ url_for('static', path='vendor/xterm-addon-fit.js') }}" defer></script>
<script src="{{ url_for('static', path='js/terminal.js') }}" defer></script>
{% endif %}
{% endblock %}
@@ -89,6 +89,10 @@
<rect x="3.5" y="14" width="17" height="6" rx="1.8"/>
<path d="M7 7h.01M7 17h.01"/>
</symbol>
<symbol id="i-terminal" viewBox="0 0 24 24">
<rect x="3" y="4" width="18" height="16" rx="2"/>
<path d="m7.5 9.5 3 2.5-3 2.5M13 15h4"/>
</symbol>
<symbol id="i-sliders" viewBox="0 0 24 24">
<path d="M4 8h10M18 8h2M4 16h4M12 16h8"/>
<circle cx="16" cy="8" r="2"/><circle cx="10" cy="16" r="2"/>
+22
View File
@@ -80,6 +80,28 @@ def fresh_generation_registry() -> Iterator[None]:
generation_service._TASKS.clear()
@pytest.fixture(autouse=True)
def fresh_terminal_registry() -> Iterator[None]:
"""Empty the open-shell registry between tests, for the same reason.
A leaked entry holds an asyncssh connection belonging to an event loop that
has since closed, and the reaper task is module-level too -- one left
running would wake up inside the next test's loop.
"""
from lembas.services.agent import terminal as terminal_service
def _clear() -> None:
reaper = terminal_service._REAPER
if reaper is not None:
reaper.cancel()
terminal_service._REAPER = None
terminal_service._SESSIONS.clear()
_clear()
yield
_clear()
@pytest.fixture
def db() -> Iterator[Session]:
session = get_session_factory()()
+4 -4
View File
@@ -117,7 +117,7 @@ def test_the_dangerous_defaults_are_all_passed_explicitly():
"""Every LLeMbas user shares one unix account, so "whatever the account has
lying around" is never the right answer. This is the most important test in
the file and it needs no server at all."""
kwargs = ssh._connect_kwargs(_spec(22, "127.0.0.1 ssh-ed25519 AAAA\n"))
kwargs = ssh.connect_kwargs(_spec(22, "127.0.0.1 ssh-ed25519 AAAA\n"))
# Bytes, never None: None turns host key checking off entirely.
assert isinstance(kwargs["known_hosts"], bytes)
@@ -132,11 +132,11 @@ def test_the_dangerous_defaults_are_all_passed_explicitly():
def test_a_profile_with_no_confirmed_host_key_refuses_to_connect():
with pytest.raises(ExecError, match="host key has not been confirmed"):
ssh._connect_kwargs(_spec(22, ""))
ssh.connect_kwargs(_spec(22, ""))
def test_a_password_profile_sends_a_password_and_no_keys():
kwargs = ssh._connect_kwargs(
kwargs = ssh.connect_kwargs(
_spec(22, "h ssh-ed25519 AAAA\n", auth="password", password="hunter2")
)
assert kwargs["password"] == "hunter2"
@@ -144,7 +144,7 @@ def test_a_password_profile_sends_a_password_and_no_keys():
def test_a_key_profile_sends_no_password():
kwargs = ssh._connect_kwargs(_spec(22, "h ssh-ed25519 AAAA\n", password="stale"))
kwargs = ssh.connect_kwargs(_spec(22, "h ssh-ed25519 AAAA\n", password="stale"))
assert kwargs["password"] is None
+437
View File
@@ -0,0 +1,437 @@
"""The shell behind the terminal panel.
Everything here runs against a real SSH server with a real PTY, because the
whole point of this module is what a pseudo-terminal does and a stub would agree
with any design at all.
"""
from __future__ import annotations
import asyncio
import pytest
from lembas.services.agent import terminal as terminal_service
from lembas.services.agent.base import ExecError
from lembas.services.agent.terminal import Session
asyncssh = pytest.importorskip("asyncssh")
# --- A machine with a shell on it ----------------------------------------------
class _Server(asyncssh.SSHServer):
def begin_auth(self, username: str) -> bool:
return False
@pytest.fixture
async def shell_host():
"""A server whose sessions behave enough like a shell to be tested against.
It announces itself, echoes what is typed at it, and remembers what the PTY
was asked for -- which is the part the client side has to get right.
"""
seen: dict = {}
async def handler(process):
seen["command"] = process.command
seen["term_type"] = process.get_terminal_type()
seen["term_size"] = process.get_terminal_size()
seen["process"] = process
process.stdout.write("READY\n")
while True:
try:
line = await process.stdin.readline()
except asyncssh.TerminalSizeChanged:
# A window change interrupts the read rather than arriving as
# data. A real shell redraws and carries on.
continue
except Exception: # noqa: BLE001 - the session ending is not a failure
break
if not line or line.rstrip("\n") == "exit":
break
process.stdout.write(f"echo:{line.rstrip()}\n")
process.exit(0)
server = await asyncssh.create_server(
_Server,
"127.0.0.1",
0,
server_host_keys=[asyncssh.generate_private_key("ssh-ed25519")],
process_factory=handler,
)
port = next(iter(server.sockets)).getsockname()[1]
from lembas.services.agent import ssh as ssh_service
line, _fingerprint = await ssh_service.capture_host_key("127.0.0.1", port)
try:
yield {"port": port, "host_key": line, "seen": seen}
finally:
# Before the server, always: a client connection still open makes
# `wait_closed` wait for it, and a test that failed mid-way has one.
await terminal_service.shutdown()
server.close()
await server.wait_closed()
def _spec(host) -> dict:
return {
"host": "127.0.0.1",
"port": host["port"],
"username": "tester",
"auth": "password",
"password": "",
"private_key": "",
"key_passphrase": "",
"host_key": host["host_key"],
"connect_timeout": 10,
}
async def _open(host, *, chat_id="chat-1", owner="user-1", project_dir="", **kwargs) -> Session:
return await terminal_service.open_session(
chat_id,
owner_id=owner,
profile_id="profile-1",
label="Test box",
spec=_spec(host),
project_dir=project_dir,
**kwargs,
)
async def _read(viewer, *, timeout: float = 5.0) -> bytes:
return await asyncio.wait_for(viewer.queue.get(), timeout=timeout)
async def _read_until(viewer, needle: bytes, *, timeout: float = 5.0) -> bytes:
"""Frames are whatever the far side wrote, so a line can arrive in pieces.
Starts from the snapshot, because a viewer that attached after the shell
had already said something finds it there rather than on the queue -- which
is the whole design, and would otherwise make this hang.
"""
seen = viewer.snapshot
while needle not in seen:
chunk = await _read(viewer, timeout=timeout)
assert chunk is not None, f"the session closed before {needle!r} arrived"
seen += chunk
return seen
# --- The PTY --------------------------------------------------------------------
async def test_the_pty_is_asked_for_with_a_term_type_and_a_size(shell_host):
session = await _open(shell_host, cols=100, rows=30)
viewer = session.attach(cols=100, rows=30)
await _read_until(viewer, b"READY")
assert shell_host["seen"]["term_type"] == "xterm-256color"
assert shell_host["seen"]["term_size"][:2] == (100, 30)
await session.close()
async def test_a_project_directory_becomes_a_cd_before_the_shell(shell_host):
session = await _open(shell_host, project_dir="/srv/work")
viewer = session.attach()
await _read_until(viewer, b"READY")
command = shell_host["seen"]["command"]
assert command is not None
assert "cd '/srv/work'" in command
assert "exec" in command
await session.close()
async def test_no_project_directory_means_the_plain_login_shell(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
assert shell_host["seen"]["command"] is None
await session.close()
async def test_a_single_quote_in_the_directory_cannot_end_the_quoting(shell_host):
session = await _open(shell_host, project_dir="/tmp/it's here; rm -rf /")
viewer = session.attach()
await _read_until(viewer, b"READY")
command = shell_host["seen"]["command"]
assert command.startswith("cd '/tmp/it'\\''s here; rm -rf /'")
await session.close()
async def test_what_is_typed_reaches_the_shell_and_comes_back(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
await session.send(b"hello\n")
assert b"echo:hello" in await _read_until(viewer, b"echo:hello")
await session.close()
async def test_a_resize_reaches_the_far_side(shell_host):
session = await _open(shell_host)
viewer = session.attach(cols=80, rows=24)
await _read_until(viewer, b"READY")
session.resize(viewer, 120, 40)
process = shell_host["seen"]["process"]
for _ in range(50):
if process.get_terminal_size()[:2] == (120, 40):
break
await asyncio.sleep(0.02)
assert process.get_terminal_size()[:2] == (120, 40)
await session.close()
async def test_a_silly_size_is_clamped_rather_than_forwarded(shell_host):
session = await _open(shell_host)
viewer = session.attach(cols=100_000, rows=100_000)
await _read_until(viewer, b"READY")
assert viewer.cols == terminal_service.MAX_COLS
assert viewer.rows == terminal_service.MAX_ROWS
await session.close()
# --- Two tabs, one shell ---------------------------------------------------------
async def test_two_attachments_share_one_shell(shell_host):
session = await _open(shell_host)
first = session.attach()
await _read_until(first, b"READY")
second = session.attach()
await session.send(b"both\n")
assert b"echo:both" in await _read_until(first, b"echo:both")
assert b"echo:both" in await _read_until(second, b"echo:both")
await session.close()
async def test_opening_twice_returns_the_same_session(shell_host):
first = await _open(shell_host)
second = await _open(shell_host)
assert first is second
await first.close()
async def test_the_smaller_viewer_decides_the_size(shell_host):
session = await _open(shell_host)
wide = session.attach(cols=200, rows=60)
await _read_until(wide, b"READY")
session.attach(cols=90, rows=25)
assert (session.cols, session.rows) == (90, 25)
await session.close()
async def test_the_size_lifts_again_when_the_smaller_window_goes(shell_host):
session = await _open(shell_host)
wide = session.attach(cols=200, rows=60)
await _read_until(wide, b"READY")
narrow = session.attach(cols=90, rows=25)
session.detach(narrow)
assert (session.cols, session.rows) == (200, 60)
await session.close()
# --- Lifetime --------------------------------------------------------------------
async def test_detaching_leaves_the_session_running(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
session.detach(viewer)
assert terminal_service.get("chat-1") is session
assert not session.closed
await session.close()
async def test_reattaching_replays_the_scrollback(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
await session.send(b"before\n")
await _read_until(viewer, b"echo:before")
session.detach(viewer)
returning = session.attach()
assert b"READY" in returning.snapshot
assert b"echo:before" in returning.snapshot
await session.close()
async def test_scrollback_is_bounded(shell_host, monkeypatch):
monkeypatch.setattr(terminal_service, "SCROLLBACK_BYTES", 1024)
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
for index in range(200):
session._remember(f"line {index} ".encode() + b"y" * 100)
assert session._scrollback_bytes <= 1024 + 110
assert len(session.attach().snapshot) <= 1024 + 110
await session.close()
async def test_a_slow_reader_is_dropped_rather_than_buffered(monkeypatch):
"""No server needed: fanning out is a plain function over the viewers.
The alternative design -- a pump that waits for a full queue -- would stall
every other viewer behind the one that stopped reading, and buffer without
bound while it did.
"""
monkeypatch.setattr(terminal_service, "VIEWER_QUEUE", 4)
session = Session("chat-1", owner_id="user-1", profile_id="p", label="box")
keeping_up = session.attach()
stalled = session.attach()
for index in range(20):
session._fan_out(f"chunk {index}\n".encode())
while not keeping_up.queue.empty():
keeping_up.queue.get_nowait()
assert stalled.dropped
assert stalled.id not in session.viewers
assert keeping_up.id in session.viewers
assert not session.closed
# Woken rather than left waiting: the sentinel is how the socket learns to
# reconnect, and the scrollback is what makes that free.
assert stalled.queue.get_nowait() is None
async def test_the_shell_exiting_closes_the_session(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
await session.send(b"exit\n")
for _ in range(200):
if session.closed:
break
await asyncio.sleep(0.02)
assert session.closed
assert session.closed_reason == terminal_service.CLOSED_EXITED
# The viewer is woken with the sentinel rather than left waiting.
frame = await _read(viewer)
while frame is not None:
frame = await _read(viewer)
async def test_a_closed_session_lingers_with_its_reason(shell_host):
session = await _open(shell_host)
viewer = session.attach()
await _read_until(viewer, b"READY")
await session.close(terminal_service.CLOSED_IDLE)
assert terminal_service.get("chat-1") is None
lingering = terminal_service.peek("chat-1")
assert lingering is not None
assert lingering.closed_reason == terminal_service.CLOSED_IDLE
async def test_an_idle_session_is_reaped(shell_host, monkeypatch):
monkeypatch.setattr(terminal_service, "REAP_INTERVAL", 0.05)
session = await _open(shell_host, idle_timeout=0.1)
viewer = session.attach()
await _read_until(viewer, b"READY")
session.detach(viewer)
for _ in range(200):
if session.closed:
break
await asyncio.sleep(0.02)
assert session.closed
assert session.closed_reason == terminal_service.CLOSED_IDLE
async def test_a_watched_session_is_never_idle(shell_host, monkeypatch):
monkeypatch.setattr(terminal_service, "REAP_INTERVAL", 0.05)
session = await _open(shell_host, idle_timeout=0.1)
viewer = session.attach()
await _read_until(viewer, b"READY")
await asyncio.sleep(0.4)
assert not session.closed
assert session.idle_for == 0.0
await session.close()
async def test_closing_a_profile_ends_its_shells(shell_host):
first = await _open(shell_host, chat_id="chat-1")
second = await _open(shell_host, chat_id="chat-2")
for session in (first, second):
await _read_until(session.attach(), b"READY")
assert await terminal_service.close_for_profile("profile-1") == 2
assert first.closed and second.closed
assert first.closed_reason == terminal_service.CLOSED_REVOKED
async def test_closing_a_chat_ends_only_its_shell(shell_host):
first = await _open(shell_host, chat_id="chat-1")
second = await _open(shell_host, chat_id="chat-2")
assert await terminal_service.close_chat("chat-1")
assert first.closed
assert not second.closed
await second.close()
async def test_shutdown_closes_every_shell(shell_host):
first = await _open(shell_host, chat_id="chat-1")
second = await _open(shell_host, chat_id="chat-2")
await terminal_service.shutdown()
assert first.closed and second.closed
assert first.closed_reason == terminal_service.CLOSED_SHUTDOWN
assert terminal_service.count() == 0
# --- Caps -------------------------------------------------------------------------
async def test_the_instance_cap_refuses_a_new_shell(shell_host):
await _open(shell_host, chat_id="chat-1", max_sessions=1)
with pytest.raises(ExecError, match="as many terminals open as it allows"):
await _open(shell_host, chat_id="chat-2", max_sessions=1)
async def test_the_per_person_cap_counts_only_that_person(shell_host):
await _open(shell_host, chat_id="chat-1", owner="user-1", max_per_user=1)
with pytest.raises(ExecError, match="already have 1 terminal"):
await _open(shell_host, chat_id="chat-2", owner="user-1", max_per_user=1)
# Somebody else is unaffected.
other = await _open(shell_host, chat_id="chat-3", owner="user-2", max_per_user=1)
assert other is terminal_service.get("chat-3")
await terminal_service.shutdown()
async def test_a_closed_session_does_not_count_against_the_cap(shell_host):
session = await _open(shell_host, chat_id="chat-1", max_per_user=1)
await terminal_service.close_chat("chat-1")
assert session.closed
replacement = await _open(shell_host, chat_id="chat-2", max_per_user=1)
assert not replacement.closed
await replacement.close()
# --- Refusals ---------------------------------------------------------------------
async def test_an_unconfirmed_host_key_is_refused_before_anything_is_sent(shell_host):
spec = _spec(shell_host)
spec["host_key"] = ""
session = Session("chat-1", owner_id="user-1", profile_id="p", label="Test box")
with pytest.raises(ExecError, match="host key has not been confirmed"):
await session.start(spec)
async def test_a_different_host_key_is_refused(shell_host):
spec = _spec(shell_host)
spec["host_key"] = f"[127.0.0.1]:{shell_host['port']} ssh-ed25519 {'A' * 68}\n"
session = Session("chat-1", owner_id="user-1", profile_id="p", label="Test box")
with pytest.raises(ExecError):
await session.start(spec)
+520
View File
@@ -0,0 +1,520 @@
"""The WebSocket behind the terminal panel: who gets one, and what it carries.
The refusals are most of this file on purpose. It is the one endpoint where
getting past the door means a shell on somebody's machine, so every "no" is
pinned -- including *where* it happens, since a browser can read a frame and
cannot read a rejected handshake.
"""
from __future__ import annotations
import json
import pytest
from fastapi.testclient import TestClient
from lembas.db.models import KIND_AGENT, KIND_CHAT, Chat, Connection, Model, SshProfile, User
from lembas.services import settings_store
from lembas.services.agent import policy
from lembas.services.agent import terminal as terminal_service
asyncssh = pytest.importorskip("asyncssh")
class _Server(asyncssh.SSHServer):
def begin_auth(self, username: str) -> bool:
return False
@pytest.fixture
def shell_host():
"""A real SSH server with a shell-ish session, on its own thread and loop.
Its own loop matters: `websocket_connect` is synchronous and blocks the
test's loop, so a server sharing it could never accept the connection the
endpoint is trying to make.
"""
import asyncio
import threading
loop = asyncio.new_event_loop()
thread = threading.Thread(target=loop.run_forever, daemon=True)
thread.start()
seen: dict = {}
async def handler(process):
seen["command"] = process.command
seen["term_size"] = process.get_terminal_size()
process.stdout.write("READY\n")
while True:
try:
line = await process.stdin.readline()
except asyncssh.TerminalSizeChanged:
# A window change interrupts the read rather than arriving as
# data. A real shell redraws and carries on, and so must this
# one, or every resize would look like the shell exiting.
seen["term_size"] = process.get_terminal_size()
continue
except Exception: # noqa: BLE001 - the session ending is not a failure
break
if not line:
break
text = line.rstrip("\n")
if text == "exit":
break
seen.setdefault("typed", []).append(text)
process.stdout.write(f"echo:{text}\n")
process.exit(0)
async def start():
from lembas.services.agent import ssh as ssh_service
server = await asyncssh.create_server(
_Server,
"127.0.0.1",
0,
server_host_keys=[asyncssh.generate_private_key("ssh-ed25519")],
process_factory=handler,
)
port = next(iter(server.sockets)).getsockname()[1]
line, fingerprint = await ssh_service.capture_host_key("127.0.0.1", port)
return server, port, line, fingerprint
server, port, host_key, fingerprint = asyncio.run_coroutine_threadsafe(start(), loop).result(10)
try:
yield {"port": port, "host_key": host_key, "fingerprint": fingerprint, "seen": seen}
finally:
async def stop():
server.close()
await server.wait_closed()
asyncio.run_coroutine_threadsafe(stop(), loop).result(10)
loop.call_soon_threadsafe(loop.stop)
thread.join(timeout=5)
@pytest.fixture(autouse=True)
def close_terminals(client, shell_host):
"""End every shell before the server goes.
A session outlives the socket on purpose, so a test that opens one leaves it
open -- and `wait_closed` then waits on a client connection that nothing is
going to close. It runs on the TestClient's portal because that is the loop
the asyncssh connection belongs to.
"""
yield
client.portal.call(terminal_service.shutdown)
def _agent_chat(db, user_id, shell_host, *, kind=KIND_AGENT, project_dir="") -> Chat:
"""A chat pointed at the test server, with everything switched on."""
settings_store.update(
db, {"enabled": True, "terminal_enabled": True}, key=settings_store.AGENTS
)
profile = SshProfile(
owner_id=user_id,
name="Test box",
host="127.0.0.1",
port=shell_host["port"],
username="tester",
host_key=shell_host["host_key"],
host_fingerprint=shell_host["fingerprint"],
)
db.add(profile)
db.commit()
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="m", capabilities_json={"tools": True}))
db.commit()
chat = Chat(
user_id=user_id,
model_id="m",
connection_id=connection.id,
kind=kind,
ssh_profile_id=profile.id,
project_dir=project_dir,
agent_mode=policy.MODE_MANUAL,
)
db.add(chat)
db.commit()
return chat
def _url(chat: Chat, **params) -> str:
query = "".join(f"&{k}={v}" for k, v in params.items())
return f"/api/chats/{chat.id}/terminal/ws?{query.lstrip('&')}"
def _headers(client: TestClient) -> dict:
"""What a browser on this origin sends. The endpoint requires both."""
return {"origin": str(client.base_url).rstrip("/"), "host": client.base_url.host}
def _first_json(socket) -> dict:
return json.loads(socket.receive_text())
def _read_until(socket, needle: bytes, *, frames: int = 40) -> bytes:
seen = b""
for _ in range(frames):
message = socket.receive()
if message.get("bytes") is not None:
seen += message["bytes"]
if needle in seen:
return seen
elif message.get("text"):
payload = json.loads(message["text"])
raise AssertionError(f"the socket said {payload} before {needle!r} arrived")
raise AssertionError(f"{needle!r} never arrived; saw {seen!r}")
# --- Getting in -------------------------------------------------------------------
def test_a_shell_opens_and_says_hello(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
hello = _first_json(socket)
assert hello["t"] == "ready"
assert hello["label"] == "Test box"
assert b"READY" in _read_until(socket, b"READY")
def test_what_is_typed_reaches_the_shell(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
assert _first_json(socket)["t"] == "ready"
_read_until(socket, b"READY")
socket.send_bytes(b"hello\n")
assert b"echo:hello" in _read_until(socket, b"echo:hello")
def test_the_requested_size_reaches_the_pty(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(
_url(chat, cols=120, rows=40), headers=_headers(client)
) as socket:
hello = _first_json(socket)
_read_until(socket, b"READY")
assert hello["cols"] == 120
assert shell_host["seen"]["term_size"][:2] == (120, 40)
def test_the_project_directory_is_where_the_shell_starts(
client, db, registered, user_id, shell_host
):
chat = _agent_chat(db, user_id, shell_host, project_dir="/srv/work")
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
assert _first_json(socket)["dir"] == "/srv/work"
_read_until(socket, b"READY")
assert "cd '/srv/work'" in shell_host["seen"]["command"]
def test_the_mode_does_not_apply_to_what_a_person_types(
client, db, registered, user_id, shell_host
):
"""Plan mode stops the *model* running anything. It is not a keyboard lock.
Pinned because it looks like a bug to anybody reading `policy.py` next to
this, and "fixing" it would make the panel useless in the mode people spend
the most time in.
"""
chat = _agent_chat(db, user_id, shell_host)
chat.agent_mode = policy.MODE_PLAN
db.commit()
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
assert _first_json(socket)["t"] == "ready"
_read_until(socket, b"READY")
socket.send_bytes(b"rm -rf /tmp/nothing\n")
_read_until(socket, b"echo:rm -rf /tmp/nothing")
assert "rm -rf /tmp/nothing" in shell_host["seen"]["typed"]
# --- Staying out ------------------------------------------------------------------
def test_a_stranger_is_refused_before_the_socket_is_accepted(client, db, user_id, shell_host):
"""No cookie, no handshake. Accepting first would mean an unauthenticated
socket existed at all, however briefly."""
chat = _agent_chat(db, user_id, shell_host)
client.cookies.clear()
# noqa: B017 - starlette raises on a rejected handshake, and the class differs
with pytest.raises(Exception), client.websocket_connect( # noqa: B017
_url(chat), headers=_headers(client)
):
pass
def test_a_cross_site_origin_is_refused_before_the_socket_is_accepted(
client, db, registered, user_id, shell_host
):
chat = _agent_chat(db, user_id, shell_host)
headers = _headers(client) | {"origin": "https://evil.example"}
with pytest.raises(Exception), client.websocket_connect( # noqa: B017
_url(chat), headers=headers
):
pass
def test_an_absent_origin_is_refused(client, db, registered, user_id, shell_host):
"""Required rather than checked-when-present: nothing without an Origin is
a browser, and a non-browser client has no business here."""
chat = _agent_chat(db, user_id, shell_host)
with pytest.raises(Exception), client.websocket_connect( # noqa: B017
_url(chat), headers={"host": client.base_url.host}
):
pass
def test_someone_elses_chat_is_refused_with_a_reason(
client, db, registered, user_id, shell_host
):
chat = _agent_chat(db, user_id, shell_host)
stranger = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(stranger)
db.commit()
chat.user_id = stranger.id
db.commit()
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "no longer exists" in payload["message"]
def test_an_ordinary_chat_has_no_machine_to_open(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host, kind=KIND_CHAT)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "ordinary chat" in payload["message"]
def test_the_instance_switch_refuses_with_a_reason(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
settings_store.update(db, {"terminal_enabled": False}, key=settings_store.AGENTS)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "switched off" in payload["message"]
def test_a_disabled_connection_refuses_with_a_reason(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
profile = db.get(SshProfile, chat.ssh_profile_id)
profile.enabled = False
db.commit()
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "not usable" in payload["message"]
def test_an_unconfirmed_host_key_says_so_rather_than_failing_silently(
client, db, registered, user_id, shell_host
):
"""The one refusal that is genuinely actionable: press Check and accept."""
chat = _agent_chat(db, user_id, shell_host)
profile = db.get(SshProfile, chat.ssh_profile_id)
profile.host_key = ""
db.commit()
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "host key has not been confirmed" in payload["message"]
def test_without_the_permission_there_is_no_terminal(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
user = db.get(User, user_id)
user.role = "user" # an admin bypasses every permission, deliberately
db.commit()
settings_store.update(
db,
{"default_permissions": {"agent.terminal": False, "tools.agent": True, "agent.ssh": True}},
)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
payload = _first_json(socket)
assert payload["t"] == "error"
assert "permission" in payload["message"]
# --- Lifetime ---------------------------------------------------------------------
def test_closing_the_socket_leaves_the_shell_running(client, db, registered, user_id, shell_host):
"""The whole reason the session is not owned by the socket."""
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
socket.send_bytes(b"remember\n")
_read_until(socket, b"echo:remember")
session = terminal_service.get(chat.id)
assert session is not None
assert not session.closed
def test_reconnecting_replays_what_was_missed(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
socket.send_bytes(b"before\n")
_read_until(socket, b"echo:before")
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
assert _first_json(socket)["t"] == "ready"
# Arrives as the scrollback, in one frame, before anything new.
assert b"echo:before" in _read_until(socket, b"echo:before")
def test_the_shell_exiting_is_announced(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
socket.send_bytes(b"exit\n")
payload = None
for _ in range(40):
message = socket.receive()
if message.get("text"):
payload = json.loads(message["text"])
break
assert payload is not None
assert payload["t"] == "closed"
assert payload["reason"] == terminal_service.CLOSED_EXITED
def test_a_resize_frame_reaches_the_far_side(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
socket.send_text(json.dumps({"t": "resize", "cols": 132, "rows": 43}))
# Round-tripped through the shell so the resize has certainly been read.
socket.send_bytes(b"after\n")
_read_until(socket, b"echo:after")
assert shell_host["seen"]["term_size"][:2] == (132, 43)
# --- The panel on the page --------------------------------------------------------
def test_the_panel_and_its_vendored_terminal_are_on_an_agent_chat(
client, db, registered, user_id, shell_host
):
chat = _agent_chat(db, user_id, shell_host)
body = client.get(f"/chat/{chat.id}").text
assert 'id="terminal"' in body
assert "vendor/xterm.js" in body
assert 'data-toggle-group="side"' in body
def test_an_ordinary_chat_loads_none_of_it(client, db, registered, user_id, shell_host):
"""280KB of terminal on a page that could never use it."""
chat = _agent_chat(db, user_id, shell_host, kind=KIND_CHAT)
body = client.get(f"/chat/{chat.id}").text
assert 'id="terminal"' not in body
assert "xterm" not in body
def test_the_panel_is_absent_when_the_instance_switch_is_off(
client, db, registered, user_id, shell_host
):
chat = _agent_chat(db, user_id, shell_host)
settings_store.update(db, {"terminal_enabled": False}, key=settings_store.AGENTS)
assert 'id="terminal"' not in client.get(f"/chat/{chat.id}").text
def test_the_panel_is_absent_without_ssh_installed(
client, db, registered, user_id, shell_host, monkeypatch
):
"""A button whose only outcome is an error frame is worse than no button."""
from lembas.services.agent import ssh as ssh_service
chat = _agent_chat(db, user_id, shell_host)
monkeypatch.setattr(ssh_service, "available", lambda: ssh_service.INSTALL_HINT)
assert 'id="terminal"' not in client.get(f"/chat/{chat.id}").text
# --- Things that must reach a shell already open ----------------------------------
def _open_and_leave(client, chat) -> None:
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
assert terminal_service.get(chat.id) is not None
def test_deleting_the_chat_closes_its_terminal(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
_open_and_leave(client, chat)
assert client.delete(f"/api/chats/{chat.id}").status_code == 204
assert terminal_service.get(chat.id) is None
def test_disabling_the_connection_closes_its_terminal(
client, db, registered, user_id, shell_host
):
"""The model stops on the next reply anyway. A shell already open would not."""
chat = _agent_chat(db, user_id, shell_host)
_open_and_leave(client, chat)
profile = db.get(SshProfile, chat.ssh_profile_id)
client.post(
f"/api/agents/{profile.id}",
data={
"name": profile.name,
"host": profile.host,
"port": str(profile.port),
"username": profile.username,
"auth": profile.auth,
"connect_timeout": "15",
},
follow_redirects=False,
)
db.expire_all()
assert db.get(SshProfile, profile.id).enabled is False
assert terminal_service.get(chat.id) is None
def test_deleting_the_connection_closes_its_terminal(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
_open_and_leave(client, chat)
profile_id = chat.ssh_profile_id
client.post(f"/api/agents/{profile_id}/delete", follow_redirects=False)
assert terminal_service.get(chat.id) is None
def test_forgetting_the_host_key_closes_its_terminal(client, db, registered, user_id, shell_host):
chat = _agent_chat(db, user_id, shell_host)
_open_and_leave(client, chat)
client.post(f"/api/agents/{chat.ssh_profile_id}/forget")
assert terminal_service.get(chat.id) is None
def test_nonsense_control_frames_are_ignored(client, db, registered, user_id, shell_host):
"""A frame is text somebody could forge; none of it may crash the socket."""
chat = _agent_chat(db, user_id, shell_host)
with client.websocket_connect(_url(chat), headers=_headers(client)) as socket:
_first_json(socket)
_read_until(socket, b"READY")
for frame in ("not json", "[]", '{"t":"unknown"}', '{"t":"resize"}'):
socket.send_text(frame)
socket.send_bytes(b"alive\n")
assert b"echo:alive" in _read_until(socket, b"echo:alive")