The terminal learns where one command ends, and can be dragged wider
"The last command and its output" was not something the panel could honestly offer. sendToChat took the last forty rows of the screen buffer, hard-wrapped at the terminal's width with no way to tell a wrap from a newline -- its own comment said so. So bash and zsh are given the OSC 133 markers VS Code and WezTerm use, and Copy, Send and an Auto toggle are built on those. The integration is written by the PTY command string itself, with printf. sshd runs that string through $SHELL -c, so it can case on the shell's own name and needs no probe, no second channel and no writable home. Passing it through the environment does not work -- every distribution ships AcceptEnv LANG LC_*, so anything else is dropped silently -- and feeding `source ...` in as keystrokes races a slow .zshrc, echoes into the scrollback and lands in shell history. Nothing needs hiding, which is the point of choosing it: the setup runs before the shell exists and never writes to the PTY's input side, so there is nothing to echo and no fan-out gate to build. Two things were wrong in the first version and both were found by running it against real shells rather than the fake one. bash: the DEBUG trap fires before every simple command *including each one inside PROMPT_COMMAND*, so $? read from there is whatever ran a moment ago -- every command reported success. The status is captured in the trap now, which also removes the two-entry PROMPT_COMMAND dance entirely. zsh: $ZDOTDIR is already ours by the time .zshenv runs, so the shims were sourcing themselves and none of the user's configuration loaded; the original is passed on the exec line. Parsing is server-side. The `behind` path resets the terminal and replays a truncated scrollback, so a client parser routinely sees a finish with no start; two tabs share one shell and can disagree; and what comes out of this ends up inside a prompt, so deriving it here leaves nothing to disbelieve. The bytes are fanned out unchanged -- xterm consumes an OSC it has no handler for. Output is bounded head and tail, 48KB and 16KB: a build that fails ten megabytes in has the invocation at the top and the error at the bottom. Carriage returns collapse to the last state of each line, which is the difference between a usable prompt and two megabytes of spinner. The fence is sized to its content, because output containing three backticks would otherwise break out and read as prose. Any shell that is not bash or zsh starts exactly as it did before. The buttons then scrape the screen and say so, and Auto is disabled rather than degraded: forty arbitrary lines on every message is worse than nothing. Also a generic [data-resize] handle, keyboard included, persisted the way the theme is. The inspector and sidebar can have it whenever they want it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -0,0 +1,164 @@
|
||||
"""One command and its output, kept so it can be handed to a model.
|
||||
|
||||
Bounded at both ends rather than only the front. A build that fails ten
|
||||
megabytes in has the invocation and the configuration at the top and the error
|
||||
at the bottom, and either half alone is the wrong half.
|
||||
|
||||
Raw bytes are kept and decoded only when somebody asks. Head/tail slicing
|
||||
splits UTF-8 characters at will, and `base.clean_output` decodes with
|
||||
`errors="replace"`, which is exactly the right handling -- decoding eagerly per
|
||||
chunk would be the same mistake the terminal pump already avoids.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import time
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from lembas.services.agent.base import clean_output
|
||||
|
||||
# What one command's output may keep, at each end.
|
||||
CAPTURE_HEAD_BYTES = 48 * 1024
|
||||
CAPTURE_TAIL_BYTES = 16 * 1024
|
||||
# The command line itself. Longer than any command and shorter than a paste.
|
||||
CAPTURE_COMMAND_BYTES = 4 * 1024
|
||||
# One line of output. A minified bundle on one line is not worth keeping whole.
|
||||
MAX_LINE_CHARS = 2000
|
||||
|
||||
# C0 except tab and newline, and the C1 block. Not in `clean_output`, which
|
||||
# `shell_run` shares: there a control character inside a file's contents is
|
||||
# data. Here it is a terminal being driven.
|
||||
_CONTROLS = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
|
||||
|
||||
|
||||
def flatten(text: str) -> str:
|
||||
"""What the screen would have shown, from what the wire carried.
|
||||
|
||||
The highest-value transform here by a distance. A progress bar redraws
|
||||
itself by returning to the start of the line and writing again; keeping
|
||||
every state turns two megabytes of `pip install` into two megabytes of
|
||||
spinner in somebody's prompt. Only the last state of a line was ever
|
||||
visible, so only the last state is kept.
|
||||
"""
|
||||
lines = []
|
||||
for line in text.replace("\r\n", "\n").split("\n"):
|
||||
if "\r" in line:
|
||||
line = line.rsplit("\r", 1)[-1]
|
||||
lines.append(_CONTROLS.sub("", line)[:MAX_LINE_CHARS])
|
||||
return "\n".join(lines).strip("\n")
|
||||
|
||||
|
||||
def fenced(text: str) -> str:
|
||||
"""A fence long enough that the content cannot end it early.
|
||||
|
||||
Output containing three backticks would otherwise break out, and everything
|
||||
after it would read to the model as prose rather than as what a machine
|
||||
printed. That is a real injection route and it costs one line to close.
|
||||
"""
|
||||
longest = max((len(run) for run in re.findall(r"`+", text)), default=0)
|
||||
ticks = "`" * max(3, longest + 1)
|
||||
return f"{ticks}console\n{text}\n{ticks}"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Capture:
|
||||
"""A command, and as much of its output as is worth keeping."""
|
||||
|
||||
seq: int = 0
|
||||
command: str = ""
|
||||
cwd: str = ""
|
||||
started: float = field(default_factory=time.monotonic)
|
||||
ended: float = 0.0
|
||||
exit_status: int | None = None # None while it is still running
|
||||
|
||||
head: bytearray = field(default_factory=bytearray)
|
||||
tail: deque[bytes] = field(default_factory=deque)
|
||||
tail_bytes: int = 0
|
||||
dropped: int = 0
|
||||
total: int = 0
|
||||
|
||||
@property
|
||||
def running(self) -> bool:
|
||||
return self.exit_status is None
|
||||
|
||||
@property
|
||||
def duration_ms(self) -> int:
|
||||
end = self.ended or time.monotonic()
|
||||
return int((end - self.started) * 1000)
|
||||
|
||||
def absorb(self, chunk: bytes) -> None:
|
||||
"""Keep the front, keep the back, count what fell out of the middle."""
|
||||
self.total += len(chunk)
|
||||
if len(self.head) < CAPTURE_HEAD_BYTES:
|
||||
take = CAPTURE_HEAD_BYTES - len(self.head)
|
||||
self.head += chunk[:take]
|
||||
chunk = chunk[take:]
|
||||
if not chunk:
|
||||
return
|
||||
self.tail.append(chunk)
|
||||
self.tail_bytes += len(chunk)
|
||||
while self.tail_bytes > CAPTURE_TAIL_BYTES and len(self.tail) > 1:
|
||||
gone = self.tail.popleft()
|
||||
self.tail_bytes -= len(gone)
|
||||
self.dropped += len(gone)
|
||||
|
||||
def output(self) -> str:
|
||||
"""The kept output as text, with the gap marked if there is one."""
|
||||
head = flatten(clean_output(bytes(self.head), limit=CAPTURE_HEAD_BYTES * 2)[0])
|
||||
if not self.dropped and not self.tail:
|
||||
return head
|
||||
tail = flatten(clean_output(b"".join(self.tail), limit=CAPTURE_TAIL_BYTES * 2)[0])
|
||||
if not self.dropped:
|
||||
return f"{head}\n{tail}" if tail else head
|
||||
gap = f"\n\n… {self.dropped / 1024:,.0f} KB dropped …\n\n"
|
||||
return f"{head}{gap}{tail}"
|
||||
|
||||
def as_text(self, *, label: str) -> str:
|
||||
"""The block that goes into a message, attribution and all.
|
||||
|
||||
The sentence sits **outside** the fence and is written here, so nothing
|
||||
the far side printed can forge it, and the `$ ` line is synthesised
|
||||
rather than lifted from the shell -- what the shell echoed carries
|
||||
readline's editing escapes and is not the command.
|
||||
"""
|
||||
where = f", in {self.cwd}" if self.cwd else ""
|
||||
if self.running:
|
||||
how = "still running"
|
||||
elif self.exit_status:
|
||||
how = f"exit {self.exit_status}"
|
||||
else:
|
||||
how = "succeeded"
|
||||
|
||||
seconds = self.duration_ms / 1000
|
||||
took = f" after {seconds:.0f}s" if seconds >= 1 else ""
|
||||
body = f"$ {self.command}\n{self.output()}".rstrip()
|
||||
return (
|
||||
f"Ran in the terminal on {label}{where} — {how}{took}:\n\n{fenced(body)}"
|
||||
)
|
||||
|
||||
def summary(self) -> str:
|
||||
"""A short label for a chip, never rendered as markup."""
|
||||
command = self.command or "(no command)"
|
||||
if len(command) > 60:
|
||||
command = command[:57] + "…"
|
||||
if self.running:
|
||||
return f"{command} · running"
|
||||
return f"{command} · exit {self.exit_status}"
|
||||
|
||||
|
||||
def trim_command(raw: str) -> str:
|
||||
text, _ = clean_output(raw, limit=CAPTURE_COMMAND_BYTES)
|
||||
return _CONTROLS.sub("", text).strip()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CAPTURE_COMMAND_BYTES",
|
||||
"CAPTURE_HEAD_BYTES",
|
||||
"CAPTURE_TAIL_BYTES",
|
||||
"Capture",
|
||||
"fenced",
|
||||
"flatten",
|
||||
"trim_command",
|
||||
]
|
||||
@@ -0,0 +1,401 @@
|
||||
"""Knowing where one command ends and the next begins, in the terminal panel.
|
||||
|
||||
Without this the panel can offer "the last forty rows of the screen", which is
|
||||
hard-wrapped at the terminal's width with no way to tell a wrap from a newline.
|
||||
That is not something to hand a model and call it the output of a command.
|
||||
|
||||
So the shell is given hooks that emit invisible markers around the prompt, the
|
||||
command and its result -- OSC 133, which is what VS Code, WezTerm and Ghostty
|
||||
all use, plus two of VS Code's private codes for the things 133 has no room
|
||||
for. A real terminal that understands 133 is not confused by ours, and one that
|
||||
does not consumes and discards them, which is why the bytes are fanned out to
|
||||
the browser unchanged.
|
||||
|
||||
**Three things about the mechanism.**
|
||||
|
||||
The integration is written by the PTY command string itself, with `printf`.
|
||||
sshd runs that string through `$SHELL -c`, so it can branch on the shell's own
|
||||
name and needs no probe, no second channel and no writable `$HOME`. Passing it
|
||||
through the environment does not work: sshd's `AcceptEnv` is `LANG LC_*` on
|
||||
every distribution anybody runs, so the variable is dropped silently -- and
|
||||
bash reads `$BASH_ENV` only when non-interactive anyway. Feeding `source …` in
|
||||
as keystrokes does work, and races a slow `.zshrc`, echoes into the scrollback,
|
||||
and lands in shell history with no portable way to remove it.
|
||||
|
||||
Nothing needs hiding, and that is the point of choosing this mechanism. The
|
||||
setup runs before the shell exists and never writes to the PTY's *input* side,
|
||||
so there is nothing for the tty to echo. The first byte a viewer sees is the
|
||||
first byte of their own prompt.
|
||||
|
||||
There is deliberately no `133;B`. It has to live at the end of `PS1`, and any
|
||||
theme that rebuilds the prompt in a hook drops it silently every time. Every
|
||||
marker here comes from a shell hook instead, so none of them depends on a
|
||||
prompt string surviving somebody's dotfiles.
|
||||
|
||||
**Markers are advisory.** A program can print `\\e]133;D;0\\a` and move a
|
||||
boundary. That is not a security problem -- the captured text is sanitised and
|
||||
fenced either way, and a program could already print anything on screen -- but
|
||||
nobody should later try to "validate" them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
|
||||
# Everything ours is under these two. 133 is FinalTerm's de-facto convention;
|
||||
# 633 is VS Code's private space, borrowed because the raw stream carries the
|
||||
# *echoed* command with readline's editing escapes in it and is not
|
||||
# recoverable, and there is no standard code for "here is the command line".
|
||||
MARK_PROMPT = "A" # 133;A -- a prompt is about to be drawn
|
||||
MARK_OUTPUT = "C" # 133;C -- output starts here
|
||||
MARK_DONE = "D" # 133;D;<exit>
|
||||
MARK_COMMAND = "E" # 633;E;<escaped command>
|
||||
MARK_CWD = "P" # 633;P;Cwd=<escaped path>
|
||||
MARK_READY = "LEMBAS" # 633;LEMBAS;<shell>;1
|
||||
|
||||
# A marker longer than this is not one of ours. `cat` of a binary file produces
|
||||
# stray ESC ] regularly, and without a bound one of them would swallow the rest
|
||||
# of the session into a buffer that never emptied.
|
||||
MAX_MARKER_BYTES = 8 * 1024
|
||||
|
||||
_ESC = 0x1B
|
||||
_BEL = 0x07
|
||||
|
||||
|
||||
# --- The snippets ------------------------------------------------------------
|
||||
# `__lembas_esc` exists because an OSC payload may contain no BEL and no ESC
|
||||
# (either would end it ambiguously) and no ';' (which would split the fields).
|
||||
# Everything else rides through. It is also what lets `base.clean_output`'s
|
||||
# existing OSC pattern swallow a whole marker in one bite.
|
||||
|
||||
BASH_RC = r"""
|
||||
# LLeMbas shell integration.
|
||||
#
|
||||
# --rcfile replaces bash's normal startup files rather than adding to them, so
|
||||
# bash's login sequence is reproduced here, in bash's own order, and nothing
|
||||
# anybody has in a dotfile is skipped.
|
||||
if [ -r /etc/profile ]; then . /etc/profile; fi
|
||||
for __lembas_rc in "$HOME/.bash_profile" "$HOME/.bash_login" "$HOME/.profile"; do
|
||||
if [ -r "$__lembas_rc" ]; then . "$__lembas_rc"; break; fi
|
||||
done
|
||||
unset __lembas_rc
|
||||
if [ -r "$HOME/.bashrc" ]; then . "$HOME/.bashrc"; fi
|
||||
|
||||
__lembas_esc() {
|
||||
local s=${1//\\/\\\\}
|
||||
s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
|
||||
s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
|
||||
builtin printf '%s' "$s"
|
||||
}
|
||||
|
||||
__lembas_emit() {
|
||||
if [ -n "$__lembas_running" ]; then
|
||||
builtin printf '\e]133;D;%s\a' "${__lembas_last:-0}"
|
||||
__lembas_running=
|
||||
__lembas_have=
|
||||
fi
|
||||
builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc "$PWD")"
|
||||
builtin printf '\e]133;A\a'
|
||||
__lembas_armed=1
|
||||
}
|
||||
|
||||
# $BASH_COMMAND is the current *simple* command, so `a | b` would give "a".
|
||||
# `history 1` is the whole line as typed, which is what somebody would
|
||||
# recognise; it falls back when history is off.
|
||||
__lembas_line() {
|
||||
local h
|
||||
h=$(HISTTIMEFORMAT='' builtin history 1 2>/dev/null) || {
|
||||
builtin printf '%s' "$BASH_COMMAND"; return; }
|
||||
if [[ $h =~ ^[[:space:]]*[0-9]+[[:space:]]+(.*)$ ]]; then
|
||||
builtin printf '%s' "${BASH_REMATCH[1]}"
|
||||
else
|
||||
builtin printf '%s' "$BASH_COMMAND"
|
||||
fi
|
||||
}
|
||||
|
||||
# The exit status is captured *in the DEBUG trap*, not in PROMPT_COMMAND.
|
||||
#
|
||||
# This is the one genuinely subtle thing in the file. DEBUG fires before every
|
||||
# simple command -- including each command inside PROMPT_COMMAND -- so anything
|
||||
# reading $? from there has already had it overwritten by whatever ran a moment
|
||||
# earlier, and by this trap's own command substitution. The first DEBUG firing
|
||||
# after the user's command is the last place the real status exists, so it is
|
||||
# taken there and held until the prompt emits it.
|
||||
#
|
||||
# `__lembas_have` is what stops the later firings (the ones inside
|
||||
# PROMPT_COMMAND) overwriting it, and `return $__s` puts $? back so nothing
|
||||
# downstream sees a status this trap invented.
|
||||
__lembas_debug() {
|
||||
local __s=$?
|
||||
if [ -n "$__lembas_running" ] && [ -z "$__lembas_have" ]; then
|
||||
__lembas_last=$__s
|
||||
__lembas_have=1
|
||||
fi
|
||||
if [ -n "$__lembas_armed" ]; then
|
||||
__lembas_armed=
|
||||
builtin printf '\e]633;E;%s\a' "$(__lembas_esc "$(__lembas_line)")"
|
||||
builtin printf '\e]133;C\a'
|
||||
__lembas_running=1
|
||||
fi
|
||||
return $__s
|
||||
}
|
||||
|
||||
trap '__lembas_debug' DEBUG
|
||||
|
||||
# Appended, so anything already there still runs and runs first.
|
||||
if [ -n "${BASH_VERSINFO[0]}" ] && [ "${BASH_VERSINFO[0]}" -ge 5 ] \
|
||||
&& [ "${PROMPT_COMMAND@a}" = "a" ]; then
|
||||
PROMPT_COMMAND=("${PROMPT_COMMAND[@]}" __lembas_emit)
|
||||
else
|
||||
PROMPT_COMMAND="${PROMPT_COMMAND:+$PROMPT_COMMAND; }__lembas_emit"
|
||||
fi
|
||||
|
||||
builtin printf '\e]633;LEMBAS;bash;1\a'
|
||||
|
||||
# Self-deleting: bash has read the whole file by the time this runs, and one
|
||||
# left in /tmp that a later shell might source is worse than any benefit.
|
||||
rm -rf -- "$(dirname -- "${BASH_SOURCE[0]}")" 2>/dev/null
|
||||
"""
|
||||
|
||||
# zsh re-reads $ZDOTDIR before *each* startup file, so every shim points it back
|
||||
# at the user's directory, sources their file, and takes it again -- otherwise
|
||||
# zsh finds the user's .zshrc and ours never loads.
|
||||
#
|
||||
# `LEMBAS_USER_ZDOTDIR` is passed in on the exec line and must not be guessed
|
||||
# here. By the time .zshenv runs, `$ZDOTDIR` is already *our* directory, so a
|
||||
# `${ZDOTDIR:-$HOME}` fallback in this file captures the wrong path and the
|
||||
# shim ends up sourcing itself -- which looks like it works, right up to the
|
||||
# point somebody notices none of their own configuration is loaded.
|
||||
_ZSH_SHIM = r"""
|
||||
: ${LEMBAS_USER_ZDOTDIR:=$HOME}
|
||||
ZDOTDIR=$LEMBAS_USER_ZDOTDIR
|
||||
[[ -r $ZDOTDIR/%(file)s ]] && source $ZDOTDIR/%(file)s
|
||||
LEMBAS_USER_ZDOTDIR=$ZDOTDIR
|
||||
ZDOTDIR=$LEMBAS_DIR
|
||||
"""
|
||||
|
||||
ZSH_ENV = _ZSH_SHIM % {"file": ".zshenv"}
|
||||
ZSH_PROFILE = _ZSH_SHIM % {"file": ".zprofile"}
|
||||
|
||||
ZSH_RC = (
|
||||
_ZSH_SHIM % {"file": ".zshrc"}
|
||||
+ r"""
|
||||
__lembas_esc() {
|
||||
local s=${1//\\/\\\\}
|
||||
s=${s//;/\\x3b}; s=${s//$'\n'/\\x0a}; s=${s//$'\r'/\\x0d}
|
||||
s=${s//$'\e'/\\x1b}; s=${s//$'\a'/\\x07}
|
||||
builtin print -rn -- $s
|
||||
}
|
||||
|
||||
__lembas_precmd() {
|
||||
local __s=$?
|
||||
if [[ -n $__lembas_running ]]; then
|
||||
builtin printf '\e]133;D;%s\a' $__s
|
||||
__lembas_running=
|
||||
fi
|
||||
builtin printf '\e]633;P;Cwd=%s\a' "$(__lembas_esc $PWD)"
|
||||
builtin printf '\e]133;A\a'
|
||||
}
|
||||
|
||||
# $1 is the line as typed, before alias and glob expansion -- what somebody
|
||||
# would recognise. $2 and $3 are progressively more expanded and less useful.
|
||||
__lembas_preexec() {
|
||||
builtin printf '\e]633;E;%s\a' "$(__lembas_esc $1)"
|
||||
builtin printf '\e]133;C\a'
|
||||
__lembas_running=1
|
||||
}
|
||||
|
||||
# Prepended, not appended: whichever precmd runs first is the only one that
|
||||
# sees the real $?, and a theme's hook will have clobbered it by the time a
|
||||
# later one runs.
|
||||
precmd_functions=(__lembas_precmd $precmd_functions)
|
||||
preexec_functions=(__lembas_preexec $preexec_functions)
|
||||
|
||||
builtin printf '\e]633;LEMBAS;zsh;1\a'
|
||||
"""
|
||||
)
|
||||
|
||||
ZSH_LOGIN = (
|
||||
_ZSH_SHIM % {"file": ".zlogin"}
|
||||
+ r"""
|
||||
# The last file zsh reads, so this is where ZDOTDIR goes back for good.
|
||||
ZDOTDIR=$LEMBAS_USER_ZDOTDIR
|
||||
[[ -n $LEMBAS_DIR ]] && rm -rf -- $LEMBAS_DIR
|
||||
unset LEMBAS_DIR LEMBAS_USER_ZDOTDIR
|
||||
"""
|
||||
)
|
||||
|
||||
|
||||
def quote(value: str) -> str:
|
||||
"""A single-quoted shell word, with embedded quotes escaped."""
|
||||
return "'" + value.replace("'", "'\\''") + "'"
|
||||
|
||||
|
||||
def command_for(project_dir: str, *, integrate: bool = True) -> str | None:
|
||||
"""What the PTY runs.
|
||||
|
||||
Returns None for the account's plain login shell in the one case that has
|
||||
always returned it -- no project directory and no integration -- so the
|
||||
existing behaviour is byte for byte what it was.
|
||||
|
||||
`$SHELL -c` is what sshd puts this through, so it is POSIX sh and branches
|
||||
on the shell's own name. Anything that is not bash or zsh falls out of the
|
||||
`case` into exactly the line that was here before, because a terminal that
|
||||
works without markers is worth more than markers that break a terminal.
|
||||
Every step is `2>/dev/null`, so a full `/tmp` or a read-only home costs the
|
||||
markers and nothing else.
|
||||
"""
|
||||
cd = f"cd {quote(project_dir)} 2>/dev/null; " if project_dir else ""
|
||||
if not integrate:
|
||||
if not project_dir:
|
||||
return None
|
||||
return f"{cd}exec ${{SHELL:-/bin/sh}} -l"
|
||||
|
||||
return (
|
||||
f"{cd}umask 077; "
|
||||
# mkdir -m 700 fails on a path that already exists, which is what makes
|
||||
# the fallback safe when mktemp is missing.
|
||||
'__L=$(mktemp -d 2>/dev/null) || { '
|
||||
'__L=${TMPDIR:-/tmp}/.lembas-$$-$RANDOM; mkdir -m 700 "$__L"; }; '
|
||||
"case ${SHELL##*/} in "
|
||||
f' bash) printf %s {quote(BASH_RC)} > "$__L/rc" 2>/dev/null && '
|
||||
' exec bash --rcfile "$__L/rc" -i ;; '
|
||||
f' zsh) printf %s {quote(ZSH_ENV)} > "$__L/.zshenv" 2>/dev/null && '
|
||||
f' printf %s {quote(ZSH_PROFILE)} > "$__L/.zprofile" 2>/dev/null && '
|
||||
f' printf %s {quote(ZSH_RC)} > "$__L/.zshrc" 2>/dev/null && '
|
||||
f' printf %s {quote(ZSH_LOGIN)} > "$__L/.zlogin" 2>/dev/null && '
|
||||
# The user's own ZDOTDIR is captured *here*, before it is replaced.
|
||||
' LEMBAS_DIR="$__L" LEMBAS_USER_ZDOTDIR="${ZDOTDIR:-$HOME}" '
|
||||
' ZDOTDIR="$__L" exec zsh -l ;; '
|
||||
"esac; "
|
||||
# Everything that did not exec lands here: an unknown shell, a failed
|
||||
# mkdir, a full /tmp. You still get a shell.
|
||||
'rm -rf -- "$__L" 2>/dev/null; exec ${SHELL:-/bin/sh} -l'
|
||||
)
|
||||
|
||||
|
||||
# --- Reading them back -------------------------------------------------------
|
||||
_UNESCAPE = re.compile(r"\\x([0-9a-fA-F]{2})")
|
||||
|
||||
|
||||
def unescape(value: str) -> str:
|
||||
"""Undo `__lembas_esc`."""
|
||||
return _UNESCAPE.sub(lambda m: chr(int(m.group(1), 16)), value).replace("\\\\", "\\")
|
||||
|
||||
|
||||
class Marks:
|
||||
"""Pulls the markers out of a stream that arrives in any pieces.
|
||||
|
||||
Deliberately not a regex over the scrollback. The pump is handed 64 KB at a
|
||||
time on no particular boundary, so the ESC and the `]` land in different
|
||||
frames often enough to matter -- which is the same reason nothing here
|
||||
decodes the bytes. This holds at most one partial marker and nothing else.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
on_mark: Callable[[str, str], None],
|
||||
on_text: Callable[[bytes], None] | None = None,
|
||||
) -> None:
|
||||
self._on_mark = on_mark
|
||||
self._on_text = on_text
|
||||
self._buffer = bytearray()
|
||||
self._in_marker = False
|
||||
self._plain = bytearray()
|
||||
|
||||
def feed(self, data: bytes) -> None:
|
||||
"""Split the stream into markers and everything else.
|
||||
|
||||
Both come back *in order* and as they are found, not after the whole
|
||||
chunk has been scanned. That matters: a shell frequently writes the
|
||||
command marker, the output and the finished marker in one 64KB read, so
|
||||
anything that scanned first and absorbed afterwards would find the
|
||||
capture already closed and keep nothing.
|
||||
"""
|
||||
for byte in data:
|
||||
if not self._in_marker:
|
||||
# A lone ESC is held: the ']' may be in the next frame.
|
||||
if byte == _ESC:
|
||||
self._flush()
|
||||
self._buffer = bytearray([byte])
|
||||
self._in_marker = True
|
||||
else:
|
||||
self._plain.append(byte)
|
||||
continue
|
||||
|
||||
if len(self._buffer) == 1:
|
||||
if byte != 0x5D: # ']' -- some other escape sequence
|
||||
# Not ours, so it is output like anything else. Emitted
|
||||
# rather than dropped: `clean_output` strips it later, and
|
||||
# swallowing it here would silently eat a colour change.
|
||||
self._in_marker = False
|
||||
self._plain += self._buffer
|
||||
self._plain.append(byte)
|
||||
self._buffer.clear()
|
||||
continue
|
||||
self._buffer.append(byte)
|
||||
continue
|
||||
|
||||
# Two legal terminators, and shells in the wild use both: BEL, and
|
||||
# ESC \ (ST). The ESC has to be *appended* rather than treated as
|
||||
# the start of something new, or the ST branch below can never fire.
|
||||
if byte == _BEL:
|
||||
self._finish()
|
||||
continue
|
||||
|
||||
self._buffer.append(byte)
|
||||
if self._buffer.endswith(b"\x1b\\"):
|
||||
del self._buffer[-2:]
|
||||
self._finish()
|
||||
continue
|
||||
# An `ESC ]` inside a marker that was never terminated starts a new
|
||||
# one. Without this a truncated marker would eat the next.
|
||||
if self._buffer.endswith(b"\x1b]"):
|
||||
self._buffer = bytearray(b"\x1b]")
|
||||
continue
|
||||
if len(self._buffer) > MAX_MARKER_BYTES:
|
||||
# Not one of ours. Output is worth more than a marker.
|
||||
self._in_marker = False
|
||||
self._plain += self._buffer
|
||||
self._buffer.clear()
|
||||
|
||||
self._flush()
|
||||
|
||||
def _flush(self) -> None:
|
||||
if not self._plain:
|
||||
return
|
||||
if self._on_text is not None:
|
||||
self._on_text(bytes(self._plain))
|
||||
self._plain.clear()
|
||||
|
||||
def _finish(self) -> None:
|
||||
payload = bytes(self._buffer[2:]).decode("utf-8", "replace")
|
||||
self._in_marker = False
|
||||
self._buffer.clear()
|
||||
|
||||
code, _, rest = payload.partition(";")
|
||||
if code not in ("133", "633") or not rest:
|
||||
return
|
||||
kind, _, value = rest.partition(";")
|
||||
self._on_mark(kind, value)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"BASH_RC",
|
||||
"MARK_COMMAND",
|
||||
"MARK_CWD",
|
||||
"MARK_DONE",
|
||||
"MARK_OUTPUT",
|
||||
"MARK_PROMPT",
|
||||
"MARK_READY",
|
||||
"MAX_MARKER_BYTES",
|
||||
"Marks",
|
||||
"ZSH_ENV",
|
||||
"ZSH_LOGIN",
|
||||
"ZSH_PROFILE",
|
||||
"ZSH_RC",
|
||||
"command_for",
|
||||
"quote",
|
||||
"unescape",
|
||||
]
|
||||
@@ -44,6 +44,7 @@ from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from lembas.services.agent import capture, shell_marks
|
||||
from lembas.services.agent.base import ExecError
|
||||
from lembas.services.agent.ssh import available, connect_kwargs
|
||||
|
||||
@@ -90,6 +91,20 @@ CLOSED_SHUTDOWN = "shutdown"
|
||||
CLOSED_REVOKED = "revoked"
|
||||
CLOSED_ERROR = "error"
|
||||
|
||||
# Whether this shell tells us where its commands begin and end.
|
||||
# live -- it does
|
||||
# loading -- the hooks went in; the first prompt has not arrived yet
|
||||
# none -- it never will: an unknown shell, or a dotfile that replaced it
|
||||
INTEGRATION_LIVE = "live"
|
||||
INTEGRATION_LOADING = "loading"
|
||||
INTEGRATION_NONE = "none"
|
||||
|
||||
# How long a shell may produce output without ever marking a prompt before we
|
||||
# conclude it is not going to. This is what catches a `.bashrc` ending in `exec
|
||||
# tmux`: the hooks were installed and then the shell replaced itself. Without
|
||||
# it the buttons stay greyed out forever with no explanation.
|
||||
INTEGRATION_GRACE = 10.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class Viewer:
|
||||
@@ -122,6 +137,7 @@ class Session:
|
||||
idle_timeout: float = 1800.0,
|
||||
cols: int = 80,
|
||||
rows: int = 24,
|
||||
integrate: bool = True,
|
||||
) -> None:
|
||||
self.chat_id = chat_id
|
||||
self.owner_id = owner_id
|
||||
@@ -134,6 +150,28 @@ class Session:
|
||||
self._scrollback: deque[bytes] = deque()
|
||||
self._scrollback_bytes = 0
|
||||
|
||||
# --- Command boundaries ---------------------------------------------
|
||||
# Three states, and the middle one matters: INTEGRATION_LOADING means
|
||||
# the hooks were installed and no marker has arrived yet, which is a
|
||||
# different thing to tell somebody than "this shell will never mark".
|
||||
self.integrate = integrate
|
||||
self.integration = INTEGRATION_LOADING if integrate else INTEGRATION_NONE
|
||||
self.shell = ""
|
||||
self._marks = shell_marks.Marks(self._on_mark, self._on_text)
|
||||
# At most two, ever. The one being written and the last finished one --
|
||||
# a history would be a second scrollback with none of the bounding.
|
||||
self.current: capture.Capture | None = None
|
||||
self.last: capture.Capture | None = None
|
||||
self._captures = 0
|
||||
# Where the shell says it is, which the panel header shows live and a
|
||||
# capture records. Seeded from the chat so it says something sensible
|
||||
# before the first prompt.
|
||||
self.cwd = project_dir
|
||||
# Set by the socket layer, which knows how to shape a frame. Called
|
||||
# when a command finishes so a panel can enable its buttons without
|
||||
# polling for something that happens a few times a minute.
|
||||
self.on_command: Any = None
|
||||
|
||||
self._conn: Any = None
|
||||
self._process: Any = None
|
||||
self._pump: asyncio.Task | None = None
|
||||
@@ -213,11 +251,79 @@ class Session:
|
||||
command nobody entered. It is single-quoted, and a failure is ignored:
|
||||
a directory that has been deleted should leave somebody at a shell to
|
||||
find out why, not with a connection that closes as it opens.
|
||||
|
||||
With integration on, the same string also writes the shell-integration
|
||||
files and execs through them; see `shell_marks` for why it is done in
|
||||
the command rather than over SFTP or through the environment.
|
||||
"""
|
||||
if not self.project_dir:
|
||||
return None
|
||||
quoted = "'" + self.project_dir.replace("'", "'\\''") + "'"
|
||||
return f"cd {quoted} 2>/dev/null; exec ${{SHELL:-/bin/sh}} -l"
|
||||
return shell_marks.command_for(self.project_dir, integrate=self.integrate)
|
||||
|
||||
# --- Where one command ends and the next begins --------------------------
|
||||
|
||||
def _on_mark(self, kind: str, value: str) -> None:
|
||||
"""One marker, from the scanner in the pump.
|
||||
|
||||
Advisory, never trusted: a program can print these itself and move a
|
||||
boundary. It is not a way in -- the text is sanitised and fenced either
|
||||
way, and a program could already print anything on screen -- but that is
|
||||
why nothing here validates them, and why none of it decides anything a
|
||||
person could not already do at the keyboard.
|
||||
"""
|
||||
if kind == shell_marks.MARK_READY:
|
||||
self.integration = INTEGRATION_LIVE
|
||||
self.shell = value.split(";")[0][:32]
|
||||
return
|
||||
|
||||
if self.integration != INTEGRATION_LIVE and kind in (
|
||||
shell_marks.MARK_PROMPT,
|
||||
shell_marks.MARK_OUTPUT,
|
||||
):
|
||||
self.integration = INTEGRATION_LIVE
|
||||
|
||||
if kind == shell_marks.MARK_CWD:
|
||||
path = shell_marks.unescape(value.partition("=")[2])[:1000]
|
||||
self.cwd = path
|
||||
return
|
||||
|
||||
if kind == shell_marks.MARK_COMMAND:
|
||||
self._captures += 1
|
||||
self.current = capture.Capture(
|
||||
seq=self._captures,
|
||||
command=capture.trim_command(shell_marks.unescape(value)),
|
||||
cwd=self.cwd,
|
||||
)
|
||||
return
|
||||
|
||||
if kind == shell_marks.MARK_DONE and self.current is not None:
|
||||
try:
|
||||
self.current.exit_status = int(value.strip() or 0)
|
||||
except ValueError:
|
||||
self.current.exit_status = 0
|
||||
self.current.ended = time.monotonic()
|
||||
self.last = self.current
|
||||
self.current = None
|
||||
if self.on_command is not None:
|
||||
self.on_command(self.last)
|
||||
|
||||
def _on_text(self, data: bytes) -> None:
|
||||
"""Everything that was not a marker, while a command is running.
|
||||
|
||||
Interleaved with `_on_mark` rather than applied to the whole chunk
|
||||
afterwards: a shell often writes the command marker, the output and the
|
||||
finished marker in one read, and absorbing after the scan would find
|
||||
the capture already closed and keep nothing at all.
|
||||
"""
|
||||
if self.current is not None:
|
||||
self.current.absorb(data)
|
||||
|
||||
def latest(self) -> capture.Capture | None:
|
||||
"""The command to act on: the one still running, else the last one.
|
||||
|
||||
In-flight counts. "Copy the last command and its output" while `make` is
|
||||
still going should give what has been printed so far, marked as still
|
||||
running -- not "nothing yet".
|
||||
"""
|
||||
return self.current or self.last
|
||||
|
||||
# --- Following -----------------------------------------------------------
|
||||
|
||||
@@ -291,6 +397,9 @@ class Session:
|
||||
if not data:
|
||||
break
|
||||
self._remember(data)
|
||||
# Before the fan-out, so a "this command finished" frame can
|
||||
# never reach a browser after the output it describes.
|
||||
self._observe(data)
|
||||
self._fan_out(data)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
@@ -300,12 +409,52 @@ class Session:
|
||||
return
|
||||
await self._finish(CLOSED_EXITED)
|
||||
|
||||
def _observe(self, data: bytes) -> None:
|
||||
"""Watch the stream for markers, and feed the command being captured.
|
||||
|
||||
Server-side rather than in the browser, for five reasons. The `behind`
|
||||
path calls `term.reset()` and replays a *truncated* scrollback, so a
|
||||
client parser routinely sees a "finished" with no matching "started".
|
||||
Two tabs share one shell and two parsers can disagree about what "the
|
||||
last command" is. The server sees the stream once however many are
|
||||
watching. And what comes out of this ends up inside a prompt -- deriving
|
||||
it here means there is nothing to disbelieve later.
|
||||
|
||||
The bytes are still fanned out unchanged, markers and all: xterm
|
||||
consumes an OSC it has no handler for and never draws it, and rewriting
|
||||
frames on the hot path would break the "nothing decodes, so nothing can
|
||||
split" property the pump depends on.
|
||||
"""
|
||||
self._marks.feed(data)
|
||||
if self.current is None and (
|
||||
self.integration == INTEGRATION_LOADING
|
||||
and time.monotonic() - self.started_at > INTEGRATION_GRACE
|
||||
):
|
||||
# Output arrived, the grace period passed, and no marker ever came.
|
||||
# Output arrived, the grace period passed, and no marker ever came.
|
||||
# Something replaced the shell -- a dotfile ending in `exec tmux` is
|
||||
# the usual one. Say so rather than leaving the buttons greyed.
|
||||
self.integration = INTEGRATION_NONE
|
||||
|
||||
def _remember(self, data: bytes) -> None:
|
||||
self._scrollback.append(data)
|
||||
self._scrollback_bytes += len(data)
|
||||
while self._scrollback_bytes > SCROLLBACK_BYTES and len(self._scrollback) > 1:
|
||||
self._scrollback_bytes -= len(self._scrollback.popleft())
|
||||
|
||||
def announce(self, text: str) -> None:
|
||||
"""Put one text frame in front of every viewer.
|
||||
|
||||
Through the same queues as the output so ordering is preserved: a
|
||||
"finished" that overtook the last of the output it describes would have
|
||||
a panel offering a capture the screen has not caught up with. Dropped
|
||||
rather than blocking on a full queue -- that viewer is already being
|
||||
disconnected and will be told again on reattach.
|
||||
"""
|
||||
for viewer in list(self.viewers.values()):
|
||||
with contextlib.suppress(asyncio.QueueFull):
|
||||
viewer.queue.put_nowait(text)
|
||||
|
||||
def _fan_out(self, data: bytes) -> None:
|
||||
for viewer in list(self.viewers.values()):
|
||||
try:
|
||||
@@ -414,6 +563,7 @@ async def open_session(
|
||||
max_per_user: int = 3,
|
||||
cols: int = 80,
|
||||
rows: int = 24,
|
||||
integrate: bool = True,
|
||||
) -> Session:
|
||||
"""The shell for this chat, opening one if it is not already there.
|
||||
|
||||
@@ -442,6 +592,7 @@ async def open_session(
|
||||
owner_id=owner_id,
|
||||
profile_id=profile_id,
|
||||
label=label,
|
||||
integrate=integrate,
|
||||
project_dir=project_dir,
|
||||
idle_timeout=idle_timeout,
|
||||
cols=cols,
|
||||
|
||||
@@ -88,6 +88,11 @@ def _agents_defaults() -> dict[str, Any]:
|
||||
# SSH connection held open, so this is a real resource, not a scruple.
|
||||
"terminal_max_sessions": 20,
|
||||
"terminal_max_per_user": 3,
|
||||
# Whether the panel's shell is given hooks that mark where one
|
||||
# command ends and the next begins. Off means the Copy and Send
|
||||
# buttons fall back to scraping the screen, and Auto is
|
||||
# unavailable -- there is nothing to key it on.
|
||||
"terminal_integration": True,
|
||||
# A listing of the project directory, put in front of the model so the
|
||||
# first rounds of a reply are not spent discovering what is there. It
|
||||
# costs its budget on *every* request in an agent chat, forever, which
|
||||
|
||||
Reference in New Issue
Block a user