Tests that found things reading did not

The testing pass: 2140 tests to 2283, and four bugs that no amount of
reading had turned up. Three came from driving the JavaScript under a
Node DOM stub, which is the practice CLAUDE.md sets out and this is the
reason it does.

The terminal dropped every keystroke after a reconnect. `onclose` closed
over the module-level socket rather than its own, and close() queues its
event -- so the old socket's close arrived after a new one was assigned
and nulled the live one. Output kept coming, because onmessage is bound
to the object, while every send gates on the variable. It also announced
"Disconnected" about a shell that had just reconnected.

Two scripts were loaded twice on /messages, once by base.html and again
by the page. Each is an IIFE with its own state, so four keyboard
shortcuts toggled their panel twice and therefore did nothing, /help
opened two dialogs, and an @ mention attached its file twice. A sweep
refuses any template re-loading what base.html has.

The microphone had no guard while the permission prompt was up, so each
click opened another stream and only the last was ever stopped. And a
skill shared with you took its name out of your own library: create
checked uniqueness against what is *visible* rather than what is owned,
against a (owner_id, name) constraint, and told you to edit a row you
cannot edit.

--ink-faint failed the contrast minimum in both themes -- 3.85 and 3.19
against 4.5 -- so the smallest text on every screen was the hardest to
read. Measured in a headless browser rather than judged by eye.

And the suite runs on 3.11 and 3.12 now as well as 3.14. It had only ever
run on 3.14 while the image ships 3.12 and the packaging claimed 3.11:
the interpreter most people would run was the one nothing had tested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-07 14:41:45 +02:00
parent 25fe81a224
commit 0514568df0
33 changed files with 2859 additions and 13 deletions
+34
View File
@@ -16,6 +16,40 @@ for 1.0.0 have something to be assembled from.
## Unreleased
## 0.9.13
**The testing pass.** 2140 tests became 2283, and writing them found four bugs
that no amount of reading had.
- Fixed: **the terminal silently stopped accepting input after a reconnect.**
Change the connection, or let the shell catch up after falling behind, and
every keystroke was dropped from then on — while output kept arriving, so the
panel looked perfectly healthy. It also announced "Disconnected. Close and
reopen to reconnect." about a shell that had just reconnected successfully.
- Fixed: **on the Messages screen, half the keyboard did nothing.** Two scripts
were loaded twice there, so `Alt+B`, `Alt+E`, `Alt+T` and `Alt+I` toggled
their panel twice — which is to say not at all — while `/help` opened two
dialogs, `/image` posted the message twice, and picking an `@` mention
attached the file twice.
- Fixed: **pressing the microphone while the permission prompt was up opened a
recording each time.** Only the last was stopped, so the browser's recording
indicator stayed on until the tab was closed.
- Fixed: **a skill shared with you took its name out of your own library.**
Creating your own was refused with "a skill called that already exists. Edit
it instead" — naming a skill you cannot edit, because sharing grants reading
only. The model's `skill_create` hit the same dead end. Sharing a curated
skill with a team is what sharing is *for*.
- Hints and timestamps are readable now. `--ink-faint` failed the accessibility
contrast minimum in **both** themes — 3.85:1 in Moria, 3.19:1 in Shire, where
4.5:1 is the bar — so the smallest text on every screen was the hardest to
read.
- The suite runs on **Python 3.11 and 3.12** as well as 3.14. It had only ever
run on 3.14, while the Docker image ships 3.12 and the packaging claimed 3.11
— so the one interpreter most people would actually run was the one nothing
had tested.
- A `docs/notes/release-checklist.md` for the half of testing a machine cannot
do: a real endpoint, a real machine, real hardware, a real pair of eyes.
## 0.9.12
**The security pass.** Six findings, all fixed. None is reachable by simply
+42 -1
View File
@@ -360,7 +360,48 @@ Questions raised before the security stage, answered by it:
button press. pip's `only-if-needed` default limits the blast radius, which is
why this is a note rather than an emergency.
## For the testing stage
## Testing — *Stage 4*
2140 tests became **2283**, and writing them found four bugs that reading had
not — three of them by driving the JavaScript under a Node DOM stub, which is
the practice `CLAUDE.md` sets out and the reason it does.
- **The terminal dropped every keystroke after a reconnect.** `socket.onclose`
closed over the module-level variable rather than its own connection, and
`close()` queues its event: both reconnect paths close the old socket and
immediately open a new one, so the old one's `close` arrived afterwards and
nulled the *live* socket. Output kept coming, because `onmessage` is bound to
the object; every send gates on `socket && readyState === OPEN`. It also said
"Disconnected" about a shell that had just reconnected, because
`closedOnPurpose` had been cleared for the new connection.
- **Two scripts were loaded twice on `/messages`** — once by `base.html` and
again by the page. Each is an IIFE with its own state, so four keyboard
shortcuts toggled their panel twice and therefore did nothing, `/help` opened
two dialogs, and an `@` mention attached its file twice. A sweep now refuses
any template re-loading what `base.html` already has.
- **The microphone had no guard while the permission prompt was up.** State only
left `idle` after `getUserMedia` resolved, so every click in that window
opened another stream; only the last was stopped.
- **A shared skill took its name out of your own library.** `create` checked
uniqueness with `by_name`, which is scoped to what is *visible* — owned or
shared — against a `(owner_id, name)` constraint. `documents.create_base` next
door asks it correctly. Found by writing HTTP tests for `api/library.py`.
Also: `--ink-faint` failed the 4.5:1 contrast minimum in **both** themes (3.85
in Moria, 3.19 in Shire), measured by computing the ratios in a headless browser
rather than by eye. Raised in both.
**Verified rather than assumed:** the suite now runs on Python **3.11, 3.12 and
3.14**. It had only ever run on 3.14, while `Dockerfile` ships 3.12 and
`requires-python` claimed 3.11 — the interpreter most people would actually run
was the one nothing had tested. Both pass.
One measurement worth keeping honest: the `slow` marker splits 365 tests that
stand up a real server, shell, PTY or git repository, but the fast subset is
111s against 140s for everything. The split is for CI, not for a dramatic
saving.
## Still open after the testing stage
- `api/library.py`: 28 routes, effectively zero HTTP coverage. Highest exposure
— auth, ownership and permission gates all live at that boundary.
+154
View File
@@ -0,0 +1,154 @@
# The manual pass, before a release
What the suite cannot reach. Everything here needs a real endpoint, a real
machine, real hardware or a real browser with a person in front of it — which is
to say, everything where the failure is "it works but nobody could use it".
Run it against the live instance. Tick nothing you have not actually seen.
Times are rough and assume things are already configured.
---
## 1. A model answers at all (5 min)
- [ ] Send a message. The reply streams in **as it is written**, not all at once
at the end. (A reply that arrives complete means something is buffering —
a proxy, or a worker that collected the response.)
- [ ] The thinking block, on a reasoning model: opens, shows a duration, and the
duration is not the same number on every round.
- [ ] Stop mid-reply. What arrived is kept, the bubble is marked stopped rather
than errored, and the composer returns to Send.
- [ ] Navigate away mid-reply and come back. The reply is still running and the
transcript catches up.
- [ ] Close the tab mid-reply, reopen the chat. The reply finished without you.
- [ ] Regenerate a reply. The old one is replaced, not appended.
- [ ] Edit an earlier message. Everything after it goes, and the conversation
runs on from there.
## 2. The composer (5 min)
- [ ] Type `/` — the menu appears on the **first** press, not the second.
- [ ] Choose a command with Enter. The box is left empty, not holding `/help`.
- [ ] Tab completes the highlighted command.
- [ ] `//` escapes: the message sends as written.
- [ ] A message that merely starts with a slash and is not a command **sends**.
- [ ] Type `@` and pick a file. The token stays in the sentence *and* a chip
appears.
- [ ] The highlighting behind `/` and `@` sits exactly over the text, at every
width, and does not drift as the box grows.
- [ ] Send. The highlighting clears with the box rather than a keystroke later.
- [ ] `Ctrl/⌘+Enter` sends from anywhere in the form.
- [ ] In an agent chat, the toolbar stays **one row** at every window width.
Send and the microphone never wrap to a second line.
## 3. Attachments and images (10 min)
- [ ] Drag an image in. It is downscaled and the model can describe it.
- [ ] Paste a screenshot. Same.
- [ ] A PDF: the text reaches the model; a scanned one says so rather than
contributing nothing silently.
- [ ] Rename a `.txt` to `.png` and upload it. It is stored as text.
- [ ] Attach from the **new-chat screen**, send, then delete the chat. The file
is gone from `data/uploads/attachments`. *(This is the 0.9.10 fix; before
it, the row went and the file stayed.)*
- [ ] Generate an image, if a ComfyUI is configured. It appears in the chat, and
deleting the chat removes the file.
## 4. Agent chats — needs a real SSH host (15 min)
- [ ] Add a connection. The fingerprint is shown **before** anything is sent.
- [ ] Each mode does what it says: **Manual** shows everything first, **Edit**
writes freely but asks before commands, **Auto** asks nothing, **Plan**
changes nothing and ends with a plan.
- [ ] Approve, refuse, and *edit* a proposed command. The edited one is what
runs, and the transcript says so.
- [ ] "Always allow this" — the next matching command runs without asking.
- [ ] Open the terminal panel. Type. Close the panel and reopen: the session
survived and the scrollback is there.
- [ ] **Change the connection while the terminal is open**, then type. Every
keystroke still reaches the shell. *(This is the 0.9.12 fix — before it,
output kept arriving and input was silently dropped.)*
- [ ] Start a long command in the background, navigate away, come back. You are
told it finished.
- [ ] Open the canvas, pick a file by browsing rather than typing a path, edit
it, save. The file changed on the far side.
- [ ] Try to point a connection at `127.0.0.1` and at `0.0.0.0`. **Both refused**
unless an administrator has opened the switch.
## 5. Things that happen later (10 min, plus waiting)
- [ ] Ask the model to schedule something ten minutes out. It uses the tool
rather than writing a note, and says the timing back **in words**.
- [ ] Check the Scheduled list: the timing shown matches what you asked for, in
your timezone.
- [ ] Wait for it to fire. A report is filed, or a message arrives.
- [ ] With the tab **closed**, a scheduled run reaches you by push (if enabled).
- [ ] The dot, the tab-title count and the system notification do not all fire
at once for the same arrival.
## 6. Sharing and permissions — needs two accounts (10 min)
- [ ] Share a note with the second account. They can read it and cannot edit it.
- [ ] "Shared with me" lists it.
- [ ] The second account cannot see anything not shared with them, **including
as an administrator**.
- [ ] Delete the second account. No share anywhere still names it.
- [ ] Set a group quota, spend past it, and confirm the reply ends with an
explanation rather than an empty bubble.
## 7. Audio — needs real hardware (5 min)
- [ ] Dictate a message. `Alt+M` starts it; the transcript lands in the box and
the highlighting repaints.
- [ ] Press the microphone **three times quickly** while the permission prompt
is up. Only one recording starts, and the browser's recording indicator
goes out when you stop. *(0.9.12.)*
- [ ] `Alt+R` reads the last reply aloud.
- [ ] Read-aloud-automatically does not re-read an old reply when you reopen a
chat.
## 8. The look of it (10 min)
Both themes, and a custom one.
- [ ] Tab through a page with the keyboard. Every control shows where you are.
- [ ] Narrow the window to a phone width on `/admin/models`, `/admin/prompts`
and a chat. Nothing is cut off and nothing needs sideways scrolling.
- [ ] Hints and timestamps are readable, not grey-on-grey. *(0.9.12 raised
`--ink-faint` in both themes; this is the one to eyeball.)*
- [ ] Switch tabs on `/admin/prompts`. The page does not jump and no screenful
of nothing appears. *(0.9.10.)*
- [ ] Make a custom theme with four colours. It composes, and the focus rings
pick up the new accent.
- [ ] Install to the home screen. The icon and the name are the branded ones.
## 9. Upgrading (15 min)
The one nobody does until it matters.
- [ ] From a **copy** of a real 0.8.x database, start the new version. It boots,
the chats are there, and nothing in the log says a column is missing.
- [ ] `/admin/updates` shows a version rather than a sha, and the release notes
come from the tag.
- [ ] Press Update. The service restarts and comes back.
- [ ] Re-run `install.sh`. The channel does **not** move on its own. *(0.9.12.)*
- [ ] `sudo ls -l /usr/local/lib/lembas/update.sh` — owned by root. If systemd's
`ExecStart` still points inside the checkout, the helper is on the old
wiring and the script says so loudly when it runs.
- [ ] A fresh install into a container, from nothing, following the README only.
---
## What the suite already covers, so you do not have to
Not a suggestion to skip it — a note on where the machine has already looked, so
your time goes where it cannot.
- Every tool's gating, and that a chat can only narrow what it was granted
- The four agent modes against a real SSH server, and the approval loop
- Reply steps, metrics, compaction, queueing and rewind
- The schema upgrade, with rows, from an 0.8.1-shaped database
- Every library route at the HTTP boundary: ownership, sharing, deletes
- The SSRF guard on every outbound path
- The whole suite on Python 3.11, 3.12 and 3.14
+8
View File
@@ -83,5 +83,13 @@ ignore = ["B008"] # FastAPI Depends() in defaults is idiomatic
[tool.pytest.ini_options]
testpaths = ["tests"]
# Registered so `-m "not slow"` works and an unknown-marker warning does not
# become an error later. `slow` is for the tests that stand up something real:
# a uvicorn subprocess on a port, an asyncssh server, a PTY, a git repository
# built with subprocess. They are the ones worth having and the ones worth
# being able to skip while iterating.
markers = [
"slow: stands up a real server, shell or repository",
]
asyncio_mode = "auto"
filterwarnings = ["ignore::DeprecationWarning"]
+1 -1
View File
@@ -1,3 +1,3 @@
"""LLeMbas - a Middle-earth themed web UI for OpenAI-compatible LLM endpoints."""
__version__ = "0.9.12"
__version__ = "0.9.13"
+29 -2
View File
@@ -67,12 +67,39 @@ def get(db: DBSession, skill_id: str, user: User | None) -> Skill | None:
def by_name(db: DBSession, name: str, user: User | None) -> Skill | None:
"""Look one up the way the model refers to it."""
"""Look one up the way the model refers to it.
Scoped to what this person can **see**, which is theirs plus anything
shared with them -- correct for `skill_get` and `skill_edit`, where a
skill somebody shared is exactly what the model is reaching for.
It is the wrong question for "is this name taken?"; see `owned_by_name`.
"""
if user is None:
return None
return db.scalar(visible(db, user).where(Skill.name == slugify(name)))
def owned_by_name(db: DBSession, name: str, owner: User) -> Skill | None:
"""One of *this person's own* skills by name.
The uniqueness check used `by_name`, which is scoped to what is visible --
so a skill somebody shared with you took that name out of your library.
Sharing a curated skill with a team is the intended use of `library.share`,
and doing it silently reserved the name for everyone it reached: creating
your own was refused with "a skill called 'weekly-report' already exists.
Edit it instead", naming a row you cannot edit, because sharing grants
reading only. The model's `skill_create` got the same dead end.
The table's constraint is `(owner_id, name)`, so the question the check
should have been asking was always this one. `documents.create_base` next
door asks it correctly.
"""
return db.scalar(
select(Skill).where(Skill.owner_id == owner.id, Skill.name == slugify(name))
)
def enabled_for(
db: DBSession, user: User | None, *, exclude: Iterable[str] = ()
) -> list[Skill]:
@@ -155,7 +182,7 @@ def create(
"A skill name must be two or more letters, numbers or hyphens, "
"such as 'weekly-report'."
)
if by_name(db, slug, owner) is not None:
if owned_by_name(db, slug, owner) is not None:
raise SkillError(f"A skill called {slug!r} already exists. Edit it instead.")
if not description.strip():
raise SkillError(
+2 -2
View File
@@ -155,7 +155,7 @@
--ink: #E4E8EC;
--ink-muted: #A2ADB8;
--ink-faint: #6E7883;
--ink-faint: #7A848F;
--ink-inverse: #0B0E11;
/* Mithril: the cool primary, used for focus and interactive accents. */
@@ -263,7 +263,7 @@
--ink: #2C2419;
--ink-muted: #6A5C48;
--ink-faint: #94856D;
--ink-faint: #756A55;
--ink-inverse: #FDFBF5;
/* Hobbit-door blue-green: the cool primary. */
+14
View File
@@ -104,6 +104,17 @@
return;
}
/* Out of `idle` **before** awaiting permission, or the button stays
clickable for as long as the browser's prompt is up -- and the dispatcher
below starts a recording on every click in that window. Each one opened
its own stream and its own MediaRecorder, only the last of which was kept
in `recorder`, so stopping released one and left the rest live: the
browser's recording indicator stayed on until the tab was closed.
`working` already exists and is already styled, so this is the state the
button was missing rather than a new one. */
setMicState(button, "working");
navigator.mediaDevices.getUserMedia({ audio: true }).then(function (stream) {
chunks = [];
recorder = new MediaRecorder(stream);
@@ -124,6 +135,9 @@
recorder.start();
setMicState(button, "recording");
}).catch(function () {
// Back to idle, or a refused permission leaves a button nothing can press
// again -- including the reader who has just gone and allowed it.
setMicState(button, "idle");
notify("The microphone could not be opened. Permission may be blocked.", "error");
});
}
+24 -5
View File
@@ -125,10 +125,27 @@
"?cols=" + (term.cols || 80) + "&rows=" + (term.rows || 24);
say("Connecting…");
socket = new WebSocket(url);
socket.binaryType = "arraybuffer";
/* Held in a local as well as in `socket`, because every handler below has
to know *which* connection it belongs to.
socket.onmessage = function (event) {
`close()` queues its event rather than firing it, and both reconnect
paths -- the `behind` frame and `repointTerminal` -- close the old
socket and immediately call this function. So the old socket's `close`
arrives *after* a new one has been assigned, and a handler that touched
the module-level `socket` was reaching past its own connection into the
live one. It set it to null: output kept arriving, because `onmessage`
is bound to the object, while `send`, `term.onData` and the paste
handler all gate on `socket && readyState === OPEN` -- so every
keystroke was dropped and no resize was ever sent again, on a panel that
looked perfectly healthy. And `closedOnPurpose` had already been cleared
for the new connection, so it said "Disconnected. Close and reopen to
reconnect." about a shell that had just reconnected. */
var opened = new WebSocket(url);
socket = opened;
opened.binaryType = "arraybuffer";
opened.onmessage = function (event) {
if (socket !== opened) return;
if (typeof event.data === "string") return control(event.data);
/* Written straight through as bytes. xterm's decoder is stateful across
calls, so a multi-byte character split across two frames still lands
@@ -136,12 +153,14 @@
term.write(new Uint8Array(event.data));
};
socket.onclose = function () {
opened.onclose = function () {
if (socket !== opened) return;
socket = null;
if (!closedOnPurpose) say("Disconnected. Close and reopen to reconnect.");
};
socket.onerror = function () {
opened.onerror = function () {
if (socket !== opened) return;
/* A failed handshake gives the page nothing: no status, no reason. So
this is a guess, and it names the likeliest cause rather than
pretending to know. */
+15 -2
View File
@@ -105,8 +105,21 @@
</div>
{% endblock %}
{#
`steps.js` only. `composer.js` and `commands.js` were listed here too, and
`base.html` already loads both on every page -- so this screen ran each of
them **twice**.
Each is an IIFE with its own state, and `stopPropagation()` does not stop a
second listener already bound to the same node. So: two composer menus stacked
on each other; one Enter on a highlighted `/` item running the command twice
(`/help` opening two dialogs, `/image` posting the message twice); an `@`
mention attaching its file twice; and `Alt+B`, `Alt+E`, `Alt+T` and `Alt+I`
toggling their panel twice, which is to say doing nothing at all.
None of it looks like a script loaded twice. Found by driving the file under a
DOM stub, which is the rule `CLAUDE.md` sets out and the reason it does.
#}
{% block scripts %}
<script src="{{ url_for('static', path='js/composer.js') }}" defer></script>
<script src="{{ url_for('static', path='js/commands.js') }}" defer></script>
<script src="{{ url_for('static', path='js/steps.js') }}" defer></script>
{% endblock %}
+106
View File
@@ -300,6 +300,112 @@ def control_named(html: str, name: str) -> dict[str, str]:
return found[0]
def script(name: str) -> str:
"""One of the shipped browser files, as text.
There is no JavaScript test runner here -- hard rule 1 keeps Node out of the
project -- so behaviour is driven by hand under a DOM stub and what the
suite pins is what the source says about itself.
"""
import lembas
return (Path(lembas.__file__).parent / "web/static/js" / name).read_text(encoding="utf-8")
def js_code(source: str) -> str:
"""The same source with its comments blanked out.
These files are heavily commented, and every comment names the thing it is
explaining -- so `"list" in refresh` is true of a paragraph saying that
writing to `list` too early was the bug. An invariant about the code must
not be satisfiable by prose describing it. String literals are left intact,
because the assertions are usually about an event name inside one.
"""
out: list[str] = []
at, end = 0, len(source)
while at < end:
char = source[at]
if char in "\"'`":
quote = char
out.append(char)
at += 1
while at < end:
if source[at] == "\\":
out.append(source[at : at + 2])
at += 2
continue
out.append(source[at])
at += 1
if source[at - 1] == quote:
break
continue
if char == "/" and source[at : at + 2] in ("//", "/*"):
stop = (
source.find("\n", at)
if source[at + 1] == "/"
else source.find("*/", at) + 2
)
if stop <= at:
stop = end
# Blanked rather than removed, so every offset still lines up.
out.append("".join(" " if c != "\n" else "\n" for c in source[at:stop]))
at = stop
continue
out.append(char)
at += 1
return "".join(out)
def js_says(text: str, *pieces: str) -> bool:
"""Whether the pieces appear, in order, whatever sits between them.
Asserting a line verbatim makes a test that fails on reindentation, which is
noise; asserting only that two words appear somewhere makes one that never
fails at all. This is the middle: the shape, not the spacing.
"""
at = 0
for piece in pieces:
at = text.find(piece, at)
if at == -1:
return False
at += len(piece)
return True
def js_function(source: str, name: str) -> str:
"""The text of one named function, braces matched.
Lets a test ask about the inside of a handler rather than about the file --
"is `build()` called before anything writes to `list`" is a question about
`refresh`, and asking it of the whole source answers yes for the wrong
reason.
"""
code = js_code(source)
# The parenthesis is load-bearing: `token` is a prefix of `tokenAt`, and
# asking about the wrong function is a test that passes for no reason.
start = code.index(f"function {name}(")
at = code.index("{", start)
depth, quote = 0, ""
while at < len(code):
char = code[at]
if quote:
if char == "\\":
at += 2
continue
if char == quote:
quote = ""
elif char in "\"'`":
quote = char
elif char == "{":
depth += 1
elif char == "}":
depth -= 1
if depth == 0:
return code[start : at + 1]
at += 1
raise AssertionError(f"no closing brace for {name}()")
@pytest.fixture
def user_id(db: Session, registered: dict[str, str]) -> str:
"""The registered user's id.
+3
View File
@@ -15,6 +15,9 @@ from sqlalchemy import select
from lembas.db.models import SshProfile, User
from lembas.services.agent import ssh as ssh_service
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+3
View File
@@ -20,6 +20,9 @@ from lembas.services import tools as tools_service
from lembas.services.agent import policy
from lembas.services.agent import ssh as ssh_service
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+2
View File
@@ -22,6 +22,8 @@ from lembas.services import settings_store
from lembas.services.agent import draft as draft_service
from lembas.services.agent import terminal as terminal_service
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
@pytest.fixture(autouse=True)
def _clean():
+3
View File
@@ -28,6 +28,9 @@ pytestmark = pytest.mark.skipif(
)
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
class LocalExecutor:
"""`SshExecutor.run`'s contract, run against the local shell.
+3
View File
@@ -16,6 +16,9 @@ from lembas.db.models import SshProfile, User
from lembas.services import settings_store
from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, encrypt
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+3
View File
@@ -14,6 +14,9 @@ import pytest
from lembas.services.agent import ssh
from lembas.services.agent.base import ExecError, ExecRequest
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+3
View File
@@ -31,6 +31,9 @@ from lembas.services.agent import policy, session
from lembas.services.agent import ssh as ssh_service
from lembas.services.tools import RISK_EXECUTE
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+200
View File
@@ -0,0 +1,200 @@
"""Dictation and read-aloud, checked without a runtime.
There is no JavaScript test runner here and hard rule 1 keeps Node out of the
project, so the behaviour is driven by hand under a DOM stub before committing.
Both halves of this file are progressive -- without it the composer and the
bubbles still work, they simply have two buttons that do nothing -- which is
also the shape of every way it can break: a microphone left open, a button left
disabled, a reply read out twice. None of them raises anything.
What the suite can pin is that there is one recording state machine and one
`<audio>` element, that every path out of a recording puts the button back, and
that writing a transcript into the composer goes through the same two calls
every other outside writer makes.
"""
from __future__ import annotations
import re
from .conftest import js_code, js_function, js_says, script
SOURCE = script("audio.js")
CODE = js_code(SOURCE)
COMMANDS = js_code(script("commands.js"))
# --- One state machine --------------------------------------------------------
def test_there_is_one_delegated_listener_for_both_buttons():
"""Delegated, because a message bubble is rendered from four places and
swapped by htmx constantly -- a listener bound per button would miss every
reply that arrived after load, which is all of them. One listener, and the
two buttons told apart by `closest`."""
assert CODE.count('document.addEventListener("click"') == 1
click = CODE[CODE.index('document.addEventListener("click"') :]
assert 'closest("[data-mic]")' in click
assert 'closest("[data-speak]")' in click
def test_the_shortcuts_press_the_button_rather_than_repeating_the_machine():
"""`Alt+M` and `Alt+R` dispatch by finding the existing control and calling
`.click()`, so this file keeps its one delegated listener and there is no
second copy of the recording state machine to fall out of step with it.
Asserted from commands.js, which is where the second copy would appear.
"""
assert js_says(COMMANDS, "[data-mic]", "mic.click()")
assert js_says(COMMANDS, "[data-speak]", ".click()")
# Nothing here is exported, so there is nothing for a shortcut to reach for
# instead -- which is what keeps that true rather than merely current.
assert not re.search(r"window\.lembas\.\w+\s*=", CODE)
assert "MediaRecorder" not in COMMANDS
def test_the_microphone_is_released_the_moment_recording_stops():
"""Leaving the track live keeps the browser's recording indicator on long
after anyone is talking -- a self-hosted tool that appears to still be
listening is the one bug here nobody would forgive. Released in the `stop`
handler, before the upload, because the upload is a network round trip."""
start = js_function(SOURCE, "startRecording")
stop_handler = start[start.index('addEventListener("stop"') :]
assert "track.stop()" in stop_handler
assert stop_handler.index("track.stop()") < stop_handler.index("upload(")
def test_a_recording_that_captured_nothing_does_not_upload():
"""An empty blob is a press and an immediate second press. Uploading it
costs a round trip to be told nothing was heard, and leaves the button
disabled while that happens."""
start = js_function(SOURCE, "startRecording")
assert js_says(start, "blob.size", "upload(", "else", 'setMicState(button, "idle")')
def test_every_way_out_of_an_upload_puts_the_button_back():
"""`working` is the only state that disables the button, and a disabled
microphone that never comes back is a feature somebody has to reload the
page to use again. Both arms of the promise reset it -- and the error arm is
the one that gets forgotten, because it is the one nobody exercises."""
upload = js_function(SOURCE, "upload")
assert 'setMicState(button, "working")' in upload
assert upload.count('setMicState(button, "idle")') == 2, "a path out leaves it disabled"
assert ".catch(" in upload
state = js_function(SOURCE, "setMicState")
assert js_says(state, "button.disabled", '"working"')
def test_the_label_says_what_the_next_press_will_do():
"""The button is a toggle with one icon slot, so the accessible name is the
only thing saying which half of the toggle it is in."""
state = js_function(SOURCE, "setMicState")
assert '"aria-label"' in state
assert '"Stop recording"' in state and '"Dictate a message"' in state
assert js_says(state, "button.title", 'getAttribute("aria-label")')
def test_plain_http_is_explained_rather_than_silently_doing_nothing():
"""`getUserMedia` is undefined on plain http, which a self-hosted install on
a LAN address often is. Without the guard the button does nothing at all,
forever, with nothing in the console a reader would find -- and the fix is
not something the page can apply for them."""
start = js_function(SOURCE, "startRecording")
guard = start[: start.index("navigator.mediaDevices.getUserMedia({")]
assert "!navigator.mediaDevices" in guard
assert 'typeof MediaRecorder === "undefined"' in guard
assert "HTTPS" in guard
def test_a_refused_microphone_is_reported():
"""Permission can be blocked at the site or at the system, and both reject
the promise. Left uncaught it is a button press that does nothing."""
start = js_function(SOURCE, "startRecording")
assert ".catch(" in start
assert "Permission" in start[start.index(".catch(") :]
def test_dictation_audio_is_never_stored():
"""It has no owner, no row, and nothing would ever sweep it. It goes to the
transcription endpoint and nowhere else -- an attachment path would give a
voice note a permanent home nobody asked for."""
assert "/api/audio/transcribe" in CODE
assert "/api/files" not in CODE
# --- What reaches the composer ------------------------------------------------
def test_the_transcript_is_appended_rather_than_replacing_what_was_typed():
"""Dictation is usually finishing a thought that was already half typed.
Overwriting is the complaint the terminal's Send was rewritten to fix, and
it would be the same complaint here."""
insert = js_function(SOURCE, "insertTranscript")
assert js_says(insert, "existing", "input.value", "existing"), "the transcript replaces the box"
assert "input.value.length" in insert, "the caret is left at the end"
def test_writing_into_the_composer_repaints_its_mirror():
"""The mirror repaints on `input` and after a swap, and this is neither -- so
without the call the highlighting sits over the previous text, shifted by
however much was just dictated, until the next keystroke."""
insert = js_function(SOURCE, "insertTranscript")
assert "paintComposer" in insert
assert "autosize" in insert
# --- Reading a reply aloud ----------------------------------------------------
def test_there_is_one_audio_element_for_the_page():
"""Two replies talking over each other is never what was wanted, and a
shared element makes that impossible rather than merely unlikely."""
player = js_function(SOURCE, "audioPlayer")
assert "if (!player)" in player
assert CODE.count("new Audio()") == 1
def test_a_second_press_stops_it():
"""The button is the only control there is -- there is no separate stop --
so pressing the one that is speaking has to mean stop rather than start
again from the beginning."""
speak = js_function(SOURCE, "speak")
assert "if (speaking === button)" in speak
assert speak.index("element.pause()") < speak.index("element.src")
def test_a_refused_autoplay_is_not_reported_as_a_failure():
"""Autoplay policies reject a `play()` the reader did not ask for. That is
the browser working as intended, and an error toast for it would fire on
every page load where a reply had just finished."""
speak = js_function(SOURCE, "speak")
tail = speak[speak.index("element.play()") :]
assert "markSpeaking(null)" in tail
assert "notify" not in tail
def test_a_finished_reply_is_read_out_once():
"""`data-speak-auto` is set on one frame only, and any swap can bring it in
-- so this watches them all. The attribute is cleared *before* the reply is
played, or a later swap of the same bubble starts it again, and the
transcript reads itself out from the top."""
arrivals = js_function(SOURCE, "playArrivals")
assert arrivals.index('removeAttribute("data-speak-auto")') < arrivals.index("speak(button)")
assert '"htmx:afterSettle"' in CODE
def test_the_microphone_leaves_idle_before_it_waits_for_permission():
"""The dispatcher starts a recording whenever the button reads `idle`, and
the state only moved once `getUserMedia` had resolved. The browser's
permission prompt sits in between, so every click while it was up opened
another stream and another MediaRecorder -- and only the last was kept, so
stopping released one and left the rest live. The browser's recording
indicator then stayed on until the tab was closed.
"""
code = js_code(script("audio.js"))
body = js_function(code, "startRecording")
# Against the *call*, not the capability check above it, which also names
# getUserMedia and legitimately comes first.
call = body.index("getUserMedia({ audio: true })")
assert body.index('setMicState(button, "working")') < call
# And a refusal has to put it back, or the button can never be pressed again
# -- including by somebody who has just gone and allowed it.
catch = body[body.index(".catch(") :]
assert 'setMicState(button, "idle")' in catch
+3
View File
@@ -12,6 +12,9 @@ import pytest
from lembas.services.agent import ssh as ssh_service
from lembas.services.agent.base import Conflict, ExecError
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+140
View File
@@ -0,0 +1,140 @@
"""The four commands, none of which had a test.
`create-admin` is the documented way back into an instance whose administrator
account has been lost, and the bootstrap step `README.md` tells somebody to run
when scripting a deployment. It had never been executed by anything but a
person, which is the worst place to discover that a command does not work: the
moment you cannot get in.
Driven through typer's `CliRunner` rather than by calling the functions, so the
options, the prompts and the exit codes are part of what is under test -- a
`typer.Exit(1)` that the shell reads as success is a script that carries on
after a failure.
"""
from __future__ import annotations
from typer.testing import CliRunner
from lembas import __version__
from lembas.cli import app
runner = CliRunner()
def test_secret_key_prints_something_usable():
"""It is pasted straight into `lembas.env`, so it has to be one line with no
surprises in it. A key with a newline silently truncates the variable."""
result = runner.invoke(app, ["secret-key"])
assert result.exit_code == 0
key = result.stdout.strip()
assert len(key) >= 40
assert "\n" not in key
# Two calls must not agree, or it is not a secret.
assert key != runner.invoke(app, ["secret-key"]).stdout.strip()
def test_info_says_where_the_data_is(db):
"""The command somebody runs when they cannot find the database, which is
exactly when a wrong answer costs the most."""
from lembas.config import settings
result = runner.invoke(app, ["info"])
assert result.exit_code == 0
assert __version__ in result.stdout
assert str(settings.db_path.resolve()) in result.stdout
assert "users" in result.stdout
def test_info_warns_about_a_generated_secret_key(db):
"""An instance running on an ephemeral key loses every session and every
stored API key on restart. It is the one thing on this screen that is a
problem rather than a fact."""
from lembas.config import settings
if not settings.secret_key_is_ephemeral: # pragma: no cover - depends on env
return
assert "GENERATED" in runner.invoke(app, ["info"]).stdout
def test_create_admin_makes_an_administrator(db):
"""The way back in. It has to produce an account that can actually sign in,
so the password is checked by hashing rather than by trusting the message."""
from sqlalchemy import select
from lembas.db.models import ROLE_ADMIN, User
from lembas.security.passwords import verify_password
result = runner.invoke(
app,
[
"create-admin",
"--email", "Frodo@Shire.test",
"--name", "Frodo",
"--password", "speak-friend-and-enter",
],
)
assert result.exit_code == 0, result.stdout
user = db.scalar(select(User).where(User.email == "frodo@shire.test"))
assert user is not None, "the address is lowercased, so look for it that way"
assert user.role == ROLE_ADMIN
assert user.active
assert verify_password("speak-friend-and-enter", user.password_hash)
def test_create_admin_promotes_an_account_that_already_exists(db):
"""Its whole purpose in the lost-password case: the account is there, it is
just no longer an administrator or no longer has a password anybody knows.
A second account with the same address would be no way back in at all."""
from sqlalchemy import func, select
from lembas.db.models import ROLE_ADMIN, ROLE_USER, User
from lembas.security.passwords import verify_password
db.add(
User(
email="sam@shire.test",
name="Sam",
password_hash="not-a-real-hash", # noqa: S106
role=ROLE_USER,
active=False,
)
)
db.commit()
result = runner.invoke(
app,
[
"create-admin",
"--email", "sam@shire.test",
"--name", "Samwise",
"--password", "second-breakfast-please",
],
)
assert result.exit_code == 0, result.stdout
assert db.scalar(select(func.count()).select_from(User)) == 1, "it made a second one"
db.expire_all()
user = db.scalar(select(User).where(User.email == "sam@shire.test"))
assert user.role == ROLE_ADMIN
assert user.active, "a deactivated account has to be let back in, or this fixes nothing"
assert verify_password("second-breakfast-please", user.password_hash)
def test_create_admin_refuses_a_password_that_would_be_rejected_elsewhere(db):
"""And exits non-zero, so a deployment script stops rather than carrying on
believing it has an administrator."""
from sqlalchemy import select
from lembas.db.models import User
result = runner.invoke(
app,
["create-admin", "--email", "x@shire.test", "--name", "X", "--password", "short"],
)
assert result.exit_code == 1
assert db.scalar(select(User)) is None
+287
View File
@@ -0,0 +1,287 @@
"""The composer's typing affordances, checked without a runtime.
There is no JavaScript test runner here and hard rule 1 keeps Node out of the
project, so the behaviour is driven by hand under a DOM stub before committing.
That is not a nicety, and this file is the reason it is written down: `refresh()`
wrote to `list` *before* anything had built it, so the first `/` or `@` ever
typed threw on a null and took the whole handler with it. The menu never
appeared, in any browser, for the entire life of the feature, and `node --check`
parses that file happily.
What the suite can pin are the properties that failure depended on, and the ones
beside it whose absence is equally silent: a listener registered in a phase the
event does not reach, a repaint bound to the htmx event that fires too early,
and a mirror sized differently from the box it sits behind. Every one of those
looks exactly like working.
"""
from __future__ import annotations
import re
from pathlib import Path
import lembas
from .conftest import js_code, js_function, js_says, script
SOURCE = script("composer.js")
CODE = js_code(SOURCE)
CSS = (Path(lembas.__file__).parent / "web/static/css/chat.css").read_text(encoding="utf-8")
def _rule(css: str, opening: str) -> str:
"""One declaration block, by the text that opens it.
The match has to *start* a rule: `.composer__mirror .tok-command {` appears
twice, once as the tail of a grouped selector, and reading the group's block
when the single one was asked for is a test that answers a question nobody
put.
"""
at = -1
while True:
at = css.index(opening, at + 1)
if not css[:at].rstrip().endswith(","):
return css[at : css.index("}", at)]
# --- The menu, and the null it used to throw on -------------------------------
def test_the_menu_is_built_before_anything_writes_to_it():
"""The bug this module exists for. `refresh()` sets `list.innerHTML` and
appends to it, and `list` is null until `build()` has run -- so with the
build left to `show()`, which is called *after* those writes, the first
keystroke threw and the menu never opened for anybody.
Asserted as the order inside `refresh`, with comments stripped: the file
explains this at length in prose, and prose satisfies a substring test.
"""
refresh = js_function(SOURCE, "refresh")
assert "build()" in refresh, "refresh no longer builds the menu"
assert refresh.index("build()") < refresh.index("list"), (
"something reads or writes `list` before the menu exists"
)
def test_building_it_twice_costs_nothing():
"""Which is what lets `refresh` and `show` both call it unconditionally.
Without the guard, every keystroke would insert another menu."""
assert js_function(SOURCE, "build").index("if (menu) return;") < 40
def test_the_late_mention_reply_checks_the_menu_is_still_there():
"""A mention list is a round trip late, and the caret may have moved off the
token -- or the menu may have been dismissed, which drops `list`. Writing
into it anyway is the same null a second time, on a slower path."""
load = js_function(SOURCE, "loadMentions")
assert load.index("!list") < load.index("list.innerHTML")
# And the kind, or a reply for `@` lands in a menu now showing commands.
assert 'kind !== "@"' in load
def test_choosing_with_the_mouse_holds_the_caret():
"""`mousedown` is cancelled inside the menu. Without it the composer loses
focus before the click lands, and the caret position the choice is about to
be written at is already gone -- so the token is replaced at the wrong
offset, or not at all."""
build = js_function(SOURCE, "build")
assert '"mousedown"' in build
assert build.index('"mousedown"') < build.index('"click"')
assert "preventDefault" in build
# --- Listeners, and the phases they must be in --------------------------------
def test_the_key_listener_is_registered_in_the_capture_phase():
"""app.js already has a document-level Enter handler that submits the form,
and listeners on the same element in the same phase fire in registration
order -- app.js loads first. In the bubble phase this one would never get to
say "that Enter chose a menu item, it did not send the message", so choosing
from the menu would send the half-typed line as well.
The property rather than the markup, for the reason `tests/test_ui_js.py`
gives: a trigger bound where the event does not go is silent.
"""
found = re.search(r'"keydown",[\s\S]*?\n\s*(true|false)\n\s*\);', SOURCE)
assert found, "the keydown listener is gone"
assert found.group(1) == "true"
def test_the_scroll_listener_is_too():
"""`scroll` does not bubble. Registered without the third argument this is
never called, and the mirror stops following the textarea the moment the box
grows past `data-max-height` -- the rectangles then sit over the wrong
words, which is the one failure the mirror exists to avoid."""
found = re.search(r'"scroll",[\s\S]*?\n\s*(true|false)\n\s*\);', SOURCE)
assert found, "the scroll listener is gone"
assert found.group(1) == "true"
def test_the_mirror_repaints_after_the_request_as_well_as_after_the_swap():
"""htmx fires afterSwap and afterSettle *before* afterRequest, and the
composer empties itself from `hx-on::after-request` -- so every repaint
bound to the first two ran while the box still held the message, and the
highlighting sat over an empty field until the next keystroke.
Both late events go through the same deferred helper, and that is the whole
assertion: `paint` bound directly to `reset` reads the value in the same
turn the event fires, which is *before* a form's fields are actually
cleared, so it paints the text that is about to vanish.
"""
after = re.search(r'addEventListener\("htmx:afterRequest",\s*(\w+)\)', CODE)
reset = re.search(r'addEventListener\("reset",\s*(\w+)\)', CODE)
assert after and reset, "the late repaints are gone"
assert after.group(1) == reset.group(1), "the two late events repaint differently"
assert after.group(1) != "paint", "a repaint in the same turn paints the old value"
deferred = js_function(SOURCE, after.group(1))
assert "requestAnimationFrame" in deferred
assert "paint" in deferred
# The early ones stay: a swap that brings a new composer in has to paint it.
for event in ("htmx:afterSwap", "htmx:afterSettle"):
assert re.search(rf'addEventListener\("{event}",\s*paint\)', CODE), event
# --- What may become markup ---------------------------------------------------
def test_nothing_typed_into_the_box_can_become_markup():
"""The mirror holds the one string a person controls exactly, character for
character. Hard rule 6 covers model output; this is the same rule pointed
the other way, and it is the only place in the application where somebody's
raw keystrokes are re-rendered as they type.
`list.innerHTML` in the mention path is deliberately not covered: that is a
server-rendered fragment, escaped where it was built.
"""
paint = js_function(SOURCE, "paint")
assert "innerHTML" not in paint
assert "replaceChildren" in paint
assert "createTextNode" in paint
assert "textContent" in js_function(SOURCE, "token")
# --- The mirror and the box are one shape -------------------------------------
METRICS = (
"font-family",
"font-size",
"line-height",
"letter-spacing",
"padding",
"white-space",
"overflow-wrap",
"word-break",
"width",
)
def test_the_mirror_and_the_box_are_sized_by_one_rule():
"""A textarea cannot style its own contents, so the mirror sits behind it
holding the same text with every character transparent. Every property that
decides where a character lands has to be declared once, for both: a single
difference and the rectangles slide off the words they are marking, and the
drift grows with the length of the line rather than showing up as an obvious
misalignment at the start.
So the shared rule is asserted to carry all of them, and neither of the
per-element rules to redeclare any -- which is the version of this that
breaks silently, because a second declaration looks like a local tweak.
"""
shared = _rule(CSS, ".composer__input,\n.composer__mirror {")
for property_ in METRICS:
assert f"{property_}:" in shared, f"{property_} is not shared"
for selector in ("\n.composer__input {", "\n.composer__mirror {"):
own = _rule(CSS, selector)
for property_ in METRICS:
assert f"{property_}:" not in own, f"{selector.strip()} redeclares {property_}"
def test_a_token_restates_what_it_would_otherwise_inherit():
"""`color: transparent` on the mirror is an inherited value, and any colour
a span declares itself beats it. The transcript's rule used to do exactly
that from across the file, painting the token in accent-coloured monospace
at 0.95em on top of the textarea's own text: doubled, and shifted from that
point on because the metrics differ."""
tokens = _rule(CSS, ".composer__mirror .tok-mention,\n.composer__mirror .tok-command {")
assert "color: transparent" in tokens
assert "font: inherit" in tokens
def test_a_token_bleeds_with_a_shadow_rather_than_with_padding():
"""The rectangle has to be a little wider than the text it sits behind.
Padding and a negative margin do that by *moving the glyph*, which is the
one thing nothing in the mirror is allowed to do -- a shadow cannot."""
tokens = _rule(CSS, ".composer__mirror .tok-mention,\n.composer__mirror .tok-command {")
assert "padding: 0" in tokens
assert "margin: 0" in tokens
for selector in (".composer__mirror .tok-mention {", ".composer__mirror .tok-command {"):
rule = _rule(CSS, selector)
assert "box-shadow" in rule, f"{selector} does not bleed"
assert "padding" not in rule, f"{selector} moves the glyph"
assert "margin" not in rule, f"{selector} moves the glyph"
def test_every_token_style_names_where_it_applies():
"""A rule written for one context matches every context. `.tok-mention`
unscoped was written for the transcript and also hit the mirror's spans,
which is the bug above. Both places qualify their selectors now, and an
unqualified one added later would do it again with nothing to notice."""
for line in CSS.splitlines():
if ".tok-" not in line:
continue
selector = line.strip()
assert selector.startswith((".msg ", ".composer__mirror ")), selector
# --- Slash commands, from the composer's side ---------------------------------
def test_an_unrecognised_slash_is_never_swallowed():
"""Eating somebody's message because it began with a slash is a far worse
failure than an unknown command. Enter only cancels the send when the value
resolves to a command -- so the `preventDefault` has to sit *inside* that
test, not beside it."""
window = CODE[CODE.index('event.key === "Enter" && !event.shiftKey') :][:600]
assert window.index("commandIn(") < window.index("preventDefault")
assert "if (command) {" in window
def test_choosing_a_command_clears_the_box_before_running_it():
"""A command is the whole message, never part of one. `/help` left sitting
in the box after running means the next Enter runs it again -- which is what
driving this under a DOM stub found, along with Tab not completing."""
choose = js_function(SOURCE, "choose")
assert js_says(choose, "input.value", '""', "runCommand")
def test_tab_completes_and_enter_runs():
"""The difference matters for a command that takes an argument: Tab writes
`/mode ` and leaves the caret after it, Enter runs what is there. Both are
one call, so the key is what decides -- and `choose` has to branch on it
before it reaches the run."""
assert js_says(CODE, "choose(all[active]", 'event.key === "Tab"')
choose = js_function(SOURCE, "choose")
assert choose.index("if (complete)") < choose.index("runCommand")
def test_the_at_menu_works_on_a_page_without_commands_js():
"""`@` and `/` share one menu but not one dependency. commands.js is absent
from some pages and may fail to parse on any of them, and either would
otherwise take the mention picker down with it."""
fallback = js_function(SOURCE, "commands")
assert "window.lembasCommands ||" in fallback
for method in ("list:", "find:", "run:"):
assert method in fallback, f"the fallback has no {method}"
# Read at call time rather than captured once, or a page whose commands.js
# loads second gets the fallback forever.
for helper in ("commandList", "commandIn", "runCommand"):
assert "commands()." in js_function(SOURCE, helper)
def test_the_directory_is_read_from_the_field_that_is_submitted():
"""The chip beside the connection shows the directory's own name so it stops
eating the composer's one row; the whole path lives in the hidden field. A
mention picker reading the label would ask the server about `worker` rather
than about `/srv/projects//worker` -- shortening a label must never shorten
a value, on this path either."""
context = js_function(SOURCE, "context")
assert "[data-dir-value]" in context
assert "data-dir-label" not in context
+981
View File
@@ -0,0 +1,981 @@
"""The library at the HTTP boundary: who may read, who may write, who may delete.
`tests/test_library.py` covers the four stores by calling the services. That
leaves untested the layer where the answers to "is this yours?" actually live:
the router's single `library.use` dependency, the `sharing.can_write` check in
front of every write, and the three ways a route says no -- a redirect to the
login page, a 403, and a 404.
Every assertion here is on a database row. A route that answers 403 and changes
the row anyway, or answers 200 and changes nothing, is exactly the failure this
file exists to catch, and neither is visible from a status code alone.
**404 versus 403 is a real distinction here, not an accident.** 404 means "you
cannot see this at all", and it is deliberately the same answer a missing id
gets, so probing ids tells a stranger nothing. 403 means "you can see it and it
is not yours to change" -- reachable only through a share, which grants reading
only. The deletes fold both into 404, and the tests below say so where it
happens rather than pretending it is uniform.
"""
from __future__ import annotations
import re
import httpx
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import (
Document,
KnowledgeBase,
Memory,
Note,
Share,
Skill,
SkillRevision,
User,
)
from lembas.services import settings_store, sharing
from lembas.services.library import documents as documents_service
from lembas.services.library import memories as memories_service
from lembas.services.library import notes as notes_service
from lembas.services.library import skills as skills_service
STRANGER = {"name": "Sam", "email": "sam@shire.test", "password": "gardening-is-hard"}
# --- People -------------------------------------------------------------------
@pytest.fixture
def owner(db, registered) -> User:
"""The first account, which is therefore the administrator."""
return db.scalars(select(User).order_by(User.created_at)).first()
@pytest.fixture
def stranger(client, registered) -> TestClient:
"""A second signed-in account, in its own browser.
Registered second on purpose: the first account becomes the administrator,
and an administrator bypasses every permission. Ownership questions have to
be asked of somebody who does not.
"""
other = TestClient(client.app)
response = other.post("/auth/register", data=STRANGER, follow_redirects=False)
assert response.status_code == 303, response.text
return other
@pytest.fixture
def stranger_user(db, stranger) -> User:
return db.scalar(select(User).where(User.email == STRANGER["email"]))
@pytest.fixture
def public(monkeypatch):
"""Make every hostname resolve to a public address, resolving nothing."""
import socket
monkeypatch.setattr(
socket, "getaddrinfo", lambda *a, **k: [(2, 1, 6, "", ("93.184.216.34", 80))]
)
# --- Rows ---------------------------------------------------------------------
def _base(db, owner: User, name: str = "Papers") -> KnowledgeBase:
return documents_service.create_base(db, owner=owner, name=name)
def _document(db, owner: User, base: KnowledgeBase | None = None, **kwargs) -> Document:
fields = {
"payload": b"The west gate of Moria was built by Narvi.",
"filename": "gate.txt",
"title": "Gate",
**kwargs,
}
return documents_service.store_upload(db, owner=owner, base=base, **fields)
def _note(db, owner: User, title: str = "Mellon", body: str = "Speak friend.") -> Note:
return notes_service.create(db, owner=owner, title=title, body=body)
def _skill(db, owner: User, name: str = "weekly-report", body: str = "Step one.") -> Skill:
return skills_service.create(
db, owner=owner, name=name, description="When a week ends.", body=body
)
def _memory(db, owner: User, content: str = "Prefers metric units.") -> Memory:
return memories_service.add(db, owner=owner, content=content)
# --- The one gate in front of everything --------------------------------------
def _routes():
"""Every route this router serves, with its path parameters filled in.
Read off the router rather than listed by hand, so a route added tomorrow is
covered by the permission test the day it appears.
"""
from lembas.api import library
for route in library.router.routes:
for method in sorted(route.methods - {"HEAD", "OPTIONS"}):
yield method, re.sub(r"\{[^}]+\}", "no-such-id", route.path)
def test_every_route_is_behind_library_use(db, client, stranger):
"""The gate is one dependency on the router, so a route added without a
thought inherits it -- and a route moved to another router silently loses it.
Without this, somebody with the library switched off keeps a working set of
URLs: the pages vanish from their sidebar and every one of them still answers.
Asserted against a fabricated id so nothing here depends on what exists.
"""
settings_store.update(db, {"default_permissions": {"library.use": False}})
refused = []
for method, path in _routes():
response = stranger.request(method, path, follow_redirects=False)
refused.append((method, path, response.status_code))
assert [row for row in refused if row[2] != 403] == []
assert len(refused) >= 28, "the router lost routes; this test is no longer covering them"
def test_the_gate_is_checked_before_the_write_happens(db, client, stranger, stranger_user):
"""A guard that runs after the row is written is not a guard. Asserted on the
absence of the note rather than on the status code, because a 403 returned
over a completed write looks identical from the outside."""
settings_store.update(db, {"default_permissions": {"library.use": False}})
stranger.post(
"/api/library/notes", data={"title": "Smuggled", "body": "x"}, follow_redirects=False
)
db.expire_all()
assert db.scalars(select(Note)).all() == []
def test_an_administrator_bypasses_the_permission(db, client, registered, owner):
"""Deliberate, and the same rule the rest of the codebase follows: an admin
can grant themselves the permission in two clicks, so withholding it is
theatre. Worth pinning because the ownership tests below rely on the
*opposite* being true of sharing."""
settings_store.update(db, {"default_permissions": {"library.use": False}})
client.post("/api/library/notes", data={"title": "Mine", "body": "x"}, follow_redirects=False)
db.expire_all()
assert [n.title for n in db.scalars(select(Note))] == ["Mine"]
def test_signed_out_changes_nothing(db, client, registered, owner):
"""A delete needs a session. Without this, a link somebody was sent removes
a note from an account nobody is signed in to."""
note = _note(db, owner)
nobody = TestClient(client.app)
nobody.post(f"/api/library/notes/{note.id}/delete", follow_redirects=False)
db.expire_all()
assert db.get(Note, note.id) is not None
# --- Reading somebody else's work ---------------------------------------------
def test_a_strangers_note_is_not_readable(db, client, stranger, owner):
"""404 rather than 403: an id somebody does not own must be indistinguishable
from an id that does not exist, or the detail page becomes an oracle for
which notes are on the instance."""
note = _note(db, owner, title="Private", body="mellon")
response = stranger.get(f"/library/notes/{note.id}")
assert response.status_code == 404
assert "mellon" not in response.text
def test_a_strangers_skill_is_not_readable(db, client, stranger, owner):
skill = _skill(db, owner, body="The secret procedure.")
response = stranger.get(f"/library/skills/{skill.id}")
assert response.status_code == 404
assert "The secret procedure." not in response.text
def test_a_strangers_base_is_not_readable(db, client, stranger, owner):
base = _base(db, owner, name="Contracts")
_document(db, owner, base, title="The lease")
response = stranger.get(f"/library/knowledge/{base.id}")
assert response.status_code == 404
assert "The lease" not in response.text
def test_a_strangers_document_is_not_readable(db, client, stranger, owner):
"""Visibility comes from the base, never the document -- so this is also the
check that `documents_service.get` resolves through the base rather than
trusting an id it was handed."""
document = _document(db, owner, _base(db, owner))
response = stranger.get(f"/library/knowledge/document/{document.id}")
assert response.status_code == 404
def test_a_strangers_document_file_is_not_served(db, client, stranger, owner):
"""The bytes, not the page. A route that authorises the detail view and
serves the file to anyone is a library that is private in the UI only."""
document = _document(db, owner, _base(db, owner), payload=b"Narvi built it.")
response = stranger.get(f"/api/library/documents/{document.id}/content")
assert response.status_code == 404
assert b"Narvi built it." not in response.content
def test_an_administrator_cannot_read_somebody_elses_note(db, client, registered, owner):
"""`permissions.resolve` gives an admin everything and `services/sharing.py`
deliberately has no admin branch. Reading somebody's private notes is not
configuration, and being able to reach the database is not being invited."""
other = User(email="merry@shire.test", name="Merry", password_hash="x") # noqa: S106
db.add(other)
db.commit()
note = _note(db, other, title="Theirs", body="not for you")
assert owner.is_admin is True
response = client.get(f"/library/notes/{note.id}")
assert response.status_code == 404
assert "not for you" not in response.text
def test_a_shared_note_is_readable(db, client, stranger, stranger_user, owner):
"""The other half of the rule above: a grant does reach the detail page.
Without this passing, the refusals could be a route that refuses everybody."""
note = _note(db, owner, title="Shared", body="Speak friend.")
sharing.set_grants(db, note, user_ids=[stranger_user.id], group_ids=[])
response = stranger.get(f"/library/notes/{note.id}")
assert response.status_code == 200
assert "Speak friend." in response.text
# --- Writing somebody else's work ---------------------------------------------
def test_a_stranger_cannot_edit_a_note(db, client, stranger, owner):
note = _note(db, owner, title="Mine", body="Original.")
stranger.post(
f"/api/library/notes/{note.id}",
data={"title": "Theirs now", "body": "Rewritten."},
follow_redirects=False,
)
db.expire_all()
assert db.get(Note, note.id).body == "Original."
def test_a_stranger_cannot_delete_a_note(db, client, stranger, owner):
note = _note(db, owner)
stranger.post(f"/api/library/notes/{note.id}/delete", follow_redirects=False)
db.expire_all()
assert db.get(Note, note.id) is not None
def test_a_shared_note_is_read_only(db, client, stranger, stranger_user, owner):
"""Sharing grants reading and nothing else. Two people editing one note with
no history and no merge is worse than the inconvenience of copying it.
The two refusals differ and that is what this pins: `update` answers 403
because the reader can see the note, while `delete` folds "not visible" and
"not yours" into one 404. Both refuse; only the wording differs.
"""
note = _note(db, owner, title="Shared", body="Original.")
sharing.set_grants(db, note, user_ids=[stranger_user.id], group_ids=[])
edited = stranger.post(
f"/api/library/notes/{note.id}", data={"body": "Rewritten."}, follow_redirects=False
)
removed = stranger.post(f"/api/library/notes/{note.id}/delete", follow_redirects=False)
assert edited.status_code == 403
assert removed.status_code == 404
db.expire_all()
assert db.get(Note, note.id).body == "Original."
def test_a_stranger_cannot_edit_a_skill(db, client, stranger, owner):
"""A skill is instructions a model follows without being asked twice.
Somebody else editing one is somebody else's words in your model's prompt."""
skill = _skill(db, owner, body="Original.")
stranger.post(
f"/api/library/skills/{skill.id}",
data={"description": "Changed.", "body": "Rewritten.", "enabled": "on"},
follow_redirects=False,
)
db.expire_all()
assert db.get(Skill, skill.id).body == "Original."
def test_a_stranger_cannot_delete_a_skill(db, client, stranger, owner):
skill = _skill(db, owner)
stranger.post(f"/api/library/skills/{skill.id}/delete", follow_redirects=False)
db.expire_all()
assert db.get(Skill, skill.id) is not None
def test_a_stranger_cannot_revert_a_skill(db, client, stranger, owner):
"""Revert is a write dressed as history: it replaces the body with an older
one. A route that checked only "does this revision exist" would let anybody
roll back anybody's skill."""
skill = _skill(db, owner, body="First.")
skills_service.update(db, skill, body="Second.")
revision = skill.revisions[0]
stranger.post(
f"/api/library/skills/{skill.id}/revert/{revision.id}", follow_redirects=False
)
db.expire_all()
assert db.get(Skill, skill.id).body == "Second."
def test_a_stranger_cannot_rename_a_base(db, client, stranger, owner):
base = _base(db, owner, name="Contracts")
stranger.post(
f"/api/library/bases/{base.id}", data={"name": "Theirs"}, follow_redirects=False
)
db.expire_all()
assert db.get(KnowledgeBase, base.id).name == "Contracts"
def test_a_shared_base_cannot_be_renamed_or_deleted(db, client, stranger, stranger_user, owner):
"""A base is the unit of sharing, so somebody who was given one has the
strongest claim to being able to change it -- and still cannot. Renaming it
would change what it says on the owner's own page."""
base = _base(db, owner, name="Contracts")
_document(db, owner, base)
sharing.set_grants(db, base, user_ids=[stranger_user.id], group_ids=[])
renamed = stranger.post(
f"/api/library/bases/{base.id}", data={"name": "Theirs"}, follow_redirects=False
)
removed = stranger.post(f"/api/library/bases/{base.id}/delete", follow_redirects=False)
assert renamed.status_code == 403
assert removed.status_code == 404
db.expire_all()
assert db.get(KnowledgeBase, base.id).name == "Contracts"
assert len(db.scalars(select(Document)).all()) == 1
def test_a_stranger_cannot_delete_a_base(db, client, stranger, owner):
"""Not visible at all, so 404 -- and the documents inside it survive, which
is the part that matters: `delete_base` takes its contents with it."""
base = _base(db, owner, name="Contracts")
_document(db, owner, base)
stranger.post(f"/api/library/bases/{base.id}/delete", follow_redirects=False)
db.expire_all()
assert db.get(KnowledgeBase, base.id) is not None
assert len(db.scalars(select(Document)).all()) == 1
def test_a_stranger_cannot_edit_a_document(db, client, stranger, owner):
document = _document(db, owner, _base(db, owner), title="The lease")
stranger.post(
f"/api/library/documents/{document.id}",
data={"title": "Theirs"},
follow_redirects=False,
)
db.expire_all()
assert db.get(Document, document.id).title == "The lease"
def test_a_stranger_cannot_delete_a_document(db, client, stranger, owner):
"""The file on disk as well as the row. A delete that removed the bytes and
then refused would leave the owner with a document that cannot be opened."""
document = _document(db, owner, _base(db, owner))
stored = documents_service.stored_path(document.stored_name)
assert stored is not None
stranger.post(f"/api/library/documents/{document.id}/delete", follow_redirects=False)
db.expire_all()
assert db.get(Document, document.id) is not None
assert stored.is_file()
def test_a_document_in_a_shared_base_is_readable_and_not_writable(
db, client, stranger, stranger_user, owner
):
"""The one store whose visibility does not come from itself. A reader who
can see a document through somebody else's base must not be able to retitle
it or delete it out of that base."""
base = _base(db, owner, name="Contracts")
document = _document(db, owner, base, title="The lease")
sharing.set_grants(db, base, user_ids=[stranger_user.id], group_ids=[])
assert stranger.get(f"/library/knowledge/document/{document.id}").status_code == 200
edited = stranger.post(
f"/api/library/documents/{document.id}", data={"title": "Theirs"}, follow_redirects=False
)
removed = stranger.post(
f"/api/library/documents/{document.id}/delete", follow_redirects=False
)
assert edited.status_code == 403
assert removed.status_code == 404
db.expire_all()
assert db.get(Document, document.id).title == "The lease"
def test_a_stranger_cannot_edit_a_memory(db, client, stranger, owner):
"""Memories are injected into every request. Somebody else writing one is
somebody else putting a standing instruction in front of your model."""
memory = _memory(db, owner, content="Prefers metric units.")
response = stranger.post(
f"/api/library/memories/{memory.id}", data={"content": "Trusts strangers."},
follow_redirects=False,
)
assert response.status_code == 404
db.expire_all()
assert db.get(Memory, memory.id).content == "Prefers metric units."
def test_a_stranger_cannot_delete_a_memory(db, client, stranger, owner):
"""Memories are not shareable at all, so there is no 403 case here: anything
that is not yours is invisible."""
memory = _memory(db, owner)
response = stranger.post(
f"/api/library/memories/{memory.id}/delete", follow_redirects=False
)
assert response.status_code == 404
db.expire_all()
assert db.get(Memory, memory.id) is not None
# --- Deletes that are supposed to work ----------------------------------------
def test_deleting_a_note_forgets_its_grants(db, client, registered, owner):
"""`Share` carries no foreign key in either direction, so nothing cascades.
A grant left behind names an id nothing owns, and would grant access to
whoever next received it."""
note = _note(db, owner)
sharing.set_grants(
db, note, user_ids=[], group_ids=["a-group-that-will-outlive-this"]
)
client.post(f"/api/library/notes/{note.id}/delete", follow_redirects=False)
db.expunge_all()
assert db.get(Note, note.id) is None
assert db.scalars(select(Share)).all() == []
def test_deleting_a_skill_takes_its_history_and_its_grants(db, client, registered, owner):
"""Revisions are the entire safety story for a model rewriting its own
instructions. Orphaned ones are rows nothing can ever reach again."""
skill = _skill(db, owner, body="First.")
skills_service.update(db, skill, body="Second.")
sharing.set_grants(db, skill, user_ids=[], group_ids=["team"])
assert db.scalars(select(SkillRevision)).all()
client.post(f"/api/library/skills/{skill.id}/delete", follow_redirects=False)
db.expunge_all()
assert db.get(Skill, skill.id) is None
assert db.scalars(select(SkillRevision)).all() == []
assert db.scalars(select(Share)).all() == []
def test_deleting_a_document_removes_the_file(db, client, registered, owner):
"""The row going and the bytes staying is a disk that fills with files
nothing references and nothing will ever delete."""
document = _document(db, owner, _base(db, owner))
stored = documents_service.stored_path(document.stored_name)
assert stored is not None and stored.is_file()
client.post(f"/api/library/documents/{document.id}/delete", follow_redirects=False)
db.expunge_all()
assert db.get(Document, document.id) is None
assert not stored.is_file()
def test_deleting_a_base_takes_its_documents_and_their_files(db, client, registered, owner):
"""A base is a place, not a label: leaving its contents behind would need an
"unfiled" concept that exists only to hold the wreckage of deletes. The
other base is here because a delete that took everything would look correct
from inside a single-base test."""
doomed = _base(db, owner, name="Contracts")
kept = _base(db, owner, name="Recipes")
inside = _document(db, owner, doomed)
elsewhere = _document(db, owner, kept)
stored = documents_service.stored_path(inside.stored_name)
sharing.set_grants(db, doomed, user_ids=[], group_ids=["team"])
client.post(f"/api/library/bases/{doomed.id}/delete", follow_redirects=False)
db.expunge_all()
assert db.get(KnowledgeBase, doomed.id) is None
assert db.get(Document, inside.id) is None
assert not stored.is_file()
assert db.get(Document, elsewhere.id) is not None
assert db.scalars(select(Share)).all() == []
def test_deleting_a_memory_works_for_its_owner(db, client, registered, owner):
memory = _memory(db, owner)
client.post(f"/api/library/memories/{memory.id}/delete", follow_redirects=False)
db.expunge_all()
assert db.get(Memory, memory.id) is None
def test_reverting_a_skill_restores_the_body_and_keeps_the_way_back(
db, client, registered, owner
):
"""Going back has to be undoable too, or a revert made by mistake is the one
change in this store with no record of what it replaced."""
skill = _skill(db, owner, body="First.")
skills_service.update(db, skill, body="Second.")
revision = skill.revisions[0]
client.post(
f"/api/library/skills/{skill.id}/revert/{revision.id}", follow_redirects=False
)
db.expire_all()
restored = db.get(Skill, skill.id)
assert restored.body == "First."
assert [r.body for r in restored.revisions] == ["Second.", "First."]
def test_a_revision_belonging_to_another_skill_is_refused(db, client, registered, owner):
"""The revision id is a second, independent handle on somebody's data. Without
the `revision.skill_id != skill.id` check, one skill could be overwritten with
the contents of any other -- including one shared to you and not yours."""
victim = _skill(db, owner, name="victim", body="Victim body.")
donor = _skill(db, owner, name="donor", body="Donor body.")
skills_service.update(db, donor, body="Donor changed.")
foreign = donor.revisions[0]
response = client.post(
f"/api/library/skills/{victim.id}/revert/{foreign.id}", follow_redirects=False
)
assert response.status_code == 404
db.expire_all()
assert db.get(Skill, victim.id).body == "Victim body."
# --- Bases --------------------------------------------------------------------
def test_creating_a_base_writes_it_to_the_signed_in_account(db, client, registered, owner):
client.post(
"/api/library/bases", data={"name": " Contracts ", "description": "Leases."},
follow_redirects=False,
)
db.expire_all()
bases = db.scalars(select(KnowledgeBase)).all()
assert [(b.name, b.owner_id) for b in bases] == [("Contracts", owner.id)]
def test_a_duplicate_base_name_makes_no_second_row(db, client, registered, owner):
"""The name is what somebody picks from when filing a document. Two called
"Contracts" is a choice nobody can make correctly."""
_base(db, owner, name="Contracts")
response = client.post(
"/api/library/bases", data={"name": "Contracts"}, follow_redirects=False
)
db.expire_all()
assert len(db.scalars(select(KnowledgeBase)).all()) == 1
assert response.status_code == 303
assert "error=" in response.headers["location"]
def test_two_people_may_each_have_a_base_of_the_same_name(db, client, stranger, owner):
"""Uniqueness is per owner. A shared instance where the first person to say
"My documents" takes the name from everybody else is unusable."""
_base(db, owner, name="Contracts")
stranger.post("/api/library/bases", data={"name": "Contracts"}, follow_redirects=False)
db.expire_all()
assert len(db.scalars(select(KnowledgeBase)).all()) == 2
def test_renaming_a_base_keeps_its_documents(db, client, registered, owner):
base = _base(db, owner, name="Contracts")
document = _document(db, owner, base)
client.post(
f"/api/library/bases/{base.id}",
data={"name": "Leases", "description": "Signed ones."},
follow_redirects=False,
)
db.expire_all()
assert db.get(KnowledgeBase, base.id).name == "Leases"
assert db.get(Document, document.id).base_id == base.id
def test_an_empty_name_leaves_the_base_named(db, client, registered, owner):
"""A rename form submitted with the field cleared must not produce a base
with no name, which is a row nobody can pick out of a list."""
base = _base(db, owner, name="Contracts")
client.post(f"/api/library/bases/{base.id}", data={"name": " "}, follow_redirects=False)
db.expire_all()
assert db.get(KnowledgeBase, base.id).name == "Contracts"
# --- Documents ----------------------------------------------------------------
def test_an_upload_lands_in_the_base_it_named(db, client, registered, owner):
base = _base(db, owner, name="Contracts")
client.post(
"/api/library/documents",
data={"title": "The lease", "base_id": base.id},
files={"file": ("lease.txt", b"Signed at Bag End.", "text/plain")},
follow_redirects=False,
)
db.expire_all()
stored = db.scalars(select(Document)).all()
assert [(d.title, d.base_id) for d in stored] == [("The lease", base.id)]
def test_an_upload_naming_an_invisible_base_goes_to_your_own(db, client, stranger, owner):
"""`get_base` cannot see it, so the request falls back to the uploader's own
default base. The property that matters is that nothing lands in somebody
else's library -- and the redirect goes to a base the uploader can open, so
the document is not silently lost either."""
theirs = _base(db, owner, name="Contracts")
stranger.post(
"/api/library/documents",
data={"base_id": theirs.id},
files={"file": ("mine.txt", b"My own file.", "text/plain")},
follow_redirects=False,
)
db.expire_all()
landed = db.scalars(select(Document)).all()
assert len(landed) == 1
assert landed[0].base_id != theirs.id
assert db.get(KnowledgeBase, landed[0].base_id).owner_id != owner.id
def test_an_upload_into_a_base_shared_to_you_is_refused(db, client, stranger, stranger_user, owner):
"""Visible and still not writable. Adding a document to somebody's base would
put your file in front of everybody they shared it with, under their name."""
base = _base(db, owner, name="Contracts")
sharing.set_grants(db, base, user_ids=[stranger_user.id], group_ids=[])
response = stranger.post(
"/api/library/documents",
data={"base_id": base.id},
files={"file": ("mine.txt", b"My own file.", "text/plain")},
follow_redirects=False,
)
assert response.status_code == 403
db.expire_all()
assert db.scalars(select(Document)).all() == []
def test_a_document_can_be_moved_between_your_own_bases(db, client, registered, owner):
"""Moving is what changes who can see a document, since visibility comes from
the base. It has to actually move."""
origin = _base(db, owner, name="Inbox")
destination = _base(db, owner, name="Contracts")
document = _document(db, owner, origin)
client.post(
f"/api/library/documents/{document.id}",
data={"title": "The lease", "base_id": destination.id},
follow_redirects=False,
)
db.expire_all()
assert db.get(Document, document.id).base_id == destination.id
def test_a_document_cannot_be_moved_into_a_base_you_only_read(
db, client, stranger, stranger_user, owner
):
"""Moving it there would hand it to that base's owner and to everybody they
shared it with. The route drops the move and keeps the rest of the save,
which is why this asserts on `base_id` rather than on a status code."""
theirs = _base(db, owner, name="Contracts")
sharing.set_grants(db, theirs, user_ids=[stranger_user.id], group_ids=[])
mine = _base(db, stranger_user, name="Mine")
document = _document(db, stranger_user, mine)
stranger.post(
f"/api/library/documents/{document.id}",
data={"title": "Kept", "base_id": theirs.id},
follow_redirects=False,
)
db.expire_all()
assert db.get(Document, document.id).base_id == mine.id
def test_a_documents_file_is_served_as_an_attachment(db, client, registered, owner):
"""An uploaded .html served inline executes in this origin, with the session
cookie. The header is the whole defence, so it is asserted rather than the
body."""
document = _document(
db, owner, _base(db, owner), payload=b"<h1>hello</h1>", filename="page.html"
)
response = client.get(f"/api/library/documents/{document.id}/content")
assert response.status_code == 200
assert response.headers["x-content-type-options"] == "nosniff"
assert response.headers["content-disposition"].startswith("attachment")
def test_a_document_whose_file_is_gone_says_so(db, client, registered, owner):
"""A stored name that no longer resolves is a 404, not a traceback: files can
go missing under a server and the page above this one still has to render."""
document = _document(db, owner, _base(db, owner))
documents_service.stored_path(document.stored_name).unlink()
assert client.get(f"/api/library/documents/{document.id}/content").status_code == 404
# --- The link route, which reaches out ----------------------------------------
def test_saving_a_link_refuses_a_loopback_address(db, client, registered, owner):
"""This route hands a user-supplied URL to a fetcher running on a server that
can reach LLeMbas itself, the router, and every other service on the box. The
refusal has to happen here, not just in the tests of `fetch`."""
response = client.post(
"/api/library/documents/link",
data={"url": "http://127.0.0.1:8080/admin"},
follow_redirects=False,
)
assert response.status_code == 400
db.expire_all()
assert db.scalars(select(Document)).all() == []
def test_saving_a_link_refuses_a_name_that_resolves_to_loopback(db, client, registered, owner):
"""The check is on the resolved address. A hostname pointing at 127.0.0.1 is
the obvious way past one that only reads the text of the URL."""
response = client.post(
"/api/library/documents/link", data={"url": "http://localhost/"}, follow_redirects=False
)
assert response.status_code == 400
db.expire_all()
assert db.scalars(select(Document)).all() == []
def test_saving_a_link_refuses_a_redirect_onto_the_server(
db, client, registered, owner, mock_http, monkeypatch
):
"""Every hop, not just the first. httpx's own following would validate the
address somebody typed and then land wherever it was sent."""
import socket
monkeypatch.setattr(
socket,
"getaddrinfo",
lambda host, *a, **k: [(2, 1, 6, "", ("93.184.216.34", 80))]
if host == "example.com"
else [(2, 1, 6, "", ("127.0.0.1", 80))],
)
mock_http(lambda _r: httpx.Response(302, headers={"location": "http://127.0.0.1:8080/admin"}))
response = client.post(
"/api/library/documents/link",
data={"url": "https://example.com/"},
follow_redirects=False,
)
assert response.status_code == 400
db.expire_all()
assert db.scalars(select(Document)).all() == []
def test_saving_a_link_keeps_the_page_as_text(db, client, registered, owner, mock_http, public):
"""The point of keeping a page is what it said. Stored as text rather than
HTML, so nothing has to be reduced again on every read."""
base = _base(db, owner, name="Reading")
mock_http(
lambda _r: httpx.Response(
200,
headers={"content-type": "text/html"},
text=(
"<html><head><title>Mallorn</title></head>"
"<body><p>A golden tree.</p></body></html>"
),
)
)
client.post(
"/api/library/documents/link",
data={"url": "https://example.com/mallorn", "base_id": base.id},
follow_redirects=False,
)
db.expire_all()
stored = db.scalars(select(Document)).all()
assert len(stored) == 1
assert stored[0].title == "Mallorn"
assert stored[0].extracted_text == "A golden tree."
assert stored[0].base_id == base.id
def test_the_administrators_switch_reaches_the_link_route(db, client, registered, owner, mock_http):
"""`allow_private_fetch` is an instance setting, and a route that never read
it would leave the switch in the admin page doing nothing at all -- the exact
failure this codebase keeps cataloguing."""
settings_store.update(db, {"allow_private_fetch": True}, key=settings_store.SEARCH)
mock_http(
lambda _r: httpx.Response(
200, headers={"content-type": "text/html"}, text="<p>An internal page.</p>"
)
)
client.post(
"/api/library/documents/link",
data={"url": "http://127.0.0.1:9/notes"},
follow_redirects=False,
)
db.expire_all()
assert [d.extracted_text for d in db.scalars(select(Document))] == ["An internal page."]
def test_saving_a_link_into_a_base_shared_to_you_is_refused(
db, client, stranger, stranger_user, owner, mock_http, public
):
"""The same gate the upload route has. Without it, the fetch is the way round
a refusal on the other door into the same base."""
base = _base(db, owner, name="Contracts")
sharing.set_grants(db, base, user_ids=[stranger_user.id], group_ids=[])
mock_http(
lambda _r: httpx.Response(200, headers={"content-type": "text/html"}, text="<p>x</p>")
)
response = stranger.post(
"/api/library/documents/link",
data={"url": "https://example.com/", "base_id": base.id},
follow_redirects=False,
)
assert response.status_code == 403
db.expire_all()
assert db.scalars(select(Document)).all() == []
# --- Route order, which fails silently ----------------------------------------
def test_the_static_segments_are_not_parsed_as_ids(db, client, registered, owner):
"""FastAPI matches in registration order, so `/library/notes/new` reaching the
detail route would 404 on a note called "new" -- and there would be no way to
write one. This has already been a bug once in the model admin."""
assert client.get("/library/notes/new").status_code == 200
assert client.get("/library/skills/new").status_code == 200
document = _document(db, owner, _base(db, owner))
assert client.get(f"/library/knowledge/document/{document.id}").status_code == 200
def test_the_link_route_is_not_parsed_as_a_document_id(db, client, registered, owner):
"""`/api/library/documents/link` is registered before
`/api/library/documents/{document_id}`. Registered after it, saving a page
would answer "that document is not available" and nobody would guess why."""
response = client.post(
"/api/library/documents/link", data={"url": "not-a-url"}, follow_redirects=False
)
assert response.status_code == 400
# --- Memory -------------------------------------------------------------------
def test_an_empty_memory_is_not_recorded(db, client, registered, owner):
"""A blank line in front of the model on every turn, forever, and no way to
tell which of several it is when removing one."""
response = client.post("/api/library/memories", data={"content": " "}, follow_redirects=False)
db.expire_all()
assert db.scalars(select(Memory)).all() == []
assert "error=" in response.headers["location"]
def test_emptying_a_memory_leaves_it_alone(db, client, registered, owner):
memory = _memory(db, owner, content="Prefers metric units.")
response = client.post(
f"/api/library/memories/{memory.id}", data={"content": ""}, follow_redirects=False
)
assert response.status_code == 400
db.expire_all()
assert db.get(Memory, memory.id).content == "Prefers metric units."
def test_the_same_memory_twice_makes_one_record(db, client, registered, owner):
"""The commonest failure in this store, and worse than wasted tokens: the
same preference saved twice makes removing it ambiguous for both."""
for _ in range(2):
client.post(
"/api/library/memories",
data={"content": "Prefers metric units."},
follow_redirects=False,
)
db.expire_all()
assert len(db.scalars(select(Memory)).all()) == 1
# --- A name somebody else owns ------------------------------------------------
def test_a_shared_skill_does_not_take_its_name_from_you(db, client, stranger, stranger_user, owner):
"""`Skill` is unique on (owner_id, name), and `create_base` next door checks
ownership -- this one checks visibility. Share one "weekly-report" with a
team and nobody on that team can make their own, which reads as the name
being reserved instance-wide by whoever got there first."""
theirs = _skill(db, owner, name="weekly-report")
sharing.set_grants(db, theirs, user_ids=[stranger_user.id], group_ids=[])
stranger.post(
"/api/library/skills",
data={
"name": "weekly-report",
"description": "How I write mine.",
"body": "Step one.",
},
follow_redirects=False,
)
db.expire_all()
mine = db.scalars(select(Skill).where(Skill.owner_id == stranger_user.id)).all()
assert [s.name for s in mine] == ["weekly-report"]
+3
View File
@@ -20,6 +20,9 @@ from lembas.services import settings_store
from lembas.services.agent import index as index_service
from lembas.services.agent import ssh as ssh_service
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+232
View File
@@ -0,0 +1,232 @@
"""The upgrade path, which every existing install takes to reach 1.0.0.
There is no Alembic here by design -- hard rule 4, additive-only, with
`db/migrations.py:sync_schema` deriving the change from the models by diffing
them against the live database. That makes the differ the *only* thing standing
between an 0.8.x deployment and a broken one, and until this file it had no test
that exercised it as a migration at all.
`tests/conftest.py` runs `create_all` and *then* `sync_schema`, so the schema it
is handed is always already current: the differ finds nothing missing, does
nothing, and reports success. Every run of the suite proved that a no-op is a
no-op. The two unit tests that did exist covered `_add_column_sql`'s DDL string
and never touched a database.
So these build an **old-shaped database with rows in it** and upgrade it for
real. The shape is not invented: `OLD_TABLES` and `OLD_COLUMNS` are what
actually arrived between `0.8.1` and this release, taken from
`git diff 00ce04a..HEAD -- src/lembas/db/models/`. `test_the_recorded_shape_is_still_real`
is what stops that list rotting into a test that upgrades nothing.
"""
from __future__ import annotations
import pytest
from sqlalchemy import inspect, text
from lembas.db.base import Base
from lembas.db.migrations import ensure_fts, sync_schema
from lembas.db.session import get_engine
# Tables that did not exist at 0.8.1. `sync_schema` has to create them.
OLD_TABLES = ("chunks", "push_subscriptions", "usage")
# Columns added to tables that already existed, and therefore already had rows.
# These are the interesting half: a new *table* is empty by definition, but a
# new *column* has to arrive beside data somebody cares about.
OLD_COLUMNS = (
("ssh_profiles", "resolves_here"),
("chats", "parent_chat_id"),
("chats", "unattended"),
("reports", "unread_notified"),
("groups", "limits_json"),
)
def _rollback_to_0_8_1(engine) -> None:
"""Take a current database back to the shape 0.8.1 left behind.
Backwards rather than forwards because the alternative is importing two
versions of the models into one interpreter, which is not a thing Python
will do -- `lembas.db.models` is already imported and registered on one
`Base.metadata`. Dropping what was added produces the same *shape* the
differ has to repair, which is what is under test.
"""
with engine.begin() as connection:
for table in OLD_TABLES:
connection.execute(text(f"DROP TABLE IF EXISTS {table}"))
for table, column in OLD_COLUMNS:
connection.execute(text(f"ALTER TABLE {table} DROP COLUMN {column}"))
def _columns(engine, table: str) -> set[str]:
return {column["name"] for column in inspect(engine).get_columns(table)}
def test_the_recorded_shape_is_still_real():
"""The rollback above names tables and columns by hand, and a name that has
since been removed or renamed would make it silently upgrade nothing --
a test that passes because it tested an empty set.
"""
tables = Base.metadata.tables
for table in OLD_TABLES:
assert table in tables, f"{table} is no longer a table; fix OLD_TABLES"
for table, column in OLD_COLUMNS:
assert table in tables, table
assert column in tables[table].c, f"{table}.{column} is gone; fix OLD_COLUMNS"
def test_an_0_8_1_database_with_data_upgrades(db):
"""The whole point. Rows written before the upgrade must still be there
afterwards, with their values, and the new columns must exist beside them.
A failure here is somebody's instance not starting after pressing Update,
or -- worse and quieter -- starting with a column that silently reads NULL.
"""
engine = get_engine()
# Data first, while the schema still has the columns the ORM expects.
from lembas.db.models import Chat, Group, User
owner = User(name="Frodo", email="f@shire.test", password_hash="x") # noqa: S106
db.add(owner)
db.commit()
group = Group(name="Fellowship")
db.add(group)
db.commit()
chat = Chat(user_id=owner.id, model_id="mithril", title="A chat from before")
db.add(chat)
db.commit()
chat_id, owner_id, group_id = chat.id, owner.id, group.id
db.close()
_rollback_to_0_8_1(engine)
# Confirm the rollback actually removed things, or the assertions below
# would pass against a database that was never old.
assert "chunks" not in inspect(engine).get_table_names()
assert "unattended" not in _columns(engine, "chats")
changes = sync_schema(engine)
assert changes, "the differ reported nothing to do on an old database"
for table in OLD_TABLES:
assert table in inspect(engine).get_table_names(), table
for table, column in OLD_COLUMNS:
assert column in _columns(engine, table), f"{table}.{column}"
# And the rows are still what they were.
with engine.connect() as connection:
row = connection.execute(
text("SELECT title, model_id, user_id FROM chats WHERE id = :id"),
{"id": chat_id},
).one()
assert row.title == "A chat from before"
assert row.model_id == "mithril"
assert row.user_id == owner_id
assert (
connection.execute(
text("SELECT name FROM groups WHERE id = :id"), {"id": group_id}
).scalar()
== "Fellowship"
)
def test_a_nullable_column_arrives_empty_rather_than_defaulted(db):
"""A nullable column is added with **no** default, so an existing row reads
NULL -- the value the model treats as absent.
An earlier version defaulted every column by type, which meant an added
foreign key arrived as `""` on old rows, and every "is this set?" check
downstream was wrong about them. `parent_chat_id` is exactly that shape: a
chat that predates subagents is not a helper, and `""` would not say so.
"""
engine = get_engine()
from lembas.db.models import Chat, User
owner = User(name="Sam", email="s@shire.test", password_hash="x") # noqa: S106
db.add(owner)
db.commit()
chat = Chat(user_id=owner.id, model_id="m")
db.add(chat)
db.commit()
chat_id = chat.id
db.close()
_rollback_to_0_8_1(engine)
sync_schema(engine)
with engine.connect() as connection:
parent = connection.execute(
text("SELECT parent_chat_id FROM chats WHERE id = :id"), {"id": chat_id}
).scalar()
assert parent is None, "an added foreign key must be absent, not empty"
def test_a_not_null_column_backfills_every_existing_row(db):
"""SQLite refuses to add a NOT NULL column without a default, so one is
derived from the type. `chats.unattended` is a boolean: an old row has to
come back False rather than NULL, or every `if chat.unattended` downstream
reads a null as a value.
"""
engine = get_engine()
from lembas.db.models import Chat, User
owner = User(name="Merry", email="m@shire.test", password_hash="x") # noqa: S106
db.add(owner)
db.commit()
db.add(Chat(user_id=owner.id, model_id="m"))
db.commit()
db.close()
_rollback_to_0_8_1(engine)
sync_schema(engine)
with engine.connect() as connection:
values = connection.execute(text("SELECT unattended FROM chats")).scalars().all()
assert values, "no rows survived the upgrade"
assert all(value in (0, False) for value in values), values
def test_upgrading_twice_changes_nothing_the_second_time(db):
"""It runs on every startup and has to converge. A second pass reporting
work would mean it was re-adding something, which on SQLite is an error that
stops the application booting."""
engine = get_engine()
_rollback_to_0_8_1(engine)
assert sync_schema(engine)
assert sync_schema(engine) == []
def test_the_search_indexes_are_rebuilt_when_missing(db):
"""FTS5 tables are not SQLAlchemy models, so `create_all` cannot make them
and the column differ cannot see them. `ensure_fts` is what converges them,
and it runs at every startup for the same reason the column sync does --
an instance whose indexes were dropped must repair itself rather than
return nothing and call it an empty library.
"""
engine = get_engine()
with engine.begin() as connection:
connection.execute(text("DROP TABLE IF EXISTS documents_fts"))
assert "documents_fts" in ensure_fts(engine)
assert "documents_fts" in inspect(engine).get_table_names()
# Converges: a second call has nothing left to make.
assert ensure_fts(engine) == []
@pytest.mark.parametrize("table", OLD_TABLES)
def test_each_new_table_is_usable_after_the_upgrade(db, table):
"""Created is not the same as correct. A table the differ made but got the
columns wrong on fails at the first insert, which is the first *reply* on an
upgraded instance rather than at startup."""
engine = get_engine()
_rollback_to_0_8_1(engine)
sync_schema(engine)
declared = {column.name for column in Base.metadata.tables[table].c}
assert _columns(engine, table) == declared
+150
View File
@@ -0,0 +1,150 @@
"""One run against a real server on a real socket.
Every other HTTP test in this suite goes through `TestClient`, which is an
in-process ASGI call: no socket, no uvicorn, no HTTP parsing. That covers the
application and covers nothing about the thing `deploy/install.sh` actually
starts. Uvicorn's own behaviour -- how it frames a streaming response, whether
it holds a connection open, what it does on shutdown with a stream still
running -- is what a deployment depends on and what nothing here touched.
Deliberately **one** test file and a handful of assertions. This is a smoke
test: it is here so that "the server starts and serves" is a fact rather than an
inference, not to re-test the application through a slower transport.
Marked `slow` because it binds a port and waits for a process.
"""
from __future__ import annotations
import socket
import subprocess
import sys
import time
import httpx
import pytest
pytestmark = pytest.mark.slow
def _free_port() -> int:
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
return probe.getsockname()[1]
@pytest.fixture
def server(tmp_path):
"""A real `lembas serve`, on a real port, with its own data directory.
Started as a subprocess rather than a thread: the point is the process the
unit file starts, and an in-thread uvicorn shares this interpreter's already
imported settings singleton.
"""
port = _free_port()
env = {
"PATH": "/usr/bin:/bin",
"HOME": str(tmp_path),
"LEMBAS_DATA_DIR": str(tmp_path),
"LEMBAS_HOST": "127.0.0.1",
"LEMBAS_PORT": str(port),
"LEMBAS_SECRET_KEY": "t" * 44,
"LEMBAS_ALLOW_SIGNUP": "true",
}
process = subprocess.Popen(
[sys.executable, "-m", "lembas.cli", "serve"],
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
base = f"http://127.0.0.1:{port}"
try:
deadline = time.monotonic() + 30
while time.monotonic() < deadline:
if process.poll() is not None:
raise AssertionError(f"the server exited: {process.stdout.read()}")
try:
httpx.get(f"{base}/healthz", timeout=1.0)
break
except httpx.TransportError:
time.sleep(0.2)
else: # pragma: no cover - only on a machine that cannot start it
raise AssertionError("the server never answered")
yield base
finally:
process.terminate()
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired: # pragma: no cover
process.kill()
def test_it_starts_and_answers(server):
"""`lembas serve` is what the systemd unit runs, and until now nothing
checked that the command in the unit file works at all."""
response = httpx.get(f"{server}/healthz", timeout=10.0)
assert response.status_code == 200
assert response.json()["status"] == "ok"
def test_a_signed_out_visitor_is_sent_to_the_sign_in_page(server):
"""Over real HTTP, with a real redirect, because the redirect is a header
and headers are the part `TestClient` does not have to get right."""
response = httpx.get(f"{server}/", follow_redirects=False, timeout=10.0)
assert response.status_code in (302, 303, 307)
assert "/auth/login" in response.headers["location"]
def test_the_stylesheets_it_serves_are_the_ones_in_the_tree(server):
"""The static mount, which is configuration rather than code and therefore
fails at deploy time rather than in a unit test."""
response = httpx.get(f"{server}/static/css/app.css", timeout=10.0)
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/css")
assert ".sidebar" in response.text
def test_a_fresh_instance_offers_a_way_in(server):
"""The first thing a new deployment does. On an empty database `/auth/login`
redirects to registration, because an instance with no accounts and a sign-in
form is a door with no key -- and the first account made becomes the
administrator.
Worth having over real HTTP: this is the exact path somebody walks thirty
seconds after `install.sh` finishes, and it is a chain of redirects, which is
the part that is headers rather than code.
"""
response = httpx.get(f"{server}/auth/login", follow_redirects=True, timeout=10.0)
assert response.status_code == 200
assert "text/html" in response.headers["content-type"]
assert "register" in str(response.url) or "register" in response.text.lower()
def test_an_account_can_be_created_and_used_over_real_http(server):
"""Registration, the session cookie, and a page that needs it -- end to end
through uvicorn. A cookie is `Set-Cookie` plus the browser sending it back,
and both halves are transport rather than application: `TestClient` has its
own cookie jar and would pass whatever the header said.
"""
with httpx.Client(base_url=server, timeout=15.0, follow_redirects=True) as client:
made = client.post(
"/auth/register",
data={
"name": "Frodo",
"email": "f@shire.test",
"password": "speak-friend-and-enter",
"confirm": "speak-friend-and-enter",
},
)
assert made.status_code == 200, made.text[:400]
assert client.cookies, "no session cookie came back"
# And the cookie actually admits us to something that requires one.
chat = client.get("/chat")
assert chat.status_code == 200
assert "auth/login" not in str(chat.url)
+105
View File
@@ -0,0 +1,105 @@
"""The wire format every reply travels over, which had no test of its own.
`services/sse.py` is twenty lines and carries all of it. That is exactly the
kind of module that never gets one: too small to look risky, and load-bearing
enough that a subtle mistake is a stream which works until a model emits a
newline -- which is to say, until the first code block.
The events are parsed back the way the browser's `EventSource` does rather than
compared against an expected string. A test asserting the literal bytes passes
on a format that is wrong in the same way it was written.
"""
from __future__ import annotations
import pytest
from lembas.services.sse import KEEPALIVE, event
def parse(frame: str) -> tuple[str, str]:
"""Read one frame the way an `EventSource` does.
Field lines are `name: value`, the space after the colon is stripped, and
several `data:` lines in one event are joined with a newline. That last rule
is the whole reason `event()` exists.
"""
name = ""
data: list[str] = []
for line in frame.split("\n"):
if line.startswith("event:"):
name = line[len("event:") :].lstrip(" ")
elif line.startswith("data:"):
data.append(line[len("data:") :].lstrip(" "))
return name, "\n".join(data)
def test_a_plain_payload_survives_the_round_trip():
assert parse(event("render", "hello")) == ("render", "hello")
def test_a_frame_ends_with_a_blank_line():
"""The blank line is what tells the browser the event is over. Without it
nothing is dispatched at all and the stream simply appears to hang."""
assert event("render", "hello").endswith("\n\n")
@pytest.mark.parametrize(
"payload",
[
"line one\nline two",
"```python\nprint('hi')\n```",
"\nleading",
"trailing\n",
"two\n\nblank",
"<p>markup</p>\n<p>and more</p>",
],
)
def test_newlines_survive_because_they_are_split_across_data_lines(payload):
"""The failure this module exists to prevent. A raw newline inside one
`data:` line ends that line, so everything after it is dropped -- the event
arrives truncated, with no error anywhere, the first time a model writes a
code block."""
name, back = parse(event("render", payload))
assert name == "render"
assert back == payload
def test_every_line_of_a_multiline_payload_is_its_own_field():
frame = event("steps", "a\nb\nc")
assert frame.count("data: ") == 3
assert "data: a\ndata: b\ndata: c\n" in frame
def test_an_empty_payload_is_still_a_frame():
"""Several frames must be able to blank themselves -- `metrics`, `status`,
`ask`, `reasoning`, `think` and `render` are sent on every version bump
*including* empty, because each has to be able to clear. An approval card
that survived being answered would be a button you could press twice."""
name, back = parse(event("ask", ""))
assert name == "ask"
assert back == ""
assert event("ask", "").endswith("\n\n")
def test_a_payload_that_looks_like_a_field_is_not_read_as_one():
"""Model output is untrusted, and `event: done` inside a reply must stay
text. It does because every line is prefixed -- but the reason is worth
pinning, since the day it is not, a model can end its own stream."""
hostile = "event: done\ndata: {}"
name, back = parse(event("render", hostile))
assert name == "render", "the payload took over the event name"
assert back == hostile
def test_the_keepalive_is_a_comment_and_not_an_event():
"""It exists to stop a proxy killing an idle stream while a model thinks.
A comment line does that without dispatching anything; an event would reach
the page and be swapped into it."""
assert KEEPALIVE.startswith(":")
assert KEEPALIVE.endswith("\n\n")
assert parse(KEEPALIVE) == ("", "")
+3
View File
@@ -15,6 +15,9 @@ from lembas.services.agent import terminal as terminal_service
from lembas.services.agent.base import ExecError
from lembas.services.agent.terminal import Session
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+3
View File
@@ -14,6 +14,9 @@ import pytest
from lembas.services.agent import capture, shell_marks
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+268
View File
@@ -0,0 +1,268 @@
"""The terminal panel, checked without a runtime.
There is no JavaScript test runner here and hard rule 1 keeps Node out of the
project, so the behaviour is driven by hand under a DOM stub before committing.
Almost everything this file gets wrong is silent: `fit()` measures `offsetWidth`,
which is 0 inside a `[hidden]` ancestor, so fitting while closed does nothing at
all and leaves an 80-column shell in a 34rem panel; a window `resize` event does
not fire when the sidebar is toggled, which is the commonest way the panel
changes size; and a handshake against the app root fails into `onerror` and
blames the proxy for a connection the reader simply has not chosen yet.
What the suite can pin are the guards those three depend on, and the properties
around them whose absence looks exactly like working.
"""
from __future__ import annotations
from pathlib import Path
import lembas
from .conftest import js_code, js_function, js_says, script
SOURCE = script("terminal.js")
CODE = js_code(SOURCE)
TEMPLATES = Path(lembas.__file__).parent / "web/templates"
def _after(text: str, marker: str, span: int = 500) -> str:
"""The code just after a marker -- for a handler with no name to ask for."""
return text[text.index(marker) :][:span]
# --- Connecting ---------------------------------------------------------------
def test_connect_refuses_a_panel_with_no_target():
"""With no connection chosen `data-url` is empty, and the URL below then
becomes `ws://host?cols=80&rows=24` -- a WebSocket handshake against the
application root. That fails into `onerror`, whose message guesses at a
proxy not passing upgrades through: a confident, wrong explanation for
something the reader has simply not chosen yet.
The guard has to come before the socket, which is the whole assertion.
"""
connect = js_function(SOURCE, "connect")
assert "!panel.dataset.url" in connect, "the empty-target guard is gone"
assert connect.index("!panel.dataset.url") < connect.index("new WebSocket"), (
"a panel with no target still opens a socket"
)
# And says so, rather than leaving the panel silent.
assert "say(" in connect[: connect.index("new WebSocket")]
def test_the_socket_url_is_built_from_the_panel_s_own_path():
"""The path is server-rendered onto the panel, and `draft.js` rewrites it
before a chat exists. Assembling one here from a chat id would be a second
place for it to be got subtly differently, and a wrong one connects to
another machine's shell rather than failing."""
connect = js_function(SOURCE, "connect")
assert "location.host + panel.dataset.url" in connect
# A page served over TLS must not open a plain socket; browsers block it,
# which reads as the terminal never connecting.
assert js_says(connect, 'location.protocol === "https:"', '"wss://"', '"ws://"')
def test_a_deliberate_close_is_flagged_before_it_is_made():
"""Both reconnect paths close a socket on purpose -- the server disconnecting
a window that stopped reading, and the composer changing which machine the
panel points at. The flag is what stops "Disconnected. Close and reopen to
reconnect." being reported for a connection nobody lost, so it has to be set
before the close, not after it."""
for marker in ('payload.t === "behind"', "repointTerminal = function"):
window = _after(CODE, marker)
assert window.index("closedOnPurpose = true") < window.index("socket.close()"), marker
def test_a_malformed_control_frame_is_ignored():
"""Text frames carry JSON and PTY bytes do not, and the two share one
`onmessage`. An exception here kills the handler for the rest of the
session, so the shell keeps running and the panel stops showing it."""
control = js_function(SOURCE, "control")
assert "try {" in control
assert control.index("JSON.parse") < control.index("catch")
def test_bytes_go_through_to_xterm_undecoded():
"""xterm's decoder is stateful across calls, so a multi-byte character split
across two frames still lands correctly -- which is exactly why the server
never decodes either. Decoding here would break the same character in the
other direction."""
assert js_says(CODE, "opened.binaryType", '"arraybuffer"')
assert js_says(CODE, "term.write(", "new Uint8Array(event.data)")
assert "TextDecoder" not in CODE
# --- Sizing -------------------------------------------------------------------
def test_a_hidden_panel_is_never_fitted():
"""`fit()` measures `offsetWidth`, which is 0 inside a `[hidden]` ancestor.
Fitting while closed therefore succeeds, silently, at nothing -- and leaves
an 80-column terminal in a panel that is not 80 columns wide, which reads as
xterm being broken rather than as a call made too early."""
refit = js_function(SOURCE, "refit")
assert refit.index("visible()") < refit.index("fit.fit()")
visible = js_function(SOURCE, "visible")
assert 'hasAttribute("hidden")' in visible
assert "offsetWidth > 0" in visible, "an ancestor's `hidden` measures as zero, not as hidden"
def test_opening_waits_a_frame_before_fitting():
"""The panel has just had `hidden` removed and has no measured width yet, so
a fit in the same turn is the silent no-op above."""
opener = js_function(SOURCE, "open")
assert opener.index("requestAnimationFrame") < opener.index("refit()")
def test_the_panel_watches_its_own_size():
"""A window `resize` event does not fire when the sidebar is toggled or a
panel opens beside this one, and those are by far the commonest ways this
panel changes size. Without the observer the terminal keeps the width it had
when it was opened and wraps every line at the wrong column."""
build = js_function(SOURCE, "build")
assert "ResizeObserver" in build
assert "observe(panel)" in build
def test_a_resize_is_told_to_the_far_side():
"""Fitting changes what this end draws; the PTY on the other end keeps its
old window size until it is told, so a full-screen program like `vim` or
`htop` paints to a geometry that is no longer there."""
refit = js_function(SOURCE, "refit")
assert js_says(refit, 't: "resize"', "cols: term.cols", "rows: term.rows")
def test_closing_the_panel_leaves_the_session_alone():
"""`write()` is queued internally, so disposing mid-output drops it; keeping
the object is what makes reopening instant; and the shell on the far side
outlives this panel by design. Closing must therefore do nothing at all."""
toggle = CODE[CODE.index('"lembas:toggle"') : CODE.index('panel.addEventListener("click"')]
assert "open()" in toggle
assert "dispose" not in toggle
assert ".close(" not in toggle
assert "dispose" not in CODE, "nothing here may dispose the terminal"
# --- The theme ----------------------------------------------------------------
def test_the_theme_is_pushed_in_when_it_changes():
"""xterm holds colours as values, not as variables, so it never notices a
`data-theme` swap. Without this, switching to `shire` leaves a black
rectangle in a light interface."""
assert '"lembas:theme"' in CODE
assert js_says(CODE, "term.options.theme", "readTheme()")
def test_a_colour_slot_with_no_token_is_left_off_entirely():
"""xterm treats `undefined` as a colour and renders it black, so a missing
ANSI token would not fall back -- it would paint that slot black on a light
theme, which is one unreadable colour rather than a visible failure."""
read = js_function(SOURCE, "readTheme")
assert js_says(read, "if (value)", "theme[name]", "value")
# --- Handing a command to the chat --------------------------------------------
def test_a_command_line_is_never_rendered_as_markup():
"""Everything shown here came off somebody's machine: a command line, a
working directory, a message from the far side. Hard rule 6 again."""
assert "innerHTML" not in CODE
assert "textContent" in js_function(SOURCE, "say")
assert "textContent" in js_function(SOURCE, "showLast")
def test_the_button_puts_it_in_the_box_rather_than_sending_it():
"""What a machine printed is exactly the sort of text somebody should read
before a model does, and the box is where that happens. Only the `auto:
send` mode skips that, and only because it was chosen."""
send = js_function(SOURCE, "sendToChat")
assert "intoComposer" in send
assert "sendStraightToChat" not in send
def test_writing_into_the_composer_repaints_its_mirror():
"""The mirror is only repainted on `input` and after a swap, and this is
neither -- so without the call the highlighting sits over the previous text
until the next keystroke, shifted by however much was just inserted."""
into = js_function(SOURCE, "intoComposer")
assert "paintComposer" in into
assert "autosize" in into
def test_the_automatic_path_never_steals_focus():
"""It fires while somebody is typing in the terminal, and yanking the caret
out of a shell mid-command is the sort of thing that gets a feature switched
off for good."""
into = js_function(SOURCE, "intoComposer")
assert js_says(into, "!quiet", "input.focus()")
# The `command` frame is the automatic path, and it is the quiet one.
assert "capture(true)" in _after(CODE, 'payload.t === "command"')
def test_a_screen_scrape_says_that_is_what_it_is():
"""Without shell integration this is the last forty rows as they appeared,
wraps and all. Presenting that as a command and its output would put a
precise-looking block in front of a model that is not one."""
capture = js_function(SOURCE, "capture")
assert "scraped()" in capture
assert "say(" in capture[capture.index("scraped()") :]
def test_auto_is_disabled_rather_than_degraded():
"""Forty arbitrary lines attached to every message is worse than nothing
attached at all, so the control is switched off where it cannot work -- and
switched back off if it was already on when the markers went away."""
apply_ = js_function(SOURCE, "applyIntegration")
assert 'integration === "live"' in apply_
assert js_says(apply_, "auto.disabled", "!usable")
assert js_says(apply_, "!usable", 'autoMode !== "off"', 'setAuto("off")')
def test_auto_is_not_remembered_between_page_loads():
"""A switch that forwards every command you run to a model is not something
to inherit from last week."""
assert "localStorage" not in CODE
assert "sessionStorage" not in CODE
# --- Where it is loaded -------------------------------------------------------
def test_xterm_is_loaded_only_where_a_shell_can_be_opened():
"""xterm is 280KB, more than everything else vendored here together. Hard
rule 1 allows it exactly once, and only on a chat that can open a terminal
-- an unconditional script tag would put it on every page in the
application."""
page = (TEMPLATES / "chat/index.html").read_text(encoding="utf-8")
guarded = ""
at = 0
while (at := page.find("{% if terminal_enabled %}", at)) != -1:
guarded += page[at : page.index("{% endif %}", at)]
at += 1
for asset in ("vendor/xterm.js", "vendor/xterm-addon-fit.js", "js/terminal.js"):
assert page.count(asset) == 1, f"{asset} is referenced more than once"
assert asset in guarded, f"{asset} is loaded outside the terminal_enabled guard"
def test_a_handler_only_touches_the_connection_it_belongs_to():
"""`close()` queues its event rather than firing it, and both reconnect
paths close the old socket and immediately open a new one. So the old
socket's `close` arrives *after* the new one is assigned -- and a handler
reading the module-level `socket` reached past its own connection into the
live one and set it to null.
Output kept arriving, because `onmessage` is bound to the object. But
`send`, `term.onData` and the paste handler all gate on
`socket && readyState === OPEN`, so every keystroke was dropped and no
resize was ever sent again -- on a panel that looked perfectly healthy. And
`closedOnPurpose` had been cleared for the new connection, so it announced
"Disconnected" about a shell that had just reconnected successfully.
"""
code = js_code(script("terminal.js"))
assert "var opened = new WebSocket(url)" in code
# Every handler compares itself against the current connection first.
for handler in ("onmessage", "onclose", "onerror"):
body = code.split(f"opened.{handler} = function", 1)[1]
guard = body[: body.index("{", body.index(")")) + 200]
assert "socket !== opened" in guard, handler
assert "socket.onclose = function" not in code, "a handler bound to the shared variable"
+3
View File
@@ -18,6 +18,9 @@ from lembas.services import settings_store
from lembas.services.agent import policy
from lembas.services.agent import terminal as terminal_service
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
asyncssh = pytest.importorskip("asyncssh")
+30
View File
@@ -298,3 +298,33 @@ def test_turning_notifications_off_also_drops_the_registration():
service for something the reader has switched off."""
assert "/api/push/unsubscribe" in SOURCE
assert "pushManager" in SOURCE
def test_no_page_loads_a_script_that_the_base_template_already_loads():
"""`/messages` listed `composer.js` and `commands.js` in its `scripts`
block, and `base.html` already loads both on every page -- so that screen
ran each of them twice.
Each is an IIFE with its own state, and `stopPropagation()` does not stop a
second listener already bound to the same node. Two composer menus stacked
on each other; one Enter on a highlighted `/` item ran the command twice, so
`/help` opened two dialogs and `/image` posted the message twice; an `@`
mention attached its file twice; and `Alt+B`, `Alt+E`, `Alt+T` and `Alt+I`
each toggled their panel twice, which is to say did nothing at all.
None of that looks like a script loaded twice, which is why this is a sweep
over every template rather than a note about one of them.
"""
import re
from pathlib import Path
templates = Path(__file__).resolve().parents[1] / "src/lembas/web/templates"
pattern = re.compile(r"path='js/([a-z_]+\.js)'")
always = set(pattern.findall((templates / "base.html").read_text()))
assert always, "base.html stopped loading any script; this test is now blind"
for template in templates.rglob("*.html"):
if template.name == "base.html":
continue
again = set(pattern.findall(template.read_text())) & always
assert not again, f"{template.name} re-loads {sorted(again)}, which base.html has"
+2
View File
@@ -13,6 +13,8 @@ import pytest
from lembas.config import settings
from lembas.services import updates
# Stands up something real -- see the `slow` marker in pyproject.toml.
pytestmark = pytest.mark.slow
@pytest.fixture
def tagged():