diff --git a/CLAUDE.md b/CLAUDE.md index 1f968ce..cf5fd58 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,14 +12,15 @@ no JavaScript build step. ```bash . .venv/bin/activate -pip install -e ".[dev,search]" # `search` adds ddgs for DuckDuckGo +pip install -e ".[dev,search,ssh]" # `search` adds ddgs for DuckDuckGo, + # `ssh` adds asyncssh for agent chats lembas serve # http://127.0.0.1:8080 lembas info # paths + counts, useful when confused lembas secret-key # generate LEMBAS_SECRET_KEY lembas create-admin # create or promote an admin -pytest # 698 tests, ~40s +pytest # 837 tests, ~50s # PLAN.md tracks what is and is not built ruff check . # lint (line length 100) python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons; @@ -83,6 +84,8 @@ src/lembas/ admin_prompts.py the prompt fragment editor and its preview admin_suggestions.py the cards offered on the new-chat screen admin_tools.py custom HTTP tools and MCP servers + admin_agents.py whether agent chats exist, and what they may spend + agents.py SSH connections, kept by the people who own them audio.py transcribe, speak, voice discovery library.py knowledge, notes, skills pages; memory CRUD files.py upload, serve, remove attachments @@ -98,6 +101,7 @@ src/lembas/ search/ ddgs, SearXNG and Firecrawl behind one shape library/ documents, notes, memories, skills, FTS mcp/ remote MCP servers: framing, transport, rows to tools + agent/ agent chats: the mode table, SSH, and the four tools audio.py OpenAI-shaped /v1/audio/* client fetch.py URL retrieval, HTML to text, the SSRF guard sharing.py one visibility rule for every library store @@ -110,6 +114,7 @@ src/lembas/ tools.py tool registry, schemas, streamed-call reassembly custom_tools.py the admin-defined HTTP tool runner tool_access.py who may be offered which admin-defined tool + interaction.py pausing a reply to ask the reader something chat.py request building, endpoint resolution, titles markdown.py markdown-it + pygments + nh3 crypto.py Fernet encrypt/decrypt/mask @@ -291,6 +296,81 @@ along with the reply, and a reload starts the turn afresh -- the model asks again. That is consistent with "a restart abandons replies in flight", but it means an approval is not a durable record of consent. +**A chat's kind and connection are fixed at creation; only the mode moves.** +`Chat.kind`, `ssh_profile_id` and `project_dir` are chosen on the new-chat screen +and refused by `update_chat` thereafter with a 409 — a transcript whose earlier +turns ran somewhere else is not one conversation. `agent_mode` is the exception +and changes freely: it decides what gets asked about, not what the conversation +is. The mode is read **once per reply**, so switching to Auto mid-reply cannot +retroactively approve what is already queued. + +**The mode is enforced in the loop, never in the prompt.** `_authorise` consults +`agent/policy.py:decide()` server-side, keyed on each `ToolDef.risk`. A model is +*told* which mode it is in so it behaves sensibly, but everything it reads — a +web page, a README, the output of the last command — is untrusted, and a rule +living only in a system message is one a poisoned file can argue with. Within an +agent chat **every** call goes through the table, including the built-ins: +`notes_edit` writes, and Plan mode meaning "look but do not touch" has to mean +that too. + +**An approved call needs telling.** Every agent runner re-checks the mode as a +backstop, so a call arriving by a path that skipped `_authorise` cannot walk +past it. That backstop refused the very thing a person had just approved — the +mode says "ask", and asking is exactly what happened. `AgentContext.approved` is +threaded per call on a *copy* of the context, because a round runs its calls +together and only some of them were allowed. + +**`registry(db)` must know every tool that can be offered, agent tools +included.** It maps an offered tool *name* back to a family, which is how the +harness decides that `tool.agent` applies. They are listed there unbound to any +chat. Without them `shell_run` resolves to no family, and an agent chat is told +nothing about the machine it is working on. The identical omission cost custom +tools their guidance once already; there is a test for it now. + +**A tool description is schema; the harness is where "where" lives.** +Descriptions are sent verbatim and are deliberately not editable, so they state +facts about the runner. Which machine, which directory and which mode belong to +*this chat* and live in the `tool.agent` fragment, where they can change without +the schema shifting under a model mid-conversation. + +**Each command is a fresh shell.** Connections are per call, so `cd build` +followed by `make` fails silently — `cwd` is a first-class parameter reaching the +executor, never spliced into the command string. This is the likeliest single +cause of "the agent seems stupid", and the harness says it out loud. So does the +other one: on a Debian-derived host `apt-get install` reports the package missing +until `apt-get update` has run. + +**Files never go through a shell.** The SSH exec protocol carries one command +*string* that the far side parses, with no argv form at all, so a model-supplied +path in a command line is unavoidably a quoting problem. `file_read`/`file_write` +/`file_list` use SFTP, where a path is a path. + +**asyncssh's defaults are wrong here, all four of them.** Every LLeMbas user +shares one unix account, so `known_hosts` unset reads a *shared* trust store +(and `None` disables checking entirely), `client_keys` unset loads whatever is in +`~/.ssh`, `config` unset lets a `ProxyCommand` redirect the connection, and +`agent_path` unset uses `$SSH_AUTH_SOCK`. All four are passed explicitly on every +connection, and the test that proves it needs no server. + +**A pinned host key belongs to a host and a port.** Moving a profile forgets it +deliberately. `capture_host_key` completes the key exchange and stops, so a host +that has not been accepted is never offered a username, let alone a credential — +which is what makes accepting a fingerprint from a button safe. + +**A plan ends the turn, but not mid-sentence.** `plan_submit` is offered in Plan +mode only, and the round after it runs with the tools withdrawn: the model gets +to say what it proposed, and cannot spend three more rounds changing its mind +about a plan somebody is being asked to approve. Carrying it out switches to +**Edit, never Auto**, and the plan goes back quoted and attributed rather than +stated — text that came out of a file the model read must not arrive wearing the +reader's authority. + +**Rewind rewinds the transcript, not the machine.** Editing or regenerating in an +agent chat stamps `Chat.rewound_at` and the harness warns that files from steps +no longer in the transcript are still there. Nothing tries to undo them: the +project directory is somebody's real working tree, and deleting their work to +match would be far worse than the inconsistency. + **Unread is polled, not pushed.** A browser on another chat has no connection to the one that finished. `/api/chats/unread` returns out-of-band dot spans and an `HX-Trigger` for the toast; `unread_notified` stops the same arrival being @@ -620,18 +700,37 @@ notes describe the machine. ## Not built yet -Agentic execution (local subprocess and SSH connection profiles), image -generation. A nav entry marks where each one goes. The tool loop in -`services/generation.py` is what they plug into — a new tool is a `ToolDef` -reaching `tools.resolve_tools()` plus a permission and a capability flag, not a -new code path. Its guidance is the same shape: a `prompts.register_source` -yielding one `Fragment` per row puts it in the harness, on the admin page and in -the preview without touching the assembler, the save handler or a template. -Custom HTTP tools and MCP servers are built and are the worked example of both. +Image generation, and a nav entry marks where it goes. The tool loop in +`services/generation.py` is what a new capability plugs into — a tool is a +`ToolDef` reaching `tools.resolve_tools()` plus a permission and a capability +flag, not a new code path. Its guidance is the same shape: a +`prompts.register_source` yielding one `Fragment` per row puts it in the harness, +on the admin page and in the preview without touching the assembler, the save +handler or a template. Custom HTTP tools, MCP servers and agent chats are the +three worked examples. -**Local MCP is deliberately absent.** Only remote servers over streamable HTTP. -Spawning `npx` is the agentic-execution feature, which wants a confirmation model -before it does anything; a URL is a different act with a different blast radius. +**Nothing executes on this machine, and that is the design.** Agent chats run +their commands on a host reached over SSH. A local sandbox was designed in +detail — bubblewrap, a masked data directory, a curated bind list — and dropped, +because every hard problem in it came from running on the machine that holds the +database and the encryption key: the service user cannot traverse `/home`, +granting it needs ACLs, `RLIMIT_NPROC` is counted per *uid* so a fork bomb +starves the server too, `--size` applies only to tmpfs so there is no disk quota, +and a bind list is a standing invitation to widen until the sandbox is +decoration. Over SSH, isolation is somebody's considered choice of host, using +tools far better at it than anything that could be built here — and it is the +only version that is honestly multi-user. + +Two consequences worth stating plainly. **The security of an agent chat is the +security of the host behind its profile**, and nothing here can tell a throwaway +container from a production server. And there is no equivalent of the +`network: False` switch the local sandbox would have had, because the network +belongs to the far side — so an instruction injected through something the model +read can, in principle, be carried out from there. + +**Local MCP is absent for the same reason.** Only remote servers over streamable +HTTP. Spawning `npx` would be a subprocess on this machine, which is the thing +that is deliberately not done. **Unknown is not zero.** `Model.context_length` of 0 means nobody has said how big the window is, which is different from "small". The context percentage is diff --git a/PLAN.md b/PLAN.md index 3a3d576..28b790d 100644 --- a/PLAN.md +++ b/PLAN.md @@ -5,8 +5,9 @@ that would be expensive to revisit. Kept current as work lands; the detail of *how* things work lives in [`CLAUDE.md`](CLAUDE.md). **Status:** usable daily. Streaming chat, attachments, reasoning, tool calling -with web search, a knowledge library, notes, memory and skills, speech in and -out, users and groups, model administration, installable as an app. 437 tests, +with web search, custom HTTP tools and MCP servers, agent chats that work on a +machine over SSH, a knowledge library, notes, memory and skills, speech in and +out, users and groups, model administration, installable as an app. 837 tests, `ruff` clean. --- @@ -66,6 +67,9 @@ be a different project, not a refactor. the next turn, for the same reasons reasoning is not - [x] A round's calls run together, and the reply says which tool is running — a remote tool taking seconds with nothing streaming looks like a hang +- [x] **A reply can stop and ask you something** — one or more questions on one + card, with answers to pick from and a box to write your own, answered + together. The same mechanism carries command approvals - [x] **Custom HTTP tools** — an administrator describes one call: a JSON Schema, a URL template, headers, an encrypted secret and how to read the answer. Arguments may fill a hole but never move the target: the scheme and host @@ -78,8 +82,36 @@ be a different project, not a refactor. a model as instructions - [x] Both gated like the built-ins — a model capability, a permission — and restrictable to groups, with guidance of their own on `/admin/prompts` -- [x] Local MCP over stdio is deliberately absent: spawning a subprocess is the - agentic-execution feature and wants a confirmation model first +- [x] Local MCP over stdio is deliberately absent: spawning a subprocess would + run on this machine, which nothing here does + +### Agent chats +- [x] A chat is a **Chat** or an **Agent**, chosen when it starts and fixed + thereafter — a transcript whose earlier turns ran somewhere else is not + one conversation. Knowledge, memories and skills are shared across both +- [x] **Nothing runs on the LLeMbas host.** Commands go to a machine reached + over SSH, so containment is somebody's considered choice of host — a + container built for the job — rather than a sandbox built here. A local + one was designed in detail and dropped; see CLAUDE.md for why +- [x] **SSH connections are user-owned**, like notes. An administrator decides + only whether the feature exists at all +- [x] Trust on first use, made explicit: adding a host does not connect to it, + **Check** shows its fingerprint with nothing sent, and only accepting + pins it. A host that later answers with a different key is refused +- [x] Four modes as a table over what each tool does to the world — + **Manual** asks about everything, **Edit** writes freely but asks before + commands, **Auto** asks about nothing, **Plan** reads freely and changes + nothing. Switchable at any time; read once per reply +- [x] Enforced in the generation loop, not in the prompt: a rule a model is + merely told is one a poisoned file can argue with +- [x] A deny list beats **Auto**; an allow list cannot be matched by a command + containing anything that joins two commands together +- [x] `shell_run`, `file_read`, `file_write`, `file_list` — files over SFTP, + never through a shell, because the SSH exec protocol has no argv form +- [x] **Plan mode ends with a plan** you can carry out with one button, which + switches to Edit and sends it back quoted rather than as an instruction +- [x] Per-reply budgets on steps, wall clock and output, with time spent + waiting for you subtracted ### The library - [x] **Knowledge bases** — documents, images and saved web pages, grouped into @@ -171,15 +203,6 @@ be a different project, not a refactor. In the order they are likely to be worth doing. -### Agentic execution -Two modes, as originally specified: -- **local** — subprocess on the machine LLeMbas runs on -- **remote** — SSH connection profiles, with `shell.run` / `fs.read` / `fs.write` - -Needs a confirmation model before it does anything. Note that the systemd unit -is deliberately only `ProtectSystem=full` rather than `strict` **because** of -this — revisit the hardening when the real filesystem needs are known. - ### Image generation Left until last from the start, as it needs heavy customisation. ComfyUI is already running on this machine and is the obvious first target. diff --git a/README.md b/README.md index dba0286..6b89900 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,17 @@ runtime. Clone it, `pip install -e .`, run it. template, a secret) and a model can make it. Or add an **MCP server** by URL and its tools appear beside the built-in ones. Both restrictable to groups, and neither can be pointed at your own network unless you say so +- **Agent chats** — start a chat as an *Agent* instead, pointed at one of your + own SSH connections and a directory on it, and a model can read files, write + files and run commands **there**. Nothing ever runs on the machine LLeMbas + itself is on. What it may do without asking is a mode you set and can change + mid-conversation: *Manual* shows you everything first, *Edit* writes freely + but asks before commands, *Auto* asks about nothing, and *Plan* reads freely, + changes nothing, and finishes by proposing steps you can carry out with one + button. Adding a host shows you its fingerprint before anything is sent to it +- **It can ask you things** — a model that needs a decision can stop and put a + few questions on one card, with answers to pick from and a box to write your + own. In any chat, not only an agent one - **Speech in and out** — dictate a message and have replies read aloud, against any OpenAI-compatible audio endpoint (whisper.cpp, Speaches, Kokoro…). Each person picks their own voice @@ -83,8 +94,7 @@ runtime. Clone it, `pip install -e .`, run it. **Planned** -Agentic execution (local and over SSH) · image generation · OCR for scanned -PDFs · semantic search in the library. +Image generation · OCR for scanned PDFs · semantic search in the library. See [PLAN.md](PLAN.md) for what is built, what is not, and why. @@ -95,7 +105,8 @@ git clone https://git.houmeres.sk/Houmeres/LLeMbas.git cd LLeMbas python -m venv .venv && . .venv/bin/activate -pip install -e ".[dev,search]" # `search` adds DuckDuckGo; drop it if unwanted +pip install -e ".[dev,search,ssh]" # search: DuckDuckGo. ssh: agent chats. + # Drop either if you do not want it cp .env.example .env lembas secret-key # paste the result into LEMBAS_SECRET_KEY @@ -141,6 +152,48 @@ Recorded audio is passed straight through and never written to disk. > The microphone needs HTTPS or localhost. Browsers do not grant it over plain > HTTP, so a LAN install without TLS will not offer dictation. +### Agent chats + +**Admin → Agents** to turn the feature on, then **Connections** in the sidebar +to add a machine. Three things have to line up before an agent chat can start: +the feature enabled, the *Run commands* permission, and a model flagged **Agent +execution**. All three are off by default, on purpose. + +Nothing an agent does runs on the machine LLeMbas is on. Commands go to a host +you name over SSH, which means **the containment is that host** — a container +built for the job is a very different thing from a key to a server you care +about, and LLeMbas cannot tell them apart. A throwaway container is the intended +shape: + +```bash +docker run -d --name agent-box -p 127.0.0.1:2222:22 +``` + +Adding a connection does not connect to it. **Check** shows you the host's +fingerprint with nothing sent — not your username, not your key — and only +accepting pins it. If that host later answers with a different key, it is +refused rather than quietly trusted. + +Then start a chat with the **Agent** toggle, pick the connection and a +directory, and choose a mode. The mode is in the chat header and changes at any +time: + +| | Reads | Writes files | Runs commands | +|---|---|---|---| +| **Manual** | asks | asks | asks | +| **Edit** | free | free | asks | +| **Auto** | free | free | free | +| **Plan** | free | asks | asks | + +The mode is enforced in the reply loop, not written into the prompt: everything +a model reads — a web page, a README, the last command's output — is untrusted, +and a rule that lives only in a system message is one a poisoned file can argue +with. In **Auto**, nothing stands between that and a command running. + +*Plan* finishes by proposing steps, with a button that carries them out — which +switches to *Edit*, never *Auto*, because the plan was written under a mode +where every command still asked. + ### The library **Sidebar → Library**, and **Settings → Memory**. Nothing is on by default for a