PWA, one send/stop button, audio in and out, web search as a tool

Four pieces of work.

**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.

**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.

**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.

**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.

Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.

Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.

338 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 17:56:50 +02:00
parent de178837b8
commit 7456525d19
63 changed files with 4597 additions and 140 deletions
+72 -12
View File
@@ -12,17 +12,18 @@ no JavaScript build step.
```bash
. .venv/bin/activate
pip install -e ".[dev]"
pip install -e ".[dev,search]" # `search` adds ddgs for DuckDuckGo
lembas serve # http://127.0.0.1:8080
lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 230 tests, ~9s
pytest # 338 tests, ~16s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate all SVG artwork
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
# needs fonttools and cairosvg)
python scripts/fetch_vendor.py # verify vendored JS against the lockfile
```
@@ -77,8 +78,11 @@ src/lembas/
admin.py connections + instance settings
admin_models.py model ordering, defaults, images, access
admin_users.py users, groups, permissions
admin_audio.py speech-to-text and text-to-speech endpoints
admin_search.py web search provider and credentials
audio.py transcribe, speak, voice discovery
files.py upload, serve, remove attachments
preferences.py per-user theme, default model, password
preferences.py per-user theme, default model, password, audio
db/
base.py Base, UUID/Timestamp mixins
session.py engine, SQLite pragmas, init_db, session_scope
@@ -87,6 +91,9 @@ src/lembas/
security/ passwords (argon2), sessions, permissions
services/
llm/openai_client.py httpx streaming + model discovery
search/ ddgs, SearXNG and Firecrawl behind one shape
audio.py OpenAI-shaped /v1/audio/* client
tools.py tool registry, schemas, streamed-call reassembly
chat.py request building, endpoint resolution, titles
markdown.py markdown-it + pygments + nh3
crypto.py Fernet encrypt/decrypt/mask
@@ -98,8 +105,8 @@ src/lembas/
web/
templating.py render() -- always use this, not TemplateResponse
templates/ Jinja
static/ css, js, vendor, img
assets/ SVG masters (generated)
static/ css, js, vendor, img, sw.js
assets/ SVG masters and PWA icons (generated)
deploy/ systemd unit, nginx vhost, install/update scripts
```
@@ -262,6 +269,59 @@ which is losing.
plain dict is not detected. The columns use `MutableDict` (`db/types.py`), but
the safe habit is `obj.field = {**obj.field, "k": v}`.
**`[hidden]` needs `!important`.** The browser's rule is `[hidden] { display:
none }`, which any class setting `display` outranks — and `.btn` is
`display: inline-flex`. That is not theoretical: it is why the old Stop button,
created and then `hidden = true`, sat permanently beside Send. `app.css` forces
the attribute to win. Anything toggled with `hidden` depends on that line.
**Send and Stop are one button.** `chat/_composer.html` renders both icons and
`ui.js` flips `data-composer-action` plus `type` (`submit``button`) when a
message in the thread is still streaming. Do not add a second button back.
**The tool loop is inside one generation.** `services/generation.py:_run()` runs
up to `tools_service.MAX_ROUNDS` request rounds for a single reply: stream,
accumulate tool calls, run them, append the results, ask again. `Generation`
accumulates content across all of them, so text emitted before a tool call
survives. Tools are only offered when search is enabled, the user has
`tools.web_search`, **and** the model is flagged `tools` — sending a `tools`
array to an endpoint without support fails the whole request, exactly as images
do without `vision`.
**Tool-call arguments arrive in fragments.** `delta.tool_calls` carries an
`index`, a name that appears once, and an `arguments` string split across
chunks. `tools.ToolCallAccumulator` rejoins them keyed on `index` — not on
name, which breaks the moment a model calls one tool twice in a turn.
**Tool results are not replayed.** Like reasoning, `Message.tool_calls_json` is
stored and rendered but never fed back as context. The answer already contains
what the model made of the results; replaying stale results and the schema into
every later request wastes the window and reliably sends a small model into a
search loop. The sources stay visible in the transcript.
**Search results are untrusted.** Hard rule 6 covers them as much as model
output. `chat/_tool_activity.html` escapes everything and only renders `http`
and `https` URLs as links — a result carrying a `javascript:` URL must never
become an anchor.
**A message bubble is rendered from four places.** `pages.py`,
`chats.post_message`, `chats.regenerate` and `chats._follow`. Each needs
`audio_service.template_flags(db, user)` or the speaker button's conditions are
undefined; the template uses `| default(false)` so a missed one degrades to no
button rather than an exception. `_follow` also passes `just_finished`, which is
what read-aloud-automatically keys off — without it, reopening a chat would
start reading its last reply out loud.
**Dictation audio never touches disk.** `api/audio.py` reads it into memory,
capped, and streams it upstream. It is not an attachment: it has no owner, no
row, and nothing would ever sweep it.
**The service worker must skip `/api/`.** A reply is an endless event stream and
passing one through a worker turns it into one delivery at the end, or nothing.
`static/js/sw.js` bails out on `/api/`, `/auth/`, `/admin/` and any request
accepting `text/event-stream`. It is served from `GET /sw.js` rather than the
static mount because a worker's scope is the path it came from.
## Changing the schema
There is no Alembic, but there *is* `db/migrations.py`. It compares the declared
@@ -306,10 +366,10 @@ notes describe the machine.
## Not built yet
Built-in tools + admin tool settings, custom tools and MCP, agentic execution
(local subprocess and SSH connection profiles), image generation. Nav entries
mark where each one goes.
Custom tools and MCP, agentic execution (local subprocess and SSH connection
profiles), image generation. Nav entries mark where each one goes. The tool
loop in `services/generation.py` is what they plug into — a second tool is a
registry entry, not a new code path.
`Model.capabilities_json` carries a `tools` flag nothing reads yet. No OCR:
a scanned PDF is stored with an explanatory `extraction_error` rather than
silently contributing nothing.
No OCR: a scanned PDF is stored with an explanatory `extraction_error` rather
than silently contributing nothing.