PWA, one send/stop button, audio in and out, web search as a tool
Four pieces of work.
**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.
**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.
**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.
**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.
Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.
Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.
338 tests, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -12,17 +12,18 @@ no JavaScript build step.
|
||||
|
||||
```bash
|
||||
. .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
pip install -e ".[dev,search]" # `search` adds ddgs for DuckDuckGo
|
||||
|
||||
lembas serve # http://127.0.0.1:8080
|
||||
lembas info # paths + counts, useful when confused
|
||||
lembas secret-key # generate LEMBAS_SECRET_KEY
|
||||
lembas create-admin # create or promote an admin
|
||||
|
||||
pytest # 230 tests, ~9s
|
||||
pytest # 338 tests, ~16s
|
||||
# PLAN.md tracks what is and is not built
|
||||
ruff check . # lint (line length 100)
|
||||
python scripts/build_artwork.py # regenerate all SVG artwork
|
||||
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
|
||||
# needs fonttools and cairosvg)
|
||||
python scripts/fetch_vendor.py # verify vendored JS against the lockfile
|
||||
```
|
||||
|
||||
@@ -77,8 +78,11 @@ src/lembas/
|
||||
admin.py connections + instance settings
|
||||
admin_models.py model ordering, defaults, images, access
|
||||
admin_users.py users, groups, permissions
|
||||
admin_audio.py speech-to-text and text-to-speech endpoints
|
||||
admin_search.py web search provider and credentials
|
||||
audio.py transcribe, speak, voice discovery
|
||||
files.py upload, serve, remove attachments
|
||||
preferences.py per-user theme, default model, password
|
||||
preferences.py per-user theme, default model, password, audio
|
||||
db/
|
||||
base.py Base, UUID/Timestamp mixins
|
||||
session.py engine, SQLite pragmas, init_db, session_scope
|
||||
@@ -87,6 +91,9 @@ src/lembas/
|
||||
security/ passwords (argon2), sessions, permissions
|
||||
services/
|
||||
llm/openai_client.py httpx streaming + model discovery
|
||||
search/ ddgs, SearXNG and Firecrawl behind one shape
|
||||
audio.py OpenAI-shaped /v1/audio/* client
|
||||
tools.py tool registry, schemas, streamed-call reassembly
|
||||
chat.py request building, endpoint resolution, titles
|
||||
markdown.py markdown-it + pygments + nh3
|
||||
crypto.py Fernet encrypt/decrypt/mask
|
||||
@@ -98,8 +105,8 @@ src/lembas/
|
||||
web/
|
||||
templating.py render() -- always use this, not TemplateResponse
|
||||
templates/ Jinja
|
||||
static/ css, js, vendor, img
|
||||
assets/ SVG masters (generated)
|
||||
static/ css, js, vendor, img, sw.js
|
||||
assets/ SVG masters and PWA icons (generated)
|
||||
deploy/ systemd unit, nginx vhost, install/update scripts
|
||||
```
|
||||
|
||||
@@ -262,6 +269,59 @@ which is losing.
|
||||
plain dict is not detected. The columns use `MutableDict` (`db/types.py`), but
|
||||
the safe habit is `obj.field = {**obj.field, "k": v}`.
|
||||
|
||||
**`[hidden]` needs `!important`.** The browser's rule is `[hidden] { display:
|
||||
none }`, which any class setting `display` outranks — and `.btn` is
|
||||
`display: inline-flex`. That is not theoretical: it is why the old Stop button,
|
||||
created and then `hidden = true`, sat permanently beside Send. `app.css` forces
|
||||
the attribute to win. Anything toggled with `hidden` depends on that line.
|
||||
|
||||
**Send and Stop are one button.** `chat/_composer.html` renders both icons and
|
||||
`ui.js` flips `data-composer-action` plus `type` (`submit` ↔ `button`) when a
|
||||
message in the thread is still streaming. Do not add a second button back.
|
||||
|
||||
**The tool loop is inside one generation.** `services/generation.py:_run()` runs
|
||||
up to `tools_service.MAX_ROUNDS` request rounds for a single reply: stream,
|
||||
accumulate tool calls, run them, append the results, ask again. `Generation`
|
||||
accumulates content across all of them, so text emitted before a tool call
|
||||
survives. Tools are only offered when search is enabled, the user has
|
||||
`tools.web_search`, **and** the model is flagged `tools` — sending a `tools`
|
||||
array to an endpoint without support fails the whole request, exactly as images
|
||||
do without `vision`.
|
||||
|
||||
**Tool-call arguments arrive in fragments.** `delta.tool_calls` carries an
|
||||
`index`, a name that appears once, and an `arguments` string split across
|
||||
chunks. `tools.ToolCallAccumulator` rejoins them keyed on `index` — not on
|
||||
name, which breaks the moment a model calls one tool twice in a turn.
|
||||
|
||||
**Tool results are not replayed.** Like reasoning, `Message.tool_calls_json` is
|
||||
stored and rendered but never fed back as context. The answer already contains
|
||||
what the model made of the results; replaying stale results and the schema into
|
||||
every later request wastes the window and reliably sends a small model into a
|
||||
search loop. The sources stay visible in the transcript.
|
||||
|
||||
**Search results are untrusted.** Hard rule 6 covers them as much as model
|
||||
output. `chat/_tool_activity.html` escapes everything and only renders `http`
|
||||
and `https` URLs as links — a result carrying a `javascript:` URL must never
|
||||
become an anchor.
|
||||
|
||||
**A message bubble is rendered from four places.** `pages.py`,
|
||||
`chats.post_message`, `chats.regenerate` and `chats._follow`. Each needs
|
||||
`audio_service.template_flags(db, user)` or the speaker button's conditions are
|
||||
undefined; the template uses `| default(false)` so a missed one degrades to no
|
||||
button rather than an exception. `_follow` also passes `just_finished`, which is
|
||||
what read-aloud-automatically keys off — without it, reopening a chat would
|
||||
start reading its last reply out loud.
|
||||
|
||||
**Dictation audio never touches disk.** `api/audio.py` reads it into memory,
|
||||
capped, and streams it upstream. It is not an attachment: it has no owner, no
|
||||
row, and nothing would ever sweep it.
|
||||
|
||||
**The service worker must skip `/api/`.** A reply is an endless event stream and
|
||||
passing one through a worker turns it into one delivery at the end, or nothing.
|
||||
`static/js/sw.js` bails out on `/api/`, `/auth/`, `/admin/` and any request
|
||||
accepting `text/event-stream`. It is served from `GET /sw.js` rather than the
|
||||
static mount because a worker's scope is the path it came from.
|
||||
|
||||
## Changing the schema
|
||||
|
||||
There is no Alembic, but there *is* `db/migrations.py`. It compares the declared
|
||||
@@ -306,10 +366,10 @@ notes describe the machine.
|
||||
|
||||
## Not built yet
|
||||
|
||||
Built-in tools + admin tool settings, custom tools and MCP, agentic execution
|
||||
(local subprocess and SSH connection profiles), image generation. Nav entries
|
||||
mark where each one goes.
|
||||
Custom tools and MCP, agentic execution (local subprocess and SSH connection
|
||||
profiles), image generation. Nav entries mark where each one goes. The tool
|
||||
loop in `services/generation.py` is what they plug into — a second tool is a
|
||||
registry entry, not a new code path.
|
||||
|
||||
`Model.capabilities_json` carries a `tools` flag nothing reads yet. No OCR:
|
||||
a scanned PDF is stored with an explanatory `extraction_error` rather than
|
||||
silently contributing nothing.
|
||||
No OCR: a scanned PDF is stored with an explanatory `extraction_error` rather
|
||||
than silently contributing nothing.
|
||||
|
||||
Reference in New Issue
Block a user