PWA, one send/stop button, audio in and out, web search as a tool

Four pieces of work.

**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.

**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.

**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.

**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.

Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.

Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.

338 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 17:56:50 +02:00
parent de178837b8
commit 7456525d19
63 changed files with 4597 additions and 140 deletions
+44 -12
View File
@@ -4,8 +4,9 @@ Where the project is, what is deliberately not built yet, and the decisions
that would be expensive to revisit. Kept current as work lands; the detail of
*how* things work lives in [`CLAUDE.md`](CLAUDE.md).
**Status:** usable daily. Streaming chat, attachments, reasoning, users and
groups, model administration. 230 tests, `ruff` clean.
**Status:** usable daily. Streaming chat, attachments, reasoning, tool calling
with web search, speech in and out, users and groups, model administration,
installable as an app. 338 tests, `ruff` clean.
---
@@ -46,6 +47,24 @@ be a different project, not a refactor.
you were elsewhere
- [x] Folders, arbitrarily nested; deleting one keeps the chats inside it
### Tools
- [x] **Tool calling** — one reply is a bounded loop of requests, not one
request. Text produced before a call is kept
- [x] **Web search** as the first tool: DuckDuckGo (no setup), SearXNG or
Firecrawl, chosen in the admin area
- [x] Only offered to models flagged `tools`, because an endpoint without
support rejects the whole request rather than ignoring the array
- [x] Sources stay in the transcript; results are **not** replayed as context on
the next turn, for the same reasons reasoning is not
### Audio
- [x] **Dictation** — record in the composer, transcribed by any OpenAI-shaped
`/v1/audio/transcriptions` endpoint. The recording never touches disk
- [x] **Read aloud** — any `/v1/audio/speech` endpoint, with the voice list
discovered from the server where it offers one
- [x] Instance defaults in Admin, per-reader overrides in Settings — voice,
speed, dictation language, and whether replies play automatically
### Models and reasoning
- [x] OpenAI-compatible connections with encrypted keys and model discovery
- [x] **Reasoning display**`reasoning_content` and inline `<think>` tags,
@@ -76,6 +95,9 @@ be a different project, not a refactor.
**outright** rather than being concatenated
### Interface
- [x] **Installable** — manifest, generated PWA icons, a service worker for the
shell and a themed offline page. The worker deliberately never touches
`/api/`: a reply is an event stream and caching one breaks it
- [x] Two themes (`moria`, `shire`) from one set of design tokens
- [x] Every control sized from `--control-h`, so rows line up by construction
- [x] Toasts and dialogs of our own; no `window.confirm` anywhere
@@ -91,15 +113,11 @@ be a different project, not a refactor.
In the order they are likely to be worth doing.
### Tools — built-in, with admin settings
A tool registry, per-tool settings in the admin area, and tool-calling wired
through the chat loop. `Model.capabilities_json` already carries a `tools` flag
that nothing reads. The chat loop currently assumes one request produces one
reply; tool calling makes it a loop, which is the real work.
### Custom tools and MCP servers
An MCP client managing configured servers, their tools surfaced alongside the
built-in ones. Depends on the tool loop above.
An MCP client managing configured servers, their tools surfaced alongside
`web_search`. The loop they plug into exists now — `services/tools.py` is a
registry and `services/generation.py` already runs bounded rounds — so this is
a client and an admin screen rather than a change to how chat works.
### Agentic execution
Two modes, as originally specified:
@@ -118,7 +136,6 @@ already running on this machine and is the obvious first target.
- **OCR** for scanned PDFs
- **Conversation branching** — `Message.parent_id` exists unused; needs a UI for
choosing between versions, which is why rewind truncates for now
- **Web search** as a built-in tool
- **Chat export** (Markdown, JSON)
- **Archived chats** — the column exists, nothing surfaces it
- **Per-user quotas**
@@ -147,6 +164,14 @@ deduplication, no size quota.
**Unread is polled every 10 seconds.** A push channel would be more responsive
but means an always-on connection per tab for the sake of a green dot.
**Installing needs HTTPS or localhost.** Service workers are unavailable over
plain HTTP, so a LAN install without TLS is a normal browser tab. The
microphone is unavailable for the same reason.
**Tool calling needs a model that supports it.** The `tools` flag is an
administrator's assertion, not something endpoints reliably advertise. Set it on
a model that cannot, and its replies fail rather than degrade.
---
## Deliberate decisions
@@ -164,7 +189,14 @@ Recorded because each looks like an oversight until you know the reason.
- **Pinning is a shortcut, not an ordering.** A picker whose order silently
differs from the admin screen is confusing.
- **Images only reach models marked `vision`.** Not graceful degradation: most
endpoints reject the entire request rather than ignoring an image part.
endpoints reject the entire request rather than ignoring an image part. Tools
are gated the same way, for the same reason.
- **Tool results are not replayed.** Like reasoning: the answer already contains
what the model made of them, and replaying stale results into every later
request wastes the window and sends small models into search loops.
- **The service worker caches the shell, never a page with a user in it.** A
cached conversation would be a snapshot that silently went stale, belonging to
whoever was signed in last.
- **Markdown rendered server-side.** One code path produces the streamed and
the stored view, so they cannot disagree.
- **This repository is public.** Deployment hostnames, ports and paths stay out