Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 19 Wiki Activity

PWA, one send/stop button, audio in and out, web search as a tool

Browse Source 
Four pieces of work.

**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.

**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.

**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.

**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.

Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.

Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.

338 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 17:56:50 +02:00
co-authored by Claude Opus 4.8
parent ca3e4fd04f
commit 436226370a
61 changed files with 4481 additions and 116 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/web/templates/admin/audio.html
+184
View File
@@ -0,0 +1,184 @@
{% extends "admin/_layout.html" %}
{% from "_macros.html" import icon %}
{% set section = "audio" %}
{% block title %}Audio - LLeMbas{% endblock %}
{% block heading %}Audio{% endblock %}
{% block admin_content %}
<p class="admin-lede">
Two endpoints speaking the OpenAI audio API: one that turns speech into text
so a message can be dictated, one that reads a reply out. They are configured
separately because they usually are separate servers — whisper.cpp and Kokoro,
say, or Speaches for both.
</p>
{% if saved %}
<div class="alert alert--success">{{ icon("check", "icon--sm") }} <span>Audio settings saved.</span></div>
{% endif %}
<form method="post" action="/admin/audio">
<section class="card">
<h2 class="card__title">
Dictation
{% if values.stt_enabled %}<span class="badge badge--success">on</span>
{% else %}<span class="badge">off</span>{% endif %}
</h2>
<p class="card__lede">
Adds a microphone to the composer. Recordings are sent to this endpoint
and never written to disk.
</p>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="stt_enabled" value="true"
{{ 'checked' if values.stt_enabled }}>
<span>Allow messages to be dictated</span>
</label>
</div>
<div class="grid grid--2">
<div class="field">
<label class="field__label" for="stt-base-url">Base URL</label>
<input class="input" id="stt-base-url" name="stt_base_url" type="url"
value="{{ values.stt_base_url }}" placeholder="http://127.0.0.1:8081">
<p class="field__hint">
Where <code>/v1/audio/transcriptions</code> lives — whisper.cpp's
<code>whisper-server</code>, Speaches, or anything else speaking it.
With or without <code>/v1</code>; either is understood.
</p>
</div>
<div class="field">
<label class="field__label" for="stt-api-key">API key</label>
<input class="input" id="stt-api-key" name="stt_api_key" type="password"
value="{{ unchanged if masked.stt else '' }}"
placeholder="{{ masked.stt or 'None needed for a local server' }}"
autocomplete="off">
<p class="field__hint">Encrypted at rest. Clear the field to remove it.</p>
</div>
<div class="field">
<label class="field__label" for="stt-model">Model</label>
<input class="input" id="stt-model" name="stt_model"
value="{{ values.stt_model }}" placeholder="whisper-1">
<p class="field__hint">
Sent even to servers that only host one; a router in front of several
needs it.
</p>
</div>
<div class="field">
<label class="field__label" for="stt-language">Language</label>
<input class="input" id="stt-language" name="stt_language" maxlength="16"
value="{{ values.stt_language }}" placeholder="detect">
<p class="field__hint">
An ISO code such as <code>en</code> or <code>sk</code>. Leave empty to
let the server detect it, which is what whisper does best.
</p>
</div>
</div>
<div class="btn-row">
<button class="btn" type="button" hx-post="/admin/audio/test/stt"
hx-target="#audio-test-stt" hx-swap="outerHTML">
{{ icon("refresh", "icon--sm") }} Test dictation
</button>
</div>
<div id="audio-test-stt"></div>
</section>
<section class="card">
<h2 class="card__title">
Read aloud
{% if values.tts_enabled %}<span class="badge badge--success">on</span>
{% else %}<span class="badge">off</span>{% endif %}
</h2>
<p class="card__lede">
Adds a speaker button to every reply. Each reader can pick their own voice
in their settings; what is chosen here is the default.
</p>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="tts_enabled" value="true"
{{ 'checked' if values.tts_enabled }}>
<span>Allow replies to be read out</span>
</label>
</div>
<div class="grid grid--2">
<div class="field">
<label class="field__label" for="tts-base-url">Base URL</label>
<input class="input" id="tts-base-url" name="tts_base_url" type="url"
value="{{ values.tts_base_url }}" placeholder="http://127.0.0.1:8880">
<p class="field__hint">
Where <code>/v1/audio/speech</code> lives — Kokoro-FastAPI, OpenAI, or
anything else speaking it.
</p>
</div>
<div class="field">
<label class="field__label" for="tts-api-key">API key</label>
<input class="input" id="tts-api-key" name="tts_api_key" type="password"
value="{{ unchanged if masked.tts else '' }}"
placeholder="{{ masked.tts or 'None needed for a local server' }}"
autocomplete="off">
</div>
<div class="field">
<label class="field__label" for="tts-model">Model</label>
<input class="input" id="tts-model" name="tts_model"
value="{{ values.tts_model }}" placeholder="tts-1">
</div>
<div class="field">
<label class="field__label" for="tts-voice">Default voice</label>
<select class="select" id="tts-voice" name="tts_voice">
{% with selected = values.tts_voice %}
{% include "partials/_voice_options.html" %}
{% endwith %}
</select>
<p class="field__hint">
{% if voice_error %}
Could not read the voice list: {{ voice_error }}
{% else %}
Read from the endpoint. Save and test to refresh it.
{% endif %}
</p>
</div>
<div class="field">
<label class="field__label" for="tts-format">Format</label>
<select class="select" id="tts-format" name="tts_format">
{% for format in formats %}
<option value="{{ format }}" {{ 'selected' if format == values.tts_format }}>
{{ format }}
</option>
{% endfor %}
</select>
</div>
<div class="field">
<label class="field__label" for="tts-speed">Speed</label>
<input class="input" id="tts-speed" name="tts_speed" type="number"
min="0.25" max="4" step="0.05" value="{{ values.tts_speed }}">
</div>
</div>
<div class="field">
<label class="checkbox">
<input type="checkbox" name="tts_autoplay" value="true"
{{ 'checked' if values.tts_autoplay }}>
<span>Read new replies aloud as they finish, by default</span>
</label>
<p class="field__hint">
Only the starting value for each account — anyone can turn it off in
their own settings, and nobody is made to listen.
</p>
</div>
<div class="btn-row">
<button class="btn" type="button" hx-post="/admin/audio/test/tts"
hx-target="#audio-test-tts" hx-swap="outerHTML">
{{ icon("refresh", "icon--sm") }} Test speech
</button>
</div>
<div id="audio-test-tts"></div>
</section>
<div class="btn-row"><button class="btn btn--primary" type="submit">Save settings</button></div>
</form>
{% endblock %}