PWA, one send/stop button, audio in and out, web search as a tool
Four pieces of work.
**Installable.** A manifest carrying the instance name, PWA icons rasterised
from the existing mark at design time, a service worker and a themed offline
page. The worker caches the shell only and bails out on /api/, /auth/, /admin/
and anything accepting text/event-stream -- passing a reply stream through a
worker turns it into one delivery at the end, or nothing. It is served from
GET /sw.js rather than the static mount because a worker's scope is the path it
came from.
**Send and Stop are one button.** They were two, and the hidden one was never
hidden: `.btn` is display: inline-flex, which outranks the browser's own
`[hidden] { display: none }`, so Stop sat permanently beside Send. app.css now
forces the attribute to win -- every control toggled with `hidden` depended on
that -- and the composer renders one button carrying both icons, with ui.js
flipping data-composer-action and the type with it.
**Audio.** Speech to text and text to speech against any OpenAI-shaped
/v1/audio/* endpoint: dictate into the composer, have a reply read out.
Instance settings in Admin, per-reader overrides in Settings, with the voice
list discovered from the server where it offers one. Recorded audio is capped
and never written to disk -- it is not an attachment, it has no owner, and
nothing would ever sweep it.
**Web search, as a tool.** This is the tool loop PLAN.md described as the real
work: one reply is now a bounded sequence of requests rather than one. The model
asks, the tool runs, the result goes back and it is asked again, up to three
rounds. Providers are DuckDuckGo (no setup), SearXNG and Firecrawl.
Two decisions worth stating. Tools are only offered to models flagged `tools`,
because an endpoint without support rejects the whole request rather than
ignoring the array -- the same reason images only reach models flagged
`vision`. And tool results are not replayed as context on the next turn, for the
same reasons reasoning is not: the answer already contains what the model made
of them, and replaying stale results into every later request wastes the window
and reliably sends a small model into a search loop. The sources stay visible in
the transcript instead.
Search results are untrusted third-party text and are treated as such: escaped,
and only http/https URLs rendered as links.
338 tests, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -120,6 +120,29 @@ def make_chat(db: Session):
|
||||
return _create
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_http():
|
||||
"""Answer every outgoing httpx request with a handler of the test's choosing.
|
||||
|
||||
The services build their own AsyncClient because each needs its own timeout,
|
||||
so there is no client to inject; patching the class is what reaches them.
|
||||
Returns a callable that installs a handler and is undone on teardown.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
original = httpx.AsyncClient
|
||||
|
||||
def install(handler):
|
||||
class Patched(original):
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(transport=httpx.MockTransport(handler), **kwargs)
|
||||
|
||||
httpx.AsyncClient = Patched
|
||||
|
||||
yield install
|
||||
httpx.AsyncClient = original
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def user_id(db: Session, registered: dict[str, str]) -> str:
|
||||
"""The registered user's id.
|
||||
|
||||
@@ -0,0 +1,363 @@
|
||||
"""Speech to text and text to speech."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from lembas.services import audio as audio_service
|
||||
from lembas.services import settings_store
|
||||
from lembas.services.llm.openai_client import Endpoint, LLMError
|
||||
from lembas.services.markdown import speakable_text
|
||||
|
||||
|
||||
# --- What gets read out ------------------------------------------------------
|
||||
def test_speakable_text_drops_code_blocks():
|
||||
"""A speech model reads every bracket and underscore of a code fence aloud,
|
||||
which is unlistenable and longer than the prose it was buried in."""
|
||||
spoken = speakable_text("Try this:\n\n```py\nprint('hello world')\n```\n\nThat is all.")
|
||||
assert "print" not in spoken
|
||||
assert "Try this:" in spoken
|
||||
assert "That is all." in spoken
|
||||
|
||||
|
||||
def test_speakable_text_drops_the_code_block_language_label():
|
||||
"""The label lives in its own div inside the block, so a naive non-greedy
|
||||
match for the wrapper's closing tag leaves both it and the code behind."""
|
||||
assert "python" not in speakable_text("```python\nx = 1\n```")
|
||||
|
||||
|
||||
def test_speakable_text_keeps_inline_code():
|
||||
assert "os.path" in speakable_text("Use `os.path` for that.")
|
||||
|
||||
|
||||
def test_speakable_text_reduces_links_to_their_words():
|
||||
spoken = speakable_text("Read [the docs](https://example.com/very/long) first.")
|
||||
assert "the docs" in spoken
|
||||
assert "https" not in spoken
|
||||
|
||||
|
||||
def test_speakable_text_unescapes_entities():
|
||||
"""The renderer escapes & and >; reading "amp semicolon" out is nonsense."""
|
||||
assert speakable_text("Salt & pepper, 5 > 3") == "Salt & pepper, 5 > 3"
|
||||
|
||||
|
||||
def test_speakable_text_of_nothing():
|
||||
assert speakable_text("") == ""
|
||||
assert speakable_text("```\nonly code\n```") == ""
|
||||
|
||||
|
||||
def test_speakable_text_is_capped():
|
||||
"""Endpoints reject or truncate very long input, and a reply this long is
|
||||
not one anybody is listening to in full."""
|
||||
from lembas.services.markdown import MAX_SPEAKABLE
|
||||
|
||||
assert len(speakable_text("word " * 10000)) == MAX_SPEAKABLE
|
||||
|
||||
|
||||
# --- Voice discovery ---------------------------------------------------------
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
{"voices": ["af_heart", "am_adam"]},
|
||||
{"voices": [{"id": "af_heart"}, {"id": "am_adam"}]},
|
||||
{"voices": [{"name": "af_heart"}, {"name": "am_adam"}]},
|
||||
["af_heart", "am_adam"],
|
||||
],
|
||||
)
|
||||
async def test_voices_reads_every_shape_a_server_might_use(mock_http, payload):
|
||||
"""Kokoro answers with objects, older builds with strings, some with a bare
|
||||
list. All three are the same information."""
|
||||
mock_http(lambda _r: httpx.Response(200, json=payload))
|
||||
audio_service.forget_voices()
|
||||
|
||||
found = await audio_service.voices(Endpoint("http://tts", "", {}))
|
||||
assert found == ["af_heart", "am_adam"]
|
||||
|
||||
|
||||
async def test_voices_falls_back_on_404(mock_http):
|
||||
"""api.openai.com has no voices endpoint. That is not an error -- its voices
|
||||
are a fixed list everybody already knows."""
|
||||
mock_http(lambda _r: httpx.Response(404))
|
||||
audio_service.forget_voices()
|
||||
|
||||
assert await audio_service.voices(Endpoint("http://tts", "", {})) == list(
|
||||
audio_service.OPENAI_VOICES
|
||||
)
|
||||
|
||||
|
||||
async def test_voices_are_cached(mock_http):
|
||||
calls = []
|
||||
|
||||
def handler(_request):
|
||||
calls.append(1)
|
||||
return httpx.Response(200, json={"voices": ["af_heart"]})
|
||||
|
||||
mock_http(handler)
|
||||
audio_service.forget_voices()
|
||||
|
||||
endpoint = Endpoint("http://tts", "", {})
|
||||
await audio_service.voices(endpoint)
|
||||
await audio_service.voices(endpoint)
|
||||
assert len(calls) == 1
|
||||
|
||||
await audio_service.voices(endpoint, refresh=True)
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
# --- Transcription -----------------------------------------------------------
|
||||
async def test_transcribe_reads_the_json_shape(mock_http):
|
||||
mock_http(lambda _r: httpx.Response(200, json={"text": " speak friend "}))
|
||||
text = await audio_service.transcribe(
|
||||
Endpoint("http://stt", "", {}),
|
||||
data=b"x",
|
||||
filename="a.wav",
|
||||
content_type="audio/wav",
|
||||
)
|
||||
assert text == "speak friend"
|
||||
|
||||
|
||||
async def test_transcribe_tolerates_a_plain_text_body(mock_http):
|
||||
"""Some servers answer in text no matter what response_format was asked
|
||||
for."""
|
||||
mock_http(lambda _r: httpx.Response(200, text="speak friend"))
|
||||
text = await audio_service.transcribe(
|
||||
Endpoint("http://stt", "", {}), data=b"x", filename="a.wav", content_type="audio/wav"
|
||||
)
|
||||
assert text == "speak friend"
|
||||
|
||||
|
||||
async def test_transcribe_omits_an_empty_language(mock_http):
|
||||
"""An empty language must be left out entirely -- sending "" makes some
|
||||
servers fail rather than detecting it."""
|
||||
seen = {}
|
||||
|
||||
def handler(request):
|
||||
seen["body"] = request.content
|
||||
return httpx.Response(200, json={"text": "ok"})
|
||||
|
||||
mock_http(handler)
|
||||
await audio_service.transcribe(
|
||||
Endpoint("http://stt", "", {}),
|
||||
data=b"x",
|
||||
filename="a.wav",
|
||||
content_type="audio/wav",
|
||||
language="",
|
||||
)
|
||||
assert b'name="language"' not in seen["body"]
|
||||
|
||||
|
||||
async def test_transcribe_reports_a_rejected_key_readably(mock_http):
|
||||
mock_http(lambda _r: httpx.Response(401, json={"error": {"message": "Bad key."}}))
|
||||
with pytest.raises(LLMError) as caught:
|
||||
await audio_service.transcribe(
|
||||
Endpoint("http://stt", "k", {}),
|
||||
data=b"x",
|
||||
filename="a.wav",
|
||||
content_type="audio/wav",
|
||||
)
|
||||
assert "rejected" in caught.value.message
|
||||
|
||||
|
||||
# --- Endpoint construction ---------------------------------------------------
|
||||
def test_endpoint_for_refuses_an_unconfigured_side():
|
||||
with pytest.raises(LLMError):
|
||||
audio_service.endpoint_for({"tts_base_url": ""}, "tts")
|
||||
|
||||
|
||||
def test_endpoint_for_decrypts_the_stored_key():
|
||||
from lembas.services.crypto import encrypt
|
||||
|
||||
endpoint = audio_service.endpoint_for(
|
||||
{"tts_base_url": "http://tts/", "tts_api_key_encrypted": encrypt("secret")}, "tts"
|
||||
)
|
||||
assert endpoint.api_key == "secret"
|
||||
assert endpoint.base_url == "http://tts"
|
||||
|
||||
|
||||
# --- Settings ----------------------------------------------------------------
|
||||
def test_audio_settings_round_trip(db):
|
||||
settings_store.update(
|
||||
db, {"tts_enabled": True, "tts_voice": "af_heart"}, key=settings_store.AUDIO
|
||||
)
|
||||
values = settings_store.audio(db)
|
||||
assert values["tts_enabled"] is True
|
||||
assert values["tts_voice"] == "af_heart"
|
||||
# Untouched keys still come back from the defaults.
|
||||
assert values["tts_format"] == "mp3"
|
||||
|
||||
|
||||
def test_audio_and_general_settings_do_not_collide(db):
|
||||
settings_store.update(db, {"instance_name": "Rivendell"})
|
||||
settings_store.update(db, {"tts_enabled": True}, key=settings_store.AUDIO)
|
||||
assert settings_store.get(db, "instance_name") == "Rivendell"
|
||||
assert "instance_name" not in settings_store.audio(db)
|
||||
|
||||
|
||||
def test_a_resubmitted_mask_keeps_the_stored_key(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"tts_enabled": "true", "tts_base_url": "http://tts", "tts_api_key": "secret"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
stored = settings_store.audio(db)["tts_api_key_encrypted"]
|
||||
assert stored
|
||||
|
||||
# Saving again with the mask in the field must not wipe the credential.
|
||||
from lembas.services.crypto import UNCHANGED_SENTINEL
|
||||
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={
|
||||
"tts_enabled": "true",
|
||||
"tts_base_url": "http://tts",
|
||||
"tts_api_key": UNCHANGED_SENTINEL,
|
||||
},
|
||||
follow_redirects=False,
|
||||
)
|
||||
assert settings_store.audio(db)["tts_api_key_encrypted"] == stored
|
||||
|
||||
|
||||
def test_an_emptied_key_field_clears_it(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"tts_base_url": "http://tts", "tts_api_key": "secret"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"tts_base_url": "http://tts", "tts_api_key": ""},
|
||||
follow_redirects=False,
|
||||
)
|
||||
assert settings_store.audio(db)["tts_api_key_encrypted"] == ""
|
||||
|
||||
|
||||
# --- The API -----------------------------------------------------------------
|
||||
def test_transcribe_is_absent_until_it_is_enabled(client: TestClient, registered):
|
||||
response = client.post(
|
||||
"/api/audio/transcribe", files={"file": ("a.wav", b"x", "audio/wav")}
|
||||
)
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_transcribe_rejects_an_empty_recording(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"stt_enabled": "true", "stt_base_url": "http://stt"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
response = client.post(
|
||||
"/api/audio/transcribe", files={"file": ("a.wav", b"", "audio/wav")}
|
||||
)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_transcribe_rejects_an_oversized_recording(client: TestClient, db, registered):
|
||||
from lembas.api.audio import MAX_AUDIO_BYTES
|
||||
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"stt_enabled": "true", "stt_base_url": "http://stt"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
response = client.post(
|
||||
"/api/audio/transcribe",
|
||||
files={"file": ("a.wav", b"x" * (MAX_AUDIO_BYTES + 10), "audio/wav")},
|
||||
)
|
||||
# Refused without the body ever reaching the transcription endpoint.
|
||||
assert response.status_code == 413
|
||||
|
||||
|
||||
def test_speech_404s_on_someone_elses_chat(client: TestClient, db, registered, make_chat):
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"tts_enabled": "true", "tts_base_url": "http://tts"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
client.post(
|
||||
"/auth/register",
|
||||
data={"name": "Sam", "email": "sam@shire.test", "password": "potatoes-and-more"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
# Now signed in as Sam, asking for Frodo's chat.
|
||||
chat_id = make_chat(email=registered["email"])
|
||||
response = client.get(f"/api/audio/speech/{chat_id}/anything")
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_the_audio_tab_appears_only_once_audio_is_configured(
|
||||
client: TestClient, db, registered
|
||||
):
|
||||
assert "tab-audio" not in client.get("/settings").text
|
||||
client.post(
|
||||
"/admin/audio",
|
||||
data={"tts_enabled": "true", "tts_base_url": "http://tts"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
assert "tab-audio" in client.get("/settings").text
|
||||
|
||||
|
||||
def test_saving_audio_preferences(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/api/preferences/audio",
|
||||
data={"voice": "am_adam", "speed": "1.5", "autoplay": "true"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
from sqlalchemy import select
|
||||
|
||||
from lembas.db.models import User
|
||||
|
||||
user = db.scalar(select(User).where(User.email == registered["email"]))
|
||||
db.refresh(user)
|
||||
assert user.settings_json["audio"] == {
|
||||
"autoplay": True,
|
||||
"voice": "am_adam",
|
||||
"speed": 1.5,
|
||||
}
|
||||
|
||||
|
||||
def test_an_unreadable_speed_leaves_the_rest_of_the_form_intact(
|
||||
client: TestClient, db, registered
|
||||
):
|
||||
client.post(
|
||||
"/api/preferences/audio",
|
||||
data={"voice": "am_adam", "speed": "quickly"},
|
||||
follow_redirects=False,
|
||||
)
|
||||
from sqlalchemy import select
|
||||
|
||||
from lembas.db.models import User
|
||||
|
||||
user = db.scalar(select(User).where(User.email == registered["email"]))
|
||||
db.refresh(user)
|
||||
assert user.settings_json["audio"]["voice"] == "am_adam"
|
||||
assert "speed" not in user.settings_json["audio"]
|
||||
|
||||
|
||||
# --- Resolving what a given reader hears -------------------------------------
|
||||
def _user_with(settings):
|
||||
from lembas.db.models import User
|
||||
|
||||
return User(name="x", email="x@x.test", password_hash="", settings_json=settings)
|
||||
|
||||
|
||||
def test_a_readers_voice_beats_the_instance_default():
|
||||
from lembas.api.audio import resolve_voice
|
||||
|
||||
user = _user_with({"audio": {"voice": "am_adam"}})
|
||||
assert resolve_voice({"tts_voice": "af_heart"}, user) == "am_adam"
|
||||
|
||||
|
||||
def test_the_instance_voice_is_used_when_the_reader_has_no_preference():
|
||||
from lembas.api.audio import resolve_voice
|
||||
|
||||
assert resolve_voice({"tts_voice": "af_heart"}, _user_with({})) == "af_heart"
|
||||
|
||||
|
||||
def test_speed_is_clamped_to_what_every_endpoint_accepts():
|
||||
from lembas.api.audio import resolve_speed
|
||||
|
||||
assert resolve_speed({}, _user_with({"audio": {"speed": 99}})) == 4.0
|
||||
assert resolve_speed({}, _user_with({"audio": {"speed": 0}})) == 0.25
|
||||
assert resolve_speed({}, _user_with({"audio": {"speed": "fast"}})) == 1.0
|
||||
@@ -0,0 +1,209 @@
|
||||
"""The tool loop: one reply, several requests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from lembas.db.models import ROLE_ASSISTANT, Chat, Connection, Message, Model
|
||||
from lembas.services import generation as generation_service
|
||||
from lembas.services import settings_store
|
||||
from lembas.services import tools as tools_service
|
||||
from lembas.services.search.base import SearchResult
|
||||
|
||||
|
||||
def _chat_with_tools(db, user_id):
|
||||
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
|
||||
db.add(connection)
|
||||
db.commit()
|
||||
db.add(Model(connection_id=connection.id, model_id="m", capabilities_json={"tools": True}))
|
||||
db.commit()
|
||||
chat = Chat(user_id=user_id, model_id="m", connection_id=connection.id)
|
||||
db.add(chat)
|
||||
db.commit()
|
||||
db.add(Message(chat_id=chat.id, role="user", content="What is a mallorn?", complete=True))
|
||||
db.commit()
|
||||
assistant = Message(chat_id=chat.id, role=ROLE_ASSISTANT, content="", complete=False)
|
||||
db.add(assistant)
|
||||
db.commit()
|
||||
return chat.id, assistant.id
|
||||
|
||||
|
||||
def _tool_call_chunk(name: str, arguments: str) -> dict:
|
||||
return {
|
||||
"choices": [
|
||||
{"delta": {"tool_calls": [{"index": 0, "id": "c1", "function": {
|
||||
"name": name, "arguments": arguments}}]}}
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def _text_chunk(text: str) -> dict:
|
||||
return {"choices": [{"delta": {"content": text}}]}
|
||||
|
||||
|
||||
def _stub_stream(rounds, seen_payloads):
|
||||
"""A stream_chat that returns a different scripted round each time."""
|
||||
|
||||
async def stream_chat(_endpoint, payload):
|
||||
seen_payloads.append(payload)
|
||||
for chunk in rounds[min(len(seen_payloads) - 1, len(rounds) - 1)]:
|
||||
yield chunk
|
||||
|
||||
return stream_chat
|
||||
|
||||
|
||||
async def test_a_tool_call_produces_a_second_request(db, user_id, monkeypatch):
|
||||
"""The whole point: one reply, two round trips, with the search result in
|
||||
the second one's messages."""
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
|
||||
async def fake_search(_config, _query, *, limit=None):
|
||||
return [SearchResult("Mallorn", "https://tolkien.test/mallorn", "A golden tree.")]
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", fake_search)
|
||||
|
||||
payloads = []
|
||||
monkeypatch.setattr(
|
||||
generation_service,
|
||||
"stream_chat",
|
||||
_stub_stream(
|
||||
[
|
||||
[_tool_call_chunk("web_search", '{"query": "mallorn"}')],
|
||||
[_text_chunk("A mallorn is a golden tree.")],
|
||||
],
|
||||
payloads,
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"lembas.services.chat.generate_title", _never_called_title
|
||||
)
|
||||
|
||||
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
await generation_service._run(generation)
|
||||
|
||||
assert len(payloads) == 2, "the model asked for a tool, so it must be asked again"
|
||||
assert generation.text == "A mallorn is a golden tree."
|
||||
|
||||
# The second request carries the assistant's own call back, then the result.
|
||||
followups = payloads[1]["messages"][-2:]
|
||||
assert followups[0]["tool_calls"][0]["function"]["name"] == "web_search"
|
||||
assert followups[1]["role"] == "tool"
|
||||
assert "https://tolkien.test/mallorn" in followups[1]["content"]
|
||||
|
||||
# And the reader gets to see what it looked up.
|
||||
assert generation.tool_events[0]["query"] == "mallorn"
|
||||
assert generation.tool_events[0]["results"][0]["url"] == "https://tolkien.test/mallorn"
|
||||
|
||||
|
||||
async def test_the_tools_array_is_absent_without_the_capability(db, user_id, monkeypatch):
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
# Search enabled, but the model is not marked as supporting tools.
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
model = db.query(Model).first()
|
||||
model.capabilities_json = {}
|
||||
db.commit()
|
||||
|
||||
payloads = []
|
||||
monkeypatch.setattr(
|
||||
generation_service, "stream_chat", _stub_stream([[_text_chunk("hi")]], payloads)
|
||||
)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
await generation_service._run(
|
||||
generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
)
|
||||
assert "tools" not in payloads[0]
|
||||
|
||||
|
||||
async def test_text_before_a_tool_call_is_kept(db, user_id, monkeypatch):
|
||||
"""A model that narrates what it is about to look up must not lose that
|
||||
when the results come back."""
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"lembas.services.search.run", _empty_search
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
generation_service,
|
||||
"stream_chat",
|
||||
_stub_stream(
|
||||
[
|
||||
[
|
||||
_text_chunk("Let me look that up. "),
|
||||
_tool_call_chunk("web_search", '{"query": "mallorn"}'),
|
||||
],
|
||||
[_text_chunk("Nothing found.")],
|
||||
],
|
||||
[],
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
await generation_service._run(generation)
|
||||
assert generation.text == "Let me look that up. Nothing found."
|
||||
|
||||
|
||||
async def test_a_model_that_only_ever_calls_tools_is_stopped(db, user_id, monkeypatch):
|
||||
"""Otherwise a small model that has decided searching is the answer keeps
|
||||
searching until the context runs out, at a full request each time."""
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", _empty_search)
|
||||
|
||||
payloads = []
|
||||
monkeypatch.setattr(
|
||||
generation_service,
|
||||
"stream_chat",
|
||||
_stub_stream([[_tool_call_chunk("web_search", '{"query": "x"}')]], payloads),
|
||||
)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
await generation_service._run(generation)
|
||||
|
||||
assert len(payloads) == tools_service.MAX_ROUNDS + 1
|
||||
# Recorded rather than silently dropped: an answer that stops here has to
|
||||
# be explicable.
|
||||
assert generation.tool_events[-1]["status"] == "error"
|
||||
|
||||
|
||||
async def test_tool_activity_is_stored_with_the_message(db, user_id, monkeypatch):
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
|
||||
async def fake_search(_config, _query, *, limit=None):
|
||||
return [SearchResult("Mallorn", "https://tolkien.test/m", "A tree.")]
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", fake_search)
|
||||
monkeypatch.setattr(
|
||||
generation_service,
|
||||
"stream_chat",
|
||||
_stub_stream(
|
||||
[
|
||||
[_tool_call_chunk("web_search", '{"query": "mallorn"}')],
|
||||
[_text_chunk("Done.")],
|
||||
],
|
||||
[],
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
await generation_service._run(
|
||||
generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
)
|
||||
|
||||
stored = db.get(Message, message_id)
|
||||
db.refresh(stored)
|
||||
assert stored.tool_calls_json[0]["query"] == "mallorn"
|
||||
assert stored.complete is True
|
||||
|
||||
|
||||
async def _empty_search(_config, _query, *, limit=None):
|
||||
return []
|
||||
|
||||
|
||||
async def _never_called_title(*_args, **_kwargs):
|
||||
"""Auto-titling makes its own request; these tests are about the tool loop."""
|
||||
return "A title"
|
||||
@@ -0,0 +1,164 @@
|
||||
"""Installing as an app, and the composer's single send/stop button."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from lembas.services import settings_store
|
||||
from lembas.web.templating import STATIC_DIR
|
||||
|
||||
|
||||
# --- Manifest ----------------------------------------------------------------
|
||||
def test_the_manifest_is_readable_when_signed_out(client: TestClient):
|
||||
"""A browser fetches the manifest outside any page's session."""
|
||||
response = client.get("/manifest.webmanifest")
|
||||
assert response.status_code == 200
|
||||
assert response.headers["content-type"].startswith("application/manifest+json")
|
||||
|
||||
|
||||
def test_the_manifest_carries_the_instance_name(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/general", data={"instance_name": "Rivendell"}, follow_redirects=False
|
||||
)
|
||||
assert client.get("/manifest.webmanifest").json()["name"] == "Rivendell"
|
||||
|
||||
|
||||
def test_the_manifest_offers_a_maskable_icon(client: TestClient):
|
||||
"""Without one, Android crops the corners off the wafer."""
|
||||
icons = client.get("/manifest.webmanifest").json()["icons"]
|
||||
assert any(icon["purpose"] == "maskable" for icon in icons)
|
||||
assert any(icon["sizes"] == "512x512" and icon["purpose"] == "any" for icon in icons)
|
||||
|
||||
|
||||
def test_every_manifest_icon_exists(client: TestClient):
|
||||
for icon in client.get("/manifest.webmanifest").json()["icons"]:
|
||||
assert client.get(icon["src"]).status_code == 200, icon["src"]
|
||||
|
||||
|
||||
def test_the_manifest_starts_at_the_chat(client: TestClient):
|
||||
payload = client.get("/manifest.webmanifest").json()
|
||||
assert payload["start_url"] == "/chat"
|
||||
assert payload["scope"] == "/"
|
||||
assert payload["display"] == "standalone"
|
||||
|
||||
|
||||
# --- Service worker ----------------------------------------------------------
|
||||
def test_the_worker_is_served_from_the_root(client: TestClient):
|
||||
"""A worker under /static/js/ would have scope /static/js/ and control
|
||||
nothing."""
|
||||
response = client.get("/sw.js")
|
||||
assert response.status_code == 200
|
||||
assert response.headers["content-type"].startswith("text/javascript")
|
||||
|
||||
|
||||
def test_the_worker_is_never_cached(client: TestClient):
|
||||
"""A stale worker keeps serving a stale cache."""
|
||||
assert "no-store" in client.get("/sw.js").headers["cache-control"]
|
||||
|
||||
|
||||
def test_the_worker_leaves_the_api_alone():
|
||||
"""The reply stream, the unread poll and attachment downloads all live
|
||||
under /api/. A cached response on any of them is at best stale."""
|
||||
source = (STATIC_DIR / "js" / "sw.js").read_text()
|
||||
assert '"/api/"' in source
|
||||
assert "text/event-stream" in source
|
||||
|
||||
|
||||
def test_every_precached_asset_exists(client: TestClient):
|
||||
"""addAll is all-or-nothing in most implementations, and a missing entry is
|
||||
invisible until someone opens the developer tools."""
|
||||
source = (STATIC_DIR / "js" / "sw.js").read_text()
|
||||
shell = source.split("var SHELL = [", 1)[1].split("];", 1)[0]
|
||||
paths = [line.strip().strip('",') for line in shell.splitlines() if '"' in line]
|
||||
|
||||
assert paths
|
||||
for path in paths:
|
||||
assert client.get(path).status_code == 200, path
|
||||
|
||||
|
||||
def test_the_offline_page_stands_on_its_own(client: TestClient):
|
||||
"""Cached at install time, so it must render with no user and no chats."""
|
||||
response = client.get("/offline")
|
||||
assert response.status_code == 200
|
||||
assert 'class="sidebar' not in response.text
|
||||
assert 'id="thread' not in response.text
|
||||
|
||||
|
||||
def test_the_page_links_the_manifest_and_the_apple_icon(client: TestClient, registered):
|
||||
page = client.get("/chat").text
|
||||
assert 'rel="manifest"' in page
|
||||
assert 'rel="apple-touch-icon"' in page
|
||||
assert 'name="theme-color"' in page
|
||||
|
||||
|
||||
# --- Send and Stop -----------------------------------------------------------
|
||||
def test_the_hidden_attribute_wins_over_component_styles():
|
||||
"""`.btn` is display: inline-flex, which beats the browser's own
|
||||
`[hidden] { display: none }`. Without this rule a button hidden from
|
||||
JavaScript stays on screen -- which is how Stop came to sit permanently
|
||||
beside Send."""
|
||||
css = (STATIC_DIR / "css" / "app.css").read_text()
|
||||
assert "[hidden]" in css
|
||||
assert "display: none !important" in css
|
||||
|
||||
|
||||
def test_the_composer_has_exactly_one_send_button(client: TestClient, db, registered):
|
||||
"""One button that becomes Stop, not two that take turns being hidden."""
|
||||
_add_a_model(db)
|
||||
page = client.get("/chat").text
|
||||
assert page.count("data-composer-action") == 1
|
||||
|
||||
|
||||
def test_the_send_button_carries_both_icons(client: TestClient, db, registered):
|
||||
"""Rendered together and chosen in CSS, so the swap costs no layout and
|
||||
cannot flash an empty button."""
|
||||
_add_a_model(db)
|
||||
page = client.get("/chat").text
|
||||
assert "composer__icon--send" in page
|
||||
assert "composer__icon--stop" in page
|
||||
|
||||
|
||||
def test_the_composer_starts_in_the_send_state(client: TestClient, db, registered):
|
||||
_add_a_model(db)
|
||||
assert 'data-composer-action="send"' in client.get("/chat").text
|
||||
|
||||
|
||||
def _add_a_model(db):
|
||||
from lembas.db.models import Connection, Model
|
||||
|
||||
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
|
||||
db.add(connection)
|
||||
db.commit()
|
||||
db.add(Model(connection_id=connection.id, model_id="m"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# --- The icons themselves ----------------------------------------------------
|
||||
def test_the_generated_icons_are_committed():
|
||||
"""They come from scripts/build_artwork.py and are committed like the SVGs;
|
||||
the running application has no rasteriser."""
|
||||
for name in (
|
||||
"icon-192.png",
|
||||
"icon-512.png",
|
||||
"icon-maskable-512.png",
|
||||
"apple-touch-icon-180.png",
|
||||
):
|
||||
path = Path(STATIC_DIR) / "img" / name
|
||||
assert path.exists(), name
|
||||
assert path.read_bytes()[:8] == b"\x89PNG\r\n\x1a\n", name
|
||||
|
||||
|
||||
def test_the_mic_appears_only_when_dictation_is_configured(
|
||||
client: TestClient, db, registered
|
||||
):
|
||||
_add_a_model(db)
|
||||
assert "data-mic" not in client.get("/chat").text
|
||||
|
||||
settings_store.update(
|
||||
db,
|
||||
{"stt_enabled": True, "stt_base_url": "http://stt"},
|
||||
key=settings_store.AUDIO,
|
||||
)
|
||||
assert "data-mic" in client.get("/chat").text
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Web search providers and their normalisation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from lembas.services import search as search_service
|
||||
from lembas.services import settings_store
|
||||
from lembas.services.crypto import encrypt
|
||||
from lembas.services.search import firecrawl, searxng
|
||||
from lembas.services.search.base import SearchError, SearchResult, clean
|
||||
|
||||
|
||||
# --- What a result is --------------------------------------------------------
|
||||
def test_only_http_urls_are_linkable():
|
||||
"""A search provider is an untrusted source. A result carrying a
|
||||
javascript: URL must never become an anchor pointing at it."""
|
||||
assert SearchResult("t", "https://example.com", "").is_linkable
|
||||
assert SearchResult("t", "http://example.com", "").is_linkable
|
||||
assert not SearchResult("t", "javascript:alert(1)", "").is_linkable
|
||||
assert not SearchResult("t", "data:text/html,<script>", "").is_linkable
|
||||
|
||||
|
||||
def test_the_host_is_pulled_out_for_display():
|
||||
assert SearchResult("t", "https://en.wikipedia.org/wiki/X", "").host == "en.wikipedia.org"
|
||||
|
||||
|
||||
def test_clean_drops_a_row_with_no_url():
|
||||
assert clean("A title", "", "text") is None
|
||||
|
||||
|
||||
def test_clean_falls_back_to_the_url_as_a_title():
|
||||
assert clean("", "https://example.com", "").title == "https://example.com"
|
||||
|
||||
|
||||
def test_clean_collapses_whitespace_and_truncates():
|
||||
from lembas.services.search.base import MAX_SNIPPET
|
||||
|
||||
result = clean(" a\n b ", "https://x.test", "word " * 500)
|
||||
assert result.title == "a b"
|
||||
assert len(result.snippet) <= MAX_SNIPPET
|
||||
|
||||
|
||||
# --- SearXNG -----------------------------------------------------------------
|
||||
async def test_searxng_normalises_its_results(mock_http):
|
||||
mock_http(
|
||||
lambda _r: httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"results": [
|
||||
{"title": "One", "url": "https://one.test", "content": "first"},
|
||||
{"title": "Two", "url": "https://two.test", "content": "second"},
|
||||
]
|
||||
},
|
||||
)
|
||||
)
|
||||
results = await searxng.search({"searxng_base_url": "http://searx"}, "q", 5)
|
||||
assert [r.title for r in results] == ["One", "Two"]
|
||||
assert results[0].snippet == "first"
|
||||
|
||||
|
||||
async def test_searxng_names_the_disabled_json_format(mock_http):
|
||||
"""A stock instance refuses the JSON format with a 403. Reporting "search
|
||||
failed" would leave the one-line fix undiscoverable."""
|
||||
mock_http(lambda _r: httpx.Response(403, text="Forbidden"))
|
||||
with pytest.raises(SearchError) as caught:
|
||||
await searxng.search({"searxng_base_url": "http://searx"}, "q", 5)
|
||||
assert "settings.yml" in caught.value.message
|
||||
|
||||
|
||||
async def test_searxng_treats_an_html_answer_as_the_same_misconfiguration(mock_http):
|
||||
mock_http(lambda _r: httpx.Response(200, text="<html>results</html>"))
|
||||
with pytest.raises(SearchError) as caught:
|
||||
await searxng.search({"searxng_base_url": "http://searx"}, "q", 5)
|
||||
assert "settings.yml" in caught.value.message
|
||||
|
||||
|
||||
async def test_searxng_needs_a_url():
|
||||
with pytest.raises(SearchError):
|
||||
await searxng.search({"searxng_base_url": ""}, "q", 5)
|
||||
|
||||
|
||||
# --- Firecrawl ---------------------------------------------------------------
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
{"data": [{"title": "One", "url": "https://one.test", "description": "first"}]},
|
||||
# Newer responses nest the list under data.web.
|
||||
{"data": {"web": [{"title": "One", "url": "https://one.test", "description": "first"}]}},
|
||||
],
|
||||
)
|
||||
async def test_firecrawl_reads_both_response_shapes(mock_http, payload):
|
||||
mock_http(lambda _r: httpx.Response(200, json=payload))
|
||||
results = await firecrawl.search(
|
||||
{"firecrawl_api_key_encrypted": encrypt("fc-x")}, "q", 5
|
||||
)
|
||||
assert [r.url for r in results] == ["https://one.test"]
|
||||
|
||||
|
||||
async def test_firecrawl_reports_a_rejected_key(mock_http):
|
||||
mock_http(lambda _r: httpx.Response(401, json={"error": "bad key"}))
|
||||
with pytest.raises(SearchError) as caught:
|
||||
await firecrawl.search({"firecrawl_api_key_encrypted": encrypt("fc-x")}, "q", 5)
|
||||
assert "rejected" in caught.value.message
|
||||
|
||||
|
||||
async def test_firecrawl_needs_a_key():
|
||||
with pytest.raises(SearchError):
|
||||
await firecrawl.search({"firecrawl_api_key_encrypted": ""}, "q", 5)
|
||||
|
||||
|
||||
# --- Dispatch ----------------------------------------------------------------
|
||||
async def test_run_refuses_an_empty_query():
|
||||
with pytest.raises(SearchError):
|
||||
await search_service.run({"provider": "ddgs"}, " ")
|
||||
|
||||
|
||||
async def test_run_refuses_an_unknown_provider():
|
||||
with pytest.raises(SearchError):
|
||||
await search_service.run({"provider": "askjeeves"}, "q")
|
||||
|
||||
|
||||
async def test_the_administrators_limit_is_a_ceiling(mock_http):
|
||||
"""A model asking for fifty results is asking for a prompt nobody can
|
||||
afford."""
|
||||
rows = [{"title": f"r{i}", "url": f"https://x{i}.test", "content": ""} for i in range(50)]
|
||||
mock_http(lambda _r: httpx.Response(200, json={"results": rows}))
|
||||
|
||||
config = {"provider": "searxng", "searxng_base_url": "http://searx", "max_results": 3}
|
||||
assert len(await search_service.run(config, "q", limit=50)) == 3
|
||||
|
||||
|
||||
# --- Admin -------------------------------------------------------------------
|
||||
def test_search_settings_round_trip(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/search",
|
||||
data={
|
||||
"enabled": "true",
|
||||
"provider": "searxng",
|
||||
"max_results": "7",
|
||||
"region": "uk-en",
|
||||
"safesearch": "strict",
|
||||
"searxng_base_url": "http://searx:8888/",
|
||||
"timeout": "30",
|
||||
},
|
||||
follow_redirects=False,
|
||||
)
|
||||
values = settings_store.search(db)
|
||||
assert values["enabled"] is True
|
||||
assert values["provider"] == "searxng"
|
||||
assert values["max_results"] == 7
|
||||
# Trailing slash stripped, so the provider does not build a double slash.
|
||||
assert values["searxng_base_url"] == "http://searx:8888"
|
||||
|
||||
|
||||
def test_an_unknown_provider_falls_back_to_the_default(client: TestClient, db, registered):
|
||||
client.post(
|
||||
"/admin/search", data={"provider": "askjeeves"}, follow_redirects=False
|
||||
)
|
||||
assert settings_store.search(db)["provider"] == "ddgs"
|
||||
|
||||
|
||||
def test_results_per_search_is_bounded(client: TestClient, db, registered):
|
||||
client.post("/admin/search", data={"max_results": "500"}, follow_redirects=False)
|
||||
assert settings_store.search(db)["max_results"] == 20
|
||||
|
||||
|
||||
def test_the_search_page_renders_every_provider(client: TestClient, registered):
|
||||
page = client.get("/admin/search").text
|
||||
for provider in search_service.PROVIDERS:
|
||||
assert provider.label in page
|
||||
@@ -0,0 +1,227 @@
|
||||
"""Tool calling: reassembling calls, deciding what is offered, running it."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from lembas.db.models import Chat, Connection, Model
|
||||
from lembas.services import settings_store
|
||||
from lembas.services import tools as tools_service
|
||||
from lembas.services.llm.openai_client import delta_tool_calls, finish_reason
|
||||
from lembas.services.search.base import SearchError, SearchResult
|
||||
|
||||
|
||||
# --- Reading the stream ------------------------------------------------------
|
||||
def test_delta_tool_calls_reads_the_normal_shape():
|
||||
chunk = {"choices": [{"delta": {"tool_calls": [{"index": 0, "id": "a"}]}}]}
|
||||
assert delta_tool_calls(chunk) == [{"index": 0, "id": "a"}]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"chunk", [{}, {"choices": []}, {"choices": [{}]}, {"choices": [{"delta": {}}]}]
|
||||
)
|
||||
def test_delta_tool_calls_tolerates_junk(chunk):
|
||||
assert delta_tool_calls(chunk) == []
|
||||
|
||||
|
||||
def test_finish_reason_is_read_when_present():
|
||||
assert finish_reason({"choices": [{"finish_reason": "tool_calls"}]}) == "tool_calls"
|
||||
assert finish_reason({"choices": [{}]}) == ""
|
||||
|
||||
|
||||
# --- The accumulator ---------------------------------------------------------
|
||||
def test_arguments_split_across_chunks_are_rejoined():
|
||||
"""Arguments arrive one token at a time; the id and name arrive once, on
|
||||
the first fragment only. This is the real shape llama.cpp produces."""
|
||||
accumulator = tools_service.ToolCallAccumulator()
|
||||
accumulator.feed(
|
||||
[{"index": 0, "id": "c1", "function": {"name": "web_search", "arguments": "{"}}]
|
||||
)
|
||||
for piece in ['"query"', ":", '"mallorn', ' tree"', "}"]:
|
||||
accumulator.feed([{"index": 0, "function": {"arguments": piece}}])
|
||||
|
||||
assert accumulator.calls == [
|
||||
{"id": "c1", "name": "web_search", "arguments": '{"query":"mallorn tree"}'}
|
||||
]
|
||||
|
||||
|
||||
def test_two_calls_are_kept_apart_by_index():
|
||||
"""Not by name: a model calling the same tool twice in one turn is exactly
|
||||
the case that breaks if name is the key."""
|
||||
accumulator = tools_service.ToolCallAccumulator()
|
||||
accumulator.feed(
|
||||
[
|
||||
{"index": 0, "id": "a", "function": {"name": "web_search", "arguments": '{"q":1}'}},
|
||||
{"index": 1, "id": "b", "function": {"name": "web_search", "arguments": '{"q":2}'}},
|
||||
]
|
||||
)
|
||||
assert [c["id"] for c in accumulator.calls] == ["a", "b"]
|
||||
assert accumulator.calls[1]["arguments"] == '{"q":2}'
|
||||
|
||||
|
||||
def test_a_missing_index_is_treated_as_the_only_call():
|
||||
"""Some servers omit index entirely when there is just one call."""
|
||||
accumulator = tools_service.ToolCallAccumulator()
|
||||
accumulator.feed([{"function": {"name": "web_search", "arguments": "{}"}}])
|
||||
assert len(accumulator.calls) == 1
|
||||
|
||||
|
||||
def test_a_call_with_no_name_is_not_a_call():
|
||||
accumulator = tools_service.ToolCallAccumulator()
|
||||
accumulator.feed([{"index": 0, "function": {"arguments": "{}"}}])
|
||||
assert accumulator.calls == []
|
||||
assert not accumulator
|
||||
|
||||
|
||||
def test_an_id_is_invented_when_the_server_supplies_none():
|
||||
"""The id is required when the results are sent back, and not every server
|
||||
provides one."""
|
||||
accumulator = tools_service.ToolCallAccumulator()
|
||||
accumulator.feed([{"index": 0, "function": {"name": "web_search", "arguments": "{}"}}])
|
||||
assert accumulator.calls[0]["id"]
|
||||
|
||||
|
||||
# --- What gets offered -------------------------------------------------------
|
||||
def _chat_with(db, user_id, *, capabilities):
|
||||
connection = Connection(name="c", base_url="http://127.0.0.1:1", api_key_encrypted="")
|
||||
db.add(connection)
|
||||
db.commit()
|
||||
db.add(
|
||||
Model(connection_id=connection.id, model_id="m", capabilities_json=capabilities)
|
||||
)
|
||||
db.commit()
|
||||
chat = Chat(user_id=user_id, model_id="m", connection_id=connection.id)
|
||||
db.add(chat)
|
||||
db.commit()
|
||||
return chat
|
||||
|
||||
|
||||
def _user(db, user_id):
|
||||
from lembas.db.models import User
|
||||
|
||||
return db.get(User, user_id)
|
||||
|
||||
|
||||
def test_nothing_is_offered_when_search_is_off(db, user_id):
|
||||
chat = _chat_with(db, user_id, capabilities={"tools": True})
|
||||
assert tools_service.enabled_tools(db, chat, _user(db, user_id)) == []
|
||||
|
||||
|
||||
def test_nothing_is_offered_to_a_model_without_the_tools_capability(db, user_id):
|
||||
"""Sending a tools array to an endpoint that does not implement tool calling
|
||||
fails the entire request, exactly as image parts do without vision."""
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat = _chat_with(db, user_id, capabilities={})
|
||||
assert tools_service.enabled_tools(db, chat, _user(db, user_id)) == []
|
||||
|
||||
|
||||
def test_web_search_is_offered_when_everything_lines_up(db, user_id):
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat = _chat_with(db, user_id, capabilities={"tools": True})
|
||||
|
||||
offered = tools_service.enabled_tools(db, chat, _user(db, user_id))
|
||||
assert len(offered) == 1
|
||||
assert offered[0]["function"]["name"] == "web_search"
|
||||
|
||||
|
||||
def test_nothing_is_offered_when_the_provider_cannot_run(db, user_id, monkeypatch):
|
||||
"""Offering a tool that will fail on every call is worse than not offering
|
||||
it at all."""
|
||||
monkeypatch.setattr(
|
||||
"lembas.services.search.availability", lambda _key: "ddgs is not installed."
|
||||
)
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat = _chat_with(db, user_id, capabilities={"tools": True})
|
||||
assert tools_service.enabled_tools(db, chat, _user(db, user_id)) == []
|
||||
|
||||
|
||||
# --- Running one -------------------------------------------------------------
|
||||
async def test_running_web_search_formats_results_for_the_model(monkeypatch):
|
||||
async def fake_run(_config, query, *, limit=None):
|
||||
return [SearchResult("A title", "https://a.test", "a snippet")]
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", fake_run)
|
||||
|
||||
outcome = await tools_service.run_tool({}, "web_search", '{"query": "mallorn"}')
|
||||
assert "A title" in outcome.content
|
||||
assert "https://a.test" in outcome.content
|
||||
assert outcome.event["status"] == "ok"
|
||||
assert outcome.event["results"][0]["host"] == "a.test"
|
||||
|
||||
|
||||
async def test_malformed_argument_json_is_treated_as_the_query(monkeypatch):
|
||||
"""Small models emit broken argument JSON often enough that this is a normal
|
||||
path, not an exceptional one."""
|
||||
seen = {}
|
||||
|
||||
async def fake_run(_config, query, *, limit=None):
|
||||
seen["query"] = query
|
||||
return []
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", fake_run)
|
||||
await tools_service.run_tool({}, "web_search", "mallorn tree")
|
||||
assert seen["query"] == "mallorn tree"
|
||||
|
||||
|
||||
async def test_a_failed_search_hands_the_model_an_explanation(monkeypatch):
|
||||
"""A failed search should produce "I could not look that up", not kill the
|
||||
whole reply."""
|
||||
|
||||
async def fake_run(_config, _query, *, limit=None):
|
||||
raise SearchError("DuckDuckGo is rate limiting this instance.")
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", fake_run)
|
||||
|
||||
outcome = await tools_service.run_tool({}, "web_search", '{"query": "x"}')
|
||||
assert "rate limiting" in outcome.content
|
||||
assert outcome.event["status"] == "error"
|
||||
|
||||
|
||||
async def test_an_unknown_tool_is_reported_rather_than_raised():
|
||||
outcome = await tools_service.run_tool({}, "launch_missiles", "{}")
|
||||
assert "no tool called" in outcome.content
|
||||
assert outcome.event["status"] == "error"
|
||||
|
||||
|
||||
async def test_a_call_with_no_query_is_reported():
|
||||
outcome = await tools_service.run_tool({}, "web_search", '{"query": " "}')
|
||||
assert outcome.event["status"] == "error"
|
||||
|
||||
|
||||
# --- The turns sent back -----------------------------------------------------
|
||||
def test_the_assistant_turn_echoes_the_calls():
|
||||
"""The endpoint needs its own tool_calls back before the tool replies, or it
|
||||
has nothing to match the tool_call_ids against."""
|
||||
turn = tools_service.assistant_turn(
|
||||
[{"id": "c1", "name": "web_search", "arguments": '{"query":"x"}'}], "Looking it up."
|
||||
)
|
||||
assert turn["role"] == "assistant"
|
||||
assert turn["content"] == "Looking it up."
|
||||
assert turn["tool_calls"][0]["id"] == "c1"
|
||||
assert turn["tool_calls"][0]["function"]["name"] == "web_search"
|
||||
|
||||
|
||||
def test_an_empty_assistant_message_becomes_null():
|
||||
"""Most endpoints reject an assistant turn whose content is an empty string
|
||||
alongside tool_calls."""
|
||||
turn = tools_service.assistant_turn([{"id": "c", "name": "n", "arguments": "{}"}], "")
|
||||
assert turn["content"] is None
|
||||
|
||||
|
||||
def test_the_tool_turn_carries_the_call_id():
|
||||
turn = tools_service.tool_turn({"id": "c1", "name": "web_search"}, "results here")
|
||||
assert turn == {
|
||||
"role": "tool",
|
||||
"tool_call_id": "c1",
|
||||
"name": "web_search",
|
||||
"content": "results here",
|
||||
}
|
||||
|
||||
|
||||
def test_the_schema_is_valid_json():
|
||||
"""It is sent verbatim to the endpoint; a schema that will not serialise
|
||||
fails every request rather than one."""
|
||||
json.dumps(tools_service.WEB_SEARCH_SCHEMA)
|
||||
assert tools_service.WEB_SEARCH_SCHEMA["function"]["parameters"]["required"] == ["query"]
|
||||
Reference in New Issue
Block a user