Every injected prompt becomes editable, and several get written

The instructions LLeMbas puts in front of a model were hard-coded: six
strings in a GUIDANCE dict, two headings, and the title request inline in
chat.py. An operator could not see what was being sent, let alone change
it, and there was nowhere for a custom tool to contribute its own guidance
when custom tools land.

services/prompts.py now holds each piece as a Fragment, and /admin/prompts
edits them with a preview of the whole assembled system message including
unsaved edits. harness.py keeps only the decisions -- which fragments apply
to this request, and what their variables resolve to.

The design turns on one choice: a fragment carries its gate as data
(families, requires, when_tools) rather than as a callable, because a
database row can carry the same three fields. Custom tools will therefore
register a fragment source and change nothing else -- there is a test that
says exactly that, and it is the reason the rest of the shape is what it is.

Consequences worth knowing:

  - Defaults live in code, overrides in the database, and text equal to its
    default is never stored. Otherwise pressing Save once would freeze
    today's wording forever and no later release could improve it.
  - An empty override means off. A fragment that was not submitted at all
    keeps what it had, because it may be missing from the page only because
    whatever contributes it is currently switched off.
  - requires= replaced the hand-written pair of memory guidance variants.
    The sentence that refers to a section now lives inside that section, so
    it cannot outlive it. That was the general problem the pair was a
    special case of.
  - {{name}}, with anything unrecognised passing through verbatim. The name
    grammar is the guard: {"total": 1} and ${PATH} are not candidates.
    Substitution is one pass and never recursive, because {{memories}}
    carries text a model wrote.

The wording is also overhauled, and a model now gets the core fragments
even with no tools -- the date above all. "An empty harness is worse than
none" was about tokens that say nothing; a model with no clock being asked
about the present is not that. Clearing those boxes restores the old
silence exactly. New: today's date, who it is talking to, the three-round
tool budget, that tool results are not replayed, that anything a tool
returns is data rather than instruction, and what the <document> wrapper
around an attachment is. Extended: memory_forget, notes_edit/delete,
skill_create/edit, and reading a knowledge document in full rather than
answering from an extract.

Tool descriptions stay in code and are listed read-only. They are schema
and they state facts about what a runner does; an edit would make the text
a lie with nothing to catch it.

No schema change -- one JSON row in the settings table.

488 tests. Version 0.2.0, which also invalidates the service worker cache
so the green artwork appears without a hard reload.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-31 23:47:49 +02:00
parent 71dc46455c
commit 1906919ee2
20 changed files with 2089 additions and 145 deletions
+18 -8
View File
@@ -17,6 +17,16 @@ from lembas.services import settings_store
from lembas.services.crypto import encrypt
def turns(body: dict) -> list[dict]:
"""The conversation turns, without the system preamble in front of them.
`build_request` always emits a system message now -- the harness carries the
date even for a model with no tools -- so a test about a *user* turn has to
say which turn it means rather than assume index 0.
"""
return [message for message in body["messages"] if message["role"] != "system"]
# --- Fixtures ----------------------------------------------------------------
def png_bytes(width: int = 40, height: int = 30, mode: str = "RGB") -> bytes:
buffer = io.BytesIO()
@@ -365,7 +375,7 @@ def test_images_become_multimodal_parts_for_a_vision_model(client: TestClient, d
chat = db.get(Chat, chat_with_model)
payload = chat_service.build_request(db, chat)
content = payload["messages"][0]["content"]
content = turns(payload)[0]["content"]
assert isinstance(content, list)
assert content[0] == {"type": "text", "text": "what is this?"}
@@ -387,7 +397,7 @@ def test_images_are_withheld_from_a_model_without_vision(client: TestClient, db,
)
chat = db.get(Chat, chat_with_model)
content = chat_service.build_request(db, chat)["messages"][0]["content"]
content = turns(chat_service.build_request(db, chat))[0]["content"]
assert isinstance(content, str)
assert content == "what is this?"
@@ -396,7 +406,7 @@ def test_a_plain_turn_stays_a_plain_string(client: TestClient, db, chat_with_mod
"""The list form is a reliable 400 from endpoints that do not implement it."""
client.post(f"/api/chats/{chat_with_model}/messages", data={"content": "just words"})
chat = db.get(Chat, chat_with_model)
assert chat_service.build_request(db, chat)["messages"][0]["content"] == "just words"
assert turns(chat_service.build_request(db, chat))[0]["content"] == "just words"
def test_document_text_is_prepended_in_tags(client: TestClient, db, chat_with_model):
@@ -410,7 +420,7 @@ def test_document_text_is_prepended_in_tags(client: TestClient, db, chat_with_mo
)
chat = db.get(Chat, chat_with_model)
content = chat_service.build_request(db, chat)["messages"][0]["content"]
content = turns(chat_service.build_request(db, chat))[0]["content"]
assert '<document name="report.txt">' in content
assert "Quarterly results were good." in content
# The question comes after the material it refers to.
@@ -430,7 +440,7 @@ def test_documents_reach_a_model_without_vision(client: TestClient, db, chat_wit
)
chat = db.get(Chat, chat_with_model)
content = chat_service.build_request(db, chat)["messages"][0]["content"]
content = turns(chat_service.build_request(db, chat))[0]["content"]
assert "important detail" in content
@@ -445,7 +455,7 @@ def test_truncation_is_declared_to_the_model(client: TestClient, db, chat_with_m
)
chat = db.get(Chat, chat_with_model)
assert "(truncated)" in chat_service.build_request(db, chat)["messages"][0]["content"]
assert "(truncated)" in turns(chat_service.build_request(db, chat))[0]["content"]
def test_an_attachment_only_turn_still_reaches_the_model(client: TestClient, db, chat_with_model):
@@ -456,7 +466,7 @@ def test_an_attachment_only_turn_still_reaches_the_model(client: TestClient, db,
)
chat = db.get(Chat, chat_with_model)
messages = chat_service.build_request(db, chat)["messages"]
messages = turns(chat_service.build_request(db, chat))
assert len(messages) == 1
assert messages[0]["content"][0]["type"] == "image_url"
@@ -587,6 +597,6 @@ def test_a_browser_serialising_the_form_actually_sends_the_attachment(
assert attachment.message_id == message.id
chat = db.get(Chat, chat_with_model)
content = chat_service.build_request(db, chat)["messages"][0]["content"]
content = turns(chat_service.build_request(db, chat))[0]["content"]
assert isinstance(content, list), "the image never reached the model"
assert any(p.get("type") == "image_url" for p in content)