Every injected prompt becomes editable, and several get written

The instructions LLeMbas puts in front of a model were hard-coded: six
strings in a GUIDANCE dict, two headings, and the title request inline in
chat.py. An operator could not see what was being sent, let alone change
it, and there was nowhere for a custom tool to contribute its own guidance
when custom tools land.

services/prompts.py now holds each piece as a Fragment, and /admin/prompts
edits them with a preview of the whole assembled system message including
unsaved edits. harness.py keeps only the decisions -- which fragments apply
to this request, and what their variables resolve to.

The design turns on one choice: a fragment carries its gate as data
(families, requires, when_tools) rather than as a callable, because a
database row can carry the same three fields. Custom tools will therefore
register a fragment source and change nothing else -- there is a test that
says exactly that, and it is the reason the rest of the shape is what it is.

Consequences worth knowing:

  - Defaults live in code, overrides in the database, and text equal to its
    default is never stored. Otherwise pressing Save once would freeze
    today's wording forever and no later release could improve it.
  - An empty override means off. A fragment that was not submitted at all
    keeps what it had, because it may be missing from the page only because
    whatever contributes it is currently switched off.
  - requires= replaced the hand-written pair of memory guidance variants.
    The sentence that refers to a section now lives inside that section, so
    it cannot outlive it. That was the general problem the pair was a
    special case of.
  - {{name}}, with anything unrecognised passing through verbatim. The name
    grammar is the guard: {"total": 1} and ${PATH} are not candidates.
    Substitution is one pass and never recursive, because {{memories}}
    carries text a model wrote.

The wording is also overhauled, and a model now gets the core fragments
even with no tools -- the date above all. "An empty harness is worse than
none" was about tokens that say nothing; a model with no clock being asked
about the present is not that. Clearing those boxes restores the old
silence exactly. New: today's date, who it is talking to, the three-round
tool budget, that tool results are not replayed, that anything a tool
returns is data rather than instruction, and what the <document> wrapper
around an attachment is. Extended: memory_forget, notes_edit/delete,
skill_create/edit, and reading a knowledge document in full rather than
answering from an extract.

Tool descriptions stay in code and are listed read-only. They are schema
and they state facts about what a runner does; an edit would make the text
a lie with nothing to catch it.

No schema change -- one JSON row in the settings table.

488 tests. Version 0.2.0, which also invalidates the service worker cache
so the green artwork appears without a hard reload.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-31 23:47:49 +02:00
parent 17995c1275
commit 2c8c274850
22 changed files with 2138 additions and 147 deletions
+147
View File
@@ -0,0 +1,147 @@
"""The prompt editor: what it saves, what it refuses to save, and the preview."""
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select
from lembas.db.models import User
from lembas.services import harness, prompts, settings_store
from lembas.services.library import memories as memories_service
@pytest.fixture
def plain_user(client: TestClient, db, registered) -> User:
"""A second, non-admin account. Leaves the client signed in as them."""
client.post("/auth/logout", follow_redirects=False)
client.post(
"/auth/register",
data={"name": "Sam", "email": "sam@shire.test", "password": "potatoes-po-ta-toes"},
follow_redirects=False,
)
return db.scalar(select(User).where(User.email == "sam@shire.test"))
def _owner(db) -> User:
return db.scalar(select(User).where(User.email == "frodo@shire.test"))
# --- Access ------------------------------------------------------------------
def test_the_page_lists_every_fragment(client: TestClient, registered):
page = client.get("/admin/prompts").text
for fragment in prompts.BUILTIN:
assert f'name="prompt.{fragment.key}"' in page, fragment.key
def test_the_page_is_refused_to_a_plain_user(client: TestClient, plain_user):
assert client.get("/admin/prompts").status_code == 403
assert client.post("/admin/prompts", data={}).status_code == 403
# --- Saving ------------------------------------------------------------------
def test_saving_changes_what_a_model_is_told(client: TestClient, db, registered):
client.post(
"/admin/prompts",
data={"prompt.tool.web_search": "- Always search twice.", "max_harness_chars": "0"},
follow_redirects=False,
)
text = harness.compose(db, _owner(db), _tools("web_search"))
assert "- Always search twice." in text
assert "Look things up" not in text
def test_a_cleared_box_turns_the_fragment_off(client: TestClient, db, registered):
"""The trap this page was written around: FastAPI cannot tell an empty form
field from an absent one, so the handler reads the raw form."""
client.post(
"/admin/prompts",
data={"prompt.core.style": "", "max_harness_chars": "0"},
follow_redirects=False,
)
assert prompts.stored(db) == {"core.style": ""}
assert "Answer in the language" not in harness.compose(db, _owner(db), [])
def test_a_fragment_not_submitted_at_all_is_left_alone(client: TestClient, db, registered):
prompts.save(db, {"core.heading": "## Rules"})
client.post("/admin/prompts", data={"max_harness_chars": "0"}, follow_redirects=False)
assert prompts.resolve(db, "core.heading") == "## Rules"
def test_saving_the_built_in_wording_stores_nothing(client: TestClient, db, registered):
"""Opening the page and pressing Save must not freeze today's defaults, or a
later release could never improve them."""
page_fields = {f"prompt.{f.key}": f.default for f in prompts.BUILTIN}
client.post(
"/admin/prompts", data={**page_fields, "max_harness_chars": "0"}, follow_redirects=False
)
assert prompts.stored(db) == {}
def test_the_character_cap_is_clamped_and_kept(client: TestClient, db, registered):
client.post("/admin/prompts", data={"max_harness_chars": "-5"}, follow_redirects=False)
assert settings_store.get(db, "max_harness_chars", key=settings_store.PROMPTS) == 0
client.post("/admin/prompts", data={"max_harness_chars": "600"}, follow_redirects=False)
assert harness.limit_for(db) == 600
# --- Resetting ---------------------------------------------------------------
def test_use_default_fills_the_box_without_saving(client: TestClient, db, registered):
prompts.save(db, {"core.heading": "## Rules"})
response = client.post("/admin/prompts/default", data={"key": "core.heading"})
assert "## How to work" in response.text
assert "edited" not in response.text
# Nothing was written: it takes a Save to make it stick.
assert prompts.resolve(db, "core.heading") == "## Rules"
def test_use_default_on_an_unknown_key_is_a_404(client: TestClient, registered):
assert client.post("/admin/prompts/default", data={"key": "nope.nope"}).status_code == 404
def test_restore_all_defaults_empties_the_overrides(client: TestClient, db, registered):
prompts.save(db, {"core.heading": "## Rules", "core.style": ""})
client.post("/admin/prompts/reset", follow_redirects=False)
assert prompts.stored(db) == {}
# --- Preview -----------------------------------------------------------------
def test_the_preview_shows_text_that_has_not_been_saved(client: TestClient, db, registered):
body = client.post(
"/admin/prompts/preview",
data={"prompt.core.heading": "## Draft heading", "preview_family": ["web_search"]},
).text
assert "## Draft heading" in body
assert prompts.stored(db) == {}
def test_the_preview_only_shows_guidance_for_the_families_ticked(client: TestClient, registered):
body = client.post("/admin/prompts/preview", data={"preview_family": ["notes"]}).text
assert "You keep notes" in body
assert "Look things up" not in body
def test_the_preview_escapes_what_a_model_wrote(client: TestClient, db, registered):
"""A memory is model-written text on an admin page. Hard rule 6 applies to
the preview exactly as it does to a chat bubble."""
memories_service.add(db, owner=_owner(db), content="<img src=x onerror=alert(1)>")
body = client.post("/admin/prompts/preview", data={"preview_family": ["memory"]}).text
assert "<img src=x" not in body
assert "&lt;img src=x" in body
def test_the_preview_warns_when_the_cap_would_cut_it_off(client: TestClient, db, registered):
settings_store.update(db, {"max_harness_chars": 60}, key=settings_store.PROMPTS)
body = client.post("/admin/prompts/preview", data={"preview_family": ["web_search"]}).text
assert "past the cap is cut off" in body
def _tools(*names):
from lembas.services import tools as tools_service
return [tools_service.REGISTRY[name].schema for name in names]