Every injected prompt becomes editable, and several get written

The instructions LLeMbas puts in front of a model were hard-coded: six
strings in a GUIDANCE dict, two headings, and the title request inline in
chat.py. An operator could not see what was being sent, let alone change
it, and there was nowhere for a custom tool to contribute its own guidance
when custom tools land.

services/prompts.py now holds each piece as a Fragment, and /admin/prompts
edits them with a preview of the whole assembled system message including
unsaved edits. harness.py keeps only the decisions -- which fragments apply
to this request, and what their variables resolve to.

The design turns on one choice: a fragment carries its gate as data
(families, requires, when_tools) rather than as a callable, because a
database row can carry the same three fields. Custom tools will therefore
register a fragment source and change nothing else -- there is a test that
says exactly that, and it is the reason the rest of the shape is what it is.

Consequences worth knowing:

  - Defaults live in code, overrides in the database, and text equal to its
    default is never stored. Otherwise pressing Save once would freeze
    today's wording forever and no later release could improve it.
  - An empty override means off. A fragment that was not submitted at all
    keeps what it had, because it may be missing from the page only because
    whatever contributes it is currently switched off.
  - requires= replaced the hand-written pair of memory guidance variants.
    The sentence that refers to a section now lives inside that section, so
    it cannot outlive it. That was the general problem the pair was a
    special case of.
  - {{name}}, with anything unrecognised passing through verbatim. The name
    grammar is the guard: {"total": 1} and ${PATH} are not candidates.
    Substitution is one pass and never recursive, because {{memories}}
    carries text a model wrote.

The wording is also overhauled, and a model now gets the core fragments
even with no tools -- the date above all. "An empty harness is worse than
none" was about tokens that say nothing; a model with no clock being asked
about the present is not that. Clearing those boxes restores the old
silence exactly. New: today's date, who it is talking to, the three-round
tool budget, that tool results are not replayed, that anything a tool
returns is data rather than instruction, and what the <document> wrapper
around an attachment is. Extended: memory_forget, notes_edit/delete,
skill_create/edit, and reading a knowledge document in full rather than
answering from an extract.

Tool descriptions stay in code and are listed read-only. They are schema
and they state facts about what a runner does; an edit would make the text
a lie with nothing to catch it.

No schema change -- one JSON row in the settings table.

488 tests. Version 0.2.0, which also invalidates the service worker cache
so the green artwork appears without a hard reload.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-31 23:47:49 +02:00
parent 17995c1275
commit 2c8c274850
22 changed files with 2138 additions and 147 deletions
+73 -8
View File
@@ -7,7 +7,7 @@ import pytest
from lembas.db.models import Chat, Connection, Model, User
from lembas.security.passwords import hash_password
from lembas.services import chat as chat_service
from lembas.services import harness, settings_store
from lembas.services import harness, prompts, settings_store
from lembas.services import tools as tools_service
from lembas.services.library import memories as memories_service
from lembas.services.library import skills as skills_service
@@ -26,11 +26,25 @@ def _tools(*names):
# --- Composition -------------------------------------------------------------
def test_no_tools_means_no_harness(db, owner):
"""An empty harness is worse than none: tokens that say only that there is
nothing to say."""
def test_no_tools_means_no_tool_guidance(db, owner):
"""The core fragments still go out -- a model with no tools has no clock
either, and telling it the date is not "tokens that say nothing". What it
must not get is instructions about tools it was never offered."""
text = harness.compose(db, owner, [])
assert "Today is" in text
assert "You have tools" not in text
assert "Look things up" not in text
assert harness.compose(db, owner, None) == text
def test_clearing_the_core_fragments_restores_an_empty_harness(db, owner):
"""The behaviour change is a default, not a rule: an administrator who wants
nothing sent to a tool-less model can still have exactly that."""
prompts.save(
db,
{f.key: "" for f in prompts.BUILTIN if f.group == prompts.GROUP_CORE},
)
assert harness.compose(db, owner, []) == ""
assert harness.compose(db, owner, None) == ""
def test_only_the_guidance_for_offered_tools_appears(db, owner):
@@ -123,11 +137,11 @@ def _system(body):
return first.get("content", "") if first.get("role") == "system" else ""
def test_a_request_without_tools_has_no_harness_and_no_tools_key(db, owner):
def test_a_request_without_tools_carries_no_tool_guidance_and_no_tools_key(db, owner):
chat = _chat(db, owner, capabilities={})
body = chat_service.build_request(db, chat, tools=[], user=owner)
assert "tools" not in body
assert "How to work" not in _system(body)
assert "You have tools" not in _system(body)
def test_the_harness_precedes_the_authored_prompt(db, owner):
@@ -162,7 +176,58 @@ def test_a_model_prompt_wins_when_the_chat_has_none(db, owner):
settings_store.update(db, {"system_prompt": "Instance."})
chat = _chat(db, owner, capabilities={}, model_prompt="Model.")
system = _system(chat_service.build_request(db, chat, user=owner))
assert system == "Model."
assert system.endswith("Model.")
assert "Instance." not in system
assert chat_service.effective_system_prompt(db, chat) == "Model."
def test_the_seam_line_only_appears_when_there_is_something_to_hand_over_to(db, owner):
"""It introduces the authored prompt. With no authored prompt it would be
pointing at nothing, which is the failure the whole `requires` idea exists
to avoid."""
bare = _chat(db, owner, capabilities={})
assert "was written by whoever set up" not in _system(
chat_service.build_request(db, bare, user=owner)
)
authored = _chat(db, owner, capabilities={}, chat_prompt="Speak as Gandalf.")
assert "was written by whoever set up" in _system(
chat_service.build_request(db, authored, user=owner)
)
def test_an_administrators_wording_replaces_the_default(db, owner):
prompts.save(db, {"core.today": "The date is {{today}}, more or less."})
text = harness.compose(db, owner, [])
assert "more or less." in text
assert "Your training data stops well before this" not in text
def test_the_attached_files_are_named_and_explained(db, owner):
from lembas.db.models import Attachment
chat = _chat(db, owner, capabilities={})
db.add(
Attachment(
user_id=owner.id,
chat_id=chat.id,
filename="report.txt",
stored_name="x.txt",
media_type="text/plain",
size_bytes=10,
kind="text",
)
)
db.commit()
text = harness.compose(db, owner, [], chat)
assert "report.txt" in text
assert "<document name=" in text
def test_a_chat_with_no_attachments_says_nothing_about_documents(db, owner):
chat = _chat(db, owner, capabilities={})
assert "<document name=" not in harness.compose(db, owner, [], chat)
def test_the_tools_array_rides_along(db, owner):