Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

A time in no particular zone, and a preview missing what it previews

Browse Source 
The first audit pass: everything from 0.8.1 to 0.9.8 read as a whole rather
than one feature at a time, starting with what a model is actually told.

Four of these had shipped as correct. The date line carried a timezone
variable that resolves to nothing until somebody chooses one -- so every
default account was told times were "in  unless they say otherwise", while
two comments asserted the line disappeared instead. The prompt preview
built its variables without a chat, which is what eleven fragments are
gated on, so the whole agent surface was absent from it whatever was
ticked. Plan mode was instructed to keep its plan current with a tool that
mode withdraws. And knowledge_get returned a document whole where every
sibling reader caps and says so, its description promising exactly that.

The subagent guidance was wrong in both directions at once: it denied a
documented parameter and named seven of twenty-three allowed commands.
Both halves are pinned by tests against the real list and the real schema
now, because prose and a constant drift the moment one is edited alone.

docs/notes/audit-0.9.md carries the findings that are not fixed here, with
why -- the ones whose fix would change what a feature does are the user's
call, not this pass's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Homer
2026-08-07 09:13:54 +02:00
co-authored by Claude Opus 5
parent 0ce8026bd2
commit e970f10cca
13 changed files with 521 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/api/admin_prompts.py
+88 -8
View File
@@ -18,6 +18,7 @@ from lembas.services import harness as harness_service
from lembas.services import prompts as prompts_service
from lembas.services import settings_store
from lembas.services import tools as tools_service
from lembas.services.agent import policy
from lembas.web.templating import render
log = logging.getLogger(__name__)
@@ -28,6 +29,49 @@ router = APIRouter(prefix="/admin/prompts", tags=["admin-prompts"])
# in place rather than imagined. An administrator can clear the field.
SAMPLE_DOCUMENTS = "report.pdf, notes.txt"
# The rest of what a preview has to pretend, and the reason it must.
#
# `harness.context_variables` fills most `requires` gates only when it is handed
# a real `Chat` -- the machine, the directory, the plan, the project listing, a
# scheduled task's instruction, the flag saying this is a helper. The preview
# passes `chat=None`, so every one of those stayed empty and **eleven gated
# fragments could never appear in it at all**: the whole agent surface, both
# scheduling fragments, and the helper warning. An administrator editing
# `tool.agent` previewed a system message with `tool.agent` missing from it, and
# nothing said so.
#
# Samples rather than a transient Chat. `compose_from` takes plain variables
# precisely so this screen never has to build one, and a constructed row would
# need a connection, a profile and a directory that exist -- inventing an SSH
# host to render a paragraph is a worse trade than inventing the paragraph's
# values. This is what `SAMPLE_DOCUMENTS` has always done, extended to the rest.
SAMPLE_AGENT = {
"agent_target": "buildbox",
"agent_dir": "/srv/www/example",
"agent_rewound": "on 3 August at 14:20",
"background": "on",
"project_files": "src/\n app.py\n models.py\nREADME.md\npyproject.toml",
"agent_instructions": "Run the tests with `make check` before proposing a change.",
"agent_instructions_file": "AGENTS.md",
"plan": "1. [done] Read the failing test\n2. [doing] Fix the parser\n3. [todo] Add a case",
}
SAMPLE_SCHEDULE = {
"schedule_instruction": "Summarise what changed in the repository since yesterday.",
"schedule_summary": "every weekday at 08:00",
}
# Situations a chat can be in that are not a tool family, so nothing on the
# "Tools offered" row can reach them. `kind` and `parent_chat_id` in the model.
SITUATION_ORDINARY = ""
SITUATION_TASK = "task"
SITUATION_HELPER = "helper"
SITUATIONS = (
(SITUATION_ORDINARY, "An ordinary chat"),
(SITUATION_TASK, "A scheduled task, running unattended"),
(SITUATION_HELPER, "A helper sent by another model"),
)
def _families_of(db: Db, names: list[str]) -> list[str]:
"""Keep only real family names, in the registry's order.
@@ -54,6 +98,8 @@ def _variables(
model_name: str = "",
bases: str = "",
documents: str = "",
situation: str = SITUATION_ORDINARY,
mode: str = "",
) -> dict[str, str]:
"""The preview's variable values.
@@ -64,7 +110,14 @@ def _variables(
No Chat row is made. `harness.compose_from` takes plain variables precisely
so that this screen never has to build a transient one.
The samples are gated exactly as `context_variables` gates the real values --
the agent block on the `agent` family, the schedule and helper blocks on the
situation rather than on any family, because neither is a tool. A preview
that admitted a fragment the real request would not is worse than one that
omitted it, so the gating is mirrored rather than approximated.
"""
from lembas.services.agent import policy
from lembas.services.library import memories as memories_service
from lembas.services.library import skills as skills_service
@@ -79,6 +132,18 @@ def _variables(
"document_names": documents,
}
)
if "agent" in families:
values.update(SAMPLE_AGENT)
# A real one out of the table, not invented prose: this bullet *is* the
# mode guidance, so a made-up sentence here would preview wording that
# no request ever carries.
values["agent_mode"] = policy.MODE_GUIDANCE.get(mode, "") or policy.MODE_GUIDANCE[
policy.MODE_EDIT
]
if situation == SITUATION_TASK:
values.update(SAMPLE_SCHEDULE)
if situation == SITUATION_HELPER:
values["subagent"] = "yes"
return values
@@ -109,16 +174,27 @@ async def prompts_page(request: Request, db: Db, user: AdminUser, saved: bool =
"variables": prompts_service.VARIABLES,
# The legend shows what each name resolves to right now, with every
# family on -- a legend nobody can check is just a list of words.
"resolved": _variables(
db,
user,
families=families,
model_name=models[0].label if models else "",
bases="Contracts, Recipes",
documents=SAMPLE_DOCUMENTS,
),
# Every situation at once, unlike the preview: a chat is either a
# scheduled task or a helper and never both, but a legend is a
# reference rather than a rendering, and a name shown as empty
# because of the situation it was built in reads as a name that
# resolves to nothing.
"resolved": {
**_variables(
db,
user,
families=families,
model_name=models[0].label if models else "",
bases="Contracts, Recipes",
documents=SAMPLE_DOCUMENTS,
situation=SITUATION_TASK,
),
"subagent": "yes",
},
"models": models,
"families": families,
"situations": SITUATIONS,
"modes": policy.MODE_LABELS,
"registry": sorted(
tools_service.registry(db).values(), key=lambda t: (t.family, t.name)
),
@@ -173,6 +249,8 @@ async def preview(request: Request, db: Db, user: AdminUser):
model_name = str(form.get("preview_model") or "")
bases = str(form.get("preview_bases") or "").strip()
documents = str(form.get("preview_documents") or "").strip()
situation = str(form.get("preview_situation") or "")
mode = str(form.get("preview_mode") or "")
variables = _variables(
db,
@@ -181,6 +259,8 @@ async def preview(request: Request, db: Db, user: AdminUser):
model_name=model_name,
bases=bases,
documents=documents,
situation=situation,
mode=mode,
)
body = harness_service.compose_from(
db,