A time in no particular zone, and a preview missing what it previews
The first audit pass: everything from 0.8.1 to 0.9.8 read as a whole rather than one feature at a time, starting with what a model is actually told. Four of these had shipped as correct. The date line carried a timezone variable that resolves to nothing until somebody chooses one -- so every default account was told times were "in unless they say otherwise", while two comments asserted the line disappeared instead. The prompt preview built its variables without a chat, which is what eleven fragments are gated on, so the whole agent surface was absent from it whatever was ticked. Plan mode was instructed to keep its plan current with a tool that mode withdraws. And knowledge_get returned a document whole where every sibling reader caps and says so, its description promising exactly that. The subagent guidance was wrong in both directions at once: it denied a documented parameter and named seven of twenty-three allowed commands. Both halves are pinned by tests against the real list and the real schema now, because prose and a constant drift the moment one is edited alone. docs/notes/audit-0.9.md carries the findings that are not fixed here, with why -- the ones whose fix would change what a feature does are the user's call, not this pass's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -986,6 +986,67 @@ async def test_an_ordinary_chat_is_not_asked_to_narrate(db, user_id, machine):
|
||||
assert "Settle what you are setting out to achieve" not in text
|
||||
|
||||
|
||||
def _give_a_plan(db, chat):
|
||||
"""A plan on the chat, the way `_plan_of` finds one: a message carrying it
|
||||
and the chat pointing at that message."""
|
||||
from lembas.services import plans
|
||||
|
||||
message = Message(
|
||||
chat_id=chat.id,
|
||||
role=ROLE_ASSISTANT,
|
||||
content="",
|
||||
plan_json=plans.build(title="Fix the parser", steps="Read it\nChange it"),
|
||||
)
|
||||
db.add(message)
|
||||
db.commit()
|
||||
chat.plan_message_id = message.id
|
||||
db.commit()
|
||||
|
||||
|
||||
async def test_plan_mode_is_not_told_to_use_a_tool_it_does_not_have(db, user_id, machine):
|
||||
"""`agent/tools.py` withdraws `plan_update` in Plan mode -- that mode ends
|
||||
with `plan_submit` instead. Its guidance was gated on {{plan}}, which is set
|
||||
whenever a plan exists in any mode, so a model in Plan mode was told to
|
||||
"keep it current with plan_update as you go" about a tool that was not in
|
||||
its list, directly under `core.tool_list` saying anything unnamed does not
|
||||
exist. The fragment's own hint asserted the two coincided.
|
||||
"""
|
||||
from lembas.services import harness
|
||||
|
||||
chat, _profile = _setup(db, user_id, machine, mode=policy.MODE_PLAN)
|
||||
_give_a_plan(db, chat)
|
||||
user = db.get(User, user_id)
|
||||
|
||||
offered = tools_service.resolve_tools(db, chat, user)
|
||||
assert "plan_update" not in offered.by_name
|
||||
|
||||
values = harness.context_variables(db, user, offered.schemas, chat)
|
||||
# The plan is still shown -- a model that cannot see it cannot submit a
|
||||
# better one. Only the instruction to *edit* it goes.
|
||||
assert values["plan"]
|
||||
assert values["plan_editable"] == ""
|
||||
|
||||
text = harness.compose(db, user, offered.schemas, chat)
|
||||
assert "Fix the parser" in text
|
||||
assert "plan_update" not in text
|
||||
|
||||
|
||||
async def test_a_mode_that_has_plan_update_is_still_told_about_it(db, user_id, machine):
|
||||
"""The other half, or the gate is just a deletion."""
|
||||
from lembas.services import harness
|
||||
|
||||
chat, _profile = _setup(db, user_id, machine, mode=policy.MODE_EDIT)
|
||||
_give_a_plan(db, chat)
|
||||
user = db.get(User, user_id)
|
||||
|
||||
offered = tools_service.resolve_tools(db, chat, user)
|
||||
assert "plan_update" in offered.by_name
|
||||
|
||||
values = harness.context_variables(db, user, offered.schemas, chat)
|
||||
assert values["plan_editable"] == values["plan"] != ""
|
||||
assert "plan_update" in harness.compose(db, user, offered.schemas, chat)
|
||||
|
||||
|
||||
async def test_an_agent_chat_is_told_its_real_round_budget(db, user_id, machine):
|
||||
"""MAX_ROUNDS is one. An agent chat gets hundreds, and telling it one would
|
||||
be a false fact about its own budget on every turn."""
|
||||
|
||||
Reference in New Issue
Block a user