An effort the model had never heard of

Reported from a live instance, on Bonsai:

  Jinja Exception: Unexpected reasoning effort high. Supported types are
  xhigh (default), medium, and low.

Effort goes out two ways because no single field works, and the second --
chat_template_kwargs -- is not a parameter the server interprets. It is
rendered into the model's own chat template, which does not ignore a value it
does not know: it calls raise_exception, and the request dies before a token.
So a perfectly ordinary option, drawn by this application in its own menu, took
the whole reply with it.

The vocabulary is per model and nobody agrees. gpt-oss takes low/medium/high.
Bonsai takes low/medium/xhigh and refuses high. OpenAI has added minimal, xhigh
and max at different points, and which of them a given model accepts varies
again. One global tuple was going to be wrong for somebody whatever it held.

A model carries its own list now, and the picker, the slash command and the
request builder all read it. A column rather than a key in capabilities_json,
for the reason context_length is one: that dict is rebuilt wholesale from the
submitted checkboxes on every save.

And it corrects itself. A refusal retries the reply once without the effort
rather than losing it -- safe only because the template renders before any
token, so nothing has been emitted, and there is a guard that keeps it that way
-- then narrows the model's list. Bonsai's error states what it does take, so
that is what gets stored.

Note the parser bug, because it is a good one: "high" is a substring of
"xhigh", so reading the advertised list by substring learned `high` from a
sentence explaining that `high` is the problem. Whole words now, with a test
named after it.

/effort reads its levels off the picker instead of a second copy of the list
kept in the browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 21:20:06 +00:00
co-authored by Claude Opus 5
parent b7bf7d728b
commit 32e2326d41
10 changed files with 382 additions and 21 deletions
+16 -1
View File
@@ -177,7 +177,11 @@ async def model_detail(
"groups": list(db.scalars(select(Group).order_by(Group.name))),
"capabilities": PROTOCOL_CAPABILITIES,
"tool_capabilities": TOOL_CAPABILITIES,
# Every effort this application understands, so an administrator
# can tick the ones their model actually takes -- and the model's
# current answer, which is the common three until somebody says.
"efforts": chat_service.EFFORTS,
"model_efforts": chat_service.efforts_for(model),
# Rows predating the split have no tool_* keys at all. Showing them
# unticked would be a lie: tools.enabled_tools treats absent as on
# when `tools` is on, so that an upgrade does not silently take web
@@ -238,6 +242,7 @@ async def update_model(
position: str = Form(""),
context_length: str = Form(""),
default_effort: str = Form(""),
reasoning_efforts: list[str] = Form(default=[]),
group_ids: list[str] = Form(default=[]),
capability: list[str] = Form(default=[]),
) -> Response:
@@ -260,9 +265,19 @@ async def update_model(
# Merged rather than rebuilt, unlike the capabilities below: params_json
# holds whatever sampling defaults an administrator has set and this form
# only carries one of them.
# Which efforts this model takes at all. Submitted as a list of ticked
# values; empty means "nobody has said", and `chat.efforts_for` answers with
# the common three. Stored in the order `EFFORTS` declares rather than the
# order a browser happened to send.
chosen = [value for value in chat_service.EFFORTS if value in (reasoning_efforts or [])]
model.reasoning_efforts = chosen
params = dict(model.params_json or {})
wanted = default_effort.strip().lower()
if wanted in chat_service.EFFORTS:
# Checked against what this model takes, not against everything this
# application has heard of -- a default of `high` on a model whose template
# refuses it is a chat that fails on its first turn.
if wanted in chat_service.efforts_for(model):
params["reasoning_effort"] = wanted
else:
params.pop("reasoning_effort", None)
+6 -4
View File
@@ -82,10 +82,12 @@ def _chat_context(db: DBSession, user: User, chat: Chat | None) -> dict:
else []
),
"attached_base_ids": [base.id for base in chat.knowledge_bases] if chat else [],
# The three a reasoning model understands. From the service so the
# command, the control and the request builder cannot disagree about
# what is a valid effort.
"efforts": chat_service.EFFORTS,
# What *this* model takes, not the three every model used to be assumed
# to take. The vocabulary is per model -- gpt-oss has no `xhigh` and
# Bonsai has no `high`, and sending the wrong one does not degrade, it
# raises inside the chat template and fails the reply. From the service
# so the command, the control and the request builder cannot disagree.
"efforts": chat_service.efforts_for(current) if current else chat_service.DEFAULT_EFFORTS,
# What the picker shows, and what `build_request` will send. One
# resolver so the two cannot disagree.
"resolved_effort": chat_service.resolved_effort(chat) if chat else "",