An effort the model had never heard of
Reported from a live instance, on Bonsai: Jinja Exception: Unexpected reasoning effort high. Supported types are xhigh (default), medium, and low. Effort goes out two ways because no single field works, and the second -- chat_template_kwargs -- is not a parameter the server interprets. It is rendered into the model's own chat template, which does not ignore a value it does not know: it calls raise_exception, and the request dies before a token. So a perfectly ordinary option, drawn by this application in its own menu, took the whole reply with it. The vocabulary is per model and nobody agrees. gpt-oss takes low/medium/high. Bonsai takes low/medium/xhigh and refuses high. OpenAI has added minimal, xhigh and max at different points, and which of them a given model accepts varies again. One global tuple was going to be wrong for somebody whatever it held. A model carries its own list now, and the picker, the slash command and the request builder all read it. A column rather than a key in capabilities_json, for the reason context_length is one: that dict is rebuilt wholesale from the submitted checkboxes on every save. And it corrects itself. A refusal retries the reply once without the effort rather than losing it -- safe only because the template renders before any token, so nothing has been emitted, and there is a guard that keeps it that way -- then narrows the model's list. Bonsai's error states what it does take, so that is what gets stored. Note the parser bug, because it is a good one: "high" is a substring of "xhigh", so reading the advertised list by substring learned `high` from a sentence explaining that `high` is the problem. Whole words now, with a test named after it. /effort reads its levels off the picker instead of a second copy of the list kept in the browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -177,7 +177,11 @@ async def model_detail(
|
||||
"groups": list(db.scalars(select(Group).order_by(Group.name))),
|
||||
"capabilities": PROTOCOL_CAPABILITIES,
|
||||
"tool_capabilities": TOOL_CAPABILITIES,
|
||||
# Every effort this application understands, so an administrator
|
||||
# can tick the ones their model actually takes -- and the model's
|
||||
# current answer, which is the common three until somebody says.
|
||||
"efforts": chat_service.EFFORTS,
|
||||
"model_efforts": chat_service.efforts_for(model),
|
||||
# Rows predating the split have no tool_* keys at all. Showing them
|
||||
# unticked would be a lie: tools.enabled_tools treats absent as on
|
||||
# when `tools` is on, so that an upgrade does not silently take web
|
||||
@@ -238,6 +242,7 @@ async def update_model(
|
||||
position: str = Form(""),
|
||||
context_length: str = Form(""),
|
||||
default_effort: str = Form(""),
|
||||
reasoning_efforts: list[str] = Form(default=[]),
|
||||
group_ids: list[str] = Form(default=[]),
|
||||
capability: list[str] = Form(default=[]),
|
||||
) -> Response:
|
||||
@@ -260,9 +265,19 @@ async def update_model(
|
||||
# Merged rather than rebuilt, unlike the capabilities below: params_json
|
||||
# holds whatever sampling defaults an administrator has set and this form
|
||||
# only carries one of them.
|
||||
# Which efforts this model takes at all. Submitted as a list of ticked
|
||||
# values; empty means "nobody has said", and `chat.efforts_for` answers with
|
||||
# the common three. Stored in the order `EFFORTS` declares rather than the
|
||||
# order a browser happened to send.
|
||||
chosen = [value for value in chat_service.EFFORTS if value in (reasoning_efforts or [])]
|
||||
model.reasoning_efforts = chosen
|
||||
|
||||
params = dict(model.params_json or {})
|
||||
wanted = default_effort.strip().lower()
|
||||
if wanted in chat_service.EFFORTS:
|
||||
# Checked against what this model takes, not against everything this
|
||||
# application has heard of -- a default of `high` on a model whose template
|
||||
# refuses it is a chat that fails on its first turn.
|
||||
if wanted in chat_service.efforts_for(model):
|
||||
params["reasoning_effort"] = wanted
|
||||
else:
|
||||
params.pop("reasoning_effort", None)
|
||||
|
||||
Reference in New Issue
Block a user