An effort the model had never heard of

Reported from a live instance, on Bonsai:

  Jinja Exception: Unexpected reasoning effort high. Supported types are
  xhigh (default), medium, and low.

Effort goes out two ways because no single field works, and the second --
chat_template_kwargs -- is not a parameter the server interprets. It is
rendered into the model's own chat template, which does not ignore a value it
does not know: it calls raise_exception, and the request dies before a token.
So a perfectly ordinary option, drawn by this application in its own menu, took
the whole reply with it.

The vocabulary is per model and nobody agrees. gpt-oss takes low/medium/high.
Bonsai takes low/medium/xhigh and refuses high. OpenAI has added minimal, xhigh
and max at different points, and which of them a given model accepts varies
again. One global tuple was going to be wrong for somebody whatever it held.

A model carries its own list now, and the picker, the slash command and the
request builder all read it. A column rather than a key in capabilities_json,
for the reason context_length is one: that dict is rebuilt wholesale from the
submitted checkboxes on every save.

And it corrects itself. A refusal retries the reply once without the effort
rather than losing it -- safe only because the template renders before any
token, so nothing has been emitted, and there is a guard that keeps it that way
-- then narrows the model's list. Bonsai's error states what it does take, so
that is what gets stored.

Note the parser bug, because it is a good one: "high" is a substring of
"xhigh", so reading the advertised list by substring learned `high` from a
sentence explaining that `high` is the problem. Whole words now, with a test
named after it.

/effort reads its levels off the picker instead of a second copy of the list
kept in the browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 21:20:06 +00:00
co-authored by Claude Opus 5
parent b7bf7d728b
commit 32e2326d41
10 changed files with 382 additions and 21 deletions
+58 -5
View File
@@ -387,7 +387,15 @@ def build_request(
):
body["tool_choice"] = {"type": "function", "function": {"name": force_tool}}
apply_effort(body, (chat.params_json or {}).get("reasoning_effort"))
# The model's own vocabulary, looked up here rather than passed in: every
# caller of `build_request` would otherwise have to remember, which is the
# trap `audio_service.template_flags` fell into.
chat_model = model_for(db, chat)
apply_effort(
body,
(chat.params_json or {}).get("reasoning_effort"),
efforts_for(chat_model) if chat_model is not None else None,
)
return body
@@ -405,7 +413,42 @@ def build_request(
# an effort on sends neither field and is byte-for-byte what it was. An endpoint
# strict about unknown parameters will refuse the extra one -- but on a chat
# somebody deliberately set an effort on, not on every chat in the instance.
EFFORTS = ("low", "medium", "high")
# Every reasoning effort this application understands, and the subset a model
# gets when nobody has said otherwise.
#
# 🚨 These are two different questions and conflating them is what broke a
# chat on Bonsai: `EFFORTS` was `("low", "medium", "high")` and was used both to
# validate what somebody chose *and* to decide what to offer, so a model whose
# vocabulary is low/medium/**xhigh** could not be given its own top setting,
# and the one it was given -- `high` -- made its chat template call
# `raise_exception` and took the whole reply with it.
#
# The known list is the union across providers, which have not agreed: OpenAI
# has added `minimal`, `xhigh` and `max` at different points; gpt-oss takes
# low/medium/high; Bonsai takes low/medium/xhigh and refuses high. `none` is
# deliberately absent -- this application already spells that `off`, and two
# spellings of off is the failure this codebase keeps cataloguing.
EFFORTS = ("minimal", "low", "medium", "high", "xhigh", "max")
# What a model is offered when its own list is empty. The three every reasoning
# model since the first one has understood.
DEFAULT_EFFORTS = ("low", "medium", "high")
def efforts_for(model) -> tuple[str, ...]:
"""The efforts this model accepts, in the order they should be offered.
A model's own list when an administrator has set one or the endpoint has
taught us one (see `generation._narrow_efforts`), and the common three
otherwise. Filtered against `EFFORTS` on the way out, so a value stored by
an older release -- or learned from an endpoint that advertised something
this application has never heard of -- cannot reach a request body.
"""
stored = list(getattr(model, "reasoning_efforts", None) or [])
chosen = [value for value in stored if value in EFFORTS]
if not chosen:
return DEFAULT_EFFORTS
return tuple(value for value in EFFORTS if value in chosen)
def resolved_effort(chat) -> str:
@@ -427,9 +470,19 @@ def resolved_effort(chat) -> str:
return value if value in EFFORTS else ""
def apply_effort(body: dict[str, Any], effort: str | None) -> None:
"""Put a chosen reasoning effort into a request body, in both forms."""
if not effort or effort not in EFFORTS:
def apply_effort(
body: dict[str, Any], effort: str | None, supported: tuple[str, ...] | None = None
) -> None:
"""Put a chosen reasoning effort into a request body, in both forms.
`supported` is the model's own vocabulary. An effort outside it is dropped
rather than sent, because the second form below is not advisory: it reaches
the model's Jinja chat template, and a template that does not know the value
raises rather than ignoring it -- which fails the whole request, not the
parameter.
"""
allowed = supported or DEFAULT_EFFORTS
if not effort or effort not in allowed:
return
body["reasoning_effort"] = effort
kwargs = dict(body.get("chat_template_kwargs") or {})