An effort the model had never heard of
Reported from a live instance, on Bonsai: Jinja Exception: Unexpected reasoning effort high. Supported types are xhigh (default), medium, and low. Effort goes out two ways because no single field works, and the second -- chat_template_kwargs -- is not a parameter the server interprets. It is rendered into the model's own chat template, which does not ignore a value it does not know: it calls raise_exception, and the request dies before a token. So a perfectly ordinary option, drawn by this application in its own menu, took the whole reply with it. The vocabulary is per model and nobody agrees. gpt-oss takes low/medium/high. Bonsai takes low/medium/xhigh and refuses high. OpenAI has added minimal, xhigh and max at different points, and which of them a given model accepts varies again. One global tuple was going to be wrong for somebody whatever it held. A model carries its own list now, and the picker, the slash command and the request builder all read it. A column rather than a key in capabilities_json, for the reason context_length is one: that dict is rebuilt wholesale from the submitted checkboxes on every save. And it corrects itself. A refusal retries the reply once without the effort rather than losing it -- safe only because the template renders before any token, so nothing has been emitted, and there is a guard that keeps it that way -- then narrows the model's list. Bonsai's error states what it does take, so that is what gets stored. Note the parser bug, because it is a good one: "high" is a substring of "xhigh", so reading the advertised list by substring learned `high` from a sentence explaining that `high` is the problem. Whole words now, with a test named after it. /effort reads its levels off the picker instead of a second copy of the list kept in the browser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -19,7 +19,7 @@ from sqlalchemy import (
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
|
||||
from lembas.db.types import JSONDict
|
||||
from lembas.db.types import JSONDict, JSONList
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# Import only for the annotation; at runtime SQLAlchemy resolves the
|
||||
@@ -137,6 +137,20 @@ class Model(UUIDPrimaryKey, Timestamps, Base):
|
||||
# ticked anything.
|
||||
context_length: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
|
||||
# Which reasoning efforts this model actually accepts. Empty means "nobody
|
||||
# has said", and `services/chat.efforts_for` answers with the common set.
|
||||
#
|
||||
# It has to be per model, because the vocabulary is: gpt-oss takes
|
||||
# low/medium/high, Bonsai takes low/medium/xhigh and *raises* on high, and
|
||||
# OpenAI's own list has grown minimal, xhigh and max at different times. A
|
||||
# single global tuple is a guess that is wrong for somebody.
|
||||
#
|
||||
# ⚠ A column and not a key in `capabilities_json`, for exactly the reason
|
||||
# `context_length` is one: that dict is rebuilt wholesale from the submitted
|
||||
# checkboxes on every save, so anything in it that is not a checkbox is
|
||||
# destroyed the next time an administrator ticks anything.
|
||||
reasoning_efforts: Mapped[list[str]] = mapped_column(JSONList, default=list)
|
||||
|
||||
connection: Mapped[Connection] = relationship(back_populates="models")
|
||||
groups: Mapped[list[Group]] = relationship(
|
||||
"Group", secondary=model_groups, back_populates="models"
|
||||
|
||||
Reference in New Issue
Block a user