Reading the answer instead of asking somebody to know it
llama-server publishes the loaded model's Jinja chat template on /props, and
that template is the very thing that rejects a reasoning effort it does not
recognise -- so the accepted set is written down, authoritatively, in a place
this application can simply read. There is a button on the model's page that
does.
The parser handles both shapes a template uses: the values inline in the test
that rejects them (Bonsai), and a named list set elsewhere with nothing near
the mention spelling them out (gpt-oss). It is deliberately conservative,
because a wrong answer here silently removes a level somebody is entitled to:
only known efforts count, an unrelated list of quoted strings is ignored, and a
single match is read as a default -- `{%- set reasoning_effort = 'medium' %}`
-- rather than as a vocabulary of one.
An endpoint with no such route says so. OpenAI and vLLM do not publish a
template, and "this cannot tell us" must not be recorded as "this model accepts
nothing".
/props sits at the server root, beside the OpenAI-compatible surface rather
than inside it, so a base URL written as .../v1 needs the suffix stripped.
Getting that wrong is a silent 404 that looks like detection simply not
working, so there is a test on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -16,6 +16,17 @@ for 1.0.0 have something to be assembled from.
|
||||
|
||||
## Unreleased
|
||||
|
||||
## 1.3.0
|
||||
|
||||
- **A model's reasoning efforts can now be detected rather than known.** There
|
||||
is a button on the model's page that asks the endpoint what its chat template
|
||||
actually accepts, and ticks those. llama.cpp publishes the loaded model's
|
||||
template, and that template is the very thing that rejects an effort it does
|
||||
not recognise — so the answer is read from the place that is authoritative
|
||||
instead of guessed at, or discovered by a failed reply.
|
||||
- Endpoints that do not publish a template — OpenAI, vLLM — say so plainly
|
||||
rather than being recorded as accepting nothing.
|
||||
|
||||
## 1.2.0
|
||||
|
||||
- Fixed: **choosing a reasoning effort could kill the reply outright**, with a
|
||||
|
||||
Reference in New Issue
Block a user