Reading the answer instead of asking somebody to know it

llama-server publishes the loaded model's Jinja chat template on /props, and
that template is the very thing that rejects a reasoning effort it does not
recognise -- so the accepted set is written down, authoritatively, in a place
this application can simply read. There is a button on the model's page that
does.

The parser handles both shapes a template uses: the values inline in the test
that rejects them (Bonsai), and a named list set elsewhere with nothing near
the mention spelling them out (gpt-oss). It is deliberately conservative,
because a wrong answer here silently removes a level somebody is entitled to:
only known efforts count, an unrelated list of quoted strings is ignored, and a
single match is read as a default -- `{%- set reasoning_effort = 'medium' %}`
-- rather than as a vocabulary of one.

An endpoint with no such route says so. OpenAI and vLLM do not publish a
template, and "this cannot tell us" must not be recorded as "this model accepts
nothing".

/props sits at the server root, beside the OpenAI-compatible surface rather
than inside it, so a base URL written as .../v1 needs the suffix stripped.
Getting that wrong is a silent 404 that looks like detection simply not
working, so there is a test on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 22:23:22 +00:00
co-authored by Claude Opus 5
parent 32e2326d41
commit b1dbca7db6
7 changed files with 323 additions and 2 deletions
+37
View File
@@ -68,6 +68,19 @@ class Endpoint:
base = f"{base}/v1"
return f"{base}/{path.lstrip('/')}"
def root_url(self, path: str) -> str:
"""A URL at the *server's* root rather than under `/v1`.
llama-server's own endpoints -- `/props` is the one that matters here --
sit beside the OpenAI-compatible surface, not inside it. A base URL may
be written either way (`http://host:8080` or `.../v1`), so the suffix is
stripped rather than assumed absent.
"""
base = self.base_url.rstrip("/")
if base.endswith("/v1"):
base = base[: -len("/v1")]
return f"{base}/{path.lstrip('/')}"
def headers(self) -> dict[str, str]:
headers = {"Content-Type": "application/json", **self.extra_headers}
# Local endpoints frequently need no key at all; sending an empty
@@ -77,6 +90,30 @@ class Endpoint:
return headers
async def fetch_chat_template(endpoint: Endpoint) -> str:
"""The model's own Jinja chat template, from llama-server's `/props`.
The one place the truth about a model's accepted values is actually
written down: `/props` returns `chat_template` verbatim, and that template
is what raises when it meets a `reasoning_effort` it does not know.
Returns "" rather than raising for anything that is not a llama-server --
OpenAI, vLLM and the rest have no such route, and "this endpoint cannot
tell us" is a normal answer here, not a failure.
"""
try:
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.get(
endpoint.root_url("props"), headers=endpoint.headers()
)
response.raise_for_status()
payload = response.json()
except (httpx.HTTPError, ValueError, json.JSONDecodeError):
return ""
template = payload.get("chat_template") if isinstance(payload, dict) else ""
return template if isinstance(template, str) else ""
def describe_http_error(exc: httpx.HTTPStatusError) -> str:
"""Turn an upstream error response into something worth reading.