A title call that could not survive a model that thinks

Reported: chat names never regenerate after the first reply. They were
regenerating; the request was being made and the answer thrown away.

`complete()` returns `message.content` verbatim, and a model that emits
`<think>` inline puts its thinking in exactly the field the title is read from.
So the title came back as "<think>Okay, the user wants a short title for" --
or, once the too-long guard caught that, as the first prompt trimmed, which is
indistinguishable from titling never having run. That is what was being seen.

Underneath it, `max_tokens: 24`. Ample for six words, and nowhere near enough
for a model that reasons first: the budget goes on thinking and the content
field comes back empty or holding an unclosed tag. Too small is not a shorter
title, it is no title at all.

Both fixed: the reply goes through `reasoning.strip_reasoning`, and the budget
is `TITLE_MAX_TOKENS` with room to think. Reproduced first against the four
shapes an endpoint actually answers with -- three of them were broken -- and
the tests are written from those.

What I did *not* do is ask for a low reasoning effort on the call, which would
make it much cheaper and was the obvious move. `reasoning_effort` and
`chat_template_kwargs` appear only where somebody has opted in, so that a
provider strict about unknown parameters sees exactly the request it always
did. An LLMError here is caught and turned into a fallback title -- so a 400
would be titling silently switching itself off, which is the failure this
commit exists to fix. The token budget makes the room instead.

The shipped prompt now asks for a leading emoji, as requested. Asked for rather
than assumed: a model that ignores it gives a title without one, and an
administrator who does not want them clears the word.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 11:23:45 +02:00
parent 9b65adf388
commit 2576755f79
4 changed files with 167 additions and 17 deletions
+18 -1
View File
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 1403 tests, ~87s
pytest # 1408 tests, ~87s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
@@ -210,6 +210,23 @@ treats a bare `Mapped[list]` as a scalar and hands back `None` instead of `[]`.
Always write `Mapped[list[Group]]`, with a `TYPE_CHECKING` import if the class
lives in another module.
**Auto-titling is a request like any other, and it was reading the wrong field
of one.** `complete()` hands back `message.content` verbatim, and a model that
emits `<think>` inline puts its thinking in exactly that field -- so a title
came back as "<think>Okay, the user wants a short title for", or, once the
too-long guard caught that, as the first prompt trimmed, which looks precisely
like titling never having run. `generate_title` puts the reply through
`reasoning.strip_reasoning` now. The budget was 24 tokens, which is ample for
six words and nowhere near enough for a model that thinks first: too small is
not a shorter title, it is no title, because the thinking consumes the budget
and content comes back empty. It is `TITLE_MAX_TOKENS` and generous.
Deliberately **not** `apply_effort(body, "low")`, tempting as that is. Those two
fields appear only where somebody has opted in, precisely so a provider strict
about unknown parameters sees the request it always did -- and an LLMError here
is caught and turned into a fallback title, so a 400 would be titling silently
switching itself off. The token budget is what makes room for the thinking.
**Reasoning arrives two ways.** A `reasoning_content` delta field (llama.cpp,
llama-swap, vLLM) or `<think>` tags inline in `content` (Ollama and friends).
`services/reasoning.py` handles the second with a streaming splitter, because