A title call that could not survive a model that thinks

Reported: chat names never regenerate after the first reply. They were
regenerating; the request was being made and the answer thrown away.

`complete()` returns `message.content` verbatim, and a model that emits
`<think>` inline puts its thinking in exactly the field the title is read from.
So the title came back as "<think>Okay, the user wants a short title for" --
or, once the too-long guard caught that, as the first prompt trimmed, which is
indistinguishable from titling never having run. That is what was being seen.

Underneath it, `max_tokens: 24`. Ample for six words, and nowhere near enough
for a model that reasons first: the budget goes on thinking and the content
field comes back empty or holding an unclosed tag. Too small is not a shorter
title, it is no title at all.

Both fixed: the reply goes through `reasoning.strip_reasoning`, and the budget
is `TITLE_MAX_TOKENS` with room to think. Reproduced first against the four
shapes an endpoint actually answers with -- three of them were broken -- and
the tests are written from those.

What I did *not* do is ask for a low reasoning effort on the call, which would
make it much cheaper and was the obvious move. `reasoning_effort` and
`chat_template_kwargs` appear only where somebody has opted in, so that a
provider strict about unknown parameters sees exactly the request it always
did. An LLMError here is caught and turned into a fallback title -- so a 400
would be titling silently switching itself off, which is the failure this
commit exists to fix. The token budget makes the room instead.

The shipped prompt now asks for a leading emoji, as requested. Asked for rather
than assumed: a model that ignores it gives a title without one, and an
administrator who does not want them clears the word.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 11:23:45 +02:00
parent 27b94c385d
commit 35b85a9cda
3 changed files with 149 additions and 16 deletions
+100
View File
@@ -156,6 +156,106 @@ async def test_the_title_prompt_carries_the_exchange(mock_http):
assert seen == ["Name this: What is lembas? / Waybread. / {{nonsense}}"]
async def test_a_title_from_a_model_that_thinks_first(mock_http):
"""The bug: a reasoning model puts `<think>` in the very field the title is
read from, so every chat on one was named "<think>Okay, the user wants a
short title for" -- or, once the guard caught that as too long, fell back to
the first prompt and looked as though titling had never run at all."""
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
json={
"choices": [
{
"message": {
"content": "<think>Six words, an emoji</think>\n"
"🌳 Mallorn trees explained"
}
}
]
},
)
mock_http(handler)
title = await chat_service.generate_title(
Endpoint("http://x.test", "", {}),
"m",
"What is a mallorn?",
"A golden tree.",
template="Name this: {{question}}",
)
assert title == "🌳 Mallorn trees explained"
async def test_the_title_call_leaves_room_to_think(mock_http):
"""24 tokens is ample for six words and nowhere near enough for a model that
reasons first: the budget went on thinking and the content came back empty.
Too small is not a shorter title, it is no title."""
seen: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
seen.append(json.loads(request.content))
return httpx.Response(200, json={"choices": [{"message": {"content": "A name"}}]})
mock_http(handler)
await chat_service.generate_title(
Endpoint("http://x.test", "", {}), "m", "q", "a", template="Name this: {{question}}"
)
assert seen[0]["max_tokens"] == chat_service.TITLE_MAX_TOKENS
assert seen[0]["max_tokens"] >= 256
async def test_the_title_call_sends_no_reasoning_effort(mock_http):
"""Tempting, and wrong. Those two fields appear only when somebody has opted
in, so a provider strict about unknown parameters sees the request it always
did -- and a 400 here is caught and turned into a fallback title, which is
titling silently switching itself off."""
seen: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
seen.append(json.loads(request.content))
return httpx.Response(200, json={"choices": [{"message": {"content": "A name"}}]})
mock_http(handler)
await chat_service.generate_title(
Endpoint("http://x.test", "", {}), "m", "q", "a", template="Name this: {{question}}"
)
assert "reasoning_effort" not in seen[0]
assert "chat_template_kwargs" not in seen[0]
async def test_a_title_that_is_only_thinking_falls_back(mock_http):
"""Nothing but reasoning means nothing to name it with. The first prompt is
a better title than an empty one."""
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(
200, json={"choices": [{"message": {"content": "<think>still deciding"}}]}
)
mock_http(handler)
title = await chat_service.generate_title(
Endpoint("http://x.test", "", {}),
"m",
"What is a mallorn?",
"A golden tree.",
template="Name this: {{question}}",
)
assert title == "What is a mallorn?"
def test_the_shipped_title_prompt_asks_for_an_emoji():
"""It makes a sidebar of twenty chats scannable, and it is asked for rather
than assumed -- a model that ignores it gives a title without one."""
from lembas.services import prompts
fragment = next(f for f in prompts.BUILTIN if f.key == "task.title")
assert "emoji" in fragment.default
assert "{{question}}" in fragment.default
assert "{{answer}}" in fragment.default
async def test_an_empty_title_prompt_asks_no_model_at_all(mock_http):
"""Clearing the fragment is how auto-titling is turned off. It must not
cost a request that is then thrown away."""