A title call that could not survive a model that thinks
Reported: chat names never regenerate after the first reply. They were regenerating; the request was being made and the answer thrown away. `complete()` returns `message.content` verbatim, and a model that emits `<think>` inline puts its thinking in exactly the field the title is read from. So the title came back as "<think>Okay, the user wants a short title for" -- or, once the too-long guard caught that, as the first prompt trimmed, which is indistinguishable from titling never having run. That is what was being seen. Underneath it, `max_tokens: 24`. Ample for six words, and nowhere near enough for a model that reasons first: the budget goes on thinking and the content field comes back empty or holding an unclosed tag. Too small is not a shorter title, it is no title at all. Both fixed: the reply goes through `reasoning.strip_reasoning`, and the budget is `TITLE_MAX_TOKENS` with room to think. Reproduced first against the four shapes an endpoint actually answers with -- three of them were broken -- and the tests are written from those. What I did *not* do is ask for a low reasoning effort on the call, which would make it much cheaper and was the obvious move. `reasoning_effort` and `chat_template_kwargs` appear only where somebody has opted in, so that a provider strict about unknown parameters sees exactly the request it always did. An LLMError here is caught and turned into a fallback title -- so a 400 would be titling silently switching itself off, which is the failure this commit exists to fix. The token budget makes the room instead. The shipped prompt now asks for a leading emoji, as requested. Asked for rather than assumed: a model that ignores it gives a title without one, and an administrator who does not want them clears the word. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -156,6 +156,106 @@ async def test_the_title_prompt_carries_the_exchange(mock_http):
|
||||
assert seen == ["Name this: What is lembas? / Waybread. / {{nonsense}}"]
|
||||
|
||||
|
||||
async def test_a_title_from_a_model_that_thinks_first(mock_http):
|
||||
"""The bug: a reasoning model puts `<think>` in the very field the title is
|
||||
read from, so every chat on one was named "<think>Okay, the user wants a
|
||||
short title for" -- or, once the guard caught that as too long, fell back to
|
||||
the first prompt and looked as though titling had never run at all."""
|
||||
|
||||
def handler(_request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"choices": [
|
||||
{
|
||||
"message": {
|
||||
"content": "<think>Six words, an emoji</think>\n"
|
||||
"🌳 Mallorn trees explained"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
)
|
||||
|
||||
mock_http(handler)
|
||||
title = await chat_service.generate_title(
|
||||
Endpoint("http://x.test", "", {}),
|
||||
"m",
|
||||
"What is a mallorn?",
|
||||
"A golden tree.",
|
||||
template="Name this: {{question}}",
|
||||
)
|
||||
assert title == "🌳 Mallorn trees explained"
|
||||
|
||||
|
||||
async def test_the_title_call_leaves_room_to_think(mock_http):
|
||||
"""24 tokens is ample for six words and nowhere near enough for a model that
|
||||
reasons first: the budget went on thinking and the content came back empty.
|
||||
Too small is not a shorter title, it is no title."""
|
||||
seen: list[dict] = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen.append(json.loads(request.content))
|
||||
return httpx.Response(200, json={"choices": [{"message": {"content": "A name"}}]})
|
||||
|
||||
mock_http(handler)
|
||||
await chat_service.generate_title(
|
||||
Endpoint("http://x.test", "", {}), "m", "q", "a", template="Name this: {{question}}"
|
||||
)
|
||||
assert seen[0]["max_tokens"] == chat_service.TITLE_MAX_TOKENS
|
||||
assert seen[0]["max_tokens"] >= 256
|
||||
|
||||
|
||||
async def test_the_title_call_sends_no_reasoning_effort(mock_http):
|
||||
"""Tempting, and wrong. Those two fields appear only when somebody has opted
|
||||
in, so a provider strict about unknown parameters sees the request it always
|
||||
did -- and a 400 here is caught and turned into a fallback title, which is
|
||||
titling silently switching itself off."""
|
||||
seen: list[dict] = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen.append(json.loads(request.content))
|
||||
return httpx.Response(200, json={"choices": [{"message": {"content": "A name"}}]})
|
||||
|
||||
mock_http(handler)
|
||||
await chat_service.generate_title(
|
||||
Endpoint("http://x.test", "", {}), "m", "q", "a", template="Name this: {{question}}"
|
||||
)
|
||||
assert "reasoning_effort" not in seen[0]
|
||||
assert "chat_template_kwargs" not in seen[0]
|
||||
|
||||
|
||||
async def test_a_title_that_is_only_thinking_falls_back(mock_http):
|
||||
"""Nothing but reasoning means nothing to name it with. The first prompt is
|
||||
a better title than an empty one."""
|
||||
|
||||
def handler(_request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(
|
||||
200, json={"choices": [{"message": {"content": "<think>still deciding"}}]}
|
||||
)
|
||||
|
||||
mock_http(handler)
|
||||
title = await chat_service.generate_title(
|
||||
Endpoint("http://x.test", "", {}),
|
||||
"m",
|
||||
"What is a mallorn?",
|
||||
"A golden tree.",
|
||||
template="Name this: {{question}}",
|
||||
)
|
||||
assert title == "What is a mallorn?"
|
||||
|
||||
|
||||
def test_the_shipped_title_prompt_asks_for_an_emoji():
|
||||
"""It makes a sidebar of twenty chats scannable, and it is asked for rather
|
||||
than assumed -- a model that ignores it gives a title without one."""
|
||||
from lembas.services import prompts
|
||||
|
||||
fragment = next(f for f in prompts.BUILTIN if f.key == "task.title")
|
||||
assert "emoji" in fragment.default
|
||||
assert "{{question}}" in fragment.default
|
||||
assert "{{answer}}" in fragment.default
|
||||
|
||||
|
||||
async def test_an_empty_title_prompt_asks_no_model_at_all(mock_http):
|
||||
"""Clearing the fragment is how auto-titling is turned off. It must not
|
||||
cost a request that is then thrown away."""
|
||||
|
||||
Reference in New Issue
Block a user