A thinking block that says how long and how much
Each block reports its own round now. `reasoning_ms` was the reply's first burst, written once, so on the fifteen-block reply GPT-OSS actually produces only the first could claim a duration and the other fourteen said "Thought" and nothing at all. `Generation.thinking_ms` accumulates per round and `close_step` stamps it cumulatively, so steps.py diffs it exactly as it already diffs the three lengths beside it. The interval between a round's first and last reasoning delta, deliberately, not a sum of gaps between deltas -- that would count the network's latency as the model's thinking. While it runs: "Thinking" with an ellipsis that types itself, and the seconds and tokens climbing beside it. The ellipsis is a `content` keyframe, so there is no timer to start, stop or clean up when the block is swapped away -- it stops existing when the element does. The numbers come from a `think` frame, and `round_thinking_ms` is written by the producer rather than computed by the follower from a start time: a model that has stopped thinking and moved on to a tool should show a settled number, not a clock that keeps running. Tokens read exactly up to 200 and as `0.4k` above it, from one helper shared by the live label and the stored one, so the two cannot drift into two conventions. The live duration is terser than the finished one -- `6s` against `6 seconds` -- because it sits beside an animating word and changes every second, where "less than a second" flickering into "1 second" reads as a glitch. Checked against the real endpoint: fourteen marks carrying 919ms through 14223ms, per-block labels from "less than a second · 111" to "4 seconds · 0.5k", and the live frames resetting each round rather than accumulating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -613,3 +613,68 @@ async def test_a_reply_that_calls_nothing_writes_no_marks(db, user_id, monkeypat
|
||||
message = db.get(Message, message_id)
|
||||
db.refresh(message)
|
||||
assert message.steps_json == []
|
||||
|
||||
|
||||
async def test_thinking_time_is_recorded_per_round(db, user_id, monkeypatch):
|
||||
"""Each thinking block reports its own round. `reasoning_ms` is the reply's
|
||||
*first* burst and is written once, so it could never say how long round
|
||||
seven thought for -- every block but the first said "Thought" and nothing.
|
||||
"""
|
||||
import asyncio as _asyncio
|
||||
|
||||
from lembas.db.models import Message
|
||||
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
|
||||
monkeypatch.setattr("lembas.services.search.run", _empty_search)
|
||||
|
||||
async def slow_stream(_endpoint, payload):
|
||||
"""Two rounds, each thinking for a measurable moment."""
|
||||
yield {"choices": [{"delta": {"reasoning_content": "mulling it over"}}]}
|
||||
await _asyncio.sleep(0.05)
|
||||
yield {"choices": [{"delta": {"reasoning_content": " some more"}}]}
|
||||
if len([m for m in payload["messages"] if m.get("role") == "tool"]) == 0:
|
||||
yield _tool_call_chunk("web_search", '{"query": "x"}')
|
||||
else:
|
||||
yield _text_chunk("Done.")
|
||||
|
||||
monkeypatch.setattr(generation_service, "stream_chat", slow_stream)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
await generation_service._run(generation)
|
||||
|
||||
assert generation.thinking_ms > 0, "the reply spent time thinking"
|
||||
assert generation.steps, "and closed a step"
|
||||
assert generation.steps[0]["thinking_ms"] > 0, "stamped on the mark"
|
||||
# Cumulative on the mark, so a block's own duration is the difference --
|
||||
# the same rule the three lengths beside it follow.
|
||||
assert generation.steps[0]["thinking_ms"] <= generation.thinking_ms
|
||||
|
||||
db.expire_all()
|
||||
stored = db.get(Message, message_id)
|
||||
assert stored.reasoning_ms == generation.thinking_ms, (
|
||||
"the row keeps the summed figure, not the first burst"
|
||||
)
|
||||
|
||||
|
||||
async def test_the_round_clock_settles_when_thinking_stops(db, user_id, monkeypatch):
|
||||
"""`round_thinking_ms` is written by the producer, not computed from a start
|
||||
time by the follower -- a model that has stopped thinking and moved on to a
|
||||
tool should show a settled number rather than a clock that keeps running."""
|
||||
settings_store.update(db, {"enabled": True}, key=settings_store.SEARCH)
|
||||
chat_id, message_id = _chat_with_tools(db, user_id)
|
||||
monkeypatch.setattr("lembas.services.search.run", _empty_search)
|
||||
monkeypatch.setattr(
|
||||
generation_service,
|
||||
"stream_chat",
|
||||
_stub_stream([[_text_chunk("No thinking at all.")]], []),
|
||||
)
|
||||
monkeypatch.setattr("lembas.services.chat.generate_title", _never_called_title)
|
||||
|
||||
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
|
||||
await generation_service._run(generation)
|
||||
|
||||
assert generation.round_thinking_ms == 0
|
||||
assert generation.thinking_ms == 0
|
||||
|
||||
Reference in New Issue
Block a user