Show what a reply cost, live and afterwards
Tokens, how full the context is, and tokens per second -- as chips under each assistant bubble, updating while the reply streams and still there when it finishes. The numbers come from one Metrics object built either from the generation still being written or from the row it left behind. That is the point rather than tidiness: the finished bubble is re-rendered from the database the instant the stream ends, so two code paths would make the figures visibly jump at exactly the moment someone is watching them. Here the only thing that changes is that an estimate may become exact. Message.usage_json has existed and been dead since the schema was written. It is the store. Two counts that look like one. prompt and completion are summed across tool rounds -- what the reply cost. context_tokens is overwritten each round with that round's prompt plus completion -- what the window actually holds. A three-round reply pays for its prompt three times and only ever occupies the window once, so a single number would be wrong for one of the two questions. Generation gains started_at as a field rather than a local in _run, because _follow is a different function that sees only the Generation and otherwise has nothing to compute a live speed against. It also carries a prompt estimate taken before the first chunk, since real usage arrives in one chunk at the very end and a percentage that appears only after the reply is useless. Everything is marked with a tilde when the endpoint reported nothing, and the percentage is simply absent when no context length is set: unknown has to stay tellable from small, and a percentage of an unknown total is a made-up number in a place people trust numbers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -120,6 +120,10 @@
|
||||
<div class="msg__waiting">
|
||||
<span class="dots"><i></i><i></i><i></i></span>
|
||||
</div>
|
||||
{# Counts as the reply is written. Everything is an estimate until the
|
||||
usage chunk lands at the very end, and the chips say so. #}
|
||||
<div class="msg__metrics" id="metrics-{{ message.id }}"
|
||||
sse-swap="metrics" hx-swap="innerHTML"></div>
|
||||
{% else %}
|
||||
{# Finished. Same order as the live view above -- thinking, then what it
|
||||
looked up, then the answer -- so a reply does not rearrange itself the
|
||||
@@ -177,6 +181,15 @@
|
||||
leave an empty box under the file. #}
|
||||
{% endif %}
|
||||
|
||||
{% if not streaming and message.role == "assistant" and message.usage_json %}
|
||||
{# Above the buttons, not among them: the actions row is things you press. #}
|
||||
<div class="msg__metrics" id="metrics-{{ message.id }}">
|
||||
{% with metrics = message.usage_json | metrics %}
|
||||
{% include "chat/_metrics.html" %}
|
||||
{% endwith %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if not streaming %}
|
||||
<footer class="msg__actions">
|
||||
<button class="btn btn--icon btn--sm" type="button"
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
{#
|
||||
What a reply cost. Chips only, no wrapper: the same markup is swapped into
|
||||
the live bubble with innerHTML and rendered into the finished one, so the
|
||||
numbers cannot change shape when the stream ends.
|
||||
|
||||
A tilde means the endpoint reported no token counts and these were worked out
|
||||
at about four characters per token. Nothing here is ever shown as exact when
|
||||
it is not.
|
||||
#}
|
||||
{% if metrics.has_anything %}
|
||||
<span class="metric" title="{% if metrics.estimated %}Estimated: this endpoint reports no token counts.
|
||||
{% endif %}{{ metrics.prompt_tokens }} in, {{ metrics.completion_tokens }} out{% if metrics.rounds > 1 %}, over {{ metrics.rounds }} rounds of tool calls{% endif %}">
|
||||
{% if metrics.estimated %}~{% endif %}{{ metrics.total_tokens }} tokens
|
||||
</span>
|
||||
|
||||
{% if metrics.context_limit %}
|
||||
<span class="metric metric--context{{ ' is-' ~ metrics.pressure if metrics.pressure }}"
|
||||
title="{% if metrics.estimated %}Estimated. {% endif %}{{ metrics.context_tokens }} of {{ metrics.context_limit }} tokens of context used">
|
||||
<span class="metric__bar">
|
||||
{# A width is data, not a design value: it is the measurement itself. #}
|
||||
<span class="metric__fill" style="width: {{ metrics.percent }}%"></span>
|
||||
</span>
|
||||
{% if metrics.estimated %}~{% endif %}{{ metrics.percent }}%
|
||||
</span>
|
||||
{% endif %}
|
||||
|
||||
{% if metrics.tokens_per_second %}
|
||||
<span class="metric" title="{% if metrics.estimated %}Estimated. {% endif %}{{ metrics.completion_tokens }} tokens in {{ metrics.elapsed_ms }} ms">
|
||||
{% if metrics.estimated %}~{% endif %}{{ '%.1f' | format(metrics.tokens_per_second) }} tok/s
|
||||
</span>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
Reference in New Issue
Block a user