A crowd in one chat
The chat's own model answers, then each other member in order, then the order runs
backwards asking each whether it disagrees, ending at the main model, which either
closes or sends them round again. Design and reasoning: LLeMbas.wiki/Crowd-chats.
THE SPEAKER SEAM, WHICH IS ALSO A BUG FIX
`chat_service.speaker_for` makes the *message* name the answering model and the
chat only the default. That closes a live half-wired feature -- `wake_chat` takes a
model override and `schedule/runner` passes one, and it reached the row and never
the request, so a schedule naming another model got the chat's model wearing the
other one's name.
The seam is wider than `build_request`: `{{model_name}}`, the authored prompt's
model layer, `vision` (where a wrong answer makes the endpoint reject the whole
request), the effort vocabulary (which raises inside the model's own chat template,
and whose refusal narrows every Model row sharing the id), `resolve_tools`,
`context_length` -> `_too_big`, and `ToolContext.model_id`. `resolve_endpoint` may
now only write back `chat.connection_id` when the speaker *is* the chat's model.
WHY N CHAINED REPLIES
`Generation` is one reply's state and `_follow` streams per message, so one
generation cannot stream into nine bubbles and `ensure` would not know which of the
nine it was after a restart. A subagent per speaker cannot work either: its answer
comes back as a tool result and tool results are never replayed, so speaker 3 could
not see speaker 2 -- which is the whole point. Chained, exactly one incomplete row
exists at a time, and `tests/test_crowd_chain.py` asserts that at every
observation.
The round lives on `Message.crowd_json`, not on the chat: the row is the authority,
and chat-level state would describe turns a rewind or a restart had removed.
`crowd.next_turn` is pure, so all eight refusals are tested with no endpoint.
THREE RULES, EACH A BUG WRITTEN THE OTHER WAY ROUND
- `if not _advance_crowd(g): _drain(g)` -- advancing must *suppress* draining, or a
queued human turn puts a second incomplete row beside the next speaker's.
- `_advance_crowd` refuses unless the finishing row is the newest, or regenerating
member 2 creates a second member 3 and two chains race down one turn.
- an error skips one speaker and two in a row end the round: the usual failure is a
small member's window overflowing, and `_drain`'s stop-on-error would kill every
crowd at whichever member is smallest.
Each other speaker's turn is relabelled as attributed user content, which is both
how a model can disagree with words it did not write and how the history keeps
alternating. The per-speaker instruction is payload-only -- as a row it could be
dropped from the request by a `created_at` tie, and every later speaker would answer
it. Compaction, titling and the notification are gated to once per turn; `_inject`
is off during a round; the way back gets no tools and a member is treated as
unattended.
Membership stores the model as text with no foreign key: "Test & refresh" deletes
and recreates Model rows, and a cascade would empty the crowd out of every chat.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -445,6 +445,93 @@
|
||||
button was pressed, which is what keeps each group's save handler writing one
|
||||
key.
|
||||
#}
|
||||
{# A third settings group on this page, saved by its own form -- the reason the
|
||||
Helpers card gives. A crowd is not an agent-chat feature either, but this is the
|
||||
page somebody opens to find out what one turn may set going. #}
|
||||
<form method="post" action="/admin/agents/crowd" class="form-grid">
|
||||
<section class="card">
|
||||
<h2 class="card__title">A crowd</h2>
|
||||
<p class="field__hint">
|
||||
A chat can have more than one model in it. The chat's own model answers, then
|
||||
each of the others in turn; then the order runs <strong>backwards</strong>,
|
||||
each one asked whether it disagrees with anything; and it ends back at the
|
||||
first, which either closes or sends them round again.
|
||||
</p>
|
||||
|
||||
<div class="alert">
|
||||
{{ icon("warning", "icon--sm") }}
|
||||
<span>
|
||||
One turn costs <strong>models × rounds × 2 − 1</strong> replies — four
|
||||
models over two rounds is fifteen — and on a single local endpoint every
|
||||
change of speaker also loads a different model. Larger crowds of smaller
|
||||
models, and sometimes of bigger ones, start going round in circles: that is
|
||||
what the round limit is for, and it is a limit ordinary work will reach
|
||||
rather than a runaway backstop.
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div class="field">
|
||||
<label class="checkbox">
|
||||
<input type="checkbox" name="enabled" value="true"
|
||||
{{ 'checked' if crowd.enabled }}>
|
||||
<span>Let a chat have a crowd</span>
|
||||
</label>
|
||||
<p class="field__hint">
|
||||
Off by default. With it on, each chat's settings panel offers the other
|
||||
models; a chat with none ticked behaves exactly as it always has.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="field">
|
||||
<label class="field__label" for="crowd_max_models">Most models besides the chat's own</label>
|
||||
<input class="input" id="crowd_max_models" name="max_models"
|
||||
type="number" min="1" max="8" step="1" value="{{ crowd.max_models }}">
|
||||
<p class="field__hint">
|
||||
Four is already eight replies a turn at one round each. More voices past
|
||||
that tend to repeat each other rather than add anything.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="field">
|
||||
<label class="field__label" for="crowd_max_rounds">Most rounds</label>
|
||||
<input class="input" id="crowd_max_rounds" name="max_rounds"
|
||||
type="number" min="1" max="5" step="1" value="{{ crowd.max_rounds }}">
|
||||
<p class="field__hint">
|
||||
A round is out and back. Two gives the first model one chance to change its
|
||||
mind after hearing the objections, which is the point of the whole thing;
|
||||
three is where going in circles starts.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="field">
|
||||
<label class="field__label" for="crowd_wall_seconds">Longest a turn may take</label>
|
||||
<input class="input" id="crowd_wall_seconds" name="wall_seconds"
|
||||
type="number" min="60" max="7200" step="30" value="{{ crowd.wall_seconds }}">
|
||||
<p class="field__hint">
|
||||
Across every speaker, not each. A member whose endpoint has stalled cannot
|
||||
then hold the round open all afternoon.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="field">
|
||||
<label class="checkbox">
|
||||
<input type="checkbox" name="collapse_agreement" value="true"
|
||||
{{ 'checked' if crowd.collapse_agreement }}>
|
||||
<span>Fold away a short "I agree" on the way back</span>
|
||||
</label>
|
||||
<p class="field__hint">
|
||||
The disagreements are what a crowd is for; a column of bubbles saying
|
||||
nothing is what makes somebody switch it off. The text is still there
|
||||
behind a disclosure.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="btn-row">
|
||||
<button class="btn btn--primary" type="submit">Save</button>
|
||||
</div>
|
||||
</section>
|
||||
</form>
|
||||
|
||||
<form method="post" action="/admin/agents/subagents" class="form-grid">
|
||||
<section class="card">
|
||||
<h2 class="card__title">Helpers</h2>
|
||||
|
||||
@@ -29,8 +29,17 @@
|
||||
`msg--machine` only overrides what should differ.
|
||||
#}
|
||||
{% set machine = (message.role == "user" and message.machine) %}
|
||||
{#
|
||||
Where this bubble sits in a crowd round, if it is in one. Nine bubbles for one
|
||||
question need orienting, and a `<details>` wrapper round the round is the wrong
|
||||
way to do it: bubbles arrive with `beforeend:#thread`, which appends *after* any
|
||||
container, so the live and reloaded renderings would disagree and a reload would
|
||||
rearrange nine bubbles under the reader. A chip on each one is the same markup
|
||||
either way.
|
||||
#}
|
||||
{% set crowd = message.crowd_json or None %}
|
||||
|
||||
<article class="msg msg--{{ message.role }}{{ ' msg--machine' if machine }}{{ ' msg--queued' if queued }}"
|
||||
<article class="msg msg--{{ message.role }}{{ ' msg--machine' if machine }}{{ ' msg--queued' if queued }}{{ ' msg--crowd-back' if crowd and crowd.get('phase') == 'back' }}"
|
||||
id="msg-{{ message.id }}"
|
||||
{% if streaming %}
|
||||
hx-ext="sse"
|
||||
@@ -74,6 +83,33 @@
|
||||
{# Only worth showing when it adds something the author line does not. #}
|
||||
<span class="msg__model" title="{{ message.model_id }}">{{ message.model_id }}</span>
|
||||
{% endif %}
|
||||
{% if crowd %}
|
||||
{# Which speaker, which pass. The count is of speakers rather than of
|
||||
replies: a round produces more bubbles than it has models in it. #}
|
||||
<span class="badge">
|
||||
{% if crowd.get("phase") == "out" %}
|
||||
{{ crowd.get("index", 0) + 1 }} of {{ crowd.get("of", 1) }}
|
||||
{% elif crowd.get("phase") == "back" %}
|
||||
on the way back
|
||||
{% else %}
|
||||
closing
|
||||
{% endif %}
|
||||
{% if crowd.get("round", 1) > 1 %} · round {{ crowd.get("round") }}{% endif %}
|
||||
</span>
|
||||
{% if crowd.get("stopped") %}
|
||||
{# Why a round ended, where it ended. Without this a crowd that ran out of
|
||||
rounds or time simply stops, which reads as the feature failing. #}
|
||||
<span class="badge badge--warning" title="The round ended here">
|
||||
{% if crowd.get("stopped") == "rounds" %}
|
||||
no rounds left
|
||||
{% elif crowd.get("stopped") == "time" %}
|
||||
out of time
|
||||
{% else %}
|
||||
two endpoints failed
|
||||
{% endif %}
|
||||
</span>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</header>
|
||||
|
||||
{% if message.attachments %}
|
||||
|
||||
@@ -243,6 +243,52 @@
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if crowd_available %}
|
||||
{# Who else answers. Nothing ticked is every chat that has ever existed:
|
||||
one model, answering on its own. #}
|
||||
<div class="field">
|
||||
<label class="field__label">Crowd</label>
|
||||
<form hx-patch="/api/chats/{{ chat.id }}" hx-swap="none" hx-trigger="change">
|
||||
{# Always submitted, for the reason the bases above are. #}
|
||||
<input type="hidden" name="crowd_model_ids" value="">
|
||||
<div class="checkbox-row">
|
||||
{% for model in crowd_available %}
|
||||
<label class="checkbox">
|
||||
<input type="checkbox" name="crowd_model_ids" value="{{ model.model_id }}"
|
||||
{{ 'checked' if model.model_id in crowd_member_ids }}>
|
||||
<span>{{ model.label }}</span>
|
||||
</label>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</form>
|
||||
<p class="field__hint">
|
||||
{% if crowd_member_ids %}
|
||||
{{ crowd_member_ids|length + 1 }} models answer each turn: this one
|
||||
first, then the others, then back through them asking whether they
|
||||
disagree, ending here.
|
||||
<strong>That is {{ crowd_replies }} replies a turn</strong>, and up to
|
||||
{{ crowd_rounds }} rounds of it.
|
||||
{% else %}
|
||||
Tick a model to have it answer after this one, then be asked whether
|
||||
it disagrees. Useful for a second opinion; expensive, because each
|
||||
one is a whole reply, and slow on one local endpoint because every
|
||||
change of speaker loads a different model.
|
||||
{% endif %}
|
||||
</p>
|
||||
{# Outside the two branches above, deliberately. A member whose model has
|
||||
gone is filtered out of `crowd_member_ids`, so if it was the only one
|
||||
this would fall into the "tick a model" branch and never mention the
|
||||
row that is still there -- which is the one thing somebody needs to
|
||||
know to tidy it up. #}
|
||||
{% if crowd_skipped %}
|
||||
<p class="field__hint">
|
||||
Skipped, because you cannot reach {{ "them" if crowd_skipped|length > 1 else "it" }}
|
||||
any more: <s>{{ crowd_skipped|join(", ") }}</s>. Untick to clear.
|
||||
</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if can.get("chat.params") %}
|
||||
<div class="grid grid--3">
|
||||
<div class="field">
|
||||
|
||||
Reference in New Issue
Block a user