Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 23 Wiki Activity

A crowd in one chat

Browse Source 
The chat's own model answers, then each other member in order, then the order runs
backwards asking each whether it disagrees, ending at the main model, which either
closes or sends them round again. Design and reasoning: LLeMbas.wiki/Crowd-chats.

THE SPEAKER SEAM, WHICH IS ALSO A BUG FIX

`chat_service.speaker_for` makes the *message* name the answering model and the
chat only the default. That closes a live half-wired feature -- `wake_chat` takes a
model override and `schedule/runner` passes one, and it reached the row and never
the request, so a schedule naming another model got the chat's model wearing the
other one's name.

The seam is wider than `build_request`: `{{model_name}}`, the authored prompt's
model layer, `vision` (where a wrong answer makes the endpoint reject the whole
request), the effort vocabulary (which raises inside the model's own chat template,
and whose refusal narrows every Model row sharing the id), `resolve_tools`,
`context_length` -> `_too_big`, and `ToolContext.model_id`. `resolve_endpoint` may
now only write back `chat.connection_id` when the speaker *is* the chat's model.

WHY N CHAINED REPLIES

`Generation` is one reply's state and `_follow` streams per message, so one
generation cannot stream into nine bubbles and `ensure` would not know which of the
nine it was after a restart. A subagent per speaker cannot work either: its answer
comes back as a tool result and tool results are never replayed, so speaker 3 could
not see speaker 2 -- which is the whole point. Chained, exactly one incomplete row
exists at a time, and `tests/test_crowd_chain.py` asserts that at every
observation.

The round lives on `Message.crowd_json`, not on the chat: the row is the authority,
and chat-level state would describe turns a rewind or a restart had removed.
`crowd.next_turn` is pure, so all eight refusals are tested with no endpoint.

THREE RULES, EACH A BUG WRITTEN THE OTHER WAY ROUND

- `if not _advance_crowd(g): _drain(g)` -- advancing must *suppress* draining, or a
  queued human turn puts a second incomplete row beside the next speaker's.
- `_advance_crowd` refuses unless the finishing row is the newest, or regenerating
  member 2 creates a second member 3 and two chains race down one turn.
- an error skips one speaker and two in a row end the round: the usual failure is a
  small member's window overflowing, and `_drain`'s stop-on-error would kill every
  crowd at whichever member is smallest.

Each other speaker's turn is relabelled as attributed user content, which is both
how a model can disagree with words it did not write and how the history keeps
alternating. The per-speaker instruction is payload-only -- as a row it could be
dropped from the request by a `created_at` tie, and every later speaker would answer
it. Compaction, titling and the notification are gated to once per turn; `_inject`
is off during a round; the way back gets no tools and a member is treated as
unattended.

Membership stores the model as text with no foreign key: "Test & refresh" deletes
and recreates Model rows, and a cascade would empty the crowd out of every chat.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Homer
2026-09-26 13:38:52 +00:00
co-authored by Claude Opus 5
parent ac51dd46cc
commit da0797ccad
27 changed files with 3018 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/services/generation.py
+248 -10
View File
@@ -33,6 +33,7 @@ from lembas.security import permissions
from lembas.services import canvas as canvas_service
from lembas.services import chat as chat_service
from lembas.services import compaction as compaction_service
from lembas.services import crowd as crowd_service
from lembas.services import interaction, settings_store, tokens, tool_labels
from lembas.services import metrics as metrics_service
from lembas.services import prompts as prompts_service
@@ -222,6 +223,15 @@ class Generation:
# -- the one frame that reaches a browser after a reply is over.
drained: bool = False
injected_ids: list[str] = field(default_factory=list)
# A crowd round, seen from one speaker's side. `crowded` says this reply's
# ending handed the turn to the next speaker -- read by `_follow`, exactly as
# `drained` is, to put the next bubble on the `done` frame. `crowd_again` is
# the main model having called `crowd_again` on its closing turn: a field
# rather than a parse of the prose, for the reason `plan_json` exists, and
# on the generation rather than the row because it is a fact about this reply
# and dies with it.
crowded: bool = False
crowd_again: bool = False
# Images this reply produced, waiting to be bound to its message row. The
# runner writes the file and the `Attachment`; only `_persist` may say which
# turn it belongs to, which is the same division of labour `canvas` above
@@ -599,7 +609,16 @@ async def _run(generation: Generation) -> None:
# assembly path. Here rather than in post_message because that route's
# whole contract is to return immediately, and a three-second
# summarisation in front of it would break exactly that.
await _maybe_compact(generation)
# Once per turn, on the reply that opens it. Three reasons, and the
# first is the one that bites: `should_compact` reads `context_limit` off
# the *last complete* assistant turn's usage, which mid-crowd is the
# previous **speaker** -- so an 8k member at position three tells a 128k
# member at position four to compact. `last_complete`'s own promise that
# the cut lands on a reply and therefore leaves a history starting on a
# user turn is also false mid-round. And compacting during a round would
# ask the way back whether it disagrees with a summary of itself.
if _opens_the_turn_id(generation):
await _maybe_compact(generation)
# Before the session opens, for the same reason compaction is: the
# listing is an SSH round trip, and holding a database session across
@@ -614,7 +633,14 @@ async def _run(generation: Generation) -> None:
generation.error = "That chat no longer exists."
return
endpoint, model_id = chat_service.resolve_endpoint(db, chat)
# Who is answering, from the row being written into rather than
# from the chat. The row is durable and this generation is not: a
# restart turns `_follow` into `ensure`, which starts a brand new
# `_run` against the same message, and everything the request depends
# on has to survive that. It is also the only thing that can make the
# bubble's avatar and the model actually asked agree.
speaker = chat_service.speaker_for(db, chat, message)
endpoint, model_id = chat_service.resolve_endpoint(db, chat, speaker)
owner = db.get(User, chat.user_id)
# Before the request is built, not while it streams. Every other
@@ -631,13 +657,42 @@ async def _run(generation: Generation) -> None:
# Resolved once, so that what the loop is allowed to *run* is the
# same set the endpoint was *offered* -- not whatever happens to
# exist by the time a call comes back.
toolset = tools_service.resolve_tools(db, chat, owner)
# Where this speaker sits in a crowd round, if it is in one. Read
# once, here, and used for three decisions: which tools it may have,
# which instruction closes its request, and whether it may ask for
# another round.
crowd_state = crowd_service.state_of(message)
crowd_settings = settings_store.crowd(db)
may_ask_again = bool(
crowd_state is not None
and crowd_state.phase == crowd_service.PHASE_CLOSE
and crowd_state.round < int(crowd_settings["max_rounds"])
)
toolset = tools_service.resolve_tools(
db, chat, owner, speaker, crowd_turn=crowd_state, crowd_again=may_ask_again
)
offered = toolset.schemas
payload = chat_service.build_request(
db, chat, upto=message, tools=offered, user=owner, force_tool=generation.force_tool
db,
chat,
upto=message,
tools=offered,
user=owner,
force_tool=generation.force_tool,
speaker=speaker,
crowd_turn=crowd_state,
# Asked of the resolved set rather than of the settings: a model
# without the tools capability gets no tools at all, so inviting it
# to call `crowd_again` would be offering a choice it cannot
# express -- and `crowd.close_final` is the wording for that.
crowd_again="crowd_again" in toolset.by_name,
)
question = _question_from(payload)
needs_title = not chat.title_generated
# Once per turn. A crowd member titling the chat would name it after
# `_question_from`'s last user turn, which under the crowd relabelling
# is another model's quoted answer -- so the chat gets called after a
# quotation. The main model's first reply is the one that titles.
needs_title = not chat.title_generated and _opens_the_turn(message)
# An agent chat is titled from its opening words and never costs a
# model call for it. That prompt is a good title already -- somebody
# starting one states an objective, not a topic -- while an ordinary
@@ -654,16 +709,22 @@ async def _run(generation: Generation) -> None:
# Read here, with the rest, because titling happens after this
# session has closed and must not open another one.
title_prompt = prompts_service.resolve(db, "task.title")
tool_context = tools_service.context_for(db, owner, chat, tools=toolset)
tool_context = tools_service.context_for(
db, owner, chat, tools=toolset, speaker=speaker
)
model = chat_service.model_for(db, chat)
# The answering model's window, not the chat's. `_too_big` is the one
# budget that stops a reply dead rather than asking it to wrap up, so
# judging a small model's request against a large model's ceiling is
# how a reply fails with no explanation in it.
model = chat_service.model_row(db, speaker)
generation.context_limit = model.context_length if model is not None else 0
# Kept for `_inject`, which builds a user turn after this session
# has closed. A turn taken in mid-reply has to be shaped exactly as
# the same words typed a moment later would have been -- images to a
# vision model, a plain string to anything else, or the endpoint
# rejects the whole request.
vision = chat_service.model_supports(db, chat, "vision")
vision = chat_service.model_supports(db, chat, "vision", speaker=speaker)
# Resolved while the session is open, like everything else here.
# Empty for an admin and for a user in no group, which is every
# instance that has not set one -- see permissions.limits_for.
@@ -1082,7 +1143,18 @@ async def _run(generation: Generation) -> None:
# `_persist` is: `_follow` breaks the instant it sees that flag, and the
# frame it then sends is the one that has to carry the next turn's
# bubbles. There is no push channel that outlives a single reply.
_drain(generation)
#
# 🚨 Advancing a crowd round *suppresses* the drain, and the order of this
# sentence is the whole of it. Written the other way round -- advance, then
# drain -- a queued human turn typed during a round would create a second
# incomplete assistant row beside the next speaker's, which is two
# generations in one chat: the state `_reply_in_flight`, `_too_many_replies`,
# `wake.lock_for` and the superseded guards in `_persist`/`_drain` all exist
# to make unreachable, and whose symptom is a Stop button pointing at
# whichever bubble comes first in the document. The queue waits for the
# round; that is what a queue is for.
if not _advance_crowd(generation):
_drain(generation)
generation.done = True
generation.finished_at = datetime.now(UTC)
generation.touch()
@@ -2112,9 +2184,166 @@ def _drain(generation: Generation) -> None:
generation.drained = True
def _advance_crowd(generation: Generation) -> bool:
"""Start the next speaker of a crowd round. True if one was started.
The imperative shell around `crowd.next_turn`, which is pure -- so everything
interesting about this (the eight ways a round declines to continue) is tested
without an endpoint, and what is left here is reading rows and writing one.
Three refusals of its own, and each is a bug if it is left out:
* **Superseded.** The same guard `_persist` and `_drain` carry: this reply is
no longer the one registered for its message.
* **Stopped.** A person pressing Stop ends the round, not just the speaker
writing at the time. `_drain` refuses after a stop for the same reason and
it is the same reason here -- somebody asked for it to end.
* **Not the newest message.** `regenerate` calls `restart`, whose `finally`
runs this again -- and the speakers after it already exist. Without this,
regenerating member 2 creates a second member 3 and two chains race down one
turn. `_drain` never needed the guard because a queued row only ever exists
*forward* of the reply.
An **error** does not end the round: `crowd.next_turn` counts consecutive
failures and abandons after two, because the commonest failure in a crowd is a
small member's context window overflowing rather than a dead endpoint, and
ending the round there would kill every crowd at whichever member is smallest.
"""
owner = _RUNNING.get(generation.message_id)
if owner is not None and owner is not generation:
return False
if generation.stopped:
return False
try:
with session_scope() as db:
chat = db.get(Chat, generation.chat_id)
message = db.get(Message, generation.message_id)
if chat is None or message is None:
return False
settings = settings_store.crowd(db)
if not settings["enabled"] or not chat.crowd:
return False
if not crowd_service.is_newest(db, message):
return False
owner_user = db.get(User, chat.user_id)
speakers = crowd_service.member_speakers(db, chat, owner_user)
speakers = speakers[: int(settings["max_models"]) + 1]
state = crowd_service.state_of(message)
# The turn a round belongs to: the user message this all answers.
turn_id = state.turn if state is not None else _turn_anchor(db, message)
following = crowd_service.next_turn(
speakers=len(speakers),
state=state,
turn_id=turn_id,
again=generation.crowd_again,
errored=bool(generation.error),
max_rounds=int(settings["max_rounds"]),
wall_seconds=int(settings["wall_seconds"]),
)
if following is None:
return False
if following.stopped:
# Recorded on the row that ended it, so the transcript can say
# why a round stopped rather than simply stopping. Nothing else
# needs writing: there is no next speaker.
message.crowd_json = following.as_json()
db.commit()
return False
speaker = speakers[following.index]
placeholder = chat_service.create_message(
db,
chat,
ROLE_ASSISTANT,
"",
complete_=False,
model_id=speaker.model_id,
)
placeholder.connection_id = speaker.connection_id
placeholder.crowd_json = following.as_json()
db.commit()
chat_id, next_id = chat.id, placeholder.id
except Exception: # noqa: BLE001 - the reply is over either way
log.exception("could not advance the crowd in chat %s", generation.chat_id)
return False
# Outside the session, like `_drain`: this starts a task.
ensure(chat_id, next_id)
generation.crowded = True
return True
def _opens_the_turn(message: Message) -> bool:
"""Whether this reply is the first one answering a question.
True for every ordinary reply, and for a crowd only for the main model's
opening turn -- which is the one with no crowd state on it at all, because a
round begins when that reply *finishes*.
"""
return crowd_service.state_of(message) is None
def _opens_the_turn_id(generation: Generation) -> bool:
"""`_opens_the_turn` before the session is open, by message id.
`_maybe_compact` runs before `_run` reads anything, so this opens its own
session -- one primary-key lookup, and only on a chat that has a crowd.
"""
try:
with session_scope() as db:
message = db.get(Message, generation.message_id)
return message is None or _opens_the_turn(message)
except Exception: # noqa: BLE001 - compaction is best-effort anyway
return True
def _ends_the_turn(message: Message) -> bool:
"""Whether this reply is the last one the person is waiting for.
True for every ordinary reply, and for a crowd only on the main model's
closing turn. What is gated on it is everything that should happen once per
question rather than once per speaker: the unread dot, the web push, and the
chat's title.
"""
state = crowd_service.state_of(message)
if state is None:
return True
return state.phase == crowd_service.PHASE_CLOSE
def _turn_anchor(db, message: Message) -> str:
"""The user turn a round answers, for a round that is only now beginning.
The last user message at or before this reply. Only read once per round -- it
is carried on every later turn's state -- and it exists so a rewind can tell
which rows belonged to which question.
"""
row = db.scalars(
select(Message)
.where(
Message.chat_id == message.chat_id,
Message.role == ROLE_USER,
Message.created_at <= message.created_at,
)
.order_by(Message.created_at.desc(), Message.id.desc())
.limit(1)
).first()
return row.id if row is not None else ""
def _inject(generation: Generation, chat_id: str, vision: bool) -> dict | None:
"""Take the oldest waiting prompt into this reply, between two rounds.
⚠ Never during a crowd round. This restamps the placeholder's `created_at` so
the reply sorts after the prompt it answers, which mid-round reorders the
speakers underneath themselves -- and the round's own bookkeeping counts an
anchor that has moved. The turn stays queued and arrives after the round as a
clean new question with a round of its own, which is what `_drain` is for.
Marked delivered and committed *before* the request goes out, so this is
at-most-once. A crash in between loses the turn, which is recoverable --
the words are still in the transcript with Send now beside them. The other
@@ -2132,6 +2361,9 @@ def _inject(generation: Generation, chat_id: str, vision: bool) -> dict | None:
"""
try:
with session_scope() as db:
message = db.get(Message, generation.message_id)
if crowd_service.state_of(message) is not None:
return None
waiting = _next_waiting(db, chat_id)
if waiting is None:
return None
@@ -2265,7 +2497,13 @@ def _persist(generation: Generation, title: str, elapsed: float) -> None:
# clears this when it is next opened. Not for a temporary chat:
# there is no sidebar row for the dot, and the toast would name a
# chat nobody can navigate to.
if generation.followers == 0 and not chat.temporary:
# 🚨 Once per *turn*, not once per speaker. `announce_later` has no
# dedupe of its own -- its docstring says so, because every site that
# calls it runs once per arrival -- so a five-model crowd with nobody
# watching would be nine web pushes and nine sidebar toasts for one
# question. The closing speaker is the arrival; everybody before it is
# the middle of one.
if generation.followers == 0 and not chat.temporary and _ends_the_turn(message):
chat.unread = True
chat.unread_notified = False
# And out to any browser that asked to be told, which is the