Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Ask several questions on one card

Browse Source 
One `ask_user` call can now carry several questions, and they come back in a
single submit. Asking one at a time cost a round trip and an interruption
each, and by the third you had forgotten the first.

Each question becomes an item with its own key; several items share a call
index, because they belong to one call and one tool turn has to answer them
all. Each answer is quoted beside the question it belongs to -- with four on
a card, a bare list would leave the model matching them up by position and
sometimes getting it wrong.

Options are radios rather than submit buttons, so picking one does not send
the form while two other questions are still blank. What you type beats what
you picked: someone who writes in the box after clicking an option meant the
writing.

`_questions_in` also reads the shapes a small model actually sends -- a bare
`question` string, a list of plain strings, one object where a list belonged.
Getting that wrong costs a whole round trip and shows a card saying nothing.

Two test fixes, both mine. `test_posting_a_message_stores_both_turns` raced
the background generation it started: against a connection that refuses
instantly the reply sometimes won, writing the error and marking the row
complete before the assertions could read it. And the generation registry is
module-global, so a test that started a reply left an entry -- and a Task
belonging to a closed event loop -- for the rest of the session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-01 22:01:46 +02:00
parent 8c3fe97939
commit 671e49cae8
10 changed files with 673 additions and 138 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/services/generation.py
+78 -24
View File
@@ -156,7 +156,13 @@ def request_stop(message_id: str) -> bool:
return True
def answer(chat_id: str, interaction_id: str, *, choice: str, text: str) -> bool:
def answer(
chat_id: str,
interaction_id: str,
*,
verdict: str = "",
answers: dict[str, str] | None = None,
) -> bool:
"""Resolve whichever running reply is parked on this interruption.
A linear scan of the registry: it holds one entry per reply in flight, and
@@ -167,12 +173,12 @@ def answer(chat_id: str, interaction_id: str, *, choice: str, text: str) -> bool
pending = generation.pending
if generation.chat_id != chat_id or pending is None or pending.id != interaction_id:
continue
outcome = choice if choice in _ANSWERS else interaction.ANSWER
return pending.resolve(outcome, text=text or choice)
outcome = verdict if verdict in _VERDICTS else interaction.ANSWER
return pending.resolve(outcome, answers=answers)
return False
_ANSWERS = (interaction.ALLOW, interaction.ALLOW_ALWAYS, interaction.DENY)
_VERDICTS = (interaction.ALLOW, interaction.ALLOW_ALWAYS, interaction.DENY)
def _prune() -> None:
@@ -566,19 +572,51 @@ def _ask_items(context, calls: list[dict]) -> list[interaction.Item]:
if not isinstance(args, dict):
args = {}
options = [str(o).strip() for o in (args.get("options") or []) if str(o).strip()]
items.append(
interaction.Item(
index=index,
kind=interaction.KIND_QUESTION,
tool_name=call["name"],
title=str(args.get("question") or "").strip() or "A question for you",
options=tuple(options[: interaction.MAX_OPTIONS]),
for asked in _questions_in(args):
options = [str(o).strip() for o in (asked.get("options") or []) if str(o).strip()]
items.append(
interaction.Item(
index=index,
key=f"q{len(items)}",
kind=interaction.KIND_QUESTION,
tool_name=call["name"],
title=str(asked.get("question") or "").strip() or "A question for you",
options=tuple(options[: interaction.MAX_OPTIONS]),
)
)
)
return items
def _questions_in(args: dict) -> list[dict]:
"""The questions in one `ask_user` call, however it was spelled.
The schema asks for a list of objects, and a capable model sends that. A
small one sends a bare `question` string, or a list of plain strings, or
one object where a list belonged -- all of which mean something obvious, so
they are read rather than refused. Getting this wrong costs a whole round
trip and produces a card saying "A question for you" and nothing else.
"""
raw = args.get("questions")
if raw is None:
raw = args.get("question")
if raw is None:
return []
if isinstance(raw, str | dict):
raw = [raw]
if not isinstance(raw, list):
return []
out: list[dict] = []
for entry in raw[: interaction.MAX_QUESTIONS]:
if isinstance(entry, str) and entry.strip():
# A bare string, possibly alongside a sibling `options` that was
# meant to go with it -- which only makes sense for a lone question.
out.append({"question": entry, "options": args.get("options") if len(raw) == 1 else []})
elif isinstance(entry, dict) and str(entry.get("question") or "").strip():
out.append(entry)
return out
async def _authorise(generation, context, calls: list[dict]) -> dict[int, ToolOutcome]:
"""Which of this round's calls may run, and what the others answer instead.
@@ -602,18 +640,24 @@ async def _authorise(generation, context, calls: list[dict]) -> dict[int, ToolOu
generation.stopped = True
return {}
return {item.index: _answered(item, reply) for item in items}
# Grouped back by call, because one `ask_user` call may have carried several
# questions and the endpoint expects exactly one tool turn per call.
grouped: dict[int, list[interaction.Item]] = {}
for item in items:
grouped.setdefault(item.index, []).append(item)
return {index: _answered(asked, reply) for index, asked in grouped.items()}
def _answered(item: interaction.Item, reply: interaction.Reply) -> ToolOutcome:
"""One question's answer, as the model will read it back."""
def _answered(items: list[interaction.Item], reply: interaction.Reply) -> ToolOutcome:
"""What one `ask_user` call gets back, however many questions it put."""
event = {
"name": item.tool_name,
"name": items[0].tool_name,
"kind": "ask",
"label": "Asked you",
"query": item.title,
"query": "; ".join(item.title for item in items),
"results": [],
}
if reply.outcome == interaction.EXPIRED:
return ToolOutcome(
"They did not answer. Carry on as best you can without it, or say "
@@ -621,16 +665,26 @@ def _answered(item: interaction.Item, reply: interaction.Reply) -> ToolOutcome:
{**event, "status": "error", "error": "No answer.", "text": ""},
)
answer_text = reply.text.strip()
if not answer_text:
answered = [(item, reply.answer_to(item)) for item in items]
given = [(item, text) for item, text in answered if text]
if not given:
return ToolOutcome(
"They closed the question without answering.",
{**event, "status": "error", "error": "No answer.", "text": ""},
)
return ToolOutcome(
f"They answered: {answer_text}",
{**event, "status": "ok", "text": answer_text},
)
# Each answer is quoted next to the question it belongs to. With four
# questions on one card, a bare list of answers would leave the model
# matching them up by position and sometimes getting it wrong.
lines = [f"{item.title}\n → {text}" for item, text in given]
skipped = [item for item, text in answered if not text]
if skipped:
lines.append(
"They left unanswered: " + "; ".join(item.title for item in skipped)
)
body = "\n\n".join(lines)
return ToolOutcome(f"They answered:\n\n{body}", {**event, "status": "ok", "text": body})
async def _run_calls(
src/lembas/services/interaction.py
+31 -20
View File
@@ -49,18 +49,29 @@ PERMITTED = (ALLOW, ALLOW_ALWAYS)
# A card offering more than this many buttons is a card nobody reads.
MAX_OPTIONS = 6
# And more than this many questions at once is a form, not a conversation. A
# model that wants twenty answers should ask for four and then ask again with
# what it learned.
MAX_QUESTIONS = 8
@dataclass(frozen=True)
class Item:
"""One thing being asked about.
"""One thing being asked about: a single question, or one command.
`index` is the position of the call in its round, so a decision can be
matched back to the call it was about -- the tool turns have to line up with
the assistant turn's `tool_calls` or an endpoint pairs the wrong result with
the right id.
`index` is the position of the *call* in its round, so an answer can be
matched back to the call it belongs to -- the tool turns have to line up
with the assistant turn's `tool_calls`, or an endpoint pairs the wrong
result with the right id. Several items can share an index, because one
`ask_user` call may carry several questions.
`key` identifies this item within the card, and is what the form field is
named after. Stable and opaque: a question's own text would make a terrible
field name, and its position alone would collide across calls.
"""
index: int
key: str
kind: str
tool_name: str
title: str
@@ -83,18 +94,7 @@ class Interruption:
def kind(self) -> str:
return KIND_QUESTION if any(i.kind == KIND_QUESTION for i in self.items) else KIND_APPROVAL
@property
def options(self) -> tuple[str, ...]:
for item in self.items:
if item.options:
return item.options
return ()
@property
def allow_free_text(self) -> bool:
return any(item.allow_free_text for item in self.items)
def resolve(self, outcome: str, *, text: str = "") -> bool:
def resolve(self, outcome: str, *, answers: dict[str, str] | None = None) -> bool:
"""Complete this pause. Idempotent -- a second answer is ignored.
Returns whether this call was the one that answered it, which is what
@@ -103,14 +103,21 @@ class Interruption:
"""
if self._future is None or self._future.done():
return False
self._future.set_result(Reply(outcome=outcome, text=text))
self._future.set_result(Reply(outcome=outcome, answers=dict(answers or {})))
return True
@dataclass(frozen=True)
class Reply:
"""How a card was answered.
`answers` is keyed by `Item.key`, so a card carrying four questions comes
back as four answers in one go. An approval carries none: the verdict is
the whole of it.
"""
outcome: str
text: str = ""
answers: dict[str, str] = field(default_factory=dict)
@property
def permitted(self) -> bool:
@@ -121,6 +128,9 @@ class Reply:
"""Whether this outcome means the whole reply should stop."""
return self.outcome == CANCELLED
def answer_to(self, item: Item) -> str:
return (self.answers.get(item.key) or "").strip()
def build(
interaction_id: str, items: list[Item] | tuple[Item, ...], *, timeout: float
@@ -163,7 +173,7 @@ def summarise(items: tuple[Item, ...]) -> str:
if not items:
return ""
if items[0].kind == KIND_QUESTION:
return "Waiting for your answer…"
return "Waiting for your answer…" if len(items) == 1 else "Waiting for your answers…"
if len(items) == 1:
return f"Waiting for you to allow {items[0].tool_name}…"
return f"Waiting for you to allow {len(items)} actions…"
@@ -179,6 +189,7 @@ __all__ = [
"KIND_APPROVAL",
"KIND_QUESTION",
"MAX_OPTIONS",
"MAX_QUESTIONS",
"PERMITTED",
"Interruption",
"Item",
src/lembas/services/tools.py
+34 -17
View File
@@ -750,31 +750,48 @@ REGISTRY: dict[str, ToolDef] = {
name="ask_user",
family=FAMILY_ASK,
description=(
"Ask the person you are talking to a question, and wait for their "
"answer before going on. Use it when you genuinely need a decision "
"only they can make — which of several approaches to take, a "
"detail you cannot infer, permission for something consequential. "
"Offer options when there is a small set of sensible answers; they "
"can always type something else instead. Do not use it for "
"anything you can work out yourself, and never ask for a password, "
"a key or any other secret."
"Ask the person you are talking to one or more questions, and wait "
"for their answers before going on. Use it when you genuinely need "
"a decision only they can make — which of several approaches to "
"take, a detail you cannot infer, permission for something "
"consequential. Offer options when there is a small set of "
"sensible answers; they can always write their own instead. "
"\n\n"
"Ask everything you need in ONE call: they answer the whole card at "
"once and it costs them a single interruption, where asking twice "
"in a row costs two. Do not use it for anything you can work out "
"yourself, and never ask for a password, a key or any other secret."
),
parameters=_object(
{
"question": {
**_STRING,
"description": "One clear question, in plain language.",
},
"options": {
"questions": {
"type": "array",
"items": _STRING,
"description": (
"Up to six answers to offer as buttons. Optional; they "
"can always write their own."
"The questions to put, answered together. Ask up to "
"about four at a time; more than that is a form, not a "
"conversation."
),
"items": {
"type": "object",
"properties": {
"question": {
**_STRING,
"description": "One question, in plain language.",
},
"options": {
"type": "array",
"items": _STRING,
"description": (
"Up to six answers to offer for this "
"question. Optional."
),
},
},
"required": ["question"],
},
},
},
["question"],
["questions"],
),
# Never resolved by this runner. The reader answers it, in every
# mode, and the loop turns their answer into the outcome -- see