Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

A question that offers real choices, and says how many you may take

Browse Source 
Three things about `ask_user`, all of them about the card being answerable
rather than about the tool being callable.

Options are required now, and they are objects: a label, and a line of
description where the label alone does not say what choosing it would mean.
"Rewrite it" and "Patch it" are two words that do not tell you which one loses
your uncommitted work. They stack one per line, because a row of chips has
nowhere to put the second line and no room to read the first.

The model says whether they are exclusive. Only it knows whether its options are
alternatives or a set, and the card has to show which -- a radio group offered
where checkboxes were meant loses every answer but one. Exclusive is the
default, being the cheaper mistake. A `multiple` question posts the same field
name once per ticked box, so the endpoint gathers choices into a list; the
`setdefault` it did before kept the first and dropped the rest, which is an
answer that says something the reader did not.

And "Something else" is added here, on every question, with the box behind it
revealed by `:has()` and no JavaScript at all. The model is told never to write
an "other" option of its own, because its version would be a choice with no box
behind it -- a word submitted that means nothing. It carries a sentinel rather
than an answer, and the endpoint swaps in what was typed beside it, or drops it
when the box was left empty rather than telling the model the answer is
"__other__".

Typing no longer beats picking. That rule belonged to a box that was always
visible next to the options; this one only exists once its own option is chosen,
so picking is the answer and the box is one of the things you can pick.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-04 19:34:47 +02:00
parent 74a3c0f9d3
commit 3065878bd0
9 changed files with 524 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+32 -4
View File
@@ -20,7 +20,7 @@ lembas info # paths + counts, useful when confused
lembas secret-key # generate LEMBAS_SECRET_KEY
lembas create-admin # create or promote an admin
pytest # 1483 tests, ~90s
pytest # 1499 tests, ~92s
# PLAN.md tracks what is and is not built
ruff check . # lint (line length 100)
python scripts/build_artwork.py # regenerate artwork (SVG + PWA icons;
@@ -375,9 +375,37 @@ Each becomes an `Item` with its own `key`; several items can share an `index`
because they belong to one call, and one tool turn answers them all with each
answer quoted beside its question. Asking one at a time would cost a round trip
and an interruption each, and answering the third would mean having forgotten
the first. `_questions_in` reads the singular form and bare strings too: a small
model sends something close to the schema rather than the schema, and getting it
wrong costs a whole round trip to show a card that says nothing.
the first. `_questions_in` reads the singular form and bare strings too, and
`_options_in` reads an option written as a bare string as well as one written as
`{label, description}`: a small model sends something close to the schema rather
than the schema, and getting it wrong costs a whole round trip to show a card
that says nothing.
**"Something else" is added by the template, never by the model, and that is why
the model is told not to offer one.** Every question gets it, with a text box
behind it revealed by `:has()` — no JavaScript, and nothing that can fall out of
step with the control. An "Other" the model wrote would be an option with no box
behind it: a choice that submits a word and means nothing. It carries
`interaction.OTHER` as its value rather than an answer, and the endpoint replaces
it with whatever was typed beside it — or drops it entirely when the box is
empty, because telling the model the answer is `__other__` is exactly the shape
of thing it would try to act on.
Options are **required** now, and stacked one per line rather than in a row: an
option carries an optional `description`, and a row of chips has nowhere to put
the second and no room to read the first. A question with no options is a blank
box, which asks the reader to do the thinking the model was meant to do.
**The model says whether its options are exclusive**, because only it knows
whether they are alternatives or a set — `multiple` picks radios or checkboxes,
and the default is exclusive because that is the cheaper mistake: a radio group
where checkboxes were meant costs one clarifying round, while checkboxes for
alternatives invite an answer that contradicts itself. A `multiple` question
posts the same field name once per ticked box, so the endpoint gathers `choice.`
fields into a **list** and joins them; the `setdefault` it used to do kept the
first and lost the rest, which is an answer that says something the reader did
not. Typing no longer beats picking — that rule belonged to a box that was always
visible, and this one only exists when its own option is chosen.
**A paused reply is deliberately not `done`.** That is what lets a page reload
reattach to it. Its *timeout* is what stops it lingering, not `_prune`, which
PLAN.md
+1 -1
View File
@@ -7,7 +7,7 @@ that would be expensive to revisit. Kept current as work lands; the detail of
**Status:** usable daily. Streaming chat, attachments, reasoning, tool calling
with web search, custom HTTP tools and MCP servers, agent chats that work on a
machine over SSH, a knowledge library, notes, memory and skills, speech in and
out, users and groups, model administration, installable as an app. 1483 tests,
out, users and groups, model administration, installable as an app. 1499 tests,
`ruff` clean.
---
src/lembas/api/chats.py
+32 -5
View File
@@ -867,7 +867,10 @@ def _ask_html(chat_id: str, pending) -> str:
if pending is None:
return ""
return templates.get_template("chat/_interaction.html").render(
{"ask": pending, "chat_id": chat_id}
# The sentinel the "Something else" row submits, passed in rather than
# written into the template, so the value the card sends and the value
# this module looks for cannot drift apart.
{"ask": pending, "chat_id": chat_id, "other_value": interaction.OTHER}
)
@@ -1412,16 +1415,40 @@ async def answer_interaction(
form = await request.form()
verdict = str(form.get("verdict") or "").strip()
answers: dict[str, str] = {}
# Gathered in two passes because one field can now arrive several times: a
# question the model marked `multiple` is checkboxes, and every ticked one
# posts under the same name. A `setdefault` would keep the first and lose
# the rest, which is an answer that says something the reader did not.
chosen: dict[str, list[str]] = {}
typed: dict[str, str] = {}
for field, value in form.multi_items():
kind, _, key = str(field).partition(".")
if not key or kind not in ("choice", "text"):
continue
written = str(value).strip()
if kind == "text" and written:
if kind == "text":
typed[key] = written
elif written:
chosen.setdefault(key, []).append(written)
answers: dict[str, str] = {}
for key, picks in chosen.items():
# The "Something else" row carries a sentinel, not an answer: what it
# means is whatever was typed beside it. Dropped entirely when the box
# was left empty, so ticking it and writing nothing is the same as not
# ticking it -- rather than the model being told the answer is
# "__other__", which is the shape of thing it would try to act on.
parts = [pick for pick in picks if pick != interaction.OTHER]
if interaction.OTHER in picks and typed.get(key):
parts.append(typed[key])
if parts:
answers[key] = ", ".join(parts)
# A box with no choice beside it: the approval card's corrected command,
# which is the one place `text.` still stands on its own.
for key, written in typed.items():
if written and key not in answers and key not in chosen:
answers[key] = written
elif kind == "choice" and written:
answers.setdefault(key, written)
# Read and recorded *before* resolving: `interaction.wait_for` clears
# `generation.pending` in its `finally`, so a moment later there is nothing
src/lembas/services/generation.py
+39 -2
View File
@@ -1377,7 +1377,6 @@ def _ask_items(context, calls: list[dict], arguments: list[dict]) -> list[intera
args = arguments[index]
for asked in _questions_in(args):
options = [str(o).strip() for o in (asked.get("options") or []) if str(o).strip()]
items.append(
interaction.Item(
index=index,
@@ -1385,12 +1384,50 @@ def _ask_items(context, calls: list[dict], arguments: list[dict]) -> list[intera
kind=interaction.KIND_QUESTION,
tool_name=call["name"],
title=str(asked.get("question") or "").strip() or "A question for you",
options=tuple(options[: interaction.MAX_OPTIONS]),
options=_options_in(asked),
multiple=bool(asked.get("multiple")),
)
)
return items
def _options_in(asked: dict) -> tuple[interaction.Option, ...]:
"""The choices offered for one question, however they were spelled.
The schema asks for objects with a `label` and an optional `description`,
and a capable model sends that. A small one sends a list of bare strings --
which is what the schema asked for until recently and is what most examples
of this pattern look like -- so that is read as a label with no description
rather than refused. Anything else in the list is dropped rather than
stringified, because `{'a': 1}` rendered as a choice is worse than one
choice fewer.
An option meaning "something else" is **not** added here. It belongs to the
template, which adds it to every question and owns the box behind it; adding
it to the data would make it indistinguishable from one the model wrote.
"""
out: list[interaction.Option] = []
for raw in asked.get("options") or []:
if isinstance(raw, str):
label, description = raw.strip(), ""
elif isinstance(raw, dict):
label = str(raw.get("label") or raw.get("name") or raw.get("value") or "").strip()
description = str(raw.get("description") or "").strip()
else:
continue
if not label:
continue
out.append(
interaction.Option(
label=label[: interaction.MAX_OPTION_CHARS],
description=description[: interaction.MAX_OPTION_CHARS],
)
)
if len(out) >= interaction.MAX_OPTIONS:
break
return tuple(out)
def _questions_in(args: dict) -> list[dict]:
"""The questions in one `ask_user` call, however it was spelled.
src/lembas/services/interaction.py
+38 -1
View File
@@ -54,6 +54,32 @@ MAX_OPTIONS = 6
# what it learned.
MAX_QUESTIONS = 8
# The value the "Something else" row submits. A sentinel rather than a real
# option, because it is the one choice on the card the model did not write: it
# is added by this code, always, to every question. That is the whole reason the
# model is told never to offer an "Other" of its own -- two of them is one that
# does nothing, and the model's version would have no box behind it.
OTHER = "__other__"
# How many characters of an option's description are kept. It is a sentence
# explaining a choice, not a paragraph, and it is model output landing in a
# card somebody is meant to read at a glance.
MAX_OPTION_CHARS = 240
@dataclass(frozen=True)
class Option:
"""One answer offered for a question.
A `label` alone reads as a button; the optional `description` is what makes
a real choice possible -- "Rewrite it" and "Patch it" say nothing about
which loses your uncommitted work. Both are model output and are escaped
where they are shown.
"""
label: str
description: str = ""
@dataclass(frozen=True)
class Item:
@@ -83,7 +109,18 @@ class Item:
# explanation somebody reads as the application's own would be a card
# vouching for it.
purpose: str = ""
options: tuple[str, ...] = ()
options: tuple[Option, ...] = ()
# Whether more than one option may be chosen. The model says which, because
# only the model knows whether its options are alternatives ("rewrite or
# patch") or a set ("which of these to include"). Exclusive is the default:
# a radio group offered where checkboxes were meant costs one clarifying
# round, while checkboxes offered for alternatives invite an answer that
# contradicts itself.
multiple: bool = False
# Whether "Something else" is offered, with the box behind it. True for a
# question -- the options are the model's guess at the answers and it can be
# wrong -- and false for an approval, where the choice is Allow or Don't and
# a third way out would mean nothing.
allow_free_text: bool = True
# Whether `detail` can be corrected before this is allowed. Only where the
# detail *is* one argument and can be put back where it came from -- a tool
src/lembas/services/tools.py
+47 -7
View File
@@ -1005,9 +1005,20 @@ REGISTRY: dict[str, ToolDef] = {
"for their answers before going on. Use it when you genuinely need "
"a decision only they can make — which of several approaches to "
"take, a detail you cannot infer, permission for something "
"consequential. Offer options when there is a small set of "
"sensible answers; they can always write their own instead. "
"\n\n"
"consequential.\n\n"
"**Always give options.** A question with no options is a blank box, "
"and a blank box asks the person to do the thinking you were meant "
"to do: offer the two to six answers you actually think are "
"plausible, in the order you would recommend them. Do NOT add an "
"option meaning “other”, “something else”, “none of these” or "
"“let me type it” — one is added for you, on every question, with a "
"box behind it. Yours would have no box and would do nothing.\n\n"
"Say whether the options are exclusive. `multiple: false` (the "
"default) is for alternatives, where picking one rules out the "
"rest; `multiple: true` is for a set, where any number may be "
"chosen. Give an option a `description` wherever the label alone "
"does not say what choosing it would mean — that is what makes a "
"real decision possible rather than a guess between two words.\n\n"
"Ask everything you need in ONE call: they answer the whole card at "
"once and it costs them a single interruption, where asking twice "
"in a row costs two. Do not use it for anything you can work out "
@@ -1031,14 +1042,43 @@ REGISTRY: dict[str, ToolDef] = {
},
"options": {
"type": "array",
"items": _STRING,
"description": (
"Up to six answers to offer for this "
"question. Optional."
"The answers to offer, two to six of them. "
"Required. Never include an “other” or "
"“something else” option — one is always "
"added for you."
),
"items": {
"type": "object",
"properties": {
"label": {
**_STRING,
"description": (
"The choice itself, in a few words."
),
},
"description": {
**_STRING,
"description": (
"Optional: one line on what "
"choosing this would mean, where "
"the label alone does not say."
),
},
},
"required": ["label"],
},
},
"multiple": {
"type": "boolean",
"description": (
"Whether more than one option may be chosen. "
"False (the default) for alternatives, true "
"for a set."
),
},
},
"required": ["question"],
"required": ["question", "options"],
},
},
},
src/lembas/web/static/css/chat.css
+60 -1
View File
@@ -494,9 +494,68 @@
border-top: 1px solid var(--border);
}
.interaction__title { margin: 0; padding: 0; color: var(--ink); font-weight: 500; }
.interaction__options { display: flex; flex-wrap: wrap; gap: var(--sp-2); }
/* Stacked, one per line. A row of chips was fine while an option was two words
and nothing else; an option now carries a description as well, and a row has
nowhere to put the second and no room to read the first. */
.interaction__options { display: flex; flex-direction: column; gap: var(--sp-2); }
.interaction__note { color: var(--ink-faint); font-size: var(--text-xs); }
/* One option: the control, then a label and an optional line under it. The
whole row is the target -- it is a `<label>`, so the description is as
clickable as the name, which is what makes a long option readable rather
than a small circle to aim at. */
.interaction__option {
display: flex;
align-items: flex-start;
gap: var(--sp-3);
padding: var(--sp-2) var(--sp-3);
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
background: var(--surface-raised);
cursor: pointer;
transition: background var(--transition-fast), border-color var(--transition-fast);
}
.interaction__option:hover { background: var(--surface-hover); }
/* The native control, kept: it carries the exclusive-versus-multiple meaning
that the whole feature turns on, and a radio and a checkbox have to look
different or the card lies about how many answers it will take. */
.interaction__option input {
margin: 0;
margin-top: 0.15rem;
flex: none;
accent-color: var(--accent);
}
.interaction__option:has(input:checked) {
border-color: var(--accent);
background: var(--surface-active);
}
.interaction__option:has(input:focus-visible) {
outline: 2px solid var(--accent);
outline-offset: 2px;
}
.interaction__option-body {
display: flex;
flex-direction: column;
gap: var(--sp-1);
min-width: 0;
}
.interaction__option-name { color: var(--ink); font-size: var(--text-sm); }
.interaction__option-note {
color: var(--ink-muted);
font-size: var(--text-xs);
line-height: var(--leading-normal);
}
/* The box behind "Something else". Hidden until that row is chosen, revealed by
`:has()` on the row itself -- no JavaScript, and nothing that can fall out of
step with the control. It is always in the DOM and always submitted; the
endpoint uses it only when the sentinel beside it was actually chosen. */
.interaction__other-input { display: none; margin-top: var(--sp-1); }
.interaction__option--other:has(input:checked) .interaction__other-input {
display: block;
}
/* An option. A radio, so picking one unpicks the last, but shaped like the
button it reads as. */
.chip { position: relative; display: inline-flex; }
src/lembas/web/templates/chat/_interaction.html
+42 -13
View File
@@ -31,27 +31,56 @@
{% if ask.kind == "question" %}
{% for item in ask.items %}
{# Stacked, one per line, never in a row. An option carries a label and
often a description, and a row of chips has nowhere to put the second
and no room to read the first.
Radio or checkbox by what the model said: `multiple` is false for
alternatives and true for a set. The name is the same either way, so
the endpoint reads one value or several without knowing which it
asked for. #}
<fieldset class="interaction__question">
<legend class="interaction__title">{{ item.title }}</legend>
{% if item.options %}
<div class="interaction__options">
{% for option in item.options %}
<label class="chip">
<input type="radio" name="choice.{{ item.key }}" value="{{ option }}">
<span>{{ option }}</span>
<label class="interaction__option">
<input type="{{ 'checkbox' if item.multiple else 'radio' }}"
name="choice.{{ item.key }}" value="{{ option.label }}">
<span class="interaction__option-body">
<span class="interaction__option-name">{{ option.label }}</span>
{% if option.description %}
<span class="interaction__option-note">{{ option.description }}</span>
{% endif %}
</span>
</label>
{% endfor %}
</div>
{% endif %}
{% if item.allow_free_text %}
{# Never type="password". A model talked into asking for a credential
must not be handed a field that looks built for one, and a chat
transcript is not a place to keep secrets. #}
<input class="input" type="text" name="text.{{ item.key }}" autocomplete="off"
placeholder="{{ 'Or write your own answer…' if item.options else 'Your answer…' }}">
{% endif %}
{% if item.allow_free_text %}
{# Added here and nowhere else: on every question, by this template,
never by the model. That is why the tool tells it not to write an
"other" option of its own -- two of them is one that does nothing,
and its version would have no box behind it.
The box is revealed by `:has()` on the row, so this needs no
JavaScript and cannot fall out of step with the checkbox. It is
always submitted; the endpoint uses it only when this option is
actually chosen. #}
<label class="interaction__option interaction__option--other">
<input type="{{ 'checkbox' if item.multiple else 'radio' }}"
name="choice.{{ item.key }}" value="{{ other_value }}">
<span class="interaction__option-body">
<span class="interaction__option-name">Something else</span>
{# Never type="password". A model talked into asking for a
credential must not be handed a field that looks built for one,
and a chat transcript is not a place to keep secrets. #}
<input class="input interaction__other-input" type="text"
name="text.{{ item.key }}" autocomplete="off"
placeholder="Your own answer…">
</span>
</label>
{% endif %}
</div>
</fieldset>
{% endfor %}
tests/test_agent_interaction.py
+233 -10
View File
@@ -137,7 +137,7 @@ async def test_the_reply_pauses_and_the_card_describes_the_question(scripted):
pending = await _until_paused(generation)
assert pending.kind == interaction.KIND_QUESTION
assert pending.items[0].title == "Tea or coffee?"
assert pending.items[0].options == ("Tea", "Coffee")
assert [o.label for o in pending.items[0].options] == ["Tea", "Coffee"]
assert "Waiting for your answer" in generation.status
pending.resolve(interaction.ANSWER, answers={"q0": "Tea"})
@@ -261,6 +261,13 @@ def test_an_interruption_with_no_future_cannot_be_resolved():
assert pause.resolve(interaction.ANSWER) is False
def _opts(*labels: str) -> tuple[interaction.Option, ...]:
"""Options as the parser produces them: a label and an optional line under
it. Bare strings are what a model sends and what `_options_in` normalises;
`Item` itself only ever holds the normalised form."""
return tuple(interaction.Option(label=label) for label in labels)
def _item(key: str = "q0", *, title: str = "Tea or coffee?", **kwargs) -> interaction.Item:
return interaction.Item(
index=0,
@@ -374,12 +381,14 @@ def test_the_card_shows_the_question_and_its_options():
pause = interaction.Interruption(
id="p1",
items=(
_item(title="Tea or coffee?", options=("Tea", "Coffee")),
_item(title="Tea or coffee?", options=_opts("Tea", "Coffee")),
),
)
html = _render(pause)
assert "Tea or coffee?" in html
assert 'value="Tea"' in html and 'value="Coffee"' in html
# Stacked, never in a row: an option carries a description now.
assert "interaction__option" in html
assert 'hx-post="/api/chats/chat-1/interaction/p1"' in html
assert "The model is asking you" in html, "attributed to the model, not to LLeMbas"
@@ -454,7 +463,7 @@ def test_everything_on_the_card_is_escaped():
_item(
title="<img src=x onerror=alert(1)>",
detail="rm -rf / <script>",
options=('"><script>bad</script>',),
options=_opts('"><script>bad</script>'),
),
),
)
@@ -675,7 +684,7 @@ def test_the_card_renders_every_question_with_its_own_fields():
pause = interaction.Interruption(
id="p1",
items=(
_item("q0", title="Which database?", options=("SQLite", "Postgres")),
_item("q0", title="Which database?", options=_opts("SQLite", "Postgres")),
_item("q1", title="Which port?"),
),
)
@@ -684,13 +693,68 @@ def test_the_card_renders_every_question_with_its_own_fields():
assert "Which database?" in html and "Which port?" in html
# Radios rather than submit buttons: picking one must not send the form
# while two other questions are still blank.
assert 'type="radio" name="choice.q0" value="SQLite"' in html
assert 'name="choice.q0" value="SQLite"' in html
assert 'name="text.q0"' in html and 'name="text.q1"' in html
# A question with no options still gets somewhere to write.
assert 'name="choice.q1"' not in html
# Every question gets "Something else" and the box behind it, including one
# the model gave no options for -- that row is added here, never by the
# model, which is why the tool tells it not to write an "other" of its own.
assert html.count(f'value="{interaction.OTHER}"') == 2
assert html.count("Something else") == 2
assert html.count("Send answers") == 1, "one submit for the whole card"
def test_options_are_radios_when_exclusive_and_checkboxes_when_not():
"""The model says which, and the card has to show it: a radio group offered
where a set was meant loses every answer but one, and checkboxes offered for
alternatives invite an answer that contradicts itself."""
one = _render(
interaction.Interruption(
id="p1", items=(_item("q0", options=_opts("Tea", "Coffee")),)
)
)
many = _render(
interaction.Interruption(
id="p1", items=(_item("q0", options=_opts("Tea", "Coffee"), multiple=True),)
)
)
assert 'type="radio"' in one and 'type="checkbox"' not in one
assert 'type="checkbox"' in many and 'type="radio"' not in many
# Including the "Something else" row, which has to match the rest or the
# card offers a set and then refuses to take one.
assert one.count('type="radio"') == many.count('type="checkbox"') == 3
def test_an_option_can_carry_a_line_explaining_it():
""""Rewrite it" and "Patch it" say nothing about which loses your
uncommitted work."""
pause = interaction.Interruption(
id="p1",
items=(
_item(
"q0",
options=(
interaction.Option("Rewrite it", "Replaces the file whole."),
interaction.Option("Patch it"),
),
),
),
)
html = _render(pause)
assert "Replaces the file whole." in html
assert html.count("interaction__option-note") == 1, "and only where there is one"
def test_an_options_description_is_escaped_like_everything_else():
pause = interaction.Interruption(
id="p1",
items=(_item("q0", options=(interaction.Option("ok", '<script>bad</script>'),)),),
)
assert "<script>bad" not in _render(pause)
def test_a_single_question_says_answer_rather_than_send_answers():
html = _render(interaction.Interruption(id="p1", items=(_item(),)))
assert "Send answers" not in html
@@ -719,8 +783,11 @@ def test_the_endpoint_gathers_every_answer_at_once(client, db, registered, user_
data={
"choice.q0": "Postgres",
"text.q0": "",
"choice.q1": "Yes",
# "Something else" ticked, and the box beside it filled in.
"choice.q1": interaction.OTHER,
"text.q1": "actually, later",
# A box with no choice beside it: the approval card's corrected
# command, which is the one place `text.` stands on its own.
"text.q2": "5433",
},
)
@@ -729,8 +796,8 @@ def test_the_endpoint_gathers_every_answer_at_once(client, db, registered, user_
chats_api.generation_service.answer = original
assert seen["answers"] == {
"q0": "Postgres", # picked, nothing written
"q1": "actually, later", # written wins over picked
"q0": "Postgres", # picked
"q1": "actually, later", # "Something else", and what was written
"q2": "5433", # written, nothing to pick
}
assert seen["verdict"] == ""
@@ -754,3 +821,159 @@ def test_the_endpoint_passes_a_verdict_through_untouched(client, db, registered,
assert seen["verdict"] == "allow"
assert seen["answers"] == {}
# --- Options as a model actually sends them -----------------------------------
def test_options_are_read_as_objects_or_as_bare_strings():
"""The schema asks for objects with a label. A small model sends a list of
strings -- which is what the schema asked for until recently, and what most
examples of this pattern look like -- so that is read rather than refused."""
objects = generation_service._options_in(
{"options": [{"label": "Rewrite", "description": "Replaces it whole."}, {"label": "Patch"}]}
)
strings = generation_service._options_in({"options": ["Rewrite", "Patch"]})
assert [(o.label, o.description) for o in objects] == [
("Rewrite", "Replaces it whole."),
("Patch", ""),
]
assert [(o.label, o.description) for o in strings] == [("Rewrite", ""), ("Patch", "")]
def test_an_option_that_is_neither_is_dropped_rather_than_stringified():
"""`{'a': 1}` rendered as a choice is worse than one choice fewer."""
options = generation_service._options_in({"options": ["ok", 7, None, {"nope": 1}, ""]})
assert [o.label for o in options] == ["ok"]
def test_too_many_options_are_cut_off():
options = generation_service._options_in({"options": [f"o{i}" for i in range(20)]})
assert len(options) == interaction.MAX_OPTIONS
def test_the_parser_never_adds_something_else_itself():
"""It belongs to the template, which adds it to every question and owns the
box behind it. In the data it would be indistinguishable from one the model
wrote -- and the model is told not to write one precisely because this row
already exists."""
options = generation_service._options_in({"options": ["Tea", "Coffee"]})
assert all(o.label != "Something else" for o in options)
assert all(o.label != interaction.OTHER for o in options)
async def test_whether_several_may_be_chosen_survives_into_the_card(db, user_id, monkeypatch):
"""Through the real loop, because the flag has to reach `Item` -- reading it
off the arguments and never passing it on would look identical here."""
import json as _json
chat_id, message_id = _chat_that_can_ask(db, user_id)
arguments = _json.dumps(
{
"questions": [
{"question": "Which of these?", "options": ["a", "b"], "multiple": True},
{"question": "And which one?", "options": ["c", "d"]},
]
}
)
chunk = {
"choices": [
{
"delta": {
"tool_calls": [
{
"index": 0,
"id": "c1",
"function": {"name": "ask_user", "arguments": arguments},
}
]
}
}
]
}
monkeypatch.setattr(
generation_service, "stream_chat", _stub_stream([[chunk], [_text_chunk("ok")]], [])
)
generation = generation_service.Generation(chat_id=chat_id, message_id=message_id)
task = asyncio.create_task(generation_service._run(generation))
pending = await _until_paused(generation)
assert pending.items[0].multiple is True
# Exclusive is the default: a radio group offered where checkboxes were
# meant costs one clarifying round, while checkboxes offered for
# alternatives invite an answer that contradicts itself.
assert pending.items[1].multiple is False
pending.resolve(interaction.ANSWER, answers={"q0": "a", "q1": "c"})
await task
def test_several_ticked_answers_all_come_back(client, db, registered, user_id):
"""A `multiple` question posts the same field name once per ticked box. The
old reader kept the first and lost the rest, which is an answer that says
something the reader did not."""
chat_id, _message_id = _chat_that_can_ask(db, user_id)
seen: dict = {}
from lembas.api import chats as chats_api
original = chats_api.generation_service.answer
chats_api.generation_service.answer = lambda chat, iid, *, verdict="", answers=None: (
seen.update(answers=answers) or True
)
try:
client.post(
f"/api/chats/{chat_id}/interaction/p1",
data={"choice.q0": ["Tea", "Coffee"], "text.q0": ""},
)
finally:
chats_api.generation_service.answer = original
assert seen["answers"] == {"q0": "Tea, Coffee"}
def test_something_else_ticked_with_an_empty_box_says_nothing(client, db, registered, user_id):
"""Rather than telling the model the answer is "__other__", which is exactly
the shape of thing it would try to act on."""
chat_id, _message_id = _chat_that_can_ask(db, user_id)
seen: dict = {}
from lembas.api import chats as chats_api
original = chats_api.generation_service.answer
chats_api.generation_service.answer = lambda chat, iid, *, verdict="", answers=None: (
seen.update(answers=answers) or True
)
try:
client.post(
f"/api/chats/{chat_id}/interaction/p1",
data={"choice.q0": interaction.OTHER, "text.q0": " "},
)
finally:
chats_api.generation_service.answer = original
assert seen["answers"] == {}
def test_a_ticked_option_and_something_else_come_back_together(client, db, registered, user_id):
chat_id, _message_id = _chat_that_can_ask(db, user_id)
seen: dict = {}
from lembas.api import chats as chats_api
original = chats_api.generation_service.answer
chats_api.generation_service.answer = lambda chat, iid, *, verdict="", answers=None: (
seen.update(answers=answers) or True
)
try:
client.post(
f"/api/chats/{chat_id}/interaction/p1",
data={"choice.q0": ["Tea", interaction.OTHER], "text.q0": "or juice"},
)
finally:
chats_api.generation_service.answer = original
assert seen["answers"] == {"q0": "Tea, or juice"}