Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Live Markdown, stop, rewind, custom picker, dialogs

Browse Source 
Seven things.

**Reasoning starts closed.** The answer is what the reader is waiting
for; the thinking is one click away.

**Image borders.** .attachments__image was a block-level <a>, so its
border stretched the full column around a narrow picture. inline-block,
and the frame is the picture. Same fix for the composer thumbnail.

**Markdown now renders during the stream.** The generator re-renders the
answer so far and sends it as a `render` event at most every 100ms,
swapped with innerHTML, instead of appending escaped tokens and
formatting everything at the end. Re-rendering whole rather than
appending is the point: a list or a code fence is only correct once its
context exists, and partial syntax resolves itself as more arrives.
Measured against a live model: 29 render events, formatting visible from
the first content token.

**Stop button.** A stop request goes into an in-process set the
generator checks between chunks; whatever arrived is kept, because a
half-written answer the reader chose to cut short is still worth having.
Measured: stream ended 0.2s after the request, 1155 characters
preserved, message marked stopped rather than errored. Navigating away
does the same thing via CancelledError.

**Rewind and edit.** Edit one of your own turns and everything after it
is deleted, then the conversation runs on from there. Deliberately not
branching: that needs a UI for choosing between versions, and "go back
and try again from here" is what was asked for. The form states how many
messages will be discarded before you confirm.

**Custom model picker.** A <select> renders only text in an <option>, so
it can never show an avatar. Built from buttons and a hidden input, with
descriptions, capability tags, a filter box past eight models, and
arrow-key navigation written out by hand since there is no native widget
doing it.

**Notification system.** lembas.notify/confirm/prompt in ui.js, built on
<dialog> so focus trapping, Escape and page inertness come from the
browser. htmx:confirm is intercepted, so every existing hx-confirm gets
the themed dialog with no change at the call site; the browser's grey
confirm() is gone from every template.

230 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 14:33:04 +02:00
parent f744232d25
commit a071d8486b
20 changed files with 1151 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/api/chats.py
+173 -2
View File
@@ -9,6 +9,7 @@ from collections.abc import AsyncIterator
from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
from fastapi.responses import HTMLResponse, Response, StreamingResponse
from sqlalchemy import select
from sqlalchemy.orm import Session as DBSession
from lembas.api.deps import Db, RequiredUser, require_permission
@@ -32,6 +33,21 @@ log = logging.getLogger(__name__)
router = APIRouter(prefix="/api/chats", tags=["chats"])
# Message ids whose generation has been asked to stop. The generator checks
# this between chunks and finalises with whatever it has.
#
# In-process, which is correct for the single-worker deployment this ships
# with: the request that stops a stream and the task producing it are in the
# same process. Running multiple workers would need this in the database or a
# broker instead -- see deploy/README.md.
_CANCELLED: set[str] = set()
# How often the partially rendered reply is pushed to the browser. Markdown is
# re-rendered from scratch each time, so this trades a little server work for
# formatting that appears as the model writes rather than all at once at the
# end. 100ms is below the threshold where the eye reads it as stepping.
RENDER_INTERVAL = 0.1
def _owned_chat(db: DBSession, chat_id: str, user_id: str) -> Chat:
chat = db.get(Chat, chat_id)
@@ -221,6 +237,9 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
splitter = ReasoningSplitter()
started = time.monotonic()
reasoning_started: float | None = None
last_render = 0.0
dirty = False
stopped = False
try:
endpoint, model_id = chat_service.resolve_endpoint(db, chat)
@@ -261,18 +280,31 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
if reasoning_started is not None and not reasoning_ms:
reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
accumulated.append(piece)
yield sse.event("token", escape_text(piece))
dirty = True
# Hand control back so the event is flushed rather than
# batched behind a fast generator.
await asyncio.sleep(0)
# Re-render the answer so far, at most every RENDER_INTERVAL.
# Markdown is rendered whole rather than appended, because a
# list or a code fence is only correct once its context is
# known -- and partial syntax resolves itself as more arrives.
now = time.monotonic()
if dirty and now - last_render >= RENDER_INTERVAL:
yield sse.event("render", render_markdown("".join(accumulated)))
last_render, dirty = now, False
await asyncio.sleep(0)
if message_id in _CANCELLED:
stopped = True
break
for kind, piece in splitter.flush():
if kind == REASONING:
thinking.append(piece)
yield sse.event("reasoning", escape_text(piece))
else:
accumulated.append(piece)
yield sse.event("token", escape_text(piece))
except LLMError as exc:
error = exc.message
@@ -280,9 +312,11 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
except asyncio.CancelledError:
# The reader navigated away or closed the tab. Keep whatever was
# produced so the partial reply is still there on reload.
_CANCELLED.discard(message_id)
message.content = "".join(accumulated)
message.reasoning = "".join(thinking)
message.complete = True
message.stopped = True
db.commit()
raise
except Exception as exc: # noqa: BLE001 - must not kill the stream silently
@@ -293,11 +327,14 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
# Reasoning ran to the end without an answer following it.
reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
_CANCELLED.discard(message_id)
message.content = "".join(accumulated)
message.reasoning = "".join(thinking)
message.reasoning_ms = reasoning_ms
message.error = error or ""
message.complete = True
message.stopped = stopped
log.debug(
"chat %s: %d chars answer, %d chars reasoning, %.1fs total",
chat_id,
@@ -340,6 +377,140 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
yield sse.event("close", "")
def _thread_context(db: DBSession, chat: Chat, user: User) -> dict:
"""Everything chat/_thread.html needs to render the conversation."""
messages = list(
db.scalars(select(Message).where(Message.chat_id == chat.id).order_by(Message.created_at))
)
return {
"chat": chat,
"user": user,
"messages": messages,
"bodies": {
m.id: render_markdown(m.content)
for m in messages
if m.role == ROLE_ASSISTANT and m.content
},
"models_by_id": {m.model_id: m for m in chat_service.available_models(db, user)},
}
def _messages_after(db: DBSession, message: Message) -> list[Message]:
return list(
db.scalars(
select(Message)
.where(Message.chat_id == message.chat_id, Message.created_at > message.created_at)
.order_by(Message.created_at)
)
)
@router.get("/{chat_id}/messages/{message_id}/edit")
async def edit_form(
request: Request, db: Db, user: RequiredUser, chat_id: str, message_id: str
) -> Response:
"""Swap one of the reader's own turns into an editable form."""
chat = _owned_chat(db, chat_id, user.id)
message = db.get(Message, message_id)
if message is None or message.chat_id != chat.id or message.role != ROLE_USER:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
return templates.TemplateResponse(
request,
"chat/_edit_form.html",
{
"request": request,
"chat": chat,
"user": user,
"message": message,
"following": len(_messages_after(db, message)),
},
)
@router.get("/{chat_id}/messages/{message_id}/cancel-edit")
async def cancel_edit(
request: Request, db: Db, user: RequiredUser, chat_id: str, message_id: str
) -> Response:
"""Put the bubble back, unchanged."""
chat = _owned_chat(db, chat_id, user.id)
message = db.get(Message, message_id)
if message is None or message.chat_id != chat.id:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
return templates.TemplateResponse(
request,
"chat/_message.html",
{
"request": request,
"chat": chat,
"user": user,
"message": message,
"body_html": "",
"models_by_id": {},
},
)
@router.post("/{chat_id}/messages/{message_id}/edit")
async def edit_message(
request: Request,
db: Db,
user: RequiredUser,
chat_id: str,
message_id: str,
content: str = Form(...),
) -> Response:
"""Rewrite one of the reader's turns and run the conversation on from there.
Everything after the edited message is deleted rather than branched. A
branch would need a UI for choosing between versions, and "go back and try
again from here" is what was actually asked for -- the simpler behaviour is
also the one people expect from every other chat client.
"""
chat = _owned_chat(db, chat_id, user.id)
message = db.get(Message, message_id)
if message is None or message.chat_id != chat.id or message.role != ROLE_USER:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
content = content.strip()
if not content and not message.attachments:
raise HTTPException(status.HTTP_400_BAD_REQUEST, "A message cannot be empty.")
message.content = content
# Attachments cascade with their message, so the files go too.
discarded = _messages_after(db, message)
for later in discarded:
db.delete(later)
db.commit()
chat_service.create_message(
db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
)
log.info("chat %s rewound to a message, %d discarded", chat.id, len(discarded))
return templates.TemplateResponse(
request, "chat/_thread.html", {"request": request, **_thread_context(db, chat, user)}
)
@router.post("/{chat_id}/messages/{message_id}/stop")
async def stop_message(db: Db, user: RequiredUser, chat_id: str, message_id: str) -> Response:
"""Ask a running generation to stop.
Whatever has arrived is kept: a half-written answer the reader chose to cut
short is still worth having, and discarding it would be a surprise.
"""
chat = _owned_chat(db, chat_id, user.id)
message = db.get(Message, message_id)
if message is None or message.chat_id != chat.id:
raise HTTPException(status.HTTP_404_NOT_FOUND, "That message no longer exists.")
_CANCELLED.add(message.id)
return Response(status_code=status.HTTP_204_NO_CONTENT)
@router.patch("/{chat_id}")
async def update_chat(request: Request, db: Db, user: RequiredUser, chat_id: str) -> Response:
"""Partially update a chat.