Houmeres/LLeMbas
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Users, groups, permissions, model settings and reasoning display

Browse Source 
Four features, plus the schema machinery they needed.

**Schema sync.** The first live instance had data in it, and create_all
only creates missing *tables* -- a new column silently never appeared.
db/migrations.py now diffs the declared models against the database and
ALTER TABLE ... ADD COLUMN for what is missing, deriving a backfill
default from the column type (SQLite refuses a NOT NULL column without
one, and a Python-side `default=dict` cannot be expressed in DDL).
Verified against a copy of the live database: eight changes applied, all
rows preserved, second run a no-op. Renames, drops and retypes are still
manual and say so.

**Permissions.** A flat set of named booleans: an instance baseline
widened by each group the user belongs to. A group grants and never
denies -- with denies, "why can this user not do X" cannot be answered
without simulating every group. Admins bypass entirely, because an admin
can grant it back to themselves in two clicks and pretending otherwise
is theatre. Model *access* is separate: public, or granted to groups.
The picker is not the boundary -- switching a chat to a model you cannot
reach is a 403.

**Model settings.** Ordering, pinned-first, an instance default and a
per-user default, display names, descriptions, capability flags, and
uploaded images. Images are stored and served locally rather than by
URL: a remote URL makes every page render a request to a third party.
Uploads are validated by magic number, not the declared content type,
and stored under a random name. Models with no image get a generated
initial whose hue is derived from the model id, so it is stable.

**Reasoning display.** Streams into its own collapsible block above the
answer, labelled "Thought for 14 seconds", collapsed once finished, and
never replayed as context on the next turn. Two sources: the
reasoning_content delta field, and <think> tags inline in content -- the
latter needs a streaming splitter because the tags arrive split across
chunks. Models emitting no reasoning show nothing, via a :has() rule
rather than JavaScript. Verified against qwen35-9b on llama-swap: 694
reasoning events, 52 answer tokens, cleanly separated.

Two bugs found and fixed while testing:

- A bare `Mapped[list]` relationship is treated by SQLAlchemy as a scalar
  and returns None instead of []. It needs the element type.
- FastAPI substitutes the default for an empty form value, so with
  `x: str | None = Form(None)` a submitted `x=` is indistinguishable from
  an absent field. That silently broke clearing a system prompt or a
  temperature. update_chat now reads the raw form and checks key presence.

143 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-07-21 11:49:32 +02:00
parent ba2fb1e13d
commit d6c87ac811
37 changed files with 2887 additions and 152 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lembas/api/chats.py
+151 -28
View File
@@ -4,19 +4,27 @@ from __future__ import annotations
import asyncio
import logging
import time
from collections.abc import AsyncIterator
from fastapi import APIRouter, Form, HTTPException, Request, status
from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
from fastapi.responses import HTMLResponse, Response, StreamingResponse
from sqlalchemy.orm import Session as DBSession
from lembas.api.deps import Db, RequiredUser
from lembas.api.deps import Db, RequiredUser, require_permission
from lembas.db.models import ROLE_ASSISTANT, ROLE_USER, Chat, Message, User
from lembas.db.session import session_scope
from lembas.security import permissions
from lembas.services import chat as chat_service
from lembas.services import sse
from lembas.services.llm.openai_client import LLMError, delta_text, stream_chat
from lembas.services.llm.openai_client import (
LLMError,
delta_reasoning,
delta_text,
stream_chat,
)
from lembas.services.markdown import escape_text, render_markdown
from lembas.services.reasoning import REASONING, ReasoningSplitter
from lembas.web.templating import render, templates
log = logging.getLogger(__name__)
@@ -33,9 +41,9 @@ def _owned_chat(db: DBSession, chat_id: str, user_id: str) -> Chat:
return chat
@router.post("")
@router.post("", dependencies=[Depends(require_permission("chat.create"))])
async def create_chat(db: Db, user: RequiredUser, folder_id: str = Form("")) -> Response:
chosen = chat_service.default_model(db)
chosen = chat_service.default_model(db, user)
chat = Chat(
user_id=user.id,
folder_id=folder_id or None,
@@ -130,7 +138,9 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
be closed by the time the first token arrives.
"""
accumulated: list[str] = []
thinking: list[str] = []
error: str | None = None
reasoning_ms = 0
with session_scope() as db:
chat = db.get(Chat, chat_id)
@@ -140,6 +150,12 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
return
first_user_text = ""
# Handles models that emit <think> tags inline in content rather than
# using the reasoning_content field.
splitter = ReasoningSplitter()
started = time.monotonic()
reasoning_started: float | None = None
try:
endpoint, model_id = chat_service.resolve_endpoint(db, chat)
payload = chat_service.build_request(db, chat, upto=message)
@@ -149,14 +165,42 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
)
async for chunk in stream_chat(endpoint, payload):
# A dedicated reasoning field is unambiguous; take it as-is.
thought = delta_reasoning(chunk)
if thought:
if reasoning_started is None:
reasoning_started = time.monotonic()
thinking.append(thought)
yield sse.event("reasoning", escape_text(thought))
await asyncio.sleep(0)
text = delta_text(chunk)
if not text:
continue
accumulated.append(text)
yield sse.event("token", escape_text(text))
# Hand control back so the event is flushed rather than
# batched behind a fast generator.
await asyncio.sleep(0)
for kind, piece in splitter.feed(text):
if kind == REASONING:
if reasoning_started is None:
reasoning_started = time.monotonic()
thinking.append(piece)
yield sse.event("reasoning", escape_text(piece))
else:
# First answer token ends the thinking phase.
if reasoning_started is not None and not reasoning_ms:
reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
accumulated.append(piece)
yield sse.event("token", escape_text(piece))
# Hand control back so the event is flushed rather than
# batched behind a fast generator.
await asyncio.sleep(0)
for kind, piece in splitter.flush():
if kind == REASONING:
thinking.append(piece)
yield sse.event("reasoning", escape_text(piece))
else:
accumulated.append(piece)
yield sse.event("token", escape_text(piece))
except LLMError as exc:
error = exc.message
@@ -165,6 +209,7 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
# The reader navigated away or closed the tab. Keep whatever was
# produced so the partial reply is still there on reload.
message.content = "".join(accumulated)
message.reasoning = "".join(thinking)
message.complete = True
db.commit()
raise
@@ -172,9 +217,22 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
error = "Something went wrong while generating this reply."
log.exception("unexpected generation failure for chat %s: %s", chat_id, exc)
if reasoning_started is not None and not reasoning_ms:
# Reasoning ran to the end without an answer following it.
reasoning_ms = int((time.monotonic() - reasoning_started) * 1000)
message.content = "".join(accumulated)
message.reasoning = "".join(thinking)
message.reasoning_ms = reasoning_ms
message.error = error or ""
message.complete = True
log.debug(
"chat %s: %d chars answer, %d chars reasoning, %.1fs total",
chat_id,
len(message.content),
len(message.reasoning),
time.monotonic() - started,
)
if not chat.title_generated and (accumulated or error):
chat.title = (
@@ -208,38 +266,103 @@ async def _generate(chat_id: str, message_id: str) -> AsyncIterator[str]:
@router.patch("/{chat_id}")
async def update_chat(
db: Db,
user: RequiredUser,
chat_id: str,
title: str | None = Form(None),
folder_id: str | None = Form(None),
model_id: str | None = Form(None),
) -> Response:
chat = _owned_chat(db, chat_id, user.id)
async def update_chat(request: Request, db: Db, user: RequiredUser, chat_id: str) -> Response:
"""Partially update a chat.
if title is not None:
cleaned = title.strip()[:300]
The raw form is read rather than declaring Form() parameters because
FastAPI substitutes the default for an empty form value, which makes
"field absent" and "field submitted empty" indistinguishable. That
difference is exactly what this endpoint needs: an empty system prompt or
temperature means *clear it*, not *leave it alone*.
"""
chat = _owned_chat(db, chat_id, user.id)
allowed = permissions.resolve(db, user)
form = await request.form()
if "title" in form:
cleaned = str(form["title"]).strip()[:300]
if cleaned:
chat.title = cleaned
# An explicit rename must not be overwritten by auto-titling later.
chat.title_generated = True
if folder_id is not None:
chat.folder_id = folder_id or None
if "folder_id" in form:
chat.folder_id = str(form["folder_id"]) or None
if model_id is not None and model_id:
chat.model_id = model_id
model_id = str(form.get("model_id", "")).strip()
if model_id:
if not allowed.get("chat.model_select"):
raise HTTPException(
status.HTTP_403_FORBIDDEN, "You may not change the model for a chat."
)
# Checked against what this user can reach, not merely what exists --
# otherwise the picker is advisory and a crafted request bypasses it.
match = next(
(m for m in chat_service.available_models(db) if m.model_id == model_id), None
(m for m in chat_service.available_models(db, user) if m.model_id == model_id),
None,
)
chat.connection_id = match.connection_id if match else None
if match is None:
raise HTTPException(status.HTTP_403_FORBIDDEN, "That model is not available to you.")
chat.model_id = model_id
chat.connection_id = match.connection_id
if "system_prompt" in form:
if not allowed.get("chat.system_prompt"):
raise HTTPException(
status.HTTP_403_FORBIDDEN, "You may not set a system prompt."
)
chat.system_prompt = str(form["system_prompt"]).strip()[:8000]
submitted_params = {name: form[name] for name in _PARAM_RANGES if name in form}
if submitted_params:
if not allowed.get("chat.params"):
raise HTTPException(
status.HTTP_403_FORBIDDEN, "You may not change sampling parameters."
)
chat.params_json = {
**(chat.params_json or {}),
**_clean_params(**{k: str(v) for k, v in submitted_params.items()}),
}
db.commit()
return Response(status_code=status.HTTP_204_NO_CONTENT)
@router.delete("/{chat_id}")
# Bounds are the ones every provider agrees on. Out-of-range values are
# dropped rather than clamped: silently changing what someone typed is worse
# than ignoring it, and the form shows what actually stuck on reload.
_PARAM_RANGES: dict[str, tuple[type, float, float]] = {
"temperature": (float, 0.0, 2.0),
"top_p": (float, 0.0, 1.0),
"max_tokens": (int, 1, 1_000_000),
}
def _clean_params(**submitted: str | None) -> dict[str, float | int | None]:
"""Parse sampling parameters, dropping anything unusable.
An empty string means "unset this and let the provider default apply", so
it maps to None rather than being ignored.
"""
cleaned: dict[str, float | int | None] = {}
for name, raw in submitted.items():
if raw is None:
continue
if not raw.strip():
cleaned[name] = None
continue
caster, low, high = _PARAM_RANGES[name]
try:
value = caster(raw)
except (TypeError, ValueError):
continue
if low <= value <= high:
cleaned[name] = value
return cleaned
@router.delete("/{chat_id}", dependencies=[Depends(require_permission("chat.delete"))])
async def delete_chat(db: Db, user: RequiredUser, chat_id: str) -> Response:
chat = _owned_chat(db, chat_id, user.id)
db.delete(chat)