Users, groups, permissions, model settings and reasoning display

Four features, plus the schema machinery they needed.

**Schema sync.** The first live instance had data in it, and create_all
only creates missing *tables* -- a new column silently never appeared.
db/migrations.py now diffs the declared models against the database and
ALTER TABLE ... ADD COLUMN for what is missing, deriving a backfill
default from the column type (SQLite refuses a NOT NULL column without
one, and a Python-side `default=dict` cannot be expressed in DDL).
Verified against a copy of the live database: eight changes applied, all
rows preserved, second run a no-op. Renames, drops and retypes are still
manual and say so.

**Permissions.** A flat set of named booleans: an instance baseline
widened by each group the user belongs to. A group grants and never
denies -- with denies, "why can this user not do X" cannot be answered
without simulating every group. Admins bypass entirely, because an admin
can grant it back to themselves in two clicks and pretending otherwise
is theatre. Model *access* is separate: public, or granted to groups.
The picker is not the boundary -- switching a chat to a model you cannot
reach is a 403.

**Model settings.** Ordering, pinned-first, an instance default and a
per-user default, display names, descriptions, capability flags, and
uploaded images. Images are stored and served locally rather than by
URL: a remote URL makes every page render a request to a third party.
Uploads are validated by magic number, not the declared content type,
and stored under a random name. Models with no image get a generated
initial whose hue is derived from the model id, so it is stable.

**Reasoning display.** Streams into its own collapsible block above the
answer, labelled "Thought for 14 seconds", collapsed once finished, and
never replayed as context on the next turn. Two sources: the
reasoning_content delta field, and <think> tags inline in content -- the
latter needs a streaming splitter because the tags arrive split across
chunks. Models emitting no reasoning show nothing, via a :has() rule
rather than JavaScript. Verified against qwen35-9b on llama-swap: 694
reasoning events, 52 answer tokens, cleanly separated.

Two bugs found and fixed while testing:

- A bare `Mapped[list]` relationship is treated by SQLAlchemy as a scalar
  and returns None instead of []. It needs the element type.
- FastAPI substitutes the default for an empty form value, so with
  `x: str | None = Form(None)` a submitted `x=` is indistinguishable from
  an absent field. That silently broke clearing a system prompt or a
  temperature. update_chat now reads the raw form and checks key presence.

143 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

src/lembas/db/migrations.py

@@ -0,0 +1,133 @@
+"""Additive schema synchronisation.
+
+This project has no Alembic, by design: it is SQLite-only and the schema is
+created at startup. That was fine until the first live instance had data in it,
+at which point adding a column to a model stopped being free -- ``create_all``
+only creates missing *tables*, so a new column silently never appears and every
+query mentioning it fails.
+
+What this module does instead is derive the migration from the models: compare
+each table's declared columns against what the database actually has, and
+``ALTER TABLE ... ADD COLUMN`` for whatever is missing. That covers new tables
+and new columns, which is essentially every schema change this project makes.
+
+What it deliberately does NOT do:
+
+* rename, drop or retype a column
+* add a PRIMARY KEY or UNIQUE constraint to an existing table
+* backfill anything requiring application logic
+
+SQLite cannot do most of those with ALTER TABLE anyway; they need the
+create-copy-swap dance. Anything in that category is a hand-written job and
+should be added to MANUAL_STEPS below so it is at least visible.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+from sqlalchemy import Engine, inspect, text
+from sqlalchemy.schema import Column, Table
+
+from lembas.db.base import Base
+
+log = logging.getLogger(__name__)
+
+# Schema changes that this module cannot perform. Kept as documentation so a
+# failure has somewhere to point rather than being a mystery.
+MANUAL_STEPS: list[str] = []
+
+
+def _literal_default(column: Column) -> str | None:
+    """A SQL literal to backfill an existing row's new column with.
+
+    SQLite refuses to add a NOT NULL column without a default, and refuses a
+    non-constant default. Python-side defaults (``default=dict``,
+    ``default=utcnow``) are callables and cannot be expressed in DDL, so the
+    value is derived from the column type instead. New rows still get the real
+    Python default; this only fills the rows that already exist.
+    """
+    default = column.default
+    if default is not None and not default.is_callable and not default.is_clause_element:
+        value: Any = default.arg
+        if isinstance(value, bool):
+            return "1" if value else "0"
+        if isinstance(value, (int, float)):
+            return str(value)
+        if isinstance(value, str):
+            escaped = value.replace("'", "''")
+            return f"'{escaped}'"
+
+    affinity = column.type.__class__.__name__.upper()
+    if "JSON" in affinity:
+        # MutableList columns must start as [] and MutableDict as {}; guessing
+        # wrong makes the first read blow up rather than return empty.
+        python_type = getattr(column.type, "python_type", None)
+        return "'[]'" if python_type is list else "'{}'"
+    if "BOOL" in affinity:
+        return "0"
+    if any(token in affinity for token in ("INT", "FLOAT", "NUMERIC", "DECIMAL")):
+        return "0"
+    if "DATE" in affinity or "TIME" in affinity:
+        return "CURRENT_TIMESTAMP"
+    if any(token in affinity for token in ("STRING", "TEXT", "VARCHAR", "CHAR")):
+        return "''"
+    return None
+
+
+def _add_column_sql(table: Table, column: Column, dialect) -> str | None:
+    type_sql = column.type.compile(dialect)
+    default = _literal_default(column)
+
+    if not column.nullable and default is None:
+        log.error(
+            "cannot add NOT NULL column %s.%s: no usable default. Add it by hand.",
+            table.name,
+            column.name,
+        )
+        return None
+
+    parts = [f'ALTER TABLE "{table.name}" ADD COLUMN "{column.name}" {type_sql}']
+    if not column.nullable:
+        parts.append("NOT NULL")
+    if default is not None:
+        parts.append(f"DEFAULT {default}")
+    return " ".join(parts)
+
+
+def sync_schema(engine: Engine) -> list[str]:
+    """Bring the database up to the declared schema. Returns what it changed."""
+    import lembas.db.models  # noqa: F401  (registers every table on the metadata)
+
+    changes: list[str] = []
+
+    inspector = inspect(engine)
+    known_tables = set(inspector.get_table_names())
+    for table in Base.metadata.sorted_tables:
+        if table.name not in known_tables:
+            changes.append(f"create table {table.name}")
+
+    # Creates anything missing; existing tables are left alone.
+    Base.metadata.create_all(bind=engine)
+
+    inspector = inspect(engine)
+    with engine.begin() as connection:
+        for table in Base.metadata.sorted_tables:
+            existing = {col["name"] for col in inspector.get_columns(table.name)}
+            for column in table.columns:
+                if column.name in existing:
+                    continue
+                statement = _add_column_sql(table, column, engine.dialect)
+                if statement is None:
+                    continue
+                connection.execute(text(statement))
+                changes.append(f"add column {table.name}.{column.name}")
+                log.info("schema: %s", statement)
+
+    if changes:
+        log.info("schema synchronised: %d change(s)", len(changes))
+    for step in MANUAL_STEPS:
+        log.warning("manual schema step still required: %s", step)
+
+    return changes

src/lembas/db/models/__init__.py

@@ -14,7 +14,7 @@ from lembas.db.models.chat import (
     Folder,
     Message,
 )
-from lembas.db.models.connection import Connection, Model
+from lembas.db.models.connection import Connection, Model, model_groups
 from lembas.db.models.setting import Setting
 from lembas.db.models.user import (
     ROLE_ADMIN,
@@ -38,6 +38,7 @@ __all__ = [
     "Group",
     "Message",
     "Model",
+    "model_groups",
     "Session",
     "Setting",
     "User",

src/lembas/db/models/chat.py

@@ -101,6 +101,14 @@ class Message(UUIDPrimaryKey, Timestamps, Base):
     # Plain-text messages leave this empty and use `content`.
     content_parts_json: Mapped[list[Any]] = mapped_column(JSONList, default=list)
 
+    # A reasoning model's visible thinking, kept separate from the answer so it
+    # can be collapsed, and so it is never fed back as context on the next turn
+    # -- providers expect the answer alone, and replaying the thinking both
+    # wastes the window and degrades the reply.
+    reasoning: Mapped[str] = mapped_column(Text, default="")
+    # Milliseconds spent producing the reasoning, for the "Thought for Xs" label.
+    reasoning_ms: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+
     model_id: Mapped[str] = mapped_column(String(300), default="")
     tool_calls_json: Mapped[list[Any]] = mapped_column(JSONList, default=list)
     usage_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)

src/lembas/db/models/connection.py

@@ -3,14 +3,38 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
-from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint
+from sqlalchemy import (
+    Boolean,
+    Column,
+    DateTime,
+    ForeignKey,
+    Integer,
+    String,
+    Table,
+    Text,
+    UniqueConstraint,
+)
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
 from lembas.db.types import JSONDict
 
+if TYPE_CHECKING:
+    # Import only for the annotation; at runtime SQLAlchemy resolves the
+    # name through its own class registry, so there is no import cycle.
+    from lembas.db.models.user import Group
+
+# Which groups may use a given model. A model with no rows here is reachable
+# only by administrators unless it is marked public.
+model_groups = Table(
+    "model_groups",
+    Base.metadata,
+    Column("model_id", String(32), ForeignKey("models.id", ondelete="CASCADE"), primary_key=True),
+    Column("group_id", String(32), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
+)
+
 
 class Connection(UUIDPrimaryKey, Timestamps, Base):
     """A configured upstream endpoint speaking the OpenAI HTTP API.
@@ -64,19 +88,45 @@ class Model(UUIDPrimaryKey, Timestamps, Base):
     )
     model_id: Mapped[str] = mapped_column(String(300), nullable=False)
     display_name: Mapped[str] = mapped_column(String(300), default="")
+    description: Mapped[str] = mapped_column(Text, default="")
     enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
 
+    # Sort order in every picker. Ties fall back to model_id so the order is
+    # stable rather than whatever SQLite feels like today.
+    position: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
+    # Pinned models are offered first, before the full list.
+    pinned: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
+
+    # Public models are usable by anyone; otherwise access comes from `groups`.
+    public: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False)
+
+    # Filename under <data>/uploads/models. Stored rather than a URL so the
+    # image cannot become a request to a third party on every page render.
+    image_path: Mapped[str] = mapped_column(String(300), default="")
+
     # Endpoints do not reliably advertise capabilities, so these are admin
-    # overrides consumed by later passes (vision uploads, tool calling).
+    # overrides. Recognised keys: vision, tools, reasoning.
     capabilities_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
     # Default sampling params applied to new chats using this model.
     params_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
     connection: Mapped[Connection] = relationship(back_populates="models")
+    groups: Mapped[list[Group]] = relationship(
+        "Group", secondary=model_groups, back_populates="models"
+    )
 
     @property
     def label(self) -> str:
         return self.display_name or self.model_id
 
+    @property
+    def supports_reasoning(self) -> bool:
+        return bool((self.capabilities_json or {}).get("reasoning"))
+
+    @property
+    def initial(self) -> str:
+        """First character of the label, for the fallback avatar."""
+        return (self.label.strip() or "?")[0].upper()
+
     def __repr__(self) -> str:
         return f"<Model {self.model_id}>"

src/lembas/db/models/user.py

@@ -3,7 +3,7 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Index, String, Table, Text
 from sqlalchemy.orm import Mapped, mapped_column, relationship
@@ -11,6 +11,10 @@ from sqlalchemy.orm import Mapped, mapped_column, relationship
 from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
 from lembas.db.types import JSONDict
 
+if TYPE_CHECKING:
+    # Annotation only; SQLAlchemy resolves the real class from its registry.
+    from lembas.db.models.connection import Model
+
 # Roles are a simple ordered ladder rather than a permission matrix. Groups
 # (below) carry finer-grained permissions once the users/groups UI lands.
 ROLE_ADMIN = "admin"
@@ -58,9 +62,16 @@ class Group(UUIDPrimaryKey, Timestamps, Base):
 
     name: Mapped[str] = mapped_column(String(120), unique=True, nullable=False)
     description: Mapped[str] = mapped_column(Text, default="")
+
+    # Only the granted keys need be present. Absent means "no opinion", not
+    # "deny" -- permissions union across a user's groups. See
+    # lembas.security.permissions.
     permissions_json: Mapped[dict[str, Any]] = mapped_column(JSONDict, default=dict)
 
     users: Mapped[list[User]] = relationship(secondary=user_groups, back_populates="groups")
+    models: Mapped[list[Model]] = relationship(
+        "Model", secondary="model_groups", back_populates="groups"
+    )
 
 
 class Session(UUIDPrimaryKey, Timestamps, Base):

src/lembas/db/session.py

@@ -10,7 +10,6 @@ from sqlalchemy import Engine, create_engine, event
 from sqlalchemy.orm import Session, sessionmaker
 
 from lembas.config import settings
-from lembas.db.base import Base
 
 log = logging.getLogger(__name__)
 
@@ -63,15 +62,17 @@ def get_session_factory() -> sessionmaker[Session]:
 
 
 def init_db() -> None:
-    """Create any missing tables.
+    """Bring the database up to the declared schema.
 
-    This is ``CREATE TABLE IF NOT EXISTS`` only -- it never alters an existing
-    table. There is no migration tool in this project by design, so changing a
-    column on a model requires migrating the database by hand.
+    Creates missing tables and adds missing columns -- see db/migrations.py for
+    what that does and does not cover. Additive changes need nothing else;
+    renames, drops and retypes are still a hand job.
     """
-    import lembas.db.models  # noqa: F401  (registers tables on the metadata)
+    from lembas.db.migrations import sync_schema
 
-    Base.metadata.create_all(bind=get_engine())
+    changes = sync_schema(get_engine())
+    if changes:
+        log.info("database schema updated: %s", ", ".join(changes))
     log.debug("schema ensured at %s", settings.db_path)