File attachments: images for vision, PDFs and text into the prompt

Drag, paste or pick a file in the composer. Images go to vision models
as multimodal content parts; PDFs and text files have their content
extracted and placed in the prompt. Verified end to end against
gemma4-e4b-q8 on llama-swap: given a drawing and a text file, it named
the red square and blue circle and read the number out of the document.

Type is decided by inspecting the bytes, never the filename or the
browser's Content-Type -- a .png full of text is stored as text. Images
are downscaled to 1400px and re-encoded: a phone photo is several
megabytes of base64, which is slow and a large slice of the context
window. PDF text is extracted once, at upload, and stored; re-extracting
per request would let a reply change because a parser was upgraded.

Design points worth keeping:

- Images are only sent to models an administrator has marked `vision`.
  This is not graceful degradation -- most endpoints reject the entire
  request rather than ignoring an image part. A plain text turn stays a
  plain string for the same reason: the list form 400s on endpoints that
  do not implement it.
- Images reach the model as base64 data URIs, not links. A local
  endpoint has no route back to LLeMbas, and a hosted one has no
  credentials for it.
- Non-images are served Content-Disposition: attachment with nosniff, so
  an uploaded .html can never execute in this origin. Stored names are
  random; the uploader's name is a label and never a path.
- Uploads are unbound until the message is sent, which is what lets a
  file be removed beforehand. claim() only takes unclaimed rows owned by
  the sender, so a forged id cannot pull in someone else's file.
  Abandoned uploads are swept at startup.
- A scanned PDF says so rather than silently contributing nothing, and
  truncation is declared to the model in the document tag so it can
  admit it did not see page 400.
- "Here, look at this" with no words is a legitimate turn, so a message
  is only empty when it carries neither text nor files.

Also fixes auto-titling, which read message["content"] as a string and
would have broken on the first multimodal turn.

186 tests, ruff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

src/lembas/services/chat.py

@@ -17,6 +17,7 @@ from lembas.db.models import (
     Message,
     Model,
 )
+from lembas.services import files as files_service
 from lembas.services.llm.openai_client import Endpoint, LLMError, complete
 
 log = logging.getLogger(__name__)
@@ -71,7 +72,66 @@ def resolve_endpoint(db: DBSession, chat: Chat) -> tuple[Endpoint, str]:
     return Endpoint.from_connection(connection), chat.model_id
 
 
-def build_messages(db: DBSession, chat: Chat, *, upto: Message | None = None) -> list[dict]:
+def document_context(message: Message) -> str:
+    """Extracted text from a message's non-image attachments.
+
+    Wrapped in named tags so the model can tell one document from another, and
+    tell all of them from what the user actually typed. Truncation is stated
+    inline rather than silently, so a model asked about page 400 of a 300-page
+    extract can say it did not see it.
+    """
+    blocks: list[str] = []
+    for attachment in message.documents:
+        if not attachment.extracted_text.strip():
+            continue
+        note = " (truncated)" if attachment.truncated else ""
+        blocks.append(
+            f'<document name="{attachment.filename}"{note}>\n'
+            f"{attachment.extracted_text.strip()}\n"
+            f"</document>"
+        )
+    return "\n\n".join(blocks)
+
+
+def message_payload(message: Message, *, vision: bool) -> dict[str, Any]:
+    """One history entry in the shape the endpoint expects.
+
+    Plain text stays a plain string: sending the multimodal list form to an
+    endpoint that does not implement it is a reliable way to get a 400, and
+    most local runners do not.
+    """
+    text = message.content.strip()
+
+    documents = document_context(message)
+    if documents:
+        # Documents lead so the question that follows has its material already
+        # in view, which is how these models are trained to read a prompt.
+        text = f"{documents}\n\n{text}" if text else documents
+
+    images = message.images if vision else []
+    if not images:
+        return {"role": message.role, "content": text}
+
+    parts: list[dict[str, Any]] = []
+    if text:
+        parts.append({"type": "text", "text": text})
+    for attachment in images:
+        uri = files_service.data_uri(attachment)
+        if uri is None:
+            # The row survived but the file did not. Better to say so than to
+            # send a turn that silently lost its picture.
+            log.warning("attachment %s has no file on disk", attachment.id)
+            continue
+        parts.append({"type": "image_url", "image_url": {"url": uri}})
+
+    if not parts:
+        return {"role": message.role, "content": text}
+    return {"role": message.role, "content": parts}
+
+
+def build_messages(
+    db: DBSession, chat: Chat, *, upto: Message | None = None, vision: bool = False
+) -> list[dict]:
     """Assemble the message list to send upstream.
 
     `upto` excludes the placeholder assistant row being generated into, and
@@ -88,24 +148,38 @@ def build_messages(db: DBSession, chat: Chat, *, upto: Message | None = None) ->
     for message in history:
         if upto is not None and message.id == upto.id:
             break
-        # Skip turns that failed or produced nothing: sending an empty
-        # assistant message upsets several providers.
-        if message.error or not message.content.strip():
+        # Skip turns that failed or produced nothing -- but a message carrying
+        # only an attachment has no text and must still be sent.
+        if message.error:
             continue
-        payload.append({"role": message.role, "content": message.content})
+        if not message.content.strip() and not message.attachments:
+            continue
+        payload.append(message_payload(message, vision=vision))
 
     return payload
 
 
+def model_supports(db: DBSession, chat: Chat, capability: str) -> bool:
+    """Whether the chat's current model is marked as having a capability."""
+    model = db.scalar(
+        select(Model).where(Model.model_id == chat.model_id).order_by(Model.position)
+    )
+    return bool(model and (model.capabilities_json or {}).get(capability))
+
+
 def build_request(db: DBSession, chat: Chat, *, upto: Message | None = None) -> dict[str, Any]:
     params = {
         key: value
         for key, value in (chat.params_json or {}).items()
         if key in FORWARDED_PARAMS and value not in (None, "")
     }
+    # Images are only sent to a model an administrator has marked as having
+    # vision. Sending them to one that has not is not a graceful degradation:
+    # most endpoints reject the whole request.
+    vision = model_supports(db, chat, "vision")
     return {
         "model": chat.model_id,
-        "messages": build_messages(db, chat, upto=upto),
+        "messages": build_messages(db, chat, upto=upto, vision=vision),
         **params,
     }