Jaroslav Beneš
d90195015c
Drag, paste or pick a file in the composer. Images go to vision models as multimodal content parts; PDFs and text files have their content extracted and placed in the prompt. Verified end to end against gemma4-e4b-q8 on llama-swap: given a drawing and a text file, it named the red square and blue circle and read the number out of the document. Type is decided by inspecting the bytes, never the filename or the browser's Content-Type -- a .png full of text is stored as text. Images are downscaled to 1400px and re-encoded: a phone photo is several megabytes of base64, which is slow and a large slice of the context window. PDF text is extracted once, at upload, and stored; re-extracting per request would let a reply change because a parser was upgraded. Design points worth keeping: - Images are only sent to models an administrator has marked `vision`. This is not graceful degradation -- most endpoints reject the entire request rather than ignoring an image part. A plain text turn stays a plain string for the same reason: the list form 400s on endpoints that do not implement it. - Images reach the model as base64 data URIs, not links. A local endpoint has no route back to LLeMbas, and a hosted one has no credentials for it. - Non-images are served Content-Disposition: attachment with nosniff, so an uploaded .html can never execute in this origin. Stored names are random; the uploader's name is a label and never a path. - Uploads are unbound until the message is sent, which is what lets a file be removed beforehand. claim() only takes unclaimed rows owned by the sender, so a forged id cannot pull in someone else's file. Abandoned uploads are swept at startup. - A scanned PDF says so rather than silently contributing nothing, and truncation is declared to the model in the document tag so it can admit it did not see page 400. - "Here, look at this" with no words is a legitimate turn, so a message is only empty when it carries neither text nor files. Also fixes auto-titling, which read message["content"] as a string and would have broken on the first multimodal turn. 186 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
75 lines
3.2 KiB
Python
75 lines
3.2 KiB
Python
"""Files attached to chat messages."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from sqlalchemy import Boolean, ForeignKey, Integer, String, Text
|
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
|
|
|
from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
|
|
|
|
# What the file is for, decided at upload time. Drives both how it is rendered
|
|
# and how it reaches the model: images become multimodal parts, everything else
|
|
# becomes text in the prompt.
|
|
KIND_IMAGE = "image"
|
|
KIND_DOCUMENT = "document" # PDF: text is extracted
|
|
KIND_TEXT = "text" # plain text, markdown, csv, source code
|
|
|
|
|
|
class Attachment(UUIDPrimaryKey, Timestamps, Base):
|
|
__tablename__ = "attachments"
|
|
|
|
user_id: Mapped[str] = mapped_column(
|
|
String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
|
)
|
|
chat_id: Mapped[str | None] = mapped_column(
|
|
String(32), ForeignKey("chats.id", ondelete="CASCADE"), index=True
|
|
)
|
|
# Null while the file is uploaded but the message has not been sent yet.
|
|
# Those orphans are swept periodically -- see services.files.sweep_orphans.
|
|
message_id: Mapped[str | None] = mapped_column(
|
|
String(32), ForeignKey("messages.id", ondelete="CASCADE"), index=True
|
|
)
|
|
|
|
# What the uploader called it. Display only, never used as a path.
|
|
filename: Mapped[str] = mapped_column(String(300), nullable=False)
|
|
# Random name on disk. See services.files for why the two are separate.
|
|
stored_name: Mapped[str] = mapped_column(String(120), nullable=False)
|
|
|
|
media_type: Mapped[str] = mapped_column(String(100), default="")
|
|
size_bytes: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
kind: Mapped[str] = mapped_column(String(16), default=KIND_DOCUMENT, nullable=False)
|
|
|
|
# Images only, after downscaling.
|
|
width: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
height: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
|
|
# Documents and text: the content that actually reaches the model. Held in
|
|
# the database rather than re-extracted per request -- extraction is slow,
|
|
# and a reply must not silently change because a PDF parser was upgraded.
|
|
extracted_text: Mapped[str] = mapped_column(Text, default="")
|
|
pages: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
truncated: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
|
|
|
# Non-empty when the file was stored but its text could not be read, e.g. a
|
|
# scanned PDF with no text layer. Shown next to the attachment so the user
|
|
# is not left wondering why the model ignored it.
|
|
extraction_error: Mapped[str] = mapped_column(Text, default="")
|
|
|
|
message: Mapped[Message] = relationship(back_populates="attachments") # noqa: F821
|
|
|
|
@property
|
|
def is_image(self) -> bool:
|
|
return self.kind == KIND_IMAGE
|
|
|
|
@property
|
|
def human_size(self) -> str:
|
|
size = float(self.size_bytes)
|
|
for unit in ("B", "KB", "MB"):
|
|
if size < 1024 or unit == "MB":
|
|
return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}"
|
|
size /= 1024
|
|
return f"{size:.1f} MB"
|
|
|
|
def __repr__(self) -> str:
|
|
return f"<Attachment {self.filename} {self.kind}>"
|