Jaroslav Beneš
fc02eb5538
An agent chat used to open with the model knowing the name of a machine and nothing about what was on it, so the first two rounds of every reply went on finding out. It now gets a listing: one read-only command, `git ls-files` where that works and `find` otherwise, falling back to an SFTP walk that always does. git first because a repository already carries somebody's considered list of what is not part of the project, and reproducing it by hand is how an index ends up mostly build output. The listing is budgeted rather than dumped. A tree of a thousand files is worse than no tree -- it costs the window on every request forever and buries the four names that mattered -- so directories that will not fit are shown as a count and the model is told to open one itself. Collapsing picks the deepest and largest first: by saving alone it would take `src/` before `src/web/static/vendor/`, because it contains it, and lose every name worth having. Read from a cache and never fetched. `harness.context_variables` is synchronous and sits on the request path; the walk happens in the generation setup, which is async and already doing network work, with a short wait. A chat whose first reply outruns its first walk simply has no listing that turn and the fragment disappears rather than appearing as an empty heading. Then `@`, over the same index and over the library, and `/` for commands with an Alt-based keyboard for the same jobs. A mentioned file arrives as contents, not a reference -- a small model asked to call file_read often does not bother -- and it arrives with its absolute path and the machine it came from, because a model handed `main.py` cannot tell which of four it is and cannot name it back when asked to change something. The rule that matters for `/`: a message that merely starts with a slash still sends. `//` escapes and an unrecognised command is posted as written. Swallowing somebody's message is a much worse failure than an unknown command. Two exceptions to Manual mode now, not one. Browsing and indexing are a person acting, not a model, so neither passes through policy.py -- the same argument the terminal panel rests on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
88 lines
4.0 KiB
Python
88 lines
4.0 KiB
Python
"""Files attached to chat messages."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from sqlalchemy import Boolean, ForeignKey, Integer, String, Text
|
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
|
|
|
from lembas.db.base import Base, Timestamps, UUIDPrimaryKey
|
|
|
|
# What the file is for, decided at upload time. Drives both how it is rendered
|
|
# and how it reaches the model: images become multimodal parts, everything else
|
|
# becomes text in the prompt.
|
|
KIND_IMAGE = "image"
|
|
KIND_DOCUMENT = "document" # PDF: text is extracted
|
|
KIND_TEXT = "text" # plain text, markdown, csv, source code
|
|
|
|
|
|
class Attachment(UUIDPrimaryKey, Timestamps, Base):
|
|
__tablename__ = "attachments"
|
|
|
|
user_id: Mapped[str] = mapped_column(
|
|
String(32), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
|
)
|
|
chat_id: Mapped[str | None] = mapped_column(
|
|
String(32), ForeignKey("chats.id", ondelete="CASCADE"), index=True
|
|
)
|
|
# Null while the file is uploaded but the message has not been sent yet.
|
|
# Those orphans are swept periodically -- see services.files.sweep_orphans.
|
|
message_id: Mapped[str | None] = mapped_column(
|
|
String(32), ForeignKey("messages.id", ondelete="CASCADE"), index=True
|
|
)
|
|
|
|
# What the uploader called it. Display only, never used as a path.
|
|
filename: Mapped[str] = mapped_column(String(300), nullable=False)
|
|
# Random name on disk. See services.files for why the two are separate.
|
|
stored_name: Mapped[str] = mapped_column(String(120), nullable=False)
|
|
|
|
media_type: Mapped[str] = mapped_column(String(100), default="")
|
|
size_bytes: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
kind: Mapped[str] = mapped_column(String(16), default=KIND_DOCUMENT, nullable=False)
|
|
|
|
# Images only, after downscaling.
|
|
width: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
height: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
|
|
# Documents and text: the content that actually reaches the model. Held in
|
|
# the database rather than re-extracted per request -- extraction is slow,
|
|
# and a reply must not silently change because a PDF parser was upgraded.
|
|
extracted_text: Mapped[str] = mapped_column(Text, default="")
|
|
pages: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
|
truncated: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
|
|
|
# Non-empty when the file was stored but its text could not be read, e.g. a
|
|
# scanned PDF with no text layer. Shown next to the attachment so the user
|
|
# is not left wondering why the model ignored it.
|
|
extraction_error: Mapped[str] = mapped_column(Text, default="")
|
|
|
|
# Where this came from, when it came from somewhere with an address.
|
|
#
|
|
# `filename` is a display name and is frequently just the basename, which
|
|
# is not enough: a model told it has been given `main.py` cannot tell which
|
|
# of four it is looking at, and cannot name the file back to you if you ask
|
|
# it to change something. So a project file carries its absolute path and
|
|
# the machine it was read from, and both go into the tag the model sees.
|
|
#
|
|
# Nullable, and empty for an ordinary upload -- a file dragged in from a
|
|
# laptop has no address this instance could meaningfully report.
|
|
source_path: Mapped[str] = mapped_column(String(1000), default="")
|
|
source_label: Mapped[str] = mapped_column(String(200), default="")
|
|
|
|
message: Mapped[Message] = relationship(back_populates="attachments") # noqa: F821
|
|
|
|
@property
|
|
def is_image(self) -> bool:
|
|
return self.kind == KIND_IMAGE
|
|
|
|
@property
|
|
def human_size(self) -> str:
|
|
size = float(self.size_bytes)
|
|
for unit in ("B", "KB", "MB"):
|
|
if size < 1024 or unit == "MB":
|
|
return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}"
|
|
size /= 1024
|
|
return f"{size:.1f} MB"
|
|
|
|
def __repr__(self) -> str:
|
|
return f"<Attachment {self.filename} {self.kind}>"
|