File attachments: images for vision, PDFs and text into the prompt
Drag, paste or pick a file in the composer. Images go to vision models as multimodal content parts; PDFs and text files have their content extracted and placed in the prompt. Verified end to end against gemma4-e4b-q8 on llama-swap: given a drawing and a text file, it named the red square and blue circle and read the number out of the document. Type is decided by inspecting the bytes, never the filename or the browser's Content-Type -- a .png full of text is stored as text. Images are downscaled to 1400px and re-encoded: a phone photo is several megabytes of base64, which is slow and a large slice of the context window. PDF text is extracted once, at upload, and stored; re-extracting per request would let a reply change because a parser was upgraded. Design points worth keeping: - Images are only sent to models an administrator has marked `vision`. This is not graceful degradation -- most endpoints reject the entire request rather than ignoring an image part. A plain text turn stays a plain string for the same reason: the list form 400s on endpoints that do not implement it. - Images reach the model as base64 data URIs, not links. A local endpoint has no route back to LLeMbas, and a hosted one has no credentials for it. - Non-images are served Content-Disposition: attachment with nosniff, so an uploaded .html can never execute in this origin. Stored names are random; the uploader's name is a label and never a path. - Uploads are unbound until the message is sent, which is what lets a file be removed beforehand. claim() only takes unclaimed rows owned by the sender, so a forged id cannot pull in someone else's file. Abandoned uploads are swept at startup. - A scanned PDF says so rather than silently contributing nothing, and truncation is declared to the model in the document tag so it can admit it did not see page 400. - "Here, look at this" with no words is a legitimate turn, so a message is only empty when it carries neither text nor files. Also fixes auto-titling, which read message["content"] as a string and would have broken on the first multimodal turn. 186 tests, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
@@ -0,0 +1,38 @@
|
||||
{% from "_macros.html" import icon %}
|
||||
{#
|
||||
A pending attachment in the composer, before the message is sent.
|
||||
|
||||
Carries its own id in a hidden input so the composer form submits it with the
|
||||
message; removing the chip removes the input, which is all the bookkeeping
|
||||
the client needs.
|
||||
#}
|
||||
<div class="chip" id="chip-{{ attachment.id }}">
|
||||
<input type="hidden" name="file_ids" value="{{ attachment.id }}">
|
||||
|
||||
{% if attachment.is_image %}
|
||||
<img class="chip__thumb" src="/api/files/{{ attachment.id }}/content" alt="">
|
||||
{% else %}
|
||||
<span class="chip__icon">
|
||||
{{ icon("attach" if attachment.kind == "document" else "copy", "icon--sm") }}
|
||||
</span>
|
||||
{% endif %}
|
||||
|
||||
<span class="chip__body">
|
||||
<span class="chip__name" title="{{ attachment.filename }}">{{ attachment.filename }}</span>
|
||||
<span class="chip__meta">
|
||||
{{ attachment.human_size }}
|
||||
{%- if attachment.pages %} · {{ attachment.pages }} page{{ '' if attachment.pages == 1 else 's' }}{% endif %}
|
||||
{%- if attachment.width %} · {{ attachment.width }}×{{ attachment.height }}{% endif %}
|
||||
{%- if attachment.truncated %} · truncated{% endif %}
|
||||
</span>
|
||||
{% if attachment.extraction_error %}
|
||||
<span class="chip__warning">{{ attachment.extraction_error }}</span>
|
||||
{% endif %}
|
||||
</span>
|
||||
|
||||
<button class="btn btn--icon btn--sm" type="button" aria-label="Remove {{ attachment.filename }}"
|
||||
hx-delete="/api/files/{{ attachment.id }}"
|
||||
hx-target="#chip-{{ attachment.id }}" hx-swap="outerHTML">
|
||||
{{ icon("x", "icon--sm") }}
|
||||
</button>
|
||||
</div>
|
||||
@@ -0,0 +1,17 @@
|
||||
{% from "_macros.html" import icon %}
|
||||
{#
|
||||
A rejected upload. Rendered in place of a chip so the reason is visible in
|
||||
the composer rather than only in the network tab. Dismissed by hand; it
|
||||
carries no hidden input, so it cannot be submitted with the message.
|
||||
#}
|
||||
<div class="chip chip--error">
|
||||
<span class="chip__icon">{{ icon("warning", "icon--sm") }}</span>
|
||||
<span class="chip__body">
|
||||
<span class="chip__name">{{ filename }}</span>
|
||||
<span class="chip__warning">{{ error }}</span>
|
||||
</span>
|
||||
<button class="btn btn--icon btn--sm" type="button" aria-label="Dismiss"
|
||||
onclick="this.closest('.chip').remove()">
|
||||
{{ icon("x", "icon--sm") }}
|
||||
</button>
|
||||
</div>
|
||||
@@ -38,6 +38,41 @@
|
||||
{% endif %}
|
||||
</header>
|
||||
|
||||
{% if message.attachments %}
|
||||
{# Above the text, matching the order they were added and the order the
|
||||
model receives them. #}
|
||||
<div class="attachments">
|
||||
{% for attachment in message.attachments %}
|
||||
{% if attachment.is_image %}
|
||||
<a class="attachments__image" href="/api/files/{{ attachment.id }}/content"
|
||||
target="_blank" rel="noopener">
|
||||
<img src="/api/files/{{ attachment.id }}/content" alt="{{ attachment.filename }}"
|
||||
loading="lazy" width="{{ attachment.width }}" height="{{ attachment.height }}">
|
||||
</a>
|
||||
{% else %}
|
||||
<div class="attachments__doc">
|
||||
{{ icon("attach", "icon--sm") }}
|
||||
<span class="attachments__doc-body">
|
||||
<a href="/api/files/{{ attachment.id }}/content">{{ attachment.filename }}</a>
|
||||
<span class="chip__meta">
|
||||
{{ attachment.human_size }}
|
||||
{%- if attachment.pages %} · {{ attachment.pages }} page{{ '' if attachment.pages == 1 else 's' }}{% endif %}
|
||||
{%- if attachment.truncated %} · truncated{% endif %}
|
||||
{%- if attachment.extracted_text %}
|
||||
· <a href="/api/files/{{ attachment.id }}/text" target="_blank"
|
||||
rel="noopener">view extracted text</a>
|
||||
{%- endif %}
|
||||
</span>
|
||||
{% if attachment.extraction_error %}
|
||||
<span class="chip__warning">{{ attachment.extraction_error }}</span>
|
||||
{% endif %}
|
||||
</span>
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if streaming %}
|
||||
{# Reasoning arrives before the answer, so this block sits above it. It
|
||||
starts open (watching a model think is the point) and the :has() rule
|
||||
@@ -92,9 +127,11 @@
|
||||
{% endif %}
|
||||
{% elif message.role == "assistant" %}
|
||||
<div class="msg__body">{{ body_html|safe }}</div>
|
||||
{% else %}
|
||||
{% elif message.content %}
|
||||
<div class="msg__body msg__body--plain">{{ message.content }}</div>
|
||||
{% endif %}
|
||||
{# An attachment-only turn has no text; rendering the bubble anyway would
|
||||
leave an empty box under the file. #}
|
||||
|
||||
{% if not streaming %}
|
||||
<footer class="msg__actions">
|
||||
|
||||
@@ -164,16 +164,45 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="composer">
|
||||
<div class="composer" {% if can.get("files.upload") %}data-dropzone{% endif %}>
|
||||
{% if can.get("files.upload") %}
|
||||
{# Uploads go up as soon as a file is chosen, so the chip (and any
|
||||
rejection) appears immediately rather than at send time. The chips
|
||||
carry hidden inputs, which is how the ids reach the message POST. #}
|
||||
<form id="upload-form" hx-post="/api/files?chat_id={{ chat.id }}"
|
||||
hx-target="#attachments" hx-swap="beforeend"
|
||||
hx-encoding="multipart/form-data"
|
||||
hx-on::after-request="this.reset()">
|
||||
<input class="visually-hidden" type="file" name="file" id="file-input"
|
||||
multiple accept="image/*,.pdf,.txt,.md,.csv,.json,.py,.js,.ts,.rs,.go,.sh,.sql,.yaml,.yml,.toml,.log"
|
||||
onchange="window.lembas.uploadFiles(this.files); this.value = ''">
|
||||
</form>
|
||||
{% endif %}
|
||||
|
||||
<div class="composer__attachments" id="attachments"></div>
|
||||
|
||||
<form class="composer__form"
|
||||
hx-post="/api/chats/{{ chat.id }}/messages"
|
||||
hx-target="#thread" hx-swap="beforeend"
|
||||
hx-on::after-request="if (event.detail.successful) {
|
||||
this.reset();
|
||||
document.getElementById('attachments').replaceChildren();
|
||||
const t = this.querySelector('textarea');
|
||||
window.lembas.autosize(t);
|
||||
window.lembas.scrollThread(true);
|
||||
}">
|
||||
{# The chips live outside this form, so their hidden inputs are pulled
|
||||
in explicitly at submit time. #}
|
||||
<div hx-include="#attachments" hidden></div>
|
||||
|
||||
{% if can.get("files.upload") %}
|
||||
<button class="btn btn--icon composer__attach" type="button"
|
||||
aria-label="Attach a file" title="Attach a file"
|
||||
onclick="document.getElementById('file-input').click()">
|
||||
{{ icon("attach") }}
|
||||
</button>
|
||||
{% endif %}
|
||||
|
||||
<textarea class="composer__input" name="content" rows="1"
|
||||
data-autosize data-max-height="320" data-composer-input
|
||||
placeholder="Send a message…" aria-label="Message"></textarea>
|
||||
@@ -181,9 +210,24 @@
|
||||
{{ icon("send", "icon--sm") }}
|
||||
</button>
|
||||
</form>
|
||||
|
||||
<p class="composer__hint">
|
||||
Enter to send, Shift+Enter for a new line.
|
||||
{% if can.get("files.upload") %}
|
||||
Drag files in, or paste an image.
|
||||
{% if current_model and not current_model.capabilities_json.get("vision") %}
|
||||
<strong>{{ current_model.label }} has no vision</strong>, so images
|
||||
will not be sent — documents still will.
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</p>
|
||||
|
||||
{% if can.get("files.upload") %}
|
||||
<div class="dropzone-overlay" aria-hidden="true">
|
||||
{{ icon("attach", "icon--lg") }}
|
||||
<span>Drop to attach</span>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</main>
|
||||
|
||||
Reference in New Issue
Block a user