Files
LLeMbas/src/lembas/web/static/js/commands.js
T
Jaroslav Beneš 5e75948069 Draw a picture, on a ComfyUI you are running
The last unbuilt capability, and built the way CLAUDE.md said it had to be: a
ToolDef reaching resolve_tools plus a permission and a capability flag, not a new
code path. The only genuinely new UI is one branch in the transcript.

services/images/ is three modules. comfy.py speaks HTTP -- submit, poll /history,
fetch the PNG, /free, and an /object_info discovery for the admin page only.
Polled and not socketed, because holding a connection open for the length of a
generation is the live-connection state the whole ssh.py design forbids, and the
thing being waited for takes tens of seconds anyway. The base URL is exempt from
the SSRF guard by construction, exactly as Connection.base_url and the audio
endpoints are -- said out loud in the docstring, because a default of
127.0.0.1:8188 is precisely the shape that guard exists to refuse and therefore
reads as a hole rather than a decision.

workflow.py fills a template, and the one thing that matters is that it walks the
parsed JSON rather than the text of it. A value that is exactly "{{steps}}"
becomes the number 20; ComfyUI validates types and refuses the string. A
placeholder inside a longer string is still text, which is what makes
"{{prompt}}, masterpiece" work -- and text substitution would additionally mean a
prompt containing a quotation mark produced a document that no longer parses, on
the one input guaranteed to hold arbitrary text. Which node holds the prompt is
the administrator's statement rather than a guess from node types: sniffing for
the first CLIPTextEncode works on the shipped workflow and on nothing else, and
swaps positive for negative the first time somebody reorders them. seed has no
fixed default, because one would make every unspecified generation identical and
make the retry loop redraw the same rejected picture four times.

tool.py is one call, one finished image. Returning every attempt to the
conversation would cost a round each, make the ceiling advisory rather than
enforced, and walk the reader past every reject -- so the reviewer lives inside
the tool and is asked about *bytes*: an attempt about to be discarded should not
leave an Attachment behind, so it sees a downscaled preview built in memory and
only the kept image is written. Anything that goes wrong in review is a keep;
losing a picture because a judging request timed out would be the check
destroying the thing it was checking. The last attempt is kept whatever the
verdict, so a request always produces something. Rejects are recorded, not
stored.

Preserve VRAM unloads the chat's own connection and nothing else, because the
memory being freed belongs to one machine: local llama-swap answers GET /unload,
and a box on the network has no reason to be unloaded when ComfyUI wants memory
here. The swap goes round the review rather than round the tool, which costs two
model loads per retry -- so the two settings are independent and the page warns
when both are on. Nothing loads the LLM back: the reply's next request does, and
that step exists in the description and not in the code, so the code says so.

Two rules elsewhere had to be drawn for the first time. message_payload sends
images only on user turns -- no assistant message had ever carried one, and the
moment one does the multimodal list form on an assistant turn is rejected by
OpenAI and most local runners, breaking every later turn in the chat. And
files.store gained keep_original, because _process_image turns anything without
alpha into JPEG q85 at 1400px: right for a phone photo, a visible loss on the one
output this feature exists to produce.

/image sends the ordinary message with force_tool, which becomes tool_choice for
the first round only -- left in place the reply would draw a picture, be asked
again, and draw another. FORCEABLE_TOOLS is an allow list because the name is
read off a form.

ToolContext gained chat_id, and that fixed a tool nobody had ever successfully
run: _run_scratch_write read context.chat_id on a dataclass with no such field,
so every call raised AttributeError, swallowed by run_tool's blanket except into
"the scratch_write tool failed" -- indistinguishable from a model calling it
wrongly. The test that existed asserted the family and the risk, which are
properties of the declaration rather than of the code.

Verified against the real ComfyUI 0.27.0 on this machine rather than against
documentation: every endpoint shape here was read off it, a generation ran end to
end through the client, the reviewer was shown a matching and a mismatched prompt
and answered KEEP and RETRY correctly, and the unload hook fired for the local
llama-swap and not for the remote box.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 14:13:19 +02:00

627 lines
25 KiB
JavaScript

/*
Slash commands, and the keyboard shortcuts that do the same jobs.
Both live here, in one table, so `/help` cannot describe a shortcut that no
longer exists. Every entry does something that was already possible by
clicking -- none of this is new server behaviour except `/usage`, which is a
question the interface could not previously answer at all.
The rule that matters: a message that merely *starts* with a slash must still
send. `//` escapes, an unrecognised command is left alone and posted as text,
and only an exact match against this table is intercepted. Silently eating
somebody's message is a far worse failure than an unknown command.
Shortcuts are Alt-based rather than Ctrl+Shift: the browser owns
Ctrl+Shift+T, N and W and will not give them up. They are matched on
`event.code`, which is the physical key, so a Dvorak or a Slovak layout gets
the same shortcuts rather than whichever letters happen to sit there.
*/
(function () {
"use strict";
function el(selector) { return document.querySelector(selector); }
function chat() {
var box = el(".composer");
return (box && box.dataset.chatId) || "";
}
function isAgent() { return !!el('[name="agent_mode"]'); }
function post(url, options) {
return fetch(url, Object.assign({ method: "POST", credentials: "same-origin" }, options || {}));
}
function note(message, kind) {
if (window.lembas && window.lembas.notify) {
window.lembas.notify(message, { kind: kind || "info" });
}
}
/* --- Shortcuts ---------------------------------------------------------- */
var SHORTCUTS = [
{ keys: "Ctrl/⌘ + K", what: "Open the command menu" },
{ keys: "Ctrl/⌘ + Enter", what: "Send, from anywhere on the page" },
{ keys: "Alt + M", what: "Dictate" },
{ keys: "Alt + R", what: "Read the last reply aloud" },
{ keys: "Alt + 1 … 4", what: "Manual, Edit, Auto, Plan" },
{ keys: "Alt + E", what: "Canvas" },
{ keys: "Alt + T", what: "Terminal" },
{ keys: "Alt + I", what: "Inspector" },
{ keys: "Alt + B", what: "Sidebar" },
{ keys: "Alt + N", what: "New chat" },
{ keys: "↑ in an empty box", what: "Edit your last message" },
{ keys: "Esc", what: "Close what is open, or stop the reply" },
{ keys: "Ctrl + Shift + C / V", what: "Copy and paste inside the terminal" }
];
/* --- The table ---------------------------------------------------------- */
var COMMANDS = [
{
name: "help",
summary: "Commands and keyboard shortcuts",
run: function () { helpSheet(); }
},
{
name: "usage",
summary: "Tokens and context used by this chat",
when: function () { return !!chat(); },
run: function () {
fetch("/api/chats/" + chat() + "/usage", { credentials: "same-origin" })
.then(function (r) { return r.text(); })
.then(function (html) { sheet("Usage", html); })
.catch(function () { note("Could not read this chat's usage.", "error"); });
}
},
{
name: "compact",
summary: "Summarise the earlier turns so they stop costing context",
when: function () { return !!chat() && !!el("#thread .msg"); },
run: compact
},
{
name: "effort",
summary: "How hard a reasoning model should think",
argument: "low | medium | high | off",
/* Offered wherever there is a model, not only where the control is.
`available()` filters `find()` and `run()` as well as the menu, so a
command hidden here is not merely unlisted -- typing it in full stops
being a command and gets sent as a message. Better to answer. */
when: function () { return !!el('[name="model_id"]'); },
run: function (rest) { setEffort(rest); }
},
{
name: "mode",
summary: "Approval mode: manual, edit, auto or plan",
argument: "manual | edit | auto | plan",
when: isAgent,
run: function (rest) {
var select = el('[name="agent_mode"]');
var wanted = (rest || "").trim().toLowerCase();
if (!wanted) return note("Modes: manual, edit, auto, plan.");
var found = Array.prototype.find.call(select.options, function (option) {
return option.value === wanted;
});
if (!found) return note("“" + wanted + "” is not a mode.", "error");
select.value = wanted;
select.dispatchEvent(new Event("change", { bubbles: true }));
note("Mode set to " + found.textContent.trim() + ".");
}
},
{
name: "title",
summary: "Rename this chat",
argument: "the new title",
when: function () { return !!chat(); },
run: function (rest) {
var wanted = (rest || "").trim();
if (!wanted) return note("Give it a title: /title Something.");
var body = new FormData();
body.append("title", wanted);
fetch("/api/chats/" + chat(), { method: "PATCH", body: body, credentials: "same-origin" })
.then(function () {
/* Both places the title appears. The heading alone left the sidebar
row showing the old name until the next reload, which reads as a
rename that half worked -- and is the reason the route now hands
back the out-of-band pair for every other caller. This one is a
bare fetch rather than htmx, so it sets them itself.
textContent, never innerHTML: this is text somebody typed. */
[el("#chat-title"), el("#chat-link-label-" + chat())].forEach(function (node) {
if (node) node.textContent = wanted;
});
note("Renamed.");
});
}
},
{
name: "index",
summary: "Read the project directory again",
when: function () { return !!chat() && isAgent(); },
run: function () { reindex(); }
},
{
name: "canvas",
summary: "Show or hide the canvas",
/* Gated, like the other two panels. An ungated command on a page with no
panel does not merely fail -- it stops being a command, and the message
is sent as written. */
when: function () { return !!el("#canvas"); },
run: function () { toggle("#canvas", "side"); }
},
{
name: "terminal",
summary: "Show or hide the terminal",
when: function () { return !!el("#terminal"); },
run: function () { toggle("#terminal", "side"); }
},
{
name: "inspector",
summary: "Show or hide the request inspector",
when: function () { return !!el("#inspector"); },
run: function () { toggle("#inspector", "side"); }
},
{ name: "sidebar", summary: "Show or hide the sidebar", run: function () { toggle("#sidebar"); } },
{
name: "theme",
summary: "Switch theme",
argument: "moria | shire",
run: function (rest) {
var wanted = (rest || "").trim().toLowerCase();
if (wanted === "moria" || wanted === "shire") window.lembas.applyTheme(wanted);
else window.lembas.toggleTheme();
}
},
{
name: "image",
summary: "Draw a picture",
argument: "what to draw",
/* Offered wherever there is a chat, not only where image generation is
switched on. `available()` filters `run()` as well as the menu, so a
command hidden here stops being a command and gets *sent as a message*
-- and "/image a red bicycle" arriving as prose is worse than being
told the feature is off. The server answers either way. */
when: function () { return !!chat(); },
run: function (rest) {
var wanted = (rest || "").trim();
if (!wanted) return note("Say what to draw: /image a red bicycle in the rain.");
var thread = el("#thread");
if (!thread) return;
var body = new FormData();
body.append("content", wanted);
/* The whole of what this command is. The turn goes through the ordinary
path -- same route, same bubbles, same stream -- and carries one extra
field that makes the first round call the image tool instead of
deciding whether to. The words still say what is wanted, so an
endpoint that ignores tool_choice steers on those alone. */
body.append("force_tool", "image_generate");
fetch("/api/chats/" + chat() + "/messages", {
method: "POST",
body: body,
credentials: "same-origin"
})
.then(function (r) { return r.text(); })
.then(function (html) {
thread.insertAdjacentHTML("beforeend", html);
/* Without this the assistant bubble's sse-connect is inert markup
and the reply never starts -- the same reason /compact processes
the thread it swapped in. */
if (window.htmx) window.htmx.process(thread);
if (window.lembas.scrollThread) window.lembas.scrollThread(true);
})
.catch(function () { note("Could not start the image.", "error"); });
}
},
{ name: "new", summary: "Start a new chat", run: function () { window.location = "/chat"; } },
{
name: "temp",
summary: "Start a temporary chat, gone after a day",
run: function () { window.location = "/chat?temporary=1"; }
},
{
name: "stop",
summary: "Stop the reply being written",
run: function () {
var button = el('[data-composer-action="stop"]');
if (button) button.click();
else note("Nothing is being written.");
}
},
{ name: "knowledge", summary: "Your library", run: go("/library/knowledge") },
{ name: "notes", summary: "Notes the model has written", run: go("/library/notes") },
{ name: "skills", summary: "Saved procedures", run: go("/library/skills") },
{ name: "connections", summary: "Your SSH connections", run: go("/agents") }
];
function go(url) {
return function () { window.location = url; };
}
/* --- Reading the project directory again --------------------------------
The listing is cached for five minutes and only ever built when a reply
starts, so anything done in the terminal panel -- a checkout, a build --
is invisible to it until then. This is the "look again now". */
var indexing = false;
function reindex() {
if (indexing) return note("Already reading the project directory.");
indexing = true;
note("Reading the project directory…");
post("/api/chats/" + chat() + "/index")
.then(function (response) {
return response.json().then(function (body) {
return { ok: response.ok, body: body };
});
})
.then(function (result) {
if (!result.ok) {
return note(result.body.detail || "Could not read the project directory.", "error");
}
note(result.body.message);
})
.catch(function () { note("Could not read the project directory.", "error"); })
.finally(function () { indexing = false; });
}
/* --- Reasoning effort ---------------------------------------------------
The command drives the same select the composer shows, so there is one
piece of state and the control updates itself when the command is used. */
var EFFORTS = ["low", "medium", "high"];
function setEffort(rest) {
var select = el("[data-effort]");
if (!select) {
return note(
"This model is not marked as a reasoning model, so effort would do " +
"nothing. An administrator can mark it on the model's page.",
"error"
);
}
var wanted = (rest || "").trim().toLowerCase();
if (!wanted) {
return note(
EFFORTS.indexOf(select.value) === -1
? "No effort is being sent. Try low, medium or high."
: "Effort is " + select.value + ". /effort low, medium, high, or off."
);
}
/* "off" is the option's real value, not an empty string: the new-chat form
cannot tell an absent field from an empty one, so the picker sends a
sentinel and this has to match it. "default" and "none" still work,
because somebody's fingers will type them. */
if (wanted === "default" || wanted === "none") wanted = "off";
else if (wanted !== "off" && EFFORTS.indexOf(wanted) === -1) {
return note("“" + wanted + "” is not an effort. Try low, medium, high or off.", "error");
}
select.value = wanted;
select.dispatchEvent(new Event("change", { bubbles: true }));
note(
wanted === "off"
? "Effort cleared; nothing is sent."
: "Effort set to " + wanted + "."
);
}
/* --- Compacting --------------------------------------------------------
One implementation, reached from the command and from the overflow menu
alike. The menu used to do its own hx-post, which meant two code paths and
a spinner on neither -- so after confirming, the menu closed and nothing
visible happened for however long the summarising model took.
The `Generation.status` channel that says "Summarising earlier messages…"
during *automatic* compaction cannot be borrowed: it lives inside the
streaming bubble, and this endpoint refuses to run while any message is
unfinished, so there is no bubble to put it in. */
var compacting = false;
function compact() {
if (compacting) return note("Already summarising.");
var thread = el("#thread");
if (!thread) return;
window.lembas.confirm(
"Summarise everything before the last reply? The messages stay in the " +
"transcript; they just stop being sent to the model.",
{ title: "Compact this chat", label: "Compact" }
).then(function (yes) {
if (!yes) return;
compacting = true;
var working = document.createElement("div");
working.className = "thread__working";
// The same words the automatic path uses, so the two do not look like
// different features.
working.innerHTML =
'<span class="dots"><i></i><i></i><i></i></span>' +
"<span>Summarising the earlier messages…</span>";
thread.appendChild(working);
if (window.lembas.scrollThread) window.lembas.scrollThread(true);
function done() {
compacting = false;
working.remove();
}
post("/api/chats/" + chat() + "/compact")
.then(function (r) { return r.ok ? r.text() : Promise.reject(r); })
.then(function (html) {
compacting = false;
// The response replaces the thread wholesale, placeholder included.
thread.innerHTML = html;
if (window.htmx) window.htmx.process(thread);
if (window.lembas.scrollThread) window.lembas.scrollThread(true);
})
.catch(function (r) {
done();
/* The endpoint's four 409s are written for a person to read -- "Wait
for the current reply to finish, then compact." -- and until now
reached nobody at all. */
if (r && r.json) {
r.json()
.then(function (body) { note(body.detail || "Could not compact.", "error"); })
.catch(function () { note("Could not compact this chat.", "error"); });
} else {
note("Could not compact this chat.", "error");
}
});
});
}
function toggle(selector, group) {
var panel = el(selector);
if (panel && window.lembas.setPanel) {
window.lembas.setPanel(selector, panel.hasAttribute("hidden"), group);
}
}
function available() {
return COMMANDS.filter(function (command) { return !command.when || command.when(); });
}
/* --- What the composer calls -------------------------------------------- */
function list(query) {
var needle = (query || "").toLowerCase();
var matches = available().filter(function (command) {
return command.name.indexOf(needle) === 0;
});
var wrap = document.createElement("div");
if (!matches.length) {
var empty = document.createElement("p");
empty.className = "muted text-sm";
empty.style.padding = "var(--sp-3)";
empty.textContent = "No command called “" + query + "”. It will be sent as a message.";
wrap.appendChild(empty);
return wrap;
}
var group = document.createElement("p");
group.className = "picker__group";
group.textContent = "Commands";
wrap.appendChild(group);
var items = document.createElement("ul");
items.className = "picker__list";
matches.forEach(function (command) {
var row = document.createElement("li");
var button = document.createElement("button");
button.type = "button";
button.className = "picker__option";
button.dataset.command = command.name;
var body = document.createElement("span");
body.className = "picker__option-body";
var name = document.createElement("span");
name.className = "picker__option-name";
name.textContent = "/" + command.name + (command.argument ? " " + command.argument : "");
var summary = document.createElement("span");
summary.className = "picker__option-note";
summary.textContent = command.summary;
body.appendChild(name);
body.appendChild(summary);
button.appendChild(body);
row.appendChild(button);
items.appendChild(row);
});
wrap.appendChild(items);
return wrap;
}
/*
A command, or null -- and null is the important half.
Anything not matching exactly is left for the composer to send as an
ordinary message, and `//` strips one slash on the way. A chat application
that swallows a message because it began with a slash has done something
much worse than failing to recognise a command.
*/
function find(value) {
if (value[0] !== "/" || value[1] === "/") return null;
var match = /^\/([a-z]+)(?:\s+([\s\S]*))?$/.exec(value.trim());
if (!match) return null;
var found = available().find(function (command) { return command.name === match[1]; });
return found ? { name: found.name, rest: match[2] || "" } : null;
}
function run(name, rest) {
var found = available().find(function (command) { return command.name === name; });
if (found) found.run(rest || "");
}
/* --- Sheets ------------------------------------------------------------- */
function sheet(title, html) {
var dialog = document.createElement("dialog");
dialog.className = "dialog dialog--wide";
var form = document.createElement("div");
form.className = "dialog__form";
var heading = document.createElement("h2");
heading.className = "dialog__title";
heading.textContent = title;
var body = document.createElement("div");
// Server-rendered and already escaped there; nothing user-typed reaches
// this path as markup.
body.innerHTML = html;
var actions = document.createElement("div");
actions.className = "dialog__actions";
var close = document.createElement("button");
close.className = "btn";
close.type = "button";
close.textContent = "Close";
actions.appendChild(close);
form.appendChild(heading);
form.appendChild(body);
form.appendChild(actions);
dialog.appendChild(form);
document.body.appendChild(dialog);
function finish() {
dialog.close();
setTimeout(function () { dialog.remove(); }, 200);
}
close.addEventListener("click", finish);
dialog.addEventListener("cancel", function (event) { event.preventDefault(); finish(); });
dialog.addEventListener("click", function (event) { if (event.target === dialog) finish(); });
dialog.showModal();
}
function helpSheet() {
var rows = available().map(function (command) {
return (
"<tr><td class='mono'>/" + command.name +
(command.argument ? " " + escapeText(command.argument) : "") +
"</td><td>" + escapeText(command.summary) + "</td></tr>"
);
});
var keys = SHORTCUTS.map(function (shortcut) {
return (
"<tr><td class='mono'>" + escapeText(shortcut.keys) + "</td><td>" +
escapeText(shortcut.what) + "</td></tr>"
);
});
sheet(
"Commands and shortcuts",
"<table class='sheet'><tbody>" + rows.join("") + "</tbody></table>" +
"<h3 class='section-title'>Keyboard</h3>" +
"<table class='sheet'><tbody>" + keys.join("") + "</tbody></table>" +
"<p class='muted text-sm'>A message that starts with a slash but is not a " +
"command is sent as written. Type <span class='mono'>//</span> to start one " +
"with a literal slash.</p>"
);
}
function escapeText(value) {
var holder = document.createElement("span");
holder.textContent = value;
return holder.innerHTML;
}
/* --- Keyboard ----------------------------------------------------------- */
var MODES = ["manual", "edit", "auto", "plan"];
document.addEventListener("keydown", function (event) {
if (event.isComposing) return;
/* Never inside the terminal: every keystroke there belongs to the shell,
and a shortcut that steals one is a shortcut that breaks vim. */
if (event.target.closest && event.target.closest("#terminal")) return;
if ((event.ctrlKey || event.metaKey) && event.code === "KeyK") {
event.preventDefault();
var input = document.querySelector("[data-composer-input]");
if (!input) return;
input.focus();
input.value = "/";
input.dispatchEvent(new Event("input", { bubbles: true }));
return;
}
/* Send, from anywhere on the page.
Enter already sends, but only with the caret inside the box (app.js), and
deliberately not at all on a touch device. This covers both: after
clicking a message to copy it, after using the model picker, after
answering an approval card, or with a hardware keyboard on a tablet.
Never Stop. Send and Stop are the same element, so Ctrl+Enter meaning
"abandon the reply" would be a trap -- and Esc already stops. */
if ((event.ctrlKey || event.metaKey) &&
(event.code === "Enter" || event.code === "NumpadEnter")) {
var action = el("[data-composer-action]");
var box = el("[data-composer-input]");
if (action && action.dataset.composerAction === "send" && box && box.value.trim()) {
event.preventDefault();
action.click();
}
return;
}
if (!event.altKey || event.ctrlKey || event.metaKey) return;
/* Dictation and read-aloud both work by clicking the button that already
does the job, so audio.js keeps its one delegated click listener and
there is no second copy of the recording state machine. Alt+M rather than
Alt+D: Alt+D is the address bar in Chrome and Firefox. */
if (event.code === "KeyM") {
var mic = el("[data-mic]");
if (mic) {
event.preventDefault();
mic.click();
}
return;
}
if (event.code === "KeyR") {
var speakers = document.querySelectorAll("#thread .msg--assistant [data-speak]");
if (speakers.length) {
event.preventDefault();
/* A second press stops it: audio.js already toggles a message that is
speaking, so this costs nothing and is the obvious second press. */
speakers[speakers.length - 1].click();
}
return;
}
/* E for editor, not C: Ctrl/Cmd+C is too near for comfort, and Alt+D is
the address bar in two browsers -- a shortcut the browser wins looks
broken. */
if (event.code === "KeyE" && el("#canvas")) {
event.preventDefault();
return toggle("#canvas", "side");
}
if (event.code === "KeyT" && el("#terminal")) {
event.preventDefault();
return toggle("#terminal", "side");
}
if (event.code === "KeyI" && el("#inspector")) {
event.preventDefault();
return toggle("#inspector", "side");
}
if (event.code === "KeyB") {
event.preventDefault();
return toggle("#sidebar");
}
if (event.code === "KeyN") {
event.preventDefault();
window.location = "/chat";
return;
}
var digit = ["Digit1", "Digit2", "Digit3", "Digit4"].indexOf(event.code);
if (digit !== -1 && isAgent()) {
event.preventDefault();
run("mode", MODES[digit]);
}
});
/* Up-arrow in an empty box edits your last turn, the way a shell recalls the
last command. Only when the box is empty, so it never eats a cursor key
somebody was using to move around what they had written. */
document.addEventListener("keydown", function (event) {
if (event.key !== "ArrowUp" || event.shiftKey || event.altKey) return;
var input = event.target.closest && event.target.closest("[data-composer-input]");
if (!input || input.value !== "") return;
var edits = document.querySelectorAll("#thread .msg--user [data-edit-message]");
if (!edits.length) return;
event.preventDefault();
edits[edits.length - 1].click();
});
window.lembasCommands = { list: list, find: find, run: run, help: helpSheet };
})();