Draw a picture, on a ComfyUI you are running

The last unbuilt capability, and built the way CLAUDE.md said it had to be: a
ToolDef reaching resolve_tools plus a permission and a capability flag, not a new
code path. The only genuinely new UI is one branch in the transcript.

services/images/ is three modules. comfy.py speaks HTTP -- submit, poll /history,
fetch the PNG, /free, and an /object_info discovery for the admin page only.
Polled and not socketed, because holding a connection open for the length of a
generation is the live-connection state the whole ssh.py design forbids, and the
thing being waited for takes tens of seconds anyway. The base URL is exempt from
the SSRF guard by construction, exactly as Connection.base_url and the audio
endpoints are -- said out loud in the docstring, because a default of
127.0.0.1:8188 is precisely the shape that guard exists to refuse and therefore
reads as a hole rather than a decision.

workflow.py fills a template, and the one thing that matters is that it walks the
parsed JSON rather than the text of it. A value that is exactly "{{steps}}"
becomes the number 20; ComfyUI validates types and refuses the string. A
placeholder inside a longer string is still text, which is what makes
"{{prompt}}, masterpiece" work -- and text substitution would additionally mean a
prompt containing a quotation mark produced a document that no longer parses, on
the one input guaranteed to hold arbitrary text. Which node holds the prompt is
the administrator's statement rather than a guess from node types: sniffing for
the first CLIPTextEncode works on the shipped workflow and on nothing else, and
swaps positive for negative the first time somebody reorders them. seed has no
fixed default, because one would make every unspecified generation identical and
make the retry loop redraw the same rejected picture four times.

tool.py is one call, one finished image. Returning every attempt to the
conversation would cost a round each, make the ceiling advisory rather than
enforced, and walk the reader past every reject -- so the reviewer lives inside
the tool and is asked about *bytes*: an attempt about to be discarded should not
leave an Attachment behind, so it sees a downscaled preview built in memory and
only the kept image is written. Anything that goes wrong in review is a keep;
losing a picture because a judging request timed out would be the check
destroying the thing it was checking. The last attempt is kept whatever the
verdict, so a request always produces something. Rejects are recorded, not
stored.

Preserve VRAM unloads the chat's own connection and nothing else, because the
memory being freed belongs to one machine: local llama-swap answers GET /unload,
and a box on the network has no reason to be unloaded when ComfyUI wants memory
here. The swap goes round the review rather than round the tool, which costs two
model loads per retry -- so the two settings are independent and the page warns
when both are on. Nothing loads the LLM back: the reply's next request does, and
that step exists in the description and not in the code, so the code says so.

Two rules elsewhere had to be drawn for the first time. message_payload sends
images only on user turns -- no assistant message had ever carried one, and the
moment one does the multimodal list form on an assistant turn is rejected by
OpenAI and most local runners, breaking every later turn in the chat. And
files.store gained keep_original, because _process_image turns anything without
alpha into JPEG q85 at 1400px: right for a phone photo, a visible loss on the one
output this feature exists to produce.

/image sends the ordinary message with force_tool, which becomes tool_choice for
the first round only -- left in place the reply would draw a picture, be asked
again, and draw another. FORCEABLE_TOOLS is an allow list because the name is
read off a form.

ToolContext gained chat_id, and that fixed a tool nobody had ever successfully
run: _run_scratch_write read context.chat_id on a dataclass with no such field,
so every call raised AttributeError, swallowed by run_tool's blanket except into
"the scratch_write tool failed" -- indistinguishable from a model calling it
wrongly. The test that existed asserted the family and the risk, which are
properties of the declaration rather than of the code.

Verified against the real ComfyUI 0.27.0 on this machine rather than against
documentation: every endpoint shape here was read off it, a generation ran end to
end through the client, the reviewer was shown a matching and a mismatched prompt
and answered KEEP and RETRY correctly, and the unload hook fired for the local
llama-swap and not for the remote box.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/lembas/api/admin.py

@@ -145,11 +145,19 @@ async def update_connection(
     base_url: str = Form(...),
     api_key: str = Form(""),
     enabled: bool = Form(False),
+    unload_url: str = Form(""),
+    unload_method: str = Form("POST"),
 ) -> Response:
     connection = _connection(db, connection_id)
     connection.name = name.strip()[:120] or connection.name
     connection.base_url = base_url.strip().rstrip("/")
     connection.enabled = enabled
+    # How to ask this endpoint to drop its model, for image generation's
+    # Preserve VRAM. Empty means it cannot be unloaded, which is the honest
+    # answer for anything not running on the machine ComfyUI is on.
+    connection.unload_url = unload_url.strip()[:500]
+    method = unload_method.strip().upper()
+    connection.unload_method = method if method in ("GET", "POST") else "POST"
 
     submitted = api_key.strip()
     if submitted and submitted != UNCHANGED_SENTINEL:

src/lembas/api/admin_images.py

@@ -0,0 +1,381 @@
+"""Image generation administration: the ComfyUI, and the workflows to run on it.
+
+Two shapes on one nav entry, because they are two different kinds of thing. The
+connection, the checkpoints and the switches are instance settings and get a
+settings page. A workflow is an authored document with a name, a description and
+a body, so the workflows are list-plus-detail -- the shape `CLAUDE.md` requires
+of any admin list, and for the reason it gives: a page that renders a ten-line
+JSON textarea per row is unusable at three rows.
+
+Route order matters and is not alphabetical. `/admin/images/workflows/new` is
+registered before `/admin/images/workflows/{workflow_id}`, or "new" is captured
+as an id and 404s. That has already been a bug twice here.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, Form, HTTPException, Request, Response, status
+from fastapi.responses import RedirectResponse
+from sqlalchemy import func, select
+
+from lembas.api.deps import AdminUser, Db
+from lembas.db.models import ImageWorkflow, Model
+from lembas.services import settings_store
+from lembas.services.crypto import UNCHANGED_SENTINEL, decrypt, keep_or_replace, mask
+from lembas.services.images import comfy
+from lembas.services.images import workflow as workflow_service
+from lembas.services.llm.openai_client import LLMError
+from lembas.web.templating import render
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/admin/images", tags=["admin-images"])
+
+SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]{0,47}$")
+
+# The placeholders a workflow has to carry to be worth having. Without a prompt
+# it draws the same picture whatever anybody types, which is the one failure
+# somebody would not think to look for.
+REQUIRED_PLACEHOLDERS = ("prompt",)
+
+
+def _lines(text: str) -> list[str]:
+    """One name per line, blanks dropped. The `admin_agents` pattern."""
+    seen: list[str] = []
+    for line in (text or "").splitlines():
+        name = line.strip()
+        if name and name not in seen:
+            seen.append(name)
+    return seen
+
+
+def _config(db: Db) -> comfy.Config:
+    values = settings_store.images(db)
+    return comfy.Config(
+        base_url=str(values.get("base_url") or ""),
+        api_key=decrypt(str(values.get("api_key_encrypted") or "")),
+        timeout=30.0,
+    )
+
+
+def _page(request: Request, db: Db, **extra) -> Response:
+    values = settings_store.images(db)
+    workflows = list(
+        db.scalars(select(ImageWorkflow).order_by(ImageWorkflow.position, ImageWorkflow.slug))
+    )
+    return render(
+        request,
+        "admin/images.html",
+        {
+            "values": values,
+            "workflows": workflows,
+            # Only models an administrator has marked as having vision can
+            # review, so the picker offers those and nothing else -- a list
+            # including text-only models would be a list of choices that
+            # silently do nothing.
+            "vision_models": list(
+                db.scalars(
+                    select(Model)
+                    .where(Model.enabled.is_(True))
+                    .order_by(Model.position, Model.model_id)
+                )
+            ),
+            "checkpoints_text": "\n".join(values.get("checkpoints") or []),
+            "masked": mask(decrypt(values.get("api_key_encrypted") or "")),
+            "unchanged": UNCHANGED_SENTINEL,
+            **extra,
+        },
+    )
+
+
+@router.get("")
+async def images_page(request: Request, db: Db, user: AdminUser, saved: str = "") -> Response:
+    return _page(request, db, saved=saved)
+
+
+@router.post("")
+async def save_images(
+    request: Request,
+    db: Db,
+    user: AdminUser,
+    enabled: bool = Form(False),
+    base_url: str = Form(""),
+    api_key: str = Form(""),
+    timeout: float = Form(600.0),
+    checkpoints: str = Form(""),
+    default_workflow_id: str = Form(""),
+    review_enabled: bool = Form(False),
+    review_model_id: str = Form(""),
+    max_tries: int = Form(4),
+    preserve_vram: bool = Form(False),
+    instructions: str = Form(""),
+) -> Response:
+    """Save the settings.
+
+    Every toggle defaults to False because an unticked checkbox is simply absent
+    from a form post -- that absence *is* the off signal, the rule
+    `admin_audio` states.
+
+    The discovered sampler and scheduler lists are deliberately not submitted
+    and not cleared here: they belong to whatever ComfyUI was tested, and a save
+    that only changed the instructions box has no opinion about them.
+    """
+    current = settings_store.images(db)
+    settings_store.update(
+        db,
+        {
+            "enabled": enabled,
+            "base_url": base_url.strip().rstrip("/"),
+            "api_key_encrypted": keep_or_replace(
+                api_key, current.get("api_key_encrypted") or ""
+            ),
+            "timeout": min(max(timeout, 10.0), 3600.0),
+            "checkpoints": _lines(checkpoints),
+            "default_workflow_id": default_workflow_id.strip(),
+            "review_enabled": review_enabled,
+            "review_model_id": review_model_id.strip(),
+            "max_tries": min(max(max_tries, 1), 10),
+            "preserve_vram": preserve_vram,
+            "instructions": instructions.strip()[:4000],
+        },
+        key=settings_store.IMAGES,
+    )
+    log.info("image generation %s by %s", "enabled" if enabled else "disabled", user.email)
+    return RedirectResponse(
+        "/admin/images?saved=Saved.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/test")
+async def test_images(request: Request, db: Db, user: AdminUser) -> Response:
+    """Ask ComfyUI what it can do, and remember the answer.
+
+    Against the *saved* settings rather than the unsaved form, so what is tested
+    is what a chat would actually reach -- the same rule `/admin/search/test`
+    follows.
+
+    The lists are stored rather than only shown, because the request path may
+    never ask ComfyUI anything: `harness.context_variables` is synchronous and
+    the tool schema is built per request, so both read what this button wrote.
+    """
+    config = _config(db)
+    if not config.configured:
+        return render(
+            request,
+            "admin/_images_result.html",
+            {"message": "Set a base URL first.", "message_kind": "error"},
+        )
+    try:
+        checkpoints, samplers, schedulers = await comfy.discover(config)
+    except LLMError as exc:
+        return render(
+            request,
+            "admin/_images_result.html",
+            {"message": exc.message, "message_kind": "error"},
+        )
+
+    stored = settings_store.images(db)
+    changes: dict = {"samplers": samplers, "schedulers": schedulers}
+    # The checkpoint list is filled in only when nobody has one yet, for the
+    # reason a refreshed connection does not overwrite a context length an
+    # administrator typed: they are usually narrowing it deliberately.
+    if not stored.get("checkpoints"):
+        changes["checkpoints"] = checkpoints
+    settings_store.update(db, changes, key=settings_store.IMAGES)
+
+    found = (
+        f"Found {len(checkpoints)} checkpoint{'' if len(checkpoints) == 1 else 's'}, "
+        f"{len(samplers)} samplers and {len(schedulers)} schedulers."
+    )
+    return render(
+        request,
+        "admin/_images_result.html",
+        {
+            "message": found,
+            "message_kind": "success",
+            "checkpoints": checkpoints,
+            "kept": bool(stored.get("checkpoints")),
+        },
+    )
+
+
+# --- Workflows -----------------------------------------------------------------
+def _workflow(db: Db, workflow_id: str) -> ImageWorkflow:
+    row = db.get(ImageWorkflow, workflow_id)
+    if row is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, "That workflow no longer exists.")
+    return row
+
+
+def _detail(request: Request, row: ImageWorkflow, *, is_new: bool, error: str = "", **extra):
+    return render(
+        request,
+        "admin/workflow_detail.html",
+        {
+            "workflow": row,
+            "is_new": is_new,
+            "error": error,
+            "placeholders": workflow_service.PLACEHOLDERS,
+            "workflow_text": extra.pop(
+                "workflow_text", json.dumps(row.workflow_json or {}, indent=2)
+            ),
+            **extra,
+        },
+    )
+
+
+def _populate(row: ImageWorkflow, form) -> None:
+    row.name = str(form.get("name") or "").strip()[:120]
+    row.description = str(form.get("description") or "").strip()[:2000]
+    row.enabled = "enabled" in form
+
+
+def _problem(db: Db, row: ImageWorkflow, form, *, existing_id: str = "") -> str:
+    """Why this cannot be saved, or an empty string.
+
+    A sentence rather than a 422, so a rejected save re-renders the form with
+    what was typed still in it -- losing forty lines of JSON to a validation
+    error is not a thing to do to somebody.
+    """
+    if not row.name:
+        return "A workflow needs a name."
+
+    slug = str(form.get("slug") or "").strip().lower()
+    if not SLUG_PATTERN.match(slug):
+        return (
+            "The name the model uses must be lowercase letters, digits, "
+            "hyphens or underscores, and start with a letter or digit."
+        )
+    clash = db.scalar(select(ImageWorkflow).where(ImageWorkflow.slug == slug))
+    if clash is not None and clash.id != existing_id:
+        return f"There is already a workflow called “{slug}”."
+    row.slug = slug
+
+    raw = str(form.get("workflow") or "").strip()
+    if not raw:
+        return "Paste the workflow, in ComfyUI's API format."
+    try:
+        parsed = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        return f"That is not valid JSON: {exc}"
+    if not isinstance(parsed, dict) or not parsed:
+        return (
+            "A ComfyUI API workflow is a JSON object keyed by node id. Use "
+            "“Export (API)” in ComfyUI rather than “Save”."
+        )
+
+    # The check worth having: a workflow with no {{prompt}} in it draws the same
+    # picture whatever anybody types, and would look like a broken model rather
+    # than an unparameterised template.
+    found = workflow_service.placeholders_in(parsed)
+    missing = [name for name in REQUIRED_PLACEHOLDERS if name not in found]
+    if missing:
+        return (
+            f"The workflow never uses {{{{{missing[0]}}}}}, so every image would be "
+            f"the same. Put it where the text prompt goes."
+        )
+    unknown = found - set(workflow_service.PLACEHOLDERS)
+    if unknown:
+        return f"Unknown placeholder {{{{{sorted(unknown)[0]}}}}}."
+
+    row.workflow_json = parsed
+    return ""
+
+
+@router.get("/workflows/new")
+async def new_workflow(request: Request, db: Db, user: AdminUser) -> Response:
+    """A draft, never persisted -- the `admin_tools` shape.
+
+    Registered before `/workflows/{workflow_id}`: FastAPI matches in
+    registration order, and with the parameterised route first "new" is an id.
+    """
+    from pathlib import Path
+
+    base = Path(__file__).resolve().parent.parent / "services/images/base_workflow.json"
+    draft = ImageWorkflow(
+        slug="",
+        name="",
+        description="",
+        workflow_json=json.loads(base.read_text(encoding="utf-8")),
+        enabled=True,
+    )
+    return _detail(request, draft, is_new=True)
+
+
+@router.post("/workflows")
+async def create_workflow(request: Request, db: Db, user: AdminUser) -> Response:
+    form = await request.form()
+    row = ImageWorkflow(workflow_json={})
+    _populate(row, form)
+    problem = _problem(db, row, form)
+    if problem:
+        return _detail(
+            request,
+            row,
+            is_new=True,
+            error=problem,
+            workflow_text=str(form.get("workflow") or ""),
+        )
+    row.position = (
+        db.scalar(select(func.coalesce(func.max(ImageWorkflow.position), -1))) or -1
+    ) + 1
+    db.add(row)
+    db.commit()
+    log.info("%s added image workflow %s", user.email, row.slug)
+    return RedirectResponse(
+        f"/admin/images?saved=Added {row.name}.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.get("/workflows/{workflow_id}/edit")
+async def edit_workflow(request: Request, db: Db, user: AdminUser, workflow_id: str) -> Response:
+    return _detail(request, _workflow(db, workflow_id), is_new=False)
+
+
+@router.post("/workflows/{workflow_id}/delete")
+async def delete_workflow(db: Db, user: AdminUser, workflow_id: str) -> Response:
+    row = _workflow(db, workflow_id)
+    name = row.name
+    db.delete(row)
+    db.commit()
+    log.info("%s deleted image workflow %s", user.email, name)
+    return RedirectResponse(
+        f"/admin/images?saved=Deleted {name}.", status_code=status.HTTP_303_SEE_OTHER
+    )
+
+
+@router.post("/workflows/{workflow_id}")
+async def update_workflow(request: Request, db: Db, user: AdminUser, workflow_id: str) -> Response:
+    row = _workflow(db, workflow_id)
+    form = await request.form()
+
+    # Validated against a draft, so a rejected save leaves the stored row alone
+    # and the form still holds what was typed.
+    draft = ImageWorkflow(workflow_json={}, position=row.position)
+    _populate(draft, form)
+    problem = _problem(db, draft, form, existing_id=row.id)
+    if problem:
+        draft.id = row.id
+        return _detail(
+            request,
+            draft,
+            is_new=False,
+            error=problem,
+            workflow_text=str(form.get("workflow") or ""),
+        )
+
+    _populate(row, form)
+    row.slug = draft.slug
+    row.workflow_json = draft.workflow_json
+    row.last_checked_at = datetime.now(UTC)
+    row.last_error = ""
+    db.commit()
+    log.info("%s updated image workflow %s", user.email, row.slug)
+    return RedirectResponse(
+        f"/admin/images?saved=Saved {row.name}.", status_code=status.HTTP_303_SEE_OTHER
+    )

src/lembas/api/admin_models.py

@@ -44,6 +44,7 @@ TOOL_CAPABILITIES = (
     ("tool_custom", "Custom tools"),
     ("tool_mcp", "MCP servers"),
     ("tool_ask", "Ask the reader"),
+    ("tool_image", "Image generation"),
     ("tool_agent", "Agent execution"),
 )
 

src/lembas/api/chats.py

@@ -67,6 +67,13 @@ METRICS_INTERVAL = 1.0
 # better than four hundred rows nobody meant to write.
 MAX_QUEUED = 10
 
+# Which tools a request may compel the model to call. An allow list rather than
+# a passthrough: this becomes `tool_choice`, and a name read straight off a form
+# would let anyone who can send a message decide what the model must do next.
+# Being on this list is not permission to *use* the tool -- `resolve_tools` still
+# decides that, and forcing one that was never offered simply does nothing.
+FORCEABLE_TOOLS = frozenset({"image_generate"})
+
 # How many things one chat may have switched off. There are a dozen families and
 # sixty skills at most, so this is not a limit anybody reaches by hand -- it is
 # there so a crafted POST cannot grow the column without bound.
@@ -816,12 +823,20 @@ async def post_message(
     chat_id: str,
     content: str = Form(""),
     file_ids: list[str] = Form(default=[]),
+    force_tool: str = Form(""),
 ) -> Response:
     """Persist the user's turn and hand back the pair of bubbles.
 
     The assistant bubble comes back empty, carrying the sse-connect attribute
     that opens the stream below. Splitting it this way means the POST returns
     immediately and the slow part is a separate, resumable connection.
+
+    `force_tool` is `/image` and nothing else. It is checked against a fixed
+    list rather than passed through: this ends up in `tool_choice`, and a name
+    taken from a form would let anybody who can send a message pick which tool
+    the model is compelled to call. Whether that tool is *offered* is still
+    decided by `resolve_tools`, so this can only ever narrow to something the
+    chat was already allowed.
     """
     chat = _owned_chat(db, chat_id, user.id)
 
@@ -831,7 +846,8 @@ async def post_message(
     if not content and not file_ids:
         return Response(status_code=status.HTTP_204_NO_CONTENT)
 
-    return _send(request, db, chat, user, content, file_ids=file_ids)
+    forced = force_tool.strip() if force_tool.strip() in FORCEABLE_TOOLS else ""
+    return _send(request, db, chat, user, content, file_ids=file_ids, force_tool=forced)
 
 
 def _reply_in_flight(db: DBSession, chat: Chat) -> bool:
@@ -871,6 +887,7 @@ def _send(
     content: str,
     *,
     file_ids: list[str] | None = None,
+    force_tool: str = "",
 ) -> Response:
     """Write a turn, start the reply, and hand back the pair of bubbles.
 
@@ -927,7 +944,7 @@ def _send(
     assistant_message = chat_service.create_message(
         db, chat, ROLE_ASSISTANT, "", complete_=False, model_id=chat.model_id
     )
-    generation_service.ensure(chat.id, assistant_message.id)
+    generation_service.ensure(chat.id, assistant_message.id, force_tool=force_tool)
 
     # `user` is required by the shared message template, which renders both
     # roles; without it the user bubble's initial blows up.