Say what actually failed, and tell the model how to use the thing

Two problems, both found by looking rather than by guessing.

ComfyUI writes its history entry in task_done and nowhere else, so the entry
appearing IS "finished" -- but it sets completed=e.success, which means an
out-of-memory, a cancelled job and a broken node all stay completed:false for
ever. await_images waited on that flag. So every failure sat for the full 600s
timeout and then reported a timeout, when ComfyUI had known within one second and
written down the node, the exception type and the message. Proved by causing both
against the real instance: an OOM now raises in 1.0s and an interrupt in 4.0s,
each naming the node.

The terminal condition is a record with a status, and status.messages is read for
the last execution_error or execution_interrupted. OutOfMemory and Interrupted
are their own classes because they are the two failures with an obvious next
move: the first tells the model to retry at a named smaller size -- worked out
from what it actually asked for, since "use a lower resolution" against a request
that was already 512x512 is advice nobody can follow -- or with a lighter
checkpoint; the second says somebody pressed stop, so do not simply start again.
Everything else gets the reason and no advice, because a model told to try again
after a broken workflow tries the identical thing.

The OOM message is cut to its first sentence. The rest is allocator advice --
PYTORCH_CUDA_ALLOC_CONF, fragmentation notes -- addressed to whoever runs the box
and meaningless to a model, in a tool result that is already a failure.

Second: the parameters were described in the register of a reference table, and
"cfg: prompt adherence, default 8" tells a model nothing it can act on. Measured
on a 4B model, same request, same everything else: with the old wording it sent
prompt and template and nothing more -- so 512x512 on an SDXL checkpoint, which
is exactly the duplicated-limbs failure the width description now warns about.
With descriptions that say what each value does to the picture and when to move
it, the same model sent a portrait 1024x1536 and a deliberate sampler. ~3KB of
schema per request in a chat that can draw, and the difference between having ten
parameters and having one.

docs/image-generation-instructions.md is the long version for the admin
instructions box, for models that need more than the harness can afford to carry
on every request in every chat.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jaroslav Beneš
2026-08-05 14:55:18 +02:00
parent b2a05e0351
commit 178742501d
6 changed files with 447 additions and 40 deletions
+31 -14
View File
@@ -1025,21 +1025,38 @@ BUILTIN: tuple[Fragment, ...] = (
group=GROUP_TOOLS,
order=243,
families=("image",),
hint="Appears when image generation is offered. The sentence about the "
"picture already being on screen is the one that earns its place: "
"without it the commonest thing a model does next is offer to show you "
"the image, which it has no way of doing and which has already "
"happened.",
hint="Appears when image generation is offered. Two sentences here earn "
"their place against the tool's own descriptions. The picture already "
"being on screen, because without it the commonest thing a model does "
"next is offer to show you the image which it cannot do and which has "
"already happened. And the shape of a prompt: a small model left to "
"itself passes the request through verbatim, which is why so many "
"generations look like nobody thought about them.",
default=(
"- You can draw a picture with image_generate. Describe what you want in "
"the prompt as fully as you can — subject, setting, lighting, style — "
"because the prompt is the whole of what the picture is made from.\n"
"- The picture appears in the conversation as soon as the tool returns. "
"It is already on screen: do not offer to show it, link to it or "
"describe how to open it.\n"
"- Only the prompt is required. Everything else has a sensible default, "
"so set a parameter when you have a reason to and leave it out "
"otherwise. Repeat a seed to get the same picture again."
"- You can draw a picture with image_generate. Only `prompt` is required.\n"
"- Write the prompt as a description, not as the request you were given. "
"Comma-separated phrases work better than a sentence, and the order matters "
"— subject first, then what it is doing, then the setting, then the light, "
"then the style and medium. \"a red bicycle\" is a worse prompt than \"a red "
"bicycle leaning on a whitewashed wall, morning light, long shadows, 35mm "
"photograph, shallow depth of field\". Expand what you were asked for into "
"one of these; do not ask the person to write it for you.\n"
"- Use `negative` for what must not appear, as plain nouns: \"blurry, extra "
"fingers, text, watermark\". Never phrase it as an instruction — \"no text\" "
"puts text in the picture.\n"
"- Set `width` and `height` to suit the subject rather than leaving both at "
"the default: taller than wide for a person, wider than tall for a place. "
"Match the size the checkpoint expects; far above it produces duplicated "
"limbs rather than more detail.\n"
"- The other parameters have sensible defaults. Change one when you have a "
"reason — fewer steps for a quick draft, lower cfg when a picture looks "
"harsh — and leave it out otherwise.\n"
"- The picture appears in the conversation as soon as the tool returns. It "
"is already on screen: do not offer to show it, link to it, or describe how "
"to open it. Say what you made and what you would change.\n"
"- If it fails because the machine ran out of video memory, try once more at "
"a smaller size or with a lighter checkpoint. Do not repeat the same request "
"unchanged."
),
),
Fragment(