Say what actually failed, and tell the model how to use the thing
Two problems, both found by looking rather than by guessing. ComfyUI writes its history entry in task_done and nowhere else, so the entry appearing IS "finished" -- but it sets completed=e.success, which means an out-of-memory, a cancelled job and a broken node all stay completed:false for ever. await_images waited on that flag. So every failure sat for the full 600s timeout and then reported a timeout, when ComfyUI had known within one second and written down the node, the exception type and the message. Proved by causing both against the real instance: an OOM now raises in 1.0s and an interrupt in 4.0s, each naming the node. The terminal condition is a record with a status, and status.messages is read for the last execution_error or execution_interrupted. OutOfMemory and Interrupted are their own classes because they are the two failures with an obvious next move: the first tells the model to retry at a named smaller size -- worked out from what it actually asked for, since "use a lower resolution" against a request that was already 512x512 is advice nobody can follow -- or with a lighter checkpoint; the second says somebody pressed stop, so do not simply start again. Everything else gets the reason and no advice, because a model told to try again after a broken workflow tries the identical thing. The OOM message is cut to its first sentence. The rest is allocator advice -- PYTORCH_CUDA_ALLOC_CONF, fragmentation notes -- addressed to whoever runs the box and meaningless to a model, in a tool result that is already a failure. Second: the parameters were described in the register of a reference table, and "cfg: prompt adherence, default 8" tells a model nothing it can act on. Measured on a 4B model, same request, same everything else: with the old wording it sent prompt and template and nothing more -- so 512x512 on an SDXL checkpoint, which is exactly the duplicated-limbs failure the width description now warns about. With descriptions that say what each value does to the picture and when to move it, the same model sent a portrait 1024x1536 and a deliberate sampler. ~3KB of schema per request in a chat that can draw, and the difference between having ten parameters and having one. docs/image-generation-instructions.md is the long version for the admin instructions box, for models that need more than the harness can afford to carry on every request in every chat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,140 @@
|
||||
# Extra instructions for image generation
|
||||
|
||||
Paste the block below into **Admin › Image generation › Extra instructions**.
|
||||
It reaches every model on the instance, above whatever each chat's own system
|
||||
prompt says, and it appears only when the image tool is actually offered.
|
||||
|
||||
It is longer than the built-in guidance on purpose. The built-in fragment has to
|
||||
suit every instance and is kept short because it costs tokens on every request
|
||||
in every chat that can draw; this is yours to make as long as your models need.
|
||||
**Small models need more of it.** A 4B model left to itself passes the request
|
||||
through verbatim — "draw me a cat" becomes the prompt "draw me a cat" — and
|
||||
leaves ten parameters at their defaults for ever. Most of what follows exists to
|
||||
stop that.
|
||||
|
||||
Trim it if your models are large enough not to need it: every line of it is sent
|
||||
on every request in every chat where image generation is on.
|
||||
|
||||
Two things it deliberately does **not** cover, because LLeMbas already tells the
|
||||
model and repeating them wastes the window:
|
||||
|
||||
- the parameter ranges and defaults — those are in the tool's own schema
|
||||
- that the picture is already on screen — that is in the built-in fragment
|
||||
|
||||
---
|
||||
|
||||
```text
|
||||
WRITING THE PROMPT
|
||||
|
||||
Never send the request as the prompt. "a cat" is a request; the prompt is what
|
||||
you write from it. Expand it into a description, in this order:
|
||||
|
||||
subject, what it is doing, setting, lighting, composition, style and medium
|
||||
|
||||
Comma-separated phrases, not a sentence. Concrete nouns and adjectives. Twenty
|
||||
to sixty words is the useful range: below that the model invents everything you
|
||||
left out, and much above it the later words stop having any effect.
|
||||
|
||||
weak: a cat
|
||||
better: a ginger tabby cat asleep on a windowsill, curled up, potted herbs
|
||||
beside it, low afternoon sun through old glass, warm rim light,
|
||||
shallow depth of field, 50mm photograph
|
||||
|
||||
Say the medium explicitly — photograph, oil painting, pencil sketch, 3D render,
|
||||
watercolour, screen print. Without it you get an averaged, plasticky look that
|
||||
belongs to no medium at all.
|
||||
|
||||
For a photograph, naming a lens and light does most of the work: 35mm, 85mm
|
||||
portrait, golden hour, overcast, backlit, studio softbox.
|
||||
For an illustration, name the tradition rather than a living artist: art
|
||||
nouveau, ukiyo-e, mid-century children's book, technical cutaway diagram.
|
||||
|
||||
Do not write instructions in the prompt. "make sure there are exactly two
|
||||
people" is not understood. Describe the result: "two people".
|
||||
|
||||
NEGATIVE PROMPTS
|
||||
|
||||
Plain nouns and adjectives for things that must not appear:
|
||||
"blurry, low quality, extra fingers, deformed hands, text, watermark, signature".
|
||||
|
||||
Never phrase it as an instruction. "no text" contains the word text and puts
|
||||
text in the picture. The negative prompt is a list of things to avoid, not a
|
||||
sentence to obey.
|
||||
|
||||
Add "extra fingers, deformed hands" whenever hands are visible, and
|
||||
"extra limbs, fused bodies" for more than one person.
|
||||
|
||||
SIZE
|
||||
|
||||
Choose the aspect ratio for the subject, then keep the total near what the
|
||||
checkpoint expects.
|
||||
|
||||
portrait of a person 512x768 (or 832x1216 on an SDXL checkpoint)
|
||||
landscape or interior 768x512 (or 1216x832)
|
||||
square, product, icon 512x512 (or 1024x1024)
|
||||
|
||||
Going far above what a checkpoint was trained for does not add detail: it adds
|
||||
second heads, extra limbs and repeated horizons. If you want more detail, add
|
||||
detail to the prompt.
|
||||
|
||||
CHOOSING A CHECKPOINT AND A TEMPLATE
|
||||
|
||||
Read the descriptions you were given and pick by what the picture needs. When
|
||||
nothing obviously fits, leave both out — the chat's usual ones are used, and a
|
||||
wrong guess costs a whole generation.
|
||||
|
||||
WHEN TO CHANGE THE OTHER PARAMETERS
|
||||
|
||||
drafting, or making several to compare steps 10-12
|
||||
the result looks harsh or over-saturated cfg 4-6
|
||||
the subject is being ignored cfg 9-11, and simplify the prompt
|
||||
fine texture matters steps 35-45, sampler dpmpp_2m,
|
||||
scheduler karras
|
||||
|
||||
Otherwise leave them alone. Changing three at once teaches you nothing about
|
||||
which one helped.
|
||||
|
||||
CHANGING A PICTURE YOU HAVE ALREADY MADE
|
||||
|
||||
You are told the seed of every image you generate. To change one thing and keep
|
||||
the rest, send the same seed with an edited prompt. To get something completely
|
||||
different, omit the seed or send -1.
|
||||
|
||||
Note that you cannot see a picture again on a later turn, so decide what to
|
||||
change from what you wrote, not from what you remember seeing.
|
||||
|
||||
WHEN IT FAILS
|
||||
|
||||
Out of video memory: generate again at about half the width and height, or with
|
||||
a lighter checkpoint. Do not resend the same request — it will fail the same
|
||||
way.
|
||||
|
||||
Cancelled: somebody stopped it deliberately. Say so and ask before starting
|
||||
another.
|
||||
|
||||
Anything else: say what failed and what you were trying to draw. Do not retry
|
||||
the identical request more than once.
|
||||
|
||||
AFTERWARDS
|
||||
|
||||
The picture is already in the conversation. Say in one or two lines what you
|
||||
made and what you would change — the checkpoint, the size and the seed are
|
||||
shown, so do not repeat them.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## A shorter version
|
||||
|
||||
For a large model, or an instance where the window is tight:
|
||||
|
||||
```text
|
||||
Write the prompt as a description, never as the request you were given:
|
||||
subject, action, setting, lighting, style and medium, comma-separated,
|
||||
twenty to sixty words. Always name the medium. Use the negative prompt for
|
||||
things to avoid, as plain nouns ("blurry, extra fingers, text") and never as
|
||||
an instruction. Choose the aspect ratio for the subject — taller for a
|
||||
person, wider for a place — and keep the total near what the checkpoint
|
||||
expects. Change the other parameters only for a reason. If it runs out of
|
||||
video memory, retry once at half the size or with a lighter checkpoint.
|
||||
```
|
||||
Reference in New Issue
Block a user