Jaroslav Beneš
60e7d0d599
Two problems, both found by looking rather than by guessing. ComfyUI writes its history entry in task_done and nowhere else, so the entry appearing IS "finished" -- but it sets completed=e.success, which means an out-of-memory, a cancelled job and a broken node all stay completed:false for ever. await_images waited on that flag. So every failure sat for the full 600s timeout and then reported a timeout, when ComfyUI had known within one second and written down the node, the exception type and the message. Proved by causing both against the real instance: an OOM now raises in 1.0s and an interrupt in 4.0s, each naming the node. The terminal condition is a record with a status, and status.messages is read for the last execution_error or execution_interrupted. OutOfMemory and Interrupted are their own classes because they are the two failures with an obvious next move: the first tells the model to retry at a named smaller size -- worked out from what it actually asked for, since "use a lower resolution" against a request that was already 512x512 is advice nobody can follow -- or with a lighter checkpoint; the second says somebody pressed stop, so do not simply start again. Everything else gets the reason and no advice, because a model told to try again after a broken workflow tries the identical thing. The OOM message is cut to its first sentence. The rest is allocator advice -- PYTORCH_CUDA_ALLOC_CONF, fragmentation notes -- addressed to whoever runs the box and meaningless to a model, in a tool result that is already a failure. Second: the parameters were described in the register of a reference table, and "cfg: prompt adherence, default 8" tells a model nothing it can act on. Measured on a 4B model, same request, same everything else: with the old wording it sent prompt and template and nothing more -- so 512x512 on an SDXL checkpoint, which is exactly the duplicated-limbs failure the width description now warns about. With descriptions that say what each value does to the picture and when to move it, the same model sent a portrait 1024x1536 and a deliberate sampler. ~3KB of schema per request in a chat that can draw, and the difference between having ten parameters and having one. docs/image-generation-instructions.md is the long version for the admin instructions box, for models that need more than the harness can afford to carry on every request in every chat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
141 lines
5.6 KiB
Markdown
141 lines
5.6 KiB
Markdown
# Extra instructions for image generation
|
||
|
||
Paste the block below into **Admin › Image generation › Extra instructions**.
|
||
It reaches every model on the instance, above whatever each chat's own system
|
||
prompt says, and it appears only when the image tool is actually offered.
|
||
|
||
It is longer than the built-in guidance on purpose. The built-in fragment has to
|
||
suit every instance and is kept short because it costs tokens on every request
|
||
in every chat that can draw; this is yours to make as long as your models need.
|
||
**Small models need more of it.** A 4B model left to itself passes the request
|
||
through verbatim — "draw me a cat" becomes the prompt "draw me a cat" — and
|
||
leaves ten parameters at their defaults for ever. Most of what follows exists to
|
||
stop that.
|
||
|
||
Trim it if your models are large enough not to need it: every line of it is sent
|
||
on every request in every chat where image generation is on.
|
||
|
||
Two things it deliberately does **not** cover, because LLeMbas already tells the
|
||
model and repeating them wastes the window:
|
||
|
||
- the parameter ranges and defaults — those are in the tool's own schema
|
||
- that the picture is already on screen — that is in the built-in fragment
|
||
|
||
---
|
||
|
||
```text
|
||
WRITING THE PROMPT
|
||
|
||
Never send the request as the prompt. "a cat" is a request; the prompt is what
|
||
you write from it. Expand it into a description, in this order:
|
||
|
||
subject, what it is doing, setting, lighting, composition, style and medium
|
||
|
||
Comma-separated phrases, not a sentence. Concrete nouns and adjectives. Twenty
|
||
to sixty words is the useful range: below that the model invents everything you
|
||
left out, and much above it the later words stop having any effect.
|
||
|
||
weak: a cat
|
||
better: a ginger tabby cat asleep on a windowsill, curled up, potted herbs
|
||
beside it, low afternoon sun through old glass, warm rim light,
|
||
shallow depth of field, 50mm photograph
|
||
|
||
Say the medium explicitly — photograph, oil painting, pencil sketch, 3D render,
|
||
watercolour, screen print. Without it you get an averaged, plasticky look that
|
||
belongs to no medium at all.
|
||
|
||
For a photograph, naming a lens and light does most of the work: 35mm, 85mm
|
||
portrait, golden hour, overcast, backlit, studio softbox.
|
||
For an illustration, name the tradition rather than a living artist: art
|
||
nouveau, ukiyo-e, mid-century children's book, technical cutaway diagram.
|
||
|
||
Do not write instructions in the prompt. "make sure there are exactly two
|
||
people" is not understood. Describe the result: "two people".
|
||
|
||
NEGATIVE PROMPTS
|
||
|
||
Plain nouns and adjectives for things that must not appear:
|
||
"blurry, low quality, extra fingers, deformed hands, text, watermark, signature".
|
||
|
||
Never phrase it as an instruction. "no text" contains the word text and puts
|
||
text in the picture. The negative prompt is a list of things to avoid, not a
|
||
sentence to obey.
|
||
|
||
Add "extra fingers, deformed hands" whenever hands are visible, and
|
||
"extra limbs, fused bodies" for more than one person.
|
||
|
||
SIZE
|
||
|
||
Choose the aspect ratio for the subject, then keep the total near what the
|
||
checkpoint expects.
|
||
|
||
portrait of a person 512x768 (or 832x1216 on an SDXL checkpoint)
|
||
landscape or interior 768x512 (or 1216x832)
|
||
square, product, icon 512x512 (or 1024x1024)
|
||
|
||
Going far above what a checkpoint was trained for does not add detail: it adds
|
||
second heads, extra limbs and repeated horizons. If you want more detail, add
|
||
detail to the prompt.
|
||
|
||
CHOOSING A CHECKPOINT AND A TEMPLATE
|
||
|
||
Read the descriptions you were given and pick by what the picture needs. When
|
||
nothing obviously fits, leave both out — the chat's usual ones are used, and a
|
||
wrong guess costs a whole generation.
|
||
|
||
WHEN TO CHANGE THE OTHER PARAMETERS
|
||
|
||
drafting, or making several to compare steps 10-12
|
||
the result looks harsh or over-saturated cfg 4-6
|
||
the subject is being ignored cfg 9-11, and simplify the prompt
|
||
fine texture matters steps 35-45, sampler dpmpp_2m,
|
||
scheduler karras
|
||
|
||
Otherwise leave them alone. Changing three at once teaches you nothing about
|
||
which one helped.
|
||
|
||
CHANGING A PICTURE YOU HAVE ALREADY MADE
|
||
|
||
You are told the seed of every image you generate. To change one thing and keep
|
||
the rest, send the same seed with an edited prompt. To get something completely
|
||
different, omit the seed or send -1.
|
||
|
||
Note that you cannot see a picture again on a later turn, so decide what to
|
||
change from what you wrote, not from what you remember seeing.
|
||
|
||
WHEN IT FAILS
|
||
|
||
Out of video memory: generate again at about half the width and height, or with
|
||
a lighter checkpoint. Do not resend the same request — it will fail the same
|
||
way.
|
||
|
||
Cancelled: somebody stopped it deliberately. Say so and ask before starting
|
||
another.
|
||
|
||
Anything else: say what failed and what you were trying to draw. Do not retry
|
||
the identical request more than once.
|
||
|
||
AFTERWARDS
|
||
|
||
The picture is already in the conversation. Say in one or two lines what you
|
||
made and what you would change — the checkpoint, the size and the seed are
|
||
shown, so do not repeat them.
|
||
```
|
||
|
||
---
|
||
|
||
## A shorter version
|
||
|
||
For a large model, or an instance where the window is tight:
|
||
|
||
```text
|
||
Write the prompt as a description, never as the request you were given:
|
||
subject, action, setting, lighting, style and medium, comma-separated,
|
||
twenty to sixty words. Always name the medium. Use the negative prompt for
|
||
things to avoid, as plain nouns ("blurry, extra fingers, text") and never as
|
||
an instruction. Choose the aspect ratio for the subject — taller for a
|
||
person, wider for a place — and keep the total near what the checkpoint
|
||
expects. Change the other parameters only for a reason. If it runs out of
|
||
video memory, retry once at half the size or with a lighter checkpoint.
|
||
```
|