Say what actually failed, and tell the model how to use the thing
Two problems, both found by looking rather than by guessing. ComfyUI writes its history entry in task_done and nowhere else, so the entry appearing IS "finished" -- but it sets completed=e.success, which means an out-of-memory, a cancelled job and a broken node all stay completed:false for ever. await_images waited on that flag. So every failure sat for the full 600s timeout and then reported a timeout, when ComfyUI had known within one second and written down the node, the exception type and the message. Proved by causing both against the real instance: an OOM now raises in 1.0s and an interrupt in 4.0s, each naming the node. The terminal condition is a record with a status, and status.messages is read for the last execution_error or execution_interrupted. OutOfMemory and Interrupted are their own classes because they are the two failures with an obvious next move: the first tells the model to retry at a named smaller size -- worked out from what it actually asked for, since "use a lower resolution" against a request that was already 512x512 is advice nobody can follow -- or with a lighter checkpoint; the second says somebody pressed stop, so do not simply start again. Everything else gets the reason and no advice, because a model told to try again after a broken workflow tries the identical thing. The OOM message is cut to its first sentence. The rest is allocator advice -- PYTORCH_CUDA_ALLOC_CONF, fragmentation notes -- addressed to whoever runs the box and meaningless to a model, in a tool result that is already a failure. Second: the parameters were described in the register of a reference table, and "cfg: prompt adherence, default 8" tells a model nothing it can act on. Measured on a 4B model, same request, same everything else: with the old wording it sent prompt and template and nothing more -- so 512x512 on an SDXL checkpoint, which is exactly the duplicated-limbs failure the width description now warns about. With descriptions that say what each value does to the picture and when to move it, the same model sent a portrait 1024x1536 and a deliberate sampler. ~3KB of schema per request in a chat that can draw, and the difference between having ten parameters and having one. docs/image-generation-instructions.md is the long version for the admin instructions box, for models that need more than the harness can afford to carry on every request in every chat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -61,9 +61,19 @@ SCHEMA: dict[str, Any] = {
|
||||
"type": "string",
|
||||
"description": "What to draw. Describe the subject, the setting and the style.",
|
||||
},
|
||||
# Every description below says what the value *does to the picture* and
|
||||
# when to move it, not what it is called. A model that is told "cfg:
|
||||
# prompt adherence, default 8" has been told nothing it can act on, and
|
||||
# the observable result is a model that sends the prompt alone and
|
||||
# leaves ten parameters at their defaults for ever.
|
||||
"negative": {
|
||||
"type": "string",
|
||||
"description": "What to keep out of the picture. Defaults to 'text, watermark'.",
|
||||
"description": (
|
||||
"Comma-separated things to keep OUT of the picture, as plain nouns and "
|
||||
"adjectives: 'blurry, extra fingers, text, watermark'. Not a sentence, "
|
||||
"and never phrased as an instruction — 'do not add text' puts *text* in "
|
||||
"the picture. Defaults to 'text, watermark'."
|
||||
),
|
||||
},
|
||||
"template": {
|
||||
"type": "string",
|
||||
@@ -71,22 +81,80 @@ SCHEMA: dict[str, Any] = {
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"description": "Which checkpoint to draw with. Omit to use this chat's usual one.",
|
||||
"description": (
|
||||
"Which checkpoint to draw with. Pick by what it is good at; omit to use "
|
||||
"this chat's usual one."
|
||||
),
|
||||
},
|
||||
"seed": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"Omit it, or pass -1, for a new random image. Repeat a seed you were "
|
||||
"told about to get the same image again."
|
||||
"told about to get that same image again — which is how you change one "
|
||||
"thing about a picture and keep the rest."
|
||||
),
|
||||
},
|
||||
"steps": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"How long to refine, 1-150. Default 20. Around 20-30 for most things; "
|
||||
"8-12 for a quick draft or when several are wanted; 40+ only for fine "
|
||||
"detail, and past about 50 it stops improving and only costs time."
|
||||
),
|
||||
},
|
||||
"cfg": {
|
||||
"type": "number",
|
||||
"description": (
|
||||
"How literally to follow the prompt, 0-30. Default 8. 3-6 gives the "
|
||||
"model room and looks more natural; 7-9 is the usual range; 12+ forces "
|
||||
"the words through and starts to look burnt and over-saturated. Lower "
|
||||
"it if the picture looks harsh, raise it if the subject is being "
|
||||
"ignored."
|
||||
),
|
||||
},
|
||||
"width": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"Pixels, 64-2048, a multiple of 8. Default 512. Use the size the "
|
||||
"checkpoint was trained for — about 512 for SD1.5, about 1024 for SDXL "
|
||||
"— and change the ratio rather than the total: 512x768 for a portrait, "
|
||||
"768x512 for a landscape. Going far above what the checkpoint expects "
|
||||
"produces duplicated limbs and repeated horizons, not more detail."
|
||||
),
|
||||
},
|
||||
"height": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"Pixels, 64-2048, a multiple of 8. Default 512. See width: the aspect "
|
||||
"ratio is the thing to choose, and taller than wide suits a person, "
|
||||
"wider than tall suits a place."
|
||||
),
|
||||
},
|
||||
"sampler": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"How the image is solved. Default euler. 'euler' is safe and fast; "
|
||||
"'dpmpp_2m' is a good general improvement; 'dpmpp_2m_sde' for more "
|
||||
"texture; 'ddim' for a clean flat look. Leave it out unless you have a "
|
||||
"reason."
|
||||
),
|
||||
},
|
||||
"scheduler": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"How the steps are spaced. Default normal. 'karras' pairs well with the "
|
||||
"dpmpp samplers and usually helps at low step counts; 'normal' "
|
||||
"otherwise. Leave it out unless you are also setting the sampler."
|
||||
),
|
||||
},
|
||||
"denoise": {
|
||||
"type": "number",
|
||||
"description": (
|
||||
"How much of the starting noise to replace, 0-1. Default 1, which is "
|
||||
"what you want for a picture drawn from nothing. Lower values only mean "
|
||||
"something for a workflow that starts from an existing image."
|
||||
),
|
||||
},
|
||||
"steps": {"type": "integer", "description": "Sampling steps. Default 20."},
|
||||
"cfg": {"type": "number", "description": "Prompt adherence. Default 8."},
|
||||
"width": {"type": "integer", "description": "Pixels. Default 512."},
|
||||
"height": {"type": "integer", "description": "Pixels. Default 512."},
|
||||
"sampler": {"type": "string", "description": "Sampler name. Default euler."},
|
||||
"scheduler": {"type": "string", "description": "Scheduler name. Default normal."},
|
||||
"denoise": {"type": "number", "description": "0 to 1. Default 1."},
|
||||
},
|
||||
"required": ["prompt"],
|
||||
}
|
||||
@@ -371,6 +439,9 @@ async def run(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
|
||||
|
||||
attempts: list[Attempt] = []
|
||||
kept: tuple[bytes, dict[str, Any]] | None = None
|
||||
# What the last attempt actually asked for, so a failure can name concrete
|
||||
# numbers back at the model rather than saying "try something smaller".
|
||||
params_used: dict[str, Any] = workflow.resolve(given)
|
||||
|
||||
try:
|
||||
for number in range(1, tries + 1):
|
||||
@@ -378,6 +449,7 @@ async def run(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
|
||||
await _unload_llm(context)
|
||||
|
||||
params = workflow.resolve({**given, "seed": args.get("seed") if number == 1 else None})
|
||||
params_used = params
|
||||
refs = await comfy.await_images(
|
||||
config, await comfy.submit(config, workflow.fill(template, params))
|
||||
)
|
||||
@@ -402,9 +474,12 @@ async def run(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
|
||||
break
|
||||
except comfy.ComfyError as exc:
|
||||
if preserve:
|
||||
# It failed *inside* the far side, so its models are still resident
|
||||
# and the language model is still unloaded. Freeing here is what
|
||||
# lets the reply carry on and say what happened.
|
||||
await comfy.free(config)
|
||||
return ToolOutcome(
|
||||
f"The image could not be generated: {exc.message}",
|
||||
f"The image could not be generated: {exc.message}{_advice(exc, params_used)}",
|
||||
{**event, "status": "error", "error": exc.message},
|
||||
)
|
||||
|
||||
@@ -454,6 +529,34 @@ async def run(context: ToolContext, args: dict[str, Any]) -> ToolOutcome:
|
||||
)
|
||||
|
||||
|
||||
def _advice(exc: comfy.ComfyError, params: dict[str, Any]) -> str:
|
||||
"""What to do about a failure, when there is something to do about it.
|
||||
|
||||
Only for the two that have an obvious next move. Everything else gets the
|
||||
reason and nothing else -- a model told to "try again" after a broken
|
||||
workflow will try the identical thing, and a suggestion invented for a
|
||||
failure nobody understands is a guess wearing the application's authority.
|
||||
|
||||
The numbers are concrete on purpose. "Use a lower resolution" against a
|
||||
request that was already 512x512 is advice that cannot be followed, so the
|
||||
halved size is worked out here where the request is known.
|
||||
"""
|
||||
if isinstance(exc, comfy.Interrupted):
|
||||
return (
|
||||
" Somebody stopped it deliberately, so do not simply start it again — say so and ask."
|
||||
)
|
||||
if not isinstance(exc, comfy.OutOfMemory):
|
||||
return ""
|
||||
|
||||
width, height = int(params.get("width") or 512), int(params.get("height") or 512)
|
||||
smaller = f"{max(256, width // 2)}x{max(256, height // 2)}"
|
||||
return (
|
||||
f" Try once more at a smaller size — {smaller} instead of {width}x{height} — "
|
||||
"or with a lighter checkpoint if one is offered. Do not repeat the same "
|
||||
"request unchanged; it will run out of memory again."
|
||||
)
|
||||
|
||||
|
||||
def _pick(rows: list[Any], wanted: str, chat_default: str, values: dict[str, Any]) -> Any:
|
||||
"""The workflow to use: asked for, then the chat's, then the instance's."""
|
||||
by_slug = {row.slug: row for row in rows}
|
||||
|
||||
Reference in New Issue
Block a user