Ideogram 4.0 Day-0 Support in ComfyUI: Open Weights and Structured Control

A 9.3B open-weights text-to-image model that reads JSON prompts like a spec sheet.

Big day for the open-source community! Ideogram just dropped their first open-weights foundation model, Ideogram 4.0, and it’s natively supported in ComfyUI on day zero.

It is a 9.3B parameter text-to-image model trained from scratch. If you have been waiting for an open model that can handle posters, layouts, and readable copy without sending prompts to a closed API, this is the one to try.

---

Precise control with structured JSON prompting

This is where Ideogram 4.0 earns its keep. A plain-text prompt gets you an image; the JSON surface gives you control. Three things you can do that a flat prompt can’t:

• Color palettes: Up to 16 hex colors per image (5 per element).

• Bounding-box layout: [y_min, x_min, y_max, x_max] in 0–1000 normalized coordinates.

• Typed text elements: Literal string to render, plus a separate styling description.

Pro tip: the more relationships you pin down in the caption (positions, colors, per-element styling) the more grounded the result. The model was trained on exhaustively described scenes, so it rewards specificity.

JSON Prompt Structure Template:

{
  "high_level_description": "",
  "style_description": {
    "aesthetics": "",
    "lighting": "",
    "photo": "",
    "medium": "",
    "color_palette": []  # hexadecimal values
  },
  "compositional_deconstruction": {
    "background": "",
    "elements": [
      {
        "type": "obj",  # for object
        "bbox": [],  # y_min, x_min, y_max, x_max
        "desc": "",
        "color_palette": []  # hexadecimal values
      },
      {
        "type": "text",  # for text
        "text": "",  # the text to render
        "bbox": [],  # y_min, x_min, y_max, x_max
        "desc": "",
        "color_palette": []  # hexadecimal values
      }
    ]
  }
}

---

Examples

---

Safety Filtering

Ideogram 4.0 ships with its own built-in safety filter that lives inside the model itself, this is not a ComfyUI filter. If a generation gets blocked or returns a blank/refused result, that’s the model’s own filter engaging, not anything ComfyUI is adding or controlling. We can’t tune, disable, or override it from our side, since it’s baked into the model’s weights as released by Ideogram. If you hit a block you didn’t expect, try rephrasing your prompt or adjusting the offending element rather than filing it as a ComfyUI bug.

---

Getting Started

1. Update ComfyUI to the latest version 0.24.0 (Cloud and Desktop will follow soon)

2. Download the workflow below, or find it in the template library

3. Follow the note in the workflow to download models and save them in the correct model directory

4. Use a natural language prompt or write your prompt in JSON format, then run the workflow

Model weights: 🤗 Comfy-Org/Ideogram-4
GitHub repository: ideogram-oss/ideogram4

As always, enjoy creating!