New ControlNet Models for Stable Diffusion 3.5 Large!
Today, ComfyUI added support for new Stable Diffusion 3.5 Large ControlNet models by Stability AI: Blur, Canny, and Depth.
Each of the models is powered by 8 billion parameters, free for both commercial and non-commercial use under the permissive Stability AI Community License. These models bring new capabilities to help you generate detailed and customized images.
Quick Overview of the 3 ControlNet Models:
Blur: Achieve extremely high fidelity upscaling, including resolutions of 8K and 16K. Perfect for tiling low-resolution images into large, detailed visuals.
Canny: Leverage Canny edge maps to structure your generated images. This control type is particularly useful for illustrations but can be adaptable across all styles.
Depth: Use depth maps, generated by DepthFM, to guide image generation. Great for architectural renderings, texturing 3D assets, and other use cases that require exact control over the composition of an image.
Get Started with Stable Diffusion 3.5 in ComfyUI
If you’re new to Stable Diffusion 3.5, check out our previous blog post to get started:
ComfyUI Now Supports Stable Diffusion 3.5!
Try Out Each of the 3 ControlNet Models
Note: We use our all-in-one SD3.5 large checkpoint in all these workflows below. You can also use the triple-clip version method to load the checkpoint, as introduced in previous blogs.
1. Blur Workflow
Update ComfyUI to the Latest
Make sure the all-in-one SD3.5 large checkpoint is in your models\checkpoints folder.
Download
sd3.5_large_controlnet_blur.safetensorsand place it in yourmodels\controlnetfolder.Drag and drop the image below into ComfyUI to load the example workflow
2. Canny Workflow
Update ComfyUI to the Latest
Make sure the all-in-one SD3.5 large checkpoint is in your models\checkpoints folder.
Download
sd3.5_large_controlnet_canny.safetensorsand place it in yourmodels\controlnetfolder.Drag and drop the image below into ComfyUI to load the example workflow, or check out our **Example Workflow Page.**
3. Depth Workflow
Update ComfyUI to the Latest
Make sure the all-in-one SD3.5 large checkpoint is in your models\checkpoints folder.
Download
sd3.5_large_controlnet_depth.safetensorsand place it in yourmodels\controlnetfolder.Drag and drop the image below into ComfyUI to load the example workflow (one custom node for depth map processing is included in this workflow).
Additional ControlNet models, including Stable Diffusion 3.5 Medium (2B) variants and new control types, are on the way!
Enjoy experiments with different workflows and creative possibilities!
 |
|
None of these images work. They're all webp format. Your link to the workflows is similarly useless.
*** Hello Internet stranger! Please read before you give up on the workflow! This works great! ***
Jo Zhang, thank you for this excellent tutorial! I would recommend updating this post to include an explicit link to the workflow JSON or actual PNG link. Otherwise, you get the WEBP format.
As noted by other commenters, you MUST left-click the image, have it open in the preview pane, then click the floating "download" icon in the top left when the image preview is active. It looks like a half box with a down arrow. (Thank you to Kuhne for pointing this out, and for gg for bringing attention to it. I almost gave up!!)
Other than this minor confusion, very well-written guide and appreciate your contributions to the generative AI community. Be well. :)