ControlNet in Depth
What is ControlNet?
ControlNet adds spatial control to image generation. Instead of letting the model freely decide composition, you provide a control map that dictates structure, pose, edges, or depth.
ControlNet Types
| Type | Use Case |
|---|---|
| Canny | Edge detection, reproduce structure |
| Depth | Depth map, preserve spatial layout |
| OpenPose | Skeleton/pose detection |
| Scribble/Sketch | Hand-drawn sketch to image |
| Lineart | Line art to colored illustration |
| SoftEdge | Soft edge detection, more creative freedom |
| Normal Map | Surface orientation for realistic lighting |
| Segmentation | Semantic zones control |
| Tile | Detail enhancement for upscaling |
ComfyUI Workflow
- Load ControlNet Model: loads the ControlNet model
- Preprocessor (Canny, Depth, OpenPose...): extracts the control map
- Apply ControlNet: injects control into conditioning
Key Parameters
- strength (0.0-2.0): control strength (0.7-0.8 recommended)
- start_percent / end_percent: when control starts/stops
Combining Multiple ControlNets
Chain multiple ControlNets for multi-dimensional control (e.g., OpenPose for pose + Depth for spatial layout). Reduce strength for each (0.5-0.7).
Practical Cases
- Reproduce a pose from a photo (OpenPose)
- Transform a sketch into an illustration (Scribble)
- Architectural style change (Depth + Canny)
- Character consistency with pose control (OpenPose + IP-Adapter)
Creating Custom Control Maps
You can manually create depth maps (grayscale), OpenPose skeletons, or segmentation maps for precise control.