Cozy Clothes Swap - ComfyUI Node For Fashion Try-on With StableVITON

A ComfyUI node to dress-up your friends or your characters. Uses a generative/diffusion-based approach. Works on CUDA. Linux-only, as it needs triton.

Made with 💚 by the CozyMantis squad.

Installation

• Copy all the node files into the custom_nodes/cozy-stableviton directory, then run pip install -r requirements.txt to install the required dependencies.
• Download VITONHD.ckpt from https://kaistackr-my.sharepoint.com/personal/rlawjdghek_kaist_ac_kr/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Frlawjdghek%5Fkaist%5Fac%5Fkr%2FDocuments%2FStableVITON&ga=1 and place it in models/stableviton

Usage

A demo ComfyUI workflow is included with your download.

Inputs you need to provide:

• A product image of the cloth to apply, against a white background, in the targeted aspect ratio.
• A picture of a model to apply the cloth to, against a white background, in the targeted aspect ratio.
• A mask for defining the inpaiting area in the model picture - where the model should apply the cloth. You can manually draw the masks, or you can use a segmenter node like the Cozy Human Parser (https://cozymantis.gumroad.com/l/cozy-human-parser-comfyui-node-body-clothes-masks)
• A densepose representation of the model picture. Use the DensePose Estimator node that comes with ComfyUI's ControlNet Auxiliary Preprocessors (https://github.com/Fannovel16/comfyui_controlnet_aux), and use the densepose_rn50_fpn_dl.torchscript model with the Parula cmap.
• To get the best results, experiment with growing/blurring the provided mask, and using different numbers of steps.

Notes

• The model has been trained on the VITON-HD dataset, so it works best with upper-body clothes, but it can also apply pants or dresses.
• The size of the output is 384 x 512. You may want to upscale the result.
• You will generally want "Repaint" enabled, as it greatly helps maintain regions unrelated to clothing.

Alternatives

• For a warping-based alternative that offers more consistency with the original cloth, try our SAL-VTON node: https://cozymantis.gumroad.com/l/cozy-clothes-swap-comfyui-node-salvton
  • This produces outputs that are more faithful to the original
  • You can manually adjust the fit by updating a series of landmarks using the provided editor
  • The output resolution is bigger
• You can also use both nodes to get multiple renderings that you can then combine/enhance for best results.

Important

• Make sure you own the rights to the images you use with this node. Do not use images that you do not have permission to use.
• Make sure to check the license for the original model: https://github.com/rlawjdghek/StableVITON

Acknowledgements

Based on the excellent paper "StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On" by Kim, Jeongho and Gu, Gyojung and Park, Minho and Park, Sunghyun and Choo, Jaegul. https://arxiv.org/pdf/2312.01725.pdf