Transforming Outfits with Roboflow Workflows and Generative AI

Roboflow Workflows lets you create complex computer vision pipelines by using a no-code visual blocks interface.

In this guide, we’ll walk through how to create a workflow that takes an input picture of a person, detects their clothing, and uses generative AI to change their outfit based on a prompt. The workflow combines YOLOv8 for object detection, SAM2 for segmentation, and Stability AI’s Stable Diffusion for generative inpainting.

Here is an example of the Workflow in action:

You can test the Workflow using your own images below:

Note: You will need to provide your own Stability AI key to run the demo.

You can fork this workflow to automatically copy it into your Roboflow account.

Project Overview

The Workflow begins by detecting clothing items like jackets and trousers in an image using YOLOv8 object detection, followed by filtering the detections to focus on specific classes of interest.

Next, SAM2 generates precise segmentation masks for the filtered items, and finally, generative AI is applied to modify the outfits based on custom prompts.

Transform Outfits with Roboflow Workflows

First, log in to your Roboflow account. Navigate to the Workflows tab on your dashboard. Then, click Create New Workflow and name it something like “transform-my-outfit”.

Add an object detection model to detect clothing

The first part of our Workflow is to set up a model that can detect items of clothing. We will then use the regions returned by our model as part of our prompt to Stability AI.

Here are the steps to follow to add the model:

1. Add the Object Detection block.
2. Search for a public model in the Roboflow Universe trained on clothing categories with labels like "clothes," "pants," and "jacket."
3. Configure the block to accept the image input.
4. Add a Bounding Box Visualization block and connect it to the output of the Object Detection block. This will display the bounding boxes for detected clothing items.
5. Add a Label Visualization block to show the class labels for each detected item. Connect this block to the Bounding Box Visualization block.

FIlter detections to show relevant classes

Next, we need to filter the detections from our pre-trained model to focus on only the items of clothing that we want to support.

1. Add the Detections Filter block to narrow down the classes of interest.
2. Connect the output of the Object Detection block to the input of the Detections Filter block.
3. Configure the filter block to include the following classes:

"sweater, Sweater, shirt, Shirt, pants, Pants, pant, Pant, clothes, hoodie, Hoodie, jacket, Jacket, hat, Hat, shoe, Shoe, shoes, Shoes, short, Short, shorts, Shorts, dress, Dress, tee, Tee"

SAM-2 Segmentation block

Next, we are going to use SAM-2 to create precise masks around items of clothing using the regions from our object detection model as a prompt.

These segmentations will then be used to prompt Stability to paint in new items of clothing later in our guide.

1. Add the Segmentation block.
2. Select Meta's SAM-2 (Segment Anything Model 2) as the model.
3. Connect the output of the Detections Filter block to the input of the Segmentation block.
4. Configure SAM2 to generate segmentation masks for each filtered detection.
5. Add a Polygon Visualization block to display the segmentation boundaries for detected clothing items. Connect it to the output of the Segmentation block.
6. Add a Mask Visualization block to overlay the segmented areas onto the original image.
7. Add a Label Visualization block to annotate the segmented regions with their corresponding labels.

Use Stability AI to in-paint new clothing

We are now ready to set up Stability AI to paint new clothing in the areas we calculated using our detection and segmentation pipeline. To set up Stability AI, follow these instructions:

1. Add the Stability AI block.
2. Connect the output of the Segmentation block to the input of the Generative AI block.
3. Configure the block to accept our custom prompt parameter, such as:
   • “Summer outfit”
   • “80s fashion”
   • “Scuba diver”
4. Enter your Stability AI API Key.

Test your workflow

We are now ready to test the Workflow.

First, configure the output block to display the detection and segmentation visualizations and the Inpainting result.

Then, upload a selfie.

Next, click Run Workflow and watch the Workflow process the input data step by step. The steps followed are:

1. YOLO detects the clothing items.
2. The filter block narrows down detections to relevant classes.
3. SAM2 generates accurate segmentation masks.
4. Stability AI modifies the outfits based on your custom prompt.

Results

Here are some examples of how the workflow performed on my images. You can see how the detected clothing items were segmented and transformed using custom prompts.

Here is the input image:

Here is the result of our object detection model, identifying a jacket:

Here is the result of the SAM-2 step, in which a segmentation mask is calculated for the jacket:

Here is the result when we ask Stability AI to edit the image with the prompt "suit and tie":

Conclusion

This workflow combines the power of YOLOv8 Object Detection, SAM2 Image Segmentation, and Stability AI Stable Diffusion Inpainting to detect and transform clothing in images. Whether you’re working on AI-driven fashion tools or simply exploring computer vision, it’s a great way to get hands-on with these models.

Ready to build your next computer vision project?

• Try Roboflow for free and explore our tutorials to get started.
• Join our community to share experiences, ask questions, and learn from others.

Cite this Post

Use the following entry to cite this post in your research:

Samuel Alexander. (Jan 24, 2025). Transforming Outfits with Roboflow Workflows and Generative AI. Roboflow Blog: https://blog.roboflow.com/transforming-outfits-with-roboflow-workflows/

Discuss this Post

If you have any questions about this blog post, start a discussion on the Roboflow Forum.