Learning outcomes 💫

By the end of this blog post, you should be able to...

  • Understand what RPA is and how it is useful
  • Know how to integrate a Power Automate flow via webhook
  • Use a Power Automate flow to send an email report

What is Robotic Process Automation (RPA)?

Robotic Process Automation is a term used to describe the automation of specific tasks, often related to business productivity. By using RPA, automation engineers can create digital workers or agents that perform specific tasks.

A simple example would be teaching an agent to filter emails to allow an end user automatically archive and filter emails. A complex example would be teaching an agent to automate an entire business process by using flows. A flow is a way to order and execute tasks, such as entering data into an excel, sending an email, and saving a picture to a database. By building a robust and orderly flow, we can teach our digital agents to do just about anything.

What is RPA used for?

RPA use cases include building automation for banking and financial services, speeding up the insurance industry, improving manufacturing efficiency, and solving healthcare challenges. All of these industries can benefit from understanding employee processes and integrating automation into those workflows.

When many tasks come together into a repeatable process, there is opportunity for automation to standardize that process via RPA. Through this automation, companies see increased productivity and efficiency across domains.

Why is RPA useful for Computer Vision?

Much like how software developers make packages, automation engineers make automation flows through use of software like UiPath or Power Automate. These platforms allow engineers to share their automation flows to other users in order to build knowledge and expedite solutions.

While these platforms are usually described as “low-code” or “no-code” they often require deep logical thinking and have similar effects to code based solutions. By stringing together various connectors and data pipelines. An automation engineer can accomplish tasks faster than a code based solution.

RPA can extend the capabilities of a computer vision model to do various tasks. Instead of building automation into your code, you can send data to Power Automate to use their low code platform to build an application.

Using RPA with Computer Vision

To get started with RPA, we will use Power Automate to build a flow to save images to Google Drive, log data to Google Sheets, and then send an email report. The rest of this post will walk you through each step in the process and by the end you will have a ready to use framework for building RPA into your computer vision application.

Send Autonomous Email with RPA

A way to send an autonomous email is through a RPA flow. Many RPA platforms have connectors that integrate directly into Outlook 365 and Gmail. With these connectors, you don’t have to worry about authentication or structuring the call with the proper SMTP server.

The syntax and authentication is handled for you and variables easily pass from the computer vision application to the email application inside your automation platform. In this example, we use Power Automate to send a basic email report. This email can be distributed to a group of people and can be extended with some filtering logic to distribute specific reports to specific people.

Sending emails with RPA

Updating Databases using RPA

RPA can be used to send data to a Google Sheet. Being able to chain data capture into a flow is exceptionally useful. There are many python packages to save data to a .csv file locally, but Google Sheets offers a quick way to save data in a shareable way. If you need something more robust, you can use the PostgreSQL connector.

Using RPA to send data to Google Sheets

Using RPA to Perform Business Logic

In the situation below, Power Automate is being used to convert a base64 string into .jpg files and storing them in folders inside Google Drive. After that, an email report is sent and the data is stored in Google Sheets.

The webhook called “When a HTTP request is received” is triggered by a computer vision model when it sends data via a JSON post request. By using Power Automate, we don’t have to find packages for email automation, Google Sheets, or Google Drive. The connectors are built into the platform and extend our computer vision capabilities.

Using Roboflow and RPA to Send an Email Report

Now that you've seen the building blocks of how RPA can be useful in different tasks, let's put them together into a real world use case. For this demo, you'll need a photo, a Roboflow model, your Roboflow API key, and your email address.

The script below will be used to send a basic computer vision report over email using Power Automate. You won’t need to set up the Power Automate endpoint because it is already configured for this demonstration.

We will use Roboflow to perform inference on a photo and then send that data to Power Automate to generate an email report. You can download the photo below as your test image, or use your own image.

TestImage.jpg

Computer Vision RPA Script

roboflow-computer-vision-utilities/trigger_power_automate.py at main · roboflow-ai/roboflow-computer-vision-utilities
Interface with the Roboflow API and Python package for running inference (receiving predictions) and customizing result images from your Roboflow Train computer vision models. - roboflow-computer-v...

Example Email

Using RPA to Build Computer Vision Applications

When it comes to building computer vision applications, there are different tools you can use at each step in your pipeline. Depending on your use case, it might make sense to use low-code RPA tools.

If you have a computer vision application in mind, get started by using an open source dataset from Roboflow Universe.