Power Automate AI Builder and Scenarios

AI Builder: Pre-Built and Custom AI models Ready to Use within the Microsoft Power Automate Platform

This blog is about AI Builder, as it is used within the Microsoft Power Automate tool. We describe what AI Builder is, its major capabilities, the different types of models it has built-in, and the different types of scenarios where AI Builder can be very useful in process automation. AI Builder brings the power of AI models (many pre-built) directly into your automation efforts without the need for coding or data science skills.

This is another paper in our series on the Microsoft Power Automate toolset[1]. We are writing this continuing series of papers to cover all the major features and functionality of Power Automate. We are also writing a companion series of papers on real-life case studies using Power Automate. These are case studies describing how we have implemented Power Automate solutions for specific business process challenges of our clients or internal business process automation within Macrosoft.

We consider both series of papers as practical and hopefully learning guides. We are writing them to generate interest in this field and to provide down-to-earth information for our clients and potential clients on Power Automate. We strongly believe this tool is the best way forward for mid-sized businesses to achieve quick and meaningful process automation and to move forward on digital transformation. This is especially true if the company is already Microsoft-based.

For those not familiar with Power Automate, it is a service that helps you create automated workflows between apps and data stores to synchronize files, get notifications, collect data, and many more such process automation. [2]  It fits within the technical scope of robotic process automation (RPA) and is highly rated compared to its RPA competitors.

Overview of AI Builder?

AI Builder, used in conjunction with Power Automate, provides AI models for optimizing and automating business processes. Many of the models are pre-designed and ready to use out of the box; others can be custom built/tuned within the tool itself to fit a company’s specific data and process needs. AI Builder allows Power Automate to automate process steps where an AI model is essential, for example, those requiring various forms of text analysis, vision and image detection, and predicting future outcomes.

AI Builder brings the power of AI through a point-and-click experience, so you do not need coding or data science skills to access the power of AI. With AI Builder, you can build custom models tailored to your special needs or choose a prebuilt model that is ready to use for many common business scenarios.

So, what kinds of things can you do with AI Builder and what kinds of business problems can it tackle. In fact, there are many things you can do with AI Builder for example:

• Analyze text for classifying it, detecting key phrases, translating the language, and deriving the sentiment behind the words.
• Predict whether an outcome or event will happen in the future based on historical and current data.
• Process business card contacts information and adds it to a contact database.
• Process text from images (~OCR) and then subject it to the same text analysis indicated above.
• Read, extract, and save information from standard document types such as invoices or receipts.
• Recognize, extract and count specific, previously defined, items in a large number of image files.

With AI Builder you can also refine AI models to your specific business needs. Available models for refinement include those listed below. We also describe a specific use case where the model would be very useful. Of course, there are countless other use cases for these models:

Category Classification:

This model can be used for a variety of use cases for example analyzing customer feedback for a hospital. In this case, this will free up the time of a hospital administrator from categorizing customer feedback, therefore leaving that person with more time to act on it and providing a better patient experience.

• Entity Extraction:

This model extracts entities like color, person, city, country, etc. This type of model is helpful in multiple use cases, for example in a retail company with branches in multiple cities. They can use an entity extraction model to extract city entities from customer complaints data, and then route those complaints to a specific customer service center or designated person in the organization. This will help the retail company streamline its operations and save time.

• Form Processing:

A form processing model can be used to extract specific and useful data from forms. This could include any type of form, such as a candidate validation form, and any type of text data within that form, such as candidate name, date of birth, etc.

• Object Detection:

This model helps in recognizing and extracting things within images, and it can also count these objects within images. This is useful for example in a retail shop where there is a maximum number of people allowed in the shop at any given time. An object detection model can be trained to count the number of people in the shop, therefore, helping the retail shop in maintaining compliance.

• Prediction:

Prediction models help us predict future outcomes based on historical and current data. A well-cited use case for a prediction model is within a bank, where the prediction is whether or not a potential customer will default on a loan?

In the Model Description section below, we give a more complete list of the types of models currently available in AI Builder. In the Business Scenarios section below, we go into more detail on the types of business scenarios where AI Builder is essential to automating a process.

How to Use AI Builder

This section describes at a high level the process you use to invoke an AI Model within Power Automate. Using AI Builder makes it easy thanks to its seamless integration with Power Automate and its intuitive UI. Adding AI intelligence to your business processes is simple!

Referring to the user screen below, the five top-level tasks proceed from it as follows:

1. Choose an AI model type: Use the model type that suits your business need. Choose from a growing set of AI solutions.
2. Connect data: Select your business-specific data from the available options.
3. Tailor your AI model: Depending on the type of model, you can tweak custom models to optimize how your AI performs.
4. Train your AI model: Training is an automatic process, where AI Builder “teaches” your AI model how to resolve your business problem (for example, how to recognize your products on an image) thanks to your business data and tailoring. When trained, your AI model can generate insights such as the result of a prediction, or the list and number of objects detected in an image.
5. Use insights from your AI model: Use the results from your AI model within Power Automate to create solutions that meet your business needs, even if you have no coding skills. For example, you can create a flow that automates document processing in Power Automate or an app that predicts whether a supplier will be out of compliance.

Model Types in AI Builder

Let’s look at the different model types that are available in AI Builder, and how they are classified. Later, we will look at common business scenarios and the model types that are suited to each of them. Please note that Microsoft is constantly adding new models to the mix, and upgrading existing models, so this list will change and expand with time.

AI Builder models are classified under the general type of AI that they use – text, vision, or prediction – and the build-type of the model. The build type indicates whether it is a customizable model that you must build, train, and publish for your specific data and business use, or whether it is a prebuilt model that’s ready to use out-of-the-box.

As a rule, custom AI Builder models are better suited for applications where you are working with data that is fairly unique and specific to your business. Prebuilt models are available for business scenarios where the data and processes are fairly common across different types of businesses.

These different types of AI models that AI Builder provides put a broad range of AI capabilities in the hands of businesses without the need for coding or data expertise.

Common Business Scenarios

What are the practical business applications of these AI models? While the possibilities are many, the list below shows some common business scenarios, and the AI model best suited to address each of them:

Concluding Comments

AI Builder offers companies a tremendous pathway to enhancing automation processes with AI capabilities, one that does not require developers or data scientists. These models are practical and can be used by business analysts to greatly improve and automate business processes within a company, and between companies.

We recommend mid-sized companies start using Power Automate with AI Builder to automate as many of their business processes as possible. Gartner refers to this view of automation as hyper-automation that is a company automates everything that can be automated!  We hold this view also. Please contact us if you want to learn more about Power Automate or its AI Builder component. As a company, Macrosoft is committed to greatly expanding our work in this area. Give us a call to discuss your ideas for automation, and we might find a way to work together on making it happen in the near future!

[1] Power Automate is one of the 4 tools in the Microsoft Power Platform family of tools, along with Power BI, Power Apps, and Power Virtual Agents.

[2] Please see other papers on Power Automate on Macrosoft’s web site.

By Talha Khan, G.N. Shah, Ronald Mueller | September 22nd, 2021 | Process Automation