Power Automate AI Builder and Scenarios

By Talha Khan, G.N. Shah, Ronald Mueller

Power Automate AI Builder and Scenarios

AI Builder: Pre-Built and Custom AI models Ready to Use within the Microsoft Power Automate Platform

This blog is about AI Builder, as it is used within the Microsoft Power Automate tool. We describe what AI Builder is, its major capabilities, the different types of models it has built-in, and the different types of scenarios where AI Builder can be very useful in process automation. AI Builder brings the power of AI models (many pre-built) directly into your automation efforts without the need for coding or data science skills.

Download Blog as PDF

Download the entire research blog as PDF for free

This is another paper in our series on the Microsoft Power Automate toolset[1]. We are writing this continuing series of papers to cover all the major features and functionality of Power Automate. We are also writing a companion series of papers on real-life case studies using Power Automate. These are case studies describing how we have implemented Power Automate solutions for specific business process challenges of our clients or internal business process automation within Macrosoft.

We consider both series of papers as practical and hopefully learning guides. We are writing them to generate interest in this field and to provide down-to-earth information for our clients and potential clients on Power Automate. We strongly believe this tool is the best way forward for mid-sized businesses to achieve quick and meaningful process automation and to move forward on digital transformation. This is especially true if the company is already Microsoft-based.

For those not familiar with Power Automate, it is a service that helps you create automated workflows between apps and data stores to synchronize files, get notifications, collect data, and many more such process automation. [2]  It fits within the technical scope of robotic process automation (RPA) and is highly rated compared to its RPA competitors.

Overview of AI Builder?

AI Builder, used in conjunction with Power Automate, provides AI models for optimizing and automating business processes. Many of the models are pre-designed and ready to use out of the box; others can be custom built/tuned within the tool itself to fit a company’s specific data and process needs. AI Builder allows Power Automate to automate process steps where an AI model is essential, for example, those requiring various forms of text analysis, vision and image detection, and predicting future outcomes.

AI Builder brings the power of AI through a point-and-click experience, so you do not need coding or data science skills to access the power of AI. With AI Builder, you can build custom models tailored to your special needs or choose a prebuilt model that is ready to use for many common business scenarios.

So, what kinds of things can you do with AI Builder and what kinds of business problems can it tackle. In fact, there are many things you can do with AI Builder for example:

  • Analyze text for classifying it, detecting key phrases, translating the language, and deriving the sentiment behind the words.
  • Predict whether an outcome or event will happen in the future based on historical and current data.
  • Process business card contacts information and adds it to a contact database.
  • Process text from images (~OCR) and then subject it to the same text analysis indicated above.
  • Read, extract, and save information from standard document types such as invoices or receipts.
  • Recognize, extract and count specific, previously defined, items in a large number of image files.

With AI Builder you can also refine AI models to your specific business needs. Available models for refinement include those listed below. We also describe a specific use case where the model would be very useful. Of course, there are countless other use cases for these models:

Category Classification:

This model can be used for a variety of use cases for example analyzing customer feedback for a hospital. In this case, this will free up the time of a hospital administrator from categorizing customer feedback, therefore leaving that person with more time to act on it and providing a better patient experience.

  • Entity Extraction:

This model extracts entities like color, person, city, country, etc. This type of model is helpful in multiple use cases, for example in a retail company with branches in multiple cities. They can use an entity extraction model to extract city entities from customer complaints data, and then route those complaints to a specific customer service center or designated person in the organization. This will help the retail company streamline its operations and save time.

  • Form Processing:

A form processing model can be used to extract specific and useful data from forms. This could include any type of form, such as a candidate validation form, and any type of text data within that form, such as candidate name, date of birth, etc.

  • Object Detection:

This model helps in recognizing and extracting things within images, and it can also count these objects within images. This is useful for example in a retail shop where there is a maximum number of people allowed in the shop at any given time. An object detection model can be trained to count the number of people in the shop, therefore, helping the retail shop in maintaining compliance.

  • Prediction:

Prediction models help us predict future outcomes based on historical and current data. A well-cited use case for a prediction model is within a bank, where the prediction is whether or not a potential customer will default on a loan?

In the Model Description section below, we give a more complete list of the types of models currently available in AI Builder. In the Business Scenarios section below, we go into more detail on the types of business scenarios where AI Builder is essential to automating a process.

How to Use AI Builder

This section describes at a high level the process you use to invoke an AI Model within Power Automate. Using AI Builder makes it easy thanks to its seamless integration with Power Automate and its intuitive UI. Adding AI intelligence to your business processes is simple!

Referring to the user screen below, the five top-level tasks proceed from it as follows:

Power Automate AI Builder Interface
  1. Choose an AI model type: Use the model type that suits your business need. Choose from a growing set of AI solutions.
  2. Connect data: Select your business-specific data from the available options.
  3. Tailor your AI model: Depending on the type of model, you can tweak custom models to optimize how your AI performs.
  4. Train your AI model: Training is an automatic process, where AI Builder “teaches” your AI model how to resolve your business problem (for example, how to recognize your products on an image) thanks to your business data and tailoring. When trained, your AI model can generate insights such as the result of a prediction, or the list and number of objects detected in an image.
  5. Use insights from your AI model: Use the results from your AI model within Power Automate to create solutions that meet your business needs, even if you have no coding skills. For example, you can create a flow that automates document processing in Power Automate or an app that predicts whether a supplier will be out of compliance.

Model Types in AI Builder

Let’s look at the different model types that are available in AI Builder, and how they are classified. Later, we will look at common business scenarios and the model types that are suited to each of them. Please note that Microsoft is constantly adding new models to the mix, and upgrading existing models, so this list will change and expand with time.

MODEL TYPES
Model typeCategoryBuild type
Category classificationTextPrebuilt and Custom
Entity extractionTextPrebuilt and Custom
Key phrase extractionTextPrebuilt
Language detectionTextPrebuilt
Sentiment analysisTextPrebuilt
Text translationTextPrebuilt
PredictionPredictionCustom
Form processingVisionCustom
Object detectionVisionCustom
Business card readerVisionPrebuilt
Text recognitionVisionPrebuilt
Receipt processingVisionPrebuilt

AI Builder models are classified under the general type of AI that they use – text, vision, or prediction – and the build-type of the model. The build type indicates whether it is a customizable model that you must build, train, and publish for your specific data and business use, or whether it is a prebuilt model that’s ready to use out-of-the-box.

As a rule, custom AI Builder models are better suited for applications where you are working with data that is fairly unique and specific to your business. Prebuilt models are available for business scenarios where the data and processes are fairly common across different types of businesses.

These different types of AI models that AI Builder provides put a broad range of AI capabilities in the hands of businesses without the need for coding or data expertise.

Download Blog as PDF

Download the entire research blog as PDF for free

Common Business Scenarios

What are the practical business applications of these AI models? While the possibilities are many, the list below shows some common business scenarios, and the AI model best suited to address each of them:

COMMON BUSINESS SCENARIOS
Business scenarioModel type
Automate customer application processingForm processing
Automate expense reportsReceipt processing
Categorize user feedback based on their focusCategory classification
Extract insights from product reviewsEntity extraction
Identify the language of textLanguage detection
Identify and classify customer feedbackSentiment analysis
Translate support requests into your languageText translation
Identify fraudulent transactionsPrediction
Get alerted to social media posts referencing your brandKey phrase extraction
Automate contact listBusiness card reader
Automate inventory takingObject detection
Take a photo of text and save it to a databaseText recognition
Detect products in imagesObject detection
Expense reports and receiptsReceipt scanning model
Prediction of marketing campaign resultsPrediction model

Concluding Comments

AI Builder offers companies a tremendous pathway to enhancing automation processes with AI capabilities, one that does not require developers or data scientists. These models are practical and can be used by business analysts to greatly improve and automate business processes within a company, and between companies.

We recommend mid-sized companies start using Power Automate with AI Builder to automate as many of their business processes as possible. Gartner refers to this view of automation as hyper-automation that is a company automates everything that can be automated!  We hold this view also. Please contact us if you want to learn more about Power Automate or its AI Builder component. As a company, Macrosoft is committed to greatly expanding our work in this area. Give us a call to discuss your ideas for automation, and we might find a way to work together on making it happen in the near future!


[1] Power Automate is one of the 4 tools in the Microsoft Power Platform family of tools, along with Power BI, Power Apps, and Power Virtual Agents.

[2] Please see other papers on Power Automate on Macrosoft’s web site.

Share this:

ByTalha Khan, G.N. Shah, Ronald Mueller | Published on September 22nd, 2021 | Last updated on August 21st, 2023 | Robotic Process Automation

About the Author

Talha Khan

Talha Khan

Talha is an accomplished Information Technology Project Manager at Macrosoft. With a robust background spanning over six years, Talha boasts extensive experience in spearheading a diverse array of projects encompassing data science, analytics, and healthcare solutions. His professional journey has seen him navigate through various industries, from healthcare to ad tech, fintech, and e-commerce, each contributing to his comprehensive understanding of project management dynamics.

Before his managerial role, Talha excelled as a Senior Software Engineer and Data Scientist, showcasing his versatility across different domains. Proficient in an array of programming languages such as Python, JavaScript, and TypeScript, Talha brings a multifaceted skill set to the table, ensuring effective project execution and delivery.

Moreover, Talha's commitment to continuous learning and professional development is evident through his acquisition of over 22 licenses and certificates spanning technology, analytics, data science, and project management domains. With his wealth of expertise and dedication to excellence, Talha stands poised to drive forward-thinking initiatives and deliver impactful solutions in the ever-evolving landscape of information technology.

Talha is passionate about leveraging his experience, training, and skills to help clients efficiently deliver their IT projects. He has a Bachelor of Science in Accounting and Finance with a minor in Economics from the University of New Hampshire.

Besides work, Talha is a nature enthusiast. He likes hiking and traveling. He regularly works out and loves networking.

G. N. Shah Chief Technology Officer of Macrosoft

G.N. Shah

Shah is a forward-thinking corporate leader with twenty years of experience delivering top-notch customer solutions in large scale and enterprise business environments. As a technology visionary and driver of strategic business systems development, allow Macrosoft to deliver best-in-class software solutions. Shah’s profound expertise includes business strategy, product development, enterprise-wide architecture, application migration, IT transformation, and international development management. Shah holds multiple professional and technical certifications, scholastic degrees, and an MBA.

Dr. Ronald Mueller Chairman and Founder of Macrosoft

Ronald Mueller

Ron is the Chairman and Founder of Macrosoft, Inc. He heads up all company strategic activities and directs day-to-day work of the Leadership Team at Macrosoft. As Macrosoft’s Chief Scientist, Ron defines and structures Macrosoft’s path forward. Ron's focus on new technologies and products, such as Cloud, Big Data, and AI/ML/WFP. Ron has a Ph.D. in Theoretical Physics from New York University and worked in physics for over a decade at Yale University, The Fusion Energy Institute in Princeton, New Jersey, and at Argonne National Laboratory.

Ron also worked at Bell Laboratories in Murray Hill, New Jersey., where he managed a group on Big Data. Ron's work focused around the early work on neural networks. Ron has a career-long passion in ultra-large-scale data processing and analysis including predictive analytics, data mining, machine learning and deep learning.

Recent Blogs

The Impact of Emerging Technologies on IT Staffing
The Impact of Emerging Technologies on IT Staffing
Read Blog
Data-Driven Insights: Optimizing Your CCM Efforts for Success
Data-Driven Insights: Optimizing Your CCM Efforts for Success
Read Blog
Harnessing the Future: Document Automation in Salesforce
Harnessing the Future: Document Automation in Salesforce
Read Blog
Data Symphony: Harmonizing Insights for Personalized Customer Journeys
Data Symphony: Harmonizing Insights for Personalized Customer Journeys
Read Blog
TOP