---
title: "Use Microsoft Azure CustomVision.ai for visual inspection"
slug: "use-microsoft-azure-customvisionai-for-visual-inspection"
updated: 2024-08-14T18:07:26Z
published: 2024-08-14T18:07:26Z
canonical: "support.tulip.co/use-microsoft-azure-customvisionai-for-visual-inspection"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://support.tulip.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Microsoft Azure CustomVision.ai for visual inspection

*Using Azure CustomVision.ai with Tulip Vision is an easy no-code way to implement visual inspection on your workstations and beyond.*

In this article we will demonstrate on how to use Microsoft Azure's CustomVision.ai service for visual inspection with Tulip. The CustomVision.ai service is for easily creating machine learning models for visual recognition tasks. With Tulip you can collect data for training the machine learning models that CustomVision.ai offers.

Visual inspection is an important part of frontline operations. It can be used to ensure only high quality products are leaving the line, reduce returned parts and re-work, and increase the true yield. Automatic visual inspection can save up on assigning manual labor to perform visual inspection, reducing overall costs and increasing efficiency. With Tulip Vision, visual inspection can be added to any workstation with speed and ease, by connecting an affordable camera to an existing computer and building a Tulip App for inspection.

## Prerequisites

- A working Tulip Vision workstation with a camera for visual inspection. Follow the [getting started guide for Tulip Vision](https://support.tulip.co/docs/getting-started-with-vision)
- An account to use on [CustomVision.ai](http://customvision.ai)/alternatively, you can use Landing AI - read more [here](https://tulip.co/library/apps/landingai-unit-test/).
- A product to use for the visual inspection task
- A dataset of at least 30 images for each category class you wish to inspect (e.g. "Pass" or "Fail", "Defect 1", "Defect 2", "Defect 3", etc.). Follow the instructions on the guide for [collecting and exporting visual inspection data from Tulip](https://support.tulip.co/docs/collecting-data-for-visual-inspection-with-vision)

### Example Visual Inspection Setup

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_499622009.png)

## Uploading the Inspection Images to CustomVision.ai

From the dataset Tulip Table click "Download Dataset" and select the relevant columns for image and annotation. Download and unzip the dataset .zip file to a folder on your computer. It should have a number of subfolders per each category of detection according to the annotation in the dataset table.

Create a new project on Customvision.ai:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_488059388.png)

Name your project and select the "Classification" Project Type and "Multiclass (Single tag per image)" Classification. Type options: (these options are selected by default)

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_488061482.png)

Click "Add Images".

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_488063381.png)

Select the images on your computer for one of the classes. You can select *all* the images in the subfolder you got from the Tulip Table unzipped dataset. Once the images are loaded in Customvision.ai you can apply a tag to all of them at once, to save on tagging them one-by-one. Since all the current images are from the same class, this is possible.

In the following example we upload all the "Normal" class images and apply the tag (class) to all of them at once:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_488763448.png)

Repeat the same upload operation for the other classes.

## Training and Publishing a Model for Visual Inspection

Once the data for training is in place, proceed to train the model. The "Train" model on the top right corner opens the training dialog.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_488767130.png)

Select the training mode appropriately. For a quick trial run to see everything is working properly, use the "Quick" option. Otherwise for best classification results, use the "Advanced" option.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490185021.png)

Once the model is trained, you will be able to inspect its performance metrics, as well as publish the model so it's accessible via an API call.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490186862.png)

Select the proper resource for the publication and continue.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490187624.png)

At this point your published model is ready to accept inference requests from Tulip. Take note of the publication URL as we will be using it shortly to connect from Tulip.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490190107.png)

![Using Microsoft Azure CustomVision.ai with Tulip Vision for Visual Inspection_490190230.png](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490190230%282%29.png)

## Widget for Making Inference Requests to the Published Model

Making inference requests to Azure CustomVision.ai service can be done on Tulip by using a Custom Widget. You'll find the Custom Widgets page under Settings.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490277422.png)

Create a new Custom Widget and add the following inputs:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490280427.png)

For the code parts, use the following:

**HTML**

```
<button class="button" type="button" onclick="detectAnomalies()">Detect Anomalies</button>
```

**JavaScript**

**Note**: Here you need to get the URL and `prediction-key` from the CustomVision.ai published model.

```
function b64toblob(image) {  
    const byteArray = Uint8Array.from(window.atob(image), c => c.charCodeAt(0));  
    return new Blob([byteArray], {type: 'application/octet-stream'});  
}  
  
async function detectAnomalies() {  
    let image = getValue("imageBase64String");  
    const url = '<<< Use the URL from CustomVision.ai>>>';  
    $.ajax({  
        url: url,  
        type: 'post',  
        data: b64toblob(image),  
        cache:false,  
        processData: false,  
        headers: {  
            'Prediction-Key': '<<< Use the prediction key >>>',  
            'Content-Type': 'application/octet-stream'  
        },  
        success: (response) => {  
            setValue("predictions", response["predictions"]);  
        },  
        error: (err) => {  
            console.log(err);  
        },  
        async: false,  
    });  
}
```

**CSS**

```
.button {  
  background-color: 

##616161;  
  border: none;  
  color: white;  
  padding: 15px 32px;  
  text-align: center;  
  text-decoration: none;  
  display: inline-block;  
  font-size: 16px;  
  width: 100%;  
}
```

Make sure to enable the jQuery external library on the custom widget. ![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Screen%20Shot%202022-10-12%20at%2011.00.35%20AM_shadow.jpg)

Your custom Widget should look like the following:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490283461.png)

## Using the Prediction Widget in a Tulip App

Now the Widget is set up you can simply add it to an app in which you will be running the inference requests. You may construct an app Step as the following:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_497394399.png)

Use a regular button to take a Snapshot from the visual inspection camera and save it in a Variable:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490299484.png)

Use the "Detect Anomalies" custom widget.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490295795.png)

Configure the widget to accept the snapshot Image Variable as a base64string.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490296728.png)

Assign the output to a variable to display on screen or use otherwise.

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_490302048.png)

Your app is now ready to run inference requests for visual inspection.

## Running the Visual Inspection App

Once your app is ready, run it on a Player machine with the inspection camera you used for data collection. It is important to replicate the same situation you used for data collection as for inspection inference, to eliminate any error from variance in lighting, distance, or angle.

Here is an example of a running visual inspection app:

![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Using%20Microsoft%20Azure%20CustomVision.ai%20with%20Tulip%20Vision%20for%20Visual%20Inspection_497398652.png)

## Further Reading

- [Getting Start with Vision](https://support.tulip.co/docs/getting-started-with-vision)
- [Collecting Data for Visual Inspection](https://support.tulip.co/docs/collecting-data-for-visual-inspection-with-vision)
- [Landing AI Custom Widget](https://support.tulip.co/docs/landingai-widget)
- [Landing AI Unit Test](https://tulip.co/library/apps/landingai-unit-test/)
- [Custom Widgets Overview](https://support.tulip.co/docs/custom-widgets-overview)

**Tulip Vision**

**Vision**is a simple no-code tool to use cameras for visual inspection, process adherence, equipment, personnel, and material tracking on the shop floor.

**Tulip Tables**

**Tulip Tables** are a global location to store your production data. **Tables** are made up of **Records** (rows). A single can be accessed from multiple apps or stations at the same time. ![](https://cdn.document360.io/7c6ff534-cad3-4fc8-9583-912c4016362f/Images/Documentation/Tulip%20Tables%20Overview%20-%20Feature%20Overview(1).gif)

**Classification**

**Classification**is the problem of identifying which of a set of categories a **Tulip Vision Snapshot** belongs to.

*ex. Given a Vision Snapshot, tell me what type of defect this part has.*

**Widget**

**Widgets**are the elements that make up a specific **App Step.**Widgets can display information to users, collect user input, or trigger app logic.

*Common widgets include: Interactive Tables, Number inputs, Machine attribute widgets, and more.*

**Step**

A view your users will see within an application. **Steps** can be viewed chronologically or in whatever order best fits your process.

Steps can be grouped into **Step Groups**to manage and organize your app Steps.

**Snapshot**

**Snapshots**are the capture of a single frame of a camera running **Tulip Vision.**These image captures can be stored to tables, passed to external APIs, and more.

**Variable**

**Variables**are a location to store app information. Variables have a specific type that must match the contents they can store.

Variables are only accessible within a single application and are cleared when the app is restarted or completed.

**Image**

**Images**are a Tulip Datatype. Images are a reference to a web-hosted location for all Tulip Images.

**Tulip Player**

**Tulip Player** is the Windows/Mac executable program where users can run Tulip apps. Tulip player allows you to create a more seamless user experience by removing the need for a web browser, and allows increased IT controls.