Getting Started with Vision

Getting Started with Vision

Learn how to incorporate computer Vision functionalities in your Tulip Apps

In this article you will learn...

• An overview of Vision's functionalities and use cases
• How to setup a Camera Configuration
• How to remotely view the video stream of a camera
• Using multiple Camera Configurations at a station

Overview

With Vision you can easily integrate computer vision functionality into your Tulip apps. Vision is integrated into Tulip Player, and requires no additional software installation.

Vision is available in beta to all Tulip Accounts and integrated with the newest release of the Tulip Player.

Vision's Functionalities

• Viewing the video stream of cameras on your shop floor
• Using a Change Detector - detect changes in pre-defined Regions
• Detecting so called Jigs - objects to which you've attached printed markers, using a Jig Detector
• Taking Snapshots

Vision's Value

• Remotely assist operators by having a real-time view into what's happening at a station
• Implement an advanced Pick To Light system by combining the Change Detector with Light Kit
• Extend your time studies with additional events detected through Vision
• Track material arrival & departure at a station by knowing when a jig appears or disappears
• Improve tracking of quality problems by capturing snapshots.

These example use cases show that, much in contrast to traditional visual inspection systems, Vision is focussed on augmenting your operators. Check out this short demo here:

Requirements

Before starting with the setup, there are some prerequisites to note:

• Tulip Player must be running on Windows (minimum version is 1.3.2).
• Your user must have a Station Supervisor role, or higher.
• The Change Detector requires an Intel D415 depth camera (see Hardware Recommendations
• The Jig Detector works with any USB camera. While Vision is in beta we are finalizing a list of officially supported cameras.
• When connecting USB cameras to Tulip, please make sure to only use USB 3.0 cables to the clearly marked USB 3.0 ports on the computer

Setting up Vision

To setup Vision, you'll need to first setup a Camera Configuration. Click on the Shop Floor tab, and navigate to the Vision page:

Click on the Create button to create a new Camera Configuration. Please give it a name, and click Next.

In the next step, you will assign the new Camera Configuration to a camera on your shop floor.

This shows a list of all cameras connected to any Tulip Player authenticated to your Tulip Account. If the list is empty, please plug in a USB camera now.

You can also duplicate/edit camera configurations by clicking on the ellipsis icon pertaining to a specific camera configuration.

To identify a specific camera you may want to un-plug it, and then plug it back in. The most recently connected camera will be shown at the top of the list.

Please click Assign, and then Done. A new page for your Camera Configuration will open:

By default, once your Camera Configuration is completed, the operator at the assigned station will be prompted with the following modal in the Player:

Note: For more information on camera stream permissions, please Vision and Privacy

Once permission has been granted in Tulip Player, the video stream of the camera you selected will appear:

Congratulations - you are now ready to create Regions, and start using Detectors. See the Further Reading section below to learn how to setup a Change Detector and Jig Detector.

Advanced Assignment of Camera Configurations

An overview of all assignments you previously made on Camera Configuration page can also be found on the Shop Floor page. This overview can be convenient, especially if you want to simultaneously use multiple cameras at a Station:

In the bottom right of the Shop Floor page, you'll see the Display Device section:

This list shows all connected cameras. Click on the View button to view the camera's live stream directly.

To edit your assigned cameras, or adjust their Camera Configuration, click the Change button:

Further Reading

• Using the Change Detector (Requires: Intel RealSense D415)
• Using the Jig Detector
• Using the Color Detector
• Using the Vision Camera Widget in Apps
• Using the Snapshot Feature in Apps
• Using the Snapshot Feature with an External Vision Service
• Using the Datamatrix and Barcode Detectors
• Vision and Privacy
• Hardware Recommendations for Vision