National Instruments NI myRIO User manual

Ed Doering
NI myRIO
Vision Essentials Guide

NI myRIO Vision Essentials Guide
Ed Doering
Electrical and Computer Engineering Department
Rose-Hulman Institute of Technology

iv
Printed July 23, 2015. Download the latest version at http://www.ni.com/myrio/
vision-guide.
c
2015 National Technology and Science Press.
All rights reserved. Neither this book, nor any portion of it, may be copied or repro-
duced in any form or by any means without written permission of the publisher.
NTS Press respects the intellectual property of others, and we ask our readers to do
the same. This book is protected by copyright and other intellectual property laws.
Where the software referred to in this book may be used to reproduce software or other
materials belonging to others, you should use such software only to reproduce materials
that you may reproduce in accordance with the terms of any applicable license or other
legal restriction.
LabVIEW and National Instruments are trademarks of National Instruments.
All other trademarks or product names are the property of their respective owners.
Additional Disclaimers: The reader assumes all risk of use of this book and of all in-
formation, theories, and programs contained or described in it. This book may contain
technical inaccuracies, typographical errors, other errors and omissions, and out-of-date
information. Neither the author nor the publisher assumes any responsibility or lia-
bility for any errors or omissions of any kind, to update any information, or for any
infringement of any patent or other intellectual property right.
Neither the author nor the publisher makes any warranties of any kind, including
without limitation any warranty as to the sufficiency of the book or of any information,
theories, or programs contained or described in it, and any warranty that use of any
information, theories, or programs contained or described in the book will not infringe
any patent or other intellectual property right. THIS BOOK IS PROVIDED “AS IS.” ALL
WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED
TO, ANY AND ALL IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT OF INTELLECTUAL
PROPERTY RIGHTS, ARE DISCLAIMED.
No right or license is granted by publisher or author under any patent or other intel-
lectual property right, expressly, or by implication or estoppel.
IN NO EVENT SHALL THE PUBLISHER OR THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, COVER, ECONOMIC, OR CONSE-
QUENTIAL DAMAGES ARISING OUT OF THIS BOOK OR ANY INFORMATION,
THEORIES, OR PROGRAMS CONTAINED OR DESCRIBED IN IT, EVEN IF AD-
VISED OF THE POSSIBILITY OF SUCH DAMAGES, AND EVEN IF CAUSED OR
CONTRIBUTED TO BY THE NEGLIGENCE OF THE PUBLISHER, THE AUTHOR,
OR OTHERS. Applicable law may not allow the exclusion or limitation of incidental or
consequential damages, so the above limitation or exclusion may not apply to you.

Contents
1 Introduction 1
I Machine Vision Fundamentals 5
2 Application Development Flow 7
3 Design Pattern for Machine Vision Applications 11
II Introductory Machine Vision Projects 25
4 Camera Setup 27
5 Stereo Vision 33
6 Coin Caliper I 39
7 Coin Caliper II 45
III Machine Vision Application Projects 55
8 Coin Counter 57
9 POS Terminal 67
10 Keyed Optical Lock 73
11 DMM Test Stand 81

vi CONTENTS
12 Gauging Station 87
13 Product Label Inspector 95
14 Component Placement Inspector 101
15 Motion Detector 107
16 Auto-Pan Camera 111
17 Marble Sorter 119
IV Appendices 129
A Recommended Equipment 131
B MXP and MSP Connector Diagrams 137
C LabVIEW Quick Tips 139
D Build a Stand-Alone Application 141
E Marble Sorter Construction Details 143
F Video Tutorial Links 145

1Introduction
Welcome to the NI myRIO Vision Essentials Guide, and welcome to the exciting
world of machine vision with NI myRIO! Machine vision systems play a critical
role in manufacturing automation, shape and color analysis, and robotics. With
little more than a USB webcam you can create NI myRIO applications that
interact with the visual world to sense motion, take physical measurements,
read barcodes and printed labels, inspect products for defects, and respond
to colors. Add LCD displays, switches, and servomotors and you can create
a complete stand-alone application that controls physical apparatus such as a
marble sorter and an auto-panning camera.
This book serves as your guide to the essential techniques necessary to add
vision to your NI myRIO project. You will learn through a mixture of written
materials, video tutorials, and guided hands-on projects. Part I, Machine Vision
Fundamentals, introduces you to a generalized application development flow
suitable for all of the machine vision projects in this book. The development
flow includes defining the application’s requirements, configuring the imaging
system and acquiring representative images, calibrating to real-world units, de-
veloping the vision processing script with NI Vision Assistant, and developing
and validating the complete machine application with NI LabVIEW. Part I also
introduces you to the “Queued State Machine” design pattern that serves as
the basis of the “Machine Vision App” (MVA) LabVIEW project template that
you will use to implement each of the application projects. With this template
you concentrate on the relevant details for each application (vision script and
application-specific LabVIEW coding) instead of getting bogged down trying
to create the entire application from scratch.
Part II, Introductory Machine Vision Projects, presents four projects to
ground you in the basics of image acquisition, stereo vision, NI vision-related
software tools, and application development with the “Machine Vision App”
project template. These introductory projects include detailed step-by-step in-

2 CHAPTER 1. INTRODUCTION
structions to help you quickly get up to speed before tackling the more advanced
projects of Part III, Machine Vision Application Projects. Each of these ten design-
oriented projects presents a set of functional requirements and then offers a
recommended technical approach to develop the vision script and LabVIEW
implementation based on the MVA template project. Each of these open-ended
design projects introduces you to new machine vision concepts and associated
NI Vision implementation techniques. Once you complete these projects you will
be able to create sophisticated embedded vision projects of your own imagining!
Intended Audience
This book is intended for students at the senior year and early graduate levels,
especially those students engaged in capstone projects and research. A back-
ground in electrical and computer engineering is helpful, but students pursuing
other disciplines will find the level of tutorial detail to be more than adequate.
Book Resources
The most-recent version of NI myRIO Vision Essentials Guide is always avail-
able as a free download at http://www.ni.com/myrio/vision-guide ;
page iv shows the document version. The download also includes the “Machine
Vision App” template LabVIEW project that serves as the starting point for each
of the hands-on projects.
This document is fully hyperlinked for section and figure references, and all
video links are live hyperlinks. Open the PDF version of this document for the
most efficient way to access all of the links; click a video hyperlink to automat-
ically launch the video in your browser. Within the PDF, use ALT+leftarrow
to navigate back to a starting point.
Part IV, Appendices, provides the following supplements:
• Appendix A, Recommended Equipment – details each piece of equipment
used to implement the lab projects including product page links,
• Appendix B, MXP and MSP Connector Diagrams – shows the NI myRIO
I/O resources and connector pin numbers,
• Appendix C, LabVIEW Quick Tips – summarizes commonly-used Lab-
VIEW keyboard shortcuts to promote coding efficiency,
• Appendix D, Build a Stand-Alone Application – learn how to deploy the
LabVIEW project as a stand-alone application on the NI myRIO so that
it runs automatically after myRIO powers up,

CHAPTER 1. INTRODUCTION 3
• Appendix E, Marble Sorter Construction Details – presents the hardware
apparatus that serves as the basis of the Marble Sorter application project
(Chapter 17), and
• Appendix F, Video Tutorial Links – summarizes all of the video links and
groups them into related categories for easy reference.
For the Instructor
All of the projects in this book are designed to be graded activities. The “Ma-
chine Vision App” template LabVIEW project includes an “images” subfolder
where students can place the representative images that they used to develop
their vision script and a “documentation” subfolder in which students can place
a lab report and any other documentation that you wish. The project folders
may then be bundled into a single .zip file and uploaded to your course man-
agement system for grading, and you will have electronic access to every file
related to the student’s project. Should you prefer a paper-based approach to
grading, request the students to print the “Main.vi” front panel diagram and all
hidden subdiagrams (see the LabVIEW “File | Print...” menu item with the “VI
documentation” option selected) as well as the front panel of the data highway
cluster type definition and you will be able to see the complete design.
A solutions manual is available to qualified instructors; please contact your
local National Instruments representative for details.

4 CHAPTER 1. INTRODUCTION
1.1 Acknowledgements
This book represents the contributions of many individuals, and without them
this book project would not have been possible. I gratefully acknowledge
Gretchen Edelmon for her technical support, advice, and encouragement re-
garding the book’s content and organization; Thomas J. Bress for his excellent
textbook Effective LabVIEW Programming
1
that introduced me to the Queued
State Machine design pattern that serves as the basis of the “Machine Vision
App” project template; Isaac Sanchez and Alex Drane for timely advice on
LabVIEW vision techniques; and Margaret Barett for inviting me to participate
in the original NI myRIO Beta Program. I am also grateful to Andrew Watchorn
for his ongoing assistance and to Tom Robbins at NTS Press for his editorial
support on this project.
Ed Doering
Department of Electrical and Computer Engineering
Rose-Hulman Institute of Technology
Terre Haute, IN 47803
[e-mail]
1http://www.ntspress.com/publications/effective-labview-programming
,
NTS Press, 2013.

Part I
Machine Vision Fundamentals

2Application Development Flow
The machine vision projects in this manual generally follow a well-defined
application development flow from original concept through validation of the ap-
plication on the NI myRIO target. The particular details vary considerably from
one project to the next, but the overall development flow remains the same.
Familiarize yourself with this development flow, and refer back to this
chapter periodically as you develop your own machine vision application.
NOTE :The application development flow described here may also be used to
create a desktop application in case you do not have access to the NI myRIO
target.
Figure 2.1 on the following page visualizes the application development flow
as an eleven-step procedure, and the remainder of this chapter lists important
considerations for each step:
1. Define the requirements of the machine vision application
• Processing objectives, e.g., measurement and gauging, classification
and sorting, and bar code recognition
•Throughput measured in parts processed per second
• Minimum feature size and number of pixels allocated to this feature
•Size of the field of view
• I/O devices for part sensing, position control, and information dis-
play
2. Configure the image acquisition system
• Determine the minimum feature size in physical units such as mil-
limeters

8 CHAPTER 2. APPLICATION DEVELOPMENT FLOW
real-world
units?
define
requirements
configure
imaging system
acquire
representative
images
calibrate
camera
develop
processing
script
develop
LabVIEW
project
validate
application
Y
N
Figure 2.1: Development flow for a machine vision application.

CHAPTER 2. APPLICATION DEVELOPMENT FLOW 9
• Determine the number of pixels to allocate to the minimum-sized fea-
ture, and thereby obtain the required spatial resolution in millimeters
per pixel
• Select the camera resolution and camera-to-object distance to achieve
the required spatial resolution, field of view (FOV), and frame rate
• Arrange the lighting system to obtain high contrast for the features
of interest and to suppress distracting background features that
complicate subsequent software processing and add unnecessary
computational cost
• Use grayscale images for efficiency unless color is required for the
application; color images tend to slow down the display, especially
at higher resolutions
3. Acquire representative images
•Connect the camera to the desktop or to the NI myRIO
• Use Vision Assistant to collect images over the range of possible
conditions: Select the “Target” as “This Computer” for a desktop-
connected camera and as “Select Network Target” when the cam-
era is connected to myRIO; use IP address 172.22.11.2 when
myRIO is tethered by its USB cable otherwise use the IP address
assigned to myRIO by the wireless network.
• Use manual camera settings, especially focus; record the settings for
later use
• Collect an image of one or more calibration targets using the same
camera settings
4. Calibrate the camera when the application requires real-world units
•Use the calibration target image(s) collected in the previous step
• Use the simplest calibration technique that achieves the required
measurement accuracy
•Do not disturb the image acquisition setup once calibrated
5. Develop the processing script
•Use the stand-alone version of NI Vision Assistant
• Begin with the color-to-luminance step (unless the application re-
quires color, of course)
• Evaluate the performance using the representative images collected
earlier

10 CHAPTER 2. APPLICATION DEVELOPMENT FLOW
•Use the minimum number of steps that satisfies the requirements
• Periodically evaluate the computational effort of each step with the
Vision Assistant “Performance Meter”
6. Create a new NI myRIO LabVIEW project
•Connect the camera to NI myRIO if not already connected
7. Configure the Vision Acquisition Express VI
•Select the “Continuous acquisition with in-line processing” option
• Adjust the camera video mode and attributes to match the settings
recorded in Step 3
•Click “Test” to confirm that the image looks correct
•Enable the frame rate output, if desired
8. Configure the Vision Assistant Express VI
•Open the Vision Assistant script created in Step 5
• Remove the color-to-luminance step (assuming the camera is in
grayscale output mode)
•Click through the steps to confirm correct operation
•Select the necessary input controls and output indicators
•Enable the “Create Destination Image” option
•Move the Express VI inside the while-loop structure
•Create the needed controls and indicators
9. Add block diagram code for I/O devices
• Use available on-board devices as needed, e.g., LEDs, accelerometer,
PWM output for servomotors, and switches
• Refer to the NI myRIO Project Essentials Guide for detailed instruc-
tions on a wide range of useful external devices such as proximity
sensors and LCD display
10. Validate the finished application
11. Deploy the VI as a stand-alone application
•Refer to Appendix D on page 141 for complete details.

3Design Pattern for Machine Vision
Applications
A LabVIEW design pattern is a particular block diagram coding style that has
emerged over time and has been “vetted” by many developers as a useful
way to develop a broad range of specific applications. Design patterns range
from the very simple to the very complex, but all design patterns provide a
systematic way to transform your concept into a functional and reliable software
application.
This chapter introduces the popular state machine design pattern, especially
the “Queued State Machine” that serves as the foundation of the “Machine
Vision App” template LabVIEW project from which all of the projects in this
lab manual may be developed.
3.1 State Machines
The excellent textbook Effective LabVIEW Programming
1
by Thomas J. Bress
presents a wide range of design patterns based on the state machine. A state em-
bodies a specific action such as incrementing a counter value, fetching the next
frame from the webcam, analyzing the image to search for a barcode, saving an
image to disk, and so on. The state “machine” moves from one state to the next
in either an unconditional fashion (State C always follows State B) or condition-
ally (go to State C if a particular front-panel button is pressed, otherwise go to
State D). See State Diagrams
2
for a more complete discussion of state diagrams.
This “classic state machine” design pattern uses a while-loop, a shift register
1http://www.ntspress.com/publications/effective-labview-programming
,
NTS Press, 2013.
2http://youtu.be/n-wJjQKCEeo (10:06)

12 CHAPTER 3. DESIGN PATTERN FOR MACHINE VISION APPLICATIONS
to maintain the current state, and a case structure to implement the activities
for each state and also to determine the next value of the state register. On each
while-loop iteration the value of the state register is read to select a particular
subdiagram of the case structure, therefore, each subdiagram embodies a single
state. Each subdiagram calculates the next value of the state register which is
then fed back via the while-loop’s shift register to become the next value of the
state register. Refer to Classic State Machine
3
for a more complete discussion
of the LabVIEW implementation of a classic state machine.
3.2 Queued State Machine
The “Queued State Machine” (QSM) is a more sophisticated approach that
decouples a state’s activity from the mechanism to determine the next state.
This decoupling makes the QSM much easier to develop and maintain because
it promotes a “divide and conquer” approach to breaking up the application
design into manageable and well-defined activities. Each state can be developed
and debugged independently of all the other states.
But how is the next state determined? A special state called the scheduler
takes care of this. The scheduler examines image analysis results, front-panel
buttons, and external sensors to determine which task — an ordered sequence of
states — should be performed next. The scheduler applies this ordered sequence
of states comprising a task to the input of a queue, also known as a FIFO (first-in,
first-out) memory structure. Once enqueued the states are now ready to be read
out one at a time, i.e., dequeued, on each while-loop iteration to select the asso-
ciated case-structure subdiagram activities in the proper sequence. The queue
serves the same role as the state register in the classic state machine. After en-
queueing a single task, the scheduler must finish by enqueueing itself so that that
state machine will return to the scheduler state after completing the current task.
Figure 3.1 on page 19 illustrates the task diagram for a generic QSM intended
for machine vision applications that target NI myRIO. Each box corresponds to a
state, and a bundle of states indicates a task. The QSM executes the start-up task
and then schedules tasks according to conditions determined by analysis results,
front-panel buttons, and external controls and sensors. The scheduler typically
selects the default task when all other task conditions evaluate to “false.” The
default task retrieves an image from the webcam, analyzes the image, adds a
nondestructive overlay, and reads and writes NI myRIO on-board and external
devices. When the front-panel Stop button is pressed or an error condition is
detected the scheduler selects the shutdown task to halt the application.
3http://youtu.be/muudWDUkypM (6:53)

CHAPTER 3. DESIGN PATTERN FOR MACHINE VISION APPLICATIONS 13
3.3 “Machine Vision App” Template
Download the .zip file from the NI myRIO Vision Essentials Guide
4
main page,
extract the “Machine Vision App” folder, open the “machine vision app” Lab-
VIEW project (double-click the .lvproj file), and then double-click Main.vi to
open its block diagram. Take a few moments to study the structure of the block
diagram, and especially scroll through the case-structure subdiagrams to see
the various states and their associated activities.
This template provides a single starting point for each of the projects in this
lab manual. As you complete each project and develop your own library of
machine vision applications you may find it easier to work from a copy of an
existing project that is similar to the requirements of the new project. In either
case, implementing a new machine vision application will become a relatively
straightforward task once you understand the basic principles of the queued
state machine design pattern.
IMPORTANT :Do not be discouraged by the seeming complexity of the
queued state machine template. You will soon discover that the template is
easy to edit and achieves expected and reliable behavior without expending
unnecessary development effort.
QSM Structure
Figure 3.2 on page 20 shows the task diagram for the “Machine Vision App”
(MVA) template project. The task diagram shows only two tasks but the
template includes many empty states that you can adapt to the needs of your
project. The default task applies an edge detector to the webcam image and
displays either this processed image or the original webcam image along with a
nondestructive graphics overlay that shows the system date and time. Figure 3.3
on page 21 shows the state sequence for the default task.
Pressing the front-panel save image action button selects the Save Image
task (Figure 3.4 on page 22); this task saves the displayed image to an image
file on the attached USB flash drive. The action button remains pressed until
released, therefore the task releases the button as its final action. Because all tasks
that respond to action buttons must conclude in a like manner, the button release
behavior is implemented in a single state that can be included in any task.
Figure 3.5 on page 23 shows the state sequence for the start-up task. These
states initialize data values, allocate memory for image buffers, reset front-panel
buttons, and configure on-board and external devices. Figure 3.6 on page 24
4http://www.ni.com/myrio/vision-guide

14 CHAPTER 3. DESIGN PATTERN FOR MACHINE VISION APPLICATIONS
shows the state sequence for the shutdown task. These states stop the webcam,
perform general “housekeeping” to close log files and deallocate memory, and
return peripheral devices to inactive states.
Template Walk-Through Tutorials
Take the time to carefully study each of the following video tutorials. Once you
have completed this study you will understand all of the essential techniques
to edit and adapt the template project to the requirements of your project:
Develop your “big picture” understanding of the Queued State Machine
design pattern:
1. MVA Project Files and Folders
5
– Open the “machine vision app” folder,
review the files and folders, open the .lvproj file, and view the organized
files.
2. MVA Main.vi Quick Tour
6
– Quick-paced tour of the “Main” VI: while-
loop, case-structure, state queue, front-panel controls, NI myRIO ports,
and data highway.
3. MVA States
7
– Review the standard states (qsm, myRIO, image), the
example application-specific state, and the expansion states.
4. Run the MVA Project
8
– Open the “machine vision app” folder, open the
.lvproj file, open Main.vi, run the VI, demo the buttons, and stop. NOTE:
The webcam must already be configured!
5. “Initialize Queue” VI9– Walk through the “initialize_queue” VI.
6. “Enqueue States” VI10 – Walk through the “enqueue_states” VI.
7. “Dequeue State” VI11 – Walk through the “dequeue_state” VI.
Template Editing Tutorials
These detail-oriented tutorials illustrate all of the essential techniques necessary
to adapt the template to your needs:
5http://youtu.be/AekhpmU9s3Y (2:04)
6http://youtu.be/sgGgU2dkocI (4:52)
7http://youtu.be/IKFeOF4Zh6Y (2:18)
8http://youtu.be/MITs9SQz-Pg (1:38)
9http://youtu.be/p97xHbXmMgg (1:20)
10http://youtu.be/AFg-_tv9MVU (2:16)
11http://youtu.be/-uIB_5KlstQ (2:30)