InterSense InertiaCam IS-1500 User manual

MNL-0024 (C)
IS-1500 NATURAL FEATURE TRACKER
USER GUIDE
© 2017 Thales Visionix, Inc.
700 Technology Park Drive, Suite 102
Billerica, MA 01821 USA
Phone [phone] • Fax [phone]
www.intersense.com

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 2 of 59
Contacting Thales Visionix
Please contact us if you need assistance.
Thales Visionix, Inc.
700 Technology Park Drive, Suite 102
Billerica, Massachusetts 01821
USA
Telephone: [phone]
Internet: http://www.intersense.com
Fax: [phone]
Technical Support: [phone]
email: techsuppo[email protected]
Sales: [phone]
email: [e-mail]
The label below identifies the protection granted by the Government of the United States to Thales Visionix for
InterSense products:
Trademarks
InterSense® is a registered trademark of Thales Visionix, Inc. InertiaCam™, InertiaCube™, VETracker™, and SimTracker™
are trademarks of Thales Visionix, Inc. All other trademarks are the property of their respective owners.
Copyright 2017
U.S. Patents
5645077, 5807284, 6162191, 6361507, 6786877, 6176837, 6409687, 7395181, 6314055 B1, 6757068 B2,
7301648, 6474159 B1, 6681629 B2, 7000469 B2, 7231063 B2, 6922632 B2, 7725253, 7900524, 8224024,
8473241, 8696458, 8762091, 8972182 and Patents Pending.
Foreign Patents
Europe 1280457 1071369 Israel: 152359 Taiwan: 162248
China: 99807510.8 Hong Kong: 1039884 Japan: 4690546.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 3 of 59
Regulatory Statements and Approvals
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two
conditions: (1) this device may not cause harmful interference, and (2) this device must accept any
interference received, including interference that may cause undesired operation.
This product meets the applicable Industry Canada technical specification.
"CE" mark indicates that this product complies with the European requirements for safety, health,
environment and customer protection.
Classified by UL to 60950-1.
Bureau Veritas NRTL Listing.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 4 of 59
Thales Visionix, Inc. End User License Agreement
This License Agreement (“Agreement”) is an agreement between you (“Licensee”) and Thales Visionix, Inc. (“Licensor”). It governs
use of the software supplied by Thales Visionix, Inc. ("the Software") and related documentation. By downloading, installing or
otherwise using the Software, the user agrees to be legally bound by the terms of this Agreement. Please read the entire contents of
this agreement. THIS SOFTWARE IS COPYRIGHT © 2017 by Thales Visionix, Inc. ALL RIGHTS RESERVED WORLD WIDE.
IMPORTANT NOTE
Backups of any important information, valuable data, or Software on the host PC should be made before installing this or any other
software. Also, regular backups of the Software and any data files should be made, so that in the event of data loss, information may
be restored.
1. DEFINITION
(a) “Documentation” means the standard end user manual for the Product provided by Licensor.
(b) “Product” means all Licensor equipment including, but not limited to, the InertiaCube, IS-900, and IS-1500 product families.
(c) “Software” means all Licensor software code including sample source code, compiled code, and embedded firmware.
2. LICENSE
(a) Grant of Rights. Licensor hereby grants to Licensee a non-exclusive, royalty-free, perpetual license to use the Software solely with
Licensor Product and solely for the operation of Product in accordance with its documentation. Licensee may redistribute Licensor
libraries, isense.dll, libisense.so, libisense.dylib, sfAccess.dll and libsfaccess.so as required for the operation of Licensee products.
(b) Restrictions. The Software (and any copy thereof) is licensed not sold, and Licensee receives no title to or ownership of the
Software and no rights other than those specifically granted in Section 2(a) above. With the exception of Licensor sample source
code, Licensee may not modify, reproduce, and create derivative works of the Software and shall not attempt to decompile or
otherwise reverse engineer the Software. Licensee shall not remove any proprietary or copyright notices of Licensor in the Software
or any copies thereof.
3. WARRANTY
This software and accompanying written materials (including instructions for use) are provided "as is" without warranty of any kind.
Further, Licensor does not warrant, guarantee, or make any representations regarding the use, or the results of use, of the software
or written materials in terms of correctness, accuracy, reliability, currency, or otherwise. The entire risk as to the results and
performance of the software is assumed by Licensee. If the software or written materials are defective, Licensee, and not Licensor or
its dealers, distributors, agents, or employees, assume the entire cost of all necessary servicing, repair, or correction.
The above is the only warranty of any kind, either express or implied, including but not limited to the implied warranties of
merchantability and fitness for a particular purpose, which is made by licensor on this product. No oral or written information or
advice given by licensor, its dealers, distributors, agents or employees shall create a warranty or in any way increase the scope of
this warranty and the user may not rely on any such information or advice.
4. LIMITATION OF LIABILITY
Neither licensor nor anyone else who has been involved in the creation, production or delivery of this product shall be liable for any
direct, indirect, consequential or incidental damages (including damages for loss of business profits, assets, business interruption,
loss of business information, and the like) arising out of the use or inability to use such product even if licensor has been advised of
the possibility of such damages.
5. TERMINATION
(a) Termination. This Agreement shall terminate automatically and immediately upon any breach of this agreement by Licensee,
including but not limited to any action in violation of the license rights and restrictions set forth in Section 2.
(b) Effects of Termination. Upon termination of this Agreement, the licenses granted in Section 2(a)
will terminate and Licensee will cease all use of the Product and delete all copies of any Software (including Upgrades and Updates
stored on Licensee computers) in its possession or control. This Section 5(b) shall survive termination of this Agreement.
6. MISCELLANEOUS
(a) Notices. Notices to Licensor pursuant to this Agreement will be sent to 22605 Gateway Center Drive, Clarksburg, Maryland
20972, Attention: Contracts Department. Such notices will be deemed received at such addresses upon the earlier of (i) actual
receipt or (ii) delivery in person, or by certified mail return receipt requested.
(b) U.S. Government Use. The Software is a “commercial item” as that term is defined at 48 C.F.R.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 5 of 59
2.101, consisting of “commercial computer software” and “commercial computer software documentation” as such terms are used
in 48 C.F.R. 12.212.
(c) Export. Licensee understands and recognizes that the Product is export-controlled and agrees to comply with all applicable export
regulations. The Product may not be exported to any country outside the country where the Product was acquired without
Licensor’s prior written consent.
(d) Assignment & Successors. Licensee may not assign this Agreement or any of its rights or obligations hereunder.
(e) Choice of Law & Jurisdiction. This Agreement will be governed by the laws of the United States of
America.
(f) Severability. To the extent permitted by applicable law, the parties hereby waive any provision of law that would render any
clause of this Agreement invalid or otherwise unenforceable in any respect. In the event that a provision of this Agreement is held to
be invalid or otherwise unenforceable, such provision will be interpreted to fulfill its intended purpose to the maximum extent
permitted by applicable law, and the remaining provisions of this Agreement will continue in full force and effect.
(g) Entire Agreement. This Agreement sets forth the entire agreement of the parties and may not be modified except by a written
agreement signed by Licensor.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 6 of 59
TABLE OF CONTENTS
1. INTRODUCTION .........................................................................................................................................................................9
1.1 OVERVIEW OF THE IS-1500..............................................................................................................................................................9
UNDERSTANDING TRACKING ...................................................................................................................................................11
2.1 TRACKING BASICS..........................................................................................................................................................................11
2.2 ACHIEVING ACCURATE 6-DOF TRACKING ..........................................................................................................................................14
2.2.1 3-DOF Tracking from the NavChip....................................................................................................................................14
2.2.2 VINS Natural Feature Tracking......................................................................................................................................... 15
2.2.3 PRA Fiducial Tracking ....................................................................................................................................................... 19
2.3 SUMMARY ...................................................................................................................................................................................21
INSTALLATION .........................................................................................................................................................................22
3.1 INSTALLING THE SOFTWARE SUITE ....................................................................................................................................................22
3.1.1 Windows .......................................................................................................................................................................... 22
3.1.2 Linux ................................................................................................................................................................................. 23
3.2 CONNECTING USING SFSTUDIO ........................................................................................................................................................24
3.3 TRACKING USING VINS IN TRACKING STATE 1..................................................................................................................................... 25
3.4 TRACKING USING FIDUCIALS IN TRACKING STATE 2 ..............................................................................................................................26
3.5 USING A CUSTOM FIDUCIAL CONSTELLATION......................................................................................................................................28
SOFTWARE...............................................................................................................................................................................29
4.1 SOFTWARE FLOW CHART................................................................................................................................................................29
4.2 SFHUB ........................................................................................................................................................................................29
4.3 INTERFACING WITH TRACKING DATA .................................................................................................................................................29
4.4 SFSTUDIO ....................................................................................................................................................................................30
4.4.1 sfStudio Toolbar ...............................................................................................................................................................31
4.4.2 sfStudio Sidebar................................................................................................................................................................31
4.4.3 Tracking Data Display ......................................................................................................................................................32
4.4.4 Optical Data Display......................................................................................................................................................... 33
4.4.5 3D Data Display................................................................................................................................................................ 34
4.4.6 Tracker Sigma Data Display .............................................................................................................................................35
4.4.7 Make Fiducials Tool..........................................................................................................................................................36
4.4.8 Simple Visual Mapper Tool............................................................................................................................................... 37
ADVANCED SETTINGS AND CONFIGURATION FILES..................................................................................................................38
5.1 INTRODUCTION............................................................................................................................................................................. 38
5.2 EDITING CONFIGURATION FILES ....................................................................................................................................................... 38
5.3 SFHUB.INFO.................................................................................................................................................................................39
5.3.1 Logging and Playback of Tracking Data ...........................................................................................................................39
5.3.2 Image Resolution in the Configuration Utility .................................................................................................................. 43
5.3.3 InertiaCam Exposure Settings ..........................................................................................................................................44
5.4 SFACCESS.INI................................................................................................................................................................................ 46
5.4.1 Boresight and Tip Offset...................................................................................................................................................46
5.4.2 Changing TCP and UDP Settings in sfAccess.ini and sfHub.ini .........................................................................................47
5.5 SFSTUDIO.INI................................................................................................................................................................................50
5.5.1 sfStudio Connect Button Settings for sfHub AutoLaunch .................................................................................................50
5.5.2 sfStudio Light and Dark Themes....................................................................................................................................... 51
USING HOBIT SENSORS WITH THE IS-1500 SOFTWARE SUITE...................................................................................................52
6.1 DIFFERENCES AND LIMITATIONS .......................................................................................................................................................52
6.1.1 Natural Feature Tracking .................................................................................................................................................52
6.1.2 isUnitySample...................................................................................................................................................................52
6.1.3 sfStudio Features..............................................................................................................................................................52
6.2 CONFIGURING SFHUB TO CONNECT TO A HOBIT.................................................................................................................................53

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 7 of 59
IR INERTIACAM ........................................................................................................................................................................54
7.1 DATA..........................................................................................................................................................................................54
7.2 CONFIGURATION SETTINGS FOR IR INERTIACAMS ................................................................................................................................55
7.2.1 Enabling and Disabling the IR Illuminator........................................................................................................................ 55
7.2.2 IR Retroreflective Fiducial Recognition............................................................................................................................. 55
TROUBLESHOOTING.................................................................................................................................................................56
8.1 FREQUENTLY ASKED QUESTIONS ...................................................................................................................................................... 56
GLOSSARY................................................................................................................................................................................59
9.1 ACRONYMS..................................................................................................................................................................................59
9.2 TERMS ........................................................................................................................................................................................59

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 8 of 59
TABLE OF FIGURES
FIGURE 1–INERTIACAM ..................................................................................................................................................................................9
FIGURE 2–IS-1500 SYSTEM ARCHITECTURE.....................................................................................................................................................10
FIGURE 3–ROTATIONAL AXES ........................................................................................................................................................................11
FIGURE 4–INERTIACAM DIMENSIONS..............................................................................................................................................................12
FIGURE 5–INITIALIZED REFERENCE FRAMES ...................................................................................................................................................... 12
FIGURE 6–TRACKED REFERENCE FRAME ..........................................................................................................................................................13
FIGURE 7–VINS FEATURES ...........................................................................................................................................................................15
FIGURE 8–VINS HAND WAVING TEST.............................................................................................................................................................16
FIGURE 9–OPTIMAL 20° PITCH......................................................................................................................................................................17
FIGURE 10 –EXERCISING AN IMU ...................................................................................................................................................................17
FIGURE 11 –DIVERGENCE ..............................................................................................................................................................................18
FIGURE 12 –CONTINUED DIVERGENCES............................................................................................................................................................18
FIGURE 13 –DEFAULT FIDUCIAL CONSTELLATION ...............................................................................................................................................19
FIGURE 14 –FIDUCIAL OVERLAY...................................................................................................................................................................... 19
FIGURE 15 –START MENU FOLDERS ................................................................................................................................................................22
FIGURE 16 –SOFTWARE DIRECTORY ................................................................................................................................................................22
FIGURE 17 –LINUX .......................................................................................................................................................................................23
FIGURE 18 –MAIN SFSTUDIO TOOLBAR............................................................................................................................................................24
FIGURE 19 –TRACKING DATA DISPLAY .............................................................................................................................................................24
FIGURE 20 –VINS IMAGE..............................................................................................................................................................................25
FIGURE 21 –3D DATA DISPLAY.......................................................................................................................................................................25
FIGURE 22 –FIDUCIAL OVERLAY...................................................................................................................................................................... 26
FIGURE 23 -TRACKING STATE CHANGES............................................................................................................................................................26
FIGURE 24 –TRACKING STATE 2RELOCALIZATION ...............................................................................................................................................27
FIGURE 25 –MAIN TOOLBAR..........................................................................................................................................................................31
FIGURE 26 –SIDEBAR....................................................................................................................................................................................31
FIGURE 27 –TRACKING DATA DISPLAY .............................................................................................................................................................32
FIGURE 28 –OPTICAL DATA DISPLAY................................................................................................................................................................33
FIGURE 29 –3D DATA DISPLAY.......................................................................................................................................................................34
FIGURE 30 –NFT SIGMAS DISPLAY.................................................................................................................................................................. 35
FIGURE 31 –MAKE FIDUCIALS TOOL ................................................................................................................................................................36
FIGURE 32 –SVM TOOL ...............................................................................................................................................................................37
FIGURE 33 –SFHUB TRACKING........................................................................................................................................................................40
FIGURE 34 –SFHUB PLAYBACK MENU ..............................................................................................................................................................41
FIGURE 35 –LOG PLAYBACK ...........................................................................................................................................................................42
FIGURE 36 –SETTING INERTIACAM IMAGE RESOLUTION ......................................................................................................................................43
FIGURE 37 –MANUAL EXPOSURE 5000µS........................................................................................................................................................45
FIGURE 38 –SFSTUDIO DARK THEME ............................................................................................................................................................... 51
FIGURE 39 –IR INERTIACAM ..........................................................................................................................................................................54
FIGURE 40 –IR INERTIACAM IMAGE IN A BRIGHTLY LIT ROOM (LEFT)VS AN UNLIT ROOM (RIGHT) ............................................................................. 54

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 9 of 59
1. INTRODUCTION
1.1 Overview of the IS-1500
The IS-1500 Natural Feature Tracker (NFT) is a motion tracking system that uses both visual and inertial input to calculate and
output position and orientation data. The system enables GPS-denied tracking and navigation in a free-roaming environment. The IS-
1500 kit includes an InertiaCam tracker, a USB-C 2.0 cable, a fiducial poster, and a sensor fusion software suite.
The InertiaCam is an optical-inertial 6-DOF (six degrees of freedom) motion tracker. It pairs a monocular camera with a specially
developed InterSense inertial measurement unit (IMU), known as the NavChip. There is a female USB-C connector at the base of the
InertiaCam used to connect to the host computer.
InertiaCams have limited internal processing capability. The majority of image and inertial data processing is performed by the
sensor fusion software installed on the host computer. The flow chart in Figure 2 provides an overview of the IS-1500 system
architecture. The arrows indicate the flow of the data processing chain, beginning with sensory input from the InertiaCam and
ending with the output of tracking data.
Figure 1 –InertiaCam

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 10 of 59
Figure 2 –IS-1500 System Architecture
As the InertiaCam receives optical data from the camera and inertial data from the NavChip, it transmits the information directly to
the sfHub program. sfHub is responsible for the bulk of the processing of the raw sensor data. It outputs packets of processed
tracking data, which are then transmitted to sfAccess.
sfAccess provides a low-level interface to the InertiaCam and sfHub and is only recommended for users requiring advanced features
of the IS-1500. The InterSense library can be used instead if the host application simply needs tracking output or if compatibility
with other InterSense tracker models is required. The suite also comes with an IS-1500 Unity Plugin, allowing users to directly
integrate the IS-1500 into their Unity applications.
sfStudio is used as the primary program for device setup, display of tracking information, and general diagnostics. sfStudio can also
be used to show the tracking status, provide graphical data plots such as tracker position and orientation over time, and display the
live camera feed from the InertiaCam.
With regards to terminology throughout the rest of this user guide, the term IS-1500 will refer to the system as a whole. The system
encompasses the InertiaCam, host computer, and full software suite. The InertiaCam may be referred to as simply the tracker.
Similarly, the user host computer may be referred to as the processor. The software components will typically be identified
specifically, although sfAccess, InterSense library, and Unity plugin may be generalized as the tracking interface as needed. As will be
discussed later in the Understand Tracking section, the IS-1500 is capable of several tracking modes, though these may be
collectively known as the tracking algorithms.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 11 of 59
UNDERSTANDING TRACKING
This section is intended for those users who wish to better understand the tracking data produced by the IS-1500 and how it acquires
that information. It starts with the basics by briefly reviewing common terms and the Cartesian coordinate reference frame. From
there it moves on to discuss how the IS-1500 uses the InertiaCam’s NavChip IMU and camera to track its position and orientation.
In explaining how the tracking works, there are many general tips for usage that come up in this section. A brief bullet list of these tips
is provided in the Summary subsection.
This section uses many graphs and screenshots from the sfStudio program to illustrate the topics covered. For users who wish to follow
along and view live motion tracking data, it may be beneficial to first go through the IS-1500 Quick Start Guide or the Installation
section to set up the system before proceeding.
2.1 Tracking Basics
The tracking environment (or tracking area) broadly refers to the area the IS-1500 is being used in. The tracking data produced by
the IS-1500 uses a Cartesian (rectangular) coordinate reference frame to describe position and orientation within the tracking
environment.
The tracking data produced by the IS-1500 reflects the InertiaCam’s position and orientation, as defined by the tracker reference
frame, relative to the world reference frame. The X, Y, and Z position of the tracker is reported in meters. Orientation is described
using Euler angles to indicate roll, pitch, and yaw.
As illustrated in Figure 3, roll, pitch, and yaw are the rotations around the X, Y, and Z axes of the world reference frame. As such, roll
changes the heading of the YZ plane of the tracker reference frame, pitch changes the XZ plane, and yaw changes the XY plane.
Figure 3 –Rotational Axes
An origin point and X, Y, and Z axes for the world reference frame are defined by the system when it is initialized. The origin point
defines the location in the tracking environment at which (X, Y, Z) is (0, 0, 0). When the IS-1500 is first initialized, the only sensory data
it can rely on to create a world reference frame is the direction of gravity. Therefore, it initializes using the accelerometers to determine
the direction of gravity. The Z axis is always in the direction of gravity, defining pitch and roll, while the direction of the X axis, defining
yaw, is effectively arbitrary. Using a right handed coordinate reference frame, the Y direction is 90° clockwise along the X axis. By
default, the IS-1500 sets the world origin point as the InertiaCam’s position when tracking is first initialized. As will be discussed in the
PRA Fiducial Tracking subsection, this initial world reference frame can be changed later by using a fiducial constellation to provide
the system with a global point of reference.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 12 of 59
Figure 4 –InertiaCam Dimensions
The tracker reference frame (or body frame)is a predetermined point and orientation on the InertiaCam and stays fixed to it. It is
indicated by markings on the InertiaCam housing, shown in Figure 4.The XY marking at point A on the front of the housing indicates
the orientation of the positive X and Y axes. The positional origin is located and marked at the back of the housing as the corner of the
rectangle, indicated by point B. (Like the markings on the front, the long and short sides of the rectangle on the back also indicate the
direction of the positive X and Y axes, respectively).
To help illustrate this, the sfStudio 3D Data Display in Figure 5 shows a tracker that has just been initialized with the InertiaCam laying
on a flat surface so the Z axis of the tracker is in the direction of gravity, the same as the world frame. The world frame in this image
has been marked in yellow. Gray gridlines lie across as the XY plane. The axes of the body frame are colored green, blue, and red for
X, Y, and Z, respectively. Note that the X and Y axes of the tracker may not line up exactly with those of the world frame.
Figure 5 –Initialized Reference Frames

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 13 of 59
Figure 6 shows another example. Here, the tracker has been moved from its initial position. Instead of lying flat with the camera
pointed up, it has been rotated into a more vertical position and moved along an arc towards a position somewhere above the
starting point. The thin yellow line shows the path the tracker took. The world frame has remained stationary, but the body frame
has moved with the InertiaCam.
Figure 6 –Tracked Reference Frame

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 14 of 59
2.2 Achieving Accurate 6-DOF Tracking
The IS-1500 uses optical-inertial tracking, meaning that it uses visual data from the camera along with inertial data from the NavChip
IMU to acquire more accurate positional information. As the IS-1500 Natural Feature Tracker name indicates, it uses natural features
it identifies in the environment to track off of. In this context, natural features are those which the system determines to be still or
motionless. However, the IS-1500 can also use fiducial tracking, which was originally developed for earlier optical-inertial trackers like
the IS-1200 and HObIT.
2.2.1 3-DOF Tracking from the NavChip
As previously mentioned, the InertiaCam tracker included with the IS-1500 kit is a 6-DOF (rotation and position) tracker with a
monocular camera and a built-in NavChip IMU. The NavChip IMU is the source of high precision 3-DOF (three degrees of freedom-
yaw, pitch, and roll) tracking data. The accuracy of the data produced by the NavChip contributes greatly to the precision performance
of the IS-1500 as a whole.
The NavChip measures changes in acceleration and rotation by using an array of individually characterized, semi-redundant
accelerometers and gyroscopes. The angular rate and linear acceleration data from the sensors is integrated to produce the standard
ΔΘ (angular displacement) and ΔV (change in velocity) IMU outputs. These outputs are integrated at 200Hz to track orientation and
position.
This data allows the NavChip to also function as an attitude and heading reference system (AHRS)to measure yaw, pitch, and roll.
The accelerometers provide measurements of pitch and roll by calculating the proportion of gravity along those vectors. However,
yaw lies in the XY plane and gravity is along the Z axis, so accelerometers cannot be used to measure yaw. Because of this, gyroscopes
are also used, which are capable of measuring rotation about each axis. While gyroscopes are less accurate than accelerometers on
the whole, they serve the dual purpose of measuring yaw and providing supplemental pitch and roll data to increase overall accuracy.
This is why IMU specifications for pitch and roll accuracy and drift are typically listed separately from yaw, which is prone to drift if not
corrected by external data.
While IMUs provide accurate orientation data, even the high precision NavChip cannot be used alone to acquire accurate 6-DOF
positional data. Every IMU is prone to a certain degree of drift, especially in yaw, and while the NavChip has been specifically designed
to minimize drift, it does not eliminate it. The drift errors are not as problematic when performing a single integration to find
orientation. However, acquiring position information requires a double integration of the sensor data, allowing drift to accumulate
quadratically with time. An example of this is illustrated towards the end of the VINS Natural Feature Tracking subsection.
In order to obtain accurate 6-DOF tracking, an external source of positional reference is also required.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 15 of 59
2.2.2 VINS Natural Feature Tracking
To track off of natural features, the IS-1500 uses a computationally lightweight Vision-aided Inertial Navigation System (VINS)
algorithm. When an image from the InertiaCam camera is received by the sfHub software, it is processed to isolate particular pixel
patches that the system considers unique and identifiable. Frequently the patches will be found in regions of sharp contrast, such as
corners or edges. These pixel patches are the natural features. Figure 7 shows the Optical Data display with Image Transfer and VINS
Image enabled. It illustrates the features found by VINS while viewing a collection of objects. Each blue box outlines a pixel patch
feature with a unique ID in green.
Figure 7 –VINS Features
As the camera continues to provide subsequent frames, they are analyzed to relocate the natural features. If they can’t be located or
become compromised, they are replaced with new features. Reasons for this might be that the feature has moved out of the frame
and is no longer visible, or that it is simply unrecognizable. (Note that many of the features in Figure 7 are partially identified by a
glare, such as 13686 on the tree branch of the ceramic plate. If the light causing the reflection were blocked, that feature would likely
become unrecognizable.) The system also has a preference for ‘natural’ (immobile) features. If a feature previously thought to be
motionless is eventually determined to be mobile, it will soon be discarded in favor of a stationary feature.
With VINS, data on each feature is not saved to permanent memory. Data is only maintained for currently tracked features. This
means that if the camera is turned away for several seconds and brought back to the same field of view, the previous features will not
be identified again. Even if the same pixel patch is identified as a feature, it will have a new feature ID and any data describing it will
be generated from scratch.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 16 of 59
The VINS algorithm is robocentric, meaning that internally the features are described with a distance and bearing vector relative to
the tracker reference frame. Humans rely on binocular vision for depth perception and estimation of distances. However, the IS-1500
uses the monocular InertiaCam. To make up for this, the distance of a feature from the InertiaCam is calculated by using parallax.
Upon first finding features, the system can only roughly estimate their distance from the camera. As the InertiaCam is moved from
side to side, the movement of the features from frame to frame is compared to the precise velocity data from the NavChip. With this
information, the IS-1500 can accurately determine the distance of each feature. In practical usage, this means that the more the
InertiaCam moves about a feature, the more the system will learn about it, improving the precision of the tracking data.
Incidentally, this is also how the system determines that a feature is mobile. If the feature is moving, its distance and bearing will not
correlate with the IMU data, and it will eventually be rejected. It is simple to demonstrate this by placing a hand in the InertiaCam’s
field of view. If the hand is kept still, it is likely to be used as a source of natural features, as seen on the left of Figure 8. However,
when the hand is waved while the InertiaCam is kept still, the features are soon discarded and replaced by alternatives external to the
hand. When the system is uncertain about the feature depth, this will be indicated by a red circle or oval, as seen in features 2172 and
2173. The longer the oval, the greater the degree of uncertainty.
Figure 8 –VINS Hand Waving Test
If both the hand and the InertiaCam are in motion, it is less likely the features will be discarded. Instead the system will continue trying
to track off the mobile features. This will feed false pose data to the system and may cause problems with tracking. For accurate
tracking, if there are moving objects in the tracking area, it is generally best to keep them out of the InertiaCam’s field of view. If
this is not possible, keep the InertiaCam still so the system can discard mobile features and replace them with natural features.
The depth estimates calculated for each feature carry a degree of possible error. The amount of possible error scales with the distance
of a feature from the InertiaCam. This means that while features can be found a long distance away on the horizon, tracking off of
them may not yield reliable data. Beyond about two feet, the greater the distance between the detected natural features and the
InertiaCam, the less reliable the tracking data will be. However, features that are further away also typically leave the field of view less
frequently, providing more stable points of reference. A general guideline is to keep the InertiaCam field of view pitched about 20°
below the perpendicular of the horizon line while tracking. This will allow the majority of features to be found on the ground and in
nearby surroundings with a few still on the horizon. At walking speed, the features on the ground will be in enough frames to provide
accurate depth information before leaving the field of view, while features on the horizon provide stability.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 17 of 59
Figure 9 –Optimal 20° Pitch
The VINS algorithm maintains an extended Kalman filter (EKF)to enable bias correction and a certain level of movement prediction.
The EKF is state based, meaning that it is updated with each new tracking calculation. This also means that when tracking is first
initialized, the filter does not have much data and can be less reliable. To ensure the EKF receives more reliable data upon
initialization, keep the tracker still while tracking is initializing (before data begins streaming from sfHub).
To help optimize tracking performance, it is good practice to begin tracking by exercising the InertiaCam’s NavChip. To do this, begin
by placing the InertiaCam on a level surface with the camera facing up. Proceed to rotate the InertiaCam about 90° in each axis, leaving
it still for five seconds or more between each rotation. Be sure not to obstruct the camera during this process. Exercising the IMU
allows the EKF to begin accumulating IMU bias data and apply accurate corrections to the raw data.
Figure 10 –Exercising an IMU
If optical information in the VINS filter is determined to be unreliable due to a particularly high error estimates in the EKF, data
from the IMU will carry more weight in the tracking calculations. For instance, when the InertiaCam is rotated very quickly, or if its
vision is temporarily obstructed, VINS tracking will discard all of the features it had previously identified and will need to characterize
a completely new set of features. Because the system doesn’t have known distances for these features yet, the quality of the tracking
data may slightly and temporarily decrease. However, the EKF data will allow the system to rely more heavily on IMU data while the
new VINS features are established. This allows the system to stay relatively stable and continue tracking accurately under these
conditions.
If the InertiaCam’s vision is obstructed for long periods of time, the system will begin to rely almost entirely on the IMU data from
the NavChip. As discussed earlier, when the IMU is used exclusively to provide position, there is a high rate of drift. As an example,
after connecting to the InertiaCam and beginning tracking, the camera of the InertiaCam is covered. Soon after this, the tracker’s
position can be seen drifting away at an increasingly rapid rate. This behavior is known as a divergence. The software is designed to
recognize this divergence and counter it by resetting the filter and returning the tracker position back to the origin. Figure 11 shows
the 3D Data Display of a divergence that has resulted in a filter reset.

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 18 of 59
Figure 11 –Divergence
The tracker will continue to diverge and reset until vision is restored and natural features can be found again, as shown in Figure 12.
When the divergence is detected and the filter reset, the system will place the position back at the last location accurate data was
recorded. Orientation will be relatively unaffected. Because the filter is reset, when recovering from a divergence, it is best to treat
the tracker as though it was just initialized. This means while the system recovers from a divergent state after the camera has been
obstructed, it is best to keep the tracker still at first, then exercise it or (if using a fiducial constellation to set a world frame) bring
fiducials in view.
Figure 12 –Continued Divergences

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 19 of 59
2.2.3 PRA Fiducial Tracking
InterSense optical inertial trackers such as the IS-1500 are capable of tracking off of fiducials like the
ones found on the poster included with the hardware kit. InterSense fiducials are circular black-and-
white patterns that act as known physical landmarks for the system. A fiducial constellation refers
to the layout of all fiducials in a tracked area. When placed on a flat, level surface, the poster serves
as the default fiducial constellation for the IS-1500 system. Constellations and their fiducials provide
predetermined points of reference for the tracker. Fiducial constellations enable another level of
drift correction and set a fixed environment reference frame for the system.
The circular patterns of the fiducials are key to their functionality. The three circles within the pattern
(the center eye and two outer eyes) are what the system’s image recognition component uses to
recognize a pattern as a fiducial. These circles also serve as the tracked points. Damage to any of
these three points, especially the center eye, can render a fiducial unidentifiable to the system.
Because the InertiaCam is a visible light tracker reading black-and-white fiducials, uneven lighting
across the constellation can also render the fiducials unidentifiable during pattern recognition.
The rest of the pattern acts like a barcode. The black and white sections are decoded by the software
and translated to a numeric value. The numerical value is known as the fiducial ID. In a single
constellation, no two fiducials can share a fiducial ID. They must all have a unique ID and therefore a unique pattern. Figure 14
shows the constellation as seen in the Optical Data display. The fiducial overlays shows a square around each recognized fiducial with
crosshairs for each eye. The numbers indicate the fiducial IDs.
Figure 14 –Fiducial Overlay
Figure 13 –Default
Fiducial Constellation

IS-1500 User Guide
Thales Visionix, Inc. MNL- 0024 (D) Page 20 of 59
The algorithm used by the IS-1500 for fiducial tracking is called the Pose Recovery Algorithm (PRA). Both VINS and PRA are run
concurrently while the system is tracking, allowing the IS-1500 to use both natural feature tracking and fiducial tracking
simultaneously.
The PRA uses the constellation map file (environmentPSEs.cfg, located in the ExampleConstellation folder) as a reference. The
constellation map file acts as a look-up table and includes a list of the IDs of the fiducials in the constellation and their pose relative
to a world reference frame. While tracking, the PRA thread cross-references the observed fiducials with their expected layout. The
tracker’s position and orientation relative to the constellation is then calculated and updated. As far as the system is concerned, the
constellation is defined by the map file. If a fiducial is listed in the map file but is never recognized by the system, it will simply never
be used for tracking. Additionally, if a fiducial is identified by PRA but the ID is not listed in the map file, it will be disregarded. However,
if a fiducial is detected, the ID is in the map file, and it is in a different position and orientation than what is described in the map
file, tracking can become compromised. Also note that the poster and its constellation must be oriented correctly relative to gravity.
If the poster is placed on the ceiling or floor (sideways or upside down), the system will not track properly unless the constellation
file is modified accordingly. The constellation file can be rotated in the included sfStudio application’s constellation display if such a
configuration is desired.
PRA data is only used when the system is able to obtain optical lock on the constellation. There are a number of requirements for
optical lock to be achieved. Most importantly, the system must be able to recognize at least four fiducials simultaneously to have
enough data points to acquire optical lock. There are also a number of quality metrics that must be met, but it is possible to obtain
optical lock on an incorrect constellation.
In practical usage, when placed in the tracked area, a constellation serves two main purposes. The first is that the constellation
defines a fixed world reference frame. Whenever the constellation is first brought into view and optical lock is acquired, the world
reference frame changes to be relative to the constellation rather than relative to wherever tracking was initialized. (This is illustrated
further in the NFT Status change from 1 to 2 in the Installation section.) If the entire constellation is moved, the world reference frame
will move with it. This change happens when the system has acquired.
The second use for fiducial tracking is that it provides another level of drift correction. When using only VINS tracking, there is a
certain degree of expected drift- about 1% of the distance travelled for distances under 200 meters. For example, if the user tracks for
50 meters in a circle, the system may report a position that is up to half a meter from the starting position upon return. However,
whenever a fiducial or constellation is brought back into view, this drift is corrected since the tracker position is now known relative
to a fixed point in the environment. Continuing the example, if there is a fiducial constellation at the starting position, after tracking a
200 meter circle, VINS may have allowed the starting position to drift approximately two meters. When the constellation is recognized,
however, the position will snap back to the actual starting position. Also, the Extended Kalman Filter will reset when the PRA achieves
optical lock on the constellation, which will correct any aberrant behavior. Exercising the tracker is not necessary when using a fiducial
constellation.
Using a constellation is recommended for applications that require tracking over large expanses or in cases where the world
reference frame needs to be precisely defined. If the provided constellation poster does not meet the needs of an application, a
custom constellation and corresponding map file can be created. Instruction for how to create a custom constellation is provided in
the Fiducial Constellation Guide. When using the IS-1500, constellation density requirements are not as strict as they are for other
InterSense optical-inertial trackers. For example, rather than covering a continuous area such as a poster or a room, fiducials can
instead be placed in clusters at key landmarks throughout a commonly tracked area.