FLIR ADK User manual

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
1
Getting Started guide for the FLIR ADK with USB, GMSL, and Ethernet Connector Options
Document Number: 102-2013-105
Version: 140
Issue Date: June 2020
FLIR Systems OEM & Emerging
6769 Hollister Avenue
Goleta, CA 93117
Phone: [phone]
www.flir.com

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
2
Table of Contents
1Introduction.................................................................................................................................................................... 3
1.1 Revision History..................................................................................................................................................3
1.2 Reference Documents......................................................................................................................................3
1.3 SCOPE.........................................................................................................................................................................4
1.4 BACKGROUND ...................................................................................................................................................... 4
2USB –Getting Started................................................................................................................................................ 4
2.1 Connect and power the USB ADK............................................................................................................. 4
2.2 FLIR Boson GUI.................................................................................................................................................... 5
2.3 External Sync......................................................................................................................................................... 7
3Ethernet –Getting Started......................................................................................................................................7
3.1 Connecting the Boson ethernet ADK to the computer.................................................................7
3.2 Ubuntu Installation............................................................................................................................................8
3.2.1 Robot Operating System.......................................................................................................................9
3.3 Windows Installation ....................................................................................................................................10
3.4 Working with the Spinnaker API...........................................................................................................10
3.4.1 Accessing Boson Camera Feature Example...........................................................................10
3.4.2 Finding Node Names, Types and Enumeration Values................................................... 11
4GMSL –Getting Started .........................................................................................................................................12
4.1 Power Over Coax..............................................................................................................................................12
4.2 NVIDIA Drive Systems..................................................................................................................................14
4.3 Upgrading the GMSL Serializer to GMSL 2.......................................................................................15
4.4 GMSL SerDes Data Transmission...........................................................................................................15
4.5 Sending Commands to the ADK over i2C.......................................................................................... 16
4.6 Sample I2C Functions for Teensy SOC................................................................................................17
4.7 Data Format and Boson Settings for GMSL 1..................................................................................20
4.8 Interpreting the Multiplexed/ Double wide IR16 video..........................................................20
4.9 Syncing the Boson using GPIO pins of the SerDes....................................................................... 20
5ADK Latency.................................................................................................................................................................21
5.1 Microbolometer Thermal Time Constant.........................................................................................21
5.2 Progressive Video Output........................................................................................................................... 22

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
3
5.3 ADK Video Processing Pipeline............................................................................................................... 22
5.4 USB latency.......................................................................................................................................................... 24
5.5 GMSL latency...................................................................................................................................................... 24
5.6 Ethernet latency ............................................................................................................................................... 24
6Intrinsic Camera Calibration.............................................................................................................................. 25
6.1 Building a Calibration Target................................................................................................................... 25
6.2 Running OpenCV algorithm.......................................................................................................................26
1Introduction
1.1 Revision History
Version
Date
Comments
100
05/02/2019
Initial Release
110
7/9/2019
16-bit Video + cable spec
120
10/14/2019
Ethernet Description
130
11/15/2019
GMSL Description + USB Description
140
6/15/2020
Latency, Intrinsic Calibration, Ethernet
API
1.2 Reference Documents
Ref Number
Document number
Comments
1
102-2013-40
Boson datasheet

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
4
1.3 SCOPE
This document describes detailed instructions on the conditions and requirements for
integrating the FLIR ADK with various communications methods including USB, Ethernet,
GMSL accessory board for communication, video streaming, and hardware synchronization.
1.4 BACKGROUND
The FLIRADKprovidesa Long Waver Inferred Image (LWIR)that is 640x512pixels. It senses
thermal radiation with wavelengths of 8-14um. The output is either a 8bit or 16bit
monochrome image.
The ADK comes in three different interfaces: USB output, GMSL output, and GMSL output
with a converter for GMSL to Ethernet. The ADK also has four different lens options.
Horizontal FOV
Vertical FOV
24
19.2
32
25.6
50
40
75
60
2USB –Getting Started
The USB option for the ADK ships with a Boson connected by USB for camera power, video
and command and control, a BNC cable for external sync input, and a Mizu-P25 connector
to power the window heater.
When connected to your computer, the USB ADK shows up as two different devices. The
first is a USB Vision Camera used for streaming video. The second is a com port used for
command and control. Any webcam driver should be able to access the USB Vision camera
device (Windows10 ‘Camera’, MacOS Photobooth, and Linux Cheese application will all
stream the ADK).
2.1 Connect and power the USB ADK
1) Connect the USB cable to the computer.
2) Optional: To power the window heater, apply 12 VDC up to 0.5A to the MIZU-P25connector
(shown below) to power the heater. The heater is protected against reverse polarity so it’s
technically ok to hookit up in reverse, it willjust pull a little more power. The USB ADK
ships with a MUZU-P25pigtail. Connect the green lead of the provided cable to positive and

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
5
the black lead to ground.
2.2 FLIR Boson GUI
An example GUI for streaming images and controlling the camera can be found here
https://www.flir.com/support-center/oem/camera-controller-gui-for-boson/
After the software is installed find the camera comport by opening Device Manager and
listing the available ports. Do this with the camera unplugged and with the camera plugged
into the computer. The port that shows up with the camera plugged in is the camera’s
comport.
With the camera’s comport known, open the BosonGUI software (icon is typically installed
on your desktop) and select the camera’s comport in the Port selection dropdown in the
lower right corner of the BosonGUI software. To start video streaming select “FLIR Video”
from the “Device Select” drop down menu on the left of the screen.

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
6
Once connected, click the “Do FFC” button on the right of the screen. The camera should
click and the image should blink. This will confirm you have control of the camera and are
streaming video.
There are many nested menus on the left side of your screen. These give you control of the
camera. The control options of most interest for automotive customers are:
“USB Video Mode” under Image Apperance -> USB Video Mode Controls
“FFC Mode” under System -> FFC Controls. Manual mode is recommended forautomotive
applications.
“Table Switch” under System -> Dynamic Range Control. This may be need is the camera
sees a significant change in temperature.
“Save Power On Defaults” and “Restore Factory Defaults” under System -> Configuration
control.This lets youboot into a custom camera configuration.
“TFPA” under Diagnostic Tools -> Status Panel. TFPAstands for Temperature of the Focal
Plane Array. This is a measure of temperature inside the camera. On the same menu you can
check your camera software version and serial numbers.
“File Save Path” under Media Settings -> Media Configuration. These show the location of
the saved files when using the Video Capture and Take Snapshot buttons in the upper left.

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
7
2.3 External Sync
The BNC coaxial connector on the USB ADK cable is for optional external sync. The ADK
triggers on the falling edge of a 3-12 volt signal with a minimum pulse width of 90nS. The
sync pulse should be 55-60hz even if the camera is in 30hz output mode. The camera will
need to be configured for external sync. Failure to provide sync pulses to a camera
configured for external sync may crash the camera requiring a reboot.
See the Flir Boson External Sync application note (102-2013-100-04 Rev 150) for more
information.
3Ethernet –Getting Started
The Ethernet option of the FLIRADK shipswith a GMSL connector onthe camera anda GMSL
to Ethernet Bridge that allows users to interact with the FLIR ADK using the GeniCam
protocol.
Figure 1: The FLIR ADK GMSLoption is shown on the left with the GMSL1 to Ethernet bridge shown on the right.
The Ethernet Bridge (also known and the Break Out Box or BoB) is a GeniCam GigE Vision
device. Any GeniCam driver should work. We offer the Spinnaker ADK as FLIR”s supported
GeniCam Client. While not supported we have heard that the open source Aravis driver
works as well.
Spinnaker can be downloaded from: https://www.flir.com/products/spinnaker-sdk/
3.1 Connecting the Boson ethernet ADK to the computer
1. Connect the FAKRA cable from the FLIR ADK to the GMSL –Ethernet bridge.
2. Connect the ethernet cable to a network interfacecard on the host computer.

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
8
3. Connect the 6 pin GPIO cable to the GMSL Ethernet bridge.
4. Apply power to the 6 pin GPIO cable. Apply 8-16 VDC to the Green wire and ground on the
Black wire. The power supply should be rated for a 2 A peak current. The system is designed
to run off the 12V supply froma car battery.
a. An optional AC power supply that will run the camera and shutter but not the heater
is available at:
https://www.flir.com/products/12v-power-supply-with-6-pin-hr10-connector/
Table 1: Pin out of the GPIO pins on the Ethernet breakout box. The Colors correspond to the GPIO cable that came with the
ADK.
Color
Pin
Function
Description
Green
1
VExt
Camera Input Power
Black
2
Not Connected
-
Red
3
Not Connected
-
White
4
Not Connected
-
Blue
5
Not Connected
-
Brown
6
GND
Camera Power Ground
3.2 Ubuntu Installation
Spinnaker is available for Ubuntu 16.04 and 18.04.
1. Downloadthe appropriate version of Spinnaker SDK from
https://www.flir.com/products/spinnaker-sdk/
2. Unpackthe tar.gz file for the appropriate system architecture.
3. Follow theREADME directionsto install necessary dependenciesand thenrun theinstaller
script.
4. It’s necessary to configure the ethernet card that the Boson is plugged into. It might be
necessary to find which ethernet card the Boson is plugged into. To do this run ‘ifconfig’ with
the ethernet cable unplugged then reconnect the ethernet cable and run ‘ifconfig’ again. You
should see a change in one of the ethernet cards.
5. Now configure that ethernet card by opening the network interfaces file with your favorite
text editor.
sudo gedit /etc/network/interfaces
Add the following lines (make sure to change enp15s0 to the appropriate card name
found by running ifconfig)

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
9
iface enp15s0 inet static
address 169.254.0.64
netmask 255.255.0.0
mtu 9000
auto enp15s0
Lastly restart the networking service to apply settings
sudo /etc/init.d/networking restart
6. The installer installs a program called SpinView. This allows you to see live feed video from
the camera. To start SpinView enter ‘spinview’ into the terminal.
7. The Linux installer also provides example programs in /usr/src/spin naker/src and full
documentation in /usr/share/doc/spinnaker-doc.
8. When SpinView is open the Boson device should be visible under the devices tab (as shown
below). Clickon the Boson device to select the camera and the click the play buttonto display
a live feed from the camera.
9. Similar to the USB GUI, spinview provides access to the Boson features in the Features tab.
Youmay need to stop streaming video to change some settings.
If you notice that some images are getting dropped or that the video feed is choppy it may
help to increase the Rx and Tx buffer size. Below is a link describing how to do that.
https://www.itechlounge.net/2015/05/linux-how-to-tune-up-receive-tx-and-
transmit-rx-buffers-on-network-interface/
3.2.1 Robot Operating System
We have an example project that uses the Ethernet ADK in the ROS framework present here.
https://github.com/flir/flir_adk_ethernet
Once the Boson is setup to run with Linux then follow the instructions in the Readme.md to
get the Boson to work with ROS.
After setup you should be able to run which will stream video on the host computer.
roslaunch flir_adk_ethernet boson.launch

FLIR ADK
Get ting Started
The information contained hereindoes not contain technology as
defined by EAR,15 CFR772, is publicly available,and therefore
not subject toEAR.
10
3.3 Windows Installation
Download and install the Windows version of the Spinnaker SDK here.
https://www.flir.com/products/spinnaker-sdk/
The Spinnaker SDK also comes with a software application called SpinView which will
display real time images from the Boson Camera.
3.4 Working with the Spinnaker API
When using the Ethernet BoB with the Boson camera, there is no direct access to the
camera. All communication with the camera is implemented with the Spinnaker API. When
connecting to the camera, the camera provides a list of features. The API provides a way to
access those features.
You are strongly encouraged to review, run and modify the example programs. On Linux
they are installed in /usr/src/spinnaker/src
3.4.1 Accessing Boson Camera Feature Example
This example show how to execute a manual FFC (also known as shutter) with the
Spinnaker API. This code was placed just before the ‘Begin acquiring images’ comment in
the AcquireImages function of the Acquisition.cpp Spinnaker example.
CCommandPtr ptrRunFFC = nodeMap.GetNode(“BosonRunFfc”);
if(IsAvailable(ptrRunFFC) && IsWritable(ptrRunFFC)){
cout << “Running FFC” << endl;
putRunFFC->Execute();
} else {
Cout << “Unable to access FFC” << endl;
}
In this example, we ask for the ‘BosonRunFFC’ node from the node map. Make sure that
node is available and that we can write to it. Then we execute the nodes function. This node
represents a command. Other node may represent strings, integers, floats or enumerations.