Argus Science ETVision User manual

ETVision
InstructionManual
MANUAL VERSION preliminary ver1.1
June, 2019
[email protected]
Web site: www.argusscience.com

E T V I S I O N M A N U A L
2
Table of Contents
1INTRODUCTION AND GENERAL SYSTEM DESCRIPTION...................................................................4
2ENVIRONMENTAL AND SAFETY CONSIDERATIONS ...........................................................................5
2.1 Statement on Safety Levels of Infrared Illumination.....................................................................................5
2.2 Safety Disclaimer...........................................................................................................................................5
3SYSTEM COMPONENTS .................................................................................................................................6
3.1 ETVision Wearable Optics.............................................................................................................................6
3.2 ETVision Controller.......................................................................................................................................8
3.3 ETVision PC ..................................................................................................................................................9
4INTERCONNECTIONS...................................................................................................................................10
5EYE TRACKING BASICS...............................................................................................................................14
6BASIC SYSTEM OPERATION.......................................................................................................................16
6.1 Charge Controller battery ............................................................................................................................16
6.2 Connect Controller to Wearable Optics Unit...............................................................................................16
6.3 Establish communication between Controller and ETVision application ....................................................17
6.3.1 Boot up Controller...............................................................................................................................17
6.3.2 Run the ETVision Application on the ETVision PC...........................................................................17
6.4 Position wearable optics unit on participant ................................................................................................18
6.5 Calibrate participant.....................................................................................................................................20
6.5.1 Perform Single Point Calibration........................................................................................................21
6.5.2 Add Calibration Points if Needed........................................................................................................22
6.6 Monitor gaze data and system status ...........................................................................................................25
6.6.1 Status Indicators..................................................................................................................................25
6.6.2 Left Eye and Right Eye windows........................................................................................................25
6.6.3 Scene Image window...........................................................................................................................25
6.6.4 Data Display window..........................................................................................................................26
6.6.5 Window Control...................................................................................................................................27
6.6.6 Manual offset correction......................................................................................................................28
6.6.7 Monitor Battery Charge.......................................................................................................................28
6.6.8 Audio....................................................................................................................................................28
6.7 Record data..................................................................................................................................................30
6.7.1 Record gaze data directly on ETV PC .................................................................................................30
6.7.2 Record video/audio data on micro SD card.........................................................................................33
6.7.3 Compute gaze data from an SD card file.............................................................................................36
6.8 Communicate in real-time with an external device......................................................................................37
6.8.1 Connecting to ETVision with ETRemote..............................................................................................39
6.8.2 Connecting to ETVision with a user created application ....................................................................41
7USING ETVISION SYSTEM OUTDOORS....................................................................................................42
8APPENDICIES..................................................................................................................................................44
8.1 Appendix A -- LED color codes..................................................................................................................44
8.2 Appendix B -- Calibration function details..................................................................................................46
8.3 Appendix C -- ETVision recorded and transmitted Data items....................................................................49
8.3.1 Data Item List ......................................................................................................................................49
8.3.2 Data Item Explanation.........................................................................................................................50

E T V I S I O N M A N U A L
3
8.4 Appendix D –System Control Table settings and Default values ...............................................................53
8.4.1 Eye Data tab........................................................................................................................................53
8.4.2 Video Source tab..................................................................................................................................54
8.4.3 Subject Calibration tab........................................................................................................................55
8.4.4 System Configuration tab.....................................................................................................................56
8.4.5 About Tab ............................................................................................................................................58
8.5 Appendix E -- Minimum specifications for ETVision PC and software installation instructions................59
8.5.1 PC specifications .................................................................................................................................59
8.5.2 ETVision software installation.............................................................................................................59
8.6 Appendix G –Removing HDMI cable from wearable optics......................................................................62
8.7 Appendix H -- SD Card Selection & Preparation........................................................................................63

E T V I S I O N M A N U A L
4
1Introduction and General System Description
ETVision (ETV) is a lightweight wearable system for measuring binocular point of gaze 180 times per
second.
The wearable unit resembles an eyeglasses frame that can be worn by itself or over the participant’s
prescription eyeglasses. It contains miniature “eye cameras” that view each eye and a “scene camera”
that views the scene in front of the participant, and a microphone. An optional visor can be mounted
to front of the headgear.
A cable extending from the earpiece connects to a small Controller (slightlylarger than a smart phone)
that easily fastens to an adjustable belt or can be held by the participant (or near the participant) in
some other way. The Controller holds a mico SD card for video and audio data recording, and
includes a rechargeable battery. It can also be powered directly from a DC power supply.
The ETVision system also includes a laptop PC that connects to the Controller in real-time via a LAN
cable or via WiFi. Alternately, the PC can read video and audio data from a microSD card previously
recorded by the wearable Controller. Note that by using the WiFi connection or recording to an SD
card for later data processing, the subject can be completely un-tethered while data recording takes
place.
An application running on the laptop PC uses the video images from the wearable cameras to compute
point-of-gaze with respect to a scene camera coordinate frame. The laptop can record the digital gaze
coordinates 180 times per second. This digital file also includes pupil diameter of each eye and other
feature detection measurements used as part of gaze computation.
In addition to recording a digital file, the PC application shows a real time display of the eye and scene
camera images and records these as wmv type video files. The 1280x720p scene image updates 30
times per second and includes a superimposed cursor showing the participant’s point of gaze in the
scene image. The scene video also includes audio from the head gear mic or a local mic. The eye
camera video displays include superimposed outlines and crosshairs to show proper recognition by the
system of eye image features.
The laptop PC can stream the 180 Hz, real-time, digital data to external devices via LAN, or stream
the real-time video scene image to external devices via LAN.
The Argus Science ETAnalysis application, usually included with the basic system, can be used to
play back and analyze gaze data recorded with ETVision. Detailed instructions for use of the
ETAnalysis application are provided in a separate manual.
An optional addition to the basic system, called ET3Space, uses head position and orientation data
from one of several third party motion tracking devices to compute gaze with respect to a room fixed
coordinate system and to determine point-of-gaze on multiple surfaces in the environment. This
optional feature is described in a separate manual.

E T V I S I O N M A N U A L
5
2Environmental and Safety Considerations
2.1 Statement on Safety Levels of Infrared Illumination
One of the most comprehensive and authoritative sources on the subject of light source safety is a
handbook entitled Safety with Lasers and Other Optical Sources, by David Sliney and Myron
Wolbarsht, first published in 1980 by Plenum Press. Quoting from page 147 of this book, “safe
chronic ocular exposure values, particularly to IR-A, probably are of the order of 10 mW/cm² or
below”. “IR-A” refers to the spectral band between 760 and 1400 nanometers, the range that includes
the Argus ETVision LEDs.
We are aware of no data, made available since the book was published, that would challenge this
conclusion. Most people might wish to be more conservative than the figure cited above, and the
ETVision illumination is at least an order of magnitude below this level. The largest irradiance value
that will be produced with the Argus ETVision optics is 0.75 mW/cm² (@ 870nm wavelength), at the
plane of the eye.
ETVison uses non-coherent illumination. There are no lasers in the headgear.
2.2 Safety Disclaimer
Argus Science LLC has taken due care in preparing this manual and accompanying
documentation including research, development and testing.
This manual reflects the state of Argus Science’s knowledge respecting the subject matter herein
at the time of its publication, and may not reflect the state of knowledge at all times in the future.
Argus Science has carefully reviewed this manual for technical accuracy. If errors are suspected,
the user should consult with Argus prior to proceeding. Argus Science makes no expressed or
implied warranty of any kind with regards to this manual or accompanying documentation.
Argus Science makes no representation, condition or warranty to the user or any other party with
respect to the adequacy of this manual or accompanying documents for any particular purpose or
with respect to its adequacy to produce a particular result. The ETVision systemis intended for
use in research conditions and Argus Science makes no declaration of suitability for safety
critical uses, including, but not limited to, clinical diagnoses, heavy machine operation or other
safety dependent activities.

E T V I S I O N M A N U A L
6
3System Components
ETVision headgear-- including eye cameras, scene camera, optional visor, and “
type-A-to-micro” HDMI cable.
ETVision Controller
ETVision PC
CAT 5 cable (for connecting ETV PC and Controller).
DC power supply for recharging (or directly powering) the Controller.
Micro SD card and adapter for use in Controller and ETVision PC.
3.1 ETVision Wearable Optics
The wearable optics unit resembles an eyeglasses frame that can be worn by itself or over the
participant’s prescription eyeglasses. It contains miniature “eye cameras” that view each eye and a
“scene camera” that views the scene in front of the participant, and a microphone. An optional visor
can be mounted to front of the headgear.
The left and right bottom sections of the frame each include a panel containing a camera and a pair of
LEDs. The panels rotate within an outer enclosure so that it is possible to adjust the camera vertical
aiming direction with respect to the optics frame. It is not usually necessary to adjust this position,

E T V I S I O N M A N U A L
7
and it is intended that be left in its standard position under most circumstances. (Consult Argus
Science if circumstances seem to require adjustmentof this position.)
The frame comes with three nosepieces that can be interchanged to adjust the vertical position of the
entire frame with respect to the face. To remove a nosepiece, simply pull the nosepiece horizontally
away from frame. Attach a nosepiece by pressing the two pins on the nosepiece into the mating holes
in the frame. The frame can also be worn with no nosepiece to position the frame at the lowest
possible position on the face.
The scene camera lens protrudes from the frame just above the nosepiece. Scene camera focus is
adjusted by rotating the lens. A microphone is located in the part of the frame that sits just above the
participant’s eyebrows.
An optional visor attaches and detaches from the frame by mating small slots at the upper corners of
the visor with small hooks on the frame.
An HDMI type cable extends from the right temple clamp (piece that extends from the temple to the
top of the ear). It connects to the frame with a micro HDMI connector in the temple clamp, but is
intended to be disconnected only if it necessary to replace the cable. Normally the cable should
remain connected to the frame. The other end of the cable has a standard type A HDMI connector and
connects to the ETVision Controller. The cable is a standard “type-A-to-micro” HDMI cable and
instructions for removing and replacing the cable are in Appendix G (section8.6).

E T V I S I O N M A N U A L
8
3.2 ETVision Controller
The ETVision Controller connects to the ETVision Wearable Optics with the HDMI cable extending
from the wearable optics frame. Besides the type A HDMI port, the Controller has a micro SD card
slot, a phono jack, a LAN port, a power supply connector, a power button and power LED, and a
record button and record LED. Both LEDs have multiple colors to indicate various possible states as
described in Appendix A (section8.1) of this manual.
The HDMI port accepts the type A HDMI connector from the ETVision Optics cable. A set screw on
the bottom of the Controller, directly under the HDMI port is used to secure the HDMI cable
connector after insertion.
The Controller can be connected to the ETVision Laptop using the LAN port and standard LAN cable,
or via Wifi. Alternately, the Controller can record video and audio data directly to a micro SD card.
If the Controller is connected to the ETVison Laptop, either with a LAN cable or Wifi, audio from the
microphone on the optics frame is transmitted to the application on the Laptop, and audio from the
Laptop microphone is transmitted to the phono jack on the Controller. If headphones or earbuds are
connected to the Controller phono jack and worn by the participant, this enables two way
communication with someone near the laptop.

E T V I S I O N M A N U A L
9
An adjustable belt supplied with system threads through two removable “loops” on the Controller
Case and can be used hold the Controller on the participant's waist. The Controller “loops” are each
attached to the case with two screws. The screws can be loosened to remove or install the belt.
The Controller contains a Lithium Polymer battery. The battery charges whenever the provided DC
power supply is connected to the Controller power connector and plugged into an AC socket. The
power unit supplied with the system is equipped with the appropriate AC adapter for the AC power
specifications of the intended destination country. The system can be run using the DC power supply
even when the battery is fully discharged, however the battery will charge most rapidly when the
system is idle. When fully charged the battery will power the system for about 5 hours when all
system features are being used. The battery will recharge in about 4.5 hours when the system is idle.
3.3 ETVision PC
ETVision systems usually include a laptop PC running Windows 10 (“ETV PC”). When supplied by
Argus Science, as part of the ETVision system, the ETVison software application is pre-installed by
Argus Science. If the ETVision PC (“ETV PC”) is not supplied by Argus Science see Appendix E
(section 8.5) for required PC specifications and software installation instructions.
The ETV PC laptop includes WiFi capability, a LAN port, USB ports, and an SD card slot. The
ETVision system uses micro SD cards. Micro SD cards must be placed in a micro-to-standard SD
card adapter before inserting in the SD card slot on the ETV PC.
If the ETVision PC was supplied by Argus Science as part of the ETVision system, the ETVision
application will be set to “load as administrator”, and both the ethernet and WiFi will be set to obtain
IPv4 address automatically. The PC should automatically connect to the controller via the LAN port
if a cable is connected or via WiFi if a cable is not connected. If the ETVision PC was not supplied
with the system, or if the ETVision app is reset to not have administrator privileges, see Appendix E
for LAN setup instructions.

E T V I S I O N M A N U A L
10
4Interconnections
Basic ETVision System Interconnections
(headphones not normally supplied as part of ETV system)

E T V I S I O N M A N U A L
11
ETVision connection to External Device –Method 1
(All other connection are as shown in the “Basic System Interconnections” diagram)

E T V I S I O N M A N U A L
12
ETVision connection to External Device –Method 2
(All other connection are as shown in the “Basic System Interconnections” diagram)

E T V I S I O N M A N U A L
13
If the ETVision PC was supplied by Argus Science as part of the system, ETVision will be set to
always “load as administrator”. If a cable connection between the Controller and the PC is detected,
the system will automatically connect via the cable. If a cable connection is not detected, the system
will automatically connect via WiFi. If the ETVision PC was not supplied with the system, or if the
ETVision app is reset to not have administrator privileges, see Appendix E (section 8.5) for LAN setup
instructions.

E T V I S I O N M A N U A L
14
5Eye Tracking Basics
To understand how eye tracking works, it is important to have a basic knowledge about the structure
of the eye.
The pupil is an aperture, or opening,
in the iris that allows light to enter
into the eye. Unless viewed exactly
on-axis with a light source, the pupil
appears black.
The iris, which is the colored part of
the eye, opens and closes to control
the pupil size.
On its way to the pupil, light passes
through the cornea, which is a thin,
film-like tissue that covers the eye.
The cornea is mostly transparent;
however, some light is also reflected
from the cornea.
The sclera is the white part of the eye.
ETVision uses a technique of eye tracking
known as “Pupil to CR” Tracking. This
method uses the relationship between two eye
features, the black pupil and mirror
reflections from the front surface of the
cornea (Corneal Reflections, or CRs), to
compute gaze within a scene.
Light from a pair of harmless near infra-red
LEDs on the head gear illuminate the eye
area. The near IR light is almost invisible to
the user, so it does not cause a distraction,
however it is visible to the eye camera. The
mirror reflections of these light sources from the front surface of the cornea (CRs) appear in the
camera image as a pair of bright dots.
When the eye rotates in its socket, the center of the pupil moves relative to the spots. By comparing
the relative position of the pupil and the CRs, the eye tracking system can compute the pointing
direction of the eye visual axis relative to a coordinate frame centered at the scene camera. During
periods when the headgear remains stationary on the head, eye gaze direction can also be computed
from pupil or CR position alone, and this can allow the system to continue to compute gaze when only
a subset of the eye image features can be successfully identified.
Pupil
Sclera
Iris
Retina
Cornea
Lens

E T V I S I O N M A N U A L
15
By computing the point at which the lines of gaze from the two eyes converge, the gaze point can
represented as spot on the scene camera image.
Since the geometry of the eye structures and the position of the optics with respect to the face vary
somewhat from person to person, gaze computation accuracy can be improved by incorporating data
from calibration points (points at which the person wearing the optics is known to be fixating a
particular point on the scene camera image). Excellent accuracy is often achieved with a single
calibration point, but can sometimes be improved with additional points.

E T V I S I O N M A N U A L
16
6Basic System Operation
6.1 Charge Controller battery
To be sure that the Controller battery will have maximum use time fully charge the batteryas follows.
•Connect the Controller power supply to the Controller and to the AC power source.
•If the power LED is off the battery is already charged. Disconnect the power supply, and
proceed to the next section.
•If the power LED is white, the battery is charging. When fully charged the LED will turn off.
(This may take up to 4.5 hours if the battery was fully discharged). Disconnect the power
supply, and proceed to the next section.
•If the power LED is green or yellow when the power supply is connected this means that the
system has been booted up (it is not in its idle state) and will not charge at the fastest rate.
Press and hold the power button for 5 seconds and then release it to put the system in the
“idle” state. The LED should turn off if the battery is already fully charged or white if the
battery is charging. When fully charged the LED will turn off. (This may take up to 4.5 hours
if the battery was fully discharged). Disconnect the power supply, and proceed to the next
section.
When the Controller is communicating with the ETVision laptop application (see section 6.3) the
battery charge percentage is displayed. Note that even if the battery is fully discharged, the system
can be operated, and including all system functions, with the power supply connected to the Controller
and plugged into AC power. The battery will charge in this case, but charging may take substantially
longer than when the Controller is in the idle state.
6.2 Connect Controller to Wearable Optics Unit
Connect the cable extending from the optics unit temple clamp (ear piece) to the Controller HDMI
port. After inserting the HDMI connector, use the setscrew on the Controller bottom panel, just below
the HDMI connector, to secure the connector. Note that it is not necessary to “tighten” this set screw.
Turn the setscrew just far enough for it to make contact with the connector and create enough friction
to keep the connector in place. Remember to loosen the setscrew before trying to disconnect the
HDMI cable from the Controller.

E T V I S I O N M A N U A L
17
6.3 Establish communication between Controller and
ETVision application
If a LAN (CAT 5) cable is connected between the ETV PC and the ETV Controller, the system will
communicate via the LAN cable. This is the most stable type of connection, but the participant is
“tethered” to the laptop. If a LAN cable is not connected between these components, the ETV PC and
Controller will communicate via WiFi. In this case the participant will be free to move about so long
as the participant carries the Controller bywearing it on the provided belt or by some other means, and
does not move beyond WiFi range from the PC.
6.3.1 Boot up Controller
Assuming the power supply is not connected to the Controller (battery only), the power LED will be
off if the Controller is not yet booted up. Press the power button and release after 1 sec. The power
LED should turn blue to indicate that the battery has at least a 15% charge and the system is booting.
The boot process will take about 1 minute. The LED will remain blue after boot up is complete.
If the power LED turns red, this means that the battery has less than 15% charge. Either return the
unit to idle and charge the battery as described in section 6.1, or plug in the power supply (LED will
turn yellow) and proceed with the power supply connected.
6.3.2 Run the ETVision Application on the ETVision PC
Run the ETVision application either by clicking the ETVision application icon on the Win10 desktop
or selecting “Argus Science->ETVision” from the Windows 10 Start menu. Note that it is also OK to
launch the ETVision application before booting up the Controller (as described in the previous
section). Note: if ETV PC was not supplied by Argus Science as part of the system, see Appendix F
for proper LAN and Wifi setup.
The ETVision application will appear as shown below. If the ETVision Controller has not finished
booting, the three video windows (left eye, right eye, and scene) will be blank. Live video should
appear in the video windows as soon as the Controller finishes its boot sequence.
Clicking the “Settings” icon on the ETVision tool bar will open a System Control Table window. If
running ETVision for the first time after installation, the default settings should be in place and will be
appropriate. These can be checked, if desired, as described in Appendix D (section8.4).

E T V I S I O N M A N U A L
18
6.4 Position wearable optics unit on participant
If desired, use the provided belt to fasten the ETVision Controller to the participant’s waist.
Alternately, have the participant hold the Controller in some other way, or support the Controller on a
surface near the participant. Assuming indoor use, it is usually not necessary to use the optional
filtered visor, but the visor maybe used if desired. (See section 7 for outdoor use).
Have the participant put on the optics frame just as they would a pair of eyeglasses. If the participant
is wearing prescription eyeglasses, the ETVision frame can simply be worn over the glasses as shown
in one of the photos below.
When the subject is wearing the ETVision frame, the top of frame should be just above the eyebrows.
If the person is wearing prescription eyeglasses, the bottom of the ETVision frame should be near the
bottom of the eyeglasses frame. An image of the left and right eye should appear in the Left Eye and
Right Eye windows, on the ETVision application. When the subject is looking more or less straight
ahead, the iris and pupil of each eye should be more or less centered vertically in the window
(although it need not be “precisely” centered). Correct positioning is shown in the photos and screen
shots below.

E T V I S I O N M A N U A L
19
subject not wearing eye glasses subject weraring eye glasses(note specular
reflections near bottom inner corners)
If the ETVision frame is not positioned correctly on the face, as described above, select a different
nosepiece or remove the nosepiece altogether to position the frame higher or lower on the face as
necessary. It is suggested that the participant completely remove the frame before the nosepiece is
swapped or removed. The photos, below, show examples of positioning that is correct as well as
examples of positioning that is either too high or too low on the face.
Correct optics position Correct optics position on subject with eye glasses
Optics too low Optics too high

E T V I S I O N M A N U A L
20
When the subject is looking more or less straight ahead both the eye pupils, and the corneal reflections
(mirror images of the two LEDs that are directed towards each eye) should be visible as shown below.
System recognition of the pupil is shown by a red ellipse superimposed on the pupil boundary and a
large green cross indicating the pupil center. Recognition of the corneal reflections (CRs) is shown by
smaller blue crosses indicating their centers.
When the participant looks above the center of the scene camera field of view, or to extreme
horizontal angles, one of both corneal reflections may reach the edge of the cornea or be obscured by
an eyelid and become undetectable or unreliable. The pupil, however should usually remain visible
and be stably detected by the system until eye angles become quite extreme. Of course, both the pupil
and corneal reflections will be obscured duringeye blinks.
The Scene window shows the image from the head mounted scene camera with a blue cross indicating
the participant’s point-of-gaze (POG). If both pupil images are being recognized, this cursor will
probably be present, but may not be a very accurate representation of POG until calibration mode is
activated as described in the next section.
6.5 Calibrate participant
This section is a simple description of the recommended calibration procedure. A more complete
description of the calibration function capabilities and options is presented in Appendix B (section
8.2).
To be sure that ETVision is set for the default calibration procedure described in this section, click the
“Settings” icon if necessary to bring up the System Control Table window and select the “Subject
Calibration” tab. Be sure “Calibration Type” is set to "Click on Scene Image Points”, and “Total
Calibration Points” is set to 9. Note that this does not mean that nine calibration points must be used,
but rather that the maximum of points is 9.