Lattice Semiconductor CrossLink-NX User manual

CrossLink-NX ROI User Tracking Using VVML
Board Demonstration
User Guide
FPGA-UG-02158-1.0
June 2022

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
2 FPGA-UG-02158-1.0
Disclaimers
Lattice makes no warranty, representation, or guarantee regarding the accuracy of information contained in this document or the suitability of its
products for any particular purpose. All information herein is provided AS IS, with all faults and associated risk the responsibility entirely of the Buyer.
Buyer shall not rely on any data and performance specifications or parameters provided herein. Products sold by Lattice have been subject to limited
testing and it is the Buyer's responsibility to independently determine the suitability of any products and to test and verify the same. No Lattice
products should be used in conjunction with mission- or safety-critical or any other application in which the failure of Lattice’s product could create a
situation where personal injury, death, severe property or environmental damage may occur. The information provided in this document is
proprietary to Lattice Semiconductor, and Lattice reserves the right to make any changes to the information in this document or to any products at
any time without notice.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 3
Contents
Acronyms in This Document.................................................................................................................................................5
1. Introduction..................................................................................................................................................................6
2. Functional Description..................................................................................................................................................6
3. Demo Setup..................................................................................................................................................................7
3.1. Hardware Requirements.....................................................................................................................................7
3.2. Software Requirements ......................................................................................................................................7
4. Programming the Demo ...............................................................................................................................................8
4.1. Package Folder Structure ....................................................................................................................................8
4.2. Load Firmware in FX3 I2C EEPROM.....................................................................................................................8
4.3. Programming the CrossLink-NX Voice and Vision SPI Flash................................................................................9
4.3.1. Erasing the CrossLink-NX Voice and Vision SRAM Prior to Reprogramming..................................................9
4.3.2. Programming the CrossLink-NX Voice and Vision Board..............................................................................11
4.3.3. Programming SensAI Firmware Binary to the CrossLink-NX Voice and Vision SPI Flash..............................12
5. Running the Demo......................................................................................................................................................17
5.1. Ideal Conditions for testing the demo ..............................................................................................................19
Technical Support Assistance .............................................................................................................................................20
Revision History..................................................................................................................................................................21

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
4 FPGA-UG-02158-1.0
Figures
Figure 2.1. VVML Rev-B board - Top View............................................................................................................................6
Figure 4.1. Demo Package Folder Structure .........................................................................................................................8
Figure 4.2. Selecting FX3 I2C EEPROM in USB Control Centre..............................................................................................9
Figure 4.3. Lattice Radiant Programmer Default Screen ....................................................................................................10
Figure 4.4. Lattice Radiant Programmer- Device Selection ................................................................................................10
Figure 4.5. Lattice Radiant Programmer –Device Operation.............................................................................................10
Figure 4.6. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing..................................11
Figure 4.7. CrossLink-NX Voice and Vision Flashing Switch –SW5 Push Button ................................................................12
Figure 4.8. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing..................................13
Figure 4.9. Radiant Programmer –Output Console............................................................................................................14
Figure 4.10. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing................................15
Figure 4.11. Radiant Programmer –Output Console..........................................................................................................16
Figure 5.1. Running the Demo............................................................................................................................................17
Figure 5.2. Running the Demo............................................................................................................................................18
Figure 5.3. Running the Demo............................................................................................................................................18

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 5
Acronyms in This Document
A list of acronyms used in this document.
Acronym
Definition
FPGA
Field-Programmable Gate Array
SPI
Serial Peripheral Interface
SRAM
Static Random-Access Memory
USB
Universal Serial Bus
VVML
Voice and Vision Machine Learning
AT
Attention
SS
Shoulder Surfing

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
6 FPGA-UG-02158-1.0
1. Introduction
This document describes User Tracking demo using the CrossLink™-NX Voice and Vision Machine Learning (VVML)
platform.
2. Functional Description
Figure 2.1 shows the top view of the VVML board used in this demonstration.
Figure 2.1. VVML Rev-B board - Top View

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 7
3. Demo Setup
This section describes the demo setup.
3.1. Hardware Requirements
•CrossLink-NX Voice and Vision Machine Learning Board, Rev. B
•USB 3.0 A to Micro B cable (To load firmware in FX3 and to run demo using video visualizing tool), USB 2.0 A to
Micro B cable (to program Crosslink-NX VVML board)
•Personal computer
3.2. Software Requirements
•Lattice Radiant™ Programmer version 3 (Refer to http://www.latticesemi.com/programmer)
•Cypress - EZ-USB FX3 Software Development Kit (Refer to https://www.cypress.com/documentation/software-
and-drivers/ez-usb-fx3-software-development-kit)
•Video visualizing tool to see output, user can use one of below tool:
1. VLC Media Player: https://www.videolan.org/vlc/download-windows.html
2. Windows Camera APP: To use windows camera app user must change default webcam if other camera device is
connected to system. For more info user can follow steps mentioned in link: https://answers.microsoft.com/en-
us/windows/forum/all/how-to-change-default-webcam-in-windows-10/1e8377c8-4fd5-4e9c-8ab6-67913c5c8e0c.
3. AMCap: http://noeld.com/programs.asp?cat=video.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
8 FPGA-UG-02158-1.0
4. Programming the Demo
This document describes the FaceID Demo running process using the Crosslink-NX™ Voice and Vision platform.
To Run the demo user will need prebuilt ‘.mcs’ and ‘.bit’ file to program the Crosslink-NX VnV board.
4.1. Package Folder Structure
Figure 4.1 shows the demo folders and files after unzipping the package.
Figure 4.1. Demo Package Folder Structure
4.2. Load Firmware in FX3 I2C EEPROM
To load the firmware:
1. Connect jumper J13, user refer to section Functional Description for location of jumper on VVML Board.
2. Connect the USB3 port of the CrossLink-NX VVML Board (Rev B) to the PC using the USB3 cable.
3. Open the USB Control Centre application. The Cypress FX3 SDK should be installed.
4. Press the SW2 button to reset the FX3 chip. Figure 4.2 shows the boot loader device screen.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 9
Figure 4.2. Selecting FX3 I2C EEPROM in USB Control Centre
5. Select Cypress USB Bootloader.
6. Click Program > FX3 > I2C E2PROM.
7. Locate and select the FX3 image file for the 640 × 480p 60 Hz 16 bit configuration.
8. The Firmware is programmed in the I2C E2PROM.
9. After the operation is completed, a message acknowledging successful programming is shown at the bottom
taskbar.
10. Remove jumper J13.
11. Power OFF and then power ON the board.
12. The FX3 boots from I2C E2PROM.
4.3. Programming the CrossLink-NX Voice and Vision SPI Flash
4.3.1. Erasing the CrossLink-NX Voice and Vision SRAM Prior to Reprogramming
If the CrossLink-NX device is already programmed (either directly, or loaded from SPI Flash), follow this procedure first
to erase the CrossLink-NX SRAM memory before re-programming the SPI Flash. If you are doing this, keep the board
powered when reprogramming the SPI Flash (so it does not reload on reboot).
To erase the CrossLink-NX device:
1. Launch Lattice Radiant™ Programmer. In the Getting Started dialog box, select Create a new blank project.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
10 FPGA-UG-02158-1.0
Figure 4.3. Lattice Radiant Programmer Default Screen
2. Click OK.
3. In the Lattice Radiant Programmer main interface, select LIFCL for Device Family, as shown in Figure 4.4.
4. Select LIFCL for Device Vendor, and LIFCL-40 for Device.
Figure 4.4. Lattice Radiant Programmer- Device Selection
5. Right-click and select Device Properties.
6. Select JTAG for Port Interface, Direct Programming for Access Mode, and Erase Only for Operation, as shown in
Figure 4.5.
Figure 4.5. Lattice Radiant Programmer –Device Operation

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 11
7. Click OK to close the Device Properties dialog box.
8. Press and hold SW5 until you see the Successful message in the Lattice Radiant log window.
9. In the Radiant Programmer main interface, click the Program button to start the erase operation.
4.3.2.Programming the CrossLink-NX Voice and Vision Board
To program the CrossLink-NX Voice and Vision SPI Flash:
1. Ensure that the CrossLink-NX Voice and Vision device SRAM is erased by performing the steps in the Erasing the
CrossLink-NX Voice and Vision SRAM Prior to Reprogramming section.
2. In the Lattice Radiant Programmer main interface, right-click Operation and select Device Properties.
3. In the Device Properties dialog box, apply the settings shown in Figure 4.6.
Figure 4.6. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing
Notes:
•In Programming file, browse and select the CrossLink-NX Voice and Vision bit file (*.bit).
•Click Load from File to update the Data file size (bytes) value.
•Ensure that the following addresses are correct:
•Start Address (Hex) – 0x00000000
•End Address (Hex) – 0x00100000

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
12 FPGA-UG-02158-1.0
4. Click OK.
5. Now press SW5 push button switch before clicking program button in below step and keep it pressed till you see
Successful message in Radiant log window.
6. Click the Program button to start the programming operation.
7. After successful programming, the Output console displays the result, as shown in Figure 4.7.
Figure 4.7. CrossLink-NX Voice and Vision Flashing Switch –SW5 Push Button
4.3.3.Programming SensAI Firmware Binary to the CrossLink-NX Voice and Vision SPI Flash
4.3.3.1. Flash SensAI Firmware Hex for Face Detection to Crosslink-NX SPI Flash
To program the CrossLink-NX SPI flash:
1. Ensure that the CrossLink-NX device SRAM is erased by performing the steps in the Erasing the CrossLink-NX Voice
and Vision SRAM Prior to Reprogramming section before flashing bitstream and sensAI firmware binary.
2. In the Lattice Radiant Programmer main interface, right-click the CrossLink-NX row and select Device Properties.
3. In the Device Properties dialog box, apply the settings as shown in Figure 4.8.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 13
Figure 4.8. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing
Notes:
•In Programming file, browse and select the CrossLink-NX sensAI firmware binary file after converting it to hex
(*.mcs).
•Click Load from File to update the Data file size (bytes) value.
•Ensure that the following addresses are correct:
•Start Address (Hex) – 0x00400000
•End Address (Hex) – 0x00480000
4. Click OK.
5. Now press SW5 push button switch before clicking program button in below step and keep it pressed till you see
Successful message in Radiant log window.
6. Click the Program button to start the programming operation.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
14 FPGA-UG-02158-1.0
7. After successful programming, the Output console displays the result, as shown in Figure 4.9.
Figure 4.9. Radiant Programmer –Output Console
4.3.3.2. Flash SensAI Firmware Hex for User Tracking Classification to Crosslink-NX SPI Flash
To program the CrossLink-NX SPI flash:
1. Ensure that the CrossLink-NX device SRAM is erased by performing the steps in Erasing the CrossLink-NX Voice and
Vision SRAM Prior to Reprogramming section before flashing bitstream and sensAI firmware binary.
2. In the Lattice Radiant Programmer main interface, right-click the CrossLink-NX row and select Device Properties.
3. In the Device Properties dialog box, apply the settings as shown in Figure 4.10.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 15
Figure 4.10. Radiant Programmer –Selecting Device Properties Options for CrossLink-NX Flashing
Notes:
•In Programming file, browse and select the CrossLink-NX sensAI firmware binary file after converting it to hex
(*.mcs).
•Click Load from File to update the Data file size (bytes) value.
•Ensure that the following addresses are correct:
•Start Address (Hex) – 0x00300000
•End Address (Hex) – 0x003B0000
4. Click OK.
5. Now press SW5 push button switch before clicking program button in below step and keep it pressed till you see
Successful message in Radiant log window.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
16 FPGA-UG-02158-1.0
6. Click the Program button to start the programming operation.
7. After successful programming, the Output console displays the result, as shown in Figure 4.11.
Figure 4.11. Radiant Programmer –Output Console

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 17
5. Running the Demo
1. To run the demo:
2. Power on the VVML board.
3. Make sure that the position of DIP SWITCH is as given below
DIP Switch
State
SWITCH 1
OFF
SWITCH 2
OFF
SWITCH 3
ON
SWITCH 4
ON
4. Flash the .bit and .mcs files.
5. Connect the VVML board to the PC through the USB3 port.
6. Open one of video tool:
7. For AMCap video display application and select the FX3 Device as source from under Devices.
8. For Windows Camera app disable other camera device if any available and set FX3 as default camera device as
mentioned in Load Firmware in FX3 I2C EEPROM.
9. For VLC Media player click Media -> Open Capture Device and select video device name as FX3.
10. The camera image with green guide lines enabled is displayed in the output Figure 5.1.
11. User Tracking output can be interpreted as following:
•Green box: Attentive user
•Blue box: Non-Attentive user
•The right bottom corner shows the Region of Interest i.e. cropped detected faces
Figure 5.1. Running the Demo

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
18 FPGA-UG-02158-1.0
Figure 5.2. Running the Demo
Figure 5.3. Running the Demo

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-UG-02158-1.0 19
5.1. Ideal Conditions for testing the demo
•Distance: Users should be in range of 1.5 to 5 feet distance.
•Descent light is needed to run demo. Too low and direct light from light source may reduce the performance of
demo.

CrossLink-NX ROI User Tracking Using VVML Board Demonstration
User Guide
© 2022 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
20 FPGA-UG-02158-1.0
Technical Support Assistance
Submit a technical support case through www.latticesemi.com/techsupport.