Lattice Semiconductor CrossLink-NX User manual

CrossLink-NX Object Counting Using VGG
Quick Start Guide
Application Note
FPGA-AN-02024-1.0
May 2020

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
2 FPGA-AN-02024-1.0
Disclaimers
Lattice makes no warranty, representation, or guarantee regarding the accuracy of information contained in this document or the suitability of its
products for any particular purpose. All information herein is provided AS IS and with all faults, and all risk associated with such information is entirely
with Buyer. Buyer shall not rely on any data and performance specifications or parameters provided herein. Products sold by Lattice have been
subject to limited testing and it is the Buyer's responsibility to independently determine the suitability of any products and to test and verify the
same. No Lattice products should be used in conjunction with mission- or safety-critical or any other application in which the failure of Lattice’s
product could create a situation where personal injury, death, severe property or environmental damage may occur. The information provided in this
document is proprietary to Lattice Semiconductor, and Lattice reserves the right to make any changes to the information in this document or to any
products at any time without notice.

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 3
Contents
Acronyms in This Document.................................................................................................................................................4
1. Introduction..................................................................................................................................................................5
1.1. Design Process Overview ....................................................................................................................................5
2. Machine Training and Creating Frozen File ..................................................................................................................7
2.1. Verifying TensorFlow and Tool Environment......................................................................................................7
2.2. Preparing the Dataset .........................................................................................................................................7
2.3. Training the Machine ..........................................................................................................................................8
2.4. Generating Frozen (*.pb) File............................................................................................................................11
3. Generating the Binary File..........................................................................................................................................12
4. Programming the Bitstream and Binary files to VIP Board and SD Card ....................................................................12
Technical Support Assistance .............................................................................................................................................13
Revision History..................................................................................................................................................................14
Figures
Figure 1.1. Lattice EVDK with MicroSD Card Adapter Board ................................................................................................5
Figure 1.2. Lattice Machine Learning Design Flow ...............................................................................................................6
Figure 2.1. TensorFlow Installation Check............................................................................................................................7
Figure 2.2. Dataset Image Size Check ...................................................................................................................................7
Figure 2.3. Dataset Folder Path Check..................................................................................................................................8
Figure 2.4. Dataset List, Image, and Label Data Path ...........................................................................................................8
Figure 2.5. Create a Label File...............................................................................................................................................8
Figure 2.6. Execute the Script...............................................................................................................................................9
Figure 2.7. TensorBoard –Generated Link...........................................................................................................................9
Figure 2.8. Training Status....................................................................................................................................................9
Figure 2.9. Image Menu......................................................................................................................................................10
Figure 2.10. Checkpoint Data Files at Log Folder................................................................................................................10
Figure 2.11. Create *.pbtxt File ..........................................................................................................................................11
Figure 2.12. Check Frozen File............................................................................................................................................11

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
4 FPGA-AN-02024-1.0
Acronyms in This Document
A list of acronyms used in this document.
Acronym
Definition
FPGA
Field-Programmable Gate Array
VIP
Video Interface Platform

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 5
1. Introduction
This document provides a quick guide on how to train a machine and create a frozen file for the Lattice Machine
Learning development using the Lattice’s Embedded Vision Development Kit. It assumes that you are familiar with the
basic Lattice FPGA design flow and mainly focuses on the Machine Learning part of the overall development process.
For detailed instructions of the design flow described in this document, refer to the CrossLink-NX Object Counting
Using VGG CNN Accelerator IP (FPGA-RD-02200).
CrossLinkTM VIP
Input Bridge Board
ECP5TM VIP
Processor Board
HDMI VIP Output
Bridge Board
MicroSD Card
Adapter Board
Camera Sensor CN2
Camera Sensor CN1
Figure 1.1. Lattice EVDK with MicroSD Card Adapter Board
1.1. Design Process Overview
The design process involves the following steps:
1. Setting up the basic environment
2. Preparing the dataset
3. Training the machine
4. Creating the frozen file (*.pb)
5. Creating the binary file with Lattice sensAI™ 3.0 program
6. Programming the binary and bitstream files to VIP board and SD card

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
6 FPGA-AN-02024-1.0
Figure 1.2. Lattice Machine Learning Design Flow

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 7
2. Machine Training and Creating Frozen File
2.1. Verifying TensorFlow and Tool Environment
Check if TensorFlow and your tool environment are installed correctly. For the detailed procedure in creating the basic
environment on PC, refer to the Setting up the Basic Environment section CrossLink-NX Object Counting Using VGG
CNN Accelerator IP (FPGA-RD-02200).
Figure 2.1. TensorFlow Installation Check
2.2. Preparing the Dataset
Prepare the image and label data (KITTI format). The image size for this design is 224 x 224 pixels.
For the detailed procedure in preparing the dataset,refer to the Preparing the Dataset section in CrossLink-NX Object
Counting Using VGG CNN Accelerator IP (FPGA-RD-02200).
Figure 2.2. Dataset Image Size Check

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
8 FPGA-AN-02024-1.0
2.3. Training the Machine
For the detailed procedure in machine training, refer to the Training the Machine section in CrossLink-NX Object
Counting Using VGG CNN Accelerator IP (FPGA-RD-02200).
To train the machine:
1. Check the training dataset path in the training script file train.sh.
Figure 2.3. Dataset Folder Path Check
The subdataset path can be set in the training code @src/dataset/kitti.py and can be used in combination with the
--data_path option while triggering training using train.py to get the desired path. For example, you can have
<data_path>/training/images and <data_path>/training/labels.
Figure 2.4. Dataset List, Image, and Label Data Path
2. Create a train.txt file.
$ cd data/humancnt/
$ python dataset_create.py
Figure 2.5. Create a Label File

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 9
3. Run machine training. In the command prompt, execute ./run command.
Figure 2.6. Execute the Script
4. Start TensorBoard.
$ tensorboard –logdir=<log directory of training>
For example: tensorboard –logdir=’./logs/’
5. Open the local host port on your web browser.
Figure 2.7. TensorBoard –Generated Link
6. Check the training status on TensorBoard.
Figure 2.8. Training Status

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
10 FPGA-AN-02024-1.0
Figure 2.9 shows the image menu of TensorBoard.
Figure 2.9. Image Menu
7. Check if the checkpoint, data, meta, index, and events (if using TensorBoard) files are created at the log directory.
These files are used for creating the frozen file (*.pb).
Figure 2.10. Checkpoint Data Files at Log Folder

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 11
2.4. Generating Frozen (*.pb) File
To generate frozen file (*.pb):
1. Run genpb.py.
$ python src/genpb.py –ckpt_dir <log directory> --freeze
For example, python src/genpb.py –ckpt_dir logs/humancnt/train --freeze
Figure 2.11. Create *.pbtxt File
2. Verify the generated frozen file.
Figure 2.12. Check Frozen File

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
12 FPGA-AN-02024-1.0
3. Generating the Binary File
For the detailed procedure in creating the binary file, refer to the Creating Binary File with sensAI section in CrossLink-
NX Object Counting Using VGG CNN Accelerator IP (FPGA-RD-02200).
4. Programming the Bitstream and Binary files to VIP Board and
SD Card
For the detailed procedure in flashing the bitstream file to the VIP board and flashing the binary file to the SD card,
refer to CrossLink-NX Object Counting Using VGG CNN Accelerator IP (FPGA-RD-02200).

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
FPGA-AN-02024-1.0 13
Technical Support Assistance
Submit a technical support case through www.latticesemi.com/techsupport.

CrossLink-NX Object Counting Using VGG Quick Start Guide
Application Note
© 2020 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
14 FPGA-AN-02024-1.0
Revision History
Revision 1.0, May 2020
Section
Change Summary
All
Initial release.

www.latticesemi.com