Lattice semiconductor Ice40 ultralite User manual

iCE40 UltraPlus Human Presence Detect

Quick Start Guide

Application Note

FPGA-AN-02005-1.1

June 2019

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

2 FPGA-AN-02005-1.1

Disclaimers

Lattice makes no warranty, representation, or guarantee regarding the accuracy of information contained in this document or the suitability of its

products for any particular purpose. All information herein is provided AS IS and with all faults, and all risk associated with such information is entirely

with Buyer. Buyer shall not rely on any data and performance specifications or parameters provided herein. Products sold by Lattice have been

subject to limited testing and it is the Buyer's responsibility to independently determine the suitability of any products and to test and verify the

same. No Lattice products should be used in conjunction with mission- or safety-critical or any other application in which the failure of Lattice’s

product could create a situation where personal injury, death, severe property or environmental damage may occur. The information provided in this

document is proprietary to Lattice Semiconductor, and Lattice reserves the right to make any changes to the information in this document or to any

products at any time without notice.

 iCE40 UltraPlus Human Presence Detect Quick Start Guide 
 Application Note 
 
© 2019 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal. 
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice. 
FPGA-AN-02005-1.1 3 
Contents 
Acronyms in This Document.................................................................................................................................................4 
1. 
Introduction
...................................................................................................................................................................5 
1.1. Design Process Overview ....................................................................................................................................5 
2. Machine Training and Creating Frozen File ..................................................................................................................7 
2.1. Verifying TensorFlow and Tool Environment......................................................................................................7 
2.2. Preparing the Dataset .........................................................................................................................................7 
2.3. Training the Machine ..........................................................................................................................................8 
2.4. Generating Frozen (*.pb) File............................................................................................................................10 
3. Generating the Binary File ..........................................................................................................................................13 
4. Programming the Bistream and Binary Files to HIMAX HM01B0 Upduino Shield Board ...........................................13 
Technical Support Assistance .............................................................................................................................................14 
Revision History ..................................................................................................................................................................15 
 
Figures 
Figure 1.1. Himax HM01B0 Upduino Shield Board ...............................................................................................................5 
Figure 1.2. Lattice Machine Learning Design Flow ...............................................................................................................6 
Figure 2.1. Tensorflow Installation Check.............................................................................................................................7 
Figure 2.2. Dataset Image Size Check ...................................................................................................................................7 
Figure 2.3. Dataset Folder Path Check..................................................................................................................................8 
Figure 2.4. Dataset List, Image, and Label Data Path ...........................................................................................................8 
Figure 2.5. Run Script File .....................................................................................................................................................8 
Figure 2.6. Execute the script ...............................................................................................................................................9 
Figure 2.7. Execute TensorBoard..........................................................................................................................................9 
Figure 2.8. TensorBoard Interface........................................................................................................................................9 
Figure 2.9. Checkpoint Data Files at Log Folder..................................................................................................................10 
Figure 2.10. Latest Checkpoint Data Files...........................................................................................................................10 
Figure 2.11. Create *.pbtxt File ..........................................................................................................................................10 
Figure 2.12. Check *.pbtxt File ...........................................................................................................................................11 
Figure 2.13. Rename and Copy Checkpoint Files................................................................................................................11 
Figure 2.14. Running trainckpt2inferencepb.py .................................................................................................................11 
Figure 2.15. Check Frozen File ............................................................................................................................................12 
 

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

4 FPGA-AN-02005-1.1

Acronyms in This Document

A list of acronyms used in this document.

Acronym

Definition

CKPT

Checkpoint

FPGA

Field-Programmable Gate Array

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 5

Introduction

This document provides a quick guide on how to train a machine and create a frozen file for the Lattice Machine

Learning development using the iCE40™HIMAX HM01B0 Upduino Shield board. It assumes that the reader is familiar

with the basic Lattice FPGA design flow and mainly focuses on the Machine Learning part of the overall development

process. This document refers to the Human Presence Detection Using Compact CNN Reference Design (FPGA-RD-

02059) for the detailed steps of the design flow.

Figure 1.1. Himax HM01B0 Upduino Shield Board

1.1. Design Process Overview

The design process involves the following steps:

Setting up the basic environment

Preparing the dataset

Training the Machine

Creating the frozen file (*.pb)

Creating the binary file with SensAI 2.0 program

Creating the FPGA Bitstream file

Programming the binary and bitstream files to iCE40 Upduino hardware

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

6 FPGA-AN-02005-1.1

Figure 1.2. Lattice Machine Learning Design Flow

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 7

2. Machine Training and Creating Frozen File

2.1. Verifying TensorFlow and Tool Environment

Check if Tensorflow and your tool environment is installed correctly. For the detailed procedure in creating the basic

environment on PC, refer to the Setting up the Basic Environment section in Human Presence Detection Using Compact

CNN Reference Design (FPGA-RD-02059).

Figure 2.1. Tensorflow Installation Check

2.2. Preparing the Dataset

Prepare the image and label data (KITTI format). The image size for this design is 64 x 64 pixels.

For the detailed procedure in preparing the dataset, refer to the Preparing the Dataset section in Human Presence

Detection Using Compact CNN Reference Design (FPGA-RD-02059).

Figure 2.2. Dataset Image Size Check

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

8 FPGA-AN-02005-1.1

2.3. Training the Machine

For the detailed procedure in machine training, refer to the Training the Machine section in Human Presence Detection

Using Compact CNN Reference Design (FPGA-RD-02059).

To train the machine:

1. Check the training dataset path in the training script file train.sh.

Figure 2.3. Dataset Folder Path Check

2. Check the subdirectory and training dataset

Figure 2.4. Dataset List, Image, and Label Data Path

3. Check the arguments of the command run.

Figure 2.5. Run Script File

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 9

4. Run machine training. In the command prompt. Execute the ./run command.

Figure 2.6. Execute the script

5. Run TensorBoard.

Execute the command tensorboard –logdir=’./logs’.

6. Open a new command prompt and web browser (such as Chrome).

Note: Check http::<>:<> port name on your terminal. It is different.

Figure 2.7. Execute TensorBoard

7. Check training status. (White Box by label dataset, Black Box by machine inference.) in the TensorBoard IMAGES

menu.

Figure 2.8. TensorBoard Interface

8. Check the checkpoint and meta data at log directory.

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

10 FPGA-AN-02005-1.1

Figure 2.9. Checkpoint Data Files at Log Folder

2.4. Generating Frozen (*.pb) File

To generate the Frozen file(*.pb):

1. Find the lastest checkpoint data at log path folder.

Figure 2.10. Latest Checkpoint Data Files

2. Run genpb.py.

3. Execute the command python src/genpb.py –ckpt_dir=’./logs/humandet/train/’.

For the detailed procedure in creating .pbtxt file, refer to the Creating Frozen File section in Human Presence

Detection Using Compact CNN Reference Design (FPGA-RD-02059).

Figure 2.11. Create *.pbtxt File

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 11

4. Check if the graph.pbtxt is generated.

Figure 2.12. Check *.pbtxt File

5. Copy the files of the train folder to the ‘/freeze/model/ ‘directory. Rename graph.pbtxt to model.pbtxt.

Figure 2.13. Rename and Copy Checkpoint Files

6. Go to the /freeze/ path at the command prompt and run trainckpt2interencepb.py

Execute command python trainckpt2interencepb.py.

For the detailed procedure in creating .pbtxt file, refer to the Creating Frozen File section in Human Presence

Detection Using Compact CNN Reference Design (FPGA-RD-02059).

Figure 2.14. Running trainckpt2inferencepb.py

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

12 FPGA-AN-02005-1.1

7. Verifiy and find the generated frozen file.

Figure 2.15. Check Frozen File

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 13

3. Generating the Binary File

For the detailed procedure in creating the binary file, refer to the Creating Binary File with SensAI section in Human

Presence Detection Using Compact CNN Reference Design (FPGA-RD-02059).

4. Programming the Bistream and Binary Files to HIMAX

HM01B0 Upduino Shield Board

For the detailed procedure in flashing the bistream and binary files to the iCE40 HIMAX HM01B0 Upduino Shield board,

refer to the Running the iCE40 Human Presence Detection Demo section in Human Presence Detection Using Compact

CNN Reference Design (FPGA-RD-02059).

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

14 FPGA-AN-02005-1.1

Technical Support Assistance

Submit a technical support case through www.latticesemi.com/techsupport.

iCE40 UltraPlus Human Presence Detect Quick Start Guide

Application Note

All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02005-1.1 15

Revision History

Revision 1.1, April 2019

Section

Change Summary

All