MYiR FZ3 User manual
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Accelerator Card User Manual
3
Table of content
FZ3 Deep Learning Computing Card User Manual ................................................................................................... 1
Introduction............................................................................................................................................................................. 4
1. Introduction to soft cores..............................................................................................................................................5
1.1
Software Introduction ................................................................................................................................5
1.2
Performance data on FZ3 for common models (unquantified cropping) ..............................5
2. Start and connect .............................................................................................................................................................. 6
2.1
Start-up preparation ................................................................................................................................... 6
2.2
Connection Method One: SSH Connection .......................................................................................... 6
2.2.1
How to connect to the internet via SSH in Windows.............................................................. 6
2.2.2
How to connect to the internet via SSH in MAC .......................................................................8
2.3
Connection Method Two: Serial Port Connection............................................................................ 8
2.3.1
Method of using SecureCRT to connect serial port in Windows........................................ 8
2.3.2
Method of using Minicom to connect serial prot in Mac OS
........................................ 10
3. Debug The device........................................................................................................................................................... 10
3.1
Change the network configuration..................................................................................................... 10
3.2
Copy files....................................................................................................................................................... 11
3.2.1
Realize File copy through FTP (suitable for Windows system)
................................ 11
3.2.2
Realize file copy through Samba protocol (For Mac OS)
.............................................. 12
3.3
Introduction to system catalog ............................................................................................................ 12
3.4.1
Example of Classification model.................................................................................................. 12
3.4.2
Example of object detection.......................................................................................................... 15
3.4.3
To output reasoning result and display.................................................................................... 17
3.4.4
Video input mode.............................................................................................................................. 19
3.5
Running EasyDL platform model prediction example................................................................ 19
3.5.1
How to use EasyDL ........................................................................................................................... 19
3.5.2
Run SDK ................................................................................................................................................ 23
3.5.3
Call HTTP Service.............................................................................................................................. 24
3.5.4
HTTP private service request description............................................................................... 25
4.Advanced Guide............................................................................................................................................................... 27
4.1
Develop Applications ............................................................................................................................... 27
4.1.1
Model acquisition.............................................................................................................................. 27
4.1.2
Connecting video data sources .................................................................................................... 27
4.1.3
Load device driver ............................................................................................................................ 27
4.1.4
Using the prediction library.......................................................................................................... 28
4.1.5
Create Application ............................................................................................................................ 28
4.2
Driver description..................................................................................................................................... 30
4.3
Prediction library description .............................................................................................................. 31
Appendix 1 Warranty & Technical Support Services.................................................................................... 33
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Accelerator Card User Manual
4
Introduction
MYIR's FZ3 is a deep learning computing card based on Xilinx Zynq Ultrascale+ CZU3EG MPSoC,
which is closely cooperated with Baidu. It integrates a 4-core ARM A53 processor + GPU + FPGA
architecture, with multi-core processing capabilities, FPGA Programmability and video streaming
hardware decoding capabilities. It has built-in deep learning soft core based on Linux operating
system + Baidu deep learning platform Paddle, deeply compatible with Baidu brain model
resource and tool platform (EasyDL/AIStudio ), can effectively and quickly implement a series of
processes such as model training-deployment-inference, which greatly reduces the threshold of
development verification, product integration, scientific research and teaching, project
deployment.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Accelerator Card User Manual
5
1. Introduction to soft cores
1.1
Software Introduction
The FZ3 computing card comes with a Linux OS, users can develop applications based on the Linux
OS. (Main calling process: 1. The application obtains video input -> 2. Calls the prediction library to
load the model -> 3. The scheduling model and the underlying driver acceleration module are used
for the calculation -> 4. Get the running result).
1.2
Performance data on FZ3 for common models (unquantified
cropping)
Network
Input Size
Single frame
time
consumption
resnet50
224 x 224
42ms
mobilenet-v1
224 x 224
10ms
inception-v2
299 x 299
41ms
inception-v3
299 x 299
70ms
resnext
224×224
69ms
mobilenet-ssd
300 x 300
24ms
mobilenet-ssd-640
640 x 640
79ms
vgg-ssd
300×300
246ms
yolov3
608×608
582ms
MYiR Tech | www.myirtech.com
6
FZ3 Deep Learning Computing Card User Manual
Note: FZ3 soft core is still under upgrading, and its performance will be improved simultaneously.
Different versions of the same network structure have different computing power requirements. If
there are specific project applications, you can contact the official team to apply for customized
optimization
2. Start and connect
2.1
Start-up preparation
1. The default start-up mode is start-up from TF card. In order to use the functions of the device
normally, please plug the TF card into the TF card slot.
2. The board supports serial port debugging and network port debugging. Serial port debugging
corresponds to the equipment’s console, it may have redundant printing information during
general debugging, so network port debugging is recommended. The board’s static IP is pre-set:
192.168.1.254, users can connect the board to the computer directly or to routing devices through
network cable (Network cable needs to be prepared by user), using the SSH protocol to log in to
the system. Please refer to the following article for specific usage.
3. Username and password (root and root) are required to be entered after the board starts-up to
login in the OS. The OS has its own deep learning pre-installed environment as well as model
inference sample. Please refer to the following article for detailed information.
2.2
Connection Method One: SSH Connection
FZ3’s de
fault parameter: static IP=192.168.1.254, netmask=255.255.255.0,
gateway=192.168.1.1
Hardware connection method: Connect FZ3 to the host computer or router, set the
computer or router IP to the same segment as FZ3, then users can login in via SSH.
Detailed Steps：
2.2.1
How to connect to the internet via SSH in Windows
1. Install debugging tools, SecureCRT (Users may Google search and install) is recommended.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
7
2. Set IP of the PC or router to the same network segment with the board. When the PC and the
board is connected directly, IP of the computer needs to be set manually: open the Control
Panel->Network and Internet->Network Connections->Local Connection->Properties-> The
Internet protocol version 4, set the IP address manually: IP address=192.168.1.111, Subnet
mask=255.255.255.0, Default gateway=192.168.1.1 as shown in below picture:
3. Create new window in SecureCRT. Click connect->New session->Protocol(select SSH2), click
next, Hostname is default IP of FZ3: 192.168.1.254, Port is 22, Username is root, password is root,
then click next to login in.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
8
2.2.2
How to connect to the internet via SSH in MAC
1. Set IP of the PC or router to the same network segment with the board.
Detailed steps：System preferences-->Network-->Advanced-->TCP/IP。
Ipv4 configuration example: Manual, Ipv4 address:192.168.1.111, Subnet mask: 255.255.255.0,
Router: 192.168.1.1.
2. Open Terminal: Launchpad->Other(folders)->Terminal
then enter username and password (root and root) to login in.
2.3
Connection Method Two: Serial Port Connection
If SSH connection fails or the IP needs to be viewed (after IP of the board is dynamically obtained),
the serial port needs to be used to enter the console of the board.
Connect USB UART of FZ3 to PC.
2.3.1
Method of using SecureCRT to connect serial port in Windows
1. Install software SecureCRT and serial driver cp210x_Windows_Drivers (The driver needs to be
installed for the first time, and the installation package can be searched from Google)
2. Make sure that the PC is connected to the USB UART interface of FZ3, and then click "my
computer" -> Properties -> device manager to check the port number mapped in the device
manager. As shown in below picture, the port number is com7.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
9
3. Open SecureCRT, create a new window Connect -> New Session -> Protocol, select serial, baud
rate 115200, not selected flow control, as shown in the picture below
4. Click [finish] and [connect] button, SecureCRT will connect to the serial port on the computing
card. After power on, users can see the start-up information. After the startup is completed, enter
the user name and password to enter the board’s OS. As shown in below picture.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
10
2.3.2
Method of using Minicom to connect serial prot in Mac OS
1. Install the serial port driver SiLabsUSBDriverDisk.dmg (it needs to be installed for the first time,
users may Google to search it) into PC
2. Install the Minicom tool
3.
Open Terminal:
Launchpad
->Others->Terminal, input command “minicom -s”to configure.
4.
The configuration content is as below. After configuration, connect FZ3 and input minicom
in terminal
select Serial port setup
：
A - Serial Device : /dev/cu.SLAB_USBtoUART
B - Lockfile Location :
/usr/local/Cellar/minicom/2.7/var C - Callin
Program :
D - Callout Program :
E - Bps/Par/Bits : 115200
8N1 F - Hardware Flow
Control : No G - Software
Flow Control : No
Save setup and dfl
3. Debug The device
3.1
Change the network configuration
The default IP address of the board is 192.168.1.254. If multiple boards are connected to the same
LAN at the same time, their IP addresses need to be configured different or changed to
dynamically obtaining IP. The path of network configuration file is /etc/network/interfaces
Static IP configuration
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
11
Dynamic IP configuration
3.2
Copy files
FZ3 supports SSH, samba, FTP and other network protocols, can easily carry out data
communication and file copy through the network. This function will be widely used in software
upgrading and user customization.
3.2.1
Realize File copy through FTP (suitable for Windows system)
1. [Press Windows + R shortcut key combination, input ipconfig], check the board’s IP by using
ipconfig command, ensure that the board’s IP and the Windows PC’s IP are in the same network
segment. Input ftp://192.168.1.254 in the folder input box. Enter the user name root and
password root according to the prompt, then users can enter the equipment system.
2. Open home -> root -> workspace directory. Workspace is the directory where the application is
located under the user root. Then copy the file to workspace, or copy files from workspace to PC.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
12
3.2.2
Realize file copy through Samba protocol (For Mac OS)
1. Configure and ensure that the board’s IP and MAC’s IP are in the same network segment (see
above 2.2.2: How to connect to the internet via SSH in MAC)
2. After the configuration is completed, click Finder -> To ->Connect to the server, enter smb://ip,
for example smb://192.168.1.254, user name root, password root.
3. The file directory of the board appears in "Finder". Open home -> root -> workspace directory.
Workspace is the directory where the application program is located under the root user. Users
can copy the files between the PC and the board through the copy and paste command.
3.3
Introduction to system catalog
Content
Catalog
Remarks
paddle-mobile
/home/root/workspace/paddle-mobile
paddle-mobile prediction
Library
Driver
/home/root/workspace/driver
Device drivers
sample
/home/root/workspace/sample
Sample codes
tools
/home/root/workspace/tools
Debug tools
3.4
Run Samples
When user’s PC is connected to FZ3, user can run the deep learning examples which MYIR
provides. It locates in /home/root/workspace/sample.
Sample
Remarks
classification
Example of classification model
detection
Example of target detection model
3.4.1
Example of Classification model
Read a local image, call the model for reasoning, and output the results.
Considering the simplicity and generality, this example shows reading the model and image
information from the JSON file, then loads and executes it. When executing, users need to specify
the corresponding configuration file:
The structure of project:
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
14
Here is an example of a configuration file.
key
value
model
Directory of model
combined_model
Whether it is a fusion model or not, only two files are fusion
models
input_width
Enter the image size for neural network. Following input
images will be scaled to this size
input_height
Enter the image size for neural network.
image
Image input for classification
mean
Average value
scale
Before entering the neural network, the budget processing is
(x - mean) * scale
format
The format required by the neural network, Opencv is BGR by
default
Other classification networks can also be implemented by adding/modifying configuration files
without modifying the code.
Example of detailed steps
：
"model":"../models/resnet50",
"image":"../models/resnet50/drink.jpeg",
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
15
1. Load driver, after the system starts, it can be loaded automatically or manually (It has been set
loaded automatically by default when leaving factory)
2.
Compile the example. FZ3 has the ability to compile. Go to the directory build of the
sample/classification to compile
After compiling, the following files will be generated in the directory build:
image_classify:
To read local image reasoning example
。
video_classify:
To
read the camera data for reasoning, which can only be used by connecting
the camera (USB camera, MIPI-CSI camera, etc.). To display the results, connect display device
via the DP interface (DP can also be transferred to HDMI or VGA)
.
For some devices working in embedded application scenarios and need Mipi camera, a Mipi lens
module is provided as an accessory which can be used as evaluation in development. The system,
software and hardware support have been made.
3.Run sample
3.4.2
Example of object detection
Different from classification, object detection can not only detect the type of the object, but also
detect the location coordinates of the object. There are two examples for object detection. One is
to detect objects on the image and draw coordinate information. The other one is the camera
collects video and detects the coordinates information, then the information would be drawn on
the screen
.
Project directory structure:
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
16
Below is an example of configuration file.
key
value
model
Directory of model
combined_model
Whether it is a fusion model or not, only two files are fusion
models
input_width
Enter the image size for neural network. Following input
images will be scaled to this size
input_height
Enter the image size for neural network.
image
Image input for classification
mean
Average value
scale
Before entering the neural network, the budget processing is
(x - mean) * scale
format
The format required by the neural network, Opencv is BGR by
default
threshold
Confidence threshold
Other classification networks can also be implemented by adding/modifying
configuration files without modifying the code.
"model":"../models/vgg-ssd",
"image":"../models/vgg-ssd/screw.jpg",
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
17
Example of detailed steps
：
1. Load drive, after the system starts, it can be loaded once (the factory has automatically loaded
by default)
2.
Compile the example. FZ3 has the ability to compile. Go to the directory build of the
sample/classification to compile
After compiling, the following files will be generated in the directory build:
image_classify:
To read local image reasoning example
。
video_classify:
To
read the camera data for reasoning, which can only be used by connecting
the camera (USB camera, MIPI-CSI camera, etc.). To display the results, connect display device
via the DP interface (DP can also be transferred to HDMI or VGA)
.
3.Run sample
3.4.3
To output reasoning result and display
1. Display via Mini DP interface: Connect FZ3 and DP monitor with male to male mini DP
cable.
MYiR Tech | www.myirtech.com
FZ3 Deep Learning Computing Card User Manual
19
The software package comes with FZ3 includes a simplified version of Linux desktop
environment which can be used to display the effect of program running in real time.
Please make sure that the monitor and FZ3 have been connected before start-up. After
entering the system, the board will enter the terminal command line environment by default.
Users may enter and exit the desktop environment using the following commands.
startx //Open desktop environment
stopx //Close desktop environment
For the demonstration example MYIR provides, opencv supports the visualization of
prediction results through window of desktop, please refer to 3.4.1 and 3.4.2 for how to use.
3.4.4
Video input mode
FZ3 supports video data input of USB, Mipi CSI, BT1120, GigE and other protocols, can be used
as the processing module of video stream in various scenarios. For example, in IPC camera
which supports BT1120 protocol, FZ3 can be used as a deep learning acceleration device
while IPC maintains its normal functions
。
After FZ3 receives the original data through BT1120 protocol and reasoning, it can send the
result back to IPC through serial port and SPI. The corresponding IPC program source code
we will also open. If user has a similar device, you can refer to the following hardware
introduction and connect it through the physical cable.
For some devices that require MIPI camera support for embedded application scenarios,
a
Mipi lens module is provided as an accessory which can be used as evaluation in development. The
system, software and hardware support have been made.
3.5
Running EasyDL platform model prediction example
3.5.1
How to use EasyDL
EasyDL is a one-stop deep learning model training and service platform that provides a visual
operation interface. Users can obtain a high-precision model by uploading a small number of
pictures. For details, please refer to the
EasyDL official website
. Detailed steps of using
EasyDL for data training is as below.
3.5.1.1
Select training category
Users may choose "image classification" or "object detection" according to the general scene.
Other manuals for FZ3
1
Table of contents
Other MYiR Computer Hardware manuals
Popular Computer Hardware manuals by other brands
Advantech
Advantech MIC-7 Series Startup manual
enerserve
enerserve SmartPi 3.0 Safety instruction
Enustech, Inc.
Enustech, Inc. imFONE BS-100 user manual
Advantech
Advantech ASMB-610V3 Startup manual
NXP Semiconductors
NXP Semiconductors QSG ADC1x13D+ECP3 DB quick start guide
Barco
Barco Encore VPx installation guide
