NXP Semiconductors SLN-VIZN3D-IOT User manual

SLN-VIZN3D-IOT Kit User Guide
NXP Semiconductors Document identifier: SLN-VIZN3D-IOT-UG
User Guide Rev. 0, 01 November 2021

Contents
Chapter 1 Introduction........................................................................................... 4
1.1 RT117F vision crossover processor overview...........................................................................4
1.2 Additional peripherals................................................................................................................4
Chapter 2 Recommendations................................................................................ 7
2.1 Recommended computer configuration.....................................................................................7
2.2 Recommended phone configuration..........................................................................................7
2.3 Usage condition.........................................................................................................................7
Chapter 3 Getting started with SLN-VIZN3D-IOT.................................................. 8
3.1 Box contents..............................................................................................................................8
3.2 Running the preconfigured smart-lock application.................................................................... 8
Chapter 4 Local face management......................................................................10
4.1 Registering a face....................................................................................................................10
4.2 Deregistering a face................................................................................................................ 12
4.3 Liveness detection...................................................................................................................14
Chapter 5 Remote face management..................................................................16
5.1 Installing the Smart Lock Android app manually..................................................................... 16
5.2 Managing kits.......................................................................................................................... 17
5.3 Managing users.......................................................................................................................19
5.3.1 Adding users............................................................................................................................. 19
5.3.2 Modifying users......................................................................................................................... 21
5.3.3 Deleting users........................................................................................................................... 22
Chapter 6 Additional features.............................................................................. 24
6.1 Low-power mode.....................................................................................................................24
6.1.1 Pushbuttons.............................................................................................................................. 24
6.1.2 PIR sensor................................................................................................................................ 25
6.1.3 Automatically enable low-power mode......................................................................................26
6.2 Serial command line interface................................................................................................. 28
6.2.1 Connect to the serial command line interface........................................................................... 28
6.2.2 List available commands...........................................................................................................29
6.3 Manage user database............................................................................................................30
6.3.1 List all registered users............................................................................................................. 30
6.3.2 Manually add users................................................................................................................... 31
6.3.3 Manually delete users............................................................................................................... 31
6.3.4 Rename users...........................................................................................................................32
6.3.5 Manually save users into flash.................................................................................................. 32
6.4 Display configuration............................................................................................................... 32
6.5 Camera configuration.............................................................................................................. 33
6.6 Audio configuration..................................................................................................................34
6.7 Configure LED brightness....................................................................................................... 35
6.7.1 IR LED.......................................................................................................................................35
6.7.2 White LED................................................................................................................................. 36
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 2 / 47

6.8 Configure automatic low-power mode..................................................................................... 36
6.9 Debug logging messages........................................................................................................36
6.10 Get BLE address................................................................................................................... 40
6.11 Get managers and manager components running................................................................40
6.12 Print version information........................................................................................................41
6.13 Reset the SLN-VIZN3D-IOT..................................................................................................41
6.14 Terminate the SLN-VIZN3D-IOT CLI.....................................................................................42
Chapter 7 Troubleshooting.................................................................................. 43
7.1 Registering a face....................................................................................................................43
7.1.1 Adjust face proximity and position.............................................................................................43
7.1.2 Debug using log messages.......................................................................................................43
Chapter 8 Document details................................................................................ 44
8.1 References.............................................................................................................................. 44
8.2 Acronyms and definitions........................................................................................................ 44
Chapter 9 Revision history...................................................................................46
NXP Semiconductors
Contents
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 3 / 47

Chapter 1
Introduction
The SLN-VIZN3D-IOT development kit implements NXP’s Edge Ready turnkey solution for 3D face recognition-based access
control, using a 3D Structured Light Module (SLM) camera. This solution is enabled by a high-performance member of the
i.MX RT1170 microcontroller (MCU) family, running a Real-Time Operating System (RTOS) at up to 1 GHz clock rates, with 2
MB on-chip SRAM. This 3D face recognition solution Software Development Kit (SDK), also called SLN-VIZN3D-IOT, provides
OEMs with a fully integrated, self-contained, software, and hardware solution that includes pre-integrated machine learning
face-recognition algorithms, as well as all required drivers for all necessary peripherals, including memories, cameras, display,
and Bluetooth Low Energy (BLE) chipset. A runtime license to this software solution is bundled with the i.MX RT117F-numbered
devices in the i.MX RT1170 family. This cost-effective, easy-to-use solution facilitates the deployment of highly accurate face
recognition with robust liveness detection capability, that leverages 3D SLM cameras to operate effectively in even the most
challenging lighting conditions. By leveraging an MCU platform, this solution is able to deliver the low cost and low power
consumption required for battery powered consumer smart locks, combined with the quick inferencing and short boot times
needed to deliver a great user experience.
TARGET APPLICATIONS
•Smart door locks: for consumer and hospitality applications, including single family homes, multiple dwelling units, and
hotels.
•Access control: for office and industrial smart-building applications.
1.1 RT117F vision crossover processor overview
The i.MX RT117F is an Edge Ready member of the i.MX RT1170 family of crossover processors, targeting secured embedded
3D face-recognition applications. It features NXPs advanced implementation of the Arm® Cortex®-M7 core, which operates at
speeds up to 1 GHz to provide high CPU performance and best realtime response. This i.MX RT117F-based solution enables
system designers to add face recognition easily and inexpensively with 2D and 3D liveness detection capabilities for a wide variety
of smart home and access control products in smart homes, smart industrial, and smart building applications. The i.MX RT117F
processor is licensed to run NXP’s i.MX RT run-time library for 3D face recognition, which includes:
• Camera drivers
• Image capture
• Image pre-processing
• Face alignment
• Face tracking
• Face detection
• Face recognition
• Liveness detection
1.2 Additional peripherals
In addition to the i.MX RT117F microcontroller, the main board embeds all the additional components, modules, and connectors
required to create a secured and connected application. These peripherals are shown in Figure 1.
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 4 / 47

Figure 1. SLN-VIZN3D-IOT main board callout
The SLN-VIZN3D-IOT kit comes fully assembled with both a 3D camera and an RGB camera module, a 2.4 inch TFT display, and
a speaker to allow for rapid prototyping of any smart lock project with 3D face-recognition capabilities. These additional peripherals
are shown in Figure 2.
Figure 2. SLN-VIZN3D-IOT kit callout
NXP Semiconductors
Introduction
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 5 / 47

Depending on the mode of operation, the VIZN3D kit can emit highly concentrated white or non-visible infrared
light which can be hazardous to human eyes. Products which incorporate these devices must follow the safety
precautions given in IEC 60825-1 and IEC 62471.
CAUTION
NXP Semiconductors
Introduction
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 6 / 47

Chapter 2
Recommendations
2.1 Recommended computer configuration
The SLN-VIZN3D-IOT SDK requires an up-to-date computer which runs MCUXpresso IDE version 11.4 or higher. It also requires
a terminal program to communicate with the device via USB.
Table 1. Supported computer configurations
Computer type OS version Terminal program
Apple Mac OS PuTTY
PC Windows 7/10 PuTTY/Tera Term
PC Linux PuTTY
2.2 Recommended phone configuration
The SLN-VIZN3D-IOT kit requires an Android™ phone App to demonstrate the remote registration.
Our engineer team has verified and tested the Smart Lock App for Android using terminals running Android 8 or higher.
2.3 Usage condition
The following information is provided per Article 10.8 of the Radio Equipment Directive 2014/53/EU:
1. Frequency bands in which the equipment operates.
2. The maximum RF power transmitted.
Table 2. Bluetooth frequency and power
PN RF technology Frequency range Maximum transmitted power
SLN-VIZN3D-IOT Bluetooth 2402 - 2480 MHz 10 dBm
EUROPEAN DECLARATION OF CONFORMITY (Simplified DoC per Article 10.9 of the Radio Equipment Directive 2014/53/EU)
This apparatus, namely SLN-VIZN3D-IOT, conforms to the Radio Equipment Directive 2014/53/EU. The full EU Declaration of
Conformity for this apparatus can be found at this location: http://www.nxp.com/mcu-vision3d
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 7 / 47

Chapter 3
Getting started with SLN-VIZN3D-IOT
3.1 Box contents
The SLN-VIZN3D-IOT kit is delivered fully assembled in a box with a printed quick start card and a USB-C cable.
Figure 3. SLN-VIZN3D-IOT box contents
Check your kit for damage or marks, and if you find any, contact your NXP representative.
3.2 Running the preconfigured smart-lock application
Before using your kit, remove the protection films on the 3D camera and the RGB camera (see Figure 4), which protect the lens
during transport. Otherwise, the depth and image capture will be wrong.
Figure 4. Removing camera protection films
To get started, take the USB-C cable provided in the kit and plug the USB-A end into your computer’s USB port and the USB-C
end into the kit’s USB port.
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 8 / 47

Figure 5. Powering the SLN-VIZN3D-IOT kit
When connected, the RGB camera video appears, framed with some application settings.
Figure 6. Boot phase completed
NXP Semiconductors
Getting started with SLN-VIZN3D-IOT
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 9 / 47

Chapter 4
Local face management
To evaluate the smart-lock application pre-programmed on the SLN-VIZN3D-IOT kit, you must first register a face.
4.1 Registering a face
The onboard pushbuttons are pre-configured to allow direct access to some of the kit’s most useful features, like face registering
and de-registering and entering and exiting Low Power Mode. To manually register a new face, press and release the SW3 button
located at the back of the kit.
Figure 7. Manual registration button
The speaker plays an audio message, confirming that the registration started. During the face registration process, the
“Registering” message appears in green at the top of the screen and a green bounding box appears to help align the user’s face.
Additionally, a timeout counter represented by a blue bar located at the bottom of the screen appears automatically and starts to
count down.
To register your face, look straight-on to the camera while aligning your face inside the bounding box guidelines presented by
the UI.
Figure 8. Registration in progress
To cancel the registration, press and release the SW3 button again or wait for the registration process to time out (~5 s).
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 10 / 47

When successfully registered, the speaker plays an audio message, confirming that the registration succeeded. Two messages
(“Registration Successful” and “USERNAME added”) appear in green. The number of registered users increases by one and the
user name shown on the screen is automatically assigned to the newly registered face.
Figure 9. Successful registration
New faces can be added and usernames can be modified through the command-line interface or the phone app
described in the following sections.
NOTE
If your face fails to register due to the registration being canceled, timing out, or due to an attempt to register the same face twice,
the speaker plays an error message and the screen displays a message in red, as shown in Figure 10.
Figure 10. Registration failed
To retry, press and release the SW3 button again.
If you have trouble successfully registering a face, try adjusting the proximity to the camera (either closer or further
away) or move your head left-right and up-down so that the camera can get a better view of different angles of your
face. For more information, see Troubleshooting.
NOTE
When a face is registered, the 3D Smart Lock solution can automatically recognize your face. When a face is recognized, the
screen displays the “Recognition Successful” message in green, the lock icon in the bottom left corner turns from red/locked to
green/unlocked, and the speaker plays an audio message, confirming that a face is successfully recognized.
NXP Semiconductors
Local face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 11 / 47

Figure 11. Successful recognition
Follow the same procedure to add additional faces into the system.
4.2 Deregistering a face
Similarly to the registration button discussed in the previous section, pressing and releasing the SW2 button (located at the back
of the kit) triggers a manual de-registration, which removes the next recognized face from the kit’s internal face database.
Figure 12. Manual deregistration button
The speaker plays an audio message, confirming that the de-registration started. During the face de-registration process, the
“Deregistering” message appears in green at the top of the screen and a red bounding box appears to help you align your face.
A timeout counter represented by a blue bar starts to count down.
To de-register your face, look straight-on to the camera and align your face in the bounding box presented by the UI.
NXP Semiconductors
Local face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 12 / 47

Figure 13. De-registration in progress
To cancel the de-registration process, press and release the SW2 button or wait for the registration process to time out (~10
s). When successfully de-registered, the speaker plays an audio message, confirming that the de-registration succeeded. An
onscreen message confirms the user name of the user that was removed and the number of registered users is decremented
by one.
Figure 14. Successful de-registration
Faces can also be deleted through the command-line interface or the phone app described in the
following sections.
NOTE
If your face fails to de-register due to the de-registration being canceled or timing out, the speaker plays an error message and
the screen displays a message in red, as shown in Figure 15.
NXP Semiconductors
Local face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 13 / 47

Figure 15. De-registration failed
To retry, press and release the SW2 button again.
If you have trouble successfully de-registering a face, adjust the proximity to the camera (either closer or further
away) and slowly move your head left-right and up-down so that the camera can get a better view of different angles
of your face. For more information, see Troubleshooting.
NOTE
4.3 Liveness detection
The 3D Smart Lock application preprogrammed in the SLN-VIZN3D-IOT kit comes with the liveness detection activated by default
to make the most of the security features of the 3D camera. This solution can discern between an actual face and a printout,
phone display picture, or 3D cast-mold of a face. This level of protection is crucial in applications that require security to prevent
“face-spoofing” attacks.
As shown in the screenshot below, attempting to display a fake face to the camera results in a “Fake Face” UI message and
audio prompt.
Figure 16. Printed picture (2D) and mask (3D) spoof attacks
By requiring a user’s actual face to unlock the system (as opposed to a simple picture of the user’s face), this feature helps to
protect you from some of the most common face-recognition “spoof” attacks, where a malicious actor uses a picture of someone
to gain access to their face-protected materials.
Using the combination of an IR camera and a 3D camera, outdoor environments with high and low lighting conditions do not impact
the performance of the face-recognition algorithm.
NXP Semiconductors
Local face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 14 / 47

Figure 17. 3D camera depth and IR views
NXP Semiconductors
Local face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 15 / 47

Chapter 5
Remote face management
The 3D Smart Lock application preprogrammed in the SLN-VIZN3D-IOT kit allows you to register and de-register faces locally
using the onboard pushbuttons or serial commands. It also supports remote face management over Bluetooth Low-Energy (BLE)
using an Android phone/tablet application. The Smart Lock Manager app for Android smartphones and tablets offers a friendly
interface where you can both register new faces remotely and manage faces that are already registered in the SLN-VIZN3D-IOT’s
local face database. This section describes some major features provided by this dedicated Android app.
5.1 Installing the Smart Lock Android app manually
The pre-compiled APK and the complete source code for the Smart Lock Manager app are on the NXP website at the
SLN-VIZN3D-IOT product page under the Software and Tools section.
To install the Smart Lock Manager app on your Android device, store the "Smart_Lock_Manager.apk" file on your smart device
by downloading it directly from your device’s web browser or by transferring it from your computer to your device. Before you can
install it on your Android device, make sure that third-party apps are allowed on your device. Go to "Menu > Settings > Security"
and tick "Unknown Sources" to allow your phone to install apps from sources other than the Google Play Store. As an alternative,
you are prompted to allow your browser or file manager to install APKs the first time you attempt to do so. Open the Files App
and search for the APK file location on your Android device (the "Downloads" folder by default), tap it, tap "Install" and wait for the
installation to complete.
Figure 18. Installing the Smart Lock Manager app for Android
After opening the Smart Lock Manager app for the first time, you are asked to authorize the app to access your smart device’s
camera, your location (to enable Bluetooth), and your photographs. It is important that you approve those services to connect new
kits or register new faces.
NXP Semiconductors
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 16 / 47

Figure 19. Allowing camera, location, and photograph access
If the Bluetooth radio is disabled on your Android device, the app asks you to turn it on. Bluetooth connectivity is required to
pair your kit with the Smart Lock Manager app and remotely manage the registered faces stored in the SLN-VIZN3D-IOT’s
local database.
Figure 20. Enabling Bluetooth
The Smart Lock Manager app is intended for use as an evaluation tool and as a reference/boiler plate upon which
a customized smartphone/tablet companion application can be built.
NOTE
5.2 Managing kits
To interact with your SLN-VIZN3D-IOT kit, pair your kit with the Smart Lock Manager app. This can be done from the app's main
menu. Press the “Device Scan” button to detect the VIZN3D devices that are within the Bluetooth range of your Android device.
The BLE feature of the SLN-VIZN3D-IOT is turned on by default, so the kit is automatically discovered by the app
shortly after powering up.
NOTE
NXP Semiconductors
Remote face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 17 / 47

While the search is in progress, any VIZN3D kits detected are displayed in the "Device" list. Select the device you want to add to
your list of supported smart locks.
Each kit broadcasts a unique Bluetooth SSID, starting with VN combined with its serial number to be
easily identified.
NOTE
Figure 21. Pairing with a new smart lock
Before accessing your device, enter a password to secure the connection between your Android device and the SLN-VIZN3D-IOT
kit you want to access. By default, this password is "000000".
Figure 22. Smart lock password screen
When (securely) connected with the VIZN3D device, the Android app displays a list of users currently registered in the kit’s local
face database.
Only one device can be connected and controlled at a time.
NOTE
NXP Semiconductors
Remote face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 18 / 47

Figure 23. Smart lock list of users currently registered
By selecting the “Change Password” command, you can modify the password previously registered to secure the Bluetooth
connection with the smart lock device. Enter the old password and the new password and click “Change”. The app automatically
returns to the previous "Smart Lock Control" menu. A message is displayed in green at the bottom of the screen to confirm that
the new password is successfully pushed to the device.
Figure 24. Changing smart lock password
5.3 Managing users
The Smart Lock Manager app can remotely manage the users registered to the SLN-VIZN3D-IOT kit using an Android device.
This includes adding, listing, and deleting users from the SLN-VIZN3D-IOT’s local face database using 2-way synchronization.
This section describes each of these features.
5.3.1 Adding users
One of the key features of the Smart Lock Manager app is registering new faces remotely from any Android device and uploading
them to the VIZN3D device via Bluetooth. To start the remote registration process, tap the “Add User” button, as shown in
Figure 25.
NXP Semiconductors
Remote face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 19 / 47

Figure 25. Remote registration menu
The Smart Lock Manager app automatically displays the live view of the front camera of your Android device. A green bounding
box appears when your face is detected. To register your face, click the shutter icon at the bottom of the camera view (as shown
in Figure 26) while the box around your face is green.
Figure 26. Remote face-capture screen
The app asks you to type a name of the new face. Tap the blue checkmark icon and select “Register” to finalize the registration.
Figure 27. Adding name of the new face
NXP Semiconductors
Remote face management
SLN-VIZN3D-IOT Kit User Guide, Rev. 0, 01 November 2021
User Guide 20 / 47