NXP Semiconductors SLN-LOCAL2-IOT User manual

SLN-LOCAL2-IOT User’s Guide
NXP Semiconductors Document identifier: SLN-LOCAL2-IOT-UG
User's Guide Rev. 0, 19 April 2021

Contents
Chapter 1 System Requirements and Prerequisites..............................................6
Chapter 2 Usage Conditions..................................................................................7
Chapter 3 Introduction........................................................................................... 8
Chapter 4 Getting Started with MCU Local Voice Control..................................... 9
4.1 Package and Collateral Content................................................................................................9
4.2 Power on................................................................................................................................... 9
4.3 Out-of-the-Box Demos.............................................................................................................10
4.3.1 Demo #1: LED Voice Control.................................................................................................... 11
4.3.2 Connecting to a Serial Terminal................................................................................................11
4.3.3 Switching Between the Out-of-the-Box Demos.........................................................................13
4.3.4 Demo #2: Smart Home in Multiple Languages......................................................................... 13
4.3.5 Demo #2: Elevator in Multiple Languages................................................................................ 16
4.3.6 Demo #2: Audio Device Control in Multiple Languages............................................................18
4.3.7 Demo #2: Washing Machine in Multiple Languages................................................................. 21
4.3.8 Demo #3: Dialog-style Voice Control........................................................................................ 23
Chapter 5 Controlling the Device......................................................................... 26
5.1 Physical Control Description....................................................................................................26
5.2 Shell Command Interface........................................................................................................27
5.2.1 WiFi Credentials........................................................................................................................27
5.2.2 Switching Out-of-the-Box Demos..............................................................................................28
5.2.3 Controlling the Volume .............................................................................................................28
5.2.4 Muting the Microphone .............................................................................................................28
5.2.5 Setting the Timeout................................................................................................................... 28
5.2.6 Enabling the Follow-up Mode....................................................................................................29
5.2.7 Enabling the Push-to-Talk (PTT) Mode.....................................................................................29
5.2.8 Checking the Software Version.................................................................................................29
5.3 USB Mass Storage Device (MSD) Mode.................................................................................30
Chapter 6 Product Specifications.........................................................................31
Chapter 7 References..........................................................................................32
Chapter 8 Revision History.................................................................................. 33
Chapter 9 Acronyms............................................................................................ 34
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 2 / 35

Figures
Figure 1. MCU Local Voice Control Kit package............................................................................................................................9
Figure 2. SLN-LOCAL2-IOT Kit content.........................................................................................................................................9
Figure 3. Connecting the SLN-LOCAL2-IOT Kit to the computer using a USB cable..................................................................10
Figure 4. RGB LED status Indicator.............................................................................................................................................10
Figure 5. Serial terminal settings................................................................................................................................................. 11
Figure 6. Available shell commands............................................................................................................................................ 12
Figure 7. Commands for Demo 1: LED Voice Control................................................................................................................. 13
Figure 8. Demo settings by Shell commands - changeto and multilingual.................................................................................. 13
Figure 9. Demo selection command............................................................................................................................................ 13
Figure 10. Language selection command....................................................................................................................................13
Figure 11. Result of changeto iot Shell command....................................................................................................................... 14
Figure 12. Result of multilingual Shell command.........................................................................................................................14
Figure 13. Examples of selecting multiple languages..................................................................................................................14
Figure 14. Available voice commands for Smart Home (IoT) demo in English, Chinese, German, and French......................... 15
Figure 15. LED Blue while listening to a voice command, same LED color for all Languages....................................................16
Figure 16. Detecting English wake word followed by a Smart Home command..........................................................................16
Figure 17. Result of changeto elevator Shell command.............................................................................................................. 16
Figure 18. Result of multilingual Shell command.........................................................................................................................16
Figure 19. Examples of selecting multiple languages..................................................................................................................17
Figure 20. Available voice commands for Elevator demo in English, Chinese, German, and French.........................................17
Figure 21. LED Blue while listening to a voice command, same LED color for all languages..................................................... 18
Figure 22. Detecting English wake word followed by an Elevator command...............................................................................18
Figure 23. Result of changeto audio Shell command.................................................................................................................. 18
Figure 24. Result of multilingual Shell command.........................................................................................................................19
Figure 25. Examples of selecting multiple languages..................................................................................................................19
Figure 26. Available voice commands for Audio Device Control demo in English, Chinese, German, and French.................... 20
Figure 27. LED Blue while listening to a voice command, same LED color for all languages..................................................... 21
Figure 28. Detecting English wake word followed by an Audio Device Control command.......................................................... 21
Figure 29. Result of changeto wash Shell command...................................................................................................................21
Figure 30. Result of multilingual Shell command.........................................................................................................................21
Figure 31. Examples of selecting multiple languages..................................................................................................................22
Figure 32. Available voice commands for Washing Machine demo in English, Chinese, German, and French......................... 22
Figure 33. LED Blue while listening to a voice command, same LED color for all languages..................................................... 23
Figure 34. Detecting English wake word followed by a Washing Machine command................................................................. 23
Figure 35. Result of changeto dialog Shell command................................................................................................................. 23
Figure 36. Available voice commands for Dialog demo...............................................................................................................23
Figure 37. Dialog-type Voice Control with oven appliance use case........................................................................................... 24
Figure 38. LED blue while listening to a voice command............................................................................................................ 24
Figure 39. Dialog demo examples............................................................................................................................................... 25
Figure 40. Shell command print to show the saved WiFi credentials.......................................................................................... 27
Figure 41. Connecting to WiFi network with incorrect credentials............................................................................................... 27
Figure 42. Erasing WiFi credentials.............................................................................................................................................28
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 3 / 35

Figure 43. Setting speaker volume to 30..................................................................................................................................... 28
Figure 44. Muting the microphone............................................................................................................................................... 28
Figure 45. Setting the waiting time to 7 seconds......................................................................................................................... 28
Figure 46. Follow-up mode use case...........................................................................................................................................29
Figure 47. Example of PTT mode usage..................................................................................................................................... 29
Figure 48. Check the software version........................................................................................................................................ 29
Figure 49. MSD update mode LED..............................................................................................................................................30
Figure 50. SLN-LOCAL2-IOT Kit mounted as a USB Mass Storage Drive..................................................................................30
NXP Semiconductors
Figures
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 4 / 35

Tables
Table 1. Tested computer configurations.......................................................................................................................................6
Table 2. Software tools and versions.............................................................................................................................................6
Table 3. Usage conditions............................................................................................................................................................. 7
Table 4. Summary of LED color and behavior.............................................................................................................................26
Table 5. Product specifications....................................................................................................................................................31
Table 6. Revision history..............................................................................................................................................................33
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 5 / 35

Chapter 1
System Requirements and Prerequisites
The MCU Local Voice Control SDK requires an up-to-date computer which runs MCUXpresso IDE. It also requires a terminal
program to communicate with the device via USB.
Table 1. Tested computer configurations
Computer type OS version Serial terminal application
PC Windows 10 TeraTerm, PuTTY
Mac macOS Serial, CoolTerm, goSerial
PC Linux PuTTY
Below are listed development tools using MCU Local Voice Control SDK.
Table 2. Software tools and versions
Software tool Version Description
Segger JLink_v6.98 or later Tool to program the flash.
MCUXpresso IDE Version 11.3.0 Eclipse base IDE for
development environment
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 6 / 35

Chapter 2
Usage Conditions
The following information is provided per Article 10.8 of the Radio Equipment Directive 2014/53/EU:
• Frequency bands in which the equipment operates
• The maximum RF power transmitted
Table 3. Usage conditions
PN RF technology (a) Frequency range (b) Max transmitted power
SLN-LOCAL2-IOT WiFi 2412MHz – 2472MHz 17.9dBm
EUROPEAN DECLARATION OF CONFORMITY (Simplified DoC per Article 10.9 of the Radio Equipment Directive 2014/53/EU)
This apparatus, namely SLN-LOCAL2-IOT, conforms to the Radio Equipment Directive 2014/53/EU. The full EU Declaration of
Conformity for this apparatus can be found at this location: https://www.nxp.com/
The product is expected to be used laying flat on a table, microphone output pointing up.
The data mode of the USB bus is not covered by the CE certification as this mode is used exceptionally to reprogram the device.
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 7 / 35

Chapter 3
Introduction
NXP’s MCU Local Voice Control 2nd generation development kit (part number: SLN-LOCAL2-IOT) is a comprehensive, secure
and cost-optimized turnkey solution with a widely-adopted development environment that enables customers to quickly get to
market with a production ready end-to-end software application.
SLN-LOCAL2-IOT embeds all the components required to produce a secure and edge-computing voice-control product without
need of Wi-Fi or Cloud connectivity. The architecture is built upon a single core:
•i.MX RT106S or RT105S for the main application, powered by an ARM® Cortex®-M7 core
SLN-LOCAL2-IOT hardware highlights:
• Up to 600 MHz (528 MHz default) Cortex-M7 MCU Core
• 1MB of On-chip RAM (512 KB TCM)
• 32 MB Hyper Flash Memory for Fast XiP (eXecute In Place)
• Three PDM MEMS Microphones
• TFA9894 Class-D Amplifier
• WiFi/Bluetooth Combo Chip
• Integrated Speaker
• GPIO Expansion Headers
SLN-LOCAL2-IOT software highlights:
• Two-Stage Bootstrap and Bootloader Allowing Flexibility in Customer’s Implementation
• Secure Boot Flow with High Assurance Booting (HAB)
• Over-the-Air (OTA) Update via WiFi
• Over-the-Wire (OTW) Update via UART
• Automated Manufacturing / Reprogramming Tools
• Speech Recognition Engine by Deep Learning
• Audio Front End (AFE) for Far-Field Automatic Speech Recognition (ASR)
SLN-LOCAL2-IOT is supported by a comprehensive and free-of-charge enablement suite from NXP and its partners including:
• MCUXpresso Development Tools
• Hardware Design Files
• Local Voice Application Software Source Code
• Software Audio Tuning Tools
• Documentation
• Training Material
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 8 / 35

Chapter 4
Getting Started with MCU Local Voice Control
4.1 Package and Collateral Content
The MCU Local Voice Control Kit arrives as shown in the following figure. Make sure to check for damage or marks and if found,
contact your NXP representative.
Figure 1. MCU Local Voice Control Kit package
When opening the box, the SLN-LOCAL2-IOT kit comes with a printed Quick Start Guide and a USB-C cable.
Figure 2. SLN-LOCAL2-IOT Kit content
4.2 Power on
Plug the USB Type-C connector into the SLN-LOCAL2-IOT kit and the dual Type-A connectors into your computer. Figure 3 shows
how to connect the kit using a USB cable.
NXP Semiconductors
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 9 / 35

Figure 3. Connecting the SLN-LOCAL2-IOT Kit to the computer using a USB cable
When you power on the kit for the first time, the device turns the LED green for about 2 seconds before turning off. Once the LED
is off, a chime is played from the onboard speaker indicating that the demo is running and listening to the wake word Hey, NXP.
Figure 4 shows the LED status while booting and being ready for listening to wake word and commands.
Figure 4. RGB LED status Indicator
4.3 Out-of-the-Box Demos
Three types of SLN-LOCAL2-IOT out-of-the-box demos are included:
• Demo #1: LED voice control
— Language: English
— Two-stage (wake word and command) ASR
• Demo #2: Smart Home (IoT) / Elevator / Audio / Washing-machine voice control in multiple languages
— Language: selectable combinations of English, Chinese, German, and French
— Two-stage ASR
• Demo #3: Oven voice control
— Language: English
— Multiturn (4-way) dialog-style ASR
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 10 / 35

4.3.1 Demo #1: LED Voice Control
Everything is ready out-of-the-box. Simply connect the kit to the computer and say Hey, NXP. The SLN-LOCAL2-IOT development
kit responds to your voice with the sound bite Can I help you? and turns the LED blue while waiting for your LED control command.
Say one of the following commands:
• L, E, D, Red
• L, E, D, Green
• L, E, D, Blue
• Cycle Fast
• Cycle Slow
If the kit detects your voice command, it changes the LED color accordingly and plays the sound bite OK!.
If the kit does not detect any of the LED control commands within a period of time, the device turns the LED purple and play a chime
to indicate the response waiting time has ended. By default the response waiting time is 8 seconds but you can change the value
with the Shell command timeout N where N is the time value in milliseconds.
4.3.2 Connecting to a Serial Terminal
The out-of-the-box demos in the following sections require a connection to a serial terminal in order to display detected wake words
and commands. Connect a serial terminal application to the USB serial device interface that enumerates (115200-8-N-1) as shown
Figure 5.
Figure 5. Serial terminal settings
Press Enter on the keyboard. The SHELL>> prompt appears.
Type help to show the available commands at the Shell. Figure 6 shows the available Shell commands with a description of each.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 11 / 35

Figure 6. Available shell commands
Type commands to ensure which demo is set in selected languages. Figure 7 indicates that the current demo is set to the LED
voice control demo.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 12 / 35

Figure 7. Commands for Demo 1: LED Voice Control
4.3.3 Switching Between the Out-of-the-Box Demos
Switching between the out-of-the-box demos can be done with Shell commands changeto and multilingual using the serial
terminal. Figure 8 shows how to switch between the demos.
Figure 8. Demo settings by Shell commands - changeto and multilingual
Figure 9 and Figure 10 shows the usage of changeto and multilingual respectively.
Figure 9. Demo selection command
Figure 10. Language selection command
4.3.4 Demo #2: Smart Home in Multiple Languages
Type changeto iot to switch the demo to Smart Home, as shown in Figure 11.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 13 / 35

Figure 11. Result of changeto iot Shell command
Type multilingual to ensure which languages are selected, as shown in Figure 12.
Figure 12. Result of multilingual Shell command
You can select any combinations of four different languages. For example, if you want to enable English and German, then enter
multilingual en de where en and de are the language codes of English and German respectively. If you want to enable four
languages, then enter multilingual en zh de fr where zh and fr are the language codes of Chinese and French respectively. You can
also enable one language only. Simply enter multilingual followed by the <language_code> you want to enable. Figure 13 shows
the examples. All the language selection is saved in flash memory, so it will remain in place even after the kit is rebooted.
Figure 13. Examples of selecting multiple languages
Type commands, to check that the current demo and the available voice commands are set in the selected languages. Figure 14
shows the commands of Smart Home in the selected four languages.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 14 / 35

Figure 14. Available voice commands for Smart Home (IoT) demo in English, Chinese, German, and French
Say a wake word in one of the selected languages. Four wake words are available:
• Hey, NXP (English)
• 你好, 恩智浦 (Chinese)
• Hallo, NXP (German)
• Salut, NXP (French)
If the SLN-LOCAL2-IOT kit is triggered by a wake word, it will respond to your voice with a sound bite in the detected language
and turns the LED blue, as shown in Figure 15, while waiting for your voice control command. The sound bite responses in the
four languages are:
• Can I help you? (English)
• 我可以帮你吗? (Chinese)
• Kann ich Ihnen helfen? (German)
• Puis-je vous aider? (French)
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 15 / 35

Figure 15. LED Blue while listening to a voice command, same LED color for all Languages
Say any command from the list shown in Figure 14. If the voice command is detected, the kit responds with the sound bite OK!
in the detected language accent.
Figure 16 shows a snapshot when detecting the English wake word Hey, NXP and Smart Home command Temperature Up.
Figure 16. Detecting English wake word followed by a Smart Home command
4.3.5 Demo #2: Elevator in Multiple Languages
Type changeto elevator to switch the demo to Elevator, as shown in Figure 17.
Figure 17. Result of changeto elevator Shell command
Type multilingual to check which languages are selected. Figure 18 shows the result of multilingual Shell command when English,
Chinese, German, and French languages are selected,
Figure 18. Result of multilingual Shell command
You can select any combinations of four different languages. For example, if you want to enable German and French, then enter
multilingual de fr where de and fr are the language codes of German and French respectively. If you want to enable four languages,
then type multilingual en zh de fr where en and zh are the language codes of English and Chinese respectively. You can also
enable one language only. To enable one language, type multilingual followed by the <language_code> you want to enable. Figure
19 shows the examples. All the language selection is saved in flash memory. So it remains even after the kit is rebooted.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 16 / 35

Figure 19. Examples of selecting multiple languages
Type commands to ensure that the current demo is set in selected languages. Figure 20 shows the commands of Elevator in the
selected four languages.
Figure 20. Available voice commands for Elevator demo in English, Chinese, German, and French
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 17 / 35

Say a wake word of selected languages. Four wake words are available:
• Hey, NXP (English)
• 你好, 恩智浦 (Chinese)
• Hallo, NXP (German)
• Salut, NXP (French)
If the SLN-LOCAL2-IOT kit is triggered by a wake word, it responds to your voice with audio in the detected language and turns the
LED blue, as shown in Figure 21, while waiting for your voice control command. The audio responses in the four languages are:
• Can I help you? (English)
• 我可以帮你吗? (Chinese)
• Kann ich Ihnen helfen? (German)
• Puis-je vous aider? (French)
Figure 21. LED Blue while listening to a voice command, same LED color for all languages
Say any command from the list shown in Figure 20. If the voice command is detected, the kit responds with the sound bite OK!
in the detected language accent.
Figure 22 shows the detection of the English wake word Hey, NXP and Elevator command Floor Three.
Figure 22. Detecting English wake word followed by an Elevator command
4.3.6 Demo #2: Audio Device Control in Multiple Languages
Type changeto audio to switch the demo to Audio Device Control, as shown in Figure 23.
Figure 23. Result of changeto audio Shell command
Type multilingual to check which languages are selected. Figure 24 shows the result of multilingual Shell command when English,
Chinese, German, and French languages are selected.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 18 / 35

Figure 24. Result of multilingual Shell command
You can select any combinations of four different languages. For example, if you want to enable English and German, then enter
multilingual en de where en and de are the language codes of English and German respectively. If you want to enable four
languages, then enter multilingual en zh de fr where zh and fr are the language codes of Chinese and French respectively. You
can also enable one language only. To enable one language, type multilingual followed by the <language_code> you want to
enable. Figure 25 shows the examples. All the language selection is saved in flash memory, so it remains in place even after the
kit is rebooted.
Figure 25. Examples of selecting multiple languages
Type commands, to check the available voice commands for the current demo in the selected languages. Figure 26 shows the
commands of Audio Device Control in the selected four languages.
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 19 / 35

Figure 26. Available voice commands for Audio Device Control demo in English, Chinese, German, and French
Say a wake word of selected languages. Four wake words are available:
• Hey, NXP (English)
• 你好, 恩智浦 (Chinese)
• Hallo, NXP (German)
• Salut, NXP (French)
If the SLN-LOCAL2-IOT kit is triggered by a wake word, it responds to your voice with a sound bite in the detected language
and turns the LED blue, as shown in Figure 27, while waiting for your voice control command. The audio responses in the four
languages are:
• Can I help you? (English)
• 我可以帮你吗? (Chinese)
NXP Semiconductors
Getting Started with MCU Local Voice Control
SLN-LOCAL2-IOT User’s Guide, Rev. 0, 19 April 2021
User's Guide 20 / 35