Espressif ESP32-S3-BOX-3 User manual

ESP32-S3-BOX-3 AIoT Development Kit User Guide
中⽂版本
Promotional Video
GitHub ESP-BOX SDK
Important Note :
Upon receiving the ESP32-S3-BOX-3, we advise updating the built-in factory firmware to ensure you
have the latest bugfixes via Espressif's online firmware flashing tool ESP Launchpad. Up till now, the
latest version is ESP-BOX-3_Demo_V1.2.3.
Hint - To verify the software version, please go to the 'About Us' Page.
Overview
In a world where innovation combines with artistry, where sharing and collaboration are paramount, we
introduce the ESP32-S3-BOX-3, a fully open-source AIoT development kit based on the powerful ESP32-
S3 SoC. This next-generation tool brings a positive change to traditional development boards, it features a
compact and beautiful enclosure also provides rich assemblies, empowering developers to easily
customize and expand its functionality with Espressif trusted software solutions such as ESP-BOX, ESP-
SR, ESP-Rainmaker and ESP-Matter. Whether you're prototyping a new project or building a complex IoT
system, the ESP32-S3-BOX-3 strikes the perfect balance of form and function, propelling the future of
development board ecology.
1

The ESP32-S3-BOX-3 comes with pre-built firmware supporting offline voice wake-up and speech
recognition in both Chinese and English. The ESP-BOX mobile APP facilitates reconfigurable AI voice
interaction, enabling you to create personalized commands for controlling smart devices. Furthermore, the
firmware showcases various sensor demos and enables users to perform IR learning, transforming the box
into a controller for in-house air conditioners.
This guide provides a comprehensive overview of the built-in firmware's capabilities to get you started on
your journey. As you read through, you'll gain valuable experience on developing your applications
independently. Let's embark on this exciting adventure together!
Target Group
The target group for this product includes individuals and organizations engaged in making, prototyping,
and development activities. It is designed to cater to a wide range of users, such as hobbyists, makers,
engineers, educators, and professionals seeking versatile and expandable solutions for their projects.
Target Application
Smart Home
Smart Agriculture
Smart City
Energy Management
IoT Prototyping and Education
Industrial Internet of Things
Smart Retail
2

Key Features
An out-of-the-box entry-level edge AI + HMI application development board.
FreeRTOS-based four-in-one voice interaction panel: offline voice recognition, network
communication, screen display, peripheral control.
Supports far-field voice interaction with 2 mics.
Offline voice wake-up and commands recognition with high wake-up rate.
Allows continuous recognition, wake-up interrupt, and over 200 customizable command words.
Flexible and versatile home automation solutions: Matter, Home Assistant, ESP-RainMaker.
Visual drag-and-drop GUI development: LVGL SquareLine Studio, Embedded Wizard, etc.
Rich development framework: ESP-IDF, Arduino, PlatformIO, Circuit Python, and more.
3

Hardware Introduction
Contents of the Kit
ESP32-S3-BOX-3, a main unit that can work on its own
ESP32-S3-BOX-3-DOCK, a functional accessory serving as a stand for the main box
ESP32-S3-BOX-3-SENSOR, a functional accessory showcasing sensor applications
ESP32-S3-BOX-3-BRACKET, an adapter accessory for mounting the main box to other devices
ESP32-S3-BOX-3-BREAD, an adapter accessory facilitating easy connection of the main box to a
standard breadboard
A USB-C Power Cable
An RGB LED module and Dupont wires for testing
ESP32-S3-BOX-3
4

ESP32-S3-BOX-3 is the main unit powered by the ESP32-S3-WROOM-1 module, which offers Wi-Fi +
Bluetooth 5 (LE) wireless capability as well as AI acceleration capabilities. On top of 512 KB SRAM
provided by the ESP32-S3 SoC, the module comes with additional 16 MB Quad flash and 16 MB Octal
PSRAM. The board is equipped a 2.4-inch 320 x 240 SPI touch screen (the ‘red circle’ supports touch),
two digital microphones, a speaker, 3-axis Gyroscope, 3-axis Accelerometer, one Type-C port for power
and download/debug, a high-density PCIe connector which allows for hardware extensibility, as well as
three functional buttons.
ESP32-S3-BOX-3-DOCK
5

ESP32-S3-BOX-3-DOCK is designed to serve as a stand for ESP32-S3-BOX-3 through the gold fingers,
and offers diverse expandability. It has two Pmod™ compatible headers, which allow users to expand
additional peripheral modules. These headers offer 16 programmable GPIOs that can supply 3.3 V power
to peripherals. One USB Type-A port for connecting to diverse USB devices such as USB camera (with a
maximum resolution of 720 P), USB disk, and other HID devices. While the another USB Type-C port for 5
V input power only.
ESP32-S3-BOX-3-SENSOR
ESP32-S3-BOX-3-SENSOR is a versatile accessory, which integrates Temp&Hum Sensor, IR Emitter and
Receiver, Radar Sensor, 18650 Rechargeable Battery slot and MicroSD Card slot. It empowers user to
create wide range of innovative projects with ease. Integrate multiple sensors for detection and control,
leverage the rechargeable battery for portability, and expand your storage capabilities using the MicroSD
Card slot (Expand storage up to 32 GB).
ESP32-S3-BOX-3-BRACKET
6

ESP32-S3-BOX-3-BRACKET can be utilized to help ESP32-S3-BOX-3 mount to other devices, opens up a
multitude of possibilities for transforming non-smart devices into smart ones. Installation of ESP32-S3-
BOX-3-BRACKET is straightforward, simply prepare 2 mounting holes and a slot using our template HERE.
By leveraging its two Pmod™ compatible headers, user can effectively integrate wireless connectivity,
voice control and screen control capabilities. The ESP32-S3-BOX-3-BRACKET adapter empowers you to
unleash the full potential of your non-smart devices.
ESP32-S3-BOX-3-BREAD
7

ESP32-S3-BOX-3-BREAD is an adapter that enables easy connection of the ESP32-S3-BOX-3 to a
standard breadboard, providing convenience for makers who enjoy hands-on projects to expand and
connect with other devices. It utilizes a high-density PCIe connector and two rows of 2.54 mm pitch pins
to expose the ESP32-S3's 16 programmable GPIOs, making ESP32-S3-BOX-3-BREAD highly practical and
functional.
8

Getting Started
Power on Your Device
Power on your device using the USB-C cable.
Once the device is powered on, the screen will play the Espressif logo boot animation.
Let's Play Around!
The initial two pages of the quick guide provide an overview of the functions of the buttons on your
BOX. Press the Nextbutton to proceed to the subsequent page.
9

The subsequent pages of the quick guide introduce how to use AI voice control. Tap OK Let’s Goto
access the menu.
The menu consists of six options: Sensor Monitor, Device Control, Network, Media Player,
Help, and About Us. You can move between these options by swiping left or right. For instance, you
can access the Device Controlscreen, tap on Lightto toggle the light on or off. Afterward, you
can return to the menu, enter the Media Playerscreen, and either play music or adjust the system
volume.
10

Press the mute button on the top of the device to disable voice wake-up and speech recognition.
Press it again to enable them.
Tap the red circle located below the screen to go back to the previous page. This red circle serves as a
customizable touch region, allowing users to customize extra touch capabilities according to their
preferences.
11

Sensor Monitor
This feature needs ESP32-S3-BOX-3 is mounted to the ESP32-S3-BOX-3-SENSOR dock. ESP32-S3-
BOX-3-SENSOR is a versatile accessory, which integrates a Temp&Hum Sensor, a pair of IR Emitter and
Receiver, and a Radar Sensor. It empowers user to create sensor network and other sensor applications
with ease. The built-in firmware provides a real-time environment Temp&Hum display, a demonstration of
human presence monitoring through a 2.4 GHz radar. Additionally, it offers an IR learning interface,
allowing users to use the ESP-BOX for IR learning of their air conditioning controller, thereby enabling
control over the air conditioner. This learning capability also extends to other in-house IR controllers, such
as Fan, TV, projector, providing a truly engaging experience.
Radar Monitoring
12

Enter the Sensor Monitorscreen and tap ON/OFFbutton to enable/ disable radar monitoring feature.
When the radar enables switch is set to ON , a red body icon will be shown if a person is detected in front
of the device, and the icon will turn gray if no person is detected within 2 minutes.
IR Learning
Below the temperature and humidity data and radar functions is the infrared learning module. Currently,
this module can only learn the ON/OFFfunction of a remote controller. Please follow the interface
instructions to iteratively learn the ON/OFFcommand of your remote controller a total of four times. The
interface upon successful learning will appear as follows:
13

Perform an ON/OFFtest on your air conditioner by pointing the ESP32-S3-BOX-3-SENSOR towards it. If
you find that the logic of the air conditioner's ON/OFFfunction is opposite to your expectations, please
click the Reversalbutton below to correct it. Of course, you can also click on Relearnto learn other
remote controllers.
Tips:
Please be aware that when the ESP32-S3-BOX-3 is not mounted to the ESP32-S3-BOX-3-SENSOR
dock, the entire Sensor Monitorfunction will not be operational.
During the period of experiencing the built-in firmware with ESP32-S3-BOX-3-DOCK, avoid hot-
plugging ESP32-S3-BOX-3-DOCK and switching to ESP32-S3-BOX-3-SENSOR. This action could
result in ESP32-S3-BOX-3-SENSOR not being recognized and functioning properly. To resolve
this, simply power again ESP32-S3-BOX-3-SENSOR to restore normal operation.
Due to the power limitations of the infrared emitter and differences among air conditioner brands,
the effective control range for the IR learning feature has been tested to be between 1 to 1.5
meters.
Offline Voice Assistant
14

Firstly, connect the RGB LED module to your device. Refer to the pin definitions in the table below, and use
Dupont wires to connect the R, G, B, and GND pins of the RGB LED module to the female header
connector on the Pmod™ interface of the ESP32-S3-BOX-3-DOCK.
You may say "Hi E. S. P." (pronounce in alphabet) at any screen to wake up your device.
When it wakes up, the screen will display the wake word you just used. If the wake word isn't
displayed, give it another try. The screen below indicates your device is listening.
Utter a command within 6 seconds after the beep, such as "Turn on the light". You will see the
command shown on the screen and the LED light on the module turned on, and you will hear an "OK".
If no more commands follow in around 6 seconds, you'll exit the voice control screen.
15

You can use voice commands to enjoy music. First, wake up the device, then say "Sing a song".
This will launch the music player and start playing built-in music. You can also use voice commands to
pause or skip songs. There are two built-in songs.
Tips:
If the LED light fails to turn on, check whether the module pins are inserted into the right ports.
If the BOX doesn't recognize any command within the designated timeframe, a Timeoutmessage
will appear, and the screen will exit in approximately 1 second.
The default commands are: turn on the light, turn off the light, turn red, turn green, turn blue, sing
a song, next song, pause playing.
Continuous Speech Recognition
More interestingly, the device supports continuous speech recognition after waking up. This feature adds
a natural and seamless quality to voice interaction, enhancing the interactive experience with a human
touch.
How to use
Say "Hi, E. S. P"to wake up the device, and you will hear a beep.
Speak your command. Upon successful recognition, you will hear "OK", and the device will remain
ready for further commands.
If no command is identified, the device will wait. Should no commands be given within 6 seconds, the
device will autonomously exit the voice control screen and you need to wake it up again.
16

Attention
If your commands are repeatedly unrecognized by the device, allow it to time out and then wake it up
again before utilizing the feature.
Upon saying the wake-up word, please do not move the device, as this could impact command
recognition.
We recommend that voice commands comprise 3-5 words for optimal results.
Currently, the device cannot recognize commands while it's playing beep.
Voice Command Customization
The ESP32-S3-BOX-3 is also equipped with Espressif proprietary AI Speech Recognition System, which
allows you to customize commands through our ESP BOX APP. We will take the LED light on the module as
an example, to show how to create your own voice commands. For a deeper understanding of the
algorithms involved, please refer to the introduction of the ESP-SR speech model in the Technical
Architecture.
1. Connect to the ESP BOX mobile app
1.1. Enter Network, and tap To install APPat the upper right corner. Scan the QR Code or search
"ESP BOX" in App Store or Google Play to install the app.
1.2. If you are new to this app, please register an account first.
1.3. Sign in with your ESP BOX account and turn on the Bluetooth on your phone. Tap +at the bottom of
the screen, and scan the QR code on your device to set up the network.
17

1.4 After adding the device, you will see the following prompts:
And this prompts in the box:
18

Tips:
Please do not exit the QR code page during the network provisioning.
Make sure you connect the device to 2.4 GHz Wi-Fi instead of 5 GHz, and enter the correct Wi-Fi
password. If the Wi-Fi password is incorrect, the prompt "Wi-Fi Authentication failed" will pop up.
Long press the Bootbutton (i.e. Funtionbutton) for 5 seconds to clear the network information
and restore"the device to factory"settings. After the device is reset, if the QR code or Bluetooth is not
working, please restart your device by pressing the Resetbutton.
2. Customize Voice Commands
2.1. Select the ESP-BOX device icon and access the screen displayed below. You can conveniently turn the
light on or off by simply toggling the button, as indicated in the image. You may need to develop the Fan
and Switch functionalities by yourself.
19

2.2 Tap Light, and the Configuretab shows the default pin information and commands. The pins for
Red, Green, and Blue can be changed as needed.
2.3 In the Configuretab, you may also customize commands to turn on or off the light and change its
color. For example, you can set "Good Morning" as the command to turn on the light. Click Saveto return
to the previous screen. Then click Saveagain as shown below.
20