NXP Semiconductors HRPNUG User manual

HRPNUG
Harpoon User's Guide
Rev. 2.2 — 16 December 2022 User guide
Document information
Information Content
Keywords i.MX 8M device family, Arm Cortex-A53 processor (Armv8-A architecture),
RTOS, Linux, hardware partitioning, Jailhouse hypervisor, NXP Linux Yocto,
Zephyr RTOS, FreeRTOS, MCUXpresso SDK
Abstract This document presents the Harpoon release 2.2 for i.MX 8M device family,
using the Arm Cortex-A53 processor (Armv8-A architecture).

NXP Semiconductors HRPNUG
Harpoon User's Guide
1 Overview
This document presents the Harpoon release 2.2 for i.MX 8M device family, using the
Arm Cortex-A53 processor (Armv8-A architecture).
Harpoon provides an environment for developing real-time demanding applications on
an RTOS running on one (or several) Cortex-A core(s) in parallel of a Linux distribution,
leveraging the 64-bit Arm architecture for higher performance.
The system starts on Linux and the Jailhouse hypervisor partitions the hardware to run
both Linux and the guest RTOS in parallel.
The hardware partitioning is configurable and depends on the use case. This release
includes an audio application, an industrial application and a real-time latency
measurement application, all available both for FreeRTOS as well as Zephyr (some
application feature limitations exist depending on the selected platform and RTOS).
This release supports the following software and hardware:
•NXP Linux Yocto
–i.MX LF 5.15.71-2.2.0: For more information, see the i.MX Yocto Project User's
Guide.
–Real-time Edge SW v2.4: For more information, see the Real-time Edge Yocto
Project v2.4 User Guide.
•i.MX 8M Series (EAR support)
–i.MX 8M Mini LPDDR4 EVKB
–i.MX 8M Nano LPDDR4 EVK
–i.MX 8M Plus LPDDR4 EVK
•Jailhouse hypervisor
•FreeRTOS V10.4.3 kernel
–Cortex-A53 port, uniprocessor
–Guest OS running on Jailhouse cell
•Zephyr RTOS 3.2.0
–Cortex-A53 port, SMP
–Guest OS running on Jailhouse cell
•MCUXpresso SDK 2.12
–GIC, Timer and MMU Cortex-A53 drivers
–CAN, ENET, ENET_QOS, GPT, I2C, SAI, and UART SoC drivers
–Audio Codec drivers
–PHY drivers
•RTOS applications
–Audio reference application
–Industrial reference application
–Real-time latency measurement application
1.1 Architecture
The following figure shows the architecture of the Harpoon solution.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
2 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Figure 1. Harpoon solution architecture
The i.MX 8M box shows the hardware partitioning between Jailhouse cells.
The boxes in dark orange (group 1) show the main hardware blocks allocated to the
Linux OS.
The boxes in blue (group 3) show the main hardware blocks allocated to the RTOS.
The boxes in light orange (group 2) show the main hardware blocks shared between
Linux and the RTOS.
Harpoon-apps is the real-time application running on Jailhouse's inmate cell. It is built on
top of Zephyr or FreeRTOS, using MCUXpresso drivers.
1.2 Hardware resource partitioning
Jailhouse hypervisor is used to run an RTOS in parallel with Linux: FreeRTOS and
Zephyr are supported in this release.
Jailhouse is a simple hypervisor that assigns hardware resources to a guest OS instead
of virtualising them. For instance, a CPU core is statically assigned to a specific guest
and is not shared with other guests.
In Jailhouse terms, the RTOS (inmate) runs in a cell. A configuration file describes which
hardware resources are assigned to this cell. This configuration file contains descriptions
of the following:
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
3 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
•CPU cores assigned to the cell
•Interrupt lines assigned to the cell
•Memory regions assigned to the cell
•Virtual PCI devices used for communication between cells
There is also a root cell configuration that describes the hardware prior to the hardware
partitioning.
The source files of the cell configurations are embedded through patches in the Jailhouse
recipe of the Harpoon meta-layer, at the following locations:
•configs/arm64/imx8m*-freertos.c for the cell configuration of the FreeRTOS
rt_latency use case
•configs/arm64/imx8m*-zephyr.c for the cell configuration of the Zephyr
rt_latency use case
•configs/arm64/imx8m*-freertos-audio.c for the cell configuration of the
FreeRTOS audio use case
•configs/arm64/imx8m*-freertos-avb.c for the cell configuration of the
FreeRTOS audio (AVB) use case
•configs/arm64/imx8m*-zephyr-audio.c for the cell configuration of the Zephyr
audio use case
•configs/arm64/imx8m*-freertos-industrial.c for the cell configuration of
the FreeRTOS industrial use case
•configs/arm64/imx8m*-zephyr-industrial.c for the cell configuration of the
Zephyr industrial use case
•configs/arm64/imx8m*.c for the root cell configuration
The CPU core allocated to the RTOS forms a bitmap in the cpu structure. Here, CPU
core 3 is assigned to the cell:
.cpus = {
0b1000,
},
For a multicore (SMP) cell, two cores can be used, for instance:
.cpus = {
0b1100,
},
Memory regions assigned to the inmate cell are listed in the mem_regions structure.
Memory regions can be reserved for the inmate cell or shared with the Linux root cell.
Memory regions can be DDR chunks for the inmate cell use as well as device memory
mapped regions such as UART or SAI.
Interrupts are mapped to the cell with the irqchips structure.
Virtual PCI devices are defined with the pci_devices structure. These virtual devices
are used by Jailhouse to implement IVSHMEM v2 communication channels.
2 Building Harpoon Yocto images
As mentioned in the overview section, Harpoon is compatible with both i.MX Yocto and
Real-Time Edge Yocto. Each distribution is addressed in a separate section below.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
4 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
2.1 i.MX Yocto
To build this release, fetch its Yocto manifest and get the meta-layers:
$ mkdir yocto
$ cd yocto
$ repo init -u https://github.com/nxp-imx/imx-manifest -b imx-
linux-kirkstone -m imx-5.15.71-2.2.0_harpoon-v2.xml
$ repo sync
Then, prepare the environment with the following command:
$ DISTRO=fsl-imx-xwayland MACHINE=<machine> source imx-harpoon-
setup-release.sh -b build.<machine>
Where, <machine> is one of the following:
•imx8mm-lpddr4-evk for i.MX 8M Mini EVKB board
•imx8mn-lpddr4-evk for i.MX 8M Nano EVKB board
•imx8mp-lpddr4-evk for i.MX 8M Plus EVK board
The end user license agreement must be accepted to continue.
Then build the image with the following command:
$ bitbake imx-image-core
The image is then available in subdirectory tmp/deploy/images/<machine>/.
Copy the disk image to a micro-SD card. For example, assuming the card is recognized
as /dev/mmcblk0 by your host machine:
$ zstdcat imx-image-core-<machine>.wic.zst | sudo dd of=/dev/
mmcblk0 bs=1M
The micro-SD card now contains the release.
2.2 Real-Time Edge Yocto
See the Real-time Edge Yocto Project User Guide to build Harpoon and prepare an SD
card for supported boards.
3 Hardware Setup
3.1 i.MX Reference Boards
This Harpoon release supports the following development boards.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
5 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Figure 2. i.MX 8M Mini EVK
Note: For more information to order the board, see https://www.nxp.com/design/
development-boards/i-mx-evaluation-and-development-boards/evaluation-kit-for-the-i-
mx-8m-mini-applications-processor:8MMINILPD4-EVK
Figure 3. i.MX 8M Nano EVK
Note: For more information to order the board, see https://www.nxp.com/design/
development-boards/i-mx-evaluation-and-development-boards/evaluation-kit-for-the-i-
mx-8m-nano-applications-processor:8MNANOD4-EVK.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
6 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Figure 4. i.MX 8M Plus EVK
Note: For more information to order the board, see https://www.nxp.com/design/
development-boards/i-mx-evaluation-and-development-boards/i-mx-8m-plus-evaluation-
kit-enabling-power-measurement:8MPLUSLPD4-PEVK.
3.2 Audio use case hardware
Harpoon audio application uses the I2S HiFiBerry audio card DAC+ ADC Pro.
Figure 5. HiFiBerry DAC+ ADC Pro (picture from HiFiBerry's website)
Note: For more information to order the board, see https://www.hifiberry.com/shop/
boards/hifiberry-dac-adc-pro/.
The HiFiBerry DAC+ ADC Pro is an audio card designed for the Raspberry Pi, but it can
be connected to EVK boards using the 40-pin connector, provided a few adaptations are
made.
The following pins on the EVK's 40-pin connector must be connected to the following
HiFiBerry's pins.
EVK HiFiBerry Function
2 2 5V
Table 1. EVK - HiFiBerry transposition
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
7 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
EVK HiFiBerry Function
3 3 I2C SDA
5 5 I2C SCK
6 6 GND
35 40 I2S TX
36 12 I2S clock
37 35 I2S word select for RX and TX
38 38 I2S RX
Table 1. EVK - HiFiBerry transposition...continued
Figure 6. Handmade transposer
A complete setup, with a handmade transposer to respect above pinout, is shown as
follows.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
8 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Figure 7. i.MX 8M Mini EVK with HiFiBerry audio card
The audio card has both an ADC (PCM1863) to record audio and a DAC (PCM5122) for
audio playback.
Record is done through the audio jack (connector highlighted in 1 in the following figure)
and playback is done through the RCA connectors (highlighted in 2).
Figure 8. HiFiBerry audio connectors (picture from HiFiBerry's website)
Note: For more information to order the board, see https://www.hifiberry.com/shop/
boards/hifiberry-dac-adc-pro/.
Control of the PCM1863 is done through I2C3, at address 0x4a.
Control of the PCM5122 is done through I2C3, at address 0x4d.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
9 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Both the PCM1863 and PCM5122 use i.MX I2S5. The I2S5 is the I2S clock master.
Two oscillators (one for sampling frequencies multiple of 44,100 Hz, one for sampling
frequencies multiple of 48,000 Hz) are present on the HiFiBerry card, and controlled by
PCM5122 GPIOs.
The following diagram shows the HiFiBerry architecture.
Figure 9. HiFiBerry architecture
The PCM1863 and the PCM5122 use the same signal for I2S word select by using SAI
synchronous mode.
3.3 Industrial use case hardware
Harpoon's industrial application may use the following hardware depending on the use
case.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
10 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Figure 10. LS1028A AVB/TSN network bridge
Note: For more information to order the board, see https://www.nxp.com/design/qoriq-
developer-resources/layerscape-ls1028a-reference-design-board:LS1028ARDB.
The LS1028A RDB is used as a TSN bridge/switch in a TSN network to demonstrate the
TSN Ethernet use case running from the inmate cell.
Figure 11. RT1170 TSN endpoint
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
11 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Note: For more information to order the board, see https://www.nxp.com/design/
development-boards/i-mx-evaluation-and-development-boards/i-mx-rt1170-evaluation-
kit:MIMXRT1170-EVK.
The RT1170 is used as a TSN endpoint in a TSN network, exchanging packets with the
i.MX 8MP board.
4 Running Harpoon Reference Applications
4.1 Basic setup
The EVK boards expose serial ports through their USB debug interface. One of these
serial ports is used by Linux for its console, and another one is used by the guest RTOS.
To run the reference applications, open both serial ports with terminal emulators, insert
the micro-SD card on which the Yocto image has been flashed in the EVK and power up
the board.
4.2 Starting Linux kernel
Linux kernel must be started with a (Harpoon specific) Jailhouse compatible device tree.
To do this, when U-Boot is executing, stop at U-Boot prompt with a terminal emulator
connected to the serial port and execute the following command (based on the board and
the application):
•For i.MX 8M Mini (audio or rt_latency):
u-boot => setenv jh_root_dtb imx8mm-evk-harpoon.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Mini (audio AVB):
u-boot => setenv jh_root_dtb imx8mm-evk-harpoon-avb.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Mini (industrial or rt_latency):
u-boot => setenv jh_root_dtb imx8mm-evk-harpoon-industrial.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Nano (audio or rt_latency):
u-boot => setenv jh_root_dtb imx8mn-evk-harpoon.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Nano (audio AVB):
u-boot => setenv jh_root_dtb imx8mn-evk-harpoon-avb.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Nano (industrial or rt_latency):
u-boot => setenv jh_root_dtb imx8mn-evk-harpoon-industrial.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Plus (audio or rt_latency):
u-boot => setenv jh_root_dtb imx8mp-evk-harpoon.dtb
u-boot => run jh_mmcboot
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
12 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
•For i.MX 8M Plus (audio AVB):
u-boot => setenv jh_root_dtb imx8mp-evk-harpoon-avb.dtb
u-boot => run jh_mmcboot
•For i.MX 8M Plus (industrial or rt_latency):
u-boot => setenv jh_root_dtb imx8mp-evk-harpoon-industrial.dtb
u-boot => run jh_mmcboot
Note: This configuration is not persistent after a reboot.
To make changes permanent, execute the following commands once (after setenv
above):
u-boot => setenv bootcmd 'run jh_mmcboot'
u-boot => saveenv
Now, at each reboot, the system starts with the Jailhouse compatible configuration and
no user interaction is required.
4.3 Audio application
4.3.1 Features of the audio application
The audio application is available in the harpoon share directory of the target's root file
system:
/usr/share/harpoon/inmates/freertos/audio.bin # FreeRTOS
binary
/usr/share/harpoon/inmates/zephyr/audio.bin # Zephyr binary
/usr/share/harpoon/inmates/freertos/audio_rpmsg.bin # FreeRTOS
binary of RPMSG based control channel
Note: In the current release, the RPMsg based control channel is only supported under
FreeRTOS on i.MX 8M Mini EVK.
This application contains several running modes that can be started and
stopped via a user space application running on Linux called harpoon_ctrl/
harpoon_ctrl_rpmsg.
Note: The harpoon_ctrl_rpmsg is used to control the binaries named with a
_rpmsg, which is controlled through the RPMsg based channel.
The different modes are:
•DTMF playback: plays a DTMF sequence.
•Sine wave playback: plays a generated sine wave.
•Loopback: record sound from HiFiBerry's input and play it live through HiFiBerry's
output.
•Full Audio pipeline: implements a flexible 3-stage pipeline with different sources (DTMF,
sine waves, SAI input) that can be routed to different sinks (SAI outputs).
•AVB Audio pipeline (FreeRTOS only): implements a 3-stage pipeline with AVB input as
a source that can be routed to different sinks (SAI outputs).
•SMP Audio pipeline (Zephyr only): splits the above pipeline in two pieces to process
them onto different cores.
All the modes support (see Notes for exceptions):
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
13 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
•Basic pipeline framework for audio processing
•44100, 48000, 88200, 96000, 176400, and 192000 Hz sample frequencies
•Audio processing period with 2, 4, 8, 16, or 32 frames
•Audio processing in 64bit float format
•Audio playback to both SAI3 (on board codec/sound jack) and SAI5 (HifiBerry)
•Audio capture from SAI5 (HifiBerry)
Note: Playback on SAI3: The i.MX 8M Plus EVK on board codec (WM8960) supports
sample rates up to 48 kHz only. 88.2 kHz and above frequency settings will fail for this
codec.
Note: SMP Audio Pipeline supports only 48000, 96000, and 192000 Hz sample
frequencies.
4.3.2 Starting the audio application with Jailhouse
The Harpoon service uses the /etc/harpoon/harpoon.conf configuration file that
contains the RTOS and the application to run. By default, the configuration file points
to the FreeRTOS audio application. To use the Zephyr audio application, the following
command can be run to generate an appropriate configuration file:
# harpoon_set_configuration.sh zephyr audio
If want to control the application through the RPMsg based channel, run the following
command to generate a configuration file:
# harpoon_set_configuration.sh freertos audio rpmsg
Note: Avoid changing the configuration while the Harpoon service is running (silent
failure when restarting the service).
To use the audio application, Jailhouse must be started first. To start Jailhouse and the
audio application, run the Harpoon service with systemd:
# systemctl start harpoon
Once the Harpoon service has been started, harpoon_ctrl/harpoon_ctrl_rpmsg is
used to start or stop the audio modes with optional parameters. The different options for
the audio application are:
Audio options:
-f <frequency> audio clock frequency (in Hz)
Supporting 44100, 48000, 88200, 176400,
96000, 192000 Hz
Will use default frequency 48000Hz if
not specified
-p <frames> audio processing period (in frames)
Supporting 2, 4, 8, 16, 32 frames
Will use default period 8 frames if not
specified
-r <id> run audio mode id:
0 - dtmf playback
1 - sine wave playback
2 - playback & recording (loopback)
3 - audio pipeline
4 - AVB audio pipeline
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
14 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
5 - SMP audio pipeline
-s stop running audio mode
Audio pipeline options:
-a <pipeline_id> audio pipeline id (default 0)
-d audio pipeline dump
Audio element options:
-a <pipeline_id> audio pipeline id (default 0)
-d audio element dump
-e <element_id> audio element id (default 0)
-t <element_type> audio element type (default 0):
0 - dtmf source
1 - routing
2 - sai sink
3 - sai source
4 - sine source
5 - avtp source
Routing audio element options:
-a <pipeline_id> audio pipeline id (default 0)
-c connect routing input/output
-d disconnect routing input/output
-e <element_id> routing element id (default 0)
-i <input_id> routing element input (default 0)
-o <output_id> routing element output (default 0)
4.3.3 Audio latency in loopback mode
The loopback mode reads audio samples from HiFiBerry's ADC in an audio buffer and
sends this buffer to the HiFiBerry's DAC when fully loaded.
The end to end latency, between the analog audio input and the analog audio output, has
been measured and is dependent on the audio buffer size and the audio sampling rate.
The RTOS and SoC combination does not alter the latency measurements.
Audio latency (μs)
Audio buffer size (frames)
Sampling
rate (kHz)
32 16 8 4 2
192 610 440 360 320 300
176.4 660 480 390 340 330
96 1210 870 700 630 580
88.2 1310 940 770 680 630
48 2380 1720 1390 1220 1140
44.1 2600 1880 1510 1310 1240
Table 2. Audio application latency
4.3.4 Running audio application: examples
If specified rpmsg option during the configuration, use the harpoon_ctrl_rpmsg
command instead.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
15 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
4.3.4.1 Playing DTMF
To start DTMF playback with default parameters (48000 Hz sampling rate):
# harpoon_ctrl audio -r 0
The DTMF is played both to the Hifiberry RCA outputs as well as the onboard jack.
To run another audio use case, the playback must be stopped with the following
command:
# harpoon_ctrl audio -s
4.3.4.2 Playing in loopback mode
In loopback mode, the SAI input is copied to the SAI output.
To start loopback mode with default parameters (48000 Hz sampling rate, 8 frame period
size):
# harpoon_ctrl audio -r 2
To run another audio use case, the playback must be stopped with the following
command:
# harpoon_ctrl audio -s
4.3.4.3 Playing a full audio pipeline
The reference audio application is based on a basic pipeline framework for audio
processing. Different audio processing elements can be assembled in a pipeline to
process audio from source(s) to sink(s). The pipeline is processed in real time, cyclically
with a fixed period.
In the audio pipeline mode there is a three stage pipeline composed of a routing element
in stage 2 which can link source elements from stage 1 to sink elements from stage 3.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
16 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Audio pipeline with multiple sources/sinks and a routing element
Figure 12. Audio pipeline
When running the audio pipeline, the routes can be configured dynamically with the
harpoon_ctrl command. This command uses source and sink indices to connect
elements.
Index Source element Comment
0 DTMF, sequence 1 Software generated source
1 DTMF, sequence 2 Software generated source
2 Sine wave, 440 Hz Software generated source
3 Sine wave, 880 Hz Software generated source
4 SAI5, left channel Hardware source
5 SAI5, right channel Hardware source
6 SAI3, left channel Hardware source
7 SAI3, right channel Hardware source
Table 3. Indices of source elements
Index Sink element Comment
0 SAI5, left channel Hardware sink
1 SAI5, right channel Hardware sink
2 SAI3, left channel Hardware sink
3 SAI3, right channel Hardware sink
Table 4. Indices of sink elements
This makes for a flexible pipeline. For instance, the following commands starts the
pipeline and configures the routing element to have a loopback between SAI5 input and
SAI3 output (i.e., sound recorded by the HiFiBerry card played by the EVK's internal
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
17 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
codec) while a DTMF sequence is played on the left channel of SAI5's output and a 440
Hz sine wave on the right channel of SAI5's output (i.e., HiFiBerry's output):
# harpoon_ctrl audio -r 3 # start audio pipeline
# harpoon_ctrl routing -i 4 -o 2 -c # SAI5's input to SAI3's
output (L)
# harpoon_ctrl routing -i 5 -o 3 -c # SAI5's input to SAI3's
output (R)
# harpoon_ctrl routing -i 0 -o 0 -c # DTMF to SAI5's output
(L)
# harpoon_ctrl routing -i 2 -o 1 -c # sinewave 440Hz to
SAI5's output (R)
4.3.4.4 Playing an AVB audio pipeline
The AVB audio pipeline embeds - as a source element - an AVB Listener, making use of
the GenAVB/TSN stack streaming API’s. This element is only responsible of the audio
data path:
•Supports one or more AVTP Listener streams
•Supports multi-channel AVTP streams
•Supports scatter of audio data
•Supports audio format conversion, from AVTP stream format to the common format
It resuses the audio application's pipeline framework for audio processing in which an
AVTP Listener is added as a source.
AVB Audio pipeline showing the AVTP Listener with multiple streams as a source.
Figure 13. AVB Audio pipeline
When running the AVB audio pipeline, the routes can be configured dynamically with
the harpoon_ctrl command. This command uses source and sink indices to connect
elements.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
18 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
Index Source element Comment
0 Sine wave, 440 Hz Software generated source
1 SAI5, left channel Hardware source
2 SAI5, right channel Hardware source
3 SAI3, left channel Hardware source
4 SAI3, right channel Hardware source
5 AVTP, stream#0 left channel AVB source from network
6 AVTP, stream#0 right channel AVB source from network
7 AVTP, stream#1 left channel AVB source from network
8 AVTP, stream#1 right channel AVB source from network
Table 5. Indices of source elements
Index Sink element Comment
0 SAI5, left channel Hardware sink
1 SAI5, right channel Hardware sink
2 SAI3, left channel Hardware sink
3 SAI3, right channel Hardware sink
Table 6. Indices of sink elements
The sections below describe how to set up an (external) AVB Audio Media Server to
enable the (Harpoon) AVB Listener.
4.3.4.4.1 AVB setup preparation
An i.MX 8M Plus EVK with Real-time Edge SW v2.4 can be used as a Talker. On the
other end, any Harpoon supported EVK can be used as a Listener.
1. Connect the headphones/speakers to the HifiBerry's RCA output or the Listener's
audio Jack port.
2. Connect both the i.MX boards with an Ethernet RJ45 cable.
3. Connect a Serial/USB cable to each i.MX board and to some USB ports of the host
PC.
4. Start consoles of the i.MX boards through the serial/USB ports.
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
19 / 54

NXP Semiconductors HRPNUG
Harpoon User's Guide
AVB Talker (blue box) connected to the AVB Listener (orange box) through Ethernet.
Figure 14. AVB Audio setup
4.3.4.4.2 AVB Talker configuration
The default AVB script needs to be modified to configure operations of the Talker entity
as using a custom Media Application. The AVB Stack is provided with a simple Media
Server application example, interfaced to the AVB stack through the GenAVB/TSN API,
and supporting reading audio samples from a media file.
To enable AVB streaming using this media application, the endpoint needs to be
configured as Endpoint AVB and the GenAVB/TSN configuration files needs to be
modified as follows:
1. Power on the i.MX board and let the boot process complete
2. Configure the GenAVB/TSN stack to Endpoint AVB mode by setting
GENAVB_TSN_CONFIG to the right value in the GenAVB/TSN mode configuration
file:
# vi /etc/genavb/config
For i.MX 8M Plus EVK:
GENAVB_TSN_CONFIG=2
3. Save and exit the file
4. Edit the GenAVB/TSN AVB configuration file using the following command:
# vi /etc/genavb/config_avb
5. Set the configuration profile to PROFILE 2
PROFILE=2
6. Save and exit the file.
7. A raw audio file sample1_for_aaf.raw is available in the /home/media
repository. The multi-stream application example looks for audio files named
HRPNUG All information provided in this document is subject to legal disclaimers. © 2022 NXP B.V. All rights reserved.
User guide Rev. 2.2 — 16 December 2022
20 / 54