Nvidia Clara AGX User manual

Clara AGX –Getting Started
NVIDIA Confidential 1
Clara AGX Developer Kit User Guide
Purpose: Provides the instructions to flash, setup, and start using a Clara AGX Developer Kit.
Disclaimer: The Clara AGX Developer Kit is not an approved medical device and is not
intended for clinical use.
Version: 1.5
Contents
Checklist for Setting up the Developer Kit ............................................................. 2
Hardware Setup............................................................................................. 2
Requirements............................................................................................. 2
Precautions ............................................................................................... 2
System Overview ........................................................................................... 3
Main Components ........................................................................................ 3
Tech Specs................................................................................................ 3
I/O and external interfaces ............................................................................ 4
Powering up the System ................................................................................... 6
Check Firmware Version and Manually Update if Needed ............................................ 7
Flashing and Updating Clara AGX Developer Kit using SDK Manager ................................ 8
Switching between iGPU and dGPU...................................................................... 8
Setting up SSD Storage.................................................................................... 10
Create Partition......................................................................................... 11
Mount Partition ......................................................................................... 12
Setting up Docker and Docker Storage on SSD ........................................................ 12
Move Docker Storage to m2 Partition................................................................ 13
Install the Clara Holoscan SDK........................................................................... 14
Known Issues ............................................................................................... 14
Additional Resources...................................................................................... 14
Compliance Information.................................................................................. 15

Clara AGX –Getting Started
NVIDIA Confidential 2
Checklist for Setting up the Developer Kit
After receiving the Clara AGX Developer Kit, ensure the following actions are taken before
developing on the kit. Each action is described in its corresponding section of this user guide.
•Read through the Hardware Setup requirements and precautions.
•Familiarize yourself with the System Overview: the main components and system I/O.
•Power up the system.
•If you had previously flashed the devkit, check firmware version and manually update if
needed. If this is your first time flashing, come back to the step at the end of the
checklist.
•Flash and update the Clara AGX Developer Kit with Jetpack using SDK Manager.
•Switch from iGPU to dGPU mode.
•Set up the 250GB SSD storage.
•Set up Docker and Docker storage.
•Install the Clara Holoscan SDK from Github.
Hardware Setup
Requirements
•A Clara AGX Developer Kit
•A compatible power cable
oYour NVIDIA Clara AGX Developer Kit may not include a power cable compatible
with your local electrical requirements.
oA compatible cable should meet the following requirements:
▪Provides a certified local 3-prong AC power plug
▪Provides a C13 connector
▪Supports ratings of 100-120VAC/6A, 200-240VAC/3A, or higher with a
minimum wire thickness of 18AWG and insulation rating of 300V or higher.
•An Ubuntu 18.04 / 20.04 host system (for use during flashing)
•A standard USB-A to USB-C or USB-C to USB-C cable with data enabled (for use during
flashing)
•Connection to the Internet for the host system before and during flashing, and for the
Clara AGX Developer Kit during flashing
•A keyboard, mouse, and monitor with HDMI for the Clara AGX Developer Kit
Precautions
•Only connect and disconnect a PCIe card (e.g. miniSAS or dGPU) when the system is
powered down.
•Apply extra care when plugging and removing PCIe cards to avoid stress on the PCIe
connectors (wearing, bending, breaking).
•The rightmost USB connector is USB 2.0 (even if the color is blue). The other two USB
connectors are USB 3.0.

Clara AGX –Getting Started
NVIDIA Confidential 3
System Overview
Main Components
The Clara AGX Developer Kit contains the following major components:
•AGX Xavier 32 GB Module
•RTX 6000 discrete GPU
•ConnectX-6 IC
•250GB Removeable SSD
Tech Specs
CPU
8-core Carmel ARM v8.2 64-bit CPU, 8MB L2 + 4MB L3
Memory
32GB 256-Bit LPDDR4x | 136.5GB/s
GPU
RTX6000 | 24 GB GDDR6 | 672 GB/s | 4,608 CUDA cores | 576 Tensor Cores
Storage
250GB SSD
I/O
USB2.0 | (2x) USB3.0 | USB-C | HDMI In/Out | (4x) DisplayPort | 1/10/100 GbE

Clara AGX –Getting Started
NVIDIA Confidential 4
Expansion
PCIe x8 (single slot width) to Xavier | PCIe x8 (single slot width) to RTX6000
Power Supply
750W | 100-240V
Dimensions
227mm W x 149mm H x 332mm L
I/O and external interfaces
1) Power cable connection
2) Power switch
3) PCIe slots for customer cards
4) dGPU outputs from RTX 6000, including four Display Ports and one mini HDMI port
a. Reference the GPU section below to determine which display output to use.

Clara AGX –Getting Started
NVIDIA Confidential 5
5) 100 GbE QSFP28 Ethernet connector to NVIDIA ConnectX-6
6) 10 GbE RJ45 Ethernet connector to NVIDIA ConnectX-6
a. The 10GbE connector only supports 10 GbE speeds.
7) 1 GbE RJ45 Ethernet connector to Xavier module
8) USB 3.0 ports (2x)
9) USB 2.0 port (1x)
10) HDMI out
a. Reference the GPU section below to determine which display output to use.
11) HDMI in: Connect to instruments that output HDMI to the platform.
12) Debug USB-C port: Connect to the Linux host system for flashing and serial port
connections.
13) x8 PCIe slot (x16 physical) for customer card, to Xavier Module
14) x8 PCIe slot (x16 physical) for customer card, to RTX6000

Clara AGX –Getting Started
NVIDIA Confidential 6
a. Used for cards supporting GPU Direct RDMA data transfer to RTX6000 GPU
15) SD card slot
16) Recovery button
17) Reset button
18) Main secondary compartment fan connector (there is an additional system fan inside the
chassis)
19) Auxiliary fan connector for card fans
20) Power button
To access ports 13-19, remove the left-hand side cover. The process is illustrated below.
Unscrew the two Phillips screws (21 and 22) that secure the cover at the back of the machine.
Next, push and slide the cover in the direction towards the back of the machine without lifting
(step 1). It should slide about 0.5 inch, or less than 1.5 cm. Finally, you should be able to lift
the cover off once it has more than one degree of freedom and can be easily lifted upwards
(step 2).
Powering up the System
1) Connect all peripherals to the system before powering up the system.
2) Connect the power cable to the system in the slot labeled (1) in the graphic above.
3) Once the power is connected, press the power button (20) for less than 1 second. It should
light up.
4) If you have a display connected, you might already see the system booting on it. During
flashing or re-flashing, use the HDMI output port (10) to connect to the display. Reference
the GPU section below to determine how to choose between display outputs in different
modes.

Clara AGX –Getting Started
NVIDIA Confidential 7
Note: The machine can be powered off by depressing the power button for approximately 10
seconds.
Check Firmware Version and Manually Update if Needed
If you are a current user of the Clara AGX Developer Kit and have flashed your devkit before,
check and if needed, update the firmware on the devkit before moving on to the next step of
flashing Jetpack 5.0 (compatible with Clara Holoscan SDK v0.2). Without the firmware update,
you might experience Known Issue #1 once Jetpack 5.0 is installed.
If you are a new user of the Clara AGX Developer Kit and have not flashed your devkit before,
follow the rest of this document to set up your devkit, then come back to this section to check
and update your firmware after switching to dGPU mode.
1. Install the tools for checking and updating firmware version
$ sudo apt install mstflint
2. Check for device address and look for the address for “Ethernet controller: Mellanox
Technologies MT28908 Family [ConnectX-6]”. The old firmware uses 0000:09:00.0 address, the
up-to-date firmware should use 0000:03:00.0.
$ lspci | grep Mellanox
3. Run the command with the device address obtained in step 2 to check for the firmware
version. If you see a version lower than 20.33.1048 then firmware update is needed. Otherwise,
there’s no need to update your firmware if the current version is 20.33.1048, you can skip the
rest of steps 4 - 7.
$ sudo mstflint -d 0000:09:00.0 q full
…
FW Version: 20.27.4006 <- Update is needed
…
$ sudo mstflint -d 0000:03:00.0 q full
…
FW Version: 20.33.1048 <- Firmware is already updated
…
4. Download the new firmware from here and unzip.
5. Install new firmware.
$ sudo mstflint -d 0000:09:00.0 -i fw-ConnectX6-rel-20_33_1048-
Nvidia_McCoy_LOM_PCIe_switch_Ax.bin burn
6. Reboot the devkit.
7. Check the newly installed firmware has correct version.
$ sudo mstflint -d 0000:03:00.0 q full
…
FW Version: 20.33.1048
…

Clara AGX –Getting Started
NVIDIA Confidential 8
Flashing and Updating Clara AGX Developer Kit using SDK Manager
1. Register and activate an NVIDIA Developer Account here to access the latest version of
Jetpack in SDK Manager.
2. If you are running a VPN on your host system, log off before flashing the Clara AGX Developer
Kit.
3. Using a VM as your host machine isn’t officially supported, but it is possible with certain
VMs such as VMWare Workstation 16 running Ubuntu 18.04. If using a VM, ensure the USB
port that connects to the USB-C port on the Clara AGX Developer Kit is always routed to the
VM.
4. From the host system, download and install the latest version of NVIDIA SDK Manager.
Instructions for downloading and setting up NVIDIA SDK Manager can be found here.
5. Connect the Clara AGX Developer Kit to the host system via USB-C (12).
6. From the NVIDIA SDK Manager, download and flash the Clara AGX Developer Kit. See the
step-by-step instructions for more details.
a. If you joined the Clara Holoscan SDK program after your initial SDK Manager login,
you will need to log out and log in again on SDK Manager for the permissions to take
effect.
b. NVIDIA recommends putting the unit to reset mode for the flashing process in Manual
Setup mode, as it has been observed that flashing can be stuck in Automatic Setup
mode.
c. Follow these steps to reset the Clara AGX Developer Kit:
i. Remove the left-hand side cover to expose ports 13-19
ii. While the unit is powered on, press the Recovery button (16) + Reset button
(17), then release both buttons.
d. If resetting the unit doesn’t work, try putting the unit into recovery mode:
i. Power off the unit.
ii. Remove the left-hand side cover to expose ports 13-19
iii. Hold down the Recovery button (16) while pressing the Power button (20) to
turn on the unit.
iv. Release the Power Button, then the Recovery button.
7. NVIDIA recommends putting the devkit into reset mode during flashing, but if for any reason
you need to know the default credentials when SDK Manager is preparing to flash the Clara
AGX Developer Kit, it is shown below
a. Username: ubuntu
b. Password: ubuntu
Switching between iGPU and dGPU
The Clara AGX Developer Kit can use either the Xavier AGX module GPU (iGPU, integrated GPU)
or the RTX6000 add-in card GPU (dGPU, discrete GPU). You can only use one type of GPU at a
time.
By default, the Clara AGX Developer Kit uses the iGPU. Switching between the iGPU and dGPU
is performed using the nvgpuswitch.py script, located at the /opt/nvidia/l4t-gputools/bin/
directory. To make the nvgpuswitch.py script accessible globally, copy it to a directory included
in $PATH if it hasn’t been already:

Clara AGX –Getting Started
NVIDIA Confidential 9
$ sudo cp /opt/nvidia/l4t-gputools/bin/nvgpuswitch.py /usr/local/bin/
To switch from the iGPU to the dGPU, follow these steps:
1. Connect the Clara AGX Developer Kit to the Internet using one of the following methods:
a. An Ethernet cable connected to a router or Wi-Fi extender
i. Use the 1GbE connector to the Xavier module at port 7
b. A USB Wi-Fi receiver
i. Not all USB Wi-Fi receivers will work out of the box on the Clara AGX
Developer Kit.
ii. The USB Wi-Fi receiver should have support for Ubuntu 20.04
iii. The TP-Link Archer T2U Nano USB Wi-Fi Adapter which has previously
been working with the Ubuntu 18.04 based Holoscan SDK versions will no
longer work with the Ubuntu 20.04 based Holoscan SDK v0.2.
2. To view the currently installed drivers and their version, use the query command:
$ nvgpuswitch.py query
iGPU (nvidia-l4t-cuda, 34.1.2-20220524101639)
3. To install the dGPU drivers, use the install command with the dGPU parameter (note that
sudo must be used to install drivers):
$ sudo nvgpuswitch.py install dGPU
The install command prints out the list of commands that will be executed as part of
the driver install, and then continues to execute those commands. This aids with
debugging if any of the commands fail to execute. The following arguments may also be
provided with the install command:
-h, --help show this help message and exit
-f, --force force reinstallation of the specified driver stack
-d, --dry do a dry run, showing the commands that would be
executed but not actually executing them
-i, --interactive run commands interactively (asks before running each
command)
-v, --verbose enable verbose output (used with --dry to describe
the commands that would be run)
-l LOG, --log LOG writes a log of the install to the specified file
-r [L4T_REPO], --l4t-repo [L4T_REPO]
specify the L4T apt repo (i.e. when using an apt
mirror; default is repo.download.nvidia.com/jetson)
4. The dGPU driver install may be verified using the query command:
$ nvgpuswitch.py query
dGPU (cuda-drivers, 510.73.08-1)
Quadro RTX 6000, 24576 MiB

Clara AGX –Getting Started
NVIDIA Confidential 10
5. After the dGPU drivers have been installed, rebooting the system will complete the switch
to the dGPU. At this point the Ubuntu desktop will be output via DisplayPort on the dGPU,
and so the display cable must be switched from the onboard HDMI (port 10) to DisplayPort
(port 4) on the dGPU.
6. Modify PATH and LD_LIBRARY_PATH . CUDA installs its runtime binaries such as nvcc into its
own versioned path that is not included by the default $PATH environment variable. Because
of this, attempts to run commands like nvcc will fail on dGPU unless the CUDA 11.6 path is
added to the $PATH variable. To add the CUDA 11.6 path for the current user, add the
following lines to $HOME/.bashrc after the switch to dGPU:
export PATH=/usr/local/cuda-11.6/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64:$LD_LIBRARY_PATH
If at any time you want to switch back to iGPU, use the install command with the iGPU
parameter:
$ sudo nvgpuswitch.py install iGPU
After the iGPU drivers have been installed, rebooting the system will complete the switch back
to iGPU. At this point the Ubuntu desktop will be output via the onboard HDMI, so the display
cable must be switched from the DisplayPort on the dGPU (4) to the onboard HDMI (10). If the
output connection isn’t switched, the terminal screen will hang during booting.
Note: The GPU settings will persist through reboots until it is changed again with
nvgpuswitch.py.
Setting up SSD Storage
Without setting up SSDK storage and moving docker storage to SSD, you might fill up the root
directory with Docker image pull operations, since a complete installation of the Jetpack leaves
only about 10GB of storage remaining in the root 32GB.
Note:
If the Clara AGX Developer Kit is reflashed with a new JetPack image, the partition table of
the m2 drive will not be modified and the contents of the partition will be retained. In this
case the Create Partition steps can be skipped, however the Mount Partition steps should
be followed again in order to remount the partition.
Any state, binaries, or docker images that persist on the m2 drive after flashing the system
may be made incompatible with new libraries or components that are flashed onto the
system. It may be required to recompile or rebuild these persistent objects to restore
runtime compatibility with the system.