Nvidia DGX A100 User manual

DU-09821-001_v06 | May 2022
DGX A100 System
User Guide

DGX A100 System DU-09821-001_v06 | ii
Table of Contents
Chapter 1. Introduction............................................................................................ 1
1.1 Hardware Overview ............................................................................................................2
1.1.1 DGX A100 Models and Component Descriptions ....................................................... 2
1.1.2 Mechanical Specifications ...........................................................................................3
1.1.3 Power Specifications.................................................................................................... 4
1.1.3.1 Support for N+N Redundancy .............................................................................. 4
1.1.3.2 DGX A100 Locking Power Cord Specification......................................................4
1.1.3.3 Using the Locking Power Cords...........................................................................5
1.1.3.3.1 Locking/Unlocking the PDU Side...................................................................5
1.1.3.3.2 Locking/Unlocking the PSU Side (Cords with Switch-Lock Mechanism).... 5
1.1.3.3.3 Locking/Unlocking the PSU Side (Cords with Twist-Lock Mechanism) ...... 6
1.1.4 Environmental Specifications......................................................................................6
1.1.5 Front Panel Connections and Controls ...................................................................... 7
1.1.5.1 With a Bezel...........................................................................................................7
1.1.5.2 With the Bezel Removed.......................................................................................8
1.1.6 Rear Panel Modules.....................................................................................................8
1.1.7 Motherboard Connections and Controls .................................................................... 9
1.1.8 Motherboard Tray Components ..................................................................................9
1.1.9 GPU Tray Components .............................................................................................. 10
1.2 Network Connections, Cables, and Adaptors................................................................. 10
1.2.1 Network Ports ............................................................................................................10
1.2.2 Supported Network Cables and Adaptors................................................................ 12
1.3 DGX A100 System Topology ............................................................................................. 12
1.4 DGX OS Software.............................................................................................................. 12
1.5 Additional Documentation ............................................................................................... 13
1.6 Customer Support............................................................................................................ 13
Chapter 2. Connecting to the DGX A100................................................................ 14
2.1 Connecting to the Console............................................................................................... 14
2.1.1 Direct Connection ...................................................................................................... 14
2.1.2 Remote Connection through the BMC...................................................................... 16
2.2 SSH Connection to the OS ............................................................................................... 18
Chapter 3. First Boot Setup................................................................................... 19
3.1 System Setup....................................................................................................................19
3.2 Post Setup Tasks.............................................................................................................. 21
3.2.1 Obtain Software Updates........................................................................................... 22
3.2.2 Enabling the srp_daemon ......................................................................................... 22

DGX A100 System DU-09821-001_v06 | iii
Chapter 4. Quick Start and Basic Operation ......................................................... 23
4.1 Installation and Configuration ......................................................................................... 23
4.2 Registration ...................................................................................................................... 23
4.3 Obtaining an NGC Account .............................................................................................. 24
4.4 Turning DGX A100 On and Off.......................................................................................... 24
4.4.1 Startup Considerations.............................................................................................. 24
4.4.2 Shutdown Considerations.......................................................................................... 24
4.5 Verifying Functionality – Quick Health Check................................................................. 24
4.6 Running a Preflight Stress Test ...................................................................................... 25
4.7 Running the NGC Containers with GPU Support ........................................................... 26
4.7.1 Using Native GPU Support ........................................................................................ 26
4.7.2 Using the NVIDIA Container Runtime for Docker .................................................... 27
4.8 Managing CPU Mitigations .............................................................................................. 28
4.8.1 Determining the CPU Mitigation State of the DGX System...................................... 28
4.8.2 Disabling CPU Mitigations......................................................................................... 29
4.8.3 Re-enabling CPU Mitigations.................................................................................... 29
Chapter 5. Additional Features and Instructions.................................................. 30
5.1 Managing the DGX Crash Dump Feature........................................................................ 30
5.1.1 Using the Script.......................................................................................................... 30
5.1.2 Connecting to Serial Over LAN to View the Console................................................ 31
Chapter 6. Managing the DGX A100 Self-Encrypting Drives................................. 32
6.1 Overview............................................................................................................................32
6.2 Installing the Software..................................................................................................... 33
6.3 Configuring Trusted Computing...................................................................................... 33
6.3.1 How to Tell if Drives Support Block SID ................................................................... 34
6.3.2 Enabling the TPM and Preventing the BIOS from Sending Block SID Requests ... 34
6.4 Initializing the System for Drive Encryption.................................................................... 35
6.5 Enabling Drive Locking .................................................................................................... 36
6.6 Initialization Examples..................................................................................................... 36
6.6.1 Example 1: Passing in the JSON File........................................................................ 36
6.6.1.1 Determining Which Drives Can be Managed as Self Encrypting ..................... 36
6.6.1.2 Creating the Drive/Password Mapping JSON Files and Using it to Initialize the
System .................................................................................................................37
6.6.2 Example 2: Generating Random Passwords............................................................ 38
6.6.3 Example 3: Specifying Passwords One at a Time When Prompted ........................ 38
6.7 Disabling Drive Locking ................................................................................................... 38
6.8 Exporting the Vault........................................................................................................... 39
6.9 Erasing your Data............................................................................................................. 39
6.10 Clearing the TPM.............................................................................................................. 39

DGX A100 System DU-09821-001_v06 | iv
6.11 Changing Disk Passwords, Adding Disks, or Replacing Disks...................................... 40
6.12 Recovering From Lost Keys............................................................................................. 40
Chapter 7. Network Configuration ........................................................................ 41
7.1 Configuring Network Proxies .......................................................................................... 41
7.1.1 For the OS and Most Applications............................................................................. 41
7.1.2 For apt ........................................................................................................................41
7.1.3 For Docker.................................................................................................................. 42
7.2 Configuring Docker IP Addresses ................................................................................... 42
7.3 Open Ports........................................................................................................................ 43
7.4 Connectivity Requirements for NGC Containers............................................................ 43
7.5 Configuring a Static IP Address for the BMC ................................................................. 44
7.5.1 Configuring a BMC Static IP Address Using ipmitool.............................................. 44
7.5.2 Configuring a BMC Static IP Address Using the System BIOS................................ 45
7.6 Configuring a BMC Static IP Address for the Network Ports........................................ 45
7.7 Switching Between InfiniBand and Ethernet.................................................................. 46
7.7.1 Starting the Mellanox Software Tools and Determining the Current Port
Configuration.............................................................................................................. 47
7.7.2 Switching the Port Configuration.............................................................................. 47
Chapter 8. Configuring Storage............................................................................. 49
8.1 Setting Filesystem Quotas............................................................................................... 50
8.2 Switching Between RAID 0 and RAID 5........................................................................... 50
8.3 Configuring Support for Custom Drive Partitioning....................................................... 51
Chapter 9. Updating and Restoring the Software ................................................. 52
9.1 Updating the DGX A100 Software .................................................................................... 52
9.1.1 Connectivity Requirements for Software Updates................................................... 52
9.1.2 Update Instructions ................................................................................................... 53
9.2 Restoring the DGX A100 Software Image ....................................................................... 53
9.2.1 Obtaining the DGX A100 Software ISO Image and Checksum File.......................... 54
9.2.2 Remotely Reimaging the System .............................................................................. 54
9.2.3 Creating a Bootable Installation Medium................................................................. 55
9.2.3.1 Creating a Bootable USB Flash Drive by Using the dd Command................... 56
9.2.3.2 Creating a Bootable USB Flash Drive by Using Akeo Rufus ............................ 56
9.2.4 Re-Imaging the System from a USB Flash Drive..................................................... 58
9.2.5 Installation Options....................................................................................................58
9.2.5.1 Retaining the RAID Partition While Installing the OS ....................................... 58
9.2.5.2 Advanced Installation Options (Encrypted Root - DGX OS 5 or later) .............. 59
9.2.5.3 Boot Into Live Environment (DGX OS 5 or later)................................................ 60
9.2.5.4 Check Disc for Defects (DGX OS 5 or later) ....................................................... 60
Chapter 10. Using the BMC ..................................................................................... 61

DGX A100 System DU-09821-001_v06 | v
10.1.1 Connecting to the BMC.............................................................................................. 61
10.2 Overview of BMC Controls ............................................................................................... 62
10.3 Common BMC Tasks........................................................................................................ 63
10.3.1 Changing BMC Login Credentials............................................................................. 63
10.3.2 Using the Remote Console........................................................................................ 64
10.3.3 Setting Up Active Directory or LDAP/E-Directory.................................................... 65
10.3.4 Configuring Platform Event Filters........................................................................... 65
10.3.5 Uploading or Generating SSL Certificates................................................................ 66
10.3.5.1 Viewing the SSL Certificate ................................................................................ 66
10.3.5.2 Generating the SSL Certificate........................................................................... 67
10.3.5.3 Uploading the SSL Certificate ............................................................................ 68
10.3.5.4 Updating the SBIOS Certificate .......................................................................... 68
Chapter 11. SBIOS Settings..................................................................................... 72
11.1 Accessing the SBIOS Setup ............................................................................................. 72
11.2 Configuring Boot Order.................................................................................................... 73
Chapter 12. Multi-Instance GPU ............................................................................. 74
Chapter 13. Security ................................................................................................ 75
13.1 User Security Measures................................................................................................... 75
13.1.1 Securing the BMC Port.............................................................................................. 75
13.2 System Security Measures .............................................................................................. 75
13.2.1 Secure Flash of DGX A100 Firmware........................................................................ 75
13.2.1.1 Encryption............................................................................................................ 75
13.2.1.2 Signing................................................................................................................. 76
13.2.1.3 NVSM Security..................................................................................................... 76
13.3 Secure Data Deletion ....................................................................................................... 76
13.3.1 Prerequisite................................................................................................................76
13.3.2 Instructions ................................................................................................................ 76
Chapter 14. Redfish APIs Support........................................................................... 78
14.1 Supported Features .........................................................................................................78
Appendix A. Installing Software on Air-Gapped DGX 100 Systems......................... 80
Appendix B. Safety ................................................................................................... 90
Appendix C. Compliance .......................................................................................... 97

DGX A100 System DU-09821-001_v06 | 1
Chapter 1. Introduction
The NVIDIA DGX™ A100 system is the universal system purpose-built for all AI infrastructure
and workloads, from analytics to training to inference. The system is built on eight NVIDIA
A100 Tensor Core GPUs.
This document is for users and administrators of the DGX A100 system.

Introduction
DGX A100 System DU-09821-001_v06 | 2
1.1 Hardware Overview
1.1.1 DGX A100 Models and Component
Descriptions
There are two models of the NVIDIA DGX A100 system: the NVIDIA DGX A100 640GB system
and the NVIDIA DGX A100 320GB system.
Table 1-1. Model Differentiation
Component NVIDIA DGX A100 640GB NVIDIA DGX A100 320GB
GPU Qty 8 NVIDIA A100 GPUs
Third-generation NVLinks
Qty 8 NVIDIA A100 GPUs
Third-generation NVLinks
Total GPU Memory 640 GB 320 GB
NVIDIA NVSwitch Qty 6
Second generation (2x faster
than first generation)
Qty 6
Second generation (2x faster than
first generation)
Networking Qty 10 (Factory ship config)
Mellanox ConnectX-6 VPI
HDR InfiniBand/200 Gb/s
Ethernet
Qty 9 (Factory ship config)
Mellanox ConnectX-6 VPI HDR IB/200
Gb/s
(Optional Add-on: Second dual- port
200 Gb/s Ethernet)
CPU 2 AMD Rome, 128 cores total 2 AMD Rome, 128 cores total
System Memory 2 TB (Factory ship config) 1 TB (Factory ship config)
(Optional Add-on: 1 TB to get 2 TB
max.)
Storage 30 TB (Factory ship config)
U.2 NVMe Drives
(Optional drive upgrade to 60 TB)
15 TB (Factory ship config)
U.2 NVMe Drives
(Optional Add-on: 15 TB to get 30 TB
max.
Optional drive upgrade to 60 TB)

Introduction
DGX A100 System DU-09821-001_v06 | 3
Table 1-2. Component Description
Component Description
GPU NVIDIA A100 GPU
CPU 2x AMD EPYC 7742 CPU w/64 cores
NVSwitch 600 GB/s GPU-to-GPU bandwidth
Storage (OS) 1.92 TB NVMe M.2 SSD (ea) in RAID 1 array
Storage (Data Cache) 3.84 TB NVMe U.2 SED (ea) in RAID 0 array
(Optional 7.68 TB NVMe U.2. SEDs)
Network (Cluster) card •Mellanox ConnectX-6 Single Port VPI InfiniBand
(default): HDR, HDR100, EDR
•Ethernet: 200GbE, 100GbE, 50GbE, 40GbE, 25GbE,
and 10GbE
Network (Storage) card Mellanox ConnectX-6 Dual Port VPI
Ethernet (default): 200GbE, 100GbE, 50GbE, 40GbE,
25GbE, and 10GbE
InfiniBand: HDR, HDR100, EDR
System Memory (DIMM) 1 TB per 16 DIMMs
BMC (out-of-band system
management) 1 GbE RJ45 interface
Supports IPMI, SNMP, KVM, and Web UI, and the Redfish
APIs
In-band system management 1 GbE RJ45 interface
Power Supply 3 kW
1.1.2 Mechanical Specifications
Table 1-3. Mechanical Specifications
Feature Description
Form Factor 6U Rackmount
Height 10.4” (264 mm)
Width 19" (482.3 mm) max
Depth 35.3" (897.1 mm) max
System Weight 271.5 lbs (123.16 kg) max

Introduction
DGX A100 System DU-09821-001_v06 | 4
1.1.3 Power Specifications
The DGX A100 system contains six power supplies with balanced distribution of the power
load.
Table 1-4. Power Specifications
Input Specification for Each Power Supply
200-240 volts AC 6.5 kW max. 3000 W @ 200-240 V, 16 A, 50-60 Hz
1.1.3.1 Support for N+N Redundancy
The DGX A100 includes six power supply units (PSU) configured for 3+3 redundancy. If three
PSUs fail, the system will continue to operate at full power with the remaining three PSUs.
Note: If only two PSUs are working, the GPUs will not be available, but the server will still boot.
This is to allow you to gather debug or system logs.
If only one PSU is working, troubleshoot the cause for the loss of power from the other PSUs and
correct. If faulty PSUs need to be replaced, shut the system down and install working PSUs.
1.1.3.2 DGX A100 Locking Power Cord Specification
The DGX A100 is shipped with a set of six (6) locking power cords that have been qualified for
use with the DGX A100 to ensure regulatory compliance. Two locking power cord types are
approved - switch-locking for the PSU side and twist-locking for the PSU side.
WARNING: To avoid electric shock or fire, only use the NVIDIA-provided power cords to connect
power to the DGX A100. For more details, see Electrical Precautions in “Safety”.
Table 1-5. Power Cord Specifications
Power Cord Feature Specification
Electrical 250VAC, 16A
Plug Standard C19/C20
Dimension 1200mm length
Compliance Cord: UL62, IEC60227
Connector/Plug: IEC60320-1

Introduction
DGX A100 System DU-09821-001_v06 | 5
1.1.3.3 Using the Locking Power Cords
Follow these instructions for using the locking power cords.
1.1.3.3.1 Locking/Unlocking the PDU Side
Power Distribution Unit side
To INSERT, push the cable into the PDU socket.
To REMOVE, press the clips together and pull the cord out of the socket.
1.1.3.3.2 Locking/Unlocking the PSU Side (Cords with Switch-Lock
Mechanism)
Power Supply (System) side - Switch locking
To INSERT or REMOVE make sure the cable is UNLOCKED and push/ pull into/out of the
socket
To UNLOCK the power cord, move
the switch to the unlocked position
(indicator will show GREEN)
To LOCK the power cord, move the
switch to the locked position
(indicator should show only RED)

Introduction
DGX A100 System DU-09821-001_v06 | 6
1.1.3.3.3 Locking/Unlocking the PSU Side (Cords with Twist-Lock
Mechanism)
Power Supply (System) side - Twist locking
To INSERT or REMOVE make sure the cable is UNLOCKED and push/ pull into/out of the
socket
1.1.4 Environmental Specifications
Table 1-6. Environmental Specifications
Feature Specification
Operating Temperature 5 ¤ C to 30¤ C (41¤ F to 86¤ F)
Relative Humidity 20% to 80% non-condensing
Airflow 840 CFM @ 80% fan PWM
Heat Output 22,179 BTU/hr
To UNLOCK the power cord, twist the
gray locking ring to the unlocked
(indicator will show an unlocked
padlock)
To LOCK the power cord, twist the
gray locking ring to the locked
position (indicator should show a
locked padlock)

Introduction
DGX A100 System DU-09821-001_v06 | 7
1.1.5 Front Panel Connections and Controls
1.1.5.1 With a Bezel
Table 1-7. Front Panel Controls
Control Description
Power Button
Press to turn the DGX A100 system On or Off
Green flashing (1 Hz): Standby (BMC booted) Green
flashing (4 Hz): POST in progress Green solid On:
Power On
ID Button
Press to cause the button blue LED to turn On
or blink (configurable through the BMC) as an
identifier during servicing.
Also causes an LED on the back of the unit to flash as
an identifier during servicing.
Fault LED Amber On: System or component faulted

Introduction
DGX A100 System DU-09821-001_v06 | 8
1.1.5.2 With the Bezel Removed
!
Important: See “Turning DGX A100 On and Off” for instructions on how to properly turn the
system on or off.
1.1.6 Rear Panel Modules

Introduction
DGX A100 System DU-09821-001_v06 | 9
1.1.7 Motherboard Connections and Controls
Table 1-8. Motherboard Controls
Control Description
Power Button Press to turn the system On or Off.
ID LED Button Blinks when ID button is pressed from the front of the unit
as an aid in identifying the unit needing servicing
BMC Reset button Press to manually reset the BMC
See “Network Connections, Cables, and Adaptors”for details on the network connections.
1.1.8 Motherboard Tray Components

Introduction
DGX A100 System DU-09821-001_v06 | 10
1.1.9 GPU Tray Components
1.2 Network Connections, Cables, and
Adaptors
1.2.1 Network Ports

Introduction
DGX A100 System DU-09821-001_v06 | 11
Table 1-9. Network Port Mapping
Slot
PCI Bus
Port Destinations RDMA
Defaults
Optional
Optional
Slot 5 not
populate
Slot 5
PopulatedPre-DGX OS
DGX OS 5
and later
0 4b:00.0 ib2 ibp75s0 enp75s0 mlx5_2 mlx5_2
1 54:00.0 ib3 ibp84s0 enp84s0 mlx5_3 mlx5_3
2 ba:00.0 ib6 ibp186s0 enp186s0 mlx5_6 mlx5_8
3 cc:00.0a
ca:00.0bb
ib2 ibp75s0 enp204s0a
enp202s0b
mlx5_7 mlx5_9
4 port 0 (top) e1:00.0 enp225s0f0 (See note) mlx5_8 mlx5_10
4 port 1
(bottom)
e1:00.1 enp225s0f1 (See note) mlx5_9 mlx5_11
5 port 0 (left) 61:00.0 enp97s0f0 (See note) - mlx5_4
5 port 1
(right)
61:00.1 enp97s0f1 (See note) - mlx5_5
6 0c:00.0 ib0 enp12s0 mlx5_0 mlx5_0
7 12:00.0 ib1 enp18s0 mlx5_1 mlx5_1
8
8d:00.1
ib4
enp141s0
mlx5_6
mlx5_6
9 94:00.0 ib5 enp148s0 mlx5_7 mlx5_7
LAN e2:00.0 enp226s0 N/A
Note: The interface enp37s0f3u1u3c2 or bmc_redfish0 is recognized by the OS and may be listed
in response to such commands as ifconfig or ip addr. This interface supports BMC
communication using Redfish APIs.
Note: The Optional column lists the port designations after reconfiguring the default InfiniBand
ports to Ethernet.
When switching from the default Ethernet to InfiniBand, the InfiniBand port designations will
vary depending on changes made to the other ports.
aBased on systems updated with DGX A100 Firmware Update Container 20.10.9 or later
bBased on systems updated with DGX A100 Firmware Update Container 20.05.12.3 or earlier.

Introduction
DGX A100 System DU-09821-001_v06 | 12
1.2.2 Supported Network Cables and Adaptors
The DGX A100 system is not shipped with network cables or adaptors. You will need to
purchase supported cables or adaptors for your network.
The ConnectX-6 firmware determines which cables and adaptors are supported. For a list of
cables and adaptors compatible with the Mellanox ConnectX-6 VPI cards installed in the DGX
A100 system.
1. Visit the Mellanox Firmware Release page.
2. From the left navigation menu, select the ConnectX model and corresponding firmware
included in the DGX A100.
3. Select Firmware Compatible Products.
1.3 DGX A100 System Topology
1.4 DGX OS Software
The DGX A100 system comes pre-installed with a DGX software stack incorporating the
following:
An Ubuntu server distribution with supporting packages
The following system management and monitoring software
•NVIDIA System Management (NVSM)
•Provides active health monitoring and system alerts for NVIDIA DGX nodes in a data
center. It also provides simple commands for checking the health of the

Introduction
DGX A100 System DU-09821-001_v06 | 13
•DGX A100 system from the command line.
•Data Center GPU Management (DCGM)
DGX A100 system support packages
The NVIDIA GPU driver
Docker Engine
NVIDIA Container Toolkit
Mellanox OpenFabrics Enterprise Distribution for Linux (MOFED)
Mellanox Software Tools (MST)
cachefilesd (daemon for managing cache data storage)
1.5 Additional Documentation
MIG User Guide
The new Multi-Instance GPU (MIG) feature allows the NVIDIA A100 GPU to be securely
partitioned into up to seven separate GPU Instances for CUDA applications.
NGC Container Registry for DGX
How to access the NGC container registry for using containerized deep learning GPU-
accelerated applications on your DGX A100 system.
NVSM Software User Guide
Contains instructions for using the NVIDIA System Management software.
DCGM Software User Guide
Contains instructions for using the Data Center GPU Manager software.
1.6 Customer Support
Contact NVIDIA Enterprise Support for assistance in reporting, troubleshooting, or diagnosing
problems with your DGX A100 system. Also contact NVIDIA Enterprise Support for assistance in
moving the DGX A100 system.
For contracted Enterprise Support questions, you can send an email to
[e-mail].
For additional details about how to obtain support, go to NVIDIA Enterprise Support.
Our support team can help collect appropriate information about your issue and involve
internal resources as needed.

DGX A100 System DU-09821-001_v06 | 14
Chapter 2. Connecting to the DGX A100
2.1 Connecting to the Console
Connect to the DGX A100 console using either a direct connection or a remote connection
through the BMC.
!
CAUTION: Connect directly to the DGX A100 console if the DGX A100 system is connected to a
172.17.xx.xx subnet.
DGX OS Server software installs Docker Engine which uses the 172.17.xx.xx sub-net by default for
Docker containers. If the DGX A100 system is on the same subnet, you will not be able to
establish a network connection to the DGX A100 system.
Refer to “Configuring Docker IP Addresses” for instructions about how to change the default
Docker network settings.
2.1.1 Direct Connection
At either the front or the back of the DGX A100 system, connect a display to the VGA connector,
and a keyboard to any of the USB ports.
Note: The display resolution must be 1440x900 or lower.

Connecting to the DGX A100
DGX A100 System DU-09821-001_v06 | 15
Figure 2-1. DGX A100 Server Front View
Figure 2-2. DGX A100 Server Rear View