HPE Apollo pc40 User manual

Part Number: 007-6488-001
Published: September 2017
Edition: Revision 1
Abstract:
This guide provides an overview of the HPE Apollo pc40 server.
HPE Apollo pc40 Server
User Guide

© 2017, Hewlett Packard Enterprise Development LP
Notices
The information contained herein is subject to change without notice. The only warranties for Hewlett Packard
Enterprise products and services are set forth in the express warranty statements accompanying such
products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett
Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use,
or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software
Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under
vendor's standard commercial license.
Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard
Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise
website.
Acknowledgments
Intel®, Itanium®, Pentium®, Intel Inside®, and the Intel Inside logo are trademarks of Intel Corporation in the
United States and other countries.
Microsoft® and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the
United States and/or other countries.
Adobe® and Acrobat® are trademarks of Adobe Systems Incorporated.
Java® and Oracle® are registered trademarks of Oracle and/or its affiliates.
UNIX® is a registered trademark of The Open Group.

3
Preface
About this Manual
This manual provides a brief summary of the HPE Apollo pc40 server. The HPE Apollo pc40
server is a 1U dual socket server featuring up to four NVIDIA® Tesla® GPUs and based on
the Intel® Xeon® Processor Scalable Family. The HPE Apollo pc40 Server is an optimized,
industry-standard server for deep learning and high performance computing (HPC) workloads
that uses the PCIe form factor for greater choice in GPU selection. Please refer to the HPE
Apollo pc40 server data sheet and QuickSpecs (www.hpe.com/servers/apollo-pc40) for a
system overview, standard features, warranty information, service and support information,
configuration information, and technical specifications. Installation and maintenance should be
performed by experienced technicians only.
Warnings
Special attention should be given to the following symbols used in this manual.
Warning! Indicates high voltage may be encountered when performing a procedure.
Warning! Indicates important information given to prevent equipment/property damage
or personal injury.

4
Contents
Chapter 1 Introduction
1.1 Overview ...............................................................................................................................6
1.2 Unpacking the System .........................................................................................................6
1.3 System Features ..................................................................................................................9
1.4 Server Chassis Features ......................................................................................................9
Control Panel ......................................................................................................................9
1.5 Motherboard Layout ...........................................................................................................12
Quick Reference Table ......................................................................................................13
Chapter 2 Server Installation
2.1 Overview .............................................................................................................................15
2.2 Preparing for Setup ............................................................................................................15
Choosing a Setup Location ...............................................................................................15
Rack Precautions ..............................................................................................................15
Server Precautions ............................................................................................................16
Rack Mounting Considerations .........................................................................................16
Ambient Operating Temperature ....................................................................................16
Airflow.............................................................................................................................16
Mechanical Loading .......................................................................................................16
Circuit Overloading ........................................................................................................17
Reliable Ground .............................................................................................................17
2.3 Installing the Rails ..............................................................................................................18
Identifying the Rails ...........................................................................................................18
Releasing the Inner Rail....................................................................................................19
Installing The Inner Rails on the Chassis ........................................................................20
Installing the Outer Rails on the Rack ..............................................................................21
Installing the System into the Rack ..................................................................................22
Removing the System From the Rack ..............................................................................23
Removing the Outer Rails From the Rack ........................................................................24

5
Chapter 3 Maintenance and Component Installation
3.1 Removing Power ................................................................................................................25
3.2 Accessing the System ........................................................................................................25
3.3 Chassis Hard Drives ..........................................................................................................27
Externally Accessible Drives ..........................................................................................27
Internal Drives ................................................................................................................28
Power Supply ....................................................................................................................29
Replacing a Power Supply Module ...............................................................................29
Appendix ABIOS Error Codes and System Specifications

6
Chapter 1
Introduction
1.1 Overview
This chapter provides a brief outline of the functions and features of the HPE
Apollo pc40 server. The HPE Apollo pc40 server is a 1U dual socket server
featuring up to four NVIDIA® Tesla® GPUs and based on the Intel® Xeon®
Processor Scalable Family. The HPE Apollo pc40 Server is an optimized,
industry-standard server for deep learning and high performance computing
(HPC) workloads that uses the PCIe form factor for greater choice in GPU
selection.
In addition to the motherboard and chassis, several important parts that are included
with the system. See QuickSpecs at: www.hpe.com/servers/apollo-pc40 for additional
details.
1.2 Unpacking the System
Inspect the box the HPE Apollo pc40 server was shipped in and note if it was damaged in
any way. If any equipment appears damaged, please file a damage claim with the carrier
who delivered it.
Decide on a suitable location for the rack unit that will hold the server. It should be situated
in a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical noise and
electromagnetic fields are generated. It will also require a grounded AC power outlet nearby.
Be sure to read the precautions and considerations (refer to information at the end of this
document).

7
The illustrations below provide an overview of the front and rear features of the HPE Apollo
pc40 server. The following page shows a top view of the system.
Front View
Item Description
1. HPE Apollo pc40 Chassis (1U)
2. 2 SFF Hot-swap SATA Drive Bays
3. Internal Fans
4. Unit Identification (UID) LED/button
5. Power Button
6. Status LEDs
Rear View
Item Description
1. Two 2000W Titanium level power supplies
2. Embedded 1 Gb NIC 1
3. Embedded 1 Gb NIC 2
4. Full-height PCIe Gen3 x16 slot
5. Full-height PCIe Gen3 x16 slot
6. Dedicated IPMI LAN port
7. Two USB 3.0 Connectors

8
Top View
Item Description
1. Full-height/Full-length PCIe 3 x16 slots for GPUs
2. DIMMs for processor 1
3. Processor 1
4. Full-height/Full-length PCIe Gen3 x16 slot
5. Processor 2
6. DIMMs for processor 2

9
Control Panel Features
Item Feature Description
1Information LED See table on the following page.
2NIC2 LED Indicates network activity on the LAN2 port when flashin
3NIC1 LED Indicates network activity on the LAN1 port when flashin
4HDD LED Indicates activity on the hard drive when flashin
5Power LED Indicates power is being supplied to the system power supply units. This LED
should normally be illuminated when the system is operating.
6UID LED
The unit identification (UID) button turns on or off the blue light function of the
Information LED and the blue LED on the rear of the chassis. These are used to
locate the server in large racks and server banks.
7Power Button
The main power button is used to apply or remove power from the power supply
to the server. Turning off system power with this button removes the main power
but maintains standby power. To perform many maintenance tasks, you must
also unplug system before servicing
Control Panel View
1.4 Server Chassis Features
Control Panel
The switches and LEDs located on the control panel are described below.
1
76
5
432
1.3 System Features
For a complete list of system features, please refer to the HPE Apollo pc40 QuickSpecs
at: www.hpe.com/servers/apollo-pc40

10
Information LED
Status Description
Continuously on and red An overheat condition has occurred.
(This may be caused by cable congestion.)
Blinking red (1Hz) Fan failure, check for an inoperative fan.
Solid blue Local UID has been activated. Use this function to
locate the server in a rackmount environment.
Blinking blue Remote UID is on. Use this function to identify the
server from a remote location.
Note: The HPE Apollo pc40 server is only considered to be N+1 in the 200-240V range;
the 100-127V range requires both power supplies to be operating.

11
SXM2 Add-on Module Block Diagram
SXM
SXM
SXM
SXM
PLX
PLX
CPU1
CPU2
NVLINK – Addional VOLTA Links
NVLINK – PASCAL
UPI
PCIE 3.0
AOM-SXM-BRG
AOM-SXM-BRG
x16
x16
x16
x16
x16
x16
x16
x16
x16
x16
AOM-SXM2 MBD-X11DGQ
RSC-GN2-66
RSC-G-A66-X1
Note: Volta NVLINKs will become available at a later date.

12
Motherboard Layout
1.5Motherboard Layout
Below is a layout of the HPE Apollo pc40 server motherboard with jumper, connector and
LED locations shown. See the table on the following page for descriptions.
X11DGQ
REV:1.00
DESIGNED IN USA
IPMI CODE BAR CODE
BIOS LICENSE
JTEMP1
JF1
M.2
JPWR4
FAN9 FAN8
JUSB3
JTPM1
JSDCARD1
FAN_PWR1
HDD_PWR1
HDD_PWR2
LE6
FAN_CTRL
JPSU1JPSU2
JRK1
JP2
JL1
JSTBY1
BT1
JVRM1 JVRM2
JWD1
JPME2
LEDM1
LE2
JBT1
I-SGPIO1
JPCIE2 JPCIE1
JSLOT6
JPCIE3
JPCIE4
JPCIE5
P2-DIMMF1
P2-DIMMD1 P2-DIMME1
P2-DIMMA1 P2-DIMMC1 P2-DIMMB1
P1-DIMMF1 P1-DIMMD1
P1-DIMME1
JPW3 JPW2
CPU1/CPU2 SLOT5 PCIE 3.0 X16/X16
CPU1 SLOT4 PCIE 3.0 X16
CPU1 SLOT3 PCIE 3.0 X16
CPU2 SLOT2 PCIE 3.0 X16
P1-DIMMC1
P1-DIMMA1
P1-DIMMB1
JPW1
USB2/3 (3.0)
I-SATA3I-SATA2I-SATA0 I-SATA1
CPU2 SLOT1 PCIE 3.0 X16

13
Quick Reference Table
Jumper Description
JBT1 Clear CMOS
JPME2 ME ManufactureMode
JVRM1/JVRM2 I2C Bus for VRM
JWD1 Watch Dog Timer Enable
Default Setting See
Pins 1-2 (Normal) Pins
1-2 (BMC: Normal) Pins
1-2 (Reset)
Connectors Description
Battery (BT1) Onboard CMOS Battery
FAN8/FAN9 CPU/System Cooling Fan Headers
FAN_CTRL Fan 1-7 control header
FAN_PWR1 Power connector for front fans
JF1 Front control panel header
JL1 Chassis intrusion header
JP2 CPLD programming header
HDD_PWR1/2 Power connectors for HDD devices
JPWR1~4 12V 8-pin GPU power supply connectors
JRK1 RAID key header
JPSU1~2 Power supply input
JSCARD1 Micro SD card slot
JSTBY1 Standby power header
JTPM1 TPM (Trusted Platform Module)/Port 80 Header
JTEMP1 Front control panel temperature header
JUSB3 USB 3.0 header
(I-)SATA0~3 Intel® SATA 3.0 Connectors from Intel PCH SATA controller
I-SGPIO1 Serial Link General Purpose I/O Header
LED Description State Status
LEDM1 BMC Heartbeat LED Green BMC Normal
LE2 Onboard Power LED Green Power On
LE6 Power Status LED Green/Red
Red Power On/Standby

14
Intel PCH C621 Chipset: System Block Diagram
Note: This is a general block diagram and may not exactly represent the features on your
motherboard. See the QuickSpecs at www.hpe.com/servers/apollo-pc40 for additional
information.
SPI
FRONT PANEL
SYSTEM POWER
CTRL
FAN SPEED
#7 USB2.0
PCH
6.0 Gb/S
USB 2.0
USB
#1
#0
SATA
#3
#2
Temp Sensor
W83773 at SMBUS
SPI
AST2500
BMC
BMC Boot Flash
DDR4
SLOT 3
5+1 PHASE
205W
DDRIV 2133/2666
P1
P1
P0
VR13
P0
#G-1
DDRIV 2133/2666
#B-1
#A-1
UPI
PCI-E X16
DMI3 DMI3
UPI 10.4G /11.2G
5+1 PHASE
205W
VR13
#H-1
#J-1
#K-1
#L-1
#M-1
VCCP1 12vVCCP0 12v
UPI
PECI:30 PECI:31
SOCKET ID:0 SOCKET ID:1
#C-1
#D-1
#E-1
#F-1
UPI
P2P2
SLOT 4
PCI-E X16
SLOT 1
PCI-E X16
SLOT 2
PCI-E X16
CPU0 PCIe X16
CPU1 PCIe X16
SLOT 5
PCI-E X16 G3 (RSC-G-6)
PCI-E X16 G3 (RSC-G-6)
PCI-E X16 G3 (RSC-G-6)
PCI-E X16 G3 (RSC-GR-6)
PCI-E X16 G3
PCI-E X16 G3 with re-driver
AOM-PIO-i2XT
RMII/NCSI
VGA
COM PORT
RGRMII
Rear IO
riser card
PCI-E X4 G2
#0-3
USB3.0 x2
#0/1 USB2.0
PCI-E X4 G2
#5
ESPI
Debug Card
TPM HEADER
SPI
BIOS
Bus S/W
#3 #2 #1 #1#2#3
DMI3
CPU0 PCIe X16
CPU1 PCIe X8
NVME X8
RSC-G-A66
RSC-GN2-A68
Two riser options
PCIE x4 or
SATA
PCIex4 or
SATA
M.2
Flexible I/O

15
Chapter 2
Server Installation
2.1 Overview
This chapter provides advice and instructions for mounting your system in a server rack.
Caution: Electrostatic Discharge (ESD) can damage electronic components. To prevent such
damage to PCBs (printed circuit boards), it is important to use a grounded wrist strap, handle
all PCBs by their edges and keep them in anti-static bags when not in use.
2.2 Preparing for Setup
The box in which the system was shipped should include the rackmount hardware needed to
install it into the rack. Please read this section in its entirety before you begin the installation.
Choosing a Setup Location
•The system should be situated in a clean, dust-free area that is well ventilated. Avoid areas
where heat, electrical noise and electromagnetic fields are generated.
•Leave enough clearance in front of the rack so that you can open the front door completely
(~25 inches) and approximately 30 inches of clearance in the back of the rack to allow
sufficient space for airflow and access when servicing.
•This product should be installed only in a Restricted Access Location (dedicated equipment
rooms, service closets, etc.).
•This product is not suitable for use with visual display workplace devices according to §2
of the German Ordinance for Work with Visual Display Units.
Rack Precautions
•Ensure that the leveling jacks on the bottom of the rack are extended to the floor so that
the full weight of the rack rests on them.

16
•In single rack installations, stabilizers should be attached to the rack. In multiple rack in-
stallations, the racks should be coupled together.
•Always make sure the rack is stable before extending a server or other component from
the rack.
•You should extend only one server or component at a time - extending two or more simul-
taneously may cause the rack to become unstable.
Server Precautions
•Review the electrical and general safety precautions - refer to reference links at the end
of this document
•Determine the placement of each component in the rack before you install the rails.
•Install the heaviest server components at the bottom of the rack first and then work your
way up.
•Use a regulating uninterruptible power supply (UPS) to protect the server from power
surges and voltage spikes and to keep your system operating in case of a power failure.
•Allow any drives and power supply modules to cool before touching them.
•When not servicing, always keep the front door of the rack and all covers/panels on the
servers closed to maintain proper cooling.
Rack Mounting Considerations
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating temperature of
the rack environment may be greater than the room's ambient temperature. Therefore,
consideration should be given to installing the equipment in an environment compatible with
the manufacturer’s maximum rated ambient temperature (TMRA).
Airow
Equipment should be mounted into a rack so that the amount of airflow required for safe
operation is not compromised.
Mechanical Loading
Equipment should be mounted into a rack so that a hazardous condition does not arise due
to uneven mechanical loading.

17
Circuit Overloading
Consideration should be given to the connection of the equipment to the power supply circuitry
and the effect that any possible overloading of circuits might have on overcurrent protection
and power supply wiring. Appropriate consideration of equipment nameplate ratings should
be used when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack itself should be
grounded. Particular attention should be given to power supply connections other than the
direct connections to the branch circuit (i.e. the use of power strips, etc.).
To prevent bodily injury when mounting or servicing this unit in a rack, you must take
special precautions to ensure that the system remains stable. The following guidelines
are provided to ensure your safety:
•This unit should be mounted at the bottom of the rack if it is the only unit in the rack.
•When mounting this unit in a partially filled rack, load the rack from the bottom to the top
with the heaviest component at the bottom of the rack.
•If the rack is provided with stabilizing devices, install the stabilizers before mounting or
servicing the unit in the rack.

18
Identifying the Rail Sections
2.3 Installing the Rails
This section provides information on installing the chassis into a rack unit with the rails
provided. There are a variety of rack units on the market, which may mean that the assembly
procedure will differ slightly from the instructions provided. You should also refer to the
installation instructions that came with the rack unit you are using.
Note: This rail will fit a rack between 27" and 32" deep.
Identifying the Rails
The chassis package includes two rail assemblies in the rack mounting kit. Each assembly
consists of three sections: an inner rail that secures directly to the chassis, an outer rail that
secures to the rack, and a middle rail that slides in the outer rail. These assemblies are
specifically designed for the left and right side of the chassis.
Middle Rails, Front
Outer Rails, Rear
Right Outer Rail
Left Outer Rail
Inner Rails

19
Note: Both front chassis rails and the rack rails have a locking tab, which serves two functions.
First, it locks the server into place when installed and pushed fully into the rack (its normal
operating position. In addition, these tabs lock the server in place when fully extended from
the rack. This prevents the server from coming completely out of the rack when pulled out
for servicing.
Extending the Rails and Releasing the Inner Rail
Warning: Stability hazard. The rack stabilizing mechanism must be in place, or the
rack must be bolted to the floor before you slide the unit out for servicing. Failure to
stabilize the rack can cause the rack to tip over.
Releasing the Inner Rail
To mount the rail onto the chassis, first release the inner rail from the outer and middle rails.
Releasing the Inner Rail from the Middle and Outer Rails
1. Lift the auto-lock latch and pull the inner rail until it is fully extended.
2. Press down the locking lever on the inside of the inner rail to release the inner rail.
Continue to pull the inner rail out of the middle rail.
Middle Rail
Locking
Lever on
Inner Rail
Outer Rail
Inner Rail
Front Latch
Front Latch on Inner Rail
Locking Lever on
Middle Rail
(on inside of rail)

20
Installing the Inner Rails
Installing The Inner Rails on the Chassis
Installing the Inner Rails
1. Identify the left and right inner rails.
2. Place the inner rail firmly against the side of the chassis, aligning the T-studs on the
side of the chassis with the slotted holes in the inner rail.
3. Slide the inner rail toward the rear of the chassis and under the T-studs.
4. An optional screw can be used to secure the rail to the chassis.
3
2
Warning: do not pick up the server with the front handles. They are designed to pull
the system from a rack only.
Slide rail mounted equipment is not to be used as a shelf or a work space.