Supermicro CSE-938NH-R2K20BP2 User manual

10
Chapter 1: Introduction
1.2 System Features
The CSE-938NH-R2K20BP2 is a 3U chassis that supports 16 front hot-swape drives and
eight rear hot-plug nodes.
Front View
Figure 1-1. System: Front View
System Features: Front
Feature Description
Control Panels Power buttons and status indicators; details on the next page
Storage Drives Sixteen 3.5" drive bays, two for each computing node; drive carriers display status lights
Drive Carrier LED Indicator
LED Color Blinking Pattern Behavior for Device
Activity LED
Blue Solid On SAS/NVMe drive installed
Blue Blinking I/O activity
Status LED
Red Solid On Failure of drive with RSTe support
Red Blinking at 1 Hz Rebuild drive with RSTe support
Red Blinking with two blinks and one
stop at 1 Hz
Hot spare for drive with RSTe support
(not supported in VMD mode)
Red On for ve seconds, then o󰀨 Power on for drive with RSTe support
Red Blinking at 4 Hz Identify drive with RSTe support
Green Solid On Safe to remove NVMe device (not
supported in VMD mode)
Amber Blinking at 1 Hz Attention state---do not remove NVMe
device (not supported in VMD mode)
1A 1B 2A 2B 3A 3B 4A 4B 5A 5B 6A 6B 7A 7B 8A 8B
Control Panel

11
Chapter 1: Introduction
Control Panel
The chassis front features a control panel to monitor node function and power o󰀨 and on the
entire system.
Figure 1-2. Control Panel
Control Panel Features
Item Feature Description
1Node Status Indicators Eight numbered LEDs that indicate the status of each node (see table
below).
2Power Fail LED Illuminated when one of the power supplies fails while any node is
powered on. It is o󰀨 during normal operation.
3Main Power button Used to apply or remove power from the power supply to the server
system. Whether on or o󰀨, standby power to the system remains on.
1
32
Node Activity LED Indicators
LED Appearance Description
Solid Green The node is powered on and operating normally.
Blinking Green The node is in the process of shutting down.
Solid Red The node is detecting an overheated condition.
1Hz Blinking Red The node is detecting a fan failure.
.25Hz Blinking Red The node is detecting a power failure.
Solid Blue The node local UID is on.
1Hz Blinking Blue The node remote UID is on.
No Illumination The node is powered down.

12
Chapter 1: Introduction
Rear View
Figure 1-3. System: Rear View
Figure 1-4. Node I/O Features
KVM Port
Power Button
USB UID Switch
Node 3 Node 1
Power Supply
Node 2Node 4Node 5Node 6Node 7Node 8
Power Supply
Dedicated and Centralized BMC LAN Ports
System Features: Rear
Feature Description
Nodes 1-8 Each node features a power button, two LAN ports and a KVM port
Power Supplies Two redundant power supply modules, PWS1 on the left, PWS2 on the right
BMC LAN Ports
Dedicated and centralized ports in cascade or redundancy mode
Cascade: connect the rst port to a management device and the second port to another
server
Redundancy: connect each port to a di󰀨erent subnetwork switch
KVM Port One VGA, one COM, and two USB 2.0 (with KVM dongle)
Power Button Node power switch and indicator (green)
USB Port USB Type C Port
UID Button Toggle switch and LED indicator for Unit ID (blue)

13
Chapter 1: Introduction
Connecting to a Node
USB, COM, and VGA port capabilities can be added to any node through the KVM port on
the rear. Plug in the adapter (dongle).
Figure 1-5. Adapter to Provide USB, COM, and VGA Ports
Node Trays
The chassis contains eight separate computing node drawers, each with its own motherboard.
Figure 1-6. AS -3015MR-H8TNR Node Tray
BMC Password
Each node supports a pull-out server tag with a unique BMC Password Label. Extend the
server tag and use the unique password to log into the BMC as the ADMIN user.
Service Tag

14
Chapter 1: Introduction
Main Node Components
Feature Description
PCIe Slot One PCIe 5.0 x8 (in x16) LP slot
Processor Single AMD Ryzen Zen4 7000 Series Processor
SAS/SATA Ports Two SAS2/SATA3 ports for AOC
DIMM Slots Four DIMM slots DDR5 (ECC/non-ECC UDIMM)
M.2 Slot One NVMe M.2 slot (2280/22110 form factor)
LAN Ports Dual 1GbE/10GbE/25GbE PCIe 5.0 x8 LAN ports (Micro LP)
1.3 System Architecture
This section highlights the features on each node.
Main Components
Figure 1-7. Node Features
PCIe Slot
M.2 Slot
LAN Ports
Processor SAS2/SATA3 Ports
DIMM Slots

15
Chapter 1: Introduction
System Block Diagram
The block diagram below shows the connections and relationships between the subsystems
and major components of the overall system.
Figure 1-5. System Block Diagram
USB 2.0(P3/P4)
2 SAS PORTS
(SAS AOC from PCIe x8 SLOT)
up to 5200MT/s
2 HDD PORTS
NVMe
AOC-SAS
AOC-SATA
(system front)
AOM-BPN-938NH
(Socket LGA-1718)
UDIMM-B2
DDR5 (CHA)
AM5
PCIe5.0 x8 (L0 - L7)
DDR5 (CHB)
UDIMM-A1
PCIe x8 SLOT(open end)
AMD Ryzen 7000X
H13SRD-F R1.00 BLOCK DIAGRAM
UDIMM-B1
UDIMM-A2
CPLD
Vender: MPS
SVI3 VRM
PCIe 5.0 x1(L26)
up to 5200MT/s
SERDES
AST2600
SMBUS
eSPI Bus
RGMII RTL8211FS
BMC
KVM connector
VGA
COM
PCIe5.0 x4 (L16-L19)
PCIe5.0 x4 (L20-L23)
LCMXO2
1 X USB Type-C
TPM2.0 Header
SPI Bus
TPM2.0
EEPROM
SPI 256Mb
M.2 PCIE PCIe 5.0 x2 (L24 - 25)
2x USB 3.2 (P0)
2x USB 2.0 (P0)
MLP SLOT
1 to 4
GL852GT-60
USB HUB 1x USB 2.0 (P1)
PCIe5.0 x8 (L8 - L15)
1x PCIe x8
2x USB 2.0
1x RMII
2x USB2.0 (Hub_P0-P1)
2x USB 2.0 (Hub_P2-P3)
With Redriver & PD
PI3EQX2004
IT8851FN-64
1x RMII

16
Chapter 1: Introduction
Figure 1-6. Motherboard Layout
1.4 Motherboard Layout
Below is a layout of the H13SRD-F motherboard with jumper, connector and LED locations
shown. See the table on the following page for descriptions. For detailed descriptions, pinout
information and jumper settings, refer to Chapter 4 or the Motherboard Manual.
PRESS FIT
MAC CODE
BAR CODE
BIOS
LICENSE
APU
SAS1
SAS2
MH2
MH1
MH4
MH3
MH5
MH7MH6
JPO1 JWD1
JSS1
MH8
MH9
JTPM1
J12
LEDM1
BMC
CPLD
JKVM1
JMD1
M.2-C PCIe 5.0 x2
USB2
(3.2 Gen2)
JVRM1
JBM1
BT1
JPT1
LED1
LED2
SW1
JUIDB1
SRW1
SRW2
JBT1
SGPIO1
DIMMA1 DIMMA2 DIMMB1 DIMMB2
MICRO-LP PCIe 5.0 x8
SLOT1 PCIe 5.0 x8
JPWR1
H13SRD-F
REV:1.00
DESIGNED IN USA
JKVM1MH1SW1JUIDB1
USB2LED2
MH8
J9 MICRO-LP PCIe 4.0 x8 Slot
MH4
BT1
S-SGPIO1
MH6
JPWR1
JVRM1
JPO1
JMB1
JWD1
DIMMA1
DIMMA2
DIMMB1
DIMMB2
LED1
MH2
MH3
Processor
SRW2
JTPM1
JBT1
JSS1
MH7
SRW1
LEDM1
J10 SLOT1 PCIe 5.0 x8
JMD1 M.2-C PCIe 5.0 x2
MH5
J12
MH9
SAS2
SAS1

17
Chapter 1: Introduction
Quick Reference Table
Jumper Description Default Setting
JBM1 Disable IPMI Share LAN Pins 1-2 Open: Enabled
JPO1 CPU Throttle when PWR_FAIL Pins 2-3: Disabled
JPT1 Onboard TPM 2.0 Enable/Disable Pins 1-2: Enabled
JSS1 AOC SAS Enable/Disable Pins 1-2: Disabled
JWD1 Watch Dog Timer Pins 1-2: Reset
LED Description Status
LED1 OH/Power Fail/FAN Fail Blinking Red: Power Fail or FAN Fail
Solid Red: Overheat
LED2 Unit Identier LED Solid Blue: Unit Identied
LEDM1 BMC Heartbeat LED Blinking Green: BMC Normal
Connector Description
BT1 Onboard Battery
J9 Micro Low-Prole PCIe 5.0 x8 Slot with two USB 2.0
J10 PCIe 5.0 x8 Slot (Open-End support up to 75W)
J12 MicroCloud Backplane Connector for 2x PCIe 5.0 x4 / Dedicated IPMI LAN / 2x AOC SATA/SAS
JKVM1 KVM Connector for USB0/1, COM1, VGA
JMD1 M.2 M-Key 2280/22110 Slot (PCIe 5.0 x2)
JPWR1 12V Input Power Connector
JTPM1 Trusted Platform Module/Port 80 Connector
JUIDB1 UID Switch
SAS1, SAS2 AOC SATA/SAS Input Connector (Link to AOC SAS Card)
S-SGPIO1 Serial Link General Purpose I/O Header
SW1 Power Switch, Power LED
USB0/1 Two USB 2.0 Ports via JKVM1
USB2 USB 3.2 Gen2 Type C Connector

18
Chapter 2: Server Installation
Chapter 2
Server Installation
2.1 Overview
This chapter provides advice and instructions for mounting your system in a server rack.
If your system is not already fully integrated with processors, system memory etc., refer to
Chapter 3 for details on installing those specic components.
Caution: Electrostatic Discharge (ESD) can damage electronic components. To prevent such
damage to PCBs (printed circuit boards), it is important to use a grounded wrist strap, handle
all PCBs by their edges and keep them in anti-static bags when not in use.
2.2 Unpacking the System
Inspect the box in which the AS -3015MR-H8TNR was shipped, and note if it was damaged
in any way. If any equipment appears damaged, le a damage claim with the carrier who
delivered it.
Decide on a suitable location for the rack unit that will hold the server. It should be situated
in a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical noise and
electromagnetic elds are generated. It will also require a grounded AC power outlet nearby.
Be sure to read the precautions and considerations noted in Appendix A.
2.3 Preparing for Setup
The box in which the system was shipped should include the rackmount hardware needed to
install it into the rack. Please read this section in its entirety before you begin the installation.
Choosing a Setup Location
• The system should be situated in a clean, dust-free area that is well ventilated. Avoid areas
where heat, electrical noise and electromagnetic elds are generated.
• Leave enough clearance in front of the rack so that you can open the front door completely
(~25 inches) and approximately 30 inches of clearance in the back of the rack to allow
su󰀩cient space for airow and access when servicing.
• This product should be installed only in a Restricted Access Location (dedicated equipment
rooms, service closets, etc.).

19
Chapter 2: Server Installation
• This product is not suitable for use with visual display workplace devices according to §2
of the German Ordinance for Work with Visual Display Units.
Rack Precautions
• Ensure that the leveling jacks on the bottom of the rack are extended to the oor so that
the full weight of the rack rests on them.
• In single rack installations, stabilizers should be attached to the rack. In multiple rack in-
stallations, the racks should be coupled together.
• Always make sure the rack is stable before extending a server or other component from
the rack.
• You should extend only one server or component at a time - extending two or more simul-
taneously may cause the rack to become unstable.
Server Precautions
• Review the electrical and general safety precautions in Appendix A.
• Determine the placement of each component in the rack before you install the rails.
• Install the heaviest server components at the bottom of the rack rst and then work your
way up.
• Use a regulating uninterruptible power supply (UPS) to protect the server from power
surges and voltage spikes and to keep your system operating in case of a power failure.
• Allow any drives and power supply modules to cool before touching them.
• When not servicing, always keep the front door of the rack and all covers/panels on the
servers closed to maintain proper cooling.
Rack Mounting Considerations
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating temperature of
the rack environment may be greater than the room's ambient temperature. Therefore,
consideration should be given to installing the equipment in an environment compatible with
the manufacturer’s maximum rated ambient temperature (TMRA).

20
Chapter 2: Server Installation
Airow
Equipment should be mounted into a rack so that the amount of airow required for safe
operation is not compromised.
Mechanical Loading
Equipment should be mounted into a rack so that a hazardous condition does not arise due
to uneven mechanical loading.
Circuit Overloading
Consideration should be given to the connection of the equipment to the power supply circuitry
and the e󰀨ect that any possible overloading of circuits might have on overcurrent protection
and power supply wiring. Appropriate consideration of equipment nameplate ratings should
be used when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack itself should be
grounded. Particular attention should be given to power supply connections other than the
direct connections to the branch circuit (i.e. the use of power strips, etc.).
To prevent bodily injury when mounting or servicing this unit in a rack, you must take
special precautions to ensure that the system remains stable. The following guidelines
are provided to ensure your safety:
• This unit should be mounted at the bottom of the rack if it is the only unit in the rack.
• When mounting this unit in a partially lled rack, load the rack from the bottom to the top
with the heaviest component at the bottom of the rack.
• If the rack is provided with stabilizing devices, install the stabilizers before mounting or
servicing the unit in the rack.
• Slide rail mounted equipment is not to be used as a shelf or a work space.
Warning: do not pick up the server with the front handles. They are designed to pull
the system from a rack only.
Slide rail mounted equipment is not to be used as a shelf or a work space.

21
Chapter 2: Server Installation
2.4 Installing the Rails
There are a variety of rack units on the market, which may require a slightly di󰀨erent assembly
procedure. Do not use a two post "telco" type rack. This rail set ts a rack between 26.5"
and 36.4" deep.
The following is a basic guideline for installing the system into a rack with the rack mounting
hardware provided. You should also refer to the installation instructions that came with the
specic rack you are using.
Identifying the Rails
The chassis package includes two rail assemblies. Each assembly consists of three sections:
An inner rail that secures directly to the chassis, an outer rail that secures to the rack, and
a middle rail which extends from the outer rail. These assemblies are specically designed
for the left and right side of the chassis and labeled.
Figure 2-1. Identifying the Outer Rail, Middle Rail and Inner Rail
(Left Rail Assembly Shown)
Note: Both front chassis rails and the rack rails have a locking tab, which serves two functions.
First, it locks the server into place when installed and pushed fully into the rack (its normal
operating position. In addition, these tabs lock the server in place when fully extended from
the rack. This prevents the server from coming completely out of the rack when pulled out
for servicing.
Inner Rail
Locking Tab
Middle Rail
Outer Rail

22
Chapter 2: Server Installation
Note: The gure above is for illustrative purposes only. Always install servers at the bottom
of the rack rst.
Stability hazard. The rack stabilizing mechanism must be in place, or the rack must
be bolted to the oor before you slide the unit out for servicing. Failure to stabilize the
rack can cause the rack to tip over.
Figure 2-2. Extending and Releasing the Inner Rail
Releasing the Inner Rail
Each inner rail has a locking latch. This latch prevents the server from coming completely
out of the rack when the chassis is pulled out for servicing.
To mount the rail onto the chassis, rst release the inner rail from the outer rails.
1. Pull the inner rail out of the outer rail until it is fully extended as illustrated below.
2. Press the locking tab down to release the inner rail.
3. Pull the inner rail all the way out.
Inner Rail
Locking Tab
Middle Rail
Outer Rail
1
2
3

23
Chapter 2: Server Installation
Installing the Inner Rails
Begin the rack mounting procedure by installing the inner rails to the chassis.
1. Identify the left and right inner rails. They are labeled.
2. Place the inner rail rmly against the side of the chassis, aligning the hooks on the side
of the chassis with the holes in the inner rail.
3. Slide the inner rail forward toward the front of the chassis and under the hooks until the
quick release bracket snaps into place, securing the rail to the chassis.
4. If desired, add screws (two at head M4 x 4mm) through the rail into the chassis for
extra security.
Figure 2-3. Installing the Rails
Warning: Do not pick up the server with the front handles. They are designed to pull
the system from a rack only.
Inner Rails
4
4
2
3

24
Chapter 2: Server Installation
Figure 2-4. Extending and Mounting the Outer Rails
Installing the Outer Rails onto the Rack
Each end of the assembled outer rail includes a bracket with hooks and spring-loaded pegs
to t into the square holes in your rack.
Installing the Outer Rail
1. Press upward on the locking tab at the rear end of the middle rail.
2. Push the middle rail back into the outer rail.
3. Hang the hooks on the front of the outer rail onto the square holes on the front of the
rack. If desired, use screws to secure the outer rails to the rack.
4. Pull out the rear of the outer rail, adjusting the length until it just ts within the posts of
the rack.
5. Hang the hooks of the rear section of the outer rail onto the square holes on the rear of
the rack. Take care that the proper holes are used so the rails are level. If desired, use
screws to secure the rear of the outer rail to the rear of the rack.
21D01
L-min=676.00(26.61")(outer rail)
1
4
2
5
3

25
Chapter 2: Server Installation
Note: Keep the ball bearing shuttle locked at the front of the middle rail during installation.
Note: Figure is for illustrative purposes only. Always install servers to the bottom of a rack rst.
2.5 Installing the Chassis into a Rack
Once rails are attached to the chassis and the rack, you can install the server.
1. Pull the middle rail out of the front of the outer rail and make sure that the ball bearing
shuttle is locked at the front of the middle rail.
2. Align the rear of the chassis rails with the middle rails and then push evenly on both
sides of the chassis until it clicks into the fully extended position.
3. Depress the locking tabs on both sides of the chassis and push the it fully into the rack.
The locking tabs should "click".
4. Optional screws may be used to hold the front of the chassis to the rack.
Figure 2-5. Installing the Server into the Rack

26
Chapter 2: Server Installation
Figure 2-6. Removing the Chassis From the Rack
Removing the Chassis from the Rack
Caution! It is dangerous for a single person to o󰀨-load the heavy chassis from the rack without
assistance. Be sure to have su󰀩cient assistance supporting the chassis when removing it
from the rack. Use a lift.
1. Pull the chassis forward out the front of the rack until it stops.
2. Press the release latches on each of the inner rails downward simultaneously and
contiue to pull the chassis forward and out of the rack.
1
2

27
Chapter 3: Maintenance and Component Installation
Chapter 3
Maintenance and Component Installation
This chapter provides instructions on installing and replacing main system components. To
prevent compatibility issues, only use components that match the specications and/or part
numbers given.
Installation or replacement of most components require that power rst be removed from the
system. Please follow the procedures given in each section.
3.1 Powering Down the System
Use the following procedure to ensure that power has been removed from the system. This
step is necessary when removing or installing non hot-swap components.
1. Use the operating system to power down the system.
2. After the system has completely shut down, disconnect the AC power cord(s) from the
power strip or outlet and remove the AC power cords from all power supply modules.
3. Disconnect the power cord(s) from the power supply module(s).
4. When performing service on non hot-swap components, remove the system from the
rack and place it on a bench or desk. Do not service with the system extended from the
rack.

28
Chapter 3: Maintenance and Component Installation
Figure 3-1. Removing the Chassis Cover
3.2 Accessing the System
The CSE-938NH chassis features a removable top cover, which allows easy access to the
inside of the chassis.
Removing the Top Cover
1. If necessary, remove power from the system as described in Section 3.1.
2. Remove the three screws securing the cover to the chassis. See Figure 3-1.
3. Lift the cover from the chassis.
Caution: Except for short periods of time, do not operate the server without the cover in place.
The chassis cover must be in place to allow for proper airow and to prevent overheating.
Remove Three
Screws
2

29
Chapter 3: Maintenance and Component Installation
Figure 3-2. Corresponding Nodes, Fans and Storage Drives
Nodes
The system includes eight removable computing nodes, each containing an individual
motherboard. A node may be removed while other nodes continue to function.
Each node controls two storage drives and shares a fan with the node beside it. If a node
is pulled out of the chassis, the storage drives associated with that node will power-down.
Corresponding Nodes, Fans and Storage Drives
Node Fan Storage Drives
Node A Fan 1 Drives A1 and A2
Node B Fan 1 Drives B1 and B2
Node C Fan 2 Drives C1 and C2
Node D Fan 2 Drives D1 and D2
Node E Fan 3 Drives E1 and E2
Node F Fan 3 Drives F1 and F2
Node G Fan 4 Drives G1 and G2
Node H Fan 4 Drives H1 and H2
Node H
Fan 1
Fan 3
Fan 2
Fan 4
Node G
Node F
Node E
Node D
Node C
Node B
Node A
B
D
F
H
A
C
E
G