Supermicro SuperServer 5038MR-H8TRF User manual

USER’S MANUAL
Revision 1.0a
SUPERSERVER®
5038MR-H8TRF

The information in this User’s Manual has been carefully reviewed and is believed to be accurate.
The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
makes no commitment to update or to keep current the information in this manual, or to notify any
person or organization of the updates. Please Note: For the most up-to-date version of this
manual, please see our web site at www.supermicro.com.
Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this manual at any time and without notice. This product, including software and
documentation, is the property of Supermicro and/or its licensors, and is supplied only under a
license. Any use or reproduction of this product is not allowed, except as expressly permitted by
the terms of said license.
IN NO EVENT WILL SUPERMICRO BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO
USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES. IN PARTICULAR, SUPERMICRO SHALL NOT HAVE LIABILITY FOR ANY
HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE
COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH
HARDWARE, SOFTWARE, OR DATA.
Any disputes arising between manufacturer and customer shall be governed by the laws of Santa
Clara County in the State of California, USA. The State of California, County of Santa Clara shall
be the exclusive venue for the resolution of any such disputes. Super Micro's total liability for all
claims will not exceed the price paid for the hardware product.
FCC Statement: This equipment has been tested and found to comply with the limits for a Class
A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide
reasonable protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the manufacturer’s instruction manual, may cause harmful
interference with radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference, in which case you will be required to correct the interference at your
own expense.
California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate
warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”
WARNING: Handling of lead solder materials used in this
product may expose you to lead, a chemical known to
the State of California to cause birth defects and other
reproductive harm.
Manual Revision 1.0a
Release Date: June 05, 2017
Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.
Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.
Copyright © 2017 by Super Micro Computer, Inc.
All rights reserved.
Printed in the United States of America

3
Preface
Preface
About this Manual
This manual is written for professional system integrators and PC technicians. It
provides information for the installation and use of the SuperServer. Installation and
maintainance should be performed by experienced technicians only.
             
supported memory, processors and operating systems (http://www.supermicro.
com).
Notes
For your system to work properly, please follow the links below to download all
necessary drivers/utilities and the user’s manual for your server.
• Supermicro product manuals: http://www.supermicro.com/support/manuals/
• Product drivers and utilities: ftp://ftp.supermicro.com
• Product safety info:
http://www.supermicro.com/about/policies/safety_information.cfm
If you have any questions, please contact our support team at:
[email protected]
This manual may be periodically updated without notice. Please check the
Supermicro Web site for possible updates to the manual revision level.
Warnings
Special attention should be given to the following symbols used in this manual.
Warning! Indicates high voltage may be encountered when performing
a procedure.
Warning! Indicates important information given to prevent equipment/
property damage or personal injury.

4
SUPERSERVER 5038MR-H8TRF User's Manual
Contents
Chapter 1 Introduction..............................................................................1-1
1-1 Overview ........................................................................................................ 1-1
1-2 Serverboard Features ..................................................................................... 1-2
Processors ..................................................................................................... 1-2
Memory ........................................................................................................... 1-2
Input/Output Ports........................................................................................... 1-2
1-3 Server Chassis Features ................................................................................ 1-4
System Power................................................................................................. 1-4
Hard Drives ..................................................................................................... 1-4
PCI Expansion Slots ....................................................................................... 1-4
User Interface.................................................................................................. 1-4
Cooling System............................................................................................... 1-4
1-4 Contacting Supermicro.................................................................................... 1-5
Chapter 2 Server Installation
2-1 Unpacking the System .................................................................................... 2-1
2-2 Preparing for Setup......................................................................................... 2-1
Choosing a Setup Location............................................................................. 2-1
2-3 Warnings and Precautions.............................................................................. 2-2
Rack Precautions............................................................................................ 2-2
Server Precautions.......................................................................................... 2-2
Rack Mounting Considerations ....................................................................... 2-3
Ambient Operating Temperature ................................................................ 2-3
 ......................................................................................... 2-3
Mechanical Loading ................................................................................... 2-3
Circuit Overloading..................................................................................... 2-3
Reliable Ground ......................................................................................... 2-3
2-4 Installing the System into a Rack ................................................................. 2-4
Identifying the Sections of the Rack Rails...................................................... 2-4
Releasing the Inner Rail ................................................................................. 2-5
Installing the Inner Rails on the Chassis ........................................................ 2-6
Installing the Outer Rails onto the Rack......................................................... 2-7
Sliding the Chassis onto the Rack Rails......................................................... 2-8
Chapter 3 System Interface......................................................................3-1
3-1 Overview ......................................................................................................... 3-1
3-2 Control Panel Buttons ..................................................................................... 3-2
Power .............................................................................................................. 3-2
Reset .............................................................................................................. 3-2

5
Preface
3-3 Control Panel LEDs ........................................................................................ 3-3
Power Fail ....................................................................................................... 3-3
Node Status LEDs .......................................................................................... 3-3
3-4 Hard Drive Carrier LEDs................................................................................. 3-4
3-5 Node Controls ................................................................................................. 3-5
Power Button and LED ................................................................................... 3-5
UID Button and LED ....................................................................................... 3-5
Failure LED ..................................................................................................... 3-5
3-6 Power Supply LEDs ........................................................................................ 3-6
Chapter 4 Standardized Warning Statements for AC Systems
About Standardized Warning Statements....................................................... 4-1
........................................................................................... 4-1
Installation Instructions.................................................................................... 4-4
Circuit Breaker ................................................................................................ 4-5
Power Disconnection Warning ........................................................................ 4-6
Equipment Installation..................................................................................... 4-8
Restricted Area................................................................................................ 4-9
Battery Handling............................................................................................ 4-10
Redundant Power Supplies (if applicable to your system)........................... 4-12
Backplane Voltage (if applicable to your system)......................................... 4-13
Comply with Local and National Electrical Codes........................................ 4-14
Product Disposal........................................................................................... 4-15
Hot Swap Fan Warning (if applicable to your system) ................................. 4-16
Power Cable and AC Adapter ...................................................................... 4-18
Chapter 5 Advanced Serverboard Setup
5-1 Handling the Serverboard ............................................................................... 5-1
Precautions ..................................................................................................... 5-1
Unpacking ....................................................................................................... 5-1
5-2 Installing the Processor and Heatsink ............................................................ 5-2
Installing an LGA 2011 Processor................................................................... 5-2
Installing a CPU Heatsink ............................................................................... 5-5
Removing the Heatsink .................................................................................. 5-5
5-3 Input/Output Ports........................................................................................... 5-6
5-4 Installing Memory............................................................................................ 5-7
Memory Support.............................................................................................. 5-7
Memory Population Guidelines ....................................................................... 5-7
5-5 Serverboard Details ........................................................................................ 5-9
 .....................................................................................5-11
Power Connectors ........................................................................................5-11

6
SUPERSERVER 5038MR-H8TRF User's Manual
Rear Input/Output Panel ................................................................................5-11
Other Connectors.......................................................................................... 5-12
5-7 Jumper Settings ............................................................................................ 5-15
5-8 Onboard Indicators........................................................................................ 5-18
5-9 SATA Ports .................................................................................................... 5-19
5-10 Installing Software......................................................................................... 5-20
SuperDoctor®5 ............................................................................................. 5-21
5-11 Onboard Battery............................................................................................ 5-22
Chapter 6 Advanced Chassis Setup........................................................6-1
6-1 Overview ......................................................................................................... 6-1
6-2 Removing Power from the System ................................................................. 6-2
6-3 Removing the Chassis Cover ......................................................................... 6-3
6-4 Corresponding Nodes, Fans and Hard Drives................................................ 6-4
6-5 Removing Motherboard Nodes ....................................................................... 6-5
6-6 Installing Hard Drives...................................................................................... 6-6
6-7 Installing an Air Shroud................................................................................... 6-9
6-8 System Fans ................................................................................................. 6-10
6-9 Power Supply.................................................................................................6-11
Replacing a Power Supply Module................................................................6-11
6-10 Removing and Installing the Backplane ...................................................... 6-13
Chapter 7 BIOS
7-1 Introduction...................................................................................................... 7-1
Starting BIOS Setup Utility.............................................................................. 7-1
......................................................... 7-1
7-2 Main Setup...................................................................................................... 7-2
Starting the Setup Utility ................................................................................. 7-2
 ...................................................................... 7-4
7-4 Event Logs .................................................................................................... 7-26
7-5 IPMI Settings................................................................................................. 7-28
7-6 Security ......................................................................................................... 7-30
7-7 Boot............................................................................................................... 7-33
7-8 Save & Exit ................................................................................................... 7-35
Appendix A BIOS Error Beep Codes ..................................................... A-1
Appendix B UEFI BIOS Recovery........................................................... B-1
Appendix C System Specications ........................................................ C-1

7
Preface
Contacting Supermicro
Headquarters
Address: Super Micro Computer, Inc.
980 Rock Ave.
San Jose, CA 95131 U.S.A.
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
Web Site: www.supermicro.com
Europe
Address: Super Micro Computer B.V.
Het Sterrenbeeld 28, 5215 ML
's-Hertogenbosch, The Netherlands
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
[email protected] (Customer Support)
Web Site: www.supermicro.nl
Asia-Pacic
Address: Super Micro Computer, Inc.
3F, No. 150, Jian 1st Rd.
Zhonghe Dist., New Taipei City 235
Taiwan (R.O.C)
Tel: [phone]
Fax: [phone]
Email: [email protected]
Web Site: www.supermicro.com.tw

8
SUPERSERVER 5038MR-H8TRF User's Manual
Notes

Chapter 1
Introduction
1-1 Overview
The SuperServer 5038MR-H8TRF is a high-density multi-node server comprised
of the SC938BH-R1620BP 3U server chassis and eight hot-pluggable computing

systems are listed at www.supermicro.com.
In addition to the above components, the server includes:
• SAS backplane supporting the HDDs (BPN-SAS-938H)
• Four 8-cm system cooling fans (FAN-0133L4)
• One rackmount rail kit (MCP-290-00057-0N)
Per node:
• Expansion card (AOC-CGP-I2-O-P)
• Heatsink (SNK-P0047PS+)
• One air shroud (MCP-310-93802-0B)
• Riser Card: (RSC-RR1U-E8-O-P)
Note: For your system to work properly, please follow the links below to download
all necessary drivers/utilities and the user’s manual for your server.
• Supermicro product manuals: http://www.supermicro.com/support/manuals/
• Product drivers and utilities: ftp://ftp.supermicro.com
• Product safety info:
http://www.supermicro.com/about/policies/safety_information.cfm
For support, email [email protected].
Chapter 1: Introduction
1-1

1-2
SUPERSERVER 5038MR-H8TRF User's Manual
1-2 Serverboard Features
Each computing node of the SuperServer 5038MR-H8TRF features the X10SRD-F,
a single processor motherboard based on the Intel PCH C612 chipset. Below are
the main features. Figure 1-1 displays a block diagram of the chipset.
Processors
The serverboard supports a single Intel Xeon E5-2600/E5-1600 v3 series processor
in LGA2011 sockets (Socket R3). Refer to the Supermicro web site for a complete
listing of supported processors (www.supermicro.com).
Memory
The serverboard has four memory sockets that can support up to 256 GB RDIMM
(Registered DIMMs) or LRDIMM. (Load-Reduced DIMMs). Memory must be ECC
DDR4, 2133/1866/1600 Hz.
Input/Output Ports
The rear I/O ports for each node include two Gb LAN ports, a dedicated IPMI LAN
port, two USB 2.0 ports (with KVM dongle), one COM port, and one VGA port.
SATA Controller
Each node has two SATA3 ports on the back panel, two SATA3 ports onboard and
two SATA2/SAS for add-on cards.

1-3
Chapter 1: Introduction
Figure 1-1. Intel PCH C612 Chipset:
System Block Diagram
Note: This is a general block diagram. Please see Chapter 5 for details.
LPC
JPCIE1
PCIE 3.0 x16
DDR3 DIMM
B1
DDR3 DIMM
A1
DMI
USB
PROCESSOR
QPI1
QPI0
PCI-E x1
PCH
DMI
HEADER
Vertical
USB CONN
6
USB 3.0
1,2
DDR3 DIMM
C1 D1
DDR3 DIMM
Socket 00
CPU0
BMC
AST2400
DDR 3
SATA3.0#2
SATA GEN 3
SATA3.0#1 SATA3.0#5 SATA3.0#6
SPI
FLASH
KVM CONNECTOR
R,G,B COM
IPMI LAN
PHY
RTL8211E
RMII
PE3PE2

1-4
SUPERSERVER 5038MR-H8TRF User's Manual
1-3 Server Chassis Features
The 5038MR-H8TRF is built upon the SC938BH-R1620BP chassis. Details and
related procedures can be found in Chapter 6.The following is a general outline of
the main features of the chassis.
System Power
The chassis features dual redundant 1620 W hot-plug power modules. They have 80

to operate if one module fails or is replaced.
Hard Drives
The chassis supports up to sixteen 3.5" hot-swap hard drives, connected through
a SAS backplane.
PCI Expansion Slots

User Interface
The chassis front control panel provides system monitoring and power control.
Multicolor LEDs indicate the status of each computing node. In addition, an LED
alerts the failure of a power supply module.
Cooling System

fans. Fan speed can be controlled based on the system temperature using IPMI.
Each power supply module also includes a cooling fan.

1-5
Chapter 1: Introduction
1-4 Contacting Supermicro
Headquarters
Address: Super Micro Computer, Inc.
980 Rock Ave.
San Jose, CA 95131 U.S.A.
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
Web Site: www.supermicro.com
Europe
Address: Super Micro Computer B.V.
Het Sterrenbeeld 28, 5215 ML
's-Hertogenbosch, The Netherlands
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
[email protected] (Customer Support)
Web Site: www.supermicro.nl
Asia-Pacic
Address: Super Micro Computer, Inc.
3F, No. 150, Jian 1st Rd.
Zhonghe Dist., New Taipei City 235
Taiwan (R.O.C)
Tel: [phone]
Fax: [phone]
Email: [email protected]
Web Site: www.supermicro.com.tw

1-6
SUPERSERVER 5038MR-H8TRF User's Manual
Notes

Chapter 2: Server Installation
2-1
Chapter 2
Server Installation
This chapter provides instructions for preparing and mounting your chassis in a rack.
2-1 Unpacking the System
You should inspect the box the chassis was shipped in and note if it was damaged

who delivered it.
2-2 Preparing for Setup
Decide on a suitable location for the rack unit that will hold your chassis. It should
be a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical

is required
The box your chassis was shipped in should include two sets of rail assemblies, two
rail mounting brackets and the mounting screws to mount the system into the rack.
Please read this chapter in its entirety before beginning the installation procedure.
Choosing a Setup Location
• Leave at least 25 inches clearance in front of the rack to open the front door
completely.
• Leave approximately 30 inches of clearance in the back of the rack to allow for

• It should be a restricted access location, such as a dedicated equipment room
or a service closet.

2-2
SUPERSERVER 5038MR-H8TRF User's Manual
2-3 Warnings and Precautions
Rack Precautions
• Ensure that the leveling jacks on the bottom of the rack are fully extended to

• In single rack installation, stabilizers should be attached to the rack. In multiple
rack installations, the racks should be coupled together.
• Always make sure the rack is stable before extending a component from the
rack.
• You should extend only one component at a time - extending two or more
simultaneously may cause the rack to become unstable.
Server Precautions
• Review the electrical and general safety precautions in Chapter 4.
• Determine the placement of each component in the rack before you install the
rails.
• 
work up.
• Use a regulating uninterruptible power supply (UPS) to protect the server from
power surges, voltage spikes and to keep your system operating in case of a
power failure.
• Allow the hot plug SAS drives and power supply modules to cool before touching
them.
• Always keep the rack's front door and all panels and components on the servers
closed when not servicing to maintain proper cooling.

Chapter 2: Server Installation
2-3
Rack Mounting Considerations
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating
temperature of the rack environment may be greater than the ambient temperature
of the room. Therefore, consideration should be given to installing the equipment
in an environment compatible with the manufacturer’s maximum rated ambient
temperature (Tmra).
Reduced Airow

for safe operation is not compromised.
Mechanical Loading
Equipment should be mounted into a rack so that a hazardous condition does not
arise due to uneven mechanical loading.
Circuit Overloading
Consideration should be given to the connection of the equipment to the power
supply circuitry and the effect that any possible overloading of circuits might have
on overcurrent protection and power supply wiring. Appropriate consideration of
equipment nameplate ratings should be used when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack
itself should be grounded. Particular attention should be given to power supply
connections other than the direct connections to the branch circuit (i.e. the use of
power strips, etc.).
Warning! To prevent bodily injury when mounting or servicing this unit in a
rack, you must take special precautions to ensure that the system remains
stable. The following guidelines are provided to ensure your safety:
• This unit should be mounted at the bottom of the rack if it is the only unit in
the rack.
• 
to the top with the heaviest component at the bottom of the rack.
• If the rack is provided with stabilizing devices, install the stabilizers before
mounting or servicing the unit in the rack.
• Slide rail mounted equipment is not to be used as a shelf or a work space.

2-4
SUPERSERVER 5038MR-H8TRF User's Manual
2-4 Installing the System into a Rack
This section provides information on installing the server into a rack unit with the
rack rails provided. There are a variety of rack units on the market, so the assembly
procedure may differ slightly. Refer to the installation instructions that came with
your rack. Note: 
Identifying the Sections of the Rack Rails
The chassis package includes two rail assemblies. Each assembly consists of three
sections: An inner rail that secures directly to the chassis, an outer rail that secures
to the rack, and a middle rail which extends from the outer rail. These assemblies

Figure 2-1. Identifying the Outer Rail, Middle Rail and Inner Rail
(Left Rail Assembly Shown)
Inner Rail
Rail Assembly
(Shown with Rails
Retracted)
This Side Faces
Outward
Locking Tab
Middle Rail
Outer Rail

Chapter 2: Server Installation
2-5
Figure 2-2. Extending and Releasing the Inner Rail
Releasing the Inner Rail
Each inner rail has a locking latch. This latch prevents the server from coming
completely out of the rack when when the chassis is pulled out for servicing.

Releasing Inner Rail from the Outer Rails
1. Pull the inner rail out of the outer rail until it is fully extended as illustrated below.
2. Press the locking tab down to release the inner rail.
3. Pull the inner rail all the way out.
4. Repeat for the other outer rail.
1
1
1
2
1
3

2-6
SUPERSERVER 5038MR-H8TRF User's Manual
Figure 2-3. Installing the Inner Rails
Installing the Inner Rails on the Chassis
Installing the Inner Rails
1. Identify the left and right inner rails. They are labeled.
2. 
the side of the chassis with the holes in the inner rail.
3. Slide the inner rail forward toward the front of the chassis until the quick release
bracket snaps into place, securing the rail to the chassis.
4. Optionally, you can further secure the inner rail to the chassis with a screw.
5. Repeat for the other inner rail.
1
3
1
4
1
4
1
2
Inner Rails