Supero SUPERSERVER 6028TR-DTR User manual

®
SUPER
USER'S MANUAL
Revision 1.0
SUPERSERVER
6028TR-DTR
6028TR-D72R

The information in this User’s Manual has been carefully reviewed and is believed to be accurate.
The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
makes no commitment to update or to keep current the information in this manual, or to notify any
person or organization of the updates. Please Note: For the most up-to-date version of this
manual, please see our web site at www.supermicro.com.
Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this manual at any time and without notice. This product, including software and
documentation, is the property of Supermicro and/or its licensors, and is supplied only under a
license. Any use or reproduction of this product is not allowed, except as expressly permitted by
the terms of said license.
IN NO EVENT WILL SUPERMICRO BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO
USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES. IN PARTICULAR, SUPERMICRO SHALL NOT HAVE LIABILITY FOR ANY
HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE
COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH
HARDWARE, SOFTWARE, OR DATA.
Any disputes arising between manufacturer and customer shall be governed by the laws of Santa
Clara County in the State of California, USA. The State of California, County of Santa Clara shall
be the exclusive venue for the resolution of any such disputes. Super Micro's total liability for all
claims will not exceed the price paid for the hardware product.
FCC Statement: This equipment has been tested and found to comply with the limits for a Class
A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide
reasonable protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the manufacturer’s instruction manual, may cause harmful
interference with radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference, in which case you will be required to correct the interference at your
own expense.
California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate
warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”
WARNING: Handling of lead solder materials used in this
product may expose you to lead, a chemical known to
the State of California to cause birth defects and other
reproductive harm.
Manual Revision 1.0
Release Date: October 17, 2014
Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.
Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.
Copyright © 2014 by Super Micro Computer, Inc.
All rights reserved.
Printed in the United States of America

iii
Preface
Preface
About This Manual
This manual is written for professional system integrators and PC technicians.
It provides information for the installation and use of the SuperServer
6028TR-DTR/D72R. Installation and maintainance should be performed by
experienced technicians only.
The SuperServer 6028TR-DTR/D72R is a high-end server based on the
SC827HD-R1K28BP 2U rackmount chassis and the dual processor X10DRT-H
serverboard.
Manual Organization
Chapter 1: Introduction
The rst chapter provides a checklist of the main components included with the
server system and describes the main features of the X10DRT-H serverboard and
the SC827HD-R1K28BP chassis.
Chapter 2: Server Installation
This chapter describes the steps necessary to install the SuperServer
6028TR-DTR/D72R into a rack and check out the server conguration prior to
powering up the system.
Chapter 3: System Interface
Refer here for details on the system interface, which includes the functions and
information provided by the control panel on the chassis as well as other LEDs
located throughout the system.
Chapter 4: Standardized Safety Warnings
You should thoroughly familiarize yourself with this chapter for a general overview
of safety precautions that should be followed when installing and servicing the
SuperServer 6028TR-DTR/D72R.
Chapter 5: Advanced Serverboard Setup
Chapter 5 provides detailed information on the X10DRT-H serverboard, including the
locations and functions of connections, headers and jumpers. Refer to this chapter
when adding or removing processors or main memory and when reconguring the
serverboard.
Chapter 6: Advanced Chassis Setup

SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
iv
Refer to Chapter 6 for detailed information on the SC827HD-R1K28BP server
chassis. You should follow the procedures given in this chapter when installing,
removing or reconguring SATA or peripheral drives and when replacing system
power supply units and cooling fans.
Chapter 7: BIOS
The BIOS chapter includes an introduction to BIOS and provides detailed information
on running the CMOS Setup Utility.
Appendix A: BIOS Error Beep Codes
Appendix B: System Specications

v
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
Notes

vi
Table of Contents
Chapter 1 Introduction
1-1 Overview ......................................................................................................... 1-1
1-2 Serverboard Features ..................................................................................... 1-2
Processors ...................................................................................................... 1-2
Memory ........................................................................................................... 1-2
SAS ................................................................................................................. 1-2
SATA .............................................................................................................. 1-2
PCI Expansion Slots ....................................................................................... 1-2
Onboard Controllers/Ports .............................................................................. 1-3
Graphics Controller ......................................................................................... 1-3
Other Features................................................................................................ 1-3
1-3 Server Chassis Features ................................................................................ 1-3
System Power................................................................................................. 1-3
SAS/SATA Subsystem..................................................................................... 1-3
Front Control Panel......................................................................................... 1-4
Cooling System............................................................................................... 1-4
Mounting Rails ................................................................................................ 1-4
1-5 Contacting Supermicro.................................................................................... 1-6
1-6 2U Twin: System Notes................................................................................... 1-7
Nodes.............................................................................................................. 1-7
System Power................................................................................................. 1-7
Hard Drive Backplane/Drives.......................................................................... 1-7
Chapter 2 Server Installation
2-1 Overview ......................................................................................................... 2-1
2-2 Unpacking the System .................................................................................... 2-1
2-3 Preparing for Setup......................................................................................... 2-1
Choosing a Setup Location............................................................................. 2-1
2-4 Warnings and Precautions .............................................................................. 2-2
Rack Precautions............................................................................................ 2-2
Server Precautions.......................................................................................... 2-2
Rack Mounting Considerations ....................................................................... 2-3
Ambient Operating Temperature ................................................................ 2-3
Reduced Airow ......................................................................................... 2-3
Mechanical Loading ................................................................................... 2-3
Circuit Overloading..................................................................................... 2-3
Reliable Ground ......................................................................................... 2-3
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL

vii
Removing the Protective Film......................................................................... 2-4
2-4 Rack Mounting Instructions............................................................................. 2-5
Separating the Sections of the Rack Rails..................................................... 2-5
Installing the Inner Rail Extensions ................................................................ 2-6
Outer Rack Rails............................................................................................. 2-7
Chapter 3 System Interface
3-1 Overview ......................................................................................................... 3-1
3-2 Control Panel Buttons ..................................................................................... 3-1
Power .............................................................................................................. 3-1
UID .................................................................................................................. 3-1
3-3 Control Panel LEDs ........................................................................................ 3-2
Overheat/Fan Fail ........................................................................................... 3-2
NIC .................................................................................................................. 3-2
3-4 Hard Drive Carrier LEDs................................................................................. 3-2
Chapter 4 Standardized Warning Statements for AC Systems
4-1 About Standardized Warning Statements....................................................... 4-1
Warning Denition ........................................................................................... 4-1
Installation Instructions.................................................................................... 4-4
Circuit Breaker ................................................................................................ 4-5
Power Disconnection Warning ........................................................................ 4-6
Equipment Installation..................................................................................... 4-8
Restricted Area................................................................................................ 4-9
Battery Handling............................................................................................ 4-10
Redundant Power Supplies .......................................................................... 4-12
Backplane Voltage ........................................................................................ 4-13
Comply with Local and National Electrical Codes........................................ 4-14
Product Disposal........................................................................................... 4-15
Hot Swap Fan Warning................................................................................. 4-16
Power Cable and AC Adapter ...................................................................... 4-18
Chapter 5 Advanced Serverboard Setup
5-1 Handling the Serverboard ............................................................................... 5-1
Precautions ..................................................................................................... 5-1
Unpacking ....................................................................................................... 5-1
5-2 Connecting Cables.......................................................................................... 5-2
Connecting Data Cables ................................................................................. 5-2
5-3 Rear I/O Ports................................................................................................. 5-3
5-4 Processor and Heatsink Installation................................................................ 5-4
Installing a Passive CPU Heatsink ................................................................. 5-7
Table of Contents

viii
Removing the Heatsink................................................................................... 5-8
5-5 Installing Memory............................................................................................ 5-9
Memory Support.............................................................................................. 5-9
5-6 Adding PCI Expansion Cards ........................................................................5-11
5-7 Serverboard Details ...................................................................................... 5-12
X10DRT-H Quick Reference ......................................................................... 5-12
5-8 Connector Denitions.................................................................................... 5-14
5-9 Jumper Settings ............................................................................................ 5-17
Explanation of Jumpers ................................................................................ 5-17
5-10 Onboard Indicators........................................................................................ 5-20
5-11 PCI-Express and SATA Connections............................................................ 5-21
5-12 Installing Software......................................................................................... 5-22
SuperDoctor® 5 ............................................................................................ 5-23
5-13 Onboard Battery............................................................................................ 5-24
Chapter 6 Advanced Chassis Setup
6-1 Static-Sensitive Devices.................................................................................. 6-1
Precautions ..................................................................................................... 6-1
Unpacking ....................................................................................................... 6-1
6-2 Control Panel .................................................................................................. 6-2
6-3 Chassis Cover................................................................................................. 6-3
6-4 Installing the Air Shrouds................................................................................ 6-4
Air Shrouds ..................................................................................................... 6-4
6-5 Checking the Airow ....................................................................................... 6-5
6-6 System Fans ................................................................................................... 6-5
Optional Fan Congurations ........................................................................... 6-5
6-7 Removing and Installing the Backplane.......................................................... 6-8
Removing the Backplane ................................................................................ 6-8
Installing the Backplane................................................................................ 6-10
6-8 Installing the Serverboard ..............................................................................6-11
I/O Shield .......................................................................................................6-11
Node Installation/Removal .............................................................................6-11
Permanent and Optional Standoffs................................................................6-11
6-9 Adapter Card Replacement........................................................................... 6-15
Expansion Card/Expansion Slot Setup......................................................... 6-16
Installing the Riser Card onto the Riser Card Bracket ................................. 6-16
6-10 Installing and Removing Hard Drives ........................................................... 6-17
6-11 Power Supply................................................................................................ 6-20
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL

Chapter 7 BIOS
7-1 Introduction...................................................................................................... 7-1
Starting BIOS Setup Utility.............................................................................. 7-1
How To Change the Conguration Data ......................................................... 7-2
Starting the Setup Utility ................................................................................. 7-2
7-2 Main Setup...................................................................................................... 7-2
7-3 Advanced Setup Congurations...................................................................... 7-4
7-4 Event Logs .................................................................................................... 7-30
7-5 IPMI............................................................................................................... 7-32
7-6 Security Settings ........................................................................................... 7-34
7-7 Boot Settings................................................................................................. 7-35
7-8 Save & Exit ................................................................................................... 7-37
Appendix A BIOS Error Beep Codes
Appendix B System Specications
ix
Table of Contents

x
Notes
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL

Chapter 1
Introduction
1-1 Overview
The SuperServer 6028TR-DTR/D72R is a high-end server comprised of two main
subsystems: the SC827HD-R1K28BP 2U server chassis and the X10DRT-H dual
processor serverboard in four hot-swap nodes. Please refer to our web site for
information on operating systems that have been certied for use with the system
(www.supermicro.com).
In addition to the serverboard and chassis, various hardware components have
been included with the SuperServer 6028TR-DTR/D72R server, as listed below:
•Four heatsinks (two SNK-P0048PW and two SNK-P0048PS)
•Two air shrouds (MCP-310-82717-0B)
•Four 80 x 80 x 38 mm cooling fans (FAN-0129L4)
•SATA/SAS Backplane
Two HD backplanes (BPN-ADP-S2208L-H6iR-O-P)
One SAS backplane for (BPN-SAS-827HD)
Twelve hot-swap 3.5" HDD trays (MCP-220-00075-0B)
Four 21-cm SATA cables (CBL-0473L)
•Two riser cards (RSC-R2UT-3E8R-O-P)
•One rail set (MCP-290-00053-0N)
Note: For your system to work properly, please follow the links below to download
all necessary drivers/utilities and the user’s manual for your server.
•Supermicro product manuals: http://www.supermicro.com/support/manuals/
•Product drivers and utilities: ftp://ftp.supermicro.com
•Product safety info: http://super-dev/about/policies/safety_information.cfm
•If you have any questions, please contact our support team at:
[email protected]
Chapter 1: Introduction
1-1

1-2
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
1-2 Serverboard Features
At the heart of the SuperServer 6028TR-DTR/D72R lies the
X10DRT-H, a dual processor serverboard based on the Intel®
C612 chipset and designed to provide maximum performance. Two of these
serverboards are mounted in the SC827HD-R1K28BP chassis.
The sections below cover the main features of the X10DRT-H serverboard (see
Figure 1-1 for a block diagram of the chipset).
Processors
The X10DRT-H supports single or dual Intel® Xeon® E5-2600 series processors
(Socket R LGA 2011). Please refer to the serverboard description pages on our
website for a complete listing of supported processors (www.supermicro.com).
Memory
The X10DRT-H has eight (8) DIMM slots supporting up to 512 GB of
DDR4-2133/1866/1600 MHz speed registered ECC SDRAM in up to
4 GB, 8 GB, 16 GB, 32 GB or 64 GB sizes at 1.2V. See Chapter 5 for details.
Note: Check the Supermicro website (www.supermicro.com) for the latest memory
support information.
SAS
The 6028TR-D72R includes an LSI 2208 SAS controller to support six SAS 2.0
hard drives per node. RAID 0, 1, 5 and 10 are supported.
SATA
A Serial ATA controller is integrated into the C612 to provide up to a three-port 6
Gb/s SATA subsystem, (two SATA 3.0 (6 Gb/s) and two SATA 2.0 (3 Gb/s), which is
either RAID 0, 1 and 10 (SATA 2.0) or RAID 0, 1, 5 and 10 (SATA 3.0) supported.
The SATA drives are hot-swappable units.
Note: The operating system you use must have RAID support to enable the hot-
swap capability and RAID function of the SATA drives.
PCI Expansion Slots
The SuperServer 6028TR-DTR/D72R has for each node one PCI Express 3.0 x16
slot (Slot 1), one PCI Express 3.0 x8 slot for a rear I/O riser card (SXB1) and one
PCI Express 3.0 x8 slot for a Supermicro proprietary daughter (add-on) card (SXB2).

1-3
Chapter 1: Introduction
Onboard Controllers/Ports
The rear I/O ports include one COM port, a VGA (monitor) port, two USB 3.0
ports (additional internal USB headers are included on the serverboard), an IPMI
dedicated LAN port and two Ethernet ports.
Note: For IPMI Conguration Instructions, please refer to the Embedded BMC
Conguration User's Guide available at http://www.supermicro.com/support/
manuals/.
Graphics Controller
The X10DRT-H features an integrated Matrox G200eW video controller.
Other Features
Other onboard features that promote system health include onboard voltage
monitors, a chassis intrusion header, auto-switching voltage regulators, chassis and
CPU overheat sensors, virus protection, node manager software and BIOS rescue.
1-3 Server Chassis Features
The following is a general outline of the main features of the SC827HD server
chassis.
System Power
Each SC827HD chassis model includes a high-efciency 80-plus Platinum certied
power supply, rated at 1280 watts plus one redundant backup power supply. In
the unlikely event your power supply fails, replacement is simple and can be
accomplished without tools.
SAS/SATA Subsystem
The SC827HD supports up to twelve 3.5" hot-swap SAS or SATA drives in trays
(six for each node). These drives are hot-swappable units and are connected to a
backplane that provides power and control.
Note: The operating system you use must have RAID support to enable the hot-
swap capability of the drives.

1-4
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
Front Control Panel
SC827HD-R1K28BP chassis features two independant control panels associated
with each serverboard (node) in the chassis. Each control panel has LEDs to
indicate power on, network activity, power fail, fan fail, system overheat conditions
and the UID LED. Each control panel also includes a main power button and a
UID button.
Cooling System
The SC827HD chassis has an innovative cooling design that features four 8-cm
high-performance fans. Fan speed may be determined by system temperature. See
Chapter 6 for details.
Mounting Rails
The SC827HD includes a set of quick-release rails, and can be placed in a rack
for secure storage and use. To setup your rack, follow the step-by-step instructions
included in this manual.

1-5
Chapter 1: Introduction
Figure 1-1. Intel C612 Chipset:
System Block Diagram
Note: This is a general block diagram and may not exactly repre-
sent the features on your motherboard. See the previous pages for
the actual specications of your motherboard. This block diagram is
intended for your reference only.
SPI
LAN3
RGMII
Debug Card
FRONT PANELCTRL
FAN SPEED
PCI-E X1 G2
USB 2.0
#12 USB2.0
#1/#2/#4/#5/6/7/8
PCH
6.0 Gb/S
USB 2.0
LPC
USB
#1
#0
SATA
RTL8211E-VB-CG
RJ45
BIOS
SPI
SPI
Temp Sensor
W83773G
TPM HEADER
USB 3.0
USB
BIOS
HEADER
SPI
AST2400
BMC
RMII/NCSI
COM1
Connector
VGA CONN
BMC Boot Flash
DDR3
5 PHASE
145W
1866/2133
1866/2133
DDRIV
P1
P1
P0
VR12.5
P0
#2-1
DDRIV
#1-4
#1-3
#1-2
#1-1
QPI
9.6G
SLOT 1
l
PCI-E X16 G3
DMI2
PCI-E X16
PCI-E X8 G3
PCI-E X8 G3
2IMD2IMD
l
SNB CORE
DDR-IV
SNB CORE
DDR-IV
QPI
9.6G
4GB/s
5 PHASE
145W
VR12.5
#3B #1 #2 #3B#2#3A #1
#2-2
#2-3
#2-4
<=1.758W (average)
2.3W (Peak)
TDP: 5W (SERVER)
Idle:0.45W
5V:1.2A
3.3V:0.1A
3.3 STBY:0.2A
1.05 PCH
1.05 ASW
1.5 PCH
PVCCIO 1.0/0.95
3.3STBY:0.5A
PCI-E X4 G3
i350
LAN1
#3
PCI-E X8 in X4 slot
SXB2
#3A
PCI-E X8 G3
1UPC0UPC
x2 rear port
x2 rear port
PCI-E X8 in X4 slot
Mellanox CX3
IB QDR/FDR
SXB1
S-SATA
I-SATA
6.0 Gb/S
Adaptor Interface
SPI Switch
SPI

1-6
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
1-5 Contacting Supermicro
Headquarters
Address: Super Micro Computer, Inc.
980 Rock Ave.
San Jose, CA 95131 U.S.A.
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
Website: www.supermicro.com
Europe
Address: Super Micro Computer B.V.
Het Sterrenbeeld 28, 5215 ML
's-Hertogenbosch, The Netherlands
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
[email protected] (Customer Support)
Website: www.supermicro.nl
Asia-Pacic
Address: Super Micro Computer, Inc.
3F, No. 150, Jian 1st Rd.
Zhonghe Dist., New Taipei City 235
Taiwan (R.O.C)
Tel: [phone]
Fax: [phone]
Email: [email protected]
Website: www.supermicro.com.tw

1-7
Chapter 1: Introduction
1-6 2U Twin: System Notes
As a 2U Twin conguration, the SuperServer 6028TR-DTR/D72R is a unique server
system. With two system boards incorporated into a single chassis acting as two
separate nodes, there are several points you should keep in mind.
Nodes
Each of the two serverboards act as a separate node in the system. As independent
nodes, each may be powered off and on without affecting the other. In addition, each
node is a hot-swappable unit that may be removed from the rear of the chassis.
The nodes are connected to the server backplane by means of an adapter card.
Note: A guide pin is located between the upper and lower nodes on the inner chassis
wall. This guide pin also acts as a “stop” when a node is fully installed. If too much
force is used when inserting a node this pin may break off. Take care to slowly slide
a node in until you hear the “click” of the locking tab seating itself.
System Power
Dual 1280 power supplies are used to provide the power for both serverboards.
Each serverboard however, can be shut down independently of the other with the
power button on its respective control panel.
Hard Drive Backplane/Drives
As a system, the SuperServer 6028TR-DTR/D72R supports the use of twelve hard
drives (SAS and SATA). A single backplane works to apply system-based control
for power and fan speed functions, yet at the same time logically connects a set
of six hard drives to each serverboard. Consequently, RAID setup is limited to a
six-drive scheme (RAID cannot be spread across all twelve drives).
See the Drive Bay Installation/Removal section in Chapter 6 for the logical hard
drive and node conguration.

1-8
SUPERSERVER 6028TR-DTR/D72R USER'S MANUAL
Notes

Chapter 2: Server Installation
2-1
Chapter 2
Server Installation
2-1 Overview
This chapter provides a quick setup checklist to get your system up and running.
Following these steps in the order given should enable you to have it operational
within a minimum amount of time. This setup assumes that your system has come to
you with the processors and memory preinstalled. If your system is not already fully
integrated with a serverboard, processors, system memory etc., please turn to the
chapter or section noted in each step for details on installing specic components.
2-2 Unpacking the System
You should inspect the box the system was shipped in and note if it was damaged
in any way. If the server itself shows damage you should le a damage claim with
the carrier who delivered it.
Decide on a suitable location for the rack unit that will hold the server. It should be
situated in a clean, dust-free area that is well ventilated. Avoid areas where heat,
electrical noise and electromagnetic elds are generated. You will also need it placed
near a grounded power outlet. Be sure to read the Rack and Server Precautions
in the next section.
2-3 Preparing for Setup
The box the server was shipped in should include the hardware needed to install the
system into the rack. Follow the steps in the order given to complete the installation
process in a minimum amount of time. Please read this section in its entirety before
you begin the installation procedure outlined in the sections that follow.
Choosing a Setup Location
•Leave enough clearance in front of the rack to enable you to open the front
door completely (~25 inches).
•Leave approximately 30 inches of clearance in the back of the rack to allow for
sufcient airow and ease in servicing.
•This product is for installation only in a Restricted Access Location (dedicated
equipment rooms, service closets and the like).

2-2
SUPERSERVER 6028TR-DTR/D72R User's Manual
•This product is not suitable for use with visual display work place devices ac-
cording to §2 of the the German Ordinance for Work with Visual Display Units.
2-4 Warnings and Precautions
Rack Precautions
•Ensure that the leveling jacks on the bottom of the rack are fully extended to
the oor with the full weight of the rack resting on them.
•In single rack installation, stabilizers should be attached to the rack.
•In multiple rack installations, the racks should be coupled together.
•Always make sure the rack is stable before extending a component from it.
•You should extend only one component at a time - extending two or more si-
multaneously may cause the rack to become unstable.
Server Precautions
•Review the electrical and general safety precautions in Chapter 4.
•Determine the placement of each component in the rack before you install the
rails.
•Install the heaviest server components on the bottom of the rack rst, and then
work up.
•Use a regulating uninterruptible power supply (UPS) to protect the server from
power surges, voltage spikes and to keep your system operating in case of a
power failure.
•Allow the hot plug SATA drives and power supply modules to cool before touch-
ing them.
•Always keep the rack's front door and all panels and components on the servers
closed when not servicing to maintain proper cooling.
•Make sure all power and data cables are properly connected and not blocking
the chassis airow. See Chapter 5 for details on cable connections.