Supermicro SuperStorage Server 2028R-NR48N User manual

USER’S MANUAL
Revision 1.0
SUpERStoRAgE SERvER®
2028R-NR48N

The information in this User’s Manual has been carefully reviewed and is believed to be accurate.
The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
makes no commitment to update or to keep current the information in this manual, or to notify any
person or organization of the updates. Please Note: For the most up-to-date version of this
manual, please see our web site at www.supermicro.com.
Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this manual at any time and without notice. This product, including software and
documentation, is the property of Supermicro and/or its licensors, and is supplied only under a
license. Any use or reproduction of this product is not allowed, except as expressly permitted by
the terms of said license.
IN NO EVENT WILL SUPERMICRO BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO
USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES. IN PARTICULAR, SUPERMICRO SHALL NOT HAVE LIABILITY FOR ANY
HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE
COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH
HARDWARE, SOFTWARE, OR DATA.
Any disputes arising between manufacturer and customer shall be governed by the laws of Santa
Clara County in the State of California, USA. The State of California, County of Santa Clara shall
be the exclusive venue for the resolution of any such disputes. Super Micro's total liability for all
claims will not exceed the price paid for the hardware product.
FCC Statement: This equipment has been tested and found to comply with the limits for a Class
A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide
reasonable protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the manufacturer’s instruction manual, may cause harmful
interference with radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference, in which case you will be required to correct the interference at your
own expense.
California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate
warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”
WARNING: Handling of lead solder materials used in this
product may expose you to lead, a chemical known to
the State of California to cause birth defects and other
reproductive harm.
Manual Revision 1.0
Release Date: June 14, 2016
Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.
Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.
Copyright © 2016 by Super Micro Computer, Inc.
All rights reserved.
Printed in the United States of America

iii
PrefacePreface
Preface
About this Manual
This manual is written for professional system integrators and PC technicians. It
provides information for the installation and use of the SuperServer. Installation and
maintenance should be performed by experienced technicians only.
Please refer to the server specications page on our Web site for updates on
supported memory, processors and operating systems (www.supermicro.com).
This manual may be periodically updated without notice. Please check the
Supermicro Web site for possible updates to the manual revision level.
Warnings
Special attention should be given to the following symbols used in this manual.
Warning! Indicates high voltage may be encountered when performing
a procedure.
Warning! Indicates important information given to prevent equipment/
property damage or personal injury.

iv
SuperStorage Server 2028R-NR48N
Contents
Chapter 1 Introduction
1-1 Overview ........................................................................................................ 1-1
1-2 Motherboard Features..................................................................................... 1-2
Processors ...................................................................................................... 1-2
Memory ........................................................................................................... 1-2
NVMe Ports .................................................................................................... 1-2
Serial ATA ...................................................................................................... 1-2
PCI Expansion Slots ...................................................................................... 1-2
Rear I/O Ports................................................................................................. 1-2
SIOM Network Slot ......................................................................................... 1-3
1-3 Chassis Features .......................................................................................... 1-4
System Power................................................................................................. 1-4
Drives .............................................................................................................. 1-4
PCI Expansion Slots ....................................................................................... 1-4
Front Control Panel......................................................................................... 1-4
Cooling System............................................................................................... 1-4
1-4 Contacting Supermicro.................................................................................... 1-5
Chapter 2 Rack Installation
2-1 Unpacking the System .................................................................................... 2-1
2-2 Preparing for Setup......................................................................................... 2-1
Choosing a Setup Location............................................................................. 2-1
2-3 Warnings and Precautions.............................................................................. 2-2
Rack Precautions............................................................................................ 2-2
General Server Precautions............................................................................ 2-2
Rack Mounting Considerations ....................................................................... 2-3
2-4 Rack Mounting Instructions............................................................................. 2-4
Identifying the Sections of the Rack Rails...................................................... 2-4
Releasing the Inner Rail ................................................................................. 2-5
Installing The Inner Rails on the Chassis ...................................................... 2-6
Installing the Outer Rails on the Rack............................................................ 2-7
Installing into the Rack.................................................................................... 2-8
Removing the Chassis From the Rack........................................................... 2-9
Removing the Outer Rails From the Rack.................................................... 2-10
2-5 Installing the Cable Management Arm ..........................................................2-11
Chapter 3 System Interface
3-1 Overview ......................................................................................................... 3-1

v
Preface
3-2 Control Panel Buttons ..................................................................................... 3-2
Power .............................................................................................................. 3-2
Reset............................................................................................................... 3-2
3-3 Control Panel LEDs ........................................................................................ 3-2
Overheating..................................................................................................... 3-4
3-4 Drive Carrier LEDs.......................................................................................... 3-4
3-5 Power Supply LEDs ........................................................................................ 3-5
Chapter 4 Standardized Warning Statements for AC Systems
About Standardized Warning Statements....................................................... 4-1
Warning Denition ........................................................................................... 4-1
Installation Instructions.................................................................................... 4-4
Circuit Breaker ................................................................................................ 4-5
Power Disconnection Warning........................................................................ 4-6
Equipment Installation..................................................................................... 4-8
Restricted Area................................................................................................ 4-9
Battery Handling............................................................................................ 4-10
Redundant Power Supplies (if applicable to your system)........................... 4-12
Backplane Voltage (if applicable to your system)......................................... 4-13
Comply with Local and National Electrical Codes........................................ 4-14
Product Disposal........................................................................................... 4-15
Hot Swap Fan Warning (if applicable to your system) ................................. 4-16
Power Cable and AC Adapter ...................................................................... 4-18
Chapter 5 Advanced Motherboard Setup
5-1 Handling the Motherboard .............................................................................. 5-1
Precautions ..................................................................................................... 5-1
5-2 Cable and Device Connectiions...................................................................... 5-2
Power Connections .................................................................................... 5-2
Fan Cabling................................................................................................ 5-2
Control Panels............................................................................................ 5-2
5-3 Processor and Heatsink Installation................................................................ 5-3
Installing a CPU Heatsink ............................................................................... 5-6
Removing the Heatsink................................................................................... 5-6
5-4 Installing Memory............................................................................................ 5-7
Memory Support.............................................................................................. 5-7
DIMM Installation ............................................................................................ 5-7
5-5 Motherboard Details...................................................................................... 5-10
X10DSC+ Quick Reference .......................................................................... 5-10
5-6 Connector Denitions ................................................................................... 5-13
Power Connectors......................................................................................... 5-13

vi
SuperStorage Server 2028R-NR48N
Control Panel Connector............................................................................... 5-14
Other Connectors.......................................................................................... 5-16
5-7 Jumper Settings ............................................................................................ 5-20
5-8 Onboard Indicators........................................................................................ 5-22
5-9 Drive Ports ................................................................................................... 5-24
SATA.............................................................................................................. 5-24
NVMe ............................................................................................................ 5-24
5-10 Installing Software......................................................................................... 5-25
SuperDoctor®5 ............................................................................................. 5-26
5-11 Onboard Battery............................................................................................ 5-27
Chapter 6 Chassis Setup and Maintenance
6-1 Overview ......................................................................................................... 6-1
6-2 Removing Power from the System ................................................................. 6-2
6-3 Removing the Top Covers............................................................................... 6-3
Mid-chassis Cover........................................................................................... 6-3
Rear Cover...................................................................................................... 6-4
6-4 Storage Drives ................................................................................................ 6-5
Drive Connections........................................................................................... 6-5
Drive Carriers ................................................................................................. 6-6
Installing a Solid State Drive (SSD)................................................................ 6-7
Installing Rear Hard Drives............................................................................. 6-8
6-5 Installing the Expansion Cards ....................................................................... 6-9
6-6 Installing a SIOM Card.................................................................................... 6-9
6-7 System Fans ..................................................................................................6-11
6-8 Installing the Air Shrouds.............................................................................. 6-13
6-9 Power Supply ............................................................................................... 6-14
Chapter 7 BIOS
7-1 Introduction...................................................................................................... 7-1
Starting BIOS Setup Utility.............................................................................. 7-1
How To Change the Conguration Data ......................................................... 7-1
7-2 Main Setup...................................................................................................... 7-2
Starting the Setup Utility ................................................................................. 7-2
7-3 Advanced Setup Congurations...................................................................... 7-4
7-4 Event Logs .................................................................................................... 7-30
7-5 IPMI............................................................................................................... 7-32
7-6 Security Settings ........................................................................................... 7-34
7-7 Boot Settings................................................................................................. 7-37
7-8 Save & Exit ................................................................................................... 7-39

vii
Preface
Appendix A BIOS Error Beep Codes
Appendix B UEFI BIOS Recovery Instructions
Appendix C System Specications

viii
SuperStorage Server 2028R-NR48N
Notes

Chapter 1
Introduction
1-1 Overview
The SuperStorage server 2028R-NR48N is a high-end solution comprised of two
main subsystems: the SC226STS-R1K62P2 2U chassis and the X10DSC+ dual
processor motherboard. It features forty-eight hot-swap drives, NVMe capabilty,
and SIOM for networking exibility. It is powered by dual redundant 1600 W high-
efciency, hot-swap power modules.
It is useful for IOPS-intensive storage applications, database applications such as
MySQL or Casandra, virtual storage environments, single instance storage and
data deduplication.
Refer to the Supermicro web site for information on operating systems that have
been certied for use with the system (www.supermicro.com).
In addition to the above components, the server includes:
• Two SAS3 backplanes (BPN-SAS3-226SEL1-N4)
• Five 8-cm system cooling fans (FAN-0118L4)
• One add-on module for NVMe (AOC-SLG3-4E4R-P)
• Two passive CPU heatsinks (SNK-P0047PS)
• One rackmount rail kit (MCP-290-00138-0N)
Note: For your system to work properly, please follow the links below to download
all necessary drivers/utilities and the user’s manual for your server.
• Supermicro product manuals: http://www.supermicro.com/support/manuals/
• Product drivers and utilities: ftp://ftp.supermicro.com
• Product safety info:
http://www.supermicro.com/about/policies/safety_information.cfm
For support, email [email protected].
Chapter 1: Introduction
1-1

1-2
SuperStorage Server 2028R-NR48N
1-2 Motherboard Features
The SuperStorage server 2028R-NR48N is built around the X10DSC+, a dual
processor motherboard based on the Intel C612 chipset. Below are the main
features. (See Figure 1-1 for a block diagram of the chipset.)
Processors
The motherboard supports single or dual Intel E5-2600 v3/v4 Series processors
in socket R3 (LGA 2011). Refer to the motherboard description pages on the
Supermicro website for a complete listing of supported processors (www.
supermicro.com).
Memory
The motherboard has 24 DIMM slots that can support Load Reduced (LRDIMM)
or Registered (RDIMM) ECC DDR4-2400/2133/1866/1600 memory. See Chapter
5 for details.
NVMe Ports
The motherboard features four NVMExpress ports (PCI-E 3.0 x4) using OCuLink.
The NVMe ports provide high-speed, low-latency connections directly from the
CPU to NVMe solid state drives (SSDs). This greatly increases SSD throughput
and signicantly reduces storage device latency by simplifying driver and software
requirements resulting from the direct PCI-E interface between the CPU and the
NVMe SSD drives.
Serial ATA
A SATA controller is integrated into the chipset to provide four SATA 3.0 (6/Gbps)
ports), which support RAID 0, 1 and 10. RAID 5 is supported with Windows OS only.
PCI Expansion Slots
The X10DSC+ has two PCI-E 3.0 x16 and one PCI-E x8 slot.
Rear I/O Ports
The I/O ports include one COM port, a VGA port, two USB 3.0 ports and a dedicated
IPMI LAN (Ethernet) port.

1-3
Chapter 1: Introduction
SIOM Network Slot
The motherboard provides a PCI-E x16 slot for a Super I/O module (SIOM)
networking card. Several SIOM congurations are available.
Figure 1-1. Intel C612 Chipset:
System Block Diagram
Note: This is a general block diagram. Please see Chapter 5 for details.
DDR4 DIMM
X10DSC+ Block Diagram
C
D
#3
#2 #1
#1
#1#1
#2
#3
#2
#3
#3
#2
QPI0
DMI
PE3PE2PE1
A
B
H
G
#3
#2
SocketID 01
#1#1
#1#1
#2
#3
#2
#3
#3
#2
E
CPU2
DDR4 DIM
M
F
QPI1
DMI
x16
NCSI
DDR3
UART
x4
8~11
x4
4~7
0~3
port 1,2
with RJ45
REAR
port 3
TYPE-A
x8
x16
PCIe Slot1
PCIe Slot2
PCIe Slot3
0~15
x8
x16 (0-15)
NCSI
NVMe1NVMe2NVMe3NVMe4
x4
12~15
x4
JMAZZ1 (SAS AOM1)
0~7
SPI
PEI
BMC
AST2400
HWM
VGA LAN
LPC
SPI USB
DDR3
VGA
32MB BMC
SPI Flash
MUX
16MB BMC
SPI Flash
SPI USB 2.0
[6/7]
LPC
PCH
PE[3] TPM Header
PHY
RTL8211E IPMI LAN
COM1
SATA Gen3 [2/3]
SATA Gen3 [0/1]
SATA_DOM1 SATA_DOM2
S-SATA1
S-SATA2
PE3 PE2 PE1 QP11QP10
32GB/s
32GB/s
16GB/s
QPI
QPI
9.6GT/s
Polarity Inversion Polarity Inversion
SocketID 00
CPU1
4GB/s
32GB/s
32GB/s
16GB/s
USB 2.0 [10/12/13]
USB 3.0 [1/2/6]
DDR4 DIM
M
DDR4 DIM
M
DDR4 DIM
M
DDR4 DIM
M
DDR4 DIMM
DDR4 DIM
M
#3
DDR4 DIM
M
DMI
#3
DDR4 DIM
M
9.6GT/s
SIOM SLOT
0~15

1-4
SuperStorage Server 2028R-NR48N
1-3 Chassis Features
The 2028R-NR48N is built upon the SC226STS-R1K62P2 chassis.
System Power
The system features dual redundant 1600 W power supplies consisting of two
hot-plug power modules. They have 80 Plus certication at Titanium Level (96%)
high-efciency. The system will continue to operate if one module fails or is replaced.
Drives
The chassis supports forty-eight hot-swap 2.5" NVMe solid state (SSDs) drives.
Twenty-four are accessble from the front, and 24 additional drives are accessable
on the top of the chassis by means of a pop-up riser bay. Each set of 24 drives is
supported by a separate backplane, the rear-most referred to herein as the mid-
plane.
The chassis also supports two additional hot-swap 2.5" hard drives at the rear.
PCI Expansion Slots
The system allows three low prole, externally accessable PCI-E cards--two x16
slots and one x8 slot. One of the x16 slots is used for the NVMe add-on card.
Front Control Panel
The chassis front control panel provides system monitoring and power control.
Status LEDs indicate system power, HDD activity, network activity, UID, and
overheat and fan failure.
Cooling System
The chassis has an innovative cooling design that features ve 8-cm hot-swap high-
performance fans. Fan speed can be determined by system temperature through
IPMI. The system also uses an air shroud to channel air where needed. Each power
supply module also includes a cooling fan.

1-5
Chapter 1: Introduction
1-4 Contacting Supermicro
Headquarters
Address: Super Micro Computer, Inc.
980 Rock Ave.
San Jose, CA 95131 U.S.A.
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
Web Site: www.supermicro.com
Europe
Address: Super Micro Computer B.V.
Het Sterrenbeeld 28, 5215 ML
's-Hertogenbosch, The Netherlands
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
[email protected] (Customer Support)
Web Site: www.supermicro.nl
Asia-Pacic
Address: Super Micro Computer, Inc.
3F, No. 150, Jian 1st Rd.
Zhonghe Dist., New Taipei City 235
Taiwan (R.O.C)
Tel: [phone]
Fax: [phone]
Email: [email protected]
Web Site: www.supermicro.com.tw

1-6
SuperStorage Server 2028R-NR48N
Notes

2-1
Chapter 2
Rack Installation
This chapter provides instructions for preparing and mounting your chassis in a rack.
2-1 Unpacking the System
You should inspect the box the chassis was shipped in and note if it was damaged
in any way. If the chassis itself shows damage, le a damage claim with the carrier
who delivered it.
2-2 Preparing for Setup
Decide on a suitable location for the rack unit that will hold your chassis. It should
be a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical
noise and electromagnetic elds are generated. A nearby grounded power outlet.
is required
The box your chassis was shipped in should include two sets of rail assemblies, two
rail mounting brackets and the mounting screws to mount the system into the rack.
Please read this chapter in its entirety before beginning the installation procedure.
Choosing a Setup Location
• Leave at least 25 inches clearance in front of the rack to open the front door
completely.
• Leave approximately 30 inches of clearance in the back of the rack to allow for
sufcient airow and access for servicing.
• It should be a restricted access location, such as a dedicated equipment room
or a service closet.

SuperStorage Server 2028R-NR48N
2-2
2-3 Warnings and Precautions
Rack Precautions
• Ensure that the leveling jacks on the bottom of the rack are fully extended to
the oor with the full weight of the rack resting on them.
• In single rack installations, stabilizers should be attached to the rack.
• In multiple rack installations, the racks should be coupled together.
• Always make sure that the rack is stable before extending a component from
the rack.
• Only one chassis should be extended from the rack at a time. Extending two or
more chassis simultaneously may cause the rack to become unstable.
General Server Precautions
• Review the electrical and general safety precautions that came with the
components you are adding to your chassis.
• Determine the placement of each component in the rack before you install the
rails.
• Install the heaviest server components on the bottom of the rack rst, and then
work upward.
• Use a regulating uninterruptible power supply (UPS) to protect the server from
power surges, voltage spikes and to keep your system operating in case of a
power failure.
• Allow the hot-swappable hard drives and power supply modules to cool before
touching them.
• Always keep the rack's front door and all panels and components on the servers
closed when not servicing to maintain proper cooling.

2-3
Rack Mounting Considerations
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating
temperature of the rack environment may be greater than the ambient temperature
of the room. Therefore, consideration should be given to installing the equipment
in an environment compatible with the manufacturer’s maximum rated ambient
temperature (TMRA).
Sufcient Airow
Equipment should be mounted into a rack so that the amount of airow required
for safe operation is not compromised.
Circuit Overloading
Consideration should be given to the connection of the equipment to the power
supply circuitry and the effect that any possible overloading of circuits might have
on overcurrent protection and power supply wiring. Appropriate consideration of
equipment nameplate ratings should be used when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack
itself should be grounded. Particular attention should be given to power supply
connections other than the direct connections to the branch circuit (i.e. the use of
power strips, etc.).
Warning: To prevent bodily injury when mounting or servicing this unit in a
rack, you must take special precautions to ensure that the system remains
stable. The following guidelines are provided to ensure your safety:
• This unit should be mounted at the bottom of the rack if it is the only unit in
the rack.
• When mounting this unit in a partially lled rack, load the rack from the bottom
to the top with the heaviest component at the bottom of the rack.
• If the rack is provided with stabilizing devices, install the stabilizers before
mounting or servicing the unit in the rack.
• Slide rail mounted equipment is not to be used as a shelf or a work space.

SuperStorage Server 2028R-NR48N
2-4
2-4 Rack Mounting Instructions
This section provides information on installing the chassis into a rack unit with the
rails provided. There are a variety of rack units on the market, which may mean
that the assembly procedure will differ slightly from the instructions provided. You
should also refer to the installation instructions that came with the rack unit you are
using. Note: This rail will t a rack between 26.5" and 36.4" deep.
Identifying the Sections of the Rack Rails
The chassis package includes two rail assemblies in the rack mounting kit. Each
assembly consists of three sections: An inner rail that secures directly to the chassis,
an outer rail that secures to the rack, and a middle rail that slides in the outer rail.
These assemblies are specically designed for the left and right side of the chassis.
Figure 2-1. Identifying the Inner Rails, Middle Rails and Outer Rails
Middle Rails,
Front
Outer Rails,
Rear
Inner Rails
Right Outer
Rail
Left Outer
Rail

2-5
Middle Rail
Locking Lever
on Inner Rail
Outer Rail
Figure 2-2. Extending the Rails and Releasing the Inner Rail
Releasing the Inner Rail
It is necessary to release the inner rail from the middle and outer rails before
installing the inner rail on the chassis.
Releasing the Inner Rail from the Middle and Outer Rails
1. Lift the front latch on the inner rail and pull the inner rail out of the middle rail,
and the middle rail out of the outer rail until the rails are fully extended.
2. Press down the locking lever on the inside of the inner rail to release the inner
rail. Continue to pull the inner rail out of the middle rail.
Inner Rail
Front Latch
Front Latch on
Inner Rail
Locking Lever on
Middle Rail
(on inside of rail)

SuperStorage Server 2028R-NR48N
2-6
Figure 2-3. Installing the Inner Rails
Installing The Inner Rails on the Chassis
Installing the Inner Rails
1. Conrm that the left and right inner rails have been correctly identied.
2. Place the inner rail rmly against the side of the chassis, aligning the pins on
the side of the chassis with the slotted thru holes in the inner rail.
3. Slide the inner rail toward the rear of the chassis until the pins are at the end of
the narrow slot, which secures the inner rail to the chassis. An optional screw
may be added for extra security.
4. Repeat for the other inner rail.
Warning: Do not pick up the server by the front handles. They are designed
to pull the system from a rack only.
3
2