Supero SUPERSERVER 1017GR-TF User manual

SUPER
SUPERSERVER
1017GR-TF
®
USER’S MANUAL
Revision 1.0

The information in this User’s Manual has been carefully reviewed and is believed to be accurate.
The vendor assumes no responsibility for any inaccuracies that may be contained in this document,
makes no commitment to update or to keep current the information in this manual, or to notify any
person or organization of the updates. Please Note: For the most up-to-date version of this
manual, please see our web site at www.supermicro.com.
Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product
described in this manual at any time and without notice. This product, including software and docu-
mentation, is the property of Supermicro and/or its licensors, and is supplied only under a license.
Any use or reproduction of this product is not allowed, except as expressly permitted by the terms
of said license.
IN NO EVENT WILLSUPERMICRO BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,
SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO
USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES. IN PARTICULAR, SUPERMICRO SHALL NOT HAVE LIABILITY FOR ANY
HARDWARE, SOFTWARE, OR DATASTORED OR USED WITH THE PRODUCT, INCLUDING THE
COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH
HARDWARE, SOFTWARE, OR DATA.
Any disputes arising between manufacturer and customer shall be governed by the laws of Santa
Clara County in the State of California, USA. The State of California, County of Santa Clara shall
be the exclusive venue for the resolution of any such disputes. Super Micro's total liability for all
claims will not exceed the price paid for the hardware product.
FCC Statement: This equipment has been tested and found to comply with the limits for a Class A
digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable
protection against harmful interference when the equipment is operated in a commercial environ-
ment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed
and used in accordance with the manufacturer’s instruction manual, may cause harmful interference
with radio communications. Operation of this equipment in a residential area is likely to cause harmful
interference, in which case you will be required to correct the interference at your own expense.
California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate warn-
ing applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate
Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”
WARNING: Handling of lead solder materials used in this
product may expose you to lead, a chemical known to the
State of California to cause birth defects and other repro-
ductive harm.
Manual Revision 1.0
Release Date: May 11, 2012
Unless you request and receive written permission from Super Micro Computer, Inc., you may not
copy any part of this document.
Information in this document is subject to change without notice. Other products and companies
referred to herein are trademarks or registered trademarks of their respective companies or mark
holders.
Copyright © 2012 by Super Micro Computer, Inc.
All rights reserved.
Printed in the United States of America

Preface
About This Manual
This manual is written for professional system integrators and PC technicians. It
provides information for the installation and use of the SuperServer 1017GR-TF.
Installation and maintenance should be performed by experienced technicians only.
The 1017GR-TF is based on the SC118G-1K43BP 1U rackmount server chassis
and the Super X9SRG-F serverboard. Please refer to our web site for an up-to-date
list of supported operating systems, processors and memory.
Manual Organization
Chapter 1: Introduction
The first chapter provides a checklist of the main components included with the
server system and describes the main features of the Super X9SRG-F serverboard
and the SC118G-1K43BP chassis.
Chapter 2: Server Installation
This chapter describes the steps necessary to install the system into a rack and
check out the server configuration prior to powering up the system. If your server
was ordered without the processor and memory components, this chapter will refer
you to the appropriate sections of the manual for their installation.
Chapter 3: System Interface
Refer to this chapter for details on the system interface, which includes the functions
and information provided by the control panel on the chassis as well as other LEDs
located throughout the system.
Chapter 4: System Safety
You should thoroughly familiarize yourself with this chapter for a general overview
of safety precautions that should be followed when installing and servicing the
system.
iii
Preface

SUPERSERVER 1017GR-TF User's Manual
iv
Chapter 5: Advanced Serverboard Setup
Chapter 5 provides detailed information on the X9SRG-F serverboard, including the
locations and functions of connectors, headers and jumpers. Refer to this chapter
when adding or removing processors or main memory and when reconfiguring the
serverboard.
Chapter 6: Advanced Chassis Setup
Refer to Chapter 6 for detailed information on the SC118G-1K43BP 1U rackmount
server chassis. You should follow the procedures given in this chapter when install-
ing, removing or reconfiguring SATAor peripheral drives and when replacing system
power supply units and cooling fans.
Chapter 7: BIOS
The BIOS chapter includes an introduction to BIOS and provides detailed informa-
tion on running the BIOS Setup Utility.
Appendix A: BIOS Error Beep Codes
Appendix B: System Specifications

v
Preface
Notes

vi
Table of Contents
Chapter 1 Introduction
1-1 Overview .........................................................................................................1-1
1-2 Serverboard Features..................................................................................... 1-2
Processors ......................................................................................................1-2
Memory ...........................................................................................................1-2
Serial ATA ....................................................................................................... 1-2
PCI Expansion Slots....................................................................................... 1-2
I/O Ports..........................................................................................................1-2
IPMI.................................................................................................................1-2
1-3 Server Chassis Features ................................................................................1-3
System Power................................................................................................. 1-3
SATA Subsystem.............................................................................................1-3
Front Control Panel.........................................................................................1-3
Cooling System............................................................................................... 1-3
1-4 GPU Subsystem..............................................................................................1-4
1-5 Contacting Supermicro....................................................................................1-6
Chapter 2 Server Installation
2-1 Overview .........................................................................................................2-1
2-2 Unpacking the System....................................................................................2-1
2-3 Preparing for Setup.........................................................................................2-1
Choosing a Setup Location............................................................................. 2-1
Rack Precautions............................................................................................ 2-2
Server Precautions..........................................................................................2-2
Rack Mounting Considerations....................................................................... 2-3
Ambient Operating Temperature................................................................2-3
Reduced Airflow .........................................................................................2-3
Mechanical Loading ................................................................................... 2-3
Circuit Overloading.....................................................................................2-3
Reliable Ground ......................................................................................... 2-3
2-4 Installing the System into a Rack ...................................................................2-4
Identifying the Sections of the Rack Rails......................................................2-4
Installing the Inner Rail Extensions ................................................................2-5
Assembling the Outer Rails............................................................................2-6
Installing the Outer Rails onto the Rack.........................................................2-7
Installing and Removing the Chassis From a Rack .......................................2-8
Installing the Server into a Telco Rack...........................................................2-9
SUPERSERVER 1017GR-TF User's Manual

vii
Table of Contents
Chapter 3 System Interface
3-1 Overview .........................................................................................................3-1
3-2 Control Panel Buttons..................................................................................... 3-1
Reset...............................................................................................................3-1
Power..............................................................................................................3-1
3-3 Control Panel LEDs ........................................................................................3-2
Overheat/Fan Fail/UID LED............................................................................ 3-2
NIC2................................................................................................................3-2
NIC1................................................................................................................3-2
HDD................................................................................................................. 3-3
Power..............................................................................................................3-3
3-4 Drive Carrier LEDs.......................................................................................... 3-3
Chapter 4 System Safety
4-1 Electrical Safety Precautions..........................................................................4-1
4-2 General Safety Precautions............................................................................ 4-2
4-3 ESD Precautions.............................................................................................4-3
4-4 Operating Precautions .................................................................................... 4-4
Chapter 5 Advanced Motherboard Setup
5-1 Handling the Motherboard .............................................................................. 5-1
Precautions .....................................................................................................5-1
Unpacking .......................................................................................................5-1
5-2 Connecting Cables..........................................................................................5-2
Connecting Data Cables.................................................................................5-2
Connecting Power Cables .............................................................................. 5-2
Connecting the Control Panel......................................................................... 5-2
5-3 I/O Ports..........................................................................................................5-3
5-4 Installing the Processor and Heatsink ............................................................5-4
Installing an LGA 2011 Processor...................................................................5-4
Installing a CPU Heatsink...............................................................................5-7
Memory Support..............................................................................................5-8
Memory Population Guidelines....................................................................... 5-9
5-6 Expansion Cards...........................................................................................5-10
5-7 Motherboard Details.......................................................................................5-11
X9SRG-F Quick Reference........................................................................... 5-12
5-8 Connector Definitions ...................................................................................5-14
5-9 Jumper Settings............................................................................................ 5-20
5-10 Onboard Indicators........................................................................................5-22
5-11 SATA Ports....................................................................................................5-23

viii
5-12 Installing Software.........................................................................................5-24
Supero Doctor III...........................................................................................5-25
Chapter 6 Advanced Chassis Setup
6-1 Static-Sensitive Devices..................................................................................6-1
Precautions .....................................................................................................6-1
6-2 Control Panel .................................................................................................. 6-2
6-3 Removing the Chassis Cover .........................................................................6-2
6-4 System Cooling............................................................................................... 6-3
Fan Numbering in BIOS ................................................................................6-3
System Fan Failure......................................................................................... 6-3
Maintaining System Fan Trays........................................................................6-4
Installing the Air Shroud..................................................................................6-6
Installing Graphics (GPU) Cards.....................................................................6-7
6-4 Drive Bay Installation/Removal....................................................................... 6-8
Accessing the Drive Bays...............................................................................6-8
Hard Drive Installation.....................................................................................6-8
6-5 Power Supply................................................................................................ 6-10
Chapter 7 BIOS
7-1 Introduction...................................................................................................... 7-1
Starting BIOS Setup Utility..............................................................................7-1
How To Change the Configuration Data......................................................... 7-1
How to Start the Setup Utility ......................................................................... 7-2
7-2 Main Setup...................................................................................................... 7-2
7-3 Advanced Setup Configurations......................................................................7-4
7-4 Event Logs....................................................................................................7-19
7-5 IPMI Settings.................................................................................................7-21
7-6 Boot Settings.................................................................................................7-23
7-7 Security Settings........................................................................................... 7-24
7-8 Save & Exit ................................................................................................... 7-25
Appendix A BIOS POST Error Codes
Appendix B System Specifications
SUPERSERVER 1017GR-TF User's Manual

Chapter 1
Introduction
1-1 Overview
The SuperServer 1017GR-TF series is a GPU-optimized server comprised of two
main subsystems: the SC118G-1K43BP 1U server chassis and the X9SRG-F
serverboard. Please refer to our web site for information on operating systems that
have been certified for use with the system (www.supermicro.com).
• Four 4-cm counter-rotating fans (FAN-0117L4)
• Four 4-cm PWM fans (FAN-0102L4)
• One air shroud (MCP-310-81802-0B)
• One passive CPU heatsink (SNK-P0047PS)
• Riser Cards
One RSC-R1U-E16R for PCI-Express 3.0 x16 card. right side
One RSC-R1UG-E16-UP for UIO card
One RSC-R1UG-E16R-UP for UP GPU, right side
• Two power cables for GPU cards (CBL-0333L)
• SATA Accessories
One SAS backplane (BPN-SAS-118TQ)
Six 2.5" drive carriers (MCP-220-00047-0B)
SATA cables (2 pcs. of CBL-0226L, CBL-0228L, CBL-0230L)
• One rail set (MCP-290-00054-0N)
• One Super Server 1017GR-TF User's Manual
Chapter 1: Introduction
1-1

1-2
SUPERSERVER 1017GR-TF User's Manual
1-2 Serverboard Features
At the heart of the SuperServer 1017GR-TF server is the X9SRG-F, a single pro-
cessor serverboard based on the Intel C602 chipset. Below are the main features
of the X9SRG-F. (See Figure 1-1 for a block diagram of the chipset).
Processors
The X9SRG-F supports a single Intel Xeon E5-2600 series processors in LGA 2011
sockets (Socket R). Please refer to the serverboard description pages on our web
site for a complete listing of supported processors (www.supermicro.com).
Memory
The X9SRG-F has eight DIMM slots that can support up to 256 GB of unbuffered
ECC DDR3-1600/1333/1066/800 RDIMMs or 64 GB of DDR3, unbuffered non-ECC
UDIMMs up to 1600 MHz. Modules of the same size and speed are recommended.
See Chapter 5 for details.
Serial ATA
ASATAcontroller is integrated into the chipset to provide a 10-port SATAsubsystem,
which is RAID 0, 1, 5 and 10 supported. Two of these are SATA 3.0 ports and eight
are SATA 2.0 ports. The SATA drives are hot-swappable units.
Note: The operating system you use must have RAID support to enable the hot-
swap capability and RAID function of the SATA drives.
PCI Expansion Slots
The X9SRG-F has two PCI-Express 3.0 x16 slots to support three double-width
GPU cards. An additional slot supports one PCI-Express 3.0 x8 low-profile card.
I/O Ports
The color-coded I/O ports include one COM port, a VGA (monitor) port, two USB
2.0 ports, two Gb Ethernet LAN ports and one dedicated IPMI LAN port.
IPMI
IPMI (Intelligent Platform Management Interface) is a hardware-level interface speci-
fication that provides remote access, monitoring and administration for Supermicro
server platforms. IPMI allows server administrators to view a server’s hardware

1-3
Chapter 1: Introduction
status remotely, receive an alarm automatically if a failure occurs, and power cycle
a system that is non-responsive.
1-3 Server Chassis Features
System Power
The SC118G-1K43BP features a high-efficiency 1400W power supply. Power must
be removed from the system and the AC power cord removed when replacing the
power supply. See Chapter 6 for details.
SATA Subsystem
The SC118G-1K43BP chassis includes six 2.5" drive bays, which may be used to
house hot-swappable SATA drives.
Front Control Panel
The control panel provides a system monitoring and control interface. LEDs indicate
system power, HDD activity, network activity, and a system overheat/fan fail/ UID
LED. A main power button and a system reset button are also included.
Cooling System
The SC118G-1K43BP has an innovative cooling design that includes four sets of
back-to-back 4-cm counter-rotating PWM (Pulse Width Modulated) fans. The power
supply modules also include a cooling fan. All chassis and power supply fans oper-
ate continuously. An air shrouds is included to further help cool the GPUs.

1-4
SUPERSERVER 1017GR-TF User's Manual
1-4 GPU Subsystem
The 1017GR-TF server represents one of Supermicro's massively parallel process-
ing multiple-GPU servers, with support for up to two GPUs. NVIDIA® Fermi™ GPUs
place this system at the forefront of today's GPU computing solutions.
Please refer to the NVIDIA web site (www.nvidia.com) for details on Fermi GPUs.
Two GPUs can be bundled with the system. See chapter 6 for installation proce-
dures.

1-5
Chapter 1: Introduction
DDR3 DIMM
PROCESSOR
Socket 2
CPU REAR
E5-2600
PE3
SLOT 5
SLOT 6
SLOT 7
SLOT 9
SLOT 10
SLOT 1
SLOT 2
SLOT 3
SLOT 4
SLOT 8
P0 P1
PE2 PE1 DMI
PCIE 3.0x8
PCIE 2.0x8 Slot 11
P00_P11
Lane Reversal
#2
#1
DDR3 DIMM
#2
#1 F
E
DDR3 DIMM
#2
#1
DDR3 DIMM
#2
#1
H
G
DDR3 DIMM
PROCESSOR
SSB
Socket 1
CPU FRONT
PE3
P1 P0
PE2 PE1 DMI
DMI
PCI
Windbond
VGA
VGA CONN
DDR2 RAM
TPM Header
BMC
PHY Super I/O HW Monitor
NCT7904D
W83527
IPMI LAN
RJ45
RTL8201F
USB [10,11] LPC
S-SATA0 S-SATA1 S-SATA2 S-SATA3
JLAN1
RJ45 JLAN2 RJ45
Dual GbE
I350AM2 PEG [0..3] SATA [0..5]
USB [0..9]
SPI
BIOS
SATA
SATA Gen3 6GB/s
3GB/s
SATA Gen2
Gen2
3GB/s
SPI Flash
#2
#1
DDR3 DIMM
#2
#1 D
C
DDR3 DIMM
QPI QPI
#2
#1
DDR3 DIMM
#2
#1
B
A
PCIE 3.0x8
S-SATA3 S-SATA4
S-SATA5 S-SATA2 S-SATA1 S-SATA0
REAR
HDR 2X5
ports 0,1 port 4 port 5
REAR TYPE A TYPE A
2,3
6,7 8,9
HDR 2X5
SATA Gen3 [0..3]
E5-2600
(x4)
Intel C602
DMI: LANE
Reversal
Figure 1-1. Intel C602 Chipset:
System Block Diagram
Note: This is a general block diagram. Please see Chapter 5 for details.

1-6
SUPERSERVER 1017GR-TF User's Manual
1-5 Contacting Supermicro
Headquarters
Address: Super Micro Computer, Inc.
980 Rock Ave.
San Jose, CA 95131 U.S.A.
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
Web Site: www.supermicro.com
Europe
Address: Super Micro Computer B.V.
Het Sterrenbeeld 28, 5215 ML
's-Hertogenbosch, The Netherlands
Tel: [phone]
Fax: [phone]
Email: [email protected] (General Information)
[email protected] (Technical Support)
[email protected] (Customer Support)
Asia-Pacific
Address: Super Micro, Taiwan
4F, No. 232-1, Liancheng Rd.
Chung-Ho 235, Taipei County
Taiwan, R.O.C.
Tel: [phone]
Fax: [phone]
Web Site: www.supermicro.com.tw
Technical Support:
Email: [email protected]
Tel: [phone]

Chapter 2: Server Installation
2-1
Chapter 2
Server Installation
2-1 Overview
This chapter provides a quick setup checklist to get your system up and running.
Following these steps in the order given should enable you to have the system
operational within a minimum amount of time. This quick setup assumes that your
system has come to you with the processors and memory preinstalled. If your sys-
tem is not already fully integrated with a serverboard, processors, system memory
etc., please turn to the chapter or section noted in each step for details on installing
specific components.
2-2 Unpacking the System
You should inspect the box the system was shipped in and note if it was damaged
in any way. If the server itself shows damage you should file a damage claim with
the carrier who delivered it.
Decide on a suitable location for the rack unit that will hold the server. It should
be situated in a clean, dust-free area that is well ventilated. Avoid areas where
heat, electrical noise and electromagnetic fields are generated. You will also need
it placed near a grounded power outlet. Read the Rack and Server Precautions in
the next section.
2-3 Preparing for Setup
The box the server was shipped in should include two sets of rail assemblies, two
rail mounting brackets and the mounting screws you will need to install the system
into the rack. Follow the steps in the order given to complete the installation process
in a minimum amount of time. Please read this section in its entirety before you
begin the installation procedure outlined in the sections that follow.
Choosing a Setup Location
• Leave enough clearance in front of the rack to enable you to open the front door
completely (~25 inches) and approximately 30 inches of clearance in the back
of the rack to allow for sufficient airflow and ease in servicing.This product is for
installation only in a Restricted Access Location (dedicated equipment rooms,
service closets and the like).

2-2
SUPERSERVER 1017GR-TF User's Manual
!
!
Warnings and Precautions!
• This product is not suitable for use with visual display work place devices
acccording to §2 of the the German Ordinance for Work with Visual Display
Units.
Rack Precautions
• Ensure that the leveling jacks on the bottom of the rack are fully extended to
the floor with the full weight of the rack resting on them.
• In single rack installation, stabilizers should be attached to the rack. In multiple
rack installations, the racks should be coupled together.
• Always make sure the rack is stable before extending a component from the
rack.
• You should extend only one component at a time - extending two or more si-
multaneously may cause the rack to become unstable.
• Rack-mounted equipment should not be used as a shelf or work space.
Server Precautions
• Review the electrical and general safety precautions in Chapter 4.
• Determine the placement of each component in the rack before you install the
rails.
• Install the heaviest server components on the bottom of the rack first, and then
work up.
• Use a regulating uninterruptible power supply (UPS) to protect the server from
power surges, voltage spikes and to keep your system operating in case of a
power failure.
• Allow the hot plug SATA drives and power supply modules to cool before touch-
ing them.

Chapter 2: Server Installation
2-3
• Always keep the rack's front door and all panels and components on the servers
closed when not servicing to maintain proper cooling.
Rack Mounting Considerations
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating tempera-
ture of the rack environment may be greater than the ambient temperature of the
room. Therefore, consideration should be given to installing the equipment in an
environment compatible with the manufacturer’s maximum rated ambient tempera-
ture (Tmra).
Reduced Airflow
Equipment should be mounted into a rack so that the amount of airflow required
for safe operation is not compromised.
Mechanical Loading
Equipment should be mounted into a rack so that a hazardous condition does not
arise due to uneven mechanical loading.
Circuit Overloading
Consideration should be given to the connection of the equipment to the power
supply circuitry and the effect that any possible overloading of circuits might have
on overcurrent protection and power supply wiring. Appropriate consideration of
equipment nameplate ratings should be used when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack itself
should be grounded. Particular attention should be given to power supply connec-
tions other than the direct connections to the branch circuit (i.e. the use of power
strips, etc.).

2-4
SUPERSERVER 1017GR-TF User's Manual
2-4 Installing the System into a Rack
This section provides information on installing the SC118GQ chassis into a rack
unit with the rails provided. There are a variety of rack units on the market, which
may mean that the assembly procedure will differ slightly. You should also refer to
the installation instructions that came with the rack unit you are using.
Note: This rail will fit a rack between 26" and 33.5" deep.
Identifying the Sections of the Rack Rails
The chassis package includes two rack rail assemblies in the rack mounting kit.
Each assembly consists of two sections: an inner fixed chassis rail that secures
directly to the server chassis and an outer fixed rack rail that secures directly to
the rack itself.
Figure 2-1. Identifying the Sections of the Rack Rails
Inner Rail
Extension:
attach to the
chassis
Outer Rails:
slide together, then
attach to the front
and rear brackets
Inner Rail (preattached
to the chassis)
Front and Rear
Brackets: attach to
the rack

Chapter 2: Server Installation
2-5
Figure 2-2. Installing the Inner Rails
1
1
1
21
3
Installing the Inner Rail Extensions
The SC118GQ chassis includes a set of inner rack rails in two sections: inner rails
(A) and inner rail extensions (B). The inner rails are preattached and do not interfere
with normal use of the chassis if you decide not to install to a server rack. Attaching
the inner rail extensions to to the inner rails stabilizes the chassis within the rack.
Installing the Inner Rail Extensions
1. Place the inner rail extensions (B) over the preattached inner rails (A) which
are attached to the side of the chassis. Align the hooks of the inner rail with
the rail extension holes. Make sure the extension faces "outward" just like the
inner rail.
2. Slide the extension toward the front of the chassis.
3. Secure the chassis with screws as illustrated.
4. Repeat steps 1-3 for the other inner rail extension.

2-6
SUPERSERVER 1017GR-TF User's Manual
Figure 2-3. Assembling the Outer Rails
Assembling the Outer Rails
Each outer rail is in two sections that must be assembled before mounting on to
the rack.
Assembling the Outer Rails
1. Identify the left and right outer rails by examining the ends, which bend
outward.
2. Slide the front section of the outer rail (A), into the rear section of the outer
rail (B).
Secure to the
front of the rack
Secure to the
rear of the rack
Slide outer rails
together
Outer rail assembled
Assembling the sections of
the outer rail
1
A
1
B