Silicon Graphics Rackable C2110G-RP5 User manual

SGI®Rackable™C2110G-RP5
System User Guide
007-5841-001

COPYRIGHT
© 2012 SGI. All rights reserved; provided portionsmay be copyright in third parties, as indicated elsewhere herein. No permission is granted to copy, distribute,
or create derivative works from the contents of this electronic documentation in any manner, in whole or in part, without the prior written permission of SGI.
LIMITED RIGHTS LEGEND
The software described in this document is“commercial computer software” provided with restricted rights (except as to included open/free source) as specified
in the FAR 52.227-19 and/or the DFAR 227.7202, or successive sections. Use beyond license provisions is a violation of worldwide intellectual property laws,
treaties and conventions. This document is provided with limited rights as defined in 52.227-14.
The electronic (software) version of this document was developed at private expense; if acquired under an agreement with the USA government or any
contractor thereto, it is acquired as “commercial computer software” subject to the provisions of its applicable license agreement, as specified in (a) 48 CFR
12.212 of the FAR; or, if acquired for Department of Defense units, (b) 48 CFR 227-7202 of the DoD FAR Supplement; or sections succeeding thereto.
Contractor/manufacturer is SGI, 46600 Landing Parkway, Fremont, CA 94538.
TRADEMARKS AND ATTRIBUTIONS
SGI, and the SGI logo are registered trademarks and Rackable is a trademark of Silicon Graphics International in the United States and/or other countries
worldwide.
Intel, Intel QuickPath Interconnect (QPI) and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and
other countries.
Fusion-MPT, Integrated RAID, MegaRAID, and LSI Logic are trademarks or registered trademarks of LSI Logic Corporation.
HyperTransport is a licensed trademark of the HyperTransport Technology Consortium.
InfiniBand is a registered trademark of the InfiniBand Trade Association.
Internet Explorer and MS-DOS are registered trademarks of Microsoft Corporation.
Java and Java Virtual Machine are trademarks or registered trademarks of Sun Microsystems, Inc.
Linux is a registered trademark of Linus Torvalds, used with permission by SGI.
Novell and Novell Netware are registered trademarks of Novell Inc.
PCIe and PCI-X are registered trademarks of PCI SIG.
Phoenix and PhoenixBIOS are registered trademarks of Phoenix Technologies Ltd.
Red Hat and all Red Hat-based trademarks are trademarks or registered trademarks of Red Hat, Inc. in the United States and other countries.
SUSE LINUX and the SUSE logo are registered trademarks of Novell, Inc.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.
All other trademarks mentioned herein are the property of their respective owners.

007-5841-001 iii
Record of Revision
Version Description
001 May, 2012
First release

007-5841-001 v
About This Guide
This guide provides an overview of the installation, architecture, general operation, and
descriptions of the major components in the SGI®Rackable™C2110G-RP5 server. It also
provides basic troubleshooting and maintenance information, BIOS information, and important
safety and regulatory specifications.
Audience
This guide is written for users of SGI Rackable C2110G-RP5 server systems. It is written with the
assumption that the reader has a good working knowledge of computers and computer systems.
This guide may be useful to installers and system administrators looking for overview information
on the server.
Chapter Descriptions
The following topics are covered in this guide:
• Chapter 1, “Introduction”
Provides an overview of the server’s components.
• Chapter 2, “Server Installation”
Provides a quick setup checklist to get the server operational.
• Chapter 3, “System Interface”
Describes several LEDs on the control panel as well as others on the SATA drive carriers
that keep you constantly informed of the overall status of the system as well as the activity
and health of specific components.
• Chapter 4, “System Safety”
Provides general system safety information.
• Chapter 5, “System Severboard Information”

vi 007-5841-001
Provides best practice procedures to work with a node board in the C2110G-RP5 chassis,
install memory DIMMs, PCIe expansion cards and hard disk drives.
• Chapter 6, “Basic Troubleshooting and Chassis Service”
Describes some basic steps required to troubleshoot your system. Additional sections in this
chapter are intended to guide you through basic component remove and replace procedures.
•Appendix A, “BIOS Error Codes,”
Provides a brief listing of BIOS error code information.
• Appendix B, “System Specifications,”
Describes system component environmental specifications and compliance.

:
007-5841-001 vii
Related Publications
The following SGI and LSI documents may be relevant to the use of your server:
•MegaRAID SAS Software User’s Guide, publication number, 860-0488-00x
•SGI Performance Suite series documentation
•SGI InfiniteStorage series documentation
• Man pages (online)
You can obtain SGI documentation (as well as the pertinent LSI books), release notes, or man
pages in the following ways:
• Refer to the SGI Technical Publications Library at http://docs.sgi.com. Various formats are
available. This library contains the most recent and most comprehensive set of online books,
release notes, man pages, and other information.
• You can also view man pages by typing man <title> on a command line.
SGI systems include a set of Linux®man pages, formatted in the standard UNIX®“man page”
style. Important system configuration files and commands are documented on man pages. These
are found online on the internal system disk (or DVD-CD) and are displayed using the man
command. For additional information about displaying man pages using the man command, see
man(1).
In addition, the apropos command locates man pages based on keywords. For example, to
display a list of man pages that describe disks, type the following on a command line:
apropos disk
For information about setting up and using apropos, see apropos(1).

viii 007-5841-001
Conventions
The following conventions are used throughout this document:
Product Support
SGI provides a comprehensive product support and maintenance program for its products. SGI
also offers services to implement and integrate Linux applications in your environment.
• Refer to http://www.sgi.com/support/
• If you are in North America, contact the Technical Assistance Center at
[phone]SGI or contact your authorized service provider.
• If you are outside North America, contact the SGI subsidiary or authorized distributor in
your country.
Convention Meaning
Command This fixed-space font denotes literal items such as commands, files,
routines, path names, signals, messages, and programming language
structures.
variable The italic typeface denotes variable entries and words or concepts being
defined. Italic typeface is also used for book titles.
user input This bold fixed-space font denotes literal items that the user enters in
interactive sessions. Output is shown in nonbold, fixed-space font.
[ ] Brackets enclose optional portions of a command or directive line.
... Ellipses indicate that a preceding element can be repeated.
man page(x) Man page section identifiers appear in parentheses after man page names.
GUI element This font denotes the names of graphical user interface (GUI) elements such
as windows, screens, dialog boxes, menus, toolbars, icons, buttons, boxes,
fields, and lists.

:
007-5841-001 ix
Reader Comments
If you have comments about the technical accuracy, content, or organization of this document,
contact SGI. Be sure to include the title and document number of the manual with your comments.
(Online, the document number is located in the front matter of the manual. In printed manuals, the
document number is located at the bottom of each page.)
You can contact SGI in any of the following ways:
• Send e-mail to the following address: techpu[email protected]
• Contact your customer service representative and ask that an incident be filed in the SGI
incident tracking system.
• Send mail to the following address:
SGI
Technical Publications
46600 Landing Parkway
Fremont, CA 94538
SGI values your comments and will respond to them promptly.

007-5841-001 xi
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 1
Server Board Features . . . . . . . . . . . . . . . . . . . . . . 3
Processors . . . . . . . . . . . . . . . . . . . . . . . . 3
QPI Interconnect . . . . . . . . . . . . . . . . . . . . . . 3
Memory. . . . . . . . . . . . . . . . . . . . . . . . . 3
Serial ATA . . . . . . . . . . . . . . . . . . . . . . . . 3
PCI Express Expansion Slots . . . . . . . . . . . . . . . . . . . 4
Onboard Controllers/Ports . . . . . . . . . . . . . . . . . . . . 4
Onboard Graphics Controller . . . . . . . . . . . . . . . . . . . 4
IPMI . . . . . . . . . . . . . . . . . . . . . . . . . 4
Other Features . . . . . . . . . . . . . . . . . . . . . . . 4
Server Chassis Features . . . . . . . . . . . . . . . . . . . . . . 5
System Power . . . . . . . . . . . . . . . . . . . . . . . 5
Serial ATA Subsystems . . . . . . . . . . . . . . . . . . . . 5
Front Control Panel. . . . . . . . . . . . . . . . . . . . . . 5
Serverboard and GPU Subsystem . . . . . . . . . . . . . . . . . . 5
GPU Features . . . . . . . . . . . . . . . . . . . . . . 6
Cooling System . . . . . . . . . . . . . . . . . . . . . . . 6
2 Server Installation . . . . . . . . . . . . . . . . . . . . . . . 9
Unpack the System . . . . . . . . . . . . . . . . . . . . . . . 9
Prepare for Setup . . . . . . . . . . . . . . . . . . . . . . 9
Choose a Setup Location . . . . . . . . . . . . . . . . . . . . 9
System Warnings and Precautions . . . . . . . . . . . . . . . . . . . 10
Server Precautions . . . . . . . . . . . . . . . . . . . . . . 11
Rack Mounting Considerations . . . . . . . . . . . . . . . . . . . . 11
Ambient Operating Temperature . . . . . . . . . . . . . . . . . . 11
Reduced Airflow . . . . . . . . . . . . . . . . . . . . . . 11

xii 007-5841-001
Contents
Mechanical Loading. . . . . . . . . . . . . . . . . . . . . . 11
Circuit Overloading . . . . . . . . . . . . . . . . . . . . . . 12
Reliable Ground . . . . . . . . . . . . . . . . . . . . . . . 12
Install the System into a Rack . . . . . . . . . . . . . . . . . . . . 12
Separate the Sections of the Rack Rails. . . . . . . . . . . . . . . . . 12
Attaching the Outer Rack Rails . . . . . . . . . . . . . . . . . . . 14
The Inner Rail Extension . . . . . . . . . . . . . . . . . . . . 15
Using the Rail Locking Tabs . . . . . . . . . . . . . . . . . . 15
Install the Server in a Rack . . . . . . . . . . . . . . . . . . . . 16
Supply Power to the System . . . . . . . . . . . . . . . . . . 18
3 System Interface. . . . . . . . . . . . . . . . . . . . . . . . 19
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Control Panel Buttons . . . . . . . . . . . . . . . . . . . . . . 20
Control Panel LEDs . . . . . . . . . . . . . . . . . . . . . . . 21
Power Fail LED . . . . . . . . . . . . . . . . . . . . . . . 21
Overheat/Fan Fail/UID LED . . . . . . . . . . . . . . . . . . . 21
NIC1. . . . . . . . . . . . . . . . . . . . . . . . . . 22
NIC2. . . . . . . . . . . . . . . . . . . . . . . . . . 22
HDD . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Power . . . . . . . . . . . . . . . . . . . . . . . . . 23
Drive Carrier LEDs . . . . . . . . . . . . . . . . . . . . . . . 23
4 System Safety . . . . . . . . . . . . . . . . . . . . . . . . 25
Electrical Safety Precautions. . . . . . . . . . . . . . . . . . . . . 25
Serverboard Battery . . . . . . . . . . . . . . . . . . . . . . 26
ESD Precautions. . . . . . . . . . . . . . . . . . . . . . . 26
Mainboard Replaceable Soldered-in Fuses . . . . . . . . . . . . . . . . 27
General Safety Precautions . . . . . . . . . . . . . . . . . . . . . 27
5 System and Serverboard Information. . . . . . . . . . . . . . . . . . 29
Handling Circuit Boards and Drives . . . . . . . . . . . . . . . . . . . 29
ESD Precautions. . . . . . . . . . . . . . . . . . . . . . . 30
Unpacking . . . . . . . . . . . . . . . . . . . . . . . . 30
System Rear I/O Ports . . . . . . . . . . . . . . . . . . . . . . 31

Contents
007-5841-001 xiii
Serverboard Details. . . . . . . . . . . . . . . . . . . . . . 31
CPUs . . . . . . . . . . . . . . . . . . . . . . . . 32
Memory . . . . . . . . . . . . . . . . . . . . . . . 32
GPUs . . . . . . . . . . . . . . . . . . . . . . . . 32
PCIe Expansion Slots . . . . . . . . . . . . . . . . . . . . 32
System Health Monitoring. . . . . . . . . . . . . . . . . . . 32
ACPI Features . . . . . . . . . . . . . . . . . . . . . . 33
Onboard I/O . . . . . . . . . . . . . . . . . . . . . . 33
Serverboard Dimensions . . . . . . . . . . . . . . . . . . . 33
Hard Disk Drives (C2110G-RP5 Chassis) . . . . . . . . . . . . . . . . . 35
Drive Configurations . . . . . . . . . . . . . . . . . . . . . 35
PCIe Expansion Cards . . . . . . . . . . . . . . . . . . . . . . 37
Power Supply Functional Rating . . . . . . . . . . . . . . . . . . 37
6 Basic Troubleshooting and Chassis Service . . . . . . . . . . . . . . . . 39
Basic Troubleshooting Procedures . . . . . . . . . . . . . . . . . . . 39
If the System Does Not Power Up . . . . . . . . . . . . . . . . . . 39
System Powers Up But Will Not Boot . . . . . . . . . . . . . . . . . 40
No Video After System Power Up . . . . . . . . . . . . . . . . . . 40
Memory Errors . . . . . . . . . . . . . . . . . . . . . . . 40
Chassis Service Information. . . . . . . . . . . . . . . . . . . . . 41
Static-Sensitive Devices . . . . . . . . . . . . . . . . . . . . 41
Precautions . . . . . . . . . . . . . . . . . . . . . . . . 41
Unpacking . . . . . . . . . . . . . . . . . . . . . . . . 42
Control Panel . . . . . . . . . . . . . . . . . . . . . . . . 42
Drive Bay Installation/Removal. . . . . . . . . . . . . . . . . . . . 42
Accessing the Drive Bays . . . . . . . . . . . . . . . . . . . . 42
Removing Hard Drives or Carriers from the Chassis . . . . . . . . . . . . . 43
The Hard Drive Backplane . . . . . . . . . . . . . . . . . . 43
Disk Drive Installation . . . . . . . . . . . . . . . . . . . . . 43
Hard Drive Carrier Assembly Usage . . . . . . . . . . . . . . . . . 44
Power Supply . . . . . . . . . . . . . . . . . . . . . . . . 47
Power Supply Failure . . . . . . . . . . . . . . . . . . . . . 48
Removing/Replacing a Power Supply . . . . . . . . . . . . . . . . . 48

xiv 007-5841-001
Contents
Removing the Power Supply . . . . . . . . . . . . . . . . . . 48
Installing a New Power Supply. . . . . . . . . . . . . . . . . . 48
Accessing the Inside of the Chassis . . . . . . . . . . . . . . . . . . . 50
System Fans . . . . . . . . . . . . . . . . . . . . . . . . . 50
System Fan Failure . . . . . . . . . . . . . . . . . . . . . . 50
Replacing System Fans . . . . . . . . . . . . . . . . . . . . . 50
Remove/Replace a Fan . . . . . . . . . . . . . . . . . . . . 51
Mid-chassis Fans: . . . . . . . . . . . . . . . . . . . . . 53
Front and GPU Fans: . . . . . . . . . . . . . . . . . . . . 54
Install/Replace a PCIe Expansion Card . . . . . . . . . . . . . . . . . . 56
Install/Replace a Full-height PCIe Card. . . . . . . . . . . . . . . . . 56
Install/Replace a Low-profile PCIe Card . . . . . . . . . . . . . . . . 59
A BIOS Error Codes . . . . . . . . . . . . . . . . . . . . . . . 61
B System Operating and Regulatory Overview . . . . . . . . . . . . . . . . 63
Operating Environment . . . . . . . . . . . . . . . . . . . . . . 63
System Input Requirements . . . . . . . . . . . . . . . . . . . . . 63
Power Supply. . . . . . . . . . . . . . . . . . . . . . . . . 64
Regulatory Compliance . . . . . . . . . . . . . . . . . . . . . . 64

007-5841-001 1
Chapter 1
1. Introduction
Important: SGI Rackable server systems may sometimes require driver versions that are not
included in the original operating system release. When required, SGI provides these drivers on
an SGI Driver CD, which may ship with the system, or on the system disk (pre-installed in the
factory). For more information on this topic check with your sales or service representative.
The Rackable C2110G-RP5 server is a 2U rackmount system (see Figure 1-1 on page 2).
In addition to the serverboard and chassis, various hardware components may be included with
the system, as listed:
• Five 8-cm chassis fans
• One internal air shroud
• One passive 1U CPU heatsink and one passive 2U CPU heatsink
• Riser cards as follows:
– One riser for two PCIe x16 cards, (left-front side internal GPU cards)
– One riser for two PCIe x16 cards, (right-front side internal GPU cards)
– One riser for one PCIe x4 card, (rear)
– One riser for one low-profile PCIe x8 card (rear)
• Four power cables for GPU cards
• SATA accessories:
– One SATA backplane
– Ten 2.5-in SATA drive carriers
Note: SAS drives are supported only in configurations using an optional PCIe RAID
controller board.
• Two power supplies
• One rackmount kit

2007-5841-001
1: Introduction
Note: The Rackable C2110G-RP5 server does not use an internal CD/DVD drive. Check with
your SGI sales or service representative for information on optional external CD/DVD drive units.
Figure 1-1 Rackable C2110G-RP5 Server Front and Rear Views
PCI expansion slots
!
2 1
S
ystem
reset
S
ystem
LEDs
Main
power
Ten disk drive bays
VGA
port
Ethernet
ports
USB
ports
IPMI
LAN

Server Board Features
007-5841-001 3
Server Board Features
At the heart of the system is a dual-processor serverboard based on the Intel C602 chipset and
designed to provide maximum performance. The main features of the serverboard are described
in the following subsections.
Processors
The serverboard supports two four, six or eight-core, Intel Xeon E5-2600 series processors. Each
processorsits in anLGA 2011socket and is interconnected viaIntelQuickPathInterconnect(QPI)
link support; see the next subsection for more information on the QPI interconnects.
QPI Interconnect
Separate QPI link pairs connect the two processors and the I/O hub in a network on the
motherboard, allowing all of the components to access other components via the network.
Each QPI comprises two 20-lane point-to-point datalinks,onein each direction (full duplex), with
a separate clock pair in each direction, for a total of 42 signals. Each signal is a differential pair,
so the total number of pins is 84. The 20 data lanes are divided onto four "quadrants" of 5 lanes
each. The basic unit of transfer is the 80-bit "flit", which is transferred in two clock cycles (four
20 bit transfers, two per clock.) The 80-bit "flit" has 8 bits for error detection, 8 bits for "link-layer
header," and 64 bits for "data". QPI bandwidths are advertised by computing the transfer of 64 bits
(8 bytes) of data every two clock cycles in each direction
Memory
The serverboard has eight DIMM slots (four per processor) that support DDR3
1600/1333/1066/800 MHz RDIMMs. Note that memory speed support is dependent on the type
of CPU used on the mother board. Up to 256 GBs of total memory is supported.
Serial ATA
A Serial ATA controller ASIC is integrated into the system serverboard to provide a 10-port
SATA disk subsystem. Drive ports 0 and 1 are SATA 3.0 ports and the remaining eight drive slots
are SATA 2.0 ports. The hot-swappable SATA drives are connected to a backplane that provides
power, bus termination and configuration settings. Optional RAID 0, 1, 5, 6 and 10 are supported.
Note that your operating system must have RAID support enabled to accommodate hot swapping

4007-5841-001
1: Introduction
of disk drives. Certain RAID configurations require use of optional hardware. Check with your
sales or service representative to obtain information on set-up procedures if your system did not
come pre-configured with SATA RAID.
PCI Express Expansion Slots
The dual processor serverboard has four PCIe 3.0 x16 slots to support four internal double-width
GPU cards. Additional slots support one PCIe 3.0 x8 low-profile card and one PCIe 2.0 x4
standard card.
See the section “PCIe Expansion Cards” in Chapter 5 for more information on these topics.
Onboard Controllers/Ports
The color-coded I/O ports include a VGA (monitor) port, two external USB 2.0 ports (an internal
COM header is located on the serverboard) and two gigabit LAN Ethernet ports. A dedicated
external IPMI LAN port is also included.
Onboard Graphics Controller
The dual-processor serverboard features an integrated Matrox G200 video controller providing a
16MB DDR2 graphics interface through the system VGA connector. The Matrox video controller
in the 2U server features low power consumption, high reliability and superior longevity.
IPMI
IPMI (Intelligent Platform Management Interface) is a hardware-level interface specification that
provides remote access, monitoring and administration for your SGI Rackable C2110G-RP5
server platforms. IPMI allows server administrators to view a server’s hardware status remotely,
receive an alarm automatically if a failure occurs, and power cycle a system that is non-responsive.
Other Features
Other onboard features that promote system health include onboard voltage monitors, a chassis
intrusion header, auto-switching voltage regulators, chassis and CPU overheat sensors, virus
protection and BIOS rescue.

Server Chassis Features
007-5841-001 5
Server Chassis Features
The following subsections provide a general outline of the main features of the SGI Rackable
C2110G-RP5 server chassis.
System Power
The Rackable C2110G-RP5 2U server chassis features a redundant power supply composed of
two separate power modules. This power redundancy feature allows you to replace a failed power
supply without shutting down the system. Note that each power supply provides up to 900 Watts
of power to the system.
Serial ATA Subsystems
The server chassis supports up to ten 2.5-in SATA drives. Drives 0 and 1 are SATA 3.0
6-Gb/second slots, drives 2-9 are 3-Gb/second (SATA 2.0) disk drives. RAID drives are
hot-swappable units and are connected to a backplane that provides power and control. Note that
the operating system you have installed must support RAID to enable the hot-swap capability of
RAID drives. Certain RAID levels require use of optional hardware to support RAIDed hard disk
drives in the server.
Front Control Panel
The control panel on the C2110G-RP5 server provides you with system monitoring and control.
LEDs that indicate system power, HDD activity, network activity, system overheat and a system
overheat/fan-fail/ UID LED. A main power button and a system reset button are also included.
Serverboard and GPU Subsystem
The C2110G-RP5 server chassis is an ATX form factor chassis designed to be used in a 2U
rackmount configuration. The serverboard’s I/O backplane supports up to four standard size
(double-width) GPUs to enable high-quality GPU computing solutions. A 15-pin VGA port, two
USB 2.0 ports, two Gigabit LAN ports and a dedicated RJ-45 IPMI LAN port are also supported.
The GPUs process complex image calculations and then route the data out through the VGA port
on the serverboard. The GPUs, which come with a passive heatsink attached, have been tested for
use with this system.

6007-5841-001
1: Introduction
Important: Check with your SGI sales or service representative prior to using any GPU not
sourced at the SGI factory.
Any combination of these cards (up to a total of four) may have come bundled with the system.
Power for the GPU cards is provided via a GPU power cable from each of the GPUs to JPW3, 4,
6 and 7 on the serverboard (one cable for each card). Figure 1-2 on page 7 shows a general block
diagram of the C2110G-RP5 server’s processor and I/O chipset.
GPU Features
Each of the GPUs will feature some or all of the following:
• Hundreds of GPU cores in each card that can deliver more than 660 Gigaflops of
double-precision and over one Teraflop of single-precision calculations.
• ECC protected internal register files, L1/L2 caches, shared memory, and external DRAM.
• Up to 6 GB of GDDR5 memory per GPU enhances performance and reduces data transfers
by keeping larger data sets in local memory attached directly to the GPU.
• Integrates the GPU subsystem with the C2110G-RP5 server’s monitoring and management
capabilities such as IPMI.
• Onboard L1 and L2 caches that accelerate algorithms and sparse-matrix multiplication.
• Provides faster context switching, concurrent kernel execution and improved thread block
scheduling.
• Enhances overall system performance by transferring data over the PCIe bus while the
computing cores are processing other data.
• Provides a flexible programming environment with broad support for various programming
languages and APIs.
Cooling System
The 2U server chassis has a cooling design that includes five internal 8-cm heavy-duty system
cooling fans located in the chassis. An air shroud channels the airflow from the system fans to
efficiently cool the processor and GPU areas of the system. Each power supply module also
includes an internal cooling fan. All chassis and power supply fans operate continuously.