Silicon Graphics Altix XE340 User manual

SGI®Altix®XE340 User’s Guide
007-5536-001

COPYRIGHT
© 2009, Silicon Graphics, Inc. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No permission is granted
to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole or in part, without the prior written
permission of Silicon Graphics, Inc.
LIMITED RIGHTS LEGEND
The electronic (software) version of this document was developed at private expense; if acquired under an agreement with the USA government or any
contractor thereto, it is acquired as "commercial computer software" subject to the provisions of its applicable license agreement, as specified in (a) 48 CFR
12.212 of the FAR; or, if acquired for Department of Defense units, (b) 48 CFR 227-7202 of the DoD FAR Supplement; or sections succeeding thereto.
Contractor/manufacturer is Silicon Graphics, Inc., 46600 Landing Parkway, Freemont, CA 94538.
TRADEMARKS AND ATTRIBUTIONS
Silicon Graphics, Altix, SGI, and the SGI logo are registered trademarks of Silicon Graphics, Inc., in the United States and/or other countries worldwide.
Intel, Itanium and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.
Infiniband is a trademark of the InfiniBand Trade Association.
Linux is a registered trademark of Linus Torvalds.
Novell is a registered trademark of Novell Inc., in the United States and other countries.
Red Hat and Red Hat Enterprise LINUX are registered trademarks of Red Hat Inc.
All other trademarks mentioned herein are the property of their respective owners.

007-5536-001 iii
Record of Revision
Version Description
-001 July, 2009
First Release

007-5536-001 v
Contents
1. Introduction and Overview . . . . . . . . . . . . . . . . . . . . 1
ESD and Safety Precautions. . . . . . . . . . . . . . . . . . . . . 1
Serverboard Features . . . . . . . . . . . . . . . . . . . . . 2
Processors . . . . . . . . . . . . . . . . . . . . . . . . 3
Memory. . . . . . . . . . . . . . . . . . . . . . . . . 3
Serial ATA/SAS . . . . . . . . . . . . . . . . . . . . . . 3
PCI Expansion Slots . . . . . . . . . . . . . . . . . . . . . 3
Ethernet Ports . . . . . . . . . . . . . . . . . . . . . . . 3
On-board Controllers/Ports. . . . . . . . . . . . . . . . . . . . 4
Graphics Controller. . . . . . . . . . . . . . . . . . . . . . 4
Other Features . . . . . . . . . . . . . . . . . . . . . . . 4
InfiniBand . . . . . . . . . . . . . . . . . . . . . . . . 4
System Power . . . . . . . . . . . . . . . . . . . . . . . 6
Disk Drive Subsystem . . . . . . . . . . . . . . . . . . . . . 6
Control Panel . . . . . . . . . . . . . . . . . . . . . . . 6
Rear I/O Panel . . . . . . . . . . . . . . . . . . . . . . . 7
Cooling System . . . . . . . . . . . . . . . . . . . . . . . 7
1U Twin: System Notes . . . . . . . . . . . . . . . . . . . . . . 7
System Power . . . . . . . . . . . . . . . . . . . . . . . 7
System Backplane and Drives . . . . . . . . . . . . . . . . . . . 8

vi 007-5536-001
Contents
2. System Operation . . . . . . . . . . . . . . . . . . . . . . . 9
Unpacking the System and Choosing an Operating Location . . . . . . . . . . . . 9
Preparing for Setup . . . . . . . . . . . . . . . . . . . . . . 9
Choosing a Setup Location . . . . . . . . . . . . . . . . . . . 9
Rack Precautions . . . . . . . . . . . . . . . . . . . . . . 10
Server Precautions . . . . . . . . . . . . . . . . . . . . . . 10
Rack Mounting Considerations . . . . . . . . . . . . . . . . . . . 11
Ambient Operating Temperature . . . . . . . . . . . . . . . . . 11
Reduced Airflow . . . . . . . . . . . . . . . . . . . . . 11
Mechanical Loading . . . . . . . . . . . . . . . . . . . . 11
Circuit Overloading. . . . . . . . . . . . . . . . . . . . . 11
Reliable Ground . . . . . . . . . . . . . . . . . . . . . 11
Installing the System into a Rack . . . . . . . . . . . . . . . . . . . 12
Identifying the Sections of the Rack Rails . . . . . . . . . . . . . . . . 12
Installing the Inner Rails . . . . . . . . . . . . . . . . . . . . 13
Installing the Outer Rails . . . . . . . . . . . . . . . . . . . . 13
Locking Tabs . . . . . . . . . . . . . . . . . . . . . . . 14
Installing the Server into the Rack . . . . . . . . . . . . . . . . . . 14
Installing the Server into a Telco Rack . . . . . . . . . . . . . . . . . 14
Server Access After Rack Installation . . . . . . . . . . . . . . . . . 15
Checking the Components and Setup Before Power On . . . . . . . . . . . . 16
Checking the Hard Disk Drives . . . . . . . . . . . . . . . . . . . 16
Checking the Airflow . . . . . . . . . . . . . . . . . . . . . 16
Providing Power. . . . . . . . . . . . . . . . . . . . . . . 17

Contents
007-5536-001 vii
3. System Interfaces Overview . . . . . . . . . . . . . . . . . . . . 19
System Interface Overview . . . . . . . . . . . . . . . . . . . . . 19
Control Panel Buttons . . . . . . . . . . . . . . . . . . . . . . 19
Control Panel LEDs . . . . . . . . . . . . . . . . . . . . . . . 19
Overheat/Fan Fail . . . . . . . . . . . . . . . . . . . . . . 20
NICs 1 and 2 . . . . . . . . . . . . . . . . . . . . . . . 20
HDD . . . . . . . . . . . . . . . . . . . . . . . . . 21
Power . . . . . . . . . . . . . . . . . . . . . . . . . 21
Disk Drive Carrier LEDs . . . . . . . . . . . . . . . . . . . . 21
4. Internal Component Replacement and Upgrades . . . . . . . . . . . . . . 23
General Precautions . . . . . . . . . . . . . . . . . . . . . . . 23
Installing System DIMM Memory . . . . . . . . . . . . . . . . . . . 23
Installing Memory Modules . . . . . . . . . . . . . . . . . . . 24
Adding PCIe Cards . . . . . . . . . . . . . . . . . . . . . . . 25
System Fans. . . . . . . . . . . . . . . . . . . . . . . . 25
System Fan Failure . . . . . . . . . . . . . . . . . . . . . . 25
Drive Bay Installation/Removal . . . . . . . . . . . . . . . . . . 26
Disk Drive Removal and Installation . . . . . . . . . . . . . . . . . . 26
Mounting a Drive in a Carrier . . . . . . . . . . . . . . . . . . . 26
Installing/Removing Hot-swap Drives . . . . . . . . . . . . . . . . . 27
Power Supply . . . . . . . . . . . . . . . . . . . . . . . . 28
Power Supply Failure . . . . . . . . . . . . . . . . . . . . . 29
Removing the Power Supply . . . . . . . . . . . . . . . . . . . 29
Installing a New Power Supply. . . . . . . . . . . . . . . . . . . 30

viii 007-5536-001
Contents
5. Server BIOS Information . . . . . . . . . . . . . . . . . . . . . 31
Starting the BIOS Setup Utility . . . . . . . . . . . . . . . . . . . . 31
How To Change the Configuration Data. . . . . . . . . . . . . . . . . . 32
Starting the Setup Utility . . . . . . . . . . . . . . . . . . . . 32
Main Setup Screen . . . . . . . . . . . . . . . . . . . . . . 32
Advanced Setup Configurations. . . . . . . . . . . . . . . . . . . 34
BOOT Features . . . . . . . . . . . . . . . . . . . . . . 34
Power Configurations . . . . . . . . . . . . . . . . . . . . 35
Processor and Clock Options . . . . . . . . . . . . . . . . . . 35
Advanced Chipset Control . . . . . . . . . . . . . . . . . . . 38
CPU Bridge Configurations . . . . . . . . . . . . . . . . . . 38
North Bridge Configuration . . . . . . . . . . . . . . . . . . 40
SouthBridge Configuration. . . . . . . . . . . . . . . . . . . 41
Remote Access Configuration . . . . . . . . . . . . . . . . . . 45
Hardware Health Monitor . . . . . . . . . . . . . . . . . . . 46
IPMI Configuration. . . . . . . . . . . . . . . . . . . . . 49
SEL PEF Configuration . . . . . . . . . . . . . . . . . . . 52
The DMI Event Log . . . . . . . . . . . . . . . . . . . . 53
Security Settings . . . . . . . . . . . . . . . . . . . . . 53
Exit Options. . . . . . . . . . . . . . . . . . . . . . . 55
BIOS Error Beep Codes . . . . . . . . . . . . . . . . . . . . . . 56
BIOS Error Beep Code List . . . . . . . . . . . . . . . . . . . . 57
A. Technical Specifications . . . . . . . . . . . . . . . . . . . . . 59
Server Specifications and Features . . . . . . . . . . . . . . . . . . . 59
Environmental Specifications . . . . . . . . . . . . . . . . . . . . 60
B. SGI Altix XE System Drivers . . . . . . . . . . . . . . . . . . . . 61
The SGI Altix XE Driver CD and Linux Updates . . . . . . . . . . . . . . . 61

007-5536-001 1
Chapter 1
1. Introduction and Overview
This chapter provides an overview of your server’s main features.
Operating precautions are provided in this chapter, followed by a general overview of the product.
Before operating your system, familiarize yourself with the safety information in the following
section:
ESD and Safety Precautions
Caution: Observe all ESD precautions. Failure to do so can result in damage to the equipment.
Wear an SGI approved wrist strap when you handle an ESD-sensitive device to eliminate possible
ESD damage to equipment. Connect the wrist strap cord directly to earth ground.
Warning: Before operating or servicing any part of this product, read the safety
precautions.
Danger: Keep fingers and conductive tools away from high-voltage areas. Failure to
follow these precautions will result in serious injury or death. The high-voltage areas of the
system are indicated with high-voltage warning labels.
Caution: Power off the system only after the system software has been shut down in an orderly
manner. If you power off the system before you halt the operating system, data may be corrupted.
!

2007-5536-001
1: Introduction and Overview
Warning: If a lithium battery is installed in your system as a soldered part, only qualified
SGI service personnel should replace this lithium battery. For a battery of another type,
replace it only with the same type or an equivalent type recommended by the battery
manufacturer, or an explosion could occur. Discard used batteries according to the
manufacturer’s instructions.
Overview
The Altix XE340 is a "1U Twin" server comprised of a 1U chassis and two (twin) “motherboards”
known as serverboards. Check with your sales or service representative before loading any
operating system on your server not provided by the SGI factory or service organization.
In addition to the two serverboards and chassis, various hardware components may be included as
part of your Altix XE340 configuration as listed below:
• SATA Accessories: Four (4) hard disk drive carriers - a minimum of one disk drive (per
serverboard) is required for operation.One (1) internal SATA backplane per serverboard.
One (1) SATA cable set per serverboard. SAS hard drive options are also available.
Note: When the Altix XE340 is used in a “diskless” cluster environment the minimum
requirement of one disk drive per serverboard is not applicable.
• Two (2) PCI Express x16 riser cards (RSC-R1U-E16R) (one per serverboard).
• Rackmount hardware with screws (CSE-PT51L):
Two (2) rack rail assemblies and (6) brackets for mounting the rack rails in a rack/telco rack.
• One (1) CD containing drivers and utilities plus optional CDs depending on order
configuration.
• One1200-Watt chassis power supply is shared by both serverboards; if the power supply is
offline both serverboards will be non-operational.
Serverboard Features
At the heart of the Altix XE340 server lies two (twin) quad-core processor based serverboards,
which are based on Intel Xeon chipsets. The following subsections describe the main features of
the serverboards. Note that the features on each board are doubled for the server, which includes
two nodes. See Figure 1-1 on page 5 for a functional block diagram of a system serverboard.

ESD and Safety Precautions
007-5536-001 3
Processors
Each serverboard supports two Intel® Xeon quad-core processors in LGA1366 sockets (a total of
4 quad-core processors per chassis). The exact type of processors provided with your system
depends on the specific configuration you ordered. Check with your sales or service representative
for information on processor upgrades.
Memory
Each of the two serverboards has twelve 240-pin DIMM sockets that can support up to 96 GB of
registered ECC DDR3-1333/1066/800 SDRAM (192 GB total for the system). See Chapter 4 for
more details on installing memory into the system.
Serial ATA/SAS
The ICH10R on each serverboard includes a Serial ATA controller that supports up to six 3.0-Gb/s
SATA drive channels (note that the XE340 serverboard uses only two of the six channels). The
hot-swappable SATA drives are connected to a backplane that provides power, bus termination
and configuration settings. Optional SAS or SATA RAID 0, 1, and 10 are supported. Contact your
sales or service provider to order a specific RAID upgrade if your system was not ordered with
that option.
PCI Expansion Slots
Each serverboard within the system has one low-profile PCI Express 2.0 x16 slot, so two PCI
Express 2.0 x16 slots are provided in the server. Riser cards are pre-installed to support two
low-profile add-on PCIe cards (one per serverboard).
Ethernet Ports
An Intel® network controller is integrated into each of the serverboards to support two Gigabit
LAN ports (100/1000Base-T/1000BaseTX, RJ45 output).

4007-5536-001
1: Introduction and Overview
On-board Controllers/Ports
On-board I/O backpanel ports on each serverboard include one COM port, a VGA port, two USB
ports, a dedicated IPMI LAN port and two Gigabit LAN (NIC) ports. An (optional) InfiniBand
port is also included on specific models of the server. There are two sets of I/O ports included in
the server (one set for each serverboard).
Graphics Controller
The system features an integrated Matrox G200eW graphics chip, which includes 8 MB of DDR2
memory.
Other Features
Other on-board features that promote system health include voltage monitors, auto-switching
voltage regulators, chassis and CPU overheat sensors, virus protection and BIOS rescue.
InfiniBand
The Altix XE340 server is available with an (optional) InfiniBand port at DDR (double-data rate)
or QDR (quad-data rate) speeds, respectively. InfiniBand is a scalable serial communications link
intended for connecting processors with high-speed peripherals or other systems. Additional
PCIe-based InfiniBand interconnects are also available for your server. Check with your sales or
service representative for available options and upgrades.

ESD and Safety Precautions
007-5536-001 5
Figure 1-1 Functional Block Diagram of an XE340 System Server Board
CPU#1 CPU#2
A
A
A
B
DDR3 DIMM
B
B
DDR3 DIMM
A
A
A
DDR3 DIMM
B
B
B
DDR3 DIMM
33
11
22
1
1
22
3
3
Port1 Port0
Ports
Ports
3,4
7,8,9,10
PCI-E x16
ICH10R
Ports
2,1
Kawela
RJ45 RJ45
ESI
PE4-1
PE5
WBD
BMC/VGA
VGA
DDR II
PCI
SATA #1
SATA #2
SATA #3
SATA #4
SATA
LPC
SPI
SST25
VF016
CLINK
CLINK
DMI
MT25408
Connect-X IB
PCI-E Gen2/DDR or QDR
QSFP
RTL8201N PHY
Dedicate LAN
5,6
Ports
LPCIO W83527
ACPI
KBC
SATA #5
SATA #6
Intel
5520

6007-5536-001
1: Introduction and Overview
Server Chassis Features
The following sections provide a general outline of the main features of the 1U system chassis.
Figure 1-2 shows the rear panel of the SGI Altix XE340 server.
Figure 1-2 Altix XE340 Rear Panel and I/O Ports
System Power
Your Altix XE340 server uses a single high-efficiency 1200W cold-swap power supply, which
provides the power to both serverboards housed in the unit.
Disk Drive Subsystem
The Altix XE340 chassis was designed to support up to four SATA or SAS hard disk drives, which
can be optionally configured as hot-swappable units.
Note that the standard XE340 only supports SATA disk drives. However by adding a SAS HBA
the XE340 will support SAS disk drives. The SAS HBA will also support RAID 0 and 1 on both
SAS and SATA disk drives. One HBA is added to each node of the XE340 server.
Control Panel
The Altix XE340 features two independent control panels associated with each node
(serverboard). Each control panel has LEDs to indicate power on, network activity, hard disk drive
USB portsLAN ports COM2 port USB portsLAN ports VGA port COM2 port InfiniBand port
(optional connector)
Power supply
PCIe x16 slot PCIe x16 slotDedicated BMC port Dedicated BMC port
VGA port
InfiniBand port
(optional connector)

1U Twin: System Notes
007-5536-001 7
activity and system overheat conditions. Each control panel also includes a main power button and
a system reset button.
Rear I/O Panel
The system is a 1U rackmount chassis. Its I/O panel provides slots for two low-profile PCI-E x16
expansion cards, two COM ports, four USB ports, two VGA ports and four Gb Ethernet ports.
Two optional InfiniBand ports are available (one per serverboard).
Cooling System
The server chassis has an innovative cooling design that features an air shroud and three sets of
4-cm counter-rotating fans per node (configured as six fans placed back-to-back in pairs). A fan
speed control setting in BIOS allows fan speed to be determined by system temperature.
1U Twin: System Notes
As a 1U Twin configuration, the Altix XE340 is different from a traditional single-motherboard
server system. With two nodes incorporated into a single chassis, there are several functional
points you should keep in mind as listed in the following subsections.
System Power
A single power supply is used to provide the power for both serverboards. Each serverboard
however, can be shut down independently of the other with the power button on its own control
panel.
Although they share a common power supply, the I2C signals used for power supply monitoring
are received by the primary serverboard only. (When viewed from the front of the chassis, the
serverboard on the right is referred to as the primary board and the serverboard on the left as the
secondary.) See Figure 1-3 on page 8 for a top-down view of the power supply and other major
components.

8007-5536-001
1: Introduction and Overview
System Backplane and Drives
As a system, the Altix XE340 server supports the use of four hard disk drives. The disk-drive
backplane works as a single backplane divided into two sections. This means that while a single
power connector is used and functions such as overheating apply to both sections together, each
pair of hard drives is logically connected to its own serverboard. Consequently, RAID setup is
limited to a two-drive scheme, (RAID cannot be spread across all four drives).
Figure 1-3 Server Top View (Cover Removed)
Note: The disk drives are hot-pluggable; however, the standard (non-RAID) hardware/software
does not seamlessly support hot-plugging. If you add a disk to a non-RAID system, you need to
reboot the system to see it. If you remove a disk from a non-RAID system and do not replace it,
then system I/O time-outs will occur when you attempt to access it.
Disk drive 1
Disk drive 2
Disk drive 3
Disk drive 4
Fan
Fan
Fan
Fan
Fan
Fan
6 memory
DIMMs Processor
socket
1
Processor
socket
26 memory
DIMMs
6 memory
DIMMs Processor
socket
1
Processor
socket
26 memory
DIMMs
Power supply
Node
1
Node
2
Gen2
Gen2

007-5536-001 9
Chapter 2
2. System Operation
This chapter describes the basic steps needed to get your Altix XE340 up and running. Following
these steps in the order given should enable you to have the system operational within a minimum
amount of time.
Unpacking the System and Choosing an Operating Location
You should inspect the box the system was shipped in and note if it was damaged in any way. If
the server itself shows damage you should file a damage claim with the carrier who delivered it.
When you decide on a suitable location for the rack unit that will hold the system, it should be
situated in a clean, dust-free area that is well ventilated. Avoid areas where heat, electrical noise
and electromagnetic fields are generated. You will also need it placed near a grounded power
outlet. Be sure to read the Rack and Server Precautions in the next section.
Preparing for Setup
The box the system was shipped in should include two sets of rail assemblies, two rail mounting
bracketsandthe mounting screws you will need toinstall thesystem into therack. Followthesteps
in the order given to complete the installation process in a minimum amount of time. Please read
this section in its entirety before you begin the installation procedure outlined in the sections that
follow.
Choosing a Setup Location
Leave enough clearance in front of the rack to enable you to open the front door completely 10 cm
(~25 inches).
Leave approximately 30 inches of clearance in the back of the rack to allow for sufficient airflow
and ease in servicing.

10 007-5536-001
2: System Operation
This product is for installation only in a Restricted Access Location (dedicated equipment rooms,
service closets and the like).
This product is not suitable for use with visual display work place devices according to the
German Ordinance for Work with Visual Display Units.
Rack Precautions
Ensure that the leveling jacks on the bottom of the rack are fully extended to the floor with the full
weight of the rack resting on them.
In single rack installation, stabilizers should be attached to the rack.
In multiple rack installations, the racks should be coupled together.
Always make sure the rack is stable before extending a component from the rack.
You should extend only one component at a time - extending two or more simultaneously may
cause the rack to become unstable.
Always keep the rack's front door and all panels and components on the servers closed when not
servicing to maintain proper cooling.
Server Precautions
Review the electrical and general safety precautions in Chapter 4.
Determine the placement of each component in the rack before you install the rails.
Install the heaviest server components on the bottom of the rack first, and then work up.
For extra protection, use a regulating uninterruptible power supply (UPS) to protect the server
from power surges, voltage spikes and to keep your system operating in case of a power failure.
This is an optional device not provided by SGI with your system.
Allow the hot plug disk drives and power supply modules to cool before touching them. Always
keep the server closed when not servicing to maintain proper cooling.
Make sure all power and data cables are properly connected and not blocking the chassis airflow.

Unpacking the System and Choosing an Operating Location
007-5536-001 11
Rack Mounting Considerations
Use the guidelines in the following subsections to properly install, use and maintain a server in a
rack.
Ambient Operating Temperature
If installed in a closed or multi-unit rack assembly, the ambient operating temperature of the rack
environment may be greater than the ambient temperature of the room. Therefore, consideration
should be given to installing the equipment in an environment compatible with the manufacturer’s
maximum rated ambient temperature (Tmra).
Reduced Airflow
Equipment should be mounted into a rack so that the amount of airflow required for safe operation
is not compromised.
Mechanical Loading
Equipment should be mounted into a rack so that a hazardous condition does not arise due to
uneven mechanical loading.
Circuit Overloading
Consideration should be given to the connection of the equipment to the power supply circuitry
and the effect that any possible overloading of circuits might have on overcurrent protection and
power supply wiring. Appropriate consideration of equipment nameplate ratings should be used
when addressing this concern.
Reliable Ground
A reliable ground must be maintained at all times. To ensure this, the rack itself should be
grounded. Particular attention should be given to power supply connections other than the direct
connections to the branch circuit (i.e. the use of power strips, etc.).

12 007-5536-001
2: System Operation
Installing the System into a Rack
This section provides information on installing the Altix XE340 server into a rack unit with the
rack rails provided. If the system has already been mounted into a rack, you can skip ahead to the
next section.
There are a variety of rack units on the market, which may mean the assembly procedure will
differ slightly. You should also refer to the installation instructions that came with the rack unit
you are using.
Identifying the Sections of the Rack Rails
You should have received two rack rail assemblies in the rack mounting kit. Each assembly
consists of two sections: an inner fixed chassis rail that secures directly to the server chassis and
an outer fixed rack rail that secures directly to the rack itself (see Figure 2-1). Two pairs of short
brackets to be used on the front side of the outer rails are also included.
Figure 2-1 SGI Altix XE340 Server Rack Rail Example
I
nner rail (pre-installed)
Outer rail (attaches to rack)
Locking tab