Penguin Computing Relion 2908GT Quick start guide

Relion 2908GT
Power Efficient Intel-based HPC Server Designed for Next-generation Datacenters
Technical Guide
Rev. 1.0
PENGUIN
COMPUTING
www.penguincomputing.com | 1-888-PENGUIN (736-4846) | twitter:@PenguinHPC

Relion 2908GT Chassis Manual
Copyright
© 2014,Penguin Computing. All rights reserved.
The trademarks mentioned in this manual are legally registered to their respective owners.
Disclaimer
Information in this manual is protected by copyright laws and is the property of Penguin Comput-
ing.Changestothespecificationsandfeaturesinthis manualmaybemadebyPenguinComputing
without priornotice. No partofthis manual may bereproduced, copied,translated,transmitted,or
publishedinanyformorbyanymeanswithoutPenguinComputing'spriorwrittenpermission.
Documentation Classifications
Inorder toassist inthe use of this product, Penguin Computing provides the following types ofdocu-
mentations:
For detailed product information, carefully readthe User's Manual.
For more information, visit our website at:
http://www.penguincomputing.com

Relion 2908GT Chassis Manual
Preface
Before using this information and the product it supports, please read the following general infor-
mation.
1. This Service Guide provides you with all technical information relating to the BASIC CON-
FIGURATION decided for Penguin Computing’s “global” product offering. To better fit local
marketrequirements and enhance product competitiveness, your regional office MAY have
decided toextend the functionality of a machine (e.g. add-on card, modem, or extra memory
capability).These LOCALIZED FEATURES will NOT be covered in this generic service
guide. In suchcases, please contact your regional offices or the responsible personnel/
channel to provide youwith further technical details.
2. Please note WHEN ORDERING SPARE PARTS, you should check the most up-to-date in-
formationavailable on your regional web or channel. For whatever reason, if a part number
change is made,it will not be noted in the printed Service Guide. For Penguin Computing-
AUTHORIZED SERVICEPROVIDERS, your Penguin Computing office may have a DIF-
FERENT part number code to thosegiven in the FRU list of this printed Service Guide. You
MUST use the list provided by yourregional Penguin Computing office to order FRU parts
for repair and service of customer machines.

- 4 -
Relion 2908GT Chassis Manual
Table of Contents
Box Contents 7
Safety, CareandRegulatory Information 8
Chapter 1 Hardware Installation 11
1-1 Installation Precautions 11
1-2 Product Specifications 12
1-3 System Block Diagram 14
Chapter 2 System HardwareInstallation 15
2-1 Removing Chassis Cover 16
2-2 Removing and Installing the Fan Duct 17
2-3 Installing the CPU 18
2-4 Installing the Heat Sink 19
2-5 Installing the Memory 21
2-5-1 Triple Channel Memory Configuration 21
2-5-2 Installing a Memory 22
2-6 Installing the PCI Expansion Card 23
2-6-1 Installing Add-on Card (MNXE2/MEZZ_2/Optional) 28
2-6-2 Installing Add-on Card (MLIZS/MEZZ_2/Optional) 29
2-7 Installing the Hard Disk Drive 31
2-8 Replacing the FANAssemblly 32
2-9 Replacing the Power Supply 37
Chapter 3 SystemAppearance 38
1Front View 38
2Rear View 39
3-3 Front Panel LED and Buttons 40
3-4 Rear System Button and LEDs 41
3-5 Hard Disk Drive LEDs 42
3-6 Hard Disk Back Plane Board Jumper Setting 43
3-7 Cable Routing 44
Chapter 4 Motherboard Components 45
4-1 MG50-G20 Motherboard Components 45
4-2 Jumper Setting 48
Chapter 5 BIOS Setup 49

1Memory Topology 89
2Memory Thermal 90
3Memory Map 91
5-3-5-4 Memory RAS Configuration 92
5-3-6 IIO Configuration 93
5-3-6-1 IOAT Configuration 94
5-3-6-2 Intel VT for Directed I/O (VT-d) 95
5-3-7 PCH Configuration 96
5-3-7-1 PCH Devices 97
5-3-7-2 PCH sSATAConfiguration 98
5-3-7-2-1 SATAMode Options 101
5-3-7-3 PCH SATAConfiguration 103
5-3-7-3-1 SATAMode Options 106
5-3-7-4 USB Configuration 108
8Miscellaneous Configuration 109
9ServerME Configuration 110
5-3-10 Runtime ErrorLogging 111
5-3-10-1Whea Setting 112
5-3-10-2Memory Error Enabling 113
- 5 -
5-2-3
5-2-4
5-2-5
5-2-6
5-2-7
5-2-8
5-9
Network Stack 63
CSM Configuration 64
Post Report Configuration 66
Trusted Computing 67
USB Configuration 68
Chipset Configuration 69
SIO Configuration 70
5-2-10 iSCSI Configuration 73
5-3 Intel RC Setup Menu 74
5-3-1 Processor Configuration 75
5-3-1-1 Pre-Socket Configuration 78
5-3-2 Advanced PowerManagement Configuration 80
5-3-2-1 CPU P State Control 81
5-3-2-2 CPU C State Control 82
5-3-2-3 CPU T State Control 83
5-3-3
5-3-4
5-3-5
Common RefCode Configuration 84
QPI Configuration 85
Memory Configuration 87
Relion 2908GT Chassis Manual
5-1 The Main Menu 51
5-2 Advanced Menu 54
5-2-1 SerialPortConsoleRedirection 55
5-2-2 PCI Subsystem Settings 59
5-2-2-1 PCI Express Settings 61

- 6 -
5-4-1
5-4-2
5-4-3
System Event Log 117
View FRU Information 118
BMC network configuration 119
5Security Menu 120
5-5-1 SecureBoot menu 121
5-5-1-1 Key Management 122
6Boot Menu 124
7Save & Exit Menu 126
5-8 BIOSPOSTCodes 128
5-9 BIOSPOSTBeep code 132
5-9-1 PEI Beep Codes 132
5-9-2 DEX Beep Codes 132
5-10 BIOSRecovery Instruction 133
Relion 2908GT Chassis Manual
5-3-10-3PCI/PCI ErrorEnabling 114
5-4 Server Management Menu 115

- 7 -
•The box contents above are for reference only and the actual items shall depend on the product package you obtain.
The box contents are subject to change without notice.
•The motherboard image is for reference only.
Relion 2908GT Chassis Manual
Box Contents
Relion 2908GT System

- 8 -
Safety, Care and Regulatory Information
Important safety information
Read and follow all instructions marked on the product and in the documentation before you operateyour sys-
tem. Retain all safety and operating instructions for future use.
•The product should be operated only from the type of power source indicated on the rating label.* If your
computer has a voltage selector switch, make sure that the switch is in the proper position foryour area.
The voltage selector switch is set at the factory to the correct voltage.
•The plug-socket combination must be accessible at all times because it serves as the main
disconnect- ing device.
•All product shipped with a three-wire electrical grounding-type plug only fits into a grounding-type
poweroutlet. This is a safety feature. The equipment grounding should be in accordance with local and
nationalelectrical codes. The equipment operates safely when it is used in accordance with its marked
electricalratings and product usageinstructions
•Do not use this product near water or a heat source.* Set up the product on a stable work surface or so
as to ensure stability of the system.
•Openings in the case are provided for ventilation. Do not block or cover these openings. Make sure
youprovide adequate space around the system for ventilation when you set up your work area. Never
insertobjects of any kind into the ventilation openings.
• To avoid electrical shock, always unplug all power cables and modem cables from the wall outletsbefore
removing covers.
•Allow the product to cool before removing coversor touching internal components.
Precaution for Product with Laser Devices
Observe the following precautions for laser devices:
•Do not open the CD-ROM drive, make adjustments, or perform procedures on a laser device other
than those specified in the product'sdocumentation.
•Only authorized servicetechnicians should repair laser devices.
Precaution for Product with Modems, Telecommunications, ot LocalAreaNetwork Options
Observe the following precautions for laser devices:
•Do not connect or use a modem or telephone during a lightning storm. There may be a risk of
electri- calshock from lightning.
• To reduce the risk of fire, use only No. 26AWG or larger telecommunications line cord.
•Do not plug a modem ortelephone cableinto the network interface controller(NIC) receptacle.
•Disconnect the modem cable before opening a product enclosure, touching or installing internalcompo-
nents, or touching an uninsulated modem cable or jack.
•Do not use a telephone line to reporta gas leak while you arein the vicinity of the leak.
Relion 2908GT Chassis Manual

Relion 2908GT Chassis Manual
Federal Communications Commission (FCC) Statement
Warning
This is a class A product. In a domestic environment this product may cause radiointerfer-
enceIn which case the user may be required to take adequate measures.
This equipment has been tested and found to comply with the limits for a Class A digital device,pursuant to
Part 15 of the FCC Rules. These limits are designed to provide reasonable protection againstharmful interfer-
ence when the equipment is operated in a commercial environment. This equipmentgenerates, uses, and can
radiate radio frequency energy and, if not installed and used in accordance withthe instruction manual, may
cause harmful interference to radio communications. Operation of thisequipment in a residential area is likely
to cause harmful interference in which case the user will berequired to correct the interference at his own ex-
pense.Properly shielded and grounded cables and connectors must be used in order to meet FCC emission-
limits. Neither the provider nor the manufacturer are responsible for any radio or television interferencecaused
by using other than recommended cables and connectors or by unauthorized changes ormodifications to this
equipment. Unauthorized changes or modifications could void the user's authority tooperate the equipment.
This devicecomplies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
(1)this device may not cause harmful interference, and
(2)this device must accept any interference received, including interference that may cause undesired opera-
tion.
Canadian Department of Communications Compliance Statement
This digital apparatus does not exceed the Class A limits for radio noise emissions from digitalapparatus as
set out in the radio interference regulations of Industry Canada.Le present appareil numerique n'emet pas
de bruits radioelectriques depassant les limites applicables auxappareils numeriques de Classe A prescrites
dans le reglement sur le brouillage radioelectrique edicte parIndustrie Canada.
Class Aequipment
This device has been tested and found to comply with the limits for a class A digital device pursuantPart 15 of
the FCC Rules. These limits are designed to provide reasonable protection againstharmful interference when
the equipment is operated in a commercial environment. This equipmentgenerate, uses, and can radiate
radio frequency energy, and if not installed and used in accordancewith the instructions, may cause harmful
interference to radio communication. Operation of thisequipment in a residential area is likely to cause harm-
ful interference, in which case the user will berequired to correct the interference at personal expence.
However, there is no guarantee that interference will not occur in a particular installation. If thisdevice does
cause harmful interference to radio or television reception, which can be determined bytuning the device off
and on, the user is encouraged to try to correct the interference by on or more ofthe following measures:
•Reorient or relocate the receiving antenna
•Increasethe separation between the device and receiver
•Connect the device into an outlet on a circuit different from that to which the receiver isconnected'Consult
the dealer or an experienced radio/television technician for help.
- 9 -

- 10 -
Relion 2908GT Chassis Manual
WEEE Symbol Statement
The symbol shown below is on the product or on its packaging, which indicates that this product
must not be disposed of with other waste. Instead, the device should be taken to the waste
collection centers for activation of the treatment, collection, recycling and disposal procedure.
The separate collection and recycling of your waste equipment at the time of disposal will help to
conservenatural resources andensure that it is recycled in a mannerthat protects human health
and the environment. For more information about where you can drop off your waste equipment for recycling,
please contact your local government office, your household waste disposal service or where you purchased
the productfor details of environmentally safe recycling.
w When your electrical or electronic equipment is no longer useful to you, "take it back" to your local or
regional waste collection administration for recycling.
w If you need further assistance in recycling, reusing in your "end of life" product, you may contact us at the
Customer Care number listed in your product's user's manual and we will be glad to help you with your
effort.
Battery Warning: Incorrectly installing a battery or using incompatible battery may increase
the risk of ifre explosion. Replace the battery only with the same or equivalent type.
•Do not disassemble, crush, punchture batteries.
•Do not store or place your battery pack next to or in a heat source such as a fire, heatgenerating
appliance, can or exhaust vent. Heating battery cells to temperatures above 65oC (149oF) can
cause explosion or fire.
•Do not attempt to open or service batteries. Do not dispose of batteries in a fire or with house-
hold waste.

HardwareInstallation - 11 -
Relion 2908GT Chassis Manual
1Installation Precautions
The motherboard/system contain numerous delicate electronic circuits and components which
can become damaged as a result of electrostatic discharge (ESD). Prior to installation, carefully
read the service guide and follow these procedures:
•Prior to installation, do not remove or break motherboard S/N (Serial Number) sticker or
warranty sticker provided by your dealer. These stickers are required for warranty
validation.
•Always remove the AC power by unplugging the power cord from the power outlet before
installing or removing the motherboard or other hardware components.
•When connecting hardware components to the internal connectors on the motherboard,
make sure they are connected tightly and securely.
•When handling the motherboard, avoid touching any metal leads or connectors.
•It is best to wear an electrostatic discharge (ESD) wrist strap when handling electronic
components such as a motherboard, CPU or memory. If you do not have an ESD wrist
strap, keep your hands dry and first touch a metal object to eliminate static electricity.
•Prior to installing the motherboard, please have it on top of an antistatic pad or within an
electrostatic shielding container.
•Before unplugging the power supply cable from the motherboard, make sure the power
supply has been turned off.
•Before turning on the power, make sure the power supply voltage has been set according to
the local voltage standard.
•Before using the product, please verify that all cables and power connectors of your
hardware components are connected.
•To prevent damage to the motherboard, do not allow screws to come in contact with the
motherboard circuit or its components.
•Make sure there are no leftover screws or metal components placed on the motherboard or
within the computer casing.
•Do not place the computer system on an uneven surface.
•Do not place the computer system in a high-temperature environment.
•Turning on the computer power during the installation process can lead to damage to
system components as well as physical harm to the user.
•If you are uncertain about any installation steps or have a problem related to the use of the
product, please consult a certified computer technician.
Chapter 1 Hardware Installation

- 12 - HardwareInstallation
Relion 2908GT Chassis Manual
1-2 Product Specifications
CPU SupportforIntel® Xeon® E5-2600V3seriesprocessorsintheLGA2011package
L3 cache varies with CPU
Supports Dual QuickPath Interconnect up to 9.6GT/s
Enhanced Intel SpeedStep Technology (EIST)
Support Intel Virtualization Technology (VT)
Chipset Intel® C612 (Wellsburg)Chipset
Memory 24x1.2VDDR4DIMMsocketssupportingupto768GBRDIMMofsystemmemory
24x1.2VDDR4DIMM socketssupportingupto1TBLRDIMMofsystemmemory
Four channel memory architecture
DDR4 2133MHz RDIMM memory modules
DDR4 2133MHz LR-DIMM memory modules
Support for ECC RDIMM/LRDIMM memory modules
LAN Intel® 82599ESsupports dual 10G SFP+ ethernet LAN ports
1 x KVM management LAN port
Expansion Slot 8 x PCI Express x16 slots, running at x16 (Gen3/PCIE_1/PCIE_2/PCIE_3/
PCIE_4)
1 x PCI Express x16 slot, running at x8 (Gen3/MEZZ_1/Proprietary slot)
1 x PCI Express x8 slot, running at x8 (Gen3/MEZZ_2/Proprietary slot)
Onboard
Graphics ASPEED® AST2400 supports 128MB DDR3 VRAM
Mass Storage 8 x 2.5” Hot-Swap SATA HDDs
Support for Intel IRSTe SATA RAID 0, RAID 1, RAID 5, RAID 10
System Fans 8 x 80x80x38mm 15000rpm
USB Up to 4 USB 3.0 ports (2 on the back panel, 2 additional ports via the USB
brackets connected to the internal USB headers)
Internal
Connectors
1 x 18-pin main power connector
1 x 14-pin ATX 12V power connector
2 x 8-pin power connectors
2 x 4-pin 12V power connectors
10 x SATA 6Gb/s connectors
1 x Front panel header
1 x HDD back plane board header
1 x SATA SGPIO header
1 x sSATA SGPIO header
1 x Front VGA port header
1 x Serial port header
1 x USB 3.0 header
1 x TPM module connector
1 x RAID KEY header
1 x IPMB connector

* Penguin Computing reserves the right to make any changes to the product specifications and product-
related information without prior notice.
HardwareInstallation - 13 -
Relion 2908GT Chassis Manual
Rear Panel I/O 2 x USB 2.0/3.0 ports
2 x 10G SFP+ LAN ports
1 x 10/100/1000 dedicated management LAN port
1 x Serial port
1 x VGAport
1 x Power switch button
1 x ID switch button
1 x Reset button
1 x NMI button
1 x System status LED
2 x LAN Link/Active LED/1G/10G Speed LED (LAN1/LAN2)
Front Panel
LED/Buttons
1 x Power button/LED
1 x ID button/LED
1 x Reset button
2 x LAN LED
1 x HDD Status LED
1 x System LED
I/O Controller ASPEED® AST2400 BMC chip
Hardware
Monitor
System voltage detection
CPU/System temperature detection
BIOS 1 x 128 Mbit flash
AMI BIOS
Environment
Ambient
Temperature
Relative
Humidity
Operating Temperature: 10oC to 35oC
Non-operating Temperature: -40oC to 60oC
8-80% operating Humidity (Non-condensing)
20-95%Non-operating Humidity (Non-condensing)
System
Dimension
871Wx86Hx482D (mm)
Electrical
Power Supply
2 x Hot-plug 1U PSU 1600W at 80 plus Platinum level
AC input 100-127V: DC Output 1000W
AC input 200-240V: DC Output 1600W

HardwareInstallation - 14 -
Relion 2908GT Chassis Manual
1-3 System Block Diagram

- 15 - HardwareInstallation
Relion 2908GT Chassis Manual
Chapter 2 System Hardware Installation
Pre-installation Instructions
Perform the steps below before you open the server or before you remove or replaceany
component.
•Back up all important system and data files before performing any hardwareconfiguration.
•Turn off the system and all the peripherals connected to it.
•Locate the pin one of the CPU. The CPU cannot be inserted if oriented incorrectly. (Or you may
locate the notcheson both sidesof the CPU and alignment keys onthe CPU socket.)
•Apply an even and thin layer of thermal grease on the surface of the CPU.
•Do not turn on the computer if the CPU cooler is not installed, otherwise overheating and
damage of the CPU may occur.
•Set the CPU host frequency in accordance with the CPU specifications. It is not recommended
that the system bus frequency be set beyond hardware specifications since it does not meet the
standard requirements for the peripherals. If you wish to set the frequency beyond the standard
specifications, please do so according to your hardware specifications including the CPU,
graphics card, memory, hard drive, etc.

HardwareInstallation - 16 -
Relion 2908GT Chassis Manual
1 Removing Chassis Cover
Before you remove or install the system cover
•Make surethe system is not turnedon or connectedtoAC power.
Follow these instructions to remove the system cover:
1. Loosen andremove the screws securing the top cover.
2. Slide the back cover toward the rear of the chassis to disengage it.
2
1
1

- 17 - HardwareInstallation
Relion 2908GT Chassis Manual
2-2 Removing and Installing the Fan Duct
Follow these instructions to remove/install the fan duct:
1. Lift up to remove the fan duct
2. To install the fan duct, align the fan duct with the guiding groove. Push down the fan duct into
chassis until its firmly seats

HardwareInstallation - 18 -
Relion 2908GT Chassis Manual
3Installing the CPU
Read the following guidelines before youbegin to install the CPU:
•Make surethat the motherboard supportsthe CPU.
•Always turn off the computer and unplug the power cord from the power outlet before installing
the CPU to prevent hardware damage.
•Unplug all cables from the power outlets.
•Disconnect all telecommunication cables from their ports.
•Place the system unit on a flat and stable surface.
•Open thesystem accordingto the instructions.
WARNING!
Failure to properly turn off the server before you start installing components may cause serious
damage. Do not attempt the procedures described in the following sections unless youare a
qualified service technician.
Follow these instructions to install the CPU:
1. Release then lift up the load lever.
2. Open the retention plate to expose the socketbody.
3. Insert the CPU with the correct orientation.
4. Close the retention plate and close the lever to the lockedposition.
12
3
4

- 19 - HardwareInstallation
Relion 2908GT Chassis Manual
2-4 Installing the Heat Sink
Follow these instructions to install the heat sinks:
1. Apply thermal compound evenlyon the top of the CPU.
2. Remove the protective cover from the underside of the heat sink.
3. Place the heat sink(s)on top of the CPU and tighten the four positioning screws.
CPU0 and CPU1 use the different CPU heat sinks. Please see the following table for installing
the correct CPU heat sink.
CPU0
CPU1

HardwareInstallation - 20 -
Relion 2908GT Chassis Manual
CPU0 CPU1
P/N: 25ST1-523200-C1R P/N: 25ST1-443101-T4R