Penguin Computing Relion 900 Quick start guide

Relion 900
Power Efficient Intel-based HPC Server Designed forNext- generation Datacenters
Technical Guide
Rev. 1.0
PENGUIN
COMPUTING
www.penguincomputing.com | 1-888-PENGUIN (736-4846) | twitter:@PenguinHPC

R130-134
R130-234
Dual LGA2011 sockets R3 motherboard for Intel®E5-2600 V3 series processors
Service Guide
Rev. 1.0

Copyright
©2014 GIGA-BYTE TECHNOLOGYCO., LTD.All rights reserved.
The trademarks mentioned in this manual are legally registered to their respective owners.
Disclaimer
Informationin this manualis protected bycopyright laws andis the propertyof GIGABYTE.
Changes to the specifications and features in this manual may be made by GIGABYTE
withoutprior notice. No part ofthis manual maybe reproduced, copied,translated, transmitted,or
publishedin any formor by anymeans without GIGABYTE'sprior written permission.
Documentation Classications
Inordertoassistintheuseofthisproduct,GIGABYTEprovidesthefollowingtypesofdocumentations:
For detailed product information, carefully read the User's Manual.
For more information, visit our website at:
http://b2b.gigabyte.com
You are a professional?
Get an access to our complete source of sales, marketing & technical materials at:
http://reseller.b2b.gigabyte.com
https://www.facebook.com/gigabyteserver

Preface
Before using this information and the product it supports, please read the following general infor-
mation.
1. This Service Guide provides you with all technical information relating to the BASIC CON-
FIGURATION decided for GIGABYTE’s “global” product offering. To better t local market-
requirements and enhance product competitiveness, your regional ofce MAY have decided
toextend the functionality of a machine (e.g. add-on card, modem, or extra memory capabil-
ity).These LOCALIZED FEATURES will NOT be covered in this generic service guide. In
suchcases, please contact your regional ofces or the responsible personnel/channel to
provide youwith further technical details.
2. Please note WHEN ORDERING SPARE PARTS, you should check the most up-to-date
informationavailable on your regional web or channel. For whatever reason, if a part num-
ber change is made,it will not be noted in the printed Service Guide. For GIGABYTE-AU-
THORIZED SERVICEPROVIDERS, your GIGABYTE ofce may have a DIFFERENT part
number code to thosegiven in the FRU list of this printed Service Guide. You MUST use the
list provided by yourregional GIGABYTE ofce to order FRU parts for repair and service of
customer machines.

- 4 -
Table of Contents
Box Contents...................................................................................................................7
Safety, Care and Regulatory Information........................................................................8
Chapter 1 Hardware Installation...................................................................................11
1-1 Installation Precautions.................................................................................. 11
1-2 Product Specications....................................................................................12
1-3 System Block Diagram...................................................................................15
Chapter 2 System Hardware Installation......................................................................16
2-1 Removing Chassis Cover...............................................................................17
2-2 Removing and Installing the Fan Duct...........................................................18
2-3 Installing the CPU .........................................................................................19
2-4 Installing the Heat Sink .................................................................................20
2-5 Installing the Memory.....................................................................................21
2-5-1 Four Channel Memory Conguration .....................................................................21
2-5-2 Installing a Memory ...............................................................................................22
2-5-3 DIMM Population Table ..........................................................................................22
2-6 Installing the PCI Expansion Card.................................................................23
2-7 Installing the Hard Disk Drive.........................................................................24
2-8 Replacing the FANAssemblly........................................................................25
2-9 Replacing the Power Supply..........................................................................26
Chapter 3 SystemAppearance.....................................................................................27
3-1 Front View......................................................................................................27
3-2 Rear View.......................................................................................................28
3-3 Front Panel LED and Buttons........................................................................29
3-4 Rear System LAN LEDs.................................................................................31
3-5 Hard Disk Drive LEDs....................................................................................32
3-6 Cable Routing................................................................................................33
Chapter 4 Motherboard Components...........................................................................34
4-1 MD60-SC1/MD60-SC2 Motherboard Components........................................34
4-2 Jumper Setting ..............................................................................................37
4-3 Back Plane Boad Components......................................................................39
4-3-1 Back Plane BoadJ2/J3 Jumper Setting..........................................................40
Chapter 5 BIOS Setup..................................................................................................41

- 5 -
5-1 The Main Menu..............................................................................................43
5-2 Advanced Menu.............................................................................................46
5-2-1 Serial Port Console Redirection.............................................................................47
5-2-2 PCI Subsystem Settings.........................................................................................51
5-2-2-1 PCI Express Settings..............................................................................................53
5-2-3 Network Stack........................................................................................................55
5-2-4 CSM Conguration .................................................................................................56
5-2-5 Post Report Conguration ......................................................................................58
5-2-6 Trusted Computing.................................................................................................59
5-2-7 USB Conguration..................................................................................................60
5-2-8 Chipset Conguration.............................................................................................61
5-9 SIO Conguration...................................................................................................62
5-2-10 iSCSI Conguration................................................................................................66
5-3 Intel RC Setup Menu......................................................................................67
5-3-1 Processor Conguration.........................................................................................68
5-3-1-1 Pre-Socket Conguration .......................................................................................71
5-3-2 Advanced Power Management Conguration........................................................73
5-3-2-1 CPU P State Control...............................................................................................74
5-3-2-2 CPU C State Control..............................................................................................75
5-3-2-3 CPU T State Control...............................................................................................76
5-3-3 Common RefCode Conguration ...........................................................................77
5-3-4 QPI Conguration...................................................................................................78
5-3-5 Memory Conguration ............................................................................................80
5-3-5-1 Memory Topology...................................................................................................82
5-3-5-2 Memory Thermal ....................................................................................................83
5-3-5-3 Memory Map...........................................................................................................84
5-3-5-4 Memory RAS Conguration....................................................................................85
5-3-6 IIO Conguration ....................................................................................................86
5-3-6-1 IOAT Conguration.................................................................................................87
5-3-6-2 Intel VT for Directed I/O (VT-d) ..............................................................................88
5-3-7 PCH Conguration .................................................................................................89
5-3-7-1 PCH Devices..........................................................................................................90
5-3-7-2 PCH sSATA Conguration ......................................................................................91
5-3-7-2-1 SATAMode Options..............................................................................................94
5-3-7-3 PCH SATA Conguration........................................................................................96
5-3-7-3-1 SATAMode Options..............................................................................................99
5-3-7-4 USB Conguration................................................................................................101
5-3-8 Miscellaneous Conguration ................................................................................102
5-3-9 Server ME Conguration ......................................................................................103
5-3-10 Runtime Error Logging.........................................................................................104
5-3-10-1Whea Setting........................................................................................................105
5-3-10-2Memory Error Enabling.........................................................................................106

- 6 -
5-3-10-3 PCI/PCI Error Enabling.........................................................................................107
5-4 Server Management Menu...........................................................................108
5-4-1 System Event Log................................................................................................110
5-4-2 View FRU Information..........................................................................................111
5-4-3 BMC network conguration ..................................................................................112
5-5 Security Menu.............................................................................................. 113
5-5-1 Secure Boot menu ...............................................................................................114
5-5-1-1 Key Management ...............................................................................................115
5-6 Boot Menu.................................................................................................... 117
5-7 Save & Exit Menu......................................................................................... 119
5-8 BIOS POST Codes ......................................................................................121
5-9 BIOS POST Beep code................................................................................125
5-9-1 PEI Beep Codes...................................................................................................125
5-9-2 DEX Beep Codes.................................................................................................125
5-10 BIOS Recovery Instruction...........................................................................126

- 7 -
Box Contents
R130-134 System
Driver CD
•The box contents above are for reference only and the actual items shall depend on the product package you obtain.
The box contents are subject to change without notice.
•The motherboard image is for reference only.

- 8 -
Safety, Care and Regulatory Information
Important safety information
Read and follow all instructions marked on the product and in the documentation before you operateyour sys-
tem. Retain all safety and operating instructions for future use.
• The product should be operated only from the type of power source indicated on the rating label.* If your
computer has a voltage selector switch, make sure that the switch is in the proper position foryour area.
The voltage selector switch is set at the factory to the correct voltage.
• The plug-socket combination must be accessible at all times because it serves as the main disconnect-
ing device.
• All product shipped with a three-wire electrical grounding-type plug only fits into a grounding-type
poweroutlet. This is a safety feature. The equipment grounding should be in accordance with local and
nationalelectrical codes. The equipment operates safely when it is used in accordance with its marked
electricalratings and product usage instructions
• Do not use this product near water or a heat source.* Set up the product on a stable work surface or so
as to ensure stability of the system.
• Openings in the case are provided for ventilation. Do not block or cover these openings. Make sure
youprovide adequate space around the system for ventilation when you set up your work area. Never
insertobjects of any kind into the ventilation openings.
• To avoid electrical shock, always unplug all power cables and modem cables from the wall outletsbefore
removing covers.
• Allow the product to cool before removing covers or touching internal components.
Precaution for Product with Laser Devices
Observe the following precautions for laser devices:
• Do not open the CD-ROM drive, make adjustments, or perform procedures on a laser device other than
those specied in the product's documentation.
• Only authorized service technicians should repair laser devices.
Precaution for Product with Modems, Telecommunications, ot Local AreaNetwork Options
Observe the following precautions for laser devices:
• Do not connect or use a modem or telephone during a lightning storm. There may be a risk of electri-
calshock from lightning.
• To reduce the risk of re, use only No. 26 AWG or larger telecommunications line cord.
• Do not plug a modem or telephone cable into the network interface controller (NIC) receptacle.
• Disconnect the modem cable before opening a product enclosure, touching or installing internalcompo-
nents, or touching an uninsulated modem cable or jack.
• Do not use a telephone line to report a gas leak while you are in the vicinity of the leak.

- 9 -
Federal Communications Commission (FCC) Statement
Warning
This is a class A product. In a domestic environment this product may cause radiointerfer-
enceIn which case the user may be required to take adequate measures.
This equipment has been tested and found to comply with the limits for a Class A digital device,pursuant to
Part 15 of the FCC Rules. These limits are designed to provide reasonable protection againstharmful interfer-
ence when the equipment is operated in a commercial environment. This equipmentgenerates, uses, and can
radiate radio frequency energy and, if not installed and used in accordance withthe instruction manual, may
cause harmful interference to radio communications. Operation of thisequipment in a residential area is likely
to cause harmful interference in which case the user will berequired to correct the interference at his own ex-
pense.Properly shielded and grounded cables and connectors must be used in order to meet FCC emission-
limits. Neither the provider nor the manufacturer are responsible for any radio or television interferencecaused
by using other than recommended cables and connectors or by unauthorized changes ormodications to this
equipment. Unauthorized changes or modications could void the user's authority tooperate the equipment.
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:
(1) this device may not cause harmful interference, and
(2) this device must accept any interference received, including interference that may cause undesired opera-
tion.
Canadian Department of Communications Compliance Statement
This digital apparatus does not exceed the Class A limits for radio noise emissions from digitalapparatus as
set out in the radio interference regulations of Industry Canada.Le present appareil numerique n'emet pas
de bruits radioelectriques depassant les limites applicables auxappareils numeriques de Classe A prescrites
dans le reglement sur le brouillage radioelectrique edicte parIndustrie Canada.
ClassAequipment
This device has been tested and found to comply with the limits for a classAdigital device pursuantPart 15 of
the FCC Rules. These limits are designed to provide reasonable protection againstharmful interference when
the equipment is operated in a commercial environment. This equipmentgenerate, uses, and can radiate
radio frequency energy, and if not installed and used in accordancewith the instructions, may cause harmful
interference to radio communication. Operation of thisequipment in a residential area is likely to cause harm-
ful interference, in which case the user will berequired to correct the interference at personal expence.
However, there is no guarantee that interference will not occur in a particular installation. If thisdevice does
cause harmful interference to radio or television reception, which can be determined bytuning the device off
and on, the user is encouraged to try to correct the interference by on or more ofthe following measures:
• Reorient or relocate the receiving antenna
• Increase the separation between the device and receiver
• Connect the device into an outlet on a circuit different from that to which the receiver isconnected'Consult
the dealer or an experienced radio/television technician for help.

- 10 -
Battery Warning: Incorrectly installing a battery or using incompatible battery may increase
the risk of ifre explosion. Replace the battery only with the same or equivalent type.
•Do not disassemble, crush, punchture batteries.
• Do not store or place your battery pack next to or in a heat source such as a re, heatgenerating
appliance, can or exhaust vent. Heating battery cells to temperatures above 65oC (149oF) can
cause explosion or re.
•Do not attempt to open or service batteries. Do not dispose of batteries in a re or with house-
hold waste.
WEEE Symbol Statement
The symbol shown below is on the product or on its packaging, which indicates that this product
must not be disposed of with other waste. Instead, the device should be taken to the waste
collection centers for activation of the treatment, collection, recycling and disposal procedure.
The separate collection and recycling of your waste equipment at the time of disposal will help to
conserve natural resources and ensure that it is recycled in a manner that protects human health
and the environment. For more information about where you can drop off your waste equipment for recycling,
please contact your local government ofce, your household waste disposal service or where you purchased
the product for details of environmentally safe recycling.
wWhen your electrical or electronic equipment is no longer useful to you, "take it back" to your local or
regional waste collection administration for recycling.
wIf you need further assistance in recycling, reusing in your "end of life" product, you may contact us at the
Customer Care number listed in your product's user's manual and we will be glad to help you with your
effort.

- 11 -
Hardware Installation
1-1 Installation Precautions
The motherboard/system contain numerous delicate electronic circuits and components which
can become damaged as a result of electrostatic discharge (ESD). Prior to installation, carefully
read the service guide and follow these procedures:
• Prior to installation, do not remove or break motherboard S/N (Serial Number) sticker or
warranty sticker provided by your dealer.These stickers are required for warranty validation.
• Always remove the AC power by unplugging the power cord from the power outlet before
installing or removing the motherboard or other hardware components.
• When connecting hardware components to the internal connectors on the motherboard,
make sure they are connected tightly and securely.
• When handling the motherboard, avoid touching any metal leads or connectors.
• It is best to wear an electrostatic discharge (ESD) wrist strap when handling electronic
components such as a motherboard, CPU or memory. If you do not have an ESD wrist
strap, keep your hands dry and rst touch a metal object to eliminate static electricity.
• Prior to installing the motherboard, please have it on top of an antistatic pad or within an
electrostatic shielding container.
• Before unplugging the power supply cable from the motherboard, make sure the power
supply has been turned off.
• Before turning on the power, make sure the power supply voltage has been set according to
the local voltage standard.
• Before using the product, please verify that all cables and power connectors of your
hardware components are connected.
• To prevent damage to the motherboard, do not allow screws to come in contact with the
motherboard circuit or its components.
• Make sure there are no leftover screws or metal components placed on the motherboard or
within the computer casing.
• Do not place the computer system on an uneven surface
.
• Do not place the computer system in a high-temperature environment.
• Turning on the computer power during the installation process can lead to damage to
system components as well as physical harm to the user.
• If you are uncertain about any installation steps or have a problem related to the use of the
product, please consult a certied computer technician.
Chapter 1 Hardware Installation

Hardware Installation
- 12 -
1-2 ProductSpecications
Motherboard R130-134: MD60-SC1
R130-234:MD60-SC2
CPU SupportforIntel®Xeon®E5-2600V3seriesprocessorsintheLGA2011package
L3 cache varies with CPU
Supports Dual QuickPath Interconnect up to 9.6GT/s
Enhanced Intel SpeedStep Technology (EIST)
Support Intel Virtualization Technology (VT)
Chipset Intel®C612 Express (Wellsburg) Chipset
Memory 16 x 1.2V DDR4 DIMM sockets supporting up to 512GB RDIMM of system
memory
16x1.2VDDR4DIMMsocketssupportingupto1TBLRDIMMofsystemmemory
Four channel memory architecture
Supports DDR4 2133MHz RDIMM memory modules
Supports DDR4 2133MHz LR-DIMM memory modules
Support for ECC RDIMM/LRDIMM memory modules
LAN Intel®I350 supports 10/100/1000 Mbps (R130-134 Only)
Intel®82599ES 10GbE LAN controller with QSFP+ (R130-234 Only)
1 x Management LAN 10/100/1000Mpbs LAN port
Expansion Slot
(R130-134) 2 x PCI Express x8 slots
Expansion Slot
(R130-234) 2 x PCI Express x16 slot, running at x16 (Gen3/PCIE_1/PCIE_3) 2 x PCI Express x8 slot, running at x8 (Gen3/PCIE_2/PCIE_4)
1 x PCI Express x8 slot, (Gigabyte extension slot/RAID_SLOT1/RAID_SLOT2/
GEN3)
2 x PCI Express x8 slots
Onboard
Graphics ASPEED®AST2400 supports 16MB DDR3 VRAM
Mass Storage 4 x 3.5” Hot-Swap SATA/SAS HDDs
2 x SATA3 6Gb/s connectors (SATA4/SATA5/Option)
Support for Intel RSTe SATA RAID 0, RAID 1, RAID 10
8 x SASGb/s connectors (SAS0/SAS1/SAS2/SAS3/SAS4/SAS5/SAS6/SAS7;
Activating SAS ports with Gigabyte extension card
2 x Mini-SAS connectors (8 additional SATA3 ports (6Gb/s)
System Fans 6 x 40x40x56mm Front: 16000rpm/Rear: 15400rpm
USB Up to 8 USB 3.0 ports (6 on the rear panel I/O, 2 at the front panel/R130-134)
Up to 4 USB 3.0 ports (2 on the rear panel I/O, 2 at the front panel/R130-234)

- 13 -
Hardware Installation
Internal
Connectors 1 x 24-pin ATX main power connector 2 x 8-pin ATX 12V power connector
2 x Mini-SAS connectors (SATA3 6Gb/s signal)
8 x SAS connectors
2 x SATA3 6Gb/s connectors
1 x PMBus header
2 x CPU fan headers
5 x System fan headers
1 x Front panel header
1 x HDD Back plane borad header
1 x USB 3.0 header
1 x TPM module connector
1 x Serial port connector
2 x SAS SPGIO headers
1 x IPMB connector
Rear Panel I/O
(R130-134) 6 x USB 3.0 ports 3 x RJ-45 ports (1 x 10/100/1000 Mbps dedicated management LAN port)
1 x COM port
1 x VGAport
1 x PS/2 connector
Rear Panel I/O
(R130-234) 2 x USB 3.0 ports 1 x QSPF LAN port
1 x RJ-45 port (10/100/1000 Mbps dedicated management LAN port)
1 x COM port
1 x VGAport
1 x PS/2 connector
Front Panel
LED/Buttons 1 x Power button/LED 1 x NMI button
1 x ID Switch button/LED
1 x Reset button
2 x LAN LED
1 x System status LED
1 x HDD status LED
I/O Controller ASPEED®AST2400 BMC chip
Hardware
Monitor System voltage detection CPU/System temperature detection
CPU/System fan speed detection
CPU/System fan speed control
* Whether the CPU/system fan speed control function is supported will depend on
the CPU/system cooler you install.
BIOS 1 x 128 Mbit ash
AMI BIOS

Hardware Installation
- 14 -
Environment
Ambient
Temperature
Relative
Humidity
Operating Temperature: 5oC to 35oC
Non-operating Temperature: 0oC to 50oC
10-80% operating Humidity at 30oC
System
Dimension 444Wx43.5Hx615D(mm)
Electrical
Power Supply Fixed 1U PSU 600W 100-240VAC at 80 plus gold level
* GIGABYTE reserves the right to make any changes to the product specications and product-related information
without prior notice.

- 15 -
Hardware Installation
1-3 System Block Diagram

Hardware Installation
- 16 -
Chapter 2 System Hardware Installation
Pre-installation Instructions
Perform the steps below before you open the server or before you remove or replaceany
component.
• Back up all important system and data les before performing any hardwareconguration.
• Turn off the system and all the peripherals connected to it.
• Locate the pin one of the CPU. The CPU cannot be inserted if oriented incorrectly. (Or you may
locate the notches on both sides of the CPU and alignment keys on the CPU socket.)
• Apply an even and thin layer of thermal grease on the surface of the CPU.
• Do not turn on the computer if the CPU cooler is not installed, otherwise overheating and
damage of the CPU may occur.
• Set the CPU host frequency in accordance with the CPU specications. It is not recommended
that the system bus frequency be set beyond hardware specications since it does not meet the
standard requirements for the peripherals. If you wish to set the frequency beyond the standard
specifications, please do so according to your hardware specifications including the CPU,
graphics card, memory, hard drive, etc.

- 17 -
Hardware Installation
2-1 Removing Chassis Cover
Before you remove or install the system cover
• Make sure the system is not turned on or connected toAC power.
Follow these instructions to remove the system cover:
1. Loosen and remove the thumbscrew securing the front chassis cover.
2. Press the the indentation located at two sides of the cover.
3. Slide the cover horizontally to the back and remove the cover in the direction of the arrow.

Hardware Installation
- 18 -
2-2 Removing and Installing the Fan Duct
Follow these instructions to remove/install the fan duct:
1. Lift up to remove the fan duct
2. To install the fan duct, align the fan duct with the guiding groove. Push down the fan duct into
chassis until its rmly seats

- 19 -
Hardware Installation
2-3 Installing the CPU
Read the following guidelines before you begin to install the CPU:
• Make sure that the motherboard supports the CPU.
• Always turn off the computer and unplug the power cord from the power outlet before installing
the CPU to prevent hardware damage.
• Unplug all cables from the power outlets.
• Disconnect all telecommunication cables from their ports.
• Place the system unit on a at and stable surface.
• Open the system according to the instructions.
WARNING!
Failure to properly turn off the server before you start installing components may cause serious
damage. Do not attempt the procedures described in the following sections unless you are a
qualied service technician.
Follow these instructions to install the CPU:
1. Raise the metal locking lever on the socket.
2. Remove the plastic covering on the CPU socket.Insert the CPU with the correct orientation. The
CPU only ts in one orientation.
3. Replace the metal cover.
4. Push the metal lever back into locked position.