Asus ESC4000 IB User manual

ESC4000 Series
Conguration Guide

ii
Copyright © 2011 ASUSTeK COMPUTER INC. All Rights Reserved.
No part of this manual, including the products and software described in it, may be reproduced, transmitted,
transcribed, stored in a retrieval system, or translated into any language in any form or by any means,
except documentation kept by the purchaser for backup purposes, without the express written permission
of ASUSTeK COMPUTER INC. (“ASUS”).
ASUS provides this manual “as is” without warranty of any kind, either express or implied, including but not
limited to the implied warranties or conditions of merchantability or tness for a particular purpose. In no
event shall ASUS, its directors, ofcers, employees, or agents be liable for any indirect, special, incidental,
or consequential damages (including damages for loss of prots, loss of business, loss of use or data,
interruption of business and the like), even if ASUS has been advised of the possibility of such damages
arising from any defect or error in this manual or product.
Specications and information contained in this manual ae furnished for informational use only, and are
subject to change at any time without notice, and should not be construed as a commitment by ASUS.
ASUS assumes no responsibility or liability for any errors or inaccuracies that may appear in this manual,
including the products and software described in it.
Product warranty or service will not be extended if: (1) the product is repaired, modied or altered, unless
such repair, modication of alteration is authorized in writing by ASUS; or (2) the serial number of the
product is defaced or missing.
Products and corporate names appearing in this manual may or may not be registered trademarks or
copyrights of their respective companies, and are used only for identication or explanation and to the
owners’ benet, without intent to infringe.
E6314
First Edition V1
May 2011

iii
Contents
Revision history
Revision Revision history Date
V1 First release of ESC4000 Series conguration guide May 2011
Revision history.......................................................................................... iii
Safety information ...................................................................................... iv
Chapter 1: Product introduction
1.1 Key features.................................................................................. 1-2
1.2 System overview.......................................................................... 1-4
1.3 Front panel features..................................................................... 1-5
1.4 Rear panel features...................................................................... 1-5
1.5 Systemspecications ................................................................. 1-6
Chapter 2: Components
2.1 Upgrading CPU and CPU heatsink............................................. 2-2
2.2 Upgrading system memory......................................................... 2-4
2.3 Upgrading hard disk drives......................................................... 2-6
2.4 Installing ASUS Server Management Board.............................. 2-7
2.5 Installing ASUS PIKE RAID module............................................ 2-8
2.6 NVIDIA®Tesla™ GPU computing modules ................................ 2-9
2.7 OS support list ........................................................................... 2-10

iv
Safety information
Electrical Safety
• Before installing or removing signal cables, ensure that the power cables for
the system unit and all attached devices are unplugged.
• To prevent electrical shock hazard, disconnect the power cable from the
electrical outlet before relocating the system.
• When adding or removing any additional devices to or from the system, contact
a qualied service technician or your dealer. Ensure that the power cables for
the devices are unplugged before the signal cables are connected. If possible,
disconnect all power cables from the existing system before you service.
• If the power supply is broken, do not try to x it by yourself. Contact a qualied
service technician or your dealer.
Operation Safety
• Servicing of this product or units is to be performed by trained service
personnel only.
• Before operating the server, carefully read all the manuals included with the
server package.
• Before using the server, make sure all cables are correctly connected and the
power cables are not damaged. If any damage is detected, contact your dealer
as soon as possible.
• To avoid short circuits, keep paper clips, screws, and staples away from
connectors, slots, sockets and circuitry.
• Avoid dust, humidity, and temperature extremes. Place the server on a stable
surface.
This product is equipped with a three-wire power cable and plug for the user’s
safety. Use the power cable with a properly grounded electrical outlet to avoid
electrical shock.
Lithium-Ion Battery Warning
CAUTION! Danger of explosion if battery is incorrectly replaced.
Replace only with the same or equivalent type recommended by the
manufacturer. Dispose of used batteries according to the manufacturer’s
instructions.
CD-ROM Drive Safety Warning
CLASS 1 LASER PRODUCT
Heavy System
CAUTION! This server system is heavy. Ask for assistance when moving or
carrying the system.

This chapter describes the key features
of ESC4000 Series. It includes
the product overview and general
specications.
Chapter 1
Product introduction

Chapter 1: Product introduction1-2
1.1 Key features
The ASUS ESC4000 is a high density GPU server presented in a compact 2U
rack mount with support for dual Intel®Xeon® 5600 series processors, 18 DIMM
slots with up to 48GB/144GB memory and 4 double-wide NVIDIA® Tesla™ GPU
computing processors. It enables high-end professional applications such as
life/medical sciences, engineering and sciences, nancial modeling, EDA and
visualization. The ESC4000 packs outstanding features, exible design and
versatile hardware, making it an ideal choice for intense parallel computing tasks.
Leading advanced system architecture
• Features dual all-new Intel®Xeon® 5600 series processors with Intel®Core™
microarchitecture for signicant performance increases
• Supports 18 DIMM slots and up to 48GB/144GB high scalability memory
for intense applications. DDR3 modules deliver higher speeds and more
bandwidth with lower power consumption
• Embeds up to four NVIDIA® Tesla™ M2050/M2070/M2090 GPUs on full PCI-E
Gen2 x16 slots with up to 2.66 teraops, doubling precision performance
• The NVIDIA® Tesla™ M2090 GPU Computing Module is the world’s fastest
parallel computing processor for HPC
8+1exibleexpansion
• Eight PCI-E Gen2 x16 slot design supports up to four double-deck GPU
computing cards for professional graphics
• Flex-E technology allows automatic switching between one PCI-E x16 and two
PCI-E x8 links
• Additional PCI-E riser module compatible with both half-length/low-prole
PCI-E cards and ASUS PIKE series cards
Intelligent system fan control
• Three independent smart system fans, two for GPU areas and one for general
motherboard cooling
• Adjusts fan speed automatically according to system load to optimize cooling
and energy saving
Flexiblehighstoragecapacity
• Designed with eight hard drive bays to support SAS or SATA drives
• Optional ASUS PIKE meets the various requirements of RAID functionality

ASUS ESC4000 Series 1-3
Highreliability
• 1400W 80+ Gold power supply delivers reliable and highly efcient power
• Hot-swap hard drive design allows downtime-free drive replacement
Remote control and management
• Bundled ASMB4-iKVM module offers advanced server management
• KVM-over internet provides OS-independent manageability
• Ideal for remote ofce applications

Chapter 1: Product introduction1-4
1.2 System overview
Items Items
1. Hard disk drive bay 2. Hard disk drive
3. Computing module bracket 4. Computing module
5. Computing module airduct 6. CPU
7. CPU heatsink 8. ASMB4-iKVM controller card
9. DDR3 DIMM 10. PCI-E x16 riser card
11. Top cover 12. Rackmount rail kit
9
34
5
11
10
12
12
6
7
8
2
1

ASUS ESC4000 Series 1-5
1.3 Front panel features
1.4 Rear panel features
*The port is for ASUS ASMB4-iKVM controller card only.
ODD USB ports
Power
button
Power LED
1
2
LAN2 LED
HDD Access LED
LAN1 LED
Message LED
Location switch
Resetbutton
Location LED
HDD 1 HDD 5 HDD 2 HDD 6 HDD 3 HDD 7 HDD 4 HDD 8
USB ports
LAN port 1
Serial port
VGA port
4 Full-length
Expansionslots
LAN port 2
Redundant
power supply
LAN port 3*
4 Full-length
Expansionslots
Half-length/Low-prole
Expansionslot
Power cord
connector
InniBandport(forESC4000/IBonly)
Power cord
connector
Redundant
power supply

Chapter 1: Product introduction1-6
1.5 Systemspecications
(continued on the next page)
Model Name ESC4000 Series
Processor / System Bus
2 x Socket LGA1366
Six/Quad-Core Intel®Xeon® X5600 Series
(130W/95W)
Quad-Core Intel®Xeon® X5500 Series (95W)
Quad-Core Intel®Xeon® E5500 Series (80W)
Quad-Core Intel®Xeon® L5500 Series (60W/38W)
QPI 4.8 / 5.86 / 6.4 GT/s
Core Logic
Intel® 5520 IOH
Intel® ICH10R I/O Controller
NVIDIA®nForce® 200 x 2
ASUS Features
Fan Speed
Control √
ASWM 2.0 √
Memory
Total Slots 18 (3-channel per CPU, 3 DIMMs per channel)
Capacity Maximum up to 144GB (RDIMM)
Maximum up to 48GB (UDIMM)
Memory Type DDR3 1333 / 1066 / 800 Reg DIMM / Unbuffered
DIMM with ECC
Memory Size 1GB, 2GB, 4GB and 8GB (RDIMM)
1GB, 2GB and 4GB (UDIMM)
ExpansionSlots
Total PCI/PCI-X/
PCI-E Slots 9
Slot Type
Full-length
- 8 x PCI-E x16 (4 at Gen2 x16 Link or 8 at Gen2
x8 Link) (Slot 1, 3, 5, 7 auto switch to x8 Link if
slot 2, 4, 6, 8 is occupied)
Half-length / Low-prole
- 1 x PCI-E x16 (Gen1 x4 Link) (PIKE SAS Card
for Storage Enhancement)
Additional Slot PIKE Riser Card
Storage
SATA Controller
Intel®ICH10R:
- 6 x SATA2 300MB/s ports
- Intel®Matrix Storage (for Windows only)
- Supports software RAID 0, 1, 5 & 10
LSIMegaRAID(forLinux/Windows):
- Supports software RAID 0, 1 & 10
SAS Controller
Optional:
ASUS PIKE 1068E 8-port SAS RAID card
ASUS PIKE 1078 8-port SAS HW RAID card
ASUS PIKE 6480 8-port SAS RAID card
ASUS PIKE 2008 8-port SAS2 6G RAID card
ASUS PIKE 2008/IMR 8-port SAS2 6G RAID card
ASUS PIKE 2108 8-port SAS2 6G H/W RAID card

ASUS ESC4000 Series 1-7
*Specicationsaresubjecttochangewithoutnotice.
HDD Bays
I = Internal
AorSwillbe
hot-swappable
8 x Hot-swap 3.5” SATAII/SAS HDD Bays
Networking LAN 2 x Intel 82574L + 1 x Mgmt LAN
Graphic VGA Aspeed AST2050 / 8MB
AuxiliaryStorageDeviceBay
(Floppy/OpticalDevice)
1 x slim-type Optical Device Bay
(Options: No Device / DVD-RW)
OnboardI/O
1 x External Serial Port
3 x RJ-45 ports (1 for ASMB4-iKVM)
4 x USB 2.0 ports (Front x 2, Rear x 2)
1 x Internal A Type USB Port
1 x VGA port
1 x QSFP Port for Inniband QDR (@ DDR speed,
for ESC4000/IB only)
OS Support
Windows® Server 2008 R2
Windows® Server 2008 Enterprise 32 / 64-bit
Windows® Server 2003 R2 Enterprise 32 / 64-bit
RedHat® Enterprise Linux AS5.5 32 / 64-bit
SuSE® Linux Enterprise Server 11 32 / 64-bit
(Subject to change without any notice)
Anti-virus Software Optional Anti-virus Software CD
Management
Solution
Out of Band
Remote
Hardware
ASMB4-iKVM for KVM-over-IP support (Bundled)
Software ASUS ASWM 2.0
Dimension(HHxWWxDD) 750mm x 444mm x 88mm (2U)
NetWeightKg(CPU,DRAM&
HDDnotincluded) 18 Kg
Power Supply 1400W (80+) 1+1 Redundant Power Supply (Default
with one Power Supply Module)
Power Rating Input: 100—140/180—240 Vac, 12—9.5A/9.5A—7A,
50—60Hz Class 1
Environment
Operation temperature: 10°C–35°C / Non operation
temperature: -40°C–70°C
Non operation humidity: 20%–90% ( Non-
condensing)

Chapter 1: Product introduction1-8

This chapter lists the key components
and optional accessories for the server
system.
Chapter 2
Components

Chapter 2: Components2-2
2.1 Upgrading CPU and CPU heatsink
ASUS ESC4000 Series supports two Intel® LGA1366 Xeon®processors per Node.
We recommend to install CPUs with the same frequency.
2
1
4
5
3
6

ASUS ESC4000 Series 2-3
Order P/N Description
90-S000U0BS0T Six Core Intel Xeon X5690, 3.46GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08Z0T Six Core Intel Xeon X5680, 3.33GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0BT0T Six Core Intel Xeon X5675, 3.06GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08Y0T Six Core Intel Xeon X5670, 2.93GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08X0T Six Core Intel Xeon X5660, 2.80GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08W0T Six Core Intel Xeon X5650, 2.66GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0BU0T Quad Core Intel Xeon E5649, 2.53GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0BV0T Quad Core Intel Xeon E5645, 2.40GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08V0T Quad Core Intel Xeon E5640, 2.66GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08U0T Quad Core Intel Xeon E5630, 2.53GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U08T0T Quad Core Intel Xeon E5620, 2.40GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0BW0T Quad Core Intel Xeon E5607, 2.26GHz, QPI 4.8GT/s, 8MB L2,
Single Pack with 1U Heatsink
90-S000U0BX0T Quad Core Intel Xeon E5606, 2.13GHz, QPI 4.8GT/s, 8MB L2,
Single Pack with 1U Heatsink
90-S000U0BY0T Quad Core Intel Xeon E5603, 1.60GHz, QPI 4.8GT/s, 4MB L2,
Single Pack with 1U Heatsink
90-S000U0BZ0T Quad Core Intel Xeon X5687, 3.60GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0990T Quad Core Intel Xeon X5677, 3.46GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0C10T Quad Core Intel Xeon X5672, 3.20GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U09A0T Quad Core Intel Xeon X5667, 3.06GHz, QPI 6.4GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U0C20T Quad Core Intel Xeon X5647, 2.93GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U09Q0T Six Core Intel Xeon L5640, 2.26GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U09B0T Quad Core Intel Xeon L5630, 2.13GHz, QPI 5.86GT/s, 12MB L2,
Single Pack with 1U Heatsink
90-S000U09C0T Quad Core Intel Xeon L5609, 1.86GHz, QPI 4.8GT/s, 12MB L2,
Single Pack with 1U Heatsink

Chapter 2: Components2-4
2.2 Upgrading system memory
The motherboard comes with eighteen (18) Double Data Rate 3 (DDR3) Dual Inline
Memory Modules (DIMM) sockets.
1
2
3
4
5
DIMM_A3
DIMM_A2
DIMM_A1
DIMM_B3
DIMM_B2
13
14
11
10
12
DIMM_D3
DIMM_D2
DIMM_D1
DIMM_E3
DIMM_E2
6DIMM_B1 15 DIMM_E1
7DIMM_C3 16 DIMM_F3
8DIMM_C2 17 DIMM_F2
9DIMM_C1 18 DIMM_F1
1
2
3
4
5
6
7
8
12 13 14 15 16 17 18
11
10
9
You may install 1 GB, 2 GB, 4 GB and 8 GB registered DDR3 DIMMs with ECC
or 1GB, 2GB and 4GB unbuffered DDR3 DIMMs with ECC into the DIMM sockets
using the memory congurations in this section.
• Always install DIMMs with the same CAS latency. For optimum
compatibility, it is recommended that you obtain memory modules from the
same vendor.
• For CPU1 conguraton, when installing only one DIMM, install it to the
orange slots labeled DIMM_A1, DIMM_B1, or DIMM_C1.
• For CPU2 conguraton, when installing only one DIMM, install it to the
orange slots labeled DIMM_D1, DIMM_E1, or DIMM_F1.
Order P/N Description
90-S000I05K0T DDR3 1333 ECC REG 2G 240P SINGLE PACK
90-S000I05M0T DDR3 1333 ECC UNB 2G 240P SINGLE PACK
90-S000I0570T DDR3 1333 ECC UNB 4G 240P SINGLE PACK
90-S000I05H0T DDR3 1333 ECC REG 4G 240P SINGLE PACK
90-S000I05I0T DDR3 1066 ECC REG 8G 240P 4RANK SINGLE PACK

ASUS ESC4000 Series 2-5
Memorypopulationtable
For UDIMM and Quad Ranks RDIMM:
CPU1Conguration
DIMM_A3 DIMM_A2 DIMM_A1 DIMM_B3 DIMM_B2 DIMM_B1 DIMM_C3 DIMM_C2 DIMM_C1
1 DIMM -- -- -- -- -- -- -- --
2 DIMMs -- -- -- -- -- -- --
3 DIMMs -- -- -- -- -- --
4 DIMMs -- -- -- -- --
5 DIMMs -- -- -- --
6 DIMMs -- -- --
CPU2Conguration
DIMM_D3 DIMM_D2 DIMM_D1 DIMM_E3 DIMM_E2 DIMM_E1 DIMM_F3 DIMM_F2 DIMM_F1
1 DIMM -- -- -- -- -- -- -- --
2 DIMMs -- -- -- -- -- -- --
3 DIMMs -- -- -- -- -- --
4 DIMMs -- -- -- -- --
5 DIMMs -- -- -- --
6 DIMMs -- -- --
ForRDIMM(SingleRank,DualRanks):
CPU1Conguration
DIMM_A3 DIMM_A2 DIMM_A1 DIMM_B3 DIMM_B2 DIMM_B1 DIMM_C3 DIMM_C2 DIMM_C1
1 DIMM -- -- -- -- -- -- -- --
2 DIMMs -- -- -- -- -- -- --
3 DIMMs -- -- -- -- -- --
4 DIMMs -- -- -- -- --
5 DIMMs -- -- -- --
6 DIMMs -- -- --
7 DIMMs -- --
8 DIMMs
9 DIMMs
CPU2Conguration
DIMM_D3 DIMM_D2 DIMM_D1 DIMM_E3 DIMM_E2 DIMM_E1 DIMM_F3 DIMM_F2 DIMM_F1
1 DIMM -- -- -- -- -- -- -- --
2 DIMMs -- -- -- -- -- -- --
3 DIMMs -- -- -- -- -- --
4 DIMMs -- -- -- -- --
5 DIMMs -- -- -- --
6 DIMMs -- -- --
7 DIMMs -- --
8 DIMMs --
9 DIMMs

Chapter 2: Components2-6
2.3 Upgrading hard disk drives
The system supports eight hot-swap SATAII/SAS hard disk drives.
We recommend that you install identical drives of the same model and capacity
for RAID conguration.
Order P/N Description
90-S000H65D1T SATA3 500G 7200rpm with tray (single pack)
90-S000H65C1T SATA3 750G 7200rpm with tray (single pack)
90-S000H65B1T SATA3 1TB 7200rpm with tray (single pack)
90-S000H64R0T SAS 6G 300G 15Krpm with tray (single pack)
90-S000H64S0T SAS 6G 450G 15Krpm with tray (single pack)
90-S000H64T0T SAS 6G 600G 15Krpm with tray (single pack)

ASUS ESC4000 Series 2-7
2.4 Installing ASUS Server Management
Board
ASUS Server Management Board is an Intelligent Platform Management Interface
(IPMI) 2.0-compliant board that allows users to monitor, control and manage a
remote server from the local or central server in local area network (LAN).
ASMB4-iKVM
• Monitor the running status of major components via LAN or Serial Port, such
as CPU and memory utilization, storage capacity and networking transfer rate.
• Advanced server management functions including “remote power on/off/reset”,
“remote control via SOL (Serial-over LAN)” and “remote SEL (system event log
reading when the system is out-of-band)”.
• Support Dynamic Host Conguration Protocol (DHCP), which avoids
manually assigning IP address by setting DHCP mode to receive IP address
automatically.
Use the LAN3 port for
server management.
LAN3 port
BaseboardManagementController(BMC)
connector(14-pinBMC_FW1)
ASUS Server Management Board

Chapter 2: Components2-8
2.5 Installing ASUS PIKE RAID module
The ASUS PIKE RAID module allows users to create RAID function from SAS hard
disk drives connected to the SAS connectors on the motherboard.
Order P/N Description
90-C1SE10-00UAY20Z PIKE 2008 8-port SAS 6G RAID Kit (RAID 0, 1, 1E)
90-C1SE15-00UAY00Z PIKE 2008/IMR 8-port SAS 6G RAID Kit (RAID 0, 1, 5)
90-C1SEF0-00UAY00Z PIKE 2008 8-port SAS 6G RAID Kit (RAID 0, 1, 5,6,10, 50, 60)
1. Firmly hold the riser card bracket, and then pull it up to detach it from the PCI
Express x16 slot on the motherboard.
2. Align and insert the golden ngers of the PIKE SAS RAID card into the card
slot on the PIKE raiser card. Ensure the card is completely seated on the slot.
3. Remove the SATA/SAS cables from the onboard SATA connectors and
connect the SATA/SAS cables to the SAS1–8 connectors on the PIKE riser
card.`
4. Connect the SGPIO2 cable (with a green label) to the SGPIO1 connector and
the SGPIO3 cable (without a label) to the SGPIO2 connector on the PIKE
riser card.
5. Install the i Button*.
* PIKE 1078 module won’t function if the i Button is not installed.
6. Align the riser card bracket to the PCI Express x16 slot on the motherboard
and press the riser card bracket until the golden ngers completely t the slot
and the bracket aligns with the rear panel.
1
5
4
3
3
6
2