NEC A1160 Installation guide

EXPRESS5800/A1160













Technical Overview 

Proprietary Notice and Liability Disclaimer
The information disclosed in this document, including all designs and related materials, is the
valuable property of NEC Corporation of America, Inc. and/or its licensors. NEC Corporation
of America and/or its licensors, as appropriate, reserve all patent, copyright and other
proprietary rights to this document, including all design, manufacturing, reproduction, use,
and sales rights thereto, except to the extent said rights are expressly granted to others.
The NEC Corporation of America product(s) discussed in this document are warranted in
accordance with the terms of the Warranty Statement accompanying each product. However,
actual performance of each product is dependent upon factors such as system configuration,
customer data, and operator control. Since implementation by customers of each product
may vary, the suitability of specific product configurations and applications must be
determined by the customer and is not warranted by NEC Corporation of America.
To allow for design and specification improvements, the information in this document is
subject to change at any time, without notice. Reproduction of this document or portions
thereof without prior written approval of NEC Corporation of America is prohibited.
Trademarks
Windows is a registered trademark of Microsoft Corporation.
Intel and Itanium are registered trademarks of Intel Corporation.
All other product, brand, or trade names used in this publication are the trademarks or
registered trademarks of their respective trademark owners.
PN: 456-01810-000 December, 2008
Copyright 2008
NEC Corporation of America
10850 Gold Center Drive, Suite 200,
Rancho Cordova, CA 95670
All Rights Reserved

iii
Contents
Section 1 Introduction ..................................................................1-1
Documentation Updates .................................................................................. 1-1
Scalable Symmetric Multiprocessing Servers.................................................. 1-1
Operating Systems .......................................................................................... 1-2
Virtual Machine Monitors ................................................................................. 1-3
Section 2 Cell Component............................................................2-1
Express5800/A1160 Cell.................................................................................. 2-1
Processors....................................................................................................... 2-2
Memory............................................................................................................ 2-5
I/O Subsystem ................................................................................................. 2-8
Power............................................................................................................. 2-12
Cooling........................................................................................................... 2-14
Peripherals..................................................................................................... 2-15
System Clocks ............................................................................................... 2-17
EXPRESSSCOPE® Monitor.......................................................................... 2-17
Fault Messages on the EXPRESSSCOPE® Monitor LCD............................2-18
Inbuilt Virtualization........................................................................................ 2-20
Trusted Platform Module (TPM).....................................................................2-20
Section 3 System Configuration..................................................3-1
Cell Interconnect and Multiple Cell Systems.................................................... 3-1
Cell Identification.............................................................................................. 3-3
System Serial Number..................................................................................... 3-3
Cabinets........................................................................................................... 3-4
Partitions.......................................................................................................... 3-5
Clustering......................................................................................................... 3-5
Redundant Configurations ............................................................................... 3-6
Replaceable Components................................................................................ 3-6
Express5800/A1160 Configuration Summary .................................................. 3-7
Section 4 Server Management .....................................................4-1

Contents
iv
System Components........................................................................................ 4-1
Network Components ...................................................................................... 4-2
Management Architecture................................................................................ 4-4
Management Board ......................................................................................... 4-5
Remote Console Interfaces ............................................................................. 4-7
BIOS ................................................................................................................ 4-8
Server Management Software......................................................................... 4-8
Hardware Components.................................................................................... 4-8
Management Software Components ............................................................... 4-9
Remote Monitoring ........................................................................................ 4-10
Appendix ACell Components......................................................... A-1
Components on the Front of a Cell..................................................................A-1
Components on the Rear of a Cell...................................................................A-2

Figures
v
Figures
Figure 1-1 Express5800/A1160 Cell .............................................................................1-1
Figure 2-1 Basic Cell.....................................................................................................2-2
Figure 2-2 Processor Socket Location on Processor Board.........................................2-3
Figure 2-3 Node Controller ...........................................................................................2-5
Figure 2-4 Memory Boards...........................................................................................2-6
Figure 2-5 DIMM Numbering ........................................................................................2-7
Figure 2-6 I/O Riser Board............................................................................................2-9
Figure 2-7 PCIe Card Carrier......................................................................................2-10
Figure 2-8 External Card Slots....................................................................................2-10
Figure 2-9 Ports on Back of the Cell...........................................................................2-11
Figure 2-10 Power Supplies........................................................................................2-12
Figure 2-11 Fan Modules............................................................................................2-15
Figure 2-12 Peripherals on the Drive Tray..................................................................2-16
Figure 2-13 EXPRESSSCOPE® Monitor ...................................................................2-18
Figure 3-1 Interconnect Cables and Interconnect Cable Ports.....................................3-1
Figure 3-2 System with Two Cells.................................................................................3-2
Figure 3-3 System with Four Cells................................................................................3-2
Figure 3-4 System Serial Number Label.......................................................................3-4
Figure 3-5 Cabinet Configuration..................................................................................3-5
Figure 4-1 Management Architecture............................................................................4-5
Figure 4-2 Management Board.....................................................................................4-6
Figure 4-3 Server Management Hardware Components..............................................4-9
Figure A-1 Front of the Cell.......................................................................................... A-1
Figure A-2 Rear of the Cell ..........................................................................................A-3

Tables
vii
Tables
Table 2-1 DIMM Capacity and Cell Memory .................................................................2-7
Table 2-2 Device Names on the EXPRESSSCOPE® Monitor LCD ...........................2-18
Table 3-1 Relation of Interconnect Port and Cable End to Cell ID................................3-3
Table 3-2 Minimum and Maximum Configurations........................................................3-7
TableA-1 Component on the Front of the Cell.............................................................A-1
TableA-2 Component on the Rear of the Cell ............................................................. A-3

Using This Guide
ix
Using This Guide
This guide contains information how to operate and administrate Express5800/A1160
server.
Who Should Use This Guide
This guide is intended for system administrators and operation personnel who are using
Express5800/A1160 server.
Symbols and Conventions
This guide uses the following text conventions and graphic symbols.
Warnings, cautions, and notes have the following meanings:
WARNING
Warnings alert you to situations that could result in serious personal injury or loss
of life.
CAUTION
Cautions indicate situations that can damage the system hardware or software.
Note: Notes give important information about the material being described.
Names of keyboard keys are printed as they appear on the keyboard. For example, Ctrl,
Alt, or Enter.
Text or keystrokes that you enter appear as boldface type. For example, type abc123
and press ENTER.
File names are printed in uppercase letters. For example, AUTOEXEC.BAT.
Related Documents
In addition to this guide, the following system documentation is useful.
NECCare™ Guide

Related Documents
x
The NECCare Guide contains information about NEC’s warranty and server registration.

Safety Notices
xi
Safety Notices
WARNING
To avoid a risk of injuries, maintenance procedures require trained technical
personnel.
In maintenance procedures with voltages of 42.4V peak or 60Vdc or more, take
safety measures, such as wearing insulated rubber gloves. Performing work
without these measures may cause electric shock.
In an emergency, such as a dangerous event that requires turning off the power
supply, turn off the breaker at the rear of the server. Turning off the breaker may
cause data destruction. Therefore, users should determine when to turn off the
breaker in accordance with specified operation criteria.
The server is equipped with a front stabilizer. Engage the front stabilizer during
installation. For stability and to distribute the weight, also attach side stabilizers.
Otherwise, the rack may topple over and cause injuries.
If you extend two or more devices from the rack at the same time, the rack may
topple over on you. Extend only one device from the rack at a time.
Exercise great care not to hurt your fingers on the rail when you mount/dismount
the equipment into/from the rack.
Lithium batteries can be dangerous. Improper handling of lithium batteries may
result in an explosion. Dispose of lithium batteries as required by local ordinance.
Replace only with the same or equivalent type battery.
A liquid crystal display is used in this server. When handling a damaged liquid
crystal display, take care to avoid exposure to the liquid inside the liquid crystal
display. The liquid can cause bodily harm. In the event the liquid is ingested,
gargle at once and consult a doctor immediately. If the liquid comes in contact
with skin or gets into the eyes, wash the skin with cool running water, or flush the
eye with cool running water for at least 15 minutes and consult a doctor.
The DVD-ROM drive uses a laser beam. Do not look or insert a mirror inside
while the system is on. A laser beam is invisible; if your eyes get exposed to it,
there is a risk of losing your eyesight.
Elevated Operating Ambient Temperature – If installed in a closed or multi-unit rack
assembly, the operating ambient temperature of the rack environment may be greater
than the room ambient environment. Therefore, consideration should be given to
installing the equipment in an environment compatible with the maximum rated ambient

Safety Notices
xii
temperature of 89.6°F.
Reduced air Flow – Installation of the equipment in a rack should be such that the
amount of air flow required for safe operation of the equipment is not compromised.
To prevent fires, and damage to rack equipment and supply wiring, make sure that the
rated load of the power branch circuit is not exceeded. Equipment nameplate ratings
should be used when addressing this concern. For more information on installation and
wiring of power-related facilities, contact your electrician or local power company.
To prevent electrical shock, connect all rack and rack support equipment to the same
electrical circuit of the building wiring. If you are unsure, check the building wiring to
avoid remote earth conditions.
For safe operation, only connect the equipment to a building supply that is in accordance
with current wiring regulations in your country. In the USA those wiring standards are
regulated by Underwriter Laboratories (UL); in the U.K. by the Institution of Electrical
Engineers, (IEE) and in Canada by the Canadian Standards Association (CSA).
WARNING
Some locations within the server have high voltage and therefore are very
dangerous. To avoid risk of electric shock, turn off all server power and
disconnect power cables before working inside the server unit.
The main power of your server is turned off by turning off the power source to the
server or removing the power cable.
Before touching the parts in the server, wait for at least 10 to 15 seconds until
residual voltage is discharged.
Online maintenance – During and after servicing, do not leave the server door open
unless necessary to perform servicing.

xiii
WARNING
Take care not to short live components with conductive tools, such as an
adjustable wrench.
To prevent shock, take care not to drop or leave conductive parts, such as a
screw, in the server when servicing the system.
Be careful when accessing a fan or rotating parts to avoid cutting your hand or
fingers.
Safety inspections – When servicing the system, check equipment that can cause harm
due to deterioration, and if necessary, replace the part.
Safety Notices for Users Outside of the U.S.A. and Canada
PELV (Protected Extra-Low Voltage) Integrity: To ensure the extra-low voltage integrity
of the equipment, connect only equipment with mains-protected electrically-compatible
circuits to the external ports.
Remote Earths: To prevent electrical shock, connect all local (individual office)
computers and computer support equipment to the same electrical circuit of the building
wiring. If you are unsure, check the building wiring to avoid remote earth conditions.
Earth Bonding: For safe operation, only connect the equipment to a building supply that
is in accordance with current wiring regulations in your country. In the USA those wiring
standards are regulated by Underwriter Laboratories (UL); in the U.K., by the Institution
of Electrical Engineers, (IEE) and in Canada by the Canadian Standards Association
(CSA).

1-1
Section 1
Introduction
This overview provides a detailed introduction to Express5800/A1160 systems and the
technologies on which they are based. It describes features that might be available to
customers in a series of hardware and software releases. For information about the
availability of any feature, contact your sales representative.
Documentation Updates
This document contains all the information that was available at the time of publication.
The latest version of the document may be found in the Product Support Web Site:
http://support.necam.com/servers/Enterprise/
Scalable Symmetric Multiprocessing Servers
Express5800/A1160 systems are the next generation of symmetric multiprocessing
(SMP) servers that support multi-core Intel Xeon processors.
Figure 1-1 Express5800/A1160 Cell
Basic Building Block
Express5800/A1160 systems employ a modular building-block design. The basic
building block is a cell—a standard 19-inch 4U rack-mounted server. A cell contains four
processor sockets, six Peripheral Component Interconnect Express (PCIe) slots, 32

Operating Systems
1-2
dual inline memory module (DIMM) sockets that support 2-GB or 4-GB fully buffered
DIMMs, and all the other components it needs to function as a self-contained computer
system.
Modular Expansion
The cell-based building-block design allows Express5800/A1160 systems to grow as
business needs evolve. By connecting cells together, you can expand an Express5800
Model 7600R system from 4 to 16 processor sockets. Memory expands from 2 to 128
DIMM sockets. With 4-GB fully buffered DIMMs, a cell can contain up to 128 GB of
memory, and a 4-cell system can contain up to 512 GB of memory. The I/O subsystem
expands from 6 to 24 PCIe slots and adding PCIe expansion modules can further
increase the number of I/O slots. Hardware to connect the cells together is added only
as the server size increases.
System Partitioning
Each cell in an Express5800/A1160 system can function as an independent partition. In
multiple-cell systems, two or more cells can be combined into a single partition. Each
partition can run an operating system different from the operating system in any other
partition.
Customer Installation and Servicing
Express5800/A1160 systems are designed to be installed by the customer without any
special tools or training. Likewise, no special tools or training are required to service
customer-replaceable units (CRUs).
Reliability, Availability, and Serviceability
Express5800/A1160 systems contain many features to detect and isolate faults.
Systems with two or more cells provide 99.995% or better hardware availability.
Distributed System Management
Management controllers within each cell of an Express5800/A1160 system cooperate to
provide management of the system. Hosted remotely, a server management application,
such as NEC Server Management software, can monitor and control the system.
Operating Systems
Windows Operating Systems
Express5800/A1160 systems support the following Windows operating systems:
yWindows Server 2003 R2 with Service Pack 2, 32-bit Enterprise Edition
yWindows Server 2003 R2 with Service Pack 2, 32-bit Datacenter Edition
yWindows Server 2003 R2, Enterprise x64 Edition
yWindows Server 2003 R2, Datacenter x64 Edition

Virtual Machine Monitors
1-3
yWindows Server 2008, Enterprise x64 Edition
yWindows Server 2008, Datacenter x64 Edition
Note: Partitions running Windows Server 2003, 32-bit Enterprise Edition, or
Windows Server 2003, 32-bit Datacenter Edition, are restricted to a maximum of
two cells.
Linux Operating Systems
Express5800/A1160 systems support the following Linux operating systems:
yRed Hat Enterprise Linux Advanced Platform 5.2 for x64
ySUSE LINUX Enterprise Server 10 (x64 with Service Pack 2)
Virtual Machine Monitors
A virtual machine monitor is the base component that enables virtualization. It is a
logical computer instance based on permanent physical hardware that runs specialized
software to host virtual machines and service partitions.
Express5800/A1160 systems support the following virtual machine monitors:
yVMware Infrastructure 3 (VMware ESX Server 3.5 update 2 or later)
yMicrosoft Hyper-V

2-1
Section 2
Cell Component
Express5800/A1160 Cell
In Express5800/A1160 systems, hardware components are organized into cells. The
main components of a cell are
•Main logic module that hosts the processors, processor voltage regulator modules,
and node controller. (The node controller interconnects the components of a cell
and provides the connection among the cells in a multiple-cell system.)
•Memory board that contains the DIMM sockets for the fully buffered DIMMs.
•I/O subsystem that includes the I/O riser board, PCIe carriers, and PCIe cards.
•Power supplies.
•Fan modules.
•Peripherals including drive tray and hard drives, DVD drive, and Serial Attached
SCSI (SAS) RAID controller.
•Management and maintenance components (EXPRESSSCOPE® Monitor and
management board).
Figure 2-1 illustrates the main components of a cell.

Processors
2-2
Figure 2-1 Basic Cell
Processors
Within each cell, the processor board in the main logic module hosts the processors.
The processor board contains the processor sockets, processor voltage regulator
modules, node controller, and the scalability ports, which are the ports that enable you to
connect up to four cells.
Processor Sockets
Each cell contains four processor sockets. Four direct front-side buses provide the
connections for the four processor sockets. Each direct front-side bus operates at 1067
MT/s.