Silicon Graphics Virtu VN200 Installation manual

SGI®Virtu VN200 Graphics Node
Hardware User’s Guide
007-5483-001

COPYRIGHT
© 2002-2008 SGI. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No permission is
granted to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole or in part,
without the prior written permission of SGI.
LIMITED RIGHTS LEGEND
The software described in this document is “commercial computer software” provided with restricted rights (except as to included open/free
source) as specified in the FAR 52.227-19 and/or the DFAR 227.7202, or successive sections. Use beyond license provisions is a violation of
worldwide intellectual property laws, treaties and conventions.This document is provided with limited rights as defined in 52.227-14.
TRADEMARKS AND ATTRIBUTIONS
Altix, Silicon Graphics, SGI, and the SGI logo, are registered trademarks of Silicon Graphics, Inc., in the United States and/or other countries
worldwide.
ATI is a trademark of Advanced Micro Devices, Inc. InfiniBand is a registered trademark of the InfiniBand Trade Association. Intel, Itanium, and
Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. InfiniBand is a
trademark of InfiniBand Trade Association. Internet Explorer is a registered trademark of Microsoft Corporation. Java and Java Virtual Machine
are trademarks or registered trademarks of Sun Microsystems, Inc. Linux is a registered trademark of Linus Torvalds, used with permission by
SGI. Mellanox is a registered trademark of Mellanox Technologies, Inc. in the United States and/or other countries. NVIDIA, the NVIDIA logo,
Quadro, and TwinView are registered trademarks or trademarks of NVIDIA Corporation in the United States and/or other countries. Phoenix
and PhoenixBIOS are registered trademarks of Phoenix Technologies Ltd. Red Hat and all Red Hat-based trademarks are trademarks or
registered trademarks of Red Hat, Inc. in the United States and other countries. ScaliManage is a trademark of Scali Inc. Supermicro and Supero
Doctor III are registered trademarks or trademarks of Super Micro Computer, Inc. SUSE LINUX and the SUSE logo are registered trademarks of
Novell, Inc. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.
Windows is a registered trademark of Microsoft Corporation.
All other trademarks mentioned herein are the property of their respective owners.

007-5483-001 iii
Record of Revision
Version Description
001 April, 2008.
Original printing.

iv 007-5483-001

007-5483-001 v
Contents
About This Guide
Audience . . . . . . . . . . . . . . . . . . . . . . xiii
Structure of this Guide . . . . . . . . . . . . . . . . . . xiii
Related Publications . . . . . . . . . . . . . . . . . . . xiv
Conventions . . . . . . . . . . . . . . . . . . . . . xv
Product Support . . . . . . . . . . . . . . . . . . . . xv
Reader Comments . . . . . . . . . . . . . . . . . . . . xvi
1 SGI Virtu VN200 Overview
Product Description . . . . . . . . . . . . . . . . . . . . 2
Warnings and Cautions . . . . . . . . . . . . . . . . . . . 4
Site Planning . . . . . . . . . . . . . . . . . . . . . . 5
Selecting a Delivery Route . . . . . . . . . . . . . . . . . 5
Selecting a Final Location . . . . . . . . . . . . . . . . . 6
Unpacking the Rack . . . . . . . . . . . . . . . . . . . . 7
Unpacking the Graphics Nodes . . . . . . . . . . . . . . . . . 9
Installing the Rack Shelf . . . . . . . . . . . . . . . . . . 10
System Configurations and Cabling. . . . . . . . . . . . . . . 11
Graphics Node Dimensions . . . . . . . . . . . . . . . . . 14
Electrical Requirements . . . . . . . . . . . . . . . . . . 15
Power Consumption for the Graphics Node . . . . . . . . . . . 15
Environmental Requirements . . . . . . . . . . . . . . . . 16
Acoustics . . . . . . . . . . . . . . . . . . . . . 16
Electromagnetic Interference . . . . . . . . . . . . . . . 16
Electrostatic Discharge . . . . . . . . . . . . . . . . . 17

vi 007-5483-001
Regulatory Specifications . . . . . . . . . . . . . . . . . . 17
Server Model Number . . . . . . . . . . . . . . . . . 17
CE Notice and Manufacturer's Declaration of Conformity . . . . . . . 18
Electromagnetic Emissions . . . . . . . . . . . . . . . . 18
FCC Notice (USA Only) . . . . . . . . . . . . . . . . 18
Industry Canada Notice (Canada Only) . . . . . . . . . . . 19
VCCI Class A Notice (Japan Only) . . . . . . . . . . . . . 19
Shielded Cables . . . . . . . . . . . . . . . . . . . 19
2SystemFeatures
Server Racks. . . . . . . . . . . . . . . . . . . . . . 22
Virtu VN200 Graphics Node . . . . . . . . . . . . . . . . . 23
Front Panel Components and Status LEDs . . . . . . . . . . . . 24
Rear Panel Components . . . . . . . . . . . . . . . . . 25
Node Board . . . . . . . . . . . . . . . . . . . . . . 26
Node Board Features . . . . . . . . . . . . . . . . . . 27
Jumper Quick-reference . . . . . . . . . . . . . . . . . 30
Memory Overview . . . . . . . . . . . . . . . . . . 31
Memory DIMM Pairs. . . . . . . . . . . . . . . . . 33
Memory Configuration Rules . . . . . . . . . . . . . . 33
Memory Part Numbers . . . . . . . . . . . . . . . . 33
AOC-SIMSO (BMC) Card. . . . . . . . . . . . . . . . . 35
Power Supplies . . . . . . . . . . . . . . . . . . . 37
Cooling Fans . . . . . . . . . . . . . . . . . . . . 38
System Disk Drives . . . . . . . . . . . . . . . . . . 39
Graphics Cards. . . . . . . . . . . . . . . . . . . . 40
Accelerated Driver Installation . . . . . . . . . . . . . . . 41

007-5483-001 vii
3 System Procedures
Connecting a Digital Monitor . . . . . . . . . . . . . . . . 43
Connecting a Keyboard and Mouse . . . . . . . . . . . . . . . 44
Configuring Firmware and Drivers . . . . . . . . . . . . . . . 45
Power On/Off the Cluster . . . . . . . . . . . . . . . . . 45
Powering Off Manually . . . . . . . . . . . . . . . . . 46
Verifying the Graphics Card Installation for Linux . . . . . . . . . . 47
Starting the X Window System . . . . . . . . . . . . . . . . 48
Commands . . . . . . . . . . . . . . . . . . . . 48
Documentation . . . . . . . . . . . . . . . . . . . 49
Start Graphics . . . . . . . . . . . . . . . . . . . . 49
Stop Graphics . . . . . . . . . . . . . . . . . . . . 49
Virtual Consoles . . . . . . . . . . . . . . . . . . . 49
xorg.conf Configuration File. . . . . . . . . . . . . . . . 50
Monitor Section . . . . . . . . . . . . . . . . . . 53
Modelines. . . . . . . . . . . . . . . . . . . . 53
Device Section . . . . . . . . . . . . . . . . . . 54
TwinView. . . . . . . . . . . . . . . . . . . . 54
TwinView Edge-matched Display . . . . . . . . . . . . . 55
TwinView Edge Blending . . . . . . . . . . . . . . . 56
Stereoscopic Viewing . . . . . . . . . . . . . . . . 56
Setting up Multiple X Servers on a Single Graphics Card . . . . . . 57
ServerLayout Section . . . . . . . . . . . . . . . . 59
Check the NVIDIA Driver . . . . . . . . . . . . . . . . . 60
Re-installation of the Graphics Drivers. . . . . . . . . . . . . 60
Upgrading Drivers . . . . . . . . . . . . . . . . . . 61
Troubleshooting . . . . . . . . . . . . . . . . . . . . 61
General. . . . . . . . . . . . . . . . . . . . . . 61
Graphics . . . . . . . . . . . . . . . . . . . . . . 62
BMC Configuration . . . . . . . . . . . . . . . . . . 62
Power . . . . . . . . . . . . . . . . . . . . . . 63
Cooling. . . . . . . . . . . . . . . . . . . . . . 63

viii 007-5483-001
Hardware Replacement Procedures . . . . . . . . . . . . . . . 64
Before You Begin . . . . . . . . . . . . . . . . . . . 64
Tools and Supplies Needed . . . . . . . . . . . . . . . . 64
Removing the Front Bezel . . . . . . . . . . . . . . . . 64
Removing the Graphics Node from the Rack . . . . . . . . . . . 65
Removing the Side Cover. . . . . . . . . . . . . . . . . 66
Replacing the Power Supplies . . . . . . . . . . . . . . . 67
Replacing a Disk Drive . . . . . . . . . . . . . . . . . 68
Replacing the Chassis Fans . . . . . . . . . . . . . . . . 69
Replacing the Graphics Card. . . . . . . . . . . . . . . . 69
Replacing the AOC-SIMSO (BMC) Daughter Card . . . . . . . . . 71
Removing the PCI Riser Board . . . . . . . . . . . . . . . 72
Removing the Air Baffles . . . . . . . . . . . . . . . . . 73
Replacing Node Board Memory DIMMs . . . . . . . . . . . . 74
Replacing Node Board Processors . . . . . . . . . . . . . . 75
Install a Processor . . . . . . . . . . . . . . . . . . . 77
Install a Heatsink . . . . . . . . . . . . . . . . . . . 81
Remove the Heatsink . . . . . . . . . . . . . . . . . . 83
Replace the Thermal Interface Material . . . . . . . . . . . . . 83
Removing TIM from a Processor . . . . . . . . . . . . . 84
Removing TIM from a Heatsink . . . . . . . . . . . . . 87
Replacing the Node Board . . . . . . . . . . . . . . . . 90
Replacing the Control Panel . . . . . . . . . . . . . . . . 91

007-5483-001 ix
Figures
Figure 1-1 SGI Altix 1300 Cluster with Virtu VN200 Graphics Nodes. . . . 1
Figure 1-2 Unpacking the Rack. . . . . . . . . . . . . . . 8
Figure 1-3 Unpacking the Graphics Nodes . . . . . . . . . . . 9
Figure 1-4 Rack Shelf . . . . . . . . . . . . . . . . 10
Figure 1-5 Altix XE Cluster Configuration . . . . . . . . . . 12
Figure 1-6 SGI Altix ICE Cluster Configuration . . . . . . . . . 13
Figure 1-7 Graphics Node Dimensions . . . . . . . . . . . 14
Figure 2-1 SGI Virtu VN200 Graphics Nodes. . . . . . . . . . 21
Figure 2-2 SGI Virtu VN200 - Front View. . . . . . . . . . . 22
Figure 2-3 Virtu VN200 Graphics Node . . . . . . . . . . . 23
Figure 2-4 Front Panel Components . . . . . . . . . . . . 24
Figure 2-5 Rear Panel Components . . . . . . . . . . . . 25
Figure 2-6 VN200 Node Board . . . . . . . . . . . . . . 26
Figure 2-7 Node Board Component Locations . . . . . . . . . 26
Figure 2-8 Node Board Block Diagram . . . . . . . . . . . 29
Figure 2-9 Memory Block Diagram . . . . . . . . . . . . 32
Figure 2-10 Node Board DIMM Layout . . . . . . . . . . . 32
Figure 2-11 AOC-SIMSO (BMC) Daughter Card . . . . . . . . . 35
Figure 2-12 Redundant Power Supplies . . . . . . . . . . . 37
Figure 2-13 Cooling Fans. . . . . . . . . . . . . . . . 38
Figure 2-14 System Disk Drives . . . . . . . . . . . . . . 39
Figure 2-15 Graphics Card . . . . . . . . . . . . . . . 40
Figure 3-1 SGI Altix XE Cluster Configuration . . . . . . . . . 44
Figure 3-2 Xorg.conf File Organization . . . . . . . . . . . 50
Figure 3-3 TwinView Edge-Matched Display . . . . . . . . . 55
Figure 3-4 TwinView Edge Blending with 10% Overlap . . . . . . 56
Figure 3-5 Front Bezel Removal . . . . . . . . . . . . . 64
Figure 3-6 Removing the Graphics Node from a Rack . . . . . . . 65
Figure 3-7 Side Cover Removal . . . . . . . . . . . . . 66
Figure 3-8 Power Supply Removal . . . . . . . . . . . . 67

x007-5483-001
Figure 3-9 Disk Drive Removal . . . . . . . . . . . . . . 68
Figure 3-10 Cooling Fans Removal . . . . . . . . . . . . . 69
Figure 3-11 Graphics Card Removal . . . . . . . . . . . . . 70
Figure 3-12 BMC Daughter Card Removal . . . . . . . . . . . 71
Figure 3-13 PCI Riser Board Removal . . . . . . . . . . . . 72
Figure 3-14 Air Baffle Removal . . . . . . . . . . . . . . 73
Figure 3-15 DIMM Removal . . . . . . . . . . . . . . . 74
Figure 3-16 Node Board Removal . . . . . . . . . . . . . 75
Figure 3-17 Install a CPU Heatsink . . . . . . . . . . . . . 82
Figure 3-18 Air Baffles . . . . . . . . . . . . . . . . . 90
Figure 3-19 Control Panel Removal . . . . . . . . . . . . . 91

007-5483-001 xi
Tables
Table 1-1 SGI Virtu VN200 System Configurations . . . . . 11
Table 1-2 Graphics Node Power Consumption. . . . . . . 15
Table 2-1 Node Board Jumper Definitions . . . . . . . . 30
Table 2-2 Node Board Connector Definitions . . . . . . . 30
Table 2-3 Memory DIMM Pairs . . . . . . . . . . . 33
Table 2-5 DIMM Configurations . . . . . . . . . . . 34
Table 2-4 Memory Sizes and SGI Part Numbers . . . . . . 34
Table 2-6 Power Supply Specifications . . . . . . . . . 37
Table 2-7 Graphics Card Specifications . . . . . . . . . 40

xii 007-5483-001

007-5483-001 xiii
About This Guide
This guide provides information about how to install, configure, administrate, and repair
an SGI Virtu VN200.
Note: Refer to “Related Publications” on page xiv, for a listing of relevant SGI Technical
Publications that can provide more detailed information about SGI cluster head nodes,
compute nodes, system software, and rack hardware.
Audience
This guide is intended for system administrators. Use this guide to:
• Unpack and install the hardware.
• Gain a basic understanding of the SGI Virtu VN200 hardware and software
components.
Structure of this Guide
This guide contains the following chapters:
• Overview — Introduces the SGI Virtu VN200 product and discusses the system
level topics such as site planning, installation, and cabling.
• System Features — Describes the components of the SGI Virtu VN200
• Procedures — Describes the procedures for configuring the system and replacing
hardware.
• Refer to SGI Altix XE320 System User's Guide, publication number 007-5466-00x, for
more detailed information about node board BIOS settings, power-on self test
(POST) codes, and the baseboard management controller (BMC) intelligent
platform management interface (IPMI) management utility.

xiv 007-5483-001
Related Publications
Your SGI Virtu VN200 system ships with a variety of hardware and software documents
in both hard copy and soft copy formats. Hard copy documents are in the packing box
and soft copy documents are located on your system hard disk in
/usr/local/Factory-Install/Docs. Additional third-party documentation may be
shipped on removable media (CD/DVD) included with your shipment. You will need to
consult the following documentation while using this guide:
•SGI Altix XE Cluster Quick Reference Guide, publication number 007-4979-00x
This document is intended as an overview of some of the common operations that
system administrators may have to perform to set-up, boot, re-configure (upgrade)
or troubleshoot the SGI Altix XE cluster.
• SGI Altix XE320 System User's Guide, publication number 007-5466-00x
This guide provides instructions administrating SGI Altix XE320 compute nodes in
the SGI Virtu VN200. The SGI Virtu VN200 uses the same node board as the
SGI Altix 320 server, so you can also use this guide for information about BIOS
settings, power-on self test (POST) codes, and the baseboard management controller
(BMC) intelligent platform management interface (IPMI) management utility.
•SGI Altix XE250 System User’s Guide, publication number 007-5467-00x
This guide provides instructions administrating SGI Altix XE250 nodes in an
SGI cluster.
•SGI Altix XE System Rack Installation Instructions, publication number 007-4902-00x
This guide provides instructions unpacking and installing an SGI Altix XE cluster
rack.
•Guide to Administration, Programming Environments, and Tools Available on
SGI Altix XE Systems, publication number 007-4901-00x
This guide is a reference document for people who manage the operation of SGI
computer systems running SGI ProPack 5 for Linux operating system on Altix XE
systems. It explains how to perform general system configuration and operations
and describes programming environments and tools available for SGI Altix XE
systems.
• Scali Manage On SGI Altix ICE System Quick Reference Guide
• Scali Manage User’s Guide

007-5483-001 xv
Conventions
The following conventions are used throughout this document:
Product Support
SGI provides a comprehensive product support and maintenance program for its
products. SGI also offers services to implement and integrate Linux applications in your
environment.
• Refer to http://www.sgi.com/support/
• If you are in North America, contact the Technical Assistance Center at
[phone]SGI or contact your authorized service provider.
• If you are outside North America, contact the SGI subsidiary or authorized
distributor in your country.
Convention Meaning
Command This fixed-space font denotes literal items such as commands, files,
routines,pathnames, signals,messages,andprogramming language
structures.
variable The italic typeface denotes variable entries and words or concepts
being defined. Italic typeface is also used for book titles.
user input This bold fixed-space font denotes literal items that the user enters in
interactive sessions. Output is shown in nonbold, fixed-space font.
[ ] Brackets enclose optional portions of a command or directive line.
... Ellipses indicate that a preceding element can be repeated.
man page(x)Man page section identifiers appear in parentheses after man page
names.

xvi 007-5483-001
Reader Comments
If you have comments about the technical accuracy, content, or organization of this
document, contact SGI. Be sure to include the title and document number of the manual
with your comments. (Online, the document number is located in the front matter of the
manual. In printed manuals, the document number is located at the bottom of each
page.)
You can contact SGI in any of the following ways:
• Send e-mail to the following address: [email protected]
• Contact your customer service representative and ask that an incident be filed in the
SGI incident tracking system.
• Send mail to the following address:
•SGI
Technical Publications
1140 East Arques Avenue
Sunnyvale, CA 94085–4602
SGI values your comments and will respond to them promptly.

007-5483-001 1
Chapter 1
1. SGI Virtu VN200 Overview
This chapter provides an introduction to the SGI Virtu VN200 graphics node (refer to
Figure 1-1) and introduces information necessary to understand the system.
Figure 1-1 SGI Altix 1300 Cluster with Virtu VN200 Graphics Nodes
PS
PS
DDR
Vol
T
AIRE
Fan
ResetE
t
h
CLI
I2C
SM
In
fo
Pwr
I
SR9024S-M
SGI Virtu VN200 Graphics Nodes
SGI Altix XE320 Compute Nodes
SGI Altix XE250 Head Node
InfiniBand Switch
GigE Switch
System Console

2007-5483-001
SGI Virtu VN200 Overview
Product Description
This document is intended as an overview of some of the common operations that
system administrators may have to perform to set-up, boot, re-configure (upgrade) or
troubleshoot an SGI Altix XE cluster with SGI Virtu VN200 graphics nodes.
The initial release of the SGI Virtu VN200 system (refer to Figure 1-1) is one or more
SGI Altix XE320 servers (compute nodes), one or more SGI Altix XE250 servers (head
nodes), and one more SGI Virtu VN200 graphics nodes (visualization/compute nodes).
The nodes are networked together using InfiniBand or Gigabit Ethernet switches and can
run parallel programs using a message passing tool like the Message Passing Interface
(MPI).
An SGI Virtu VN200 system is a distributed memory server as opposed to a shared
memory system like that used in the SGI Altix 450 or SGI Altix 4700 high-performance
compute servers. Instead of passing pointers into a shared virtual address space, parallel
processes in an application pass messages, with each process requiring its own dedicated
processor and address space.
Just like a multi-processor shared memory system, a cluster can be shared among
multiple applications. For instance, one application may run on 16 processors in the
cluster while another application runs on a different set of 8 processors. Very large
clusters may run dozens of separate, independent applications at the same time.
Typically, each process of an MPI job runs exclusively on a processor. Multiple processes
can share a single processor, through standard Linux context switching, but this can have
a significant effect on application performance. A parallel program can only finish when
all of its sub-processes have finished. If one process is delayed because it is sharing a
processor and memory with another application, then the entire parallel program is
delayed. This gets slightly more complicated when systems have multiple processors
(and/or multiple cores) that share memory, but the basic rule is that a process is run on
a dedicated processor core.

Product Description
007-5483-001 3
The primary hardware components in an SGI Virtu VN200 system are:
• Head node(s) (SGI Altix XE250 servers)
• Compute nodes (SGI Altix XE320 servers)
• Graphics nodes (SGI Virtu VN200 graphics nodes)
• Network interconnect components (Gigabit Ethernet switches, InfiniBand switches,
PCI cards, and cables)
• System console, monitor, keyboard, and mouse
The head node is connected to the interconnect network and also to the public network,
typically via the local area network (LAN). The head node is the point of submittal for all
MPI application jobs for the cluster. An MPI job is started from the head node and the
sub-processes aredistributed to thecluster compute nodes from the head node.The main
process on the head node waits for the sub-processes to finish. For large clusters or
clusters that run many MPI jobs, multiple head nodes may be used to distribute the load.
The compute nodes are identical computing systems that run the primary processes of
MPI applications. These compute nodes are connected to each other through the
interconnect network.
A graphics node is similar to a compute node in that it contains processors and memory,
but it has an additional high-performance 3D graphics card installed.
The network interconnect components are typically Gigabit Ethernet or InfiniBand. The
MPI messages are passed across this network between the processes. This compute node
network does not connect directly to the public network because mixing external and
internal cluster network traffic could impact application performance. Visualization
nodes may be connected to the public network to act as a login or application gateway
for remote visualization.
Note: Refer to “Related Publications” on page xiv, for a listing of relevant SGI Technical
Publications that can provide more detailed information about SGI cluster head nodes,
compute nodes, and system rack.

4007-5483-001
SGI Virtu VN200 Overview
Warnings and Cautions
Read the following safety and warnings and cautions when operating your
SGI Virtu VN200 cluster.
Warning: System power on/off: The power button DOES NOT turn off the system
AC power. To remove power from system, you must unplug the AC power cord from
the wall outlet. Make sure the AC power cord is unplugged before you open the
chassis, add, or remove any components. Failure to do so may result in serious injury
or death.
Caution: Electrostatic discharge (ESD)and ESD protection: ESD can damage disk drives,
boards, and other parts. We recommend that you perform all replacement procedures on
an ESD-safe worksurface. If one is not available, provide some ESD protection by
wearing an antistatic wrist strap attached to the chassis ground or any unpainted metal
surface on your server when handling parts.
Caution: Always handle boards carefully. They can be extremely sensitive to ESD. Hold
boards only by their edges. After removing a board from its protective wrapper or from
the server, place the board component side up on a grounded, static free surface. Use a
conductive foam pad if available but not the board wrapper. Do not slide board over any
surface.
Caution: Do not place the node board on a conductive surface. A conductive surface can
damage the BIOS battery and prevent the system from booting. Always place the node
board on an ESD-safe worksurface.
!
!
!
!