Sun Microsystems Sun Netra CP3260 User manual

Sun Microsystems, Inc.
www.sun.com
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Sun Netra™CP3260Blade Server
User’s Guide
Part No. 820-0457-11
April 2009, Revision 01

Please
Recycle
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
This distribution may include materials developed by third parties.
Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in
the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd.
Sun, Sun Microsystems, the Sun logo, Netra, Sun Ray, the Netra logo and the Solaris logo are trademarks or registered trademarks of Sun
Microsystems, Inc., and its subsidiaries, in the U.S. and other countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other
countries. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems, Inc.
Use of any spare or replacement CPUs is limited to repair or one-for-one replacement of CPUs in products exported in compliance with U.S.
export laws. Use of CPUs as product upgrades unless authorized by the U.S. Government is strictly prohibited.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.
Cette distribution peut comprendre des composants développés par des tierces parties.
Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque
déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.
Sun, Sun Microsystems, le logo Sun, Netra, Sun Ray, le logo Netra et le logo Solaris sont des marques de fabrique ou des marques déposées de
Sun Microsystems, Inc. ou ses filiales, aux Etats-Unis et dans d’autres pays.
Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc.
aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun
Microsystems, Inc.
see above L’utilisation de pieces detachees ou d’unites centrales de remplacement est limitee aux reparations ou a l’echange standard d’unites
centrales pour les produits exportes, conformement a la legislation americaine en matiere d’exportation. Sauf autorisation par les autorites des
Etats-Unis, l’utilisation d’unites centrales pour proceder a des mises a jour de produits est rigoureusement interdite.
LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFACON.

iii
Contents
Preface xiii
1. Introduction 1–1
1.1 Overview 1–1
1.2 Features 1–2
1.3 System Configurations 1–8
1.4 Advanced Rear Transition Modules 1–11
1.5 Hot-Swap Support 1–14
1.6 System Requirements and Options 1–14
1.6.1 Hardware Requirements and Options 1–14
1.6.2 Software Requirements 1–15
1.7 Technical Support and Warranty 1–15
1.7.1 Part Number, Serial Number, and Revision Number
Identification 1–16
2. Hardware Installation 2–1
2.1 Equipment and Operator Safety 2–1
2.2 Materials and Tools Required 2–3
2.3 Preparing for the Installation 2–3
2.3.1 Checking Power, Thermal, Environmental, and Space
Requirements 2–3

iv Netra CP3260 Blade Server User’s Guide • April 2009
2.4 Installation Procedure Summary 2–4
2.5 Configuring On-Board Hardware 2–5
2.5.1 Adding or Replacing FB-DIMM Memory Modules 2–5
2.5.1.1 To Remove FB-DIMM Memory Modules 2–10
2.5.1.2 To Install FB-DIMM Memory Modules 2–11
2.5.2 Adding or Replacing Compact Flash Card 2–13
2.5.3 Adding or Replacing TOD Clock Battery 2–14
2.6 Installing an Advanced Rear Transition Module (Optional) 2–18
2.7 Installing the Netra CP3260 Blade Server 2–21
2.8 Connecting External I/O Cables 2–23
2.9 Hot-Swapping the Netra CP3260 Blade Server 2–25
3. Software Installation 3–1
3.1 Operating Systems and Patches 3–1
3.2 Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3 3–
2
3.3 Installing as a Diskless Clients 3–4
3.3.1 Creating a Boot Server for Diskless Clients 3–4
3.3.2 Adding a Diskless Client 3–5
3.3.3 Determining Local Network IP Addresses and Host Names 3–7
3.4 Firmware Updates 3–8
3.5 Configuring for 1-GbE or 10-GbE Switches 3–8
3.6 Downloading and Installing SunVTS Software 3–9
3.7 Formatting the Optional Compact Flash Card 3–10
4. Firmware and Blade Server Management 4–1
4.1 System Firmware 4–2
4.2 Power-On Self-Test Diagnostics 4–3
4.2.1 POST Test Coverage 4–3
4.2.2 POST Diagnostic and Error Message Format 4–4

Contents v
4.3 OpenBoot Firmware 4–4
4.3.1 Getting to the ok Prompt 4–5
4.3.2 Auto-Boot Options 4–6
4.3.3 OpenBoot Commands 4–6
4.3.3.1 probe-scsi and probe-scsi-all Commands 4–7
4.3.3.2 probe-ide Command 4–8
4.3.3.3 show-devs Command 4–8
4.3.3.4 Using watch-net and watch-net-all Commands to
Check the Network 4–11
4.3.4 OpenBoot Configuration Variables 4–12
4.3.4.1 Viewing and Setting OpenBoot Configuration
Variables 4–14
4.4 Error Handling Summary 4–15
4.5 Automatic System Recovery 4–16
4.5.1 Enabling and Disabling Automatic System Recovery 4–17
4.5.1.1 To Enable Automatic System Recovery 4–17
4.5.1.2 To Disable Automatic System Recovery 4–17
4.6 Network Device Aliases 4–18
4.7 Retrieving Device Information 4–19
4.8 Multiplexing to Zones 2 and 3 4–27
5. Hardware Functional Descriptions 5–1
5.1 Hardware Architecture 5–1
5.1.1 UltraSPARC T2 Processor and Memory 5–2
5.1.2 I/O 5–3
5.1.3 Base and Fabric Interfaces 5–3
5.1.4 Additional I/O 5–3
5.1.5 Power PC 5–3
5.1.6 ARTM Support 5–4
5.2 Hardware Modules 5–4

vi Netra CP3260 Blade Server User’s Guide • April 2009
5.2.1 UltraSPARC T2 Processor 5–4
5.2.1.1 Electronic Fuse 5–7
5.2.1.2 Cores 5–8
5.2.1.3 L2 Cache 5–8
5.2.1.4 Memory Controller 5–8
5.2.1.5 I/O Interface 5–8
5.2.2 Memory Subsystem 5–9
5.2.2.1 Memory Capacity 5–9
5.2.2.2 Memory Speed 5–9
5.2.3 I/O Subsystem 5–9
5.2.3.1 PCI Express Switch 5–10
5.2.3.2 Base Interface 5–10
5.2.3.3 Fabric Interface 5–10
5.2.3.4 Common ARTM 5–10
5.2.4 Other ARTM Interfaces 5–11
5.2.4.1 Serial Ports 5–11
5.2.4.2 Ethernet Management Port 5–11
5.2.5 Front Panel I/O 5–11
5.2.5.1 Ethernet Management Port 5–11
5.2.5.2 Serial Port 5–11
5.2.5.3 Dual USB Ports 5–12
5.2.6 Compact Flash Socket 5–12
5.2.7 Service Processor MPC885 5–12
5.2.7.1 Field-Programmable Gate Array 5–12
5.2.8 Intelligent Platform Management Controller 5–13
5.2.8.1 Intelligent Platform Management Bus 5–14
5.2.8.2 Interface to the PPC 5–14
5.2.8.3 IPMB-L Interface 5–14

Contents vii
5.2.8.4 ATCA Hot-Swap Latch 5–14
5.2.8.5 LEDs 5–15
5.2.8.6 Power Control 5–15
5.2.8.7 System Monitor (ADM1026) and Thresholds 5–15
5.2.8.8 FRUID PROMs 5–17
5.2.9 I/O Subsystem Resets 5–17
5.2.10 ATCA Power Module (−48V to 12V) 5–18
5.2.11 TOD Clock Battery 5–18
A. Physical Characteristics A–1
A.1 Form Factor A–1
A.2 Layout A–1
A.3 Front Panel A–3
A.3.1 Visual Indicators A–3
A.3.2 Ports A–3
A.4 Connectors and Pinout A–3
A.4.1 Front Panel Connectors A–3
A.4.1.1 Ethernet Port A–4
A.4.1.2 Dual USB Port A–4
A.4.1.3 Serial Port A–5
A.4.2 Compact Flash Connector A–6
A.4.3 Midplane Power Connector (Zone 1) A–6
A.4.4 Data Transport Connector (Zone 2) A–8
A.4.5 Sun Netra ARTM Connectors (Zone 3) A–10
A.4.6 TOD Clock Battery Holder A–13
B. Sun OEM IPMI Commands B–1
B.1 Get Version Command B–2
B.2 Get RTM Status Command B–3

viii Netra CP3260 Blade Server User’s Guide • April 2009
B.3 Solaris OS Graceful Shutdown Commands B–4
B.4 Send Sensor State Command B–5
Index Index–1

ix
Figures
FIGURE 1-1 Netra CP3260 Blade Server Front Panel 1–4
FIGURE 1-2 Netra CP3260 Blade Server (Top View) 1–5
FIGURE 1-3 Netra CP3260 Blade Server in an ATCA Shelf Enclosure 1–9
FIGURE 1-4 Netra CP3260 Blade Server, Midplane, and Netra CP32x0 ARTM 1–12
FIGURE 1-5 Netra CP3260 Blade Server Barcode Labeling 1–17
FIGURE 2-1 FB-DIMM Memory Locations 2–6
FIGURE 2-2 Removing an FB-DIMM Memory Module 2–11
FIGURE 2-3 Installing a FB-DIMM Memory Module 2–12
FIGURE 2-4 Compact Flash Card Location 2–14
FIGURE 2-5 TOD Battery Location 2–16
FIGURE 2-6 Installing a Netra CP32X0 ARTM 2–19
FIGURE 2-7 Installing Blade Server Into Chassis Slot 2–22
FIGURE 2-8 Netra CP3260 Blade Server Latches and Locking Screws 2–23
FIGURE 2-9 Hot-Swap Latch and Hot-Swap LED 2–26
FIGURE 5-1 Netra CP3260 Blade Server Block Diagram 5–2
FIGURE 5-2 UltraSPARC T2 Multicore Processor Block Diagram 5–5
FIGURE A-1 Netra CP3260 Blade Server Layout A–2
FIGURE A-2 Ethernet RJ-45 Connector A–4
FIGURE A-3 Dual USB Connector A–5
FIGURE A-4 Front Panel Serial Port Diagram A–6

xNetra CP3260 Blade Server User’s Guide • April 2009
FIGURE A-5 Power Distribution Connector (Zone 1) P10 A–7
FIGURE A-6 Zone 2 Connectors A–9
FIGURE A-7 Zone 3 Signal Connectors A–10
FIGURE A-8 Zone 3 Power Connector A–12
FIGURE A-9 TOD Battery Location A–14

xi
Tables
TABLE 1-1 Netra CP3260 Blade Server Feature Summary 1–2
TABLE 1-2 I/O Configurations 1–13
TABLE 2-1 Ethernet Device Names 2–24
TABLE 3-1 Local Network Information 3–7
TABLE 4-1 Ways of Accessing the ok Prompt 4–5
TABLE 4-2 OpenBoot Configuration Variables 4–12
TABLE 4-3 Network Device Aliases 4–18
TABLE 4-4 PICL Frutree Entries and Description for the Netra CP3260 Board 4–19
TABLE 5-1 Voltage Sensor Thresholds 5–16
TABLE 5-2 CPU Temperature Alarms 5–16
TABLE A-1 Ethernet Port Connector Pin Assignments A–4
TABLE A-2 USB Connector Pin Assignments A–5
TABLE A-3 Serial Port RJ-45 Connector Pinouts A–6
TABLE A-4 Power Distribution Connector Pin Assignments A–7
TABLE A-5 Zone 2 J23 Connector Pin Assignments A–9
TABLE A-6 Zone 2 J20 Connector Pin Assignments A–9
TABLE A-7 Zone 3 J31 Connector Pin Assignments A–11
TABLE A-8 Zone 3 J32 Connector Pin Assignments A–11
TABLE A-9 Zone 3 J33 Connector Pin Assignments A–12
TABLE A-10 Zone 3 Power Connector Pin Assignments A–13

xii Netra CP3260 Blade Server User’s Guide • April 2009
TABLE B-1 Get Version Command Data Bytes B–2
TABLE B-2 Get RTM Status Command Data Bytes B–3
TABLE 1 Solaris OS Graceful Shutdown Parameters in /etc/fsmd.conf B–4
TABLE B-3 Send Sensor State Command Data Bytes B–5

xiii
Preface
The Sun Netra CP3260 Blade Server User’s Guide provides information about features,
installation, configuration, functional hardware components, and physical properties
of this blade server. The Sun Netra CP3260 Blade Server User’s Guide is written for
system integration engineers, field applications and service engineers, and others
involved in the integration of these blade servers into systems.
How This Document Is Organized
Chapter 1 provides an overview of the Sun Netra™ CP3260 blade server.
Chapter 2 provides instructions on hardware installation.
Chapter 3 provides instructions on the software configuration.
Chapter 4 provides information about the Netra CP3260 firmware.
Chapter 5 provides hardware and functional descriptions of the Netra CP3260 blade
server.
Appendix A provides information about the physical characteristics of the Netra
CP3260 blade server.
Appendix B describes the Sun-specific OEM-defined Intelligent Platform
Management Interface (IPMI) commands.

xiv Sun Netra CP3260 Blade Server User’s Guide • April 2009
Using UNIX Commands
This document might not contain information on basic UNIX®commands and
procedures such as shutting down the system, booting the system, and configuring
devices. Refer to the following for this information:
■Software documentation that you received with your system
■Solaris™ Operating System documentation, which is at:
http://docs.sun.com/app/docs/prod/solaris
Shell Prompts
Shell Prompt
C shell machine-name%
C shell superuser machine-name#
Bourne shell and Korn shell $
Bourne shell and Korn shell superuser #

Preface xv
Typographic Conventions
Note – Characters display differently depending on browser settings. If characters
do not display correctly, change the character encoding in your browser to Unicode
UTF-8.
Related Documentation
The Netra CP3260 blade server documentation is listed in the following table. Except
for the Important Safety Information for Sun Hardware Systems, all the documents listed
are available online at:
http://docs.sun.com/app/docs/prod/cp3260.brd
Typeface Meaning Examples
AaBbCc123 The names of commands, files,
and directories; on-screen
computer output
Edit your.login file.
Use ls -a to list all files.
% You have mail.
AaBbCc123 What you type, when contrasted
with on-screen computer output
%su
Password:
AaBbCc123 Book titles, new words or terms,
words to be emphasized.
Replace command-line variables
with real names or values.
Read Chapter 6 in the User’s Guide.
These are called class options.
You must be superuser to do this.
To delete a file, type rm filename.
Title Part Number
Netra CP3260 Board Product Notes 820-0455
Netra CP3260 Board Getting Started Guide 820-0456
Important Safety Information for Sun Hardware Systems (printed version only)816-7190

xvi Sun Netra CP3260 Blade Server User’s Guide • April 2009
Documentation, Support, and Training
Third-Party Web Sites
Sun is not responsible for the availability of third-party web sites mentioned in this
document. Sun does not endorse and is not responsible or liable for any content,
advertising, products, or other materials that are available on or through such sites
Application Title Part Number Format Location
Installation and
Configuration
Sun Netra CP32x0 SAS Storage
Advanced Rear Transition Module,
Dual HD User’s Guide
820-3147 PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.sas#hic
Installation and
Configuration
Sun Netra™ CP32x0 Quad GbE,
Dual Fibre Channel, Advanced Rear
Transition Module, User’s Guide
820-3148 PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.4gbefc?l=en#hic
Installation and
Configuration
Sun Netra™ CP32x0 10GbE
Advanced Rear Transition Module,
Dual Port User’s Guide
820-3150 PDF,
HTML
Online
http://docs.sun.com/app/docs/prod/cp32x0.10gbee?l=en#hic
Sun Function URL
Documentation http://www.sun.com/documentation/
Support http://www.sun.com/support/
Training http://www.sun.com/training/

Preface xvii
or resources. Sun will not be responsible or liable for any actual or alleged damage
or loss caused by or in connection with the use of or reliance on any such content,
goods, or services that are available on or through such sites or resources.
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun Netra CP3260 Blade Server User’s Guide, part number 820-0457-11

xviii Sun Netra CP3260 Blade Server User’s Guide • April 2009

1-1
CHAPTER
1-CHAPTER
Introduction
This chapter contains the following sections:
■Section 1.1, “Overview” on page 1-1
■Section 1.2, “Features” on page 1-2
■Section 1.3, “System Configurations” on page 1-8
■Section 1.5, “Hot-Swap Support” on page 1-14
■Section 1.6, “System Requirements and Options” on page 1-14
■Section 1.7, “Technical Support and Warranty” on page 1-15
1.1 Overview
The Netra CP3260 blade server (FIGURE 1-1 and FIGURE 1-2) is a high-performance
single-board computer based on one UltraSPARC®T2 multicore processor and
designed for high availability in a switched network computing environment. This
blade server is compliant with Advanced Telecommunications Computing
Architecture (AdvancedTCA®, ATCA®) specifications (PICMG®3.0 and PICMG 3.1).
The PICMG (PCI Industrial Computer Manufacturers Group) standards committee
has developed the new ATCA (or PICMG 3.x) standard to address the issues posed
by previous standards based on cPCI and cPSB (PICMG 2.x). The PICMG 3.x
specification brought the following changes to the existing PICMG 2.x family of
products:
■Larger board space (8U high compared to 6U for cPCI), which allows more
features and processing power
■On-board power supplies deriving local power from redundant −48V power from
the midplane (rather than separate power supplies)
■6-HP slot width, allowing greater component height
■Advanced Mezzanine card (AMC) support and options

1-2 Netra CP3260 Blade Server User’s Guide • April 2009
■Elimination of PCI connectivity between the blade servers in the system and
reallocation of connectivity to serial interconnects, eliminating single points of
failure
■Mandatory use of Intelligent Platform Management Interface (IPMI) management
interfaces
■Flexible user I/O
■Power and thermal management guidelines enforced by the management
infrastructure
■Separation of control and data traffic by supporting the Base (PICMG 3.0) and
Fabric (PICMG 3.1) interfaces
The ATCA standard consists of the PICMG 3.0, PICMG 3.1, PICMG 3.2, and PICMG
3.3 specifications. The Netra CP3260 blade server complies with:
■PICMG 3.0, the base specification that defines the mechanical, power distribution,
system management, data transport, and regulatory guidelines
■PICMG 3.1, which builds upon the PICMG 3.0 base specification and the IEEE
802.3-2003 standard
1.2 Features
TABLE 1-1 provides a summary of Netra CP3260 features.
TABLE 1-1 Netra CP3260 Blade Server Feature Summary
Feature Description
CPU • One UltraSPARC T2 processor (6 or 8 cores, 8 threads/core)
• CPU core speed: 1.2 GHz
Memory • Fully-Buffered DDR2-based memory DIMMs (FB-DIMMs)
• Eight FB-DIMM slots (one per channel or two per memory controller)
• 1-Gbyte or 2-Gbyte FB-DIMMs, for a total memory of 16 Gbytes
Service
processor
• Service processor (MPC885) for CPU reset, boot, partition, and fault
management architecture (FMA)
Power
requirement
• 200 watts maximum
• Dual redundant input 48V to derive on-board power
Cooling
requirement
• 32 - 35 cfm at 55C