Sun Microsystems Netra CP3060 User manual

Sun Microsystems, Inc.
www.sun.com
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Sun Netra™CP3060 Blade Server
User’s Guide
Part No. 819-4967-11
April 2009, Revision A

Please
Recycle
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
This distribution may include materials developed by third parties.
Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in
the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd.
Sun, Sun Microsystems, the Sun logo, Netra, Sun Ray, the Netra logo and the Solaris logo are trademarks or registered trademarks of Sun
Microsystems, Inc., or its subsidiaries, in the U.S. and other countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other
countries. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems, Inc.
Use of any spare or replacement CPUs is limited to repair or one-for-one replacement of CPUs in products exported in compliance with U.S.
export laws. Use of CPUs as product upgrades unless authorized by the U.S. Government is strictly prohibited.
DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright © 2009 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.
Cette distribution peut comprendre des composants développés par des tierces parties.
Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque
déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.
Sun, Sun Microsystems, le logo Sun, Netra, Sun Ray, le logo Netra et le logo Solaris sont des marques de fabrique ou des marques déposées de
Sun Microsystems, Inc., ou ses filiales, aux Etats-Unis et dans d’autres pays.
Toutes les marques SPARC sont utilisées sous licence et sont des marques de fabrique ou des marques déposées de SPARC International, Inc.
aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun
Microsystems, Inc.
L’utilisation de pieces detachees ou d’unites centrales de remplacement est limitee aux reparations ou a l’echange standard d’unites centrales
pour les produits exportes, conformement a la legislation americaine en matiere d’exportation. Sauf autorisation par les autorites des Etats-
Unis, l’utilisation d’unites centrales pour proceder a des mises a jour de produits est rigoureusement interdite.
LA DOCUMENTATION EST FOURNIE "EN L’ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFACON.

iii
Contents
Preface xi
1. Introduction to the Sun Netra CP3060 Blade Server 1–1
1.1 Overview of the Sun Netra CP3060 Blade Server 1–1
1.2 Features of the Sun Netra CP3060 Blade Server 1–2
1.3 Sun Netra CP3060 Blade Server System Configurations 1–7
1.3.1 AMC Modules 1–8
1.3.2 Rear Transition Module 1–8
1.4 Hot-Swap Support 1–11
1.5 System Requirements 1–11
1.5.1 Hardware Requirements 1–12
1.5.2 Software Requirements 1–12
1.6 Technical Support and Warranty 1–13
1.6.1 Blade Server Part Number, Serial Number, and Revision Number
Identification 1–13
2. Hardware Installation 2–1
2.1 Equipment and Operator Safety 2–1
2.2 Materials and Tools Required 2–2
2.3 Preparing for the Installation 2–3

iv Netra CP3060 Board User’s Guide • April 2009
2.3.1 Checking Power, Thermal, Environmental, and Space
Requirements 2–3
2.3.2 Determining Local Network IP Addresses and Host Names 2–4
2.3.3 Installation Procedure Summary 2–4
2.4 Configuring the Blade Server Hardware 2–5
2.4.1 Installation of DDR-2 DIMM Memory Modules 2–5
2.4.1.1 Installing a DDR-2 DIMM Memory Module 2–7
2.4.1.2 Removing a DDR-2 DIMM Memory Module 2–8
2.4.2 Installation of Optional Compact Flash Card 2–9
2.4.3 Configuring Rear Transition Module Hardware 2–11
2.5 Installing the Sun Netra CP3060 Blade Server 2–11
2.5.1 Installing the Sun Netra CP3060 Blade Server With a Rear
Transition Module 2–11
2.5.1.1 Installing a Rear Transition Module 2–12
2.5.2 Installing the Sun Netra CP3060 Blade Server 2–15
2.6 Connecting External I/O Cables 2–17
2.7 Installation of Optional AMC Modules 2–18
2.7.0.1 Installing an Optional AMC Module 2–19
3. Software Installation 3–1
3.1 Operating Systems and Patches 3–1
3.2 Firmware Updates 3–2
3.3 Mandatory /etc/system File Entry 3–2
▼To Check and Create the Mandatory /etc/system File Entry 2
3.4 Installing Diskless Clients 3–3
3.4.1 Creating a Boot Server for Diskless Clients 3–3
3.4.2 Adding a Diskless Client 3–5
3.5 Downloading and Installing SunVTS Software 3–7
4. Firmware and Blade Server Management 4–1

Contents v
4.1 System Firmware 4–1
4.2 Power-On Self-Test Diagnostics 4–2
4.2.1 POST Test Coverage 4–2
4.2.2 POST Diagnostic and Error Message Format 4–3
4.3 OpenBoot Firmware 4–3
4.3.1 Getting to the ok Prompt 4–4
4.3.2 Auto-Boot Options 4–5
4.3.3 OpenBoot Commands 4–5
4.3.3.1 probe-ide Command 4–6
4.3.3.2 show-devs Command 4–6
4.3.3.3 Checking Network Using watch-net and watch-net-
all Commands 4–8
4.3.4 OpenBoot Configuration Variables 4–9
4.3.4.1 Viewing and Setting OpenBoot Configuration
Variables 4–12
4.4 Error Handling Summary 4–12
4.5 Automatic System Recovery 4–13
4.5.1 Enabling and Disabling Automatic System Recovery 4–14
4.5.1.1 To Enable Automatic System Recovery 4–14
4.5.1.2 To Disable Automatic System Recovery 4–15
4.6 Hot-Swap Information 4–15
4.6.1 Hot-Swapping the Sun Netra CP3060 Blade Server 4–15
4.6.1.1 Hot-Swap LED 4–16
4.7 Network Device Aliases 4–17
4.8 Retrieving Device Information 4–17
4.9 Mandatory /etc/system File Entry 4–20
▼To Check and Create the Mandatory /etc/system File Entry 20
5. Hardware and Functional Descriptions 5–1
5.1 Hardware Architecture 5–1

vi Netra CP3060 Board User’s Guide • April 2009
5.1.1 UltraSPARC T1 Processor 5–4
5.1.1.1 Overview 5–4
5.1.1.2 Cores and Cache 5–5
5.1.1.3 Memory Controller 5–6
5.1.1.4 Instruction Set 5–6
5.1.1.5 Interrupts 5–6
5.1.1.6 UltraSPARC T1 RAS Features 5–6
5.1.1.7 UltraSPARC T1 Processor Speed 5–6
5.1.2 DDR-2 Memory Subsystem 5–7
5.1.2.1 DIMMs 5–7
5.1.2.2 Memory Subsystem RAS Features 5–8
5.1.2.3 Serial Presence Detect 5–9
5.1.3 System JBus 5–9
5.1.3.1 JBus RAS Features 5–9
5.1.4 I/O Subsystem 5–9
5.1.4.1 JBus-to-PCI-E Bridge 5–10
5.1.4.2 PCI-E Switch 5–11
5.1.4.3 PCI-E to Dual GbE Controller 5–11
5.1.4.4 Ethernet Switch 5–12
5.1.4.5 AMC Slot 5–12
5.1.4.6 Southbridge 5–16
5.1.5 Service Processor MPC885 5–17
5.1.5.1 MPC Bus External Devices 5–18
5.1.5.2 Field-Programmable Gate Array 5–18
5.1.5.3 XBus External devices 5–19
5.1.6 Intelligent Platform Management Controller 5–19
5.1.6.1 Intelligent Platform Management Bus 5–19
5.1.6.2 Interface to the MPC 5–19

Contents vii
5.1.6.3 IPMB-L Interface 5–20
5.1.6.4 ATCA Hot-Swap Latch 5–20
5.1.6.5 LEDs 5–20
5.1.6.6 Power Control 5–20
5.1.6.7 Extended Interface E-keying Control 5–21
5.1.6.8 I2C Architecture 5–21
5.1.6.9 I2C Device Address Map 5–22
5.1.6.10 System Monitor and Thresholds 5–24
5.2 Power-on Sequence 5–26
5.3 Power 5–27
5.3.1 Power Input 5–27
5.3.2 Fuses 5–27
5.3.3 ATCA Power Module (-48V to 12V) 5–28
5.3.4 On-Board DC/DC Regulators 5–28
5.3.5 TOD Clock Battery 5–28
A. Physical Characteristics A–1
A.1 Form Factor A–1
A.2 Layout A–1
A.3 Front Panel A–2
A.3.1 Visual Indicators A–3
A.3.2 Ports A–3
A.3.3 AMC Slot A–3
A.4 Connectors and Pinout A–3
A.4.1 Front Panel Connectors A–3
A.4.1.1 Ethernet Ports A–4
A.4.1.2 Serial Port A–4
A.4.2 AMC Connector A–5
A.4.3 Compact Flash Connector A–7

viii Netra CP3060 Board User’s Guide • April 2009
A.4.4 Midplane Power Connector (Zone 1) A–7
A.4.5 Data Transport Connector (Zone 2) A–9
A.4.6 RTM Connector (Zone 3) A–11
A.4.7 TOD Clock Battery Holder A–14
B. Sun OEM IPMI Commands B–1
B.1 Get Version Command B–2
B.2 Get RTM Status Command B–3
Index Index–1

ix
Figures
FIGURE 1-1 Sun Netra CP3060 Blade Server Front Panel 1–4
FIGURE 1-2 Sun Netra CP3060 Blade Server (Top View) 1–4
FIGURE 1-3 Sun Netra CP3060 Blade Server in an ATCA Shelf Enclosure 1–8
FIGURE 1-4 Sun Netra CP3060 Rear Transition Module 1–9
FIGURE 1-5 Relationship of the Sun Netra CP3060 Blade Server, Midplane, and RTM 1–10
FIGURE 1-6 Sun Netra CP3060 Blade Server Barcode Labeling 1–14
FIGURE 2-1 DDR-2 DIMM Memory Locations 2–6
FIGURE 2-2 Installing a DDR-2 DIMM Memory Module 2–8
FIGURE 2-3 Removing a DDR-2 DIMM Memory Module 2–9
FIGURE 2-4 Compact Flash Card Connector 2–10
FIGURE 2-5 Installing the Sun Netra CP3060 Rear Transition Module 2–12
FIGURE 2-6 RTM Injector/Ejector Latch and Locking Screw 2–14
FIGURE 2-7 Installing Node Blade Server Into Chassis Slot 2–16
FIGURE 2-8 Sun Netra CP3060 Blade Server Latches and Locking Screws 2–17
FIGURE 2-9 Removing AMC Slot Filler Panel 2–19
FIGURE 2-10 Inserting AMC Module Into AMC Connector 2–20
FIGURE 4-1 Hot-Swap Latch and Hot-Swap LED 4–16
FIGURE 5-1 Block Diagram 5–3
FIGURE 5-2 UltraSPARC T1 Multicore Processor Block Diagram 5–5
FIGURE 5-3 DDR Memory Diagram 5–8

xNetra CP3060 Board User’s Guide • April 2009
FIGURE 5-4 AMC Power and Management Infrastructure 5–14
FIGURE 5-5 Clock Synchronization 5–16
FIGURE 5-6 I2C Block Diagram 5–22
FIGURE A-1 Sun Netra CP3060 Blade Server Layout A–2
FIGURE A-2 Ethernet RJ-45 Connector A–4
FIGURE A-3 Front Panel Serial Port Diagram A–5
FIGURE A-4 Power Distribution Connector (Zone 1) P10 A–8
FIGURE A-5 Zone 2 Connectors A–10
FIGURE A-6 Zone 3 Connectors A–12

xi
Preface
The Sun Netra CP3060 Blade Server User’s Guide provides information about features,
installation, configuration, functional hardware components, and physical properties
of this blade server. The Sun Netra CP3060 Blade Server User’s Guide is written for
system integration engineers, field applications and service engineers, and others
involved in the integration of these blade servers into systems.
How This Document Is Organized
Chapter 1 provides an overview of the Sun Netra CP3060 blade server.
Chapter 2 provides instructions on hardware installation.
Chapter 3 provides instructions on the software configuration.
Chapter 4 provides information about the Sun Netra CP3060 firmware.
Chapter 5 provides hardware and functional descriptions of the Sun Netra CP3060
blade server.
Appendix A provides information about the physical characteristics of the Sun Netra
CP3060 blade server.
Appendix B describes the Sun-specific OEM-defined Intelligent Platform
Management Interface (IPMI) commands.

xii Netra CP3060 Board User’s Guide • April 2009
Using UNIX Commands
This document might not contain information on basic UNIX®commands and
procedures such as shutting down the system, booting the system, and configuring
devices. Refer to the following for this information:
■Software documentation that you received with your system
■Solaris™ Operating System documentation, which is at:
http://docs.sun.com
Shell Prompts
Shell Prompt
C shell machine-name%
C shell superuser machine-name#
Bourne shell and Korn shell $
Bourne shell and Korn shell superuser #

Preface xiii
Typographic Conventions
Related Documentation
For additional information about the Sun Netra CP3060 blade server and the Sun
Netra CP3060 rear transition module (RTM), refer to the following documents.
Except for Important Safety Information for Sun Hardware Systems, all the documents
listed are available online at:
http://www.sun.com/documentation
Typeface*
* The settings on your browser might differ from these settings.
Meaning Examples
AaBbCc123 The names of commands, files,
and directories; on-screen
computer output
Edit your.login file.
Use ls -a to list all files.
% You have mail.
AaBbCc123 What you type, when contrasted
with on-screen computer output
%su
Password:
AaBbCc123 Book titles, new words or terms,
words to be emphasized.
Replace command-line variables
with real names or values.
Read Chapter 6 in the User’s Guide.
These are called class options.
You must be superuser to do this.
To delete a file, type rm filename.
Title Part Number
Sun Netra CP3060 Blade Server Product Notes 819-4966
Sun Netra CP3060 Blade Server Getting Started Guide 819-4971
Sun Netra CP3060 Blade Server Programming Guide 819-4969
Sun Netra CP3060 Rear Transition Module Getting Started Guide 819-6692
Sun Netra CP3060 Rear Transition Module User’s Guide 819-6689
Important Safety Information for Sun Hardware Systems (printed version only)816-7190-10

xiv Netra CP3060 Board User’s Guide • April 2009
Documentation, Support, and Training
Third-Party Web Sites
Sun is not responsible for the availability of third-party web sites mentioned in this
document. Sun does not endorse and is not responsible or liable for any content,
advertising, products, or other materials that are available on or through such sites
or resources. Sun will not be responsible or liable for any actual or alleged damage
or loss caused by or in connection with the use of or reliance on any such content,
goods, or services that are available on or through such sites or resources.
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun Netra CP3060 Blade Server User’s Guide, part number 819-4967-11
Sun Function URL
Documentation http://www.sun.com/documentation/
Support http://www.sun.com/support/
Training http://www.sun.com/training/

1-1
CHAPTER 1
Introduction to the Sun Netra
CP3060 Blade Server
This chapter contains the following sections:
■Section 1.2, “Features of the Sun Netra CP3060 Blade Server” on page 1-2
■Section 1.3, “Sun Netra CP3060 Blade Server System Configurations” on page 1-7
■Section 1.4, “Hot-Swap Support” on page 1-11
■Section 1.5, “System Requirements” on page 1-11
■Section 1.6, “Technical Support and Warranty” on page 1-13
1.1 Overview of the Sun Netra CP3060 Blade
Server
The Sun Netra CP3060 blade server is a high-performance single-blade server
computer based on one UltraSPARC®T1 multicore processor and designed for high
availability in a switched network computing environment. This blade server is
compliant with Advanced Telecom Computing Architecture®(ATCA) specifications
(PICMG®3.0 and PICMG 3.1) and can achieve greater performance levels than
compactPCI (cPCI) standards-based products targeted for telco markets.
The PICMG (PCI Industrial Computer Manufacturers Group) standards committee
has developed the new ATCA (or PICMG 3.x) standard to address the issues posed
by previous standards based on cPCI and cPSB (PICMG 2.x). The PICMG 3.x
specification brought the following changes to the existing PICMG 2.x family of
products:
■Larger board space (8U high compared to 6U for cPCI), which allows more
features and processing power
■On-board power supplies deriving local power from redundant −48V power from
the midplane (rather than separate power supplies)

1-2 Netra CP3060 Board User’s Guide • April 2009
■6-HP slot width, allowing greater component height
■Advanced Mezzanine card (AMC) support and options
■Elimination of PCI connectivity between the blade servers in the system and
reallocation of connectivity to serial interconnects, eliminating single points of
failure
■Mandatory use of Intelligent Platform Management Interface (IPMI) management
interfaces
■Flexible user I/O
■Power and thermal management guidelines enforced by the management
infrastructure
■Separation of control and data traffic by supporting the Base (PICMG 3.0) and
Extended (PICMG 3.1) interfaces
The ATCA standard consists of the PICMG 3.0, PICMG 3.1, PICMG 3.2, and PICMG
3.3 specifications. The Sun Netra CP3060 blade server complies with:
■PICMG 3.0, the base specification that defines the mechanical, power distribution,
system management, data transport, and regulatory guidelines
■PICMG 3.1, which builds upon the PICMG 3.0 base specification and the IEEE
802.3-2003 standard
1.2 Features of the Sun Netra CP3060 Blade
Server
The Sun Netra CP3060 blade server provides two 1000BASE-T Ethernet interfaces for
the Base interface (a requirement of PICMG 3.0) and two serializer, deserializer
(SERDES) gigabit Ethernet interfaces (PICMG 3.1) for the Extended interface. The
Base interface is used as the control interface and the Extended interface can be used
for data traffic. Both Base and Extended interfaces are configured as Dual Star
topologies.
Sun Netra CP3060 blade server features include:
■Single-wide ATCA module
■Compliant with PCIMG3.0/3.1 specification
■UltraSPARC T1 4-core, 6-core, or 8-core, 1.0-GHz processor
■Service processor (MPC885) for CPU reset, boot, partition, and fault management
architecture (FMA)
■Memory subsystem
■Eight standard DDR-2 Very Low Profile (VLP) DIMM sockets

Chapter 1 Introduction to the Sun Netra CP3060 Blade Server 1-3
■200 MHz DIMMs, buffered and registered with error-correcting code (ECC)
■Up to 16 Gbytes of DIMM memory with 2-Gbyte DIMMs
■JBus-to-PCI-E application-specific integrated circuits (ASICs) to bridge processors
and the PCI-E I/O subsystem
■Two on-board 1000BASE-T Ethernet interfaces used as ATCA Base Interface
■Two on-board 1000 SERDES Ethernet interfaces used as ATCA Extended Interface
■One AMC slot for x8 PCI-E I/O expansion or compatible disk drive
■Compliant with PCIMG AMC.0/.1/.3 specifications
■Two 3-Gbps Serial Advanced Technology Attachment (SATA) ports routed to
AMC connector
■Two 10/100/1000 BASE-T Ethernet RJ-45 ports on front panel
■One asynchronous serial port on front panel
■Compact Flash socket to support an 8-Gbyte user flash type I/II memory card
■System management support using Pigeon Point Systems Intelligent Platform
Management (IPM) controller, providing a redundant IPMI channel to
communicate with the ATCA shelf manager
■Rear I/O access using a compatible Sun Netra CP3060 rear transition module
(RTM)
■Maximum power consumption: 200 watts (including Sun Netra CP3060 RTM)
FIGURE 1-1 and FIGURE 1-2 show the Sun Netra CP3060 blade server.

1-4 Netra CP3060 Board User’s Guide • April 2009
FIGURE 1-1 Sun Netra CP3060 Blade Server Front Panel
1 - Top latch 6 - Out of Service LED (yellow)
2 - AMC Hard Disk Drive (optional) 7 - OK LED (green)
3 - Ethernet port A (RJ-45) 8 - Hot-Swap LED (blue)
4 - Ethernet port B (RJ-45) 9 - Hot-Swap switch and bottom latch
5 - Serial port (RJ-45, ttya)
12
34567
8
9

Chapter 1 Introduction to the Sun Netra CP3060 Blade Server 1-5
FIGURE 1-2 Sun Netra CP3060 Blade Server (Top View)
AMC slot
enclosure
Compact
Flash connector
DIMMs
DIMMs
Zone 3
connectors
Zone 1
power
connector
Zone 2
connectors
Processor
(under heat
sinks)
AMC HDD
(optional)

1-6 Netra CP3060 Board User’s Guide • April 2009
TABLE 1-1 lists features of the blade server.
TABLE 1-1 Feature Summary for the Sun Netra CP3060 Blade Server
Feature Description
CPU • One UltraSPARC T1 processor (4, 6, or 8 core)
• CPU core speed: 1.0 GHz
Memory • Eight DDR-2 240-pin Very Low Profile (VLP) DIMMs, buffered,
registered
• 1-Gbyte or 2-Gbyte DIMMs, for a total memory of 16 Gbytes
• ECC supported
Power
requirement
ATCA 3.0 standard of 200 watts maximum
PICMG
compliance
• PICMG 3.0 R1.0
• PICMG 3.1 R1.0
Node board
support
Functions as a CPU node board with the Solaris software package
Operating
system
Solaris 10 6/06 OS and subsequent compatible versions with supported
Netra patches
Internal I/O
(connections to
ATCA
midplane)
• Dual gigabit Ethernet for Base interface
• Dual SERDES interface as Extended interface
• Dual IPMI channel connects to the midplane for communicating with
the Shelf Management card
External I/O • Two 10/100/1000 BASE-T Ethernet ports on front panel
• One asynchronous serial port for maintenance on front panel
• One AMC slot with front panel access for AMC I/O
• Rear access support using Sun Netra CP3060 RTM:
–One 10/100 Mbps Ethernet port
–One asynchronous serial port. Access is available in front and rear
when RTM is present. Note: Either serial port can be used; but only
one of the serial ports can be used at a time.
AMC I/O Provision for adding an independent hardware vendor (IHV)-supplied
AMC disk or PCI-E card on front panel
IPMI system
management
Uses IPMI communications with baseboard management controller
(BMC); performs advanced system monitoring (ASM) on local board
interface (for example, temperature sense, FRU ID, and control)
Hot-swap
support
• Basic, full, and high-availability (HA) hot-swap support for node board
• AMC disk/card hot-swap support