Cisco SFS InfiniBand Server Switch SFS 7000P Manual

Corporate Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]
Cisco SFS 7000P Hardware
Installation Guide
May 2006
Customer Order Number:
Text Part Number: OL-9167-01 Rev. A0

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT
NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT
ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR
THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION
PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO
LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class
A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when
the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed
and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a
residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.
The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate
radio-frequency energy. If it is not installed in accordance with Cisco’s installation instructions, it may cause interference with radio and television
reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in
part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation.
However, there is no guarantee that interference will not occur in a particular installation.
Modifying the equipment without Cisco’s written authorization may result in the equipment no longer complying with FCC requirements for Class
A or Class B digital devices. In that event, your right to use the equipment may be limited by FCC regulations, and you may be required to correct
any interference to radio or television communications at your own expense.
You can determine whether your equipment is causing interference by turning it off. If the interference stops, it was probably caused by the Cisco
equipment or one of its peripheral devices. If the equipment causes interference to radio or television reception, try to correct the interference by
using one or more of the following measures:
• Turn the television or radio antenna until the interference stops.
• Move the equipment to one side or the other of the television or radio.
• Move the equipment farther away from the television or radio.
• Plug the equipment into an outlet that is on a different circuit from the television or radio. (That is, make certain the equipment and the television
or radio are on circuits controlled by different circuit breakers or fuses.)
Modifications to this product not authorized by Cisco Systems, Inc. could void the FCC approval and negate your authority to operate the product.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as
part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE
PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED
OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL
DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR
INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.

CCSP, CCVP, the Cisco Square Bridge logo, Follow Me Browsing, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We
Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA,
CCDP, CCIE, CCIP, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems
Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, FormShare, GigaDrive,
GigaStack, HomeLink, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, LightStream, Linksys, MeetingPlace,
MGX, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, ProConnect, RateMUX,
ScriptShare, SlideCast, SMARTnet, The Fastest Way to Increase Your Internet Quotient, and TransPath are registered trademarks of Cisco Systems,
Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply
a partnership relationship between Cisco and any other company. (0601R)
Cisco SFS 7000P Hardware Installation Guide
© 2006 Cisco Systems, Inc. All rights reserved.

v
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
CONTENTS
Audience ix
Organization x
Conventions x
Statement 1071—Warning Definition xii
Related Documentation xvi
Obtaining Documentation xvii
Cisco.com xvii
Product Documentation DVD xvii
Ordering Documentation xviii
Documentation Feedback xviii
Cisco Product Security Overview xviii
Reporting Security Problems in Cisco Products xix
Obtaining Technical Assistance xx
Cisco Technical Support & Documentation Website xx
Submitting a Service Request xxi
Definitions of Service Request Severity xxi
Obtaining Additional Publications and Information xxii
CHAPTER 1Product Overview 1-1
Switch Description 1-1
Connectors 1-2
Switch Components 1-2
LEDs 1-2
Power Supplies and Fan Units 1-2

Contents
vi
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Power Supply Bay 1-2
Power Supplies and Fan Trays 1-3
System Features 1-3
InfiniBand Connectivity 1-3
Scalability 1-3
High Availability 1-3
Hardware 1-4
Ports 1-4
Fabric 1-4
Nonblocking Architecture 1-5
Administrative Features 1-5
Real-Time Clock 1-5
Latency 1-5
Nonvolatile Memory 1-5
Vital Product Data Storage 1-5
Diagnostics 1-6
CHAPTER 2Installing the Cisco SFS 7000P Switch 2-1
Safety 2-2
Preparing for Installation 2-3
Configuring Basic Connectivity 2-3
Attaching a Serial Console Cable to a PC or Terminal 2-3
Powering Up the Chassis 2-4
Rack-Mounting the Cisco SFS 7000P Switch 2-5
Requirements 2-5
Rack-Mounting the Switch 2-6
Installing the Switch with One Person 2-6
Installing the Switch with Two People 2-11
Connecting Network Devices 2-11

vii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Contents
Connecting InfiniBand Devices 2-12
Connecting Management Devices 2-15
Managing the System 2-15
CHAPTER 3Installing Field Replaceable Units 3-1
Power and Fan Modules 3-1
Locating the Power Supply or Fan Unit 3-1
Failed Power Supplies or Fan Units 3-2
Installing a Power Supply or Fan Unit 3-2
Removing Power Supplies and Fan Units 3-5
CHAPTER 4Managing the Cisco SFS 7000P Switch 4-1
LEDs 4-1
Chassis Status LEDs 4-1
InfiniBand Port LEDs 4-2
Power Supply and Fan LEDs 4-3
Managing the System with Element Manager 4-3
Using the Summary Tab 4-4
Using the Power Supplies Tab 4-5
Using the Fans Tab 4-5
Using the Sensors Tab 4-6
Displaying System Information 4-6
Displaying Power Supply Information 4-7
Displaying Fan Information 4-7
Displaying Sensor Information 4-8
CHAPTER 5Hardware Diagnostic Tests 5-1
About Diagnostic Tests 5-1
LED Tests 5-1

Contents
viii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Self-Tests 5-2
Running Card Tests 5-2
Running a Card Self-Test 5-2
Running Chassis Tests 5-3
Running a Chassis Standard Test 5-3
Running Fan Tests 5-5
Running a Self-Test on a Fan 5-5
Running Power Supply Tests 5-7
Running a LED Test on the Power Supply 5-7
Running a Self-Test on a Power Supply 5-7
Displaying Hardware Errors 5-9
APPENDIX ASpecifications and Compliance Certifications A-1
Chassis and Management Interface A-1
Electrical Specifications A-2
I NDEX

ix
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
This preface describes who should read the Cisco SFS 7000P Hardware
Installation Guide, how it is organized, and its document conventions.
Audience
Only trained and qualified service personnel (as defined in IEC 60950 and
AS/NZS3260) should install, replace, or service the equipment described in this
publication.

Preface
Organization
x
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Organization
This publication is organized as follows:
Conventions
This publication uses the following conventions:
Chapter Title Description
Chapter 1 Product Overview Provides an overview of the Cisco SFS 7000P switch
features.
Chapter 2 Installing the Cisco SFS
7000P Switch
Describes how to install the switch in a rack.
Chapter 3 Installing Field
Replaceable Units
Describes how to install and remove field-replaceable
units (FRUs) on the switch.
Chapter 4 Managing the Cisco SFS
7000P Switch
Provides upgrade procedures for switch components.
Chapter 5 Hardware Diagnostic
Tests
Describes how to monitor switch performance.
Appendix A Specifications and
Compliance
Certifications
Provides specifications and compliance information.
Convention Description
boldface font Commands, command options, and keywords are in boldface.
italic font Arguments for which you supply values are in italics.
[ ] Elements in square brackets are optional.
{ x | y | z } Alternative keywords are grouped in braces and separated by vertical bars.
[ x | y | z ] Optional alternative keywords are grouped in brackets and separated by
vertical bars.
string A nonquoted set of characters. Do not use quotation marks around the string
or the string will include the quotation marks.

xi
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
Conventions
Notes use the following conventions:
Note Means reader take note. Notes contain helpful suggestions or references to
material not covered in the publication.
Cautions use the following conventions:
Caution Means reader be careful. In this situation, you might do something that could
result in equipment damage or loss of data.
screen font Terminal sessions and information the system displays are in screen font.
boldface screen font Information you must enter is in boldface screen font.
italic screen font Arguments for which you supply values are in italic screen font.
^ The symbol ^ represents the key labeled Control. For example, the key
combination ^D in a screen display means hold down the Control key while
you press the D key.
< > Nonprinting characters, such as passwords, are in angle brackets.
Convention Description

Preface
Conventions
xii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Warnings use the following conventions:
Statement 1071—Warning Definition
Warning IMPORTANT SAFETY INSTRUCTIONS
This warning symbol means danger. You are in a situation that could cause
bodily injury. Before you work on any equipment, be aware of the hazards
involved with electrical circuitry and be familiar with standard practices for
preventing accidents. Use the statement number provided at the end of each
warning to locate its translation in the translated safety warnings that
accompanied this device.
SAVE THESE INSTRUCTIONS
Waarschuwing
BELANGRIJKE VEILIGHEIDSINSTRUCTIES
Dit waarschuwingssymbool betekent gevaar. U verkeert in een situatie die
lichamelijk letsel kan veroorzaken. Voordat u aan enige apparatuur gaat
werken, dient u zich bewust te zijn van de bij elektrische schakelingen
betrokken risico's en dient u op de hoogte te zijn van de standaard praktijken
om ongelukken te voorkomen. Gebruik het nummer van de verklaring
onderaan de waarschuwing als u een vertaling van de waarschuwing die bij
het apparaat wordt geleverd, wilt raadplegen.
BEWAAR DEZE INSTRUCTIES
Varoitus
TÄRKEITÄ TURVALLISUUSOHJEITA
Tämä varoitusmerkki merkitsee vaaraa. Tilanne voi aiheuttaa ruumiillisia
vammoja. Ennen kuin käsittelet laitteistoa, huomioi sähköpiirien
käsittelemiseen liittyvät riskit ja tutustu onnettomuuksien yleisiin
ehkäisytapoihin. Turvallisuusvaroitusten käännökset löytyvät laitteen
mukana toimitettujen käännettyjen turvallisuusvaroitusten joukosta
varoitusten lopussa näkyvien lausuntonumeroiden avulla.
SÄILYTÄ NÄMÄ OHJEET

xiii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
Conventions
Attention
IMPORTANTES INFORMATIONS DE SÉCURITÉ
Ce symbole d'avertissement indique un danger. Vous vous trouvez dans une
situation pouvant entraîner des blessures ou des dommages corporels. Avant
de travailler sur un équipement, soyez conscient des dangers liés aux circuits
électriques et familiarisez-vous avec les procédures couramment utilisées
pour éviter les accidents. Pour prendre connaissance des traductions des
avertissements figurant dans les consignes de sécurité traduites qui
accompagnent cet appareil, référez-vous au numéro de l'instruction situé à la
fin de chaque avertissement.
CONSERVEZ CES INFORMATIONS
Warnung
WICHTIGE SICHERHEITSHINWEISE
Dieses Warnsymbol bedeutet Gefahr. Sie befinden sich in einer Situation, die
zu Verletzungen führen kann. Machen Sie sich vor der Arbeit mit Geräten mit
den Gefahren elektrischer Schaltungen und den üblichen Verfahren zur
Vorbeugung vor Unfällen vertraut. Suchen Sie mit der am Ende jeder Warnung
angegebenen Anweisungsnummer nach der jeweiligen Übersetzung in den
übersetzten Sicherheitshinweisen, die zusammen mit diesem Gerät
ausgeliefert wurden.
BEWAHREN SIE DIESE HINWEISE GUT AUF.
Avvertenza
IMPORTANTI ISTRUZIONI SULLA SICUREZZA
Questo simbolo di avvertenza indica un pericolo. La situazione potrebbe
causare infortuni alle persone. Prima di intervenire su qualsiasi
apparecchiatura, occorre essere al corrente dei pericoli relativi ai circuiti
elettrici e conoscere le procedure standard per la prevenzione di incidenti.
Utilizzare il numero di istruzione presente alla fine di ciascuna avvertenza per
individuare le traduzioni delle avvertenze riportate in questo documento.
CONSERVARE QUESTE ISTRUZIONI

Preface
Conventions
xiv
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Advarsel
VIKTIGE SIKKERHETSINSTRUKSJONER
Dette advarselssymbolet betyr fare. Du er i en situasjon som kan føre til skade
på person. Før du begynner å arbeide med noe av utstyret, må du være
oppmerksom på farene forbundet med elektriske kretser, og kjenne til
standardprosedyrer for å forhindre ulykker. Bruk nummeret i slutten av hver
advarsel for å finne oversettelsen i de oversatte sikkerhetsadvarslene som
fulgte med denne enheten.
TA VARE PÅ DISSE INSTRUKSJONENE
Aviso
INSTRUÇÕES IMPORTANTES DE SEGURANÇA
Este símbolo de aviso significa perigo. Você está em uma situação que poderá
ser causadora de lesões corporais. Antes de iniciar a utilização de qualquer
equipamento, tenha conhecimento dos perigos envolvidos no manuseio de
circuitos elétricos e familiarize-se com as práticas habituais de prevenção de
acidentes. Utilize o número da instrução fornecido ao final de cada aviso para
localizar sua tradução nos avisos de segurança traduzidos que acompanham
este dispositivo.
GUARDE ESTAS INSTRUÇÕES
¡Advertencia!
INSTRUCCIONES IMPORTANTES DE SEGURIDAD
Este símbolo de aviso indica peligro. Existe riesgo para su integridad física.
Antes de manipular cualquier equipo, considere los riesgos de la corriente
eléctrica y familiarícese con los procedimientos estándar de prevención de
accidentes. Al final de cada advertencia encontrará el número que le ayudará
a encontrar el texto traducido en el apartado de traducciones que acompaña
a este dispositivo.
GUARDE ESTAS INSTRUCCIONES

xv
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
Conventions
Varning!
VIKTIGA SÄKERHETSANVISNINGAR
Denna varningssignal signalerar fara. Du befinner dig i en situation som kan
leda till personskada. Innan du utför arbete på någon utrustning måste du vara
medveten om farorna med elkretsar och känna till vanliga förfaranden för att
förebygga olyckor. Använd det nummer som finns i slutet av varje varning för
att hitta dess översättning i de översatta säkerhetsvarningar som medföljer
denna anordning.
SPARA DESSA ANVISNINGAR

Preface
Related Documentation
xvi
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Related Documentation
For instructions on installing and configuring Cisco SFS 7000P and 7008P
switches, refer to these publications:
•Cisco SFS 7008P Hardware Installation Guide
•Cisco SFS 7000P Switch Installation and Configuration Note
•Cisco SFS 7008P Switch Installation and Configuration Note
•Cisco SFS 7000 Series Product Family Chassis Manager User Guide
•Cisco SFS 7000 Series Product Family Element Manager User Guide
•Regulatory Compliance and Safety Information for the Cisco SFS 7000 Series
Switches

xvii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
Obtaining Documentation
Obtaining Documentation
Cisco documentation and additional literature are available on Cisco.com. Cisco
also provides several ways to obtain technical assistance and other technical
resources. These sections explain how to obtain technical information from Cisco
Systems.
Cisco.com
You can access the most current Cisco documentation at this URL:
http://www.cisco.com/techsupport
You can access the Cisco website at this URL:
http://www.cisco.com
You can access international Cisco websites at this URL:
http://www.cisco.com/public/countries_languages.shtml
Product Documentation DVD
The Product Documentation DVD is a comprehensive library of technical product
documentation on a portable medium. The DVD enables you to access multiple
versions of installation, configuration, and command guides for Cisco hardware
and software products. With the DVD, you have access to the same HTML
documentation that is found on the Cisco website without being connected to the
Internet. Certain products also have .PDF versions of the documentation
available.
The Product Documentation DVD is available as a single unit or as a subscription.
Registered Cisco.com users (Cisco direct customers) can order a Product
Documentation DVD (product number DOC-DOCDVD= or
DOC-DOCDVD=SUB) from Cisco Marketplace at this URL:
http://www.cisco.com/go/marketplace/

Preface
Documentation Feedback
xviii
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Ordering Documentation
Registered Cisco.com users may order Cisco documentation at the Product
Documentation Store in the Cisco Marketplace at this URL:
http://www.cisco.com/go/marketplace/
Nonregistered Cisco.com users can order technical documentation from 8:00 a.m.
to 5:00 p.m. (0800 to 1700) PDT by calling 1 866 463-3487 in the United States
and Canada, or elsewhere by calling 011 408 519-5055. You can also order
documentation by e-mail at [e-mail] or by fax at
1 408 519-5001 in the United States and Canada, or elsewhere at 011 408
519-5001.
Documentation Feedback
You can rate and provide feedback about Cisco technical documents by
completing the online feedback form that appears with the technical documents
on Cisco.com.
You can submit comments about Cisco documentation by using the response card
(if present) behind the front cover of your document or by writing to the following
address:
Cisco Systems
Attn: Customer Document Ordering
170 West Tasman Drive
San Jose, CA 95134-9883
We appreciate your comments.
Cisco Product Security Overview
Cisco provides a free online Security Vulnerability Policy portal at this URL:
http://www.cisco.com/en/US/products/products_security_vulnerability_policy.ht
ml

xix
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
Preface
Cisco Product Security Overview
From this site, you will find information about how to:
•Report security vulnerabilities in Cisco products.
•Obtain assistance with security incidents that involve Cisco products.
•Register to receive security information from Cisco.
A current list of security advisories, security notices, and security responses for
Cisco products is available at this URL:
http://www.cisco.com/go/psirt
To see security advisories, security notices, and security responses as they are
updated in real time, you can subscribe to the Product Security Incident Response
Team Really Simple Syndication (PSIRT RSS) feed. Information about how to
subscribe to the PSIRT RSS feed is found at this URL:
http://www.cisco.com/en/US/products/products_psirt_rss_feed.html
Reporting Security Problems in Cisco Products
Cisco is committed to delivering secure products. We test our products internally
before we release them, and we strive to correct all vulnerabilities quickly. If you
think that you have identified a vulnerability in a Cisco product, contact PSIRT:
•For Emergencies only —[email protected]
An emergency is either a condition in which a system is under active attack
or a condition for which a severe and urgent security vulnerability should be
reported. All other conditions are considered nonemergencies.
•For Nonemergencies—[email protected]
In an emergency, you can also reach PSIRT by telephone:
•1 877 228-7302
•1 408 525-6532
Tip We encourage you to use Pretty Good Privacy (PGP) or a compatible product (for
example, GnuPG) to encrypt any sensitive information that you send to Cisco.
PSIRT can work with information that has been encrypted with PGP versions 2.x
through 9.x.
Never use a revoked or an expired encryption key. The correct public key to use

Preface
Obtaining Technical Assistance
xx
Cisco SFS 7000P Hardware Installation Guide
OL-9167-01 Rev. A0
in your correspondence with PSIRT is the one linked in the Contact Summary
section of the Security Vulnerability Policy page at this URL:
http://www.cisco.com/en/US/products/products_security_vulnerability_policy.ht
ml
The link on this page has the current PGP key ID in use.
If you do not have or use PGP, contact PSIRT at the aforementioned e-mail
addresses or phone numbers before sending any sensitive material to find other
means of encrypting the data.
Obtaining Technical Assistance
Cisco Technical Support provides 24-hour-a-day award-winning technical
assistance. The Cisco Technical Support & Documentation website on Cisco.com
features extensive online support resources. In addition, if you have a valid Cisco
service contract, Cisco Technical Assistance Center (TAC) engineers provide
telephone support. If you do not have a valid Cisco service contract, contact your
reseller.
Cisco Technical Support & Documentation Website
The Cisco Technical Support & Documentation website provides online
documents and tools for troubleshooting and resolving technical issues with Cisco
products and technologies. The website is available 24 hours a day, at this URL:
http://www.cisco.com/techsupport
Access to all tools on the Cisco Technical Support & Documentation website
requires a Cisco.com user ID and password. If you have a valid service contract
but do not have a user ID or password, you can register at this URL:
http://tools.cisco.com/RPF/register/register.do
Note Use the Cisco Product Identification (CPI) tool to locate your product serial
number before submitting a web or phone request for service. You can access the
CPI tool from the Cisco Technical Support & Documentation website by clicking