Cisco SFS 7008P Operating and maintenance instructions

Corporate Headquarters:
© 2006 Cisco Systems, Inc. All rights reserved.
Cisco Systems, Inc., 170 West Tasman Drive, San Jose, CA 95134-1706 USA
Cisco SFS 7008P InfiniBand Server Switch
Installation and Configuration Note
Product number: PID=78-17434-01 Rev. A0
This installation and configuration note describes how to install and configure the Cisco SFS 7008P
switch. This publication contains these sections:
•Safety Overview, page 2
•Parts List, page 2
•Required Tools, Equipment, and Personnel, page 2
•Before You Install, page 3
•Installing the Cisco SFS 7008P Switch into a Rack, page 6
•Troubleshooting, page 20
•Translated Safety Warnings, page 21
•Obtaining Documentation, page 27
•Documentation Feedback, page 27
•Cisco Product Security Overview, page 28
•Obtaining Technical Assistance, page 29
•Obtaining Additional Publications and Information, page 31

2
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Safety Overview
Safety Overview
Safety warnings appear throughout this publication in procedures that may harm you if performed
incorrectly. A warning symbol precedes each warning statement.
Parts List
The following items are included in your SFS 7008 package:
•Chassis with TopSpin OS v2.5.0 preloaded
•2 Serial Cable Kits
– Serial Cable
– Serial cable document
– DB-9 connector male to RJ-45
– DB-9 connector female to RJ-45
– MX-9-1 roll-over cable
•Power cord for each power supply
•Bezel
•Parts bag
•Regulation and Compliance Statement
•1 year warranty package (Cisco PN 83-1190-01)
Required Tools, Equipment, and Personnel
In addition to the accessories provided with the switch, you should have the following:
•A #2 Phillips screw driver
•Screws (not provided) to attach the mounting brackets to the rack
•Any associated mounting clips to secure the brackets to your rack (two for each rail of the rack)
•A rack with enough clearance for the Cisco SFS 7008P switch and cables.
Warning IMPORTANT SAFETY INSTRUCTIONS
This warning symbol means danger. You are in a situation that could cause bodily injury. Before you
work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar
with standard practices for preventing accidents. Use the statement number provided at the end of
each warning to locate its translation in the translated safety warnings that accompanied this
device.
Statement 1071
SAVE THESE INSTRUCTIONS

3
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Before You Install
•Three people to complete the installation (highly recommended). The Cisco SFS 7008P switch
chassis weighs up to 106 pounds fully loaded.
Before You Install
Before you begin the installation, do the following:
•Make sure that you have the right cables and sufficient ventilation.
•Open the Cisco SFS 7008P package and remove the chassis, rack brackets, parts bag, and
documentation from the box. Remove the packaging from around the chassis. After you have
removed the straps, lift the box up and away from the switch and pallet. Do not cut the packaging,
as you could damage the switch. Save the box and packaging.
•Prepare a management workstation, such as a PC or laptop running a terminal program with a
rollover M/F DB-9 serial cable (included).
Verifying Basic Hardware
If you have not purchased a fully loaded chassis with high-availability components installed, verify that
you have the minimum components necessary to run the chassis. Specific components in the Cisco SFS
7008P switch (see Figure 1 and Figure 2) have interdependencies on other components.
Check for basic hardware by following these steps:
Step 1 Verify that you have at least one management interface module installed in slot 15 at the back of the unit.
(See Figure 1.) Slot 15 is paired with the Fabric Controller in slot 11. (See Figure 2.)

4
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Verifying Basic Hardware
Figure 1 Back of Unit
Step 2 Verify that you have a Chassis ID module installed. (See Figure 1).
Step 3 Verify that you have a Fabric Controller installed in the Core slot (see slot 11 in Figure 2). A second
Fabric Controller module (installed in slot 12) is not required for the chassis to function, but is required
to have a non-blocking fabric.
144988
Management I.O
modules
Power
plug
2
Power
plug
1
Chassis ID
Main air
exhaust
4x copper
line interface
modules

5
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Verifying Basic Hardware
Figure 2 Cisco SFS 7008P Switch Front ViewWithout Bezel
Step 4 Verify that you have at least one power supply installed. (See Figure 2.)
Step 5 Verify that you have at least one fan module installed. (See Figure 2.)
Step 6 Verify that you have at least one fabric module installed in one of the node slots. The node slots are 9,
10, 13 and 14. (See Figure 2.)
Step 7 Verify that you have two LIMs installed for each fabric module in the node slots. See Figure 3 for an
example of the connections between the LIMs and the fabric modules in the node slots:
154497
System
Status LED
pp y
Status LED
Power Supply
identity LED
Power supplies
Fan Fan
Node slot 1 Slot 9
Slot 10
Slot 11
Slot 12
Slot 13
Slot 14
Node slot 2
Node slot 3
Node slot 4
Core slot 1
Core slot 2
Fan tray LED

6
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Installing the Cisco SFS 7008P Switch into a Rack
Figure 3 LIM and Fabric Module Connections
Installing the Cisco SFS 7008P Switch into a Rack
This section describes how to install the Cisco SFS 7008P switch in an equipment rack. If you purchased
the custom Cisco SFS 7008P switch shelf, follow the directions in the Cisco SFS 7008 InfiniBand Server
Switch Hardware Installation Guide. To mount the Cisco SFS 7008P switch chassis in an
industry-standard rack, follow these steps:
Step 1 Install a shelf for the chassis as shown in Figure 4, if you have not already done so.
The shelf must support the weight of the chassis, which can be up to 106 pounds if all possible
components are installed.
Fabric card in slot 9
LIM card in slot 1
LIM card in slot 2
LIM card in slot 3
LIM card in slot 4
LIM card in slot 5
LIM card in slot 6
LIM card in slot 7
LIM card in slot 8
Fabric card in slot 10
Fabric card in slot 13
Fabric card in slot 14
154550

7
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Installing the Cisco SFS 7008P Switch into a Rack
Figure 4 Install Appropriate Shelf
Step 2 Ground yourself using an approved ground wrist-strap.
Step 3 Determine the direction that the switch will be installedand justified in the rack.
The direction that the switch will be justified determines which set of mounting brackets need to be
removed in the next step.
Step 4 Remove one set of mounting brackets, as shown in Figure 5:
•If the switch will be justified toward the bezel-side in the rack, then unscrew the mounting brackets
from the port-side of the switch.
•If the switch will be justified toward the port-side in the rack, then unscrew the mounting brackets
from the bezel-side of the switch.
144964

8
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Installing the Cisco SFS 7008P Switch into a Rack
Figure 5 Unscrew Extra Mounting Brackets from the Chassis
Step 5 Separate the mounting brackets from the chassis when all screws have been loosened.
Step 6 With 2 or 3 people working together, lift the switch into the rack as shown in Figure 6.
Figure 6 Chassis Set onto Shelf
Step 7 Push the chassis into the rack until the mounting brackets are flush with the rails.
Step 8 Use screws that fit your rack to attach both mounting brackets to the rack posts.
144965
144966

9
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Installing the Cisco SFS 7008P Switch into a Rack
Installing Optional High-Availability Components
If you purchased a High-Availability (HA) package, install the following components:
•Redundant Power Supply; see the power supply locations in Figure 7. Install the second power
supply according to the directions in the Power Supply Module Installation Guide in the package.
Figure 7 Power Supplies and Fans Located in the Front of the Unit with the Bezel Removed
•Redundant Fan Tray; see the fan locations in Figure 7. Install the second fan according to the Fan
Module Installation Guide in the package.
•Redundant Management Interface Module; see the location of the Management I/O Modules in
Figure 8. Install the second I/O Module according to the directions in the Management Interface
Module Installation Guide in the package.
154497
System
Status LED
Power Supply
Status LED
Power Supply
identity LED
Power supplies
Fan Fan
Node slot 1 Slot 9
Slot 10
Slot 11
Slot 12
Slot 13
Slot 14
Node slot 2
Node slot 3
Node slot 4
Core slot 1
Core slot 2
Fan tray LED

10
Cisco SFS 7008P InfiniBand Server Switch Installation and Configuration Note
78-17434-01 Rev. A0
Installing the Cisco SFS 7008P Switch into a Rack
Figure 8 Management I/O Modules Located in the Back of the Unit
Configuring and Connecting Host Channel Expansion Card for IBM BladeCenter
This section briefly describes configuring your InfiniBand host. Download the configuration software
and an integrated suite of drivers including IPoIB, SRP, SDP, uDAPL and MPI from the Cisco Software
Center at this URL:
http://www.cisco.com/public/sw-center
Install the Host Channel Expansion Card for IBM BladeCenter by doing the following:
Step 1 Power down the host system following proper grounding procedures.
Step 2 Unpack the HCA module and insert it into an open 64-bit PCI or PCI-X slot.
Step 3 Power on the host system.
Installing Host Channel Expansion Card Linux Drivers
Make sure your Linux kernel is supported. See www.cisco.com for a list of supported versions.
144988
Management I.O
modules
Power
plug
2
Power
plug
1
Chassis ID
Main air
exhaust
4x copper
line interface
modules