IBM SAN384B-2 Manual

IBM System Storage SAN384B-2
Installation, Service, and User Guide
Service information: 2499-416
GA32-0894-04
Read Before Using
This product contains software that is licensed under written license agreements. Your use of such software is subject to
the license agreements under which they are provided.


IBM System Storage SAN384B-2
Installation, Service, and User Guide
Service information: 2499-416
GA32-0894-04


Note:
Before using this information and the product it supports, read the information in “Notices” on page 155.
Portions Copyright © 2011, 2012 Brocade Communications Systems, Inc. All Rights Reserved.
The paragraph does not apply to any country (or region) where such provisions are inconsistent with local law.
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT
WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states (or
regions) do not allow disclaimer of express or implied warranties in certain transactions; therefore, this statement
may not apply to you.
© Copyright IBM Corporation 2011, 2013.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.

Read this first
Summary of changes
This is the fifth edition of the IBM System Storage SAN384B-2 Installation, Service,
and User Guide. The content changes since the last edition of this publication are
noted by a vertical line placed in the left margin beside each change. Minor edits
are not identified by this mark. A summary of the changes for each edition are
listed below.
Fifth edition
The changes were made in this edition:
vAddition of information regarding the need for clean optics with higher
transmission speeds.
vAddition of information for tuning the system.
vLink troubleshooting information added.
Fourth edition
The changes were made in this edition:
vAddition of Enhanced 8 Gbps 32-port blade–Feature code 3633
vAddition of Enhanced 8 Gbps 48-port blade–Feature code 3649
vAddition of FIPS tamper-proof seals–Feature code 7230
vEnhanced inter-chassis link (ICL) capability–Feature code 7872
vUpdated Fabric OS version to 7.0.1 or higher
Third edition
The changes were made in this edition:
vAddition of FICON Accelerator and FICON with CUP Activation optional
features
vUpdated FOS version to 7.0.0c
vAdded port speed to Encryption blade information
Second edition
The changes were made in the second edition:
vThe addition of 16 Gbps long wave length (LWL) SFP feature
vThe addition of encryption, including the Encryption Blade (FS8-18)
vUpdated instructions on ordering the lift tool load plate (now included with the
lift tool)
vOther minor content updates
Getting help
For the latest version of your product documentation, visit the web at
www.ibm.com/shop/publications/order. Search by form number or title.
For more information about IBM®SAN products, see the website:
www.ibm.com/servers/storage/san/
© Copyright IBM Corp. 2011, 2013 iii
|
|
|
|
|
|
|

IBM Redbooks®often provide in depth information about product best practices,
configurations, and more technical information. For redbooks associated with this
product, enter search terms on the website: www.redbooks.ibm.com/.
For support information for this and other IBM products, see the IBM Support
Portal, www.ibm.com/supportportal. Search for the product Machine type or
product name.
For Fabric OS Release Notes and access to Fabric OS firmware downloads, go to
the IBM Support Portal, www.ibm.com/supportportal. Search for the product
Machine type or product name, and then follow links for Downloads. More
detailed instructions are available through the Accessing firmware updates and
OS documentation updates link on the product documentation CD that is shipped
with this product.
You can also contact IBM Service within the United States at 1-800-IBMSERV
(1-800-426-7378). For support outside the United States, you can find the service
number at: www.ibm.com/planetwide/.
To expedite your call, before contacting IBM Service, please have the information
available:
vMachine type and model (2499-416)
vProduct serial number
vLicense ID (you can use the licenseIdShow command to display the license ID)
vWorld Wide Name (WWN)
vFOS version installed
vError numbers and messages received
vsupportSave command output
vDetailed description of the problem and specific questions
vDescription of any troubleshooting steps already performed and the results
vSerial console and Telnet session logs
vSyslog message logs
Visit www.ibm.com/contact for the contact information for your country or region.
For detailed information about the Fibre Channel standards, see the Fibre Channel
Industry Association (FCIA) website at: www.fibrechannel.org/
For information about storage industry standards, see the Storage Networking
Industry Association (SNIA) website at: www.snia.org/
Taiwan Contact Information
IBM Taiwan Product Service Contact Info:
IBM Taiwan Corporation
3F, No 7, Song Ren Rd., Taipei Taiwan
Tel: [phone]
f2c00790
iv SAN384B-2 Installation, Service, and User Guide

How to send your comments
Your feedback is important in helping us provide the most accurate and
high-quality information. If you have comments or suggestions for improving this
document, send us your comments by email to [email protected] . Be sure to
include the information:
vExact publication title
vPublication form number (for example, GC26-1234-02)
vPage, table, or illustration numbers
vA detailed description of any information that should be changed
Read this first v

vi SAN384B-2 Installation, Service, and User Guide

Contents
Read this first ............iii
Summary of changes ...........iii
Fifth edition .............iii
Fourth edition ............iii
Third edition .............iii
Second edition ............iii
Getting help ..............iii
Taiwan Contact Information ........iv
How to send your comments .........v
Figures ...............xi
Tables ...............xiii
Safety notices ...........xv
Safety notices and labels ..........xv
Danger notices ............xv
Caution notices ...........xviii
Safety labels .............xx
Attention notices ...........xxi
Rack safety ..............xxii
Rack installation ...........xxii
Rack relocation (19" rack) ........xxiii
Safety inspections ...........xxiv
Removing AC power .........xxiv
External machine checks ........xxiv
Internal machine checks ........xxiv
Product recycling and disposal .......xxv
About this document .......xxvii
Who should read this document ......xxvii
Product documents ...........xxvii
Brocade documents ...........xxvii
IBM and Brocade product matrix .....xxviii
Accessibility features for SAN384B-2 ....xxviii
Chapter 1. Introduction ........1
Overview of the SAN384B-2 .........1
Hardware components ...........2
SAN384B-2 blades ............5
High availability .............7
Reliability ...............8
Serviceability ..............8
Software features .............9
Security ...............9
Network Manageability .........10
Chapter 2. Installing a SAN384B-2 in a
cabinet...............11
Items included with the SAN384B-2 ......11
Ordering the lift tool ...........12
World trade locations ..........12
United States locations .........12
Installation guidelines ...........13
Installing a SAN384B-2 in a 2109 C36 cabinet . . . 14
Time required ............14
Unpacking and assembling the lift tool ....16
Unpacking the SAN384B-2 ........17
Installing the port-side exhaust kit......18
Torque requirements ..........21
Installing the exhaust kit hardware .....21
Installing the SAN384B-2 into the cabinet . . . 25
Port numbering .............29
Chassis slots ..............31
Cable organization ............31
Chapter 3. Starting and configuring the
SAN384B-2 .............33
Providing power to the SAN384B-2 ......35
Connecting the SAN384B-2 and host with a serial
cable ................35
Logging in to the serial console port ......36
Configuring the IP addresses ........36
Logging off the serial console port and
disconnecting the serial cable .......37
Establishing an Ethernet connection ......38
Customizing the switch name ........38
Customizing a chassis name .........38
Setting the Domain ID...........39
Setting the date and time ..........39
Setting the date ............39
Setting the time zone ..........40
Synchronizing local time .........40
Verifying the persistent PID feature ......41
Determining installed software licenses .....41
Installing transceivers and attaching cables ....42
Managing cables.............44
Planning for encryption (optional) .......45
Configuring for encryption (optional)......46
Summary of procedure .........46
Avoid double encryption .........47
Verifying correct operation and backing up the
configuration ..............47
Fabric OS firmware updates .........48
Downloading and installing firmware ....49
Powering off the SAN384B-2 ........49
Chapter 4. Monitoring system
components ............51
Port, application, or encryption blade status . . . 55
Control processor blade (CP8) status ......59
Core switch blade (CR16-4) status .......61
Power supply status ...........63
Blower assembly status ..........63
WWN card status ............65
© Copyright IBM Corp. 2011, 2013 vii
||

Chapter 5. Removing and installing
components ............67
Removing and installing the chassis door ....67
Time and items required .........67
Removing the chassis door ........67
Removing and installing cable management finger
assemblies ...............68
Time and items required .........68
Removing a cable management finger assembly 68
Installing a cable management finger assembly 69
Removing and installing port, application, and
encryption blades ............70
Time and items required .........70
Removing a port or application blade ....70
Installing a blade ...........72
Removing and installing blade filler panels....72
Removing a filler panel .........72
Installing a filler panel..........73
Removing and installing a CP8 control processor
blade ................73
Time and items required .........74
Verifying the need for replacement .....74
Recording critical SAN384B-2 information . . . 75
Removing a control processor blade (CP8) . . . 76
Installing a control processor blade (CP8) . . . 77
Verifying operation of the new CP blade....78
Downloading firmware from an FTP server. . . 78
Downloading firmware from a USB device . . . 80
Completing the CP8 blade replacement ....81
Removing and installing a CR16-4 core switch blade 82
Verifying the necessity of replacement ....82
Time and items required .........82
Removing a CR16-4 core switch blade ....83
Installing a CR16-4 core switch blade .....83
Removing and installing a power supply ....84
Time and items required .........84
Removing a power supply ........84
Installing a power supply ........86
Removing and installing a blower assembly . . . 86
Time and items required .........86
Removing a blower assembly .......87
Installing a blower assembly........87
Removing and installing a WWN bezel and WWN
card.................88
Time and items required .........88
Verifying necessity of replacement ......88
Removing the WWN bezel and WWN card. . . 88
Installing the WWN bezel and WWN card . . . 89
Removing and installing transceivers ......90
Time required ............90
Items required ............90
Removing and installing an SFP+ optical
transceiver..............90
Removing and installing an mSFP optical
transceiver..............92
Removing and replacing inter-chassis link (QSFP)
cables ................93
Time and items required .........94
Removing an inter-chassis link (QSFP) cable . . 95
Installing an ICL cable..........95
Possible ICL configurations ........96
Removing and replacing a SAN384B-2 chassis . . . 98
Time required ............99
Items required ............99
Verifying need for replacement .......99
Recording critical SAN384B-2 and SAN
information .............99
Disconnecting from the network and fabric . . 102
Removing components from the chassis . . . 103
Removing a SAN384B-2 from the cabinet . . . 104
Installing the replacement chassis .....105
Installing components into the new chassis . . 105
Downloading the configuration ......106
Verifying correct operation of system ....107
Reconnecting the system to the network and
fabric ...............108
Verifying correct configuration of the fabric . . 108
Cable routing table template .......110
Removing the batteries ..........111
Chapter 6. Installing new features . . 115
FC3632, FC3648- Installing a 16-Gb port blade (32
or 48 ports) ..............115
Time required ............115
Items required ............116
Procedures .............116
FC3633, FC3649- Installing an Enhanced 8-Gb port
blade (32 or 48 ports) ...........117
Time required ............117
Items required ............117
Procedures .............117
FC3864 - Installing an 8-Gb 64-port port blade . . 118
Time required ............118
Items required ............118
Procedures .............119
FC3890 - Installing an 8 Gbps Extension Blade
(FX8-24) ...............120
Time required ............120
Items required ............120
Procedures .............120
FC3895 - Installing an Encryption Blade (FS8-18) 121
Time required ............121
Items required ............122
Procedures .............122
Avoid double encryption ........123
FC7230 - Installing FIPS 140-2 Level 2 tamper proof
seals ................123
FC7871 - Installing a QSFP and inter-chassis license 123
Time required ............123
Items required ............123
Procedures .............123
FC7872 - Installing a QSFP and Enterprise
inter-chassis license ...........124
Time required ............124
Items required ............124
Procedures .............124
Appendix A. Product specifications 125
General specifications ..........125
System architecture ...........125
System size and weights..........127
viii SAN384B-2 Installation, Service, and User Guide

System blade and FRU weights ......128
Facility requirements ...........128
Environmental requirements ........129
Fibre Channel port specifications .......129
Power specifications ...........130
Data transmission ranges .........132
Power cords ..............132
Appendix B. Application and
encryption blades .........133
FS8-18 blade..............133
FX8-24 blade .............133
Appendix C. Diagnostics and
troubleshooting ..........135
Obtaining chassis and component status ....135
Interpreting POST and boot results ......136
POST ...............136
Boot ...............137
Diagnostics ..............137
General troubleshooting ..........137
Link troubleshooting ...........140
Fault isolation ............140
Dust, dirt, or other contaminants ......141
Best practices for minimizing link loss ....141
Attenuation on LWL connections ......142
Appendix D. Tuning the system . . . 145
Identifying whether you need to tune your system 145
Before tuning ............147
Appendix E. Port numbering .....151
Notices ..............155
Trademarks ..............156
Electronic emission notices .........156
Federal Communications Commission (FCC)
Class A Statement ...........156
Industry Canada Class A Emission Compliance
Statement .............157
Avis de conformité à la réglementation
d'Industrie Canada ..........157
European Union EMC Directive Conformance
Statement .............157
Germany Electromagnetic Compatibility
Directive ..............157
People's Republic of China Class A Electronic
Emission Statement ..........158
Japan Voluntary Control Council for Interference
(VCCI) Class A Statement ........159
Japan Electronics and Information Technology
Industries Association (JEITA) Statement . . . 159
Korea Communications Commission (KCC)
Class A Statement ...........159
Russia Electromagnetic Interference (EMI) Class
A Statement .............159
Australia and New Zealand Class A Statement 159
Taiwan Class A Electronic Emission Statement 160
Index ...............161
Contents ix
||
||
||
||
||
||
||
||

xSAN384B-2 Installation, Service, and User Guide

Figures
1. Port side of the SAN384B-2 (sample
configuration) ............3
2. Port side of the SAN384B-2 with the exhaust kit
installed (sample configuration) ......4
3. Blower (non-port) side of the SAN384B-2 5
4. Assembled lift tool with 24-inch load plate 17
5. Lift tool positioned next to the chassis on the
shipping tray ............18
6. Port-side exhaust kit assembly for 27 to 31 in.
(68.58 to 78.74 cm) cabinets .......20
7. Clip and retainer nut locations on exhaust side
cabinet rails (example showing relative
positions) .............22
8. Shelf installed in cabinet ........23
9. Air duct installed into side slot in shelf 23
10. Installing the top rail assembly to the duct
assembly and cabinet rails .......24
11. Lift tool and chassis positioned at rear of
cabinet ..............25
12. Load plate extended inside the cabinet to the
exhaust kit shelf ...........26
13. Chassis half way into cabinet on load plate 27
14. Location of screws to attach the chassis to the
cabinet rails ............28
15. Attaching the vertical cable management
finger assemblies to the cabinet rails ....29
16. Cable design for the mSFP patch cables for the
FC8-64 high density port blade ......32
17. Flow of configuration tasks for the SAN384B-2 34
18. Encryption configuration ........47
19. FC16-32 port blade ..........55
20. FC16-48 port blade ..........56
21. FC8-32E port blade ..........56
22. FC8-48E port blade ..........56
23. FC8-64 port blade ..........57
24. FS8-18 encryption blade ........57
25. FX8-24 extension blade.........58
26. Control processor blade (CP8) ports and LEDs 60
27. Core switch blade (CR16-4) .......61
28. Power supply ............63
29. Blower assembly LEDs.........64
30. WWN bezel ............66
31. Chassis door ............68
32. Cable management finger assemblies ....69
33. Port, application, and encryption blade
removal and installation (FC16-48 port blade
shown) ..............71
34. Filler panel removal and installation ....73
35. Removing the control processor (CP8) blade 77
36. Removing and replacing the core switch blade 83
37. Location of the two power supplies ....85
38. Removing and installing a power supply 86
39. Blower assembly removal and replacement 87
40. WWN bezel and card removal and installation 89
41. Optical transceiver (SFP+) extraction tool 91
42. Installing and removing a 16 Gbps SFP+ with
pull tab (shown without cable attached) . . . 91
43. Removing and replacing an SFP+ optical
transceiver .............92
44. Optical mSFP transceiver ........93
45. QSFP cable and transceiver .......94
46. QSFP cable and transceiver (bail open) 95
47. QSFP cable connections for two SAN768B-2
chassis (sample configuration) ......96
48. Core/edge ICL topology for SAN384B-2 and
SAN768B-2 chassis ..........97
49. full mesh ICL topology for SAN384B-2 and
SAN768B-2 chassis ..........98
50. Battery locations on the CP8 blade ....112
51. Encryption blade battery location .....113
52. Identifying the origin of failure .....140
53. CR16-4 core blade ..........151
54. FC16-32 port blade..........152
55. FC16-48 port blade..........152
56. FC8-32E port blade .........152
57. FC8-48E port blade .........153
58. FC8-64 port blade ..........153
59. FS8-18 encryption blade ........153
60. FX8-24 extension blade ........153
© Copyright IBM Corp. 2011, 2013 xi
||

xii SAN384B-2 Installation, Service, and User Guide

Tables
1. Brocade and IBM product and model
number matrix ..........xxviii
2. Blades available for the SAN384B-2 .....5
3. Installation tasks, time, and items required 15
4. Parts list for port-side exhaust kit for C36
cabinets ..............19
5. Torque requirements for mounting screws 21
6. Port and application blades available on the
SAN384B-2.............29
7. Configuration parameters ........36
8. Port, application, and encryption blade LED
descriptions ............58
9. CP blade LED descriptions .......60
10. CR16-4 blade LED descriptions ......62
11. Power supply LED descriptions......63
12. Blower assembly LED descriptions .....64
13. Messages that may indicate WWN card failure 65
14. Steps for upgrading through multiple versions
of Fabric OS ............78
15. ICL connector port LEDs ........94
16. Critical information checklist example 100
17. Cable routing table for SAN384B-2 (64 ports
shown)..............110
18. General product specifications ......125
19. System architecture .........125
20. System sizes and weights .......127
21. System FRU weights .........128
22. Environmental requirements ......129
23. Power specifications .........130
24. Power demands per component .....131
25. Supported optics, speeds, cables, and
distances .............132
26. Environmental status and maintenance
commands ............136
27. Troubleshooting...........137
28. Specifications of LWL 10km transceivers 142
29. Maximum receive power of 2 Gbps LWL SFPs 143
30. External port to slotShow port mapping for
CR16-4 core blades..........151
© Copyright IBM Corp. 2011, 2013 xiii
||
||

xiv SAN384B-2 Installation, Service, and User Guide

Safety notices
This section contains information about:
v“Safety notices and labels”
v“Rack safety” on page xxii
v“Safety inspections” on page xxiv
Safety notices and labels
When using this product, observe the danger, caution, and attention notices
contained in this guide. The notices are accompanied by symbols that represent the
severity of the safety condition. The danger and caution notices are listed in
numerical order based on their IDs, which are displayed in parentheses, for
example (D004), at the end of each notice. Use this ID to locate the translation of
these danger and caution notices in the Safety Notices publication that is shipped
with this product.
The notices and statements are used in IBM documents. They are listed below in
order of increasing severity of potential hazards. Follow the links for more detailed
descriptions and examples of the danger, caution, and attention notices in the
sections that follow.
vNote: These notices provide important tips, guidance, or advice.
v“Attention notices” on page xxi: These notices indicate potential damage to
programs, devices, or data.
v“Caution notices” on page xviii: These statements indicate situations that can be
potentially hazardous to you.
v“Danger notices”: These statements indicate situations that can be potentially
lethal or extremely hazardous to you. Safety labels are also attached directly to
products to warn of these situations.
vIn addition to these notices, “Safety labels” on page xx may be attached to the
product to warn of potential hazards.
Danger notices
A danger notice calls attention to a situation that is potentially lethal or extremely
hazardous to people. A lightning bolt symbol accompanies a danger notice to
represent a dangerous electrical condition. Read and comply with the danger
notices before installing or servicing this device.
DANGER
To prevent a possible shock from touching two surfaces with
different protective ground (earth), use one hand, when possible, to
connect or disconnect signal cables. (D001)
© Copyright IBM Corp. 2011, 2013 xv

DANGER
Overloading a branch circuit is potentially a fire hazard and a
shock hazard under certain conditions. To avoid these hazards,
ensure that your system electrical requirements do not exceed
branch circuit protection requirements. Refer to the information
that is provided with your device or the power rating label for
electrical specifications. (D002)
DANGER
If the receptacle has a metal shell, do not touch the shell until you
have completed the voltage and grounding checks. Improper wiring
or grounding could place dangerous voltage on the metal shell. If
any of the conditions are not as described, STOP. Ensure the
improper voltage or impedance conditions are corrected before
proceeding. (D003)
DANGER
An electrical outlet that is not correctly wired could place
hazardous voltage on metal parts of the system or the devices that
attach to the system. It is the responsibility of the customer to
ensure that the outlet is correctly wired and grounded to prevent
an electrical shock. (D004)
A general electrical danger notice provides instructions on how to avoid shock
hazards when servicing equipment. Unless instructed otherwise, follow the
procedures in the danger notice.
xvi SAN384B-2 Installation, Service, and User Guide

DANGER
When working on or around the system, observe the precautions:
Electrical voltage and current from power, telephone, and
communication cables are hazardous. To avoid a shock hazard:
vConnect power to this unit only with the IBM provided power
cord. Do not use the IBM provided power cord for any other
product.
vDo not open or service any power supply assembly.
vDo not connect or disconnect any cables or perform installation,
maintenance, or reconfiguration of this product during an
electrical storm.
vThe product might be equipped with multiple power cords. To
remove all hazardous voltages, disconnect all power cords.
vConnect all power cords to a properly wired and grounded
electrical outlet. Ensure that the outlet supplies proper voltage
and phase rotation according to the system rating plate.
vConnect any equipment that will be attached to this product to
properly wired outlets.
vWhen possible, use one hand only to connect or disconnect
signal cables.
vNever turn on any equipment when there is evidence of fire,
water, or structural damage.
vDisconnect the attached power cords, telecommunications
systems, networks, and modems before you open the device
covers, unless instructed otherwise in the installation and
configuration procedures.
vConnect and disconnect cables as described below when
installing, moving, or opening covers on this product or attached
devices.
To disconnect:
1. Turn off everything (unless instructed otherwise).
2. Remove the power cords from the outlets.
3. Remove the signal cables from the connectors.
4. Remove all cables from the devices.
To connect:
1. Turn off everything (unless instructed otherwise).
2. Attach all cables to the devices.
3. Attach the signal cables to the connectors.
4. Attach the power cords to the outlets.
5. Turn on the devices.
(D005)
The weight of the SAN cabinet (2109-C36) with two SAN384B-2s is greater than
227 kg (500 lb), and has a fully loaded capacity of 816 kg (1795 lb). Under these
conditions, the statement and notice apply.
Safety notices xvii

Delivery and subsequent transportation of the equipment
The customer should prepare his environment to accept the new product based on
the installation planning information provided, with assistance from an IBM
Installation Planning Representative (IPR) or IBM authorized service provider. In
anticipation of the equipment delivery, the final installation site should be prepared
in advance such that professional movers/riggers can transport the equipment to
the final installation site within the computer room. If for some reason, this is not
possible at the time of delivery, the customer will need to make arrangements to
have professional movers/riggers return to finish the transportation at a later date.
Only professional movers/riggers should transport the equipment. The IBM
authorized service provider will only perform minimal frame repositioning within
the computer room, as needed, to perform required service actions. The customer
is also responsible for using professional movers/riggers in the case of equipment
relocation or disposal.
>(>)500 lbs. 227 kg.
a69i0333
DANGER
Heavy equipment—personal injury or equipment damage might
result if mishandled. (D006)
Caution notices
A caution notice calls attention to a situation that is potentially hazardous to
people because of some existing condition. A caution notice can be accompanied
by different symbols, as in the examples below:
If the symbol
is... It means....
A hazardous electrical condition with less severity than electrical danger.
A generally hazardous condition not represented by other safety
symbols.
svc00169
55 kg ( 121.2 lbs)
>55kg (121.2 lb)
A specification of product weight that requires safe lifting practices. The
weight range of the product is listed below the graphic, and the wording
of the caution varies, depending on the weight of the device.
P/N 18P5850-B
SJ000752
A potential hazard of pinching the hand or other body parts between
parts.
A hazardous condition due to moving parts nearby.
A hazardous condition due to the use of a laser in the product. Laser
symbols are always accompanied by the classification of the laser as
defined by the U.S. Department of Health and Human Services (for
example, Class I, Class II, and so forth).
xviii SAN384B-2 Installation, Service, and User Guide