Mellanox Technologies MIS5025 User manual

www.mellanox.com
36 Port InfiniBand QDR Switch Platform
Installation Guide
PNs: MIS5025, MIS5030, MIS5035
Rev 1.8

Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: [phone]
Fax: [phone]
Mellanox Technologies, Ltd.
PO Box 586 Hermon Building
Yokneam 20692
Israel
Tel: [phone]
Fax: [phone]
© Copyright 2011. Mellanox Technologies. All rights reserved.
Mellanox®, BridgeX®, ConnectX®, InfiniBlast®, InfiniBridge®, InfiniHost®, InfiniRISC®,
InfiniScale®, InfiniPCI®, PhyX® and Virtual Protocol Interconnect® are registered trademarks
of Mellanox Technologies, Ltd. CORE-Direct and FabricIT are trademarks of Mellanox
Technologies, Ltd. All other marks and names mentioned herein may be trademarks of their
respective companies.
36 Port InfiniBand QDR Switch Platform Installation Guide
Document Number: 3135
Rev 1.8
Mellanox Technologies
2
NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE
PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE
OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE
CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX
TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX
TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE
HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE
DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT,
INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO,
PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND
RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Rev 1.8
Mellanox Technologies 3
Table of Contents
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Internally Managed vs. Externally Managed . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Installing the Switch in the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Minimum and Maximum Rack Depth for this Switch . . . . . . . . . . . . . . . . . .11
Installing the Switch in the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Grounding the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Configuring the Switch for the First Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Unified Fabric Manager (UFM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
FabricIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Starting an SSH Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Starting a WebUI Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Resetting the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Rev 1.8
Mellanox Technologies
4
List of Figures
Figure 1: Rack Rail Kit Parts MIS00083/85 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Figure 2: Rack Rail Kit Parts MIS000079 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Figure 3: Installation Options Available for Short Switches . . . . . . . . . . . . . . . . . . . . . . . .14
Figure 4: Installation Options Available for Standard Length Switches . . . . . . . . . . . . . . .14
Figure 5: For Short Switches Which Side of the Rack Do You Want the Connectors ? . . .15
Figure 6: Making Room for the Power Cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Figure 7: Screwing on the Bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Figure 8: Screwing on the Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 9: Inserting the Caged Nuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 10: Connect Bracket to Rack Vertical support . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Figure 11: Externally Managed Management Interface . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Figure 12: License Key Entitlement Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Figure 13: License Key Generation Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Figure 14: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Rev 1.8
Mellanox Technologies 5
List of Tables
Table 1: Revision History of this User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Table 2: Reference Documents and Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Table 3: Switch Management by OPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Table 4: Switches and Rail Kits for Various Rack Depths . . . . . . . . . . . . . . . . . . . . . . . . .10
Table 5: Mellanox Part Numbering Legend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Table 6: Serial Terminal Program Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Table 7: Configuration Wizard Session - IP Configuration by DHCP . . . . . . . . . . . . . . . .21
Table 8: Configuration Wizard Session - IP Zeroconf Configuration . . . . . . . . . . . . . . . . .23
Table 9: Configuration Wizard Session - Static IP Configuration . . . . . . . . . . . . . . . . . . .24

Rev 1.8
Mellanox Technologies
6
Revision History
Table 1 - Revision History of this User’s Manual
Revision Date Details
1.8 November
2012 • Added Note to See “Installing the Switch in the
Rack” on page 12.regarding lack of support for stan-
dard length switch installation with power side next to
the vertical support
• Added Figure 4 and Figure 5 and text regarding lack
of support for Standard installation with the connector
side deeper in the rack
1.7 April 2012 Fixed disclaimer typo
Fixed TOC hierarchy
1.6 Sept. 2011 Changes to the Reset button section
Removed 5031switch
1.5 May 2011 Added section on UFM
1.4 July 2010 Added files to change this into a book.Added last page
with barcode
1.3 June 2010 Added “Getting the License Key”
1.2 May 2010 Added Note: ETH Ports Must be 10/100M
1.1 April 2010 Minor formatting
1.0 April 2010 Initial Release

Rev 1.8
Mellanox Technologies 7
About this Manual
This manual provides installation and set-up instructions for the IS50XX series QSFP top of rack
InfiniBand Switch Platform.
Intended Audience
This manual is intended for users and system administrators responsible for installing and setting
up the switch platform.
The manual assumes familiarity with the InfiniBand®architecture specification.
Related Documentation
The documentation set accompanying the QSFP Chassis InfiniBand Switch platform includes the
following:
Table 2 - Reference Documents and Web Sites
Document Conventions
When discussing memory sizes, MB and MBytes are used in this document to mean size in mega bytes. The use of
Mb or Mbits (small b) indicates size in mega bits.
Switch Firmware and Firmware Update Tools See 
http://www.mellanox.com/content/
pages.php?pg=management_tools&menu_section=34
Note that the Switch System described in this manual is
based on Mellanox Technologies’ InfiniScale IV switch
device.
FabricIT TM Enterprise Fabric Management
(EFM) Software User’s Manual Talk to your Mellanox representative for information
regarding licensing and implementation of the FabricIT
Enterprise Fabric Management Software System.
See
This symbol indicates a recommendation.
This symbol indicates importatnt information for the user and installer.
This symbol indicates a potential danger of damage to the hardware or software.

Rev 1.8
Mellanox Technologies
8
This symbol indicates a potential danger of injury to users and / or installers and poten-
tial damage to the hardware or software.

Rev 1.8
Mellanox Technologies 9
Internally Managed vs. Externally Managed
The following table shows which switches come with a management CPU and which do not.
Managed switches can be upgraded from simple chassis management to full fabric management
by purchasing an additional license. For more information on buying a license see the user’s
manual.
Unmanaged switches are plug and play out of the box. All switches come with the latest firm-
ware burned on the Flash. Update the firmware on unmanaged switches in-band only. When new
firmware is available you will receive an email with the link to the Mellanox firmware download
site. The download site has the Mellanox firmware tool package and full instructions for updating
firmware.
All managed switches have internal chassis management and can manage up to 108 nodes.
FabricIT can support IB fabrics of up to 648 nodes. Managed switches need an initial configura-
tion before they will start working. See Table 3 for details.
Should your unmanaged switch have CONSOLE, Ethernet, and USB connectors, they will not
work. Only the I2C connector will work on unmanaged switches.
Table 3 - Switch Management by OPN
Family Managed /
Unmanaged Management Connections
IS5025 Unman-
aged No Management
Plug and play 
All firmware updates should be done in-band using Mellanox Firm-
ware Management Tools. 12C port access using MTUSB-1 device is
required for firmware updates if in-band burning is not possible.
IS5030 Managed RS232 cable DB9 to RJ45 included in the box to connect to host PC
for initial configuration of the switch. After initial configuration the
switch can be managed through the Ethernet using a remote connec-
tion.
This switch is capable of Fabric management for an unlimited num-
ber of nodes, with the purchase of Mellanox UFM.
IS5035 Managed RS232 cable DB9 to RJ45 included in the box to connect to host PC
for initial configuration of the switch. After initial configuration the
switch can be managed through the Ethernet using a remote connec-
tion.
This switch is capable of Fabric management for an unlimited num-
ber of nodes, with the purchase of Mellanox UFM.

Rev 1.8
Mellanox Technologies
10
Table 4 - Switches and Rail Kits for Various Rack Depths
Switch S(Standard)/ B (Short) Rack Size Rail Kit #
Short 50cm to 80cm MIS00008
3
Short 38cm to 50cm MIS00007
9
Standard 50cm to 80cm MIS00008
5
Table 5 - Mellanox Part Numbering Legend
Place Field Decoder
IS System Type InfiniScale Switch
50 Model Family
FF Form factor 25 = 36 Ports Unmanaged
30 = 36 Ports and Chassis Management with
Fabric
Management up to 108 nodes
35 = 36 ports with chassis management and Fab-
ric
Management up to 2000 nodes
CInfiniBand Port Config. Q= QDR, D= DDR
- Separator
P# of Power Supplies 0=0, 1=1, 2=2....
MDepth of the Unit S = standard depth, B = short depth
YAir Flow direction R= Connector side to PSU side airflow
F= PSU side to Connector side airflow
R RoHS C=RoHS5, X=RoHS6