Voltaire Grid Director 4036 User manual

.
Voltaire
®
4036/2036 Grid Director
April 2009
Doc P/N: DOC-00619
Rev: A01
4036/2036 Software Version: 2.0

VOLTAIRE, INC. AND ITS AFFILIATES (“VOLTAIRE”) FURNISH THIS DOCUMENT “AS IS,” WITHOUT WARRANTY OF
ANY KIND. VOLTAIRE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION,
THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEENT
AND THOSE ARISING FROM A COURSE OF PERFORMANCE, A COURSE OF DEALING, OR TRADE USAGE.
VOLTAIRE SHALL NOT BE LIABLE FOR ANY ERROR, OMISSION, DEFECT, DEFICIENCY OR NONCONFORMITY IN
THIS DOCUMENT AND DISCLAIMS ALL LIABILITY, INCLUDING LIABILITY FOR INFRINGEMENT OF ANY
INTELLECTUAL PROPERTY RIGHTS RELATED TO THE INFORMATION CONTAINED IN THIS DOCUMENT.
No license, expressed or implied, to any intellectual property rights is granted under this document. This document, as well
as the software described in it, are furnished under a separate license and shall only be used or copied in accordance with
the terms of the applicable license. The information in this document is furnished for informational use only, is subject to
change without notice, and should not be construed as any commitment by Voltaire. Except as permitted by the applicable
license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any
means without the express written consent of Voltaire.
Names and logos identifying products of Voltaire in this document are registered trademarks or trademarks of Voltaire. All
other trademarks mentioned in this document are the property of their respective owners.
Copyright © 2009 Voltaire, Inc. All rights reserved.
Voltaire Document Part Number: DOC-00619
Business Headquarters
Voltaire Inc.
6 Fortune Drive, Suite 301
Billerica, MA
USA 01821
Tel: [phone]
Fax: [phone]
R&D Center
Voltaire Ltd.
13 Zarchin St
Raanana 43662
Israel
Tel: [phone]
Fax: [phone]

DOC-00619 A01
4036/2036 User Manual Page iii
Overview
Voltaire’s integrated family of switching hardware and network virtualization software delivers
a high-performance, intelligent solution for deploying clusters and grids. Leveraging
InfiniBand technology, Voltaire solutions offer improved application performance, resource
utilization and data center scalability for its customers’ high-performance computing needs.
Voltaire switching solutions are available in a multitude of port counts to facilitate the
deployment of powerful clusters ranging from a few to thousands of nodes.
The 4036 and the 2036 are part of the Grid Director family:
This Manual is divided as follows:
Part 1: Getting Started and Voltaire Device Manager
Part 2: CLI Reference Section
Part 3: Appendix and Glossary
NOTE:
Terms and CLI examples are interchangeable between the 2036 and 4036 unless specified
otherwise.
CAUTION:
Voltaire highly recommends against using root permissions for its switches. Use of root
permission for the switches is at the user’s own risk.

DOC-00619 A01
4036/2036 User Manual Page iii
About this Manual
This preface describes the audience, organization and command syntax conventions of this
4036/2036 User Manual. It also provides information on how to obtain related documentation
and technical assistance.
Information about hardware installation of the respective Voltaire switches can be found in the
4036/2036 Installation Manual [DOC-00467]
NOTE:
Refer to the latest Voltaire release notes for last minute updates and restrictions.
The Voltaire Technical Support Center (TSC) is at your service. You may access Warranty
Service through our Web Request Form by using the following link:
http://www.voltaire.com/support.html
Contact Us:
Please send your documentation-related comments and feedback or report mistakes
to [email protected].
We are committed to constant and never-ending improvement. Your input will greatly
help us in our endeavor.
Audience
This manual is primarily intended for system administrators who are familiar with the
fundamentals of router-based internetworking and network storage devices, but who might
not be familiar with the specifics of Voltaire products or the routing protocols supported by
Voltaire products
It is assumed that readers are familiar with InfiniBand technology and terminology.

Overview
DOC-00619 Rev. A01
4036/2036 User Manual Page iv
Related Documentation
4036/2036 Getting Started Guide - Voltaire QDR Solution (LIT-00037)
Voltaire 4036/2036 Installation Manual [DOC-00467]
Unified Fabric Manager (UFM) User Manual, Voltaire [DOC-00600]
Document Conventions
NOTE:
Text set off in this manner presents clarifying information, specific instructions, commentary,
sidelights, or interesting points of information.
IMPORTANT:
Text set off in this manner indicates important information regarding a specific feature.
CAUTION:
Text set off in this manner indicates that failure to follow directions could result in damage to
equipment or loss of information.

DOC-00619 Rev. A01
4036/2036 User Manual Page v
Contents
Overview ..................................................................................................................................................... iii
About this Manual...............................................................................................................................iii
Audience ............................................................................................................................................iii
Related Documentation..................................................................................................................... iv
Document Conventions..................................................................................................................... iv
PART 1: 4036/2036 Device Manager......................................................................................................... ix
Chapter 1. Introduction ....................................................................................................................... 1-1
1.1 About
InfiniBand....................................................................................................................1-2
1.2 Scaling-Out Data Centers with QDR ....................................................................................1-3
1.3 4036
System Overview.........................................................................................................1-4
1.4 2036
System Overview.........................................................................................................1-4
1.5 4036/2036
Functional Block Diagram...................................................................................1-5
1.6 Voltaire Switch Management ................................................................................................1-5
Chapter 2. CLI Overview...................................................................................................................... 2-1
2.1 Interfaces for CLI Connection ...............................................................................................2-2
2.2 Telnet/SSH Client Settings ...................................................................................................2-2
2.3 CLI
Usage Tips .....................................................................................................................2-3
2.4 Keyboard Usage in CLI Operation........................................................................................2-3
2.5 CLI
Command Prompt ..........................................................................................................2-5
2.6 CLI
Command Structure .......................................................................................................2-6
2.7 CLI Case Sensitivity..............................................................................................................2-7
2.8 Built-in Test ...........................................................................................................................2-7
Chapter 3. Initial System Setup (CLI)................................................................................................. 3-1
3.1 Prerequisite Tasks ................................................................................................................3-2
3.2 First-Time Configuration .......................................................................................................3-2
3.3 Collecting Required Information ...........................................................................................3-2
3.4 Connecting a Console ..........................................................................................................3-4
3.5 Starting a CLI Session ..........................................................................................................3-5
3.6 Initial System Configuration for Management.......................................................................3-6
3.7 Time Settings ........................................................................................................................3-7
3.8 Configuring Passwords.........................................................................................................3-8
3.9 Verifying Configuration .........................................................................................................3-9
Chapter 4. Built-in-Test ....................................................................................................................... 4-1
4.1 Overview ...............................................................................................................................4-2
4.2 Tests, Description and Explanations ....................................................................................4-2

Contents
DOC-00619 Rev. A01
4036/2036 User Manual Page vi
Chapter 5. Updating, Backing up, and Upgrading the Software ..................................................... 5-1
5.1 Prerequisites .........................................................................................................................5-2
5.2 Software Update ...................................................................................................................5-3
5.3 Firmware Upgrade ................................................................................................................5-6
Chapter 6. Handover Mechanism and Redundancy/High Availability............................................ 6-1
6.1 SM (Subnet Manager) Handover..........................................................................................6-2
6.2 Redundant
Power Supplies ..................................................................................................6-2
6.3 Fans ......................................................................................................................................6-3
Chapter 7. 4036/2036 Device Manager............................................................................................... 7-1
7.1 Device Manager Overview....................................................................................................7-2
7.2 4036/2036
- Obtaining Device Information ...........................................................................7-2
7.3 Obtaining Detailed Information via the Device M .................................................................7-3
7.4 4036/2036 Local SM Information..........................................................................................7-4
7.5 4036/2036
Temperature Information ....................................................................................7-6
7.6 4036/2036 Reset...................................................................................................................7-7
7.7 Viewing the Event and Error Log ..........................................................................................7-8
Chapter 8. Remote Configuration Settings .......................................................................................8-1
8.1 Overview ...............................................................................................................................8-2
8.2 Setting the Remote Configuration Parameters.....................................................................8-2
Chapter 9. SecuritySettings ............................................................................................................... 9-5
9.1 Overview ...............................................................................................................................9-6
9.2 SCP - Secure Export and Import ..........................................................................................9-6
9.3 Enabling/Disabling Non-secure Protocols ............................................................................9-7
Chapter 10. Unified Fabric Manager Support....................................................................................10-8
Chapter 11. FactoryDefaults ............................................................................................................ 11-10
Chapter 12. QSFP Port Management ...............................................................................................12-11
Chapter 13. Configuring the Subnet Manager ................................................................................ 13-13
Chapter 14. Configuring Routing Algorithms .................................................................................14-14
14.1 Supported Algorithms .......................................................................................................14-15
14.2 Min Hop (Balanced Routing) Scheme ..............................................................................14-15
14.3 Up/Down Routing Scheme ...............................................................................................14-15
14.4 Setting the Routing Algorithm ...........................................................................................14-16
Chapter 15. Utilities............................................................................................................................ 15-17
PART 2: CLI Reference Section.......................................................................................................... 15-18
Chapter 16. Getting Started with the CLI and Top Level Structure ................................................ 16-1
16.1 CLI Overview ......................................................................................................................16-2
16.2 CLI Initial Setup...................................................................................................................16-2
16.3 CLI
Structure.......................................................................................................................16-2
Chapter 17. Guest Mode...................................................................................................................... 17-1
17.1 Accessing the Guest Mode.................................................................................................17-2
17.2 Guest Mode Command Reference .....................................................................................17-2

Contents
DOC-00619 Rev. A01
4036/2036 User Manual Page vii
Chapter 18. Admin Mode..................................................................................................................... 18-1
18.1 Overview .............................................................................................................................18-2
18.2 Accessing the Admin (Privileged) Mode.............................................................................18-2
18.3 Admin (Privileged) Mode Command Reference .................................................................18-3
18.4 Health Monitor...................................................................................................................18-12
18.5 Logs Mode ........................................................................................................................18-13
18.6 Utilities Mode ....................................................................................................................18-18
Chapter 19. Configuration (config) Mode ........................................................................................19-35
19.1 Overview ...........................................................................................................................19-36
19.2 Accessing the config Mode...............................................................................................19-36
19.3 config Mode Command Reference ...................................................................................19-36
19.4 Interface Mode ..................................................................................................................19-39
19.5 Names Mode.....................................................................................................................19-42
19.6 NTP Mode.........................................................................................................................19-45
19.7 Ports Mode........................................................................................................................19-46
19.8 Remote Mode ...................................................................................................................19-48
19.9 Security Mode ...................................................................................................................19-55
19.10 SM (Subnet Manager) Mode ............................................................................................19-57
PART 3: Appendices............................................................................................................................ 19-65
Appendix A Acronyms.......................................................................................................................... A-1
Appendix B InfiniBandTM Standard Glossary ................................................................................... B-4

DOC-00619 Rev. A01
4036/2036 User Manual Page viii
Figures
Figure 1-1. Voltaire 4036/2036 Grid Director Functional Diagram............................................................ 1-5
Figure 2-1. Telnet/SSH Client Settings – Example (using Putty).............................................................. 2-2
Tables
Table 2-1. CLI Key Functions ................................................................................................................... 2-3
Table 2-2. CLI Command Prompts ........................................................................................................... 2-5
Table 2-3. Basic Commands..................................................................................................................... 2-6
Table 3-1. Switch Installation Checklist .................................................................................................... 3-3
Table 3-2. Terminal Emulation Configuration ........................................................................................... 3-4
Table 7-1. Information (Front Show) Parameters ..................................................................................... 7-3
Table 7-2. Local SM Information Parameters ...........................................................................................7-4
Table 7-3. Temperature Show Parameters...............................................................................................7-7
Table 17-1. Common Options............................................................................................................... 18-19

P
PA AR RT T 1 1: :
4
40 03 36 6/ /2 20 03 36 6 D De ev vi ic ce e M Ma an na ag ge er r