Mellanox Technologies Mellanox SX1018 User manual

www.mellanox.com
Mellanox Technologies Confidential
Mellanox MLNX-OS® User Manual
for SX1018HP Ethernet Managed Blade Switch
Rev 1.6.9
Software Version 3.3.4100

Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: [phone]
Fax: [phone]
Mellanox Technologies, Ltd.
Beit Mellanox
PO Box 586 Yokneam 20692
Israel
www.mellanox.com
Tel: [phone]
Fax: [phone]
© Copyright 2013. Mellanox Technologies. All Rights Reserved.
Mellanox®, Mellanox logo, BridgeX®, ConnectX®, CORE-Direct®, InfiniBridge®, InfiniHost®, InfiniScale®,
MLNX-OS®, PhyX®, SwitchX®, UFM®, Virtual Protocol Interconnect® and Voltaire® are registered trademarks of
Mellanox Technologies, Ltd.
Connect-IB™, ExtendX™, FabricIT™, Mellanox Open Ethernet™, Mellanox Virtual Modular Switch™, MetroX™,
MetroDX™, ScalableHPC™, Unbreakable-Link™ are trademarks of Mellanox Technologies, Ltd.
All other trademarks are property of their respective owners.
NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED
DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY
KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE
THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT
HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE
PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND
DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST
QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES
FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND
(INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED
DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Document Number: 3959HP-
Rev 1.6.9
Mellanox Technologies
2
Mellanox Technologies Confidential

Rev 1.6.9
Mellanox Technologies 3
Mellanox Technologies Confidential
Table of Contents
Document Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.1 MLNX-OS Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 2 Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1 Configuring the Switch for the First Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.1 Re-Running the Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Starting the Command Line (CLI). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Starting the Web Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.1 Installing MLNX-OS® License (CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.2 Installing MLNX-OS License (Web) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.3 Retrieving a Lost License Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Chapter 3 User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1 Command Line Interface (CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1.1 CLI Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1.2 Syntax Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.3 Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.4 Prompt and Response Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.5 Using the “no” Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.1.6 Parameter Key. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Web Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.1 Setup Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2.2 System Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2.3 Security Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.4 Ports Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.5 Status Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.6 IB SM Mgmt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.7 Fabric Inspector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.2.8 ETH Mgmt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Chapter 4 System Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 Management Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.1 Configuring Management Interfaces with Static IP Addresses . . . . . . . . . . . . . . 36
4.1.2 Configuring IPv6 Address on the Management Interface. . . . . . . . . . . . . . . . . . . 36
4.1.3 Dynamic Host Configuration Protocol (DHCP) . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.4 Default Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1.5 In-Band Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 Software Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Rev 1.6.9
Mellanox Technologies 4
Mellanox Technologies Confidential
4.2.1 Upgrading MLNX-OS® Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.2 Deleting Unused Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.3 Downgrading MLNX-OS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.4 Upgrading System Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3 File Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.1 Saving a Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.2 Loading a Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3.3 Restoring Factory Default Configuration on a Switch System 
(Single Management Module) 47
4.4 Managing Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.4.1 BIN Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.4.2 Text Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.5.1 Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.5.2 Remote Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.6 Event Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.6.1 Supported Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.6.2 SNMP Trap Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.6.3 Terminal Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.6.4 Email Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.7 USB Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.7.1 Accessing a USB Device for Read/Write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.8 Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.8.1 Retrieving Return Codes when Executing Remote Commands . . . . . . . . . . . . . . 53
4.9 mDNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.10 User Management and Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.10.1 Authentication, Authorization and Accounting (AAA) . . . . . . . . . . . . . . . . . . . . 54
4.10.2 Secure Shell (SSH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.10.3 User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.11 Network Management Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.11.1 SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.11.2 MLNX-OS XML API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Chapter 5 Ethernet Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.1 Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.1.1 Break-Out Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.2 Link Aggregation Group (LAG) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.2.1 Configuring Static Link Aggregation Group (LAG) . . . . . . . . . . . . . . . . . . . . . . 66
5.2.2 Configuring Link Aggregation Control Protocol (LACP) . . . . . . . . . . . . . . . . . . 67
5.3 VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3.1 Configuring Access Mode and Assigning Port VLAN ID (PVID). . . . . . . . . . . . 68
5.3.2 Configuring Hybrid Mode and Assigning Port VLAN ID (PVID). . . . . . . . . . . . 68
5.3.3 Configuring Trunk Mode VLAN Membership. . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.3.4 Configuring Hybrid Mode VLAN Membership . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.4 MAC Address Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.4.1 Configuring Unicast Static MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Rev 1.6.9
Mellanox Technologies 5
Mellanox Technologies Confidential
5.5 Spanning Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.5.1 Port Priority and Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.5.2 Port Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.5.3 BPDU Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.5.4 Loop Guard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.5.5 Root Guard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.6 IGMP Snooping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.6.1 Configuring IGMP Snooping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.6.2 Defining a Multicast Router Port on a VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.7 Link Layer Discovery Protocol (LLDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.7.1 Configuring LLDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.8 Quality of Service (QoS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.8.1 Priority Flow Control and Link Level Flow Control . . . . . . . . . . . . . . . . . . . . . . 75
5.8.2 Enhanced Transmission Selection (ETS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.9 Access Control List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.9.1 Configuring Access Control List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.9.2 ACL Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.10 Port Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.10.1 Mirroring Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.10.2 Configuring Mirroring Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.10.3 Verifying Mirroring Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.11 sFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.11.1 Flow Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.11.2 Statistical Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.11.3 sFlow Datagrams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.11.4 Sampled Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.11.5 Configuring sFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.11.6 Verifying sFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Table 1 - Document Revision History - Ethernet
Document
Revision Date Changes
Rev 1.6.9 September
2013
Updated Section 4.6, “Event Notifications,” on page 49
Updated Section 4.4.2, “Text Configuration Files,” on page 48
Added Section 4.9, “mDNS,” on page 53
Rev 1.6.8 August 2013 Added Section 4.4.2, “Text Configuration Files,” on page 48
Updated Section 2.4, “Licenses,” on page 19.
Rev 1.6.7 June 2013 Updated Section 6.2.3, “Configuring OSPF,” on page 101.
Updated Section 4.5, “Logging,” on page 49.
Updated Table 22, “Supported Event Notifications and
MIB Mapping,” on page 50.
Rev. 1.6.6 Apr. 2013 Added Section 4.1.5, “In-Band Management,” on page 37.
Added Section 4.6.1, “Supported Events,” on page 50.
Updated Section 5.5, “Spanning Tree,” on page 70.
Rev. 1.6.4 Mar 2013 Added Section 5.10, “Port Mirroring,” on page 80.
Added Section 5.11, “sFlow,” on page 86.
Rev. 1.5.4 Sep. 2012 Updated Section 4.2.3, “Downgrading MLNX-OS Software,”
on page 42.
Rev. 1.5.3 Aug. 2012 Updated Section 5.1.1, “Break-Out Cables,” on page 63.
Rev 1.5.2 June 2012 Updated Section 4.11.1, “SNMP,” on page 57.
Rev 1.5.1 May 2012 Added Section 5.7, “Link Layer Discovery Protocol (LLDP),”
on page 74.
Rev 1.5 May 2012 Initial document.
Rev 1.6.9
Mellanox Technologies 6
Mellanox Technologies Confidential
Document Revision History

Rev 1.6.9
Mellanox Technologies 7
Mellanox Technologies Confidential
About this Manual
This manual provides general information concerning the scope and organization of this User’s
Manual.
Intended Audience
This manual is intended for network administrators who are responsible for configuring and
managing Mellanox Technologies’ SwitchX based Switch Platforms.
Related Documentation
The following table lists the documents referenced in this User’s Manual.
Table 2 - Reference Documents
Document Name Description
InfiniBand Architecture Specification, Vol. 1,
Release 1.2.1
The InfiniBand Architecture Specification that is pro-
vided by IBTA.
Switch Installation Guide Each Mellanox Technologies' switch platform is
shipped with an Installation Guide document to
bring-up and initialize the switch platform.
System Hardware User Manual This document contains hardware descriptions, LED
assignments and hardware specifications among other
things.
Switch Product Release Notes Please look up the relevant SwitchX®-based switch
system/series release note file
MLNX-OS® Command Reference Guide Command Reference Guide for MLNX-OS listing all
of the commands available through MLNX-OS with
explanations and examples.
All of these documents can be found on the Mellanox website. They are available either through
the product pages or through the support page with a login and password.

Rev 1.6.9
Mellanox Technologies 8
Mellanox Technologies Confidential
Glossary
Table 3 - Glossary
AAA Authentication, Authorization, and Accounting.
Authentication - verifies user credentials (username and password).
Authorization - grants or refuses privileges to a user/client for accessing spe-
cific services.
Accounting - tracks network resources consumption by users.
ARP Address Resolution Protocol. A protocol that translates IP addresses into
MAC addresses for communication over a local area network (LAN).
CLI Command Line Interface. A user interface in which you type commands at the
prompt
DCB Data Center Bridging
DCBX DCBX protocol is an extension of the Link Layer Discovery Protocol
(LLDP). DCBX end points exchange request and acknowledgment messages.
For flexibility, parameters are coded in a type-length-value (TLV) format.
DHCP The Dynamic Host Configuration Protocol (DHCP) is an automatic configura-
tion protocol used on IP networks.
DNS Domain Name System. A hierarchical naming system for devices in a com-
puter network
ETS ETS provides a common management framework for assignment of band-
width to traffic classes.
FTP/TFTP/sFTP File Transfer Protocol (FTP) is a standard network protocol used to transfer
files from one host to another over a TCP-based network, such as the Internet.
Gateway A network node that interfaces with another network using a different network
protocol
HA (High Availability) A system design protocol that provides redundancy of system components,
thus enables overcoming single or multiple failures in minimal downtime
Host A computer platform executing an Operating System which may control one
or more network adapters
LACP Link Aggregation Control Protocol (LACP) provides a method to control the
bundling of several physical ports together to form a single logical channel.
LACP allows a network device to negotiate an automatic bundling of links by
sending LACP packets to the peer (directly connected device that also imple-
ments LACP).
LDAP The Lightweight Directory Access Protocol is an application protocol for
reading and editing directories over an IP network.
MAC A Media Access Control address (MAC address) is a unique identifier
assigned to network interfaces for communications on the physical network
segment. MAC addresses are used for numerous network technologies and
most IEEE 802 network technologies including Ethernet.

Rev 1.6.9
Mellanox Technologies 9
Mellanox Technologies Confidential
MTU (Maximum Transfer Unit) The maximum size of a packet payload (not including headers) that can be
sent /received from a port
Network Adapter A hardware device that allows for communication between computers in a
network
PFC/FC Priority Based Flow Control applies pause functionality to traffic classes OR
classes of service on the Ethernet link.
RADIUS Remote Authentication Dial In User Service. A networking protocol that
enables AAA centralized management for computers to connect and use a net-
work service.
RDMA (Remote Direct Memory
Access)
Accessing memory in a remote side without involvement of the remote CPU
RSTP Rapid Spanning Tree Protocol. A spanning-tree protocol used to prevent loops
in bridge configurations. RSTP is not aware of VLANs and blocks ports at the
physical level.
SA (Subnet Administrator) The interface for querying and manipulating subnet management data
SCP Secure Copy or SCP is a means of securely transferring computer files
between a local and a remote host or between two remote hosts. It is based on
the Secure Shell (SSH) protocol.
SNMP Simple Network Management Protocol. A network protocol for the manage-
ment of a network and the monitoring of network devices and their functions
NTP Network Time Protocol. A protocol for synchronizing computer clocks in a
network
SSH Secure Shell. A protocol (program) for securely logging in to and running pro-
grams on remote machines across a network. The program authenticates
access to the remote machine and encrypts the transferred information through
the connection.
syslog A standard for forwarding log messages in an IP network
TACACS+ Terminal Access Controller Access-Control System Plus. A networking proto-
col that enables access to a network of devices via one or more centralized
servers. TACACS+ provides separate AAA services.
XML Gateway Extensible Markup Language Gateway. Provides an XML request-response
protocol for setting and retrieving HW management information.
Table 3 - Glossary

Rev 1.6.9
Mellanox Technologies 10
Mellanox Technologies Confidential
1 Introduction
Mellanox® Operating System (MLNX-OS®) enables the management and configuration of Mel-
lanox Technologies’ SwitchX® silicon based switch platforms. MLNX-OS supports the Virtual
Protocol Interconnect (VPI) technology which enables it to be used for both Ethernet and Infini-
Band technology providing the user with greater flexibility.
MLNX-OS provides a full suite of management options, including support for Mellanox’s Uni-
fied Fabric Manager® (UFM), SNMP V1,2,3, and web user interface. In addition, it incorporates
a familiar industry-standard CLI, which enables administrators to easily configure and manage
the system.
1.1 MLNX-OS Features
Table 4 - General System Features
Feature Description
Software Management • Dual software image
• Software and firmware updates
File management •FTP
•TFTP
•SCP
Logging • Event history log
• SysLog support
Management Interface • DHCP/Zeroconf
•IPv6
Chassis Management • Monitoring environmental controls
Network Management Inter-
faces
• SNMP v1,v2c,v3
• REST interfaces (XML Gateway)
Security • SSH
• Telnet
• RADIUS
•TACACS+
Date and Time •NTP
Cables & Transceivers • Transceiver info
Virtual Port Interconnect®
(VPI)
• Ethernet
• InfiniBand
Table 5 - Ethernet Features
Feature Description
General • Jumbo Frames (9K)
• ACL - 24K rules (permit/deny)
• Breakout cables

Rev 1.6.9
Mellanox Technologies 11
Mellanox Technologies Confidential
Ethernet support • 48K Unicast MAC addresses
• VLAN (802.1Q) - 4K
• LAG/LACP (802.3ad), 16 links per LAG (36 LAGs)
• Rapid Spanning Tree (802.1w)
• Flow control (802.3x)
• IGMP snooping v1,2
•LLDP
• ETS (802.1Qaz)
• PFC (802.1Qbb)
IP routing • VLAN interface
•ECMP
•OSPF
Table 5 - Ethernet Features
Feature Description
Figure 1: Managing an Ethernet Fabric Using MLNX-OS

Rev 1.6.9
Mellanox Technologies 12
Mellanox Technologies Confidential
2 Getting Started
The procedures described in this chapter assume that you have already installed and powered on
your switch according to the instructions in the Hardware Installation Guide, which was shipped
with the product.
2.1 Configuring the Switch for the First Time
Step 1. Connect to the HP Chassis Manager and run “connect interconnect <slot number 1-7>”.
Step 2. Configure a serial terminal from EM with the settings described below.
Step 3. Login as admin and use admin as password.
Step 4. Go through the Mellanox configuration wizard.
The following table shows an example of a wizard session.
No remote IP connection is available at this stage.
Table 6 - Serial Terminal Program Configuration
Parameter Setting
Baud Rate 9600
Data bits 8
Stop bits 1
Parity None
Flow Control None
Table 7 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 1 of 3)
Wizard Session Display (Example) Comments
Mellanox configuration wizard
Do you want to use the wizard for initial con-
figuration? yes
You must perform this configuration the first
time you operate the switch or after resetting
the switch to the factory defaults. Type “y” and
then press <Enter>.
Step1: Hostname? [switch-1] If you wish to accept the default hostname,
then press <Enter>. Otherwise, type a different
hostname and press <Enter>.

Rev 1.6.9
Mellanox Technologies 13
Mellanox Technologies Confidential
Step 2: Use DHCP on mgmt0 interface? [yes] Perform this step to obtain an IP address for the
switch. (mgmt0 is the management port of the
switch.)
If you wish the DHCP server to assign the IP
address, type “yes” and press <Enter>.
If you type “no” (no DHCP), then you will be
asked whether you wish to use the “zeroconf”
configuration or not. If you enter “yes” (yes
Zeroconf), the session will continue as shown
in Table 8.
If you enter “no” (no Zeroconf), then you need
to enter a static IP, and the session will con-
tinue as shown in Table 9.
Step 3: Enable IPv6 [yes] Perform this step to enable IPv6 on manage-
ment ports.
If you wish to enable IPv6, type “yes” and
press <Enter>.
If you enter “no” (no IPv6), then you will auto-
matically be referred to Step 5.
Step 4: Enable IPv6 autoconfig (SLAAC) on
mgmt0 interface
Perform this step to enable StateLess address
autoconfig on external management port.
If you wish to enable it, type “yes” and press
<Enter>.
If you wish to disable it, enter “no”.
Step 5: Admin password (Press <Enter> to
leave unchanged)? <new_password>
Step 4: Confirm admin password?
<new_password>
To avoid illegal access to the machine, please
type a password and then press <Enter>. Then
confirm the password by re-entering it.
Note that password characters are not printed.
Table 7 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 2 of 3)
Wizard Session Display (Example) Comments

Rev 1.6.9
Mellanox Technologies 14
Mellanox Technologies Confidential
You have entered the following information:
1. Hostname: <switch name>
2. Use DHCP on mgmt0 interface: yes
3. Enable IPv6: yes
4. Enable IPv6 autoconfig (SLAAC) on mgmt0
interface: yes
5. Admin password (Enter to leave unchanged):
(CHANGED)
To change an answer, enter the step number to
return to.
Otherwise hit <enter> to save changes and exit.
Choice: <Enter>
Configuration changes saved.
To return to the wizard from the CLI, enter the
“configuration jump-start” command from
configuration mode. Launching CLI...
<switch name> [standalone: master] >
The wizard displays a summary of your
choices and then asks you to confirm the
choices or to re-edit them.
Either press <Enter> to save changes and exit,
or enter the configuration step number that you
wish to return to.
Note:
To run the command “configuration jump-
start” you must be in Config mode.
Table 7 - Configuration Wizard Session - IP Configuration by DHCP (Sheet 3 of 3)
Wizard Session Display (Example) Comments

Rev 1.6.9
Mellanox Technologies 15
Mellanox Technologies Confidential
Table 8 - Configuration Wizard Session - IP Zeroconf Configuration
Wizard Session Display - IP Zeroconf Configuration (Example)
Mellanox configuration wizard
Do you want to use the wizard for initial configuration? y
Step 1: Hostname? [switch-112126]
Step 2: Use DHCP on mgmt0 interface? [no]
Step 3: Use zeroconf on mgmt0 interface? [no] yes
Step 4: Default gateway? [192.168.10.1]
Step 5: Primary DNS server?
Step 6: Domain name?
Step 7: Enable IPv6? [yes] yes
Step 8: Enable IPv6 autoconfig (SLAAC) on mgmt0 interface? [no] no
Step 9: Admin password (Enter to leave unchanged)?
You have entered the following information:
1. Hostname: switch-112126
2. Use DHCP on mgmt0 interface: no
3. Use zeroconf on mgmt0 interface: yes
4. Default gateway: 192.168.10.1
5. Primary DNS server:
6. Domain name:
7. Enable IPv6: yes
8. Enable IPv6 autoconfig (SLAAC) on mgmt0 interface: yes
9. Admin password (Enter to leave unchanged): (unchanged)
To change an answer, enter the step number to return to.
Otherwise hit <enter> to save changes and exit.
Choice:
Configuration changes saved.
To return to the wizard from the CLI, enter the “configuration jump-start”
command from configure mode. Launching CLI...
<switch name> [standalone: master] >

Rev 1.6.9
Mellanox Technologies 16
Mellanox Technologies Confidential
Table 9 - Configuration Wizard Session - Static IP Configuration
Wizard Session Display - Static IP Configuration (Example)
Mellanox configuration wizard
Do you want to use the wizard for initial configuration? y
Step 1: Hostname? [switch-112126]
Step 2: Use DHCP on mgmt0 interface? [yes] n
Step 3: Use zeroconf on mgmt0 interface? [no]
Step 4: Primary IP address? 192.168.10.4
Mask length may not be zero if address is not zero (interface mgmt0)
Step 5: Netmask? [0.0.0.0] 255.255.255.0
Step 6: Default gateway? 192.168.10.1
Step 7: Primary DNS server?
Step 8: Domain name?
Step 9: Enable IPv6? [yes] yes
Step 10: Enable IPv6 autoconfig (SLAAC) on mgmt0 interface? [no] no
Step 11: Admin password (Enter to leave unchanged)?
You have entered the following information:
1. Hostname: switch-112126
2. Use DHCP on mgmt0 interface: no
3. Use zeroconf on mgmt0 interface: no
4. Primary IP address: 192.168.10.4
5. Netmask: 255.255.255.0
6. Default gateway: 192.168.10.1
7. Primary DNS server:
8. Domain name:
9. Enable IPv6: yes
10. Enable IPv6 autoconfig (SLAAC) on mgmt0 interface: no
11. Admin password (Enter to leave unchanged): (unchanged)
To change an answer, enter the step number to return to.
Otherwise hit <enter> to save changes and exit.
Choice:
Configuration changes saved.
To return to the wizard from the CLI, enter the “configuration jump-start”
command from configure mode. Launching CLI...
<switch name>[standalone: master] >

Rev 1.6.9
Mellanox Technologies 17
Mellanox Technologies Confidential
Step 5. Check the mgmt0 interface configuration before attempting a remote (for example, SSH) con-
nection to the switch. Specifically, verify the existence of an IP address.
2.1.1 Re-Running the Wizard
To rerun the wizard:
Step 1. Enter the config mode.
Step 2. Rerun the wizard.
2.2 Starting the Command Line (CLI)
Step 1. Set up an Ethernet connection between the switch and a local network machine using a stan-
dard RJ-45 connector.
Step 2. Start a remote secured shell (SSH) to the switch using the command “ssh -l <username>
<switch ip address>.”
switch # show interfaces mgmt0
Interface mgmt0 state
Admin up: yes
Link up: yes
IP address: 169.254.15.134
Netmask: 255.255.0.0
IPv6 enabled: yes
Autoconf enabled: yes
Autoconf route: yes
Autoconf privacy: no
IPv6 addresses: 1
IPv6 address: fe80::202:c9ff:fe11:a1b2/64
Speed: 1000Mb/s (auto)
Duplex: full (auto)
Interface type: ethernet
Interface source: physical
MTU: 1500
HW address: 00:02:C9:11:A1:B2
Comment:
RX bytes: 11700449 TX bytes: 15139846
RX packets: 55753 TX packets: 28452
RX mcast packets: 0 TX discards: 0
RX discards: 0 TX errors: 0
RX errors: 0 TX overruns: 0
RX overruns: 0 TX carrier: 0
RX frame: 0 TX collisions: 0
TX queue len: 1000
switch > enable
switch # config terminal
switch (config) # configuration jump-start
rem_mach1 > ssh -l <username> <ip address>
Mellanox MLNX-OS Switch Management
Last login: Thu Apr 28 11:24:13 2011 from 192.168.10.1

Rev 1.6.9
Mellanox Technologies 18
Mellanox Technologies Confidential
Step 3. Login to the switch (default username is admin, password admin)
Step 4. Once you get the prompt, you are ready to use the system. Refer to MLNX-OS Command Refer-
ence Guide for additional information on the CLI commands.
2.3 Starting the Web Interface
To start a WebUI connection to the switch platform:
Step 1. Set up an Ethernet connection between the switch and a local network machine using a stan-
dard RJ-45 connector.
Step 2. Open a web browser – Internet Explorer 7.0 Chrome or Mozilla Firefox 3.0.
Note: Make sure the screen resolution is set to 1024*768 or higher.
Step 3. Type in the IP address of the switch or its DNS name in the format: http://
<switch_IP_address>.
Step 4. Login to the switch (default user name is admin, password admin).
The following figure shows an example of the login window for remote management of the
switch.
Figure 2: MLNX-OS Login Window
After you log in to MLNX-OS, a (default) status summary window will be displayed containing
the following information:
Mellanox Switch
switch >

Rev 1.6.9
Mellanox Technologies 19
Mellanox Technologies Confidential
Figure 3: Display After Login
2.4 Licenses
MLNX-OS software package can be extended with premium features. Installing a license allows
you to access the specified premium features.
The following licenses are offered with MLNX-OS software:
This section is relevant only to switch systems with an internal management capability.
Table 10 - MLNX-OS Licenses
OPN Valid on product Description
UPGR-6012-GW SX6012 Ethernet L2/L3, Gateway
UPGR-6012F-56E SX6012 56GbE link speed
UPGR-1012-GW SX1012 InfiniBand, Ethernet L3, Gateway
UPGR-1012-56E SX1012 56GbE link speed
UPGR-6018-GW SX6018 Ethernet L2/L3, Gateway
UPGR-6018F-56E SX6018 56GbE link speed
UPGR-6036-GW SX6036 Ethernet L2/L3, Gateway

Rev 1.6.9
Mellanox Technologies 20
Mellanox Technologies Confidential
2.4.1 Installing MLNX-OS® License (CLI)
To install an MLNX-OS license via CLI:
Step 1. Login as admin and change to Config mode.
Step 2. Install the license using the key. Run:
Step 3. Display the installed license(s) using the following command.
Make sure that the “Valid” and “Active” fields both indicate “yes”.
Step 4. Save the configuration to complete the license installation. Run:
2.4.2 Installing MLNX-OS License (Web)
To install an MLNX-OS license via CLI:
Step 1. Log in as admin.
Step 2. Click the Setup tab and then Licensing in the left side navigation pane.
UPGR-6036F-56E SX6036 56GbE link speed
UPGR-1036-GW SX1036 InfiniBand, Ethernet L3, Gateway
UPGR-1036F-56E SX1036 56GbE link speed
UPGR-1024-GW SX1024 InfiniBand, Ethernet L3, Gateway
UPGR-1024-56E SX1024 56GbE link speed
LIC-fabric-inspector SX6036F/T/
SX65XX
InfiniBand fabric inspector monitoring and health.
switch > enable
switch # config terminal
switch (config) # license install <license key>
switch (config) # show licenses
License 1: <license key>
Feature: EFM_SX
Valid: yes
Active: yes
switch (config) #
switch (config) # configuration write
If you do not save the installation session, you will lose the license at the next system
start up.
Table 10 - MLNX-OS Licenses
OPN Valid on product Description