Mellanox Technologies Grid Director 4700 User manual

www.mellanox.com
Mellanox Grid Director 4700
Installation Manual

2
Document Number: DOC-00726 A06
NOTE:
MELLANOX TECHNOLOGIES, INC. AND ITS AFFILIATES ("MELLANOX") FURNISH THIS DOCUMENT "AS IS," WITHOUT
WARRANTY OF ANY KIND. MELLANOX DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, WITHOUT
LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE,
NON-INFRINGEMENT AND THOSE ARISING FROM A COURSE OF PERFORMANCE, A COURSE OF DEALING, OR TRADE
USAGE. MELLANOX SHALL NOT BE LIABLE FOR ANY ERROR, OMISSION, DEFECT, DEFICIENCY OR NONCONFORMITY IN
THIS DOCUMENT AND DISCLAIMS ALL LIABILITY, INCLUDING LIABILITY FOR INFRINGEMENT OF ANY INTELLECTUAL
PROPERTY RIGHTS RELATED TO THE INFORMATION CONTAINED IN THIS DOCUMENT.
No license, expressed or implied, to any intellectual property rights is granted under this document. This document, as well as the software
described in it, are furnished under a separate license and shall only be used or copied in accordance with the terms of the applicable license. The
information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as any
commitment by Mellanox. Except as permitted by the applicable license, no part of this document may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means without the express written consent of Mellanox.
Names and logos identifying products of Mellanox in this document are registered trademarks or trademarks of Mellanox. Voltaire is a
registered trademark of Mellanox Technologies, Ltd. All other trademarks mentioned in this document are the property of their respective
owners.
Copyright © 2011 Mellanox Technologies, Inc. All rights reserved.
Mellanox Technologies, Inc.
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: [phone]
Fax: [phone]
Mellanox Technologies Ltd
PO Box 586 Hermon Building
Yokneam 20692
Israel
Tel: [phone]
Fax: [phone]
© Copyright 2011. Mellanox Technologies. All rights reserved.
Mellanox®, BridgeX®, ConnectX®, CORE-Direct®, InfiniBlast®, InfiniBridge®, InfiniHost®, InfiniRISC®, InfiniScale®, InfiniPCI®,
PhyX®, Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies, Ltd.
FabricIT and SwitchX are trademarks of Mellanox Technologies, Ltd.
All other trademarks are property of their respective owners.

Contents
3
Contents
Preface.................................................................................................................................................... 8
1Introduction ................................................................................................................................... 10
1.1 Mellanox Grid Director™ 4700 Overview ............................................................................. 10
1.2 Main Features....................................................................................................................... 13
1.3 Product Architecture............................................................................................................. 14
1.3.1Main Building Blocks............................................................................................... 14
1.3.2 Reset Buttons ......................................................................................................... 16
1.4 Management......................................................................................................................... 17
2System Configurations................................................................................................................. 18
2.1 Configuration Summary........................................................................................................ 18
2.1.1 Module Configuration per Chassis ......................................................................... 18
2.1.2 Specifications Summary......................................................................................... 19
2.2 4700 Configuration and Part Numbers................................................................................. 19
2.3 Switch Internal Topology...................................................................................................... 20
2.3.1 HyperScale Fabric Boards...................................................................................... 20
2.3.2 Switching Topologies.............................................................................................. 21
2.3.3 HyperScale™ Configurations ................................................................................. 23
34700 Specifications....................................................................................................................... 24
3.1 4700 Architecture.................................................................................................................. 24
3.2 4700 Fabric Board (sFB-4700) Specifications...................................................................... 24
3.3 4700 Line Board (sLB-4018) Specifications......................................................................... 24
3.4 4700 Management Board (sMB-CM) Specifications ............................................................ 25
3.5 Chassis Slot Numbering....................................................................................................... 25
3.6 Torque Settings .................................................................................................................... 26
3.7 Module Configuration Rules ................................................................................................. 27
3.8 Clearance Requirements...................................................................................................... 27
3.9 Power Data........................................................................................................................... 27
3.9.1 Type of Power Cord on the Power Supply ............................................................. 27
3.9.2 Power Requirements .............................................................................................. 27
3.9.3 Power Characteristics............................................................................................. 28
3.9.4 Power Data per Module.......................................................................................... 28
3.10 Calculating Heat Output in BTUs.......................................................................................... 29
3.11 Maximum Heat Dissipation................................................................................................... 30
3.12 Airflow................................................................................................................................... 30
3.13 Physical Characteristics ....................................................................................................... 30

Contents
4
3.14 Weights and Dimensions...................................................................................................... 30
3.15 LED Indicators...................................................................................................................... 31
3.15.1 sLB-4018 Line Board LEDs.................................................................................... 32
3.15.2 Fabric Board (sFB-4700) LEDs .............................................................................. 32
3.15.3 HyperScale Fabric Board (sFB-4700-2X) LEDs..................................................... 33
3.15.4 Management Board (sMB-CM) LEDs..................................................................... 33
3.15.5 sFU-40H Horizontal Fan Unit LEDs........................................................................ 34
3.15.6 sPSU-S Power Supply LEDs.................................................................................. 35
3.16 Environmental Specifications ............................................................................................... 35
3.17 Acoustic Data........................................................................................................................ 35
3.18 Reliability (MTBF)................................................................................................................. 35
3.19 Certifications......................................................................................................................... 36
3.20 Declarations.......................................................................................................................... 37
3.20.1 Declaration of Conformity....................................................................................... 37
3.20.2 Hazardous Substances (RoHS 6) Compliance Declaration................................... 37
3.21 Label(s)................................................................................................................................. 38
3.22 Gost-R Certification .............................................................................................................. 38
3.23 KCC-Korea Certification ....................................................................................................... 38
3.24 Oscillator Speeds.................................................................................................................. 38
3.25 Shock and Vibration Test Reports........................................................................................ 39
3.25.1 4700 Test for Transportation without a Rack.......................................................... 39
3.26 Export Information ................................................................................................................ 40
3.27 Shipping Restrictions............................................................................................................ 40
3.28 Center of Gravity Data.......................................................................................................... 41
3.29 Statement of Volatility........................................................................................................... 41
4Packing and Unpacking ............................................................................................................... 42
4.1 Package Contents ................................................................................................................ 42
4.2 Unpacking the Grid Director 4700........................................................................................ 44
4.3 Packing the Grid Director 4700 Switch................................................................................. 45
5Preparing for Installation ............................................................................................................. 47
5.1 Tools Required for Installation.............................................................................................. 47
5.2 Site Planning......................................................................................................................... 47
5.3 Rack Requirements.............................................................................................................. 48
5.4 Clearance Requirements...................................................................................................... 48
5.5 Site Environment Specifications........................................................................................... 48
5.6 Chassis Grounding............................................................................................................... 48
5.7 Power Requirements............................................................................................................ 52
6Rack-Mounting the 4700 Chassis................................................................................................ 54

Contents
5
6.1 Mounting Process Summary ................................................................................................ 54
6.2 Assembling and Mounting the Rails..................................................................................... 54
6.2.1 Rail Kit and Mounting Components........................................................................ 55
6.2.2 Rail Kit Assembly Overview.................................................................................... 55
6.2.3 Rail Kit Assembly Procedure.................................................................................. 57
6.3 Mounting the 4700 in a Rack (Standard Installation) ........................................................... 60
6.4 Mounting the HyperScale Chassis in a Rack ....................................................................... 62
6.5 4700 Power-Up..................................................................................................................... 64
7Installing System Boards............................................................................................................. 65
7.1Chassis Board Configuration................................................................................................ 65
7.1.1 4700 Chassis.......................................................................................................... 65
7.2 Inserting/Extracting Boards .................................................................................................. 65
7.3 Installing Fabric Boards........................................................................................................ 67
7.4 Installing the Line Board (sLB-4018 )................................................................................... 69
7.5 Installing the Management Board (sMB-CM) ....................................................................... 70
7.6 Installing the Power Supply Unit (sPSU-S) .......................................................................... 70
7.7 Fan Units (sFUs)................................................................................................................... 72
7.7.1 Vertical Fan Unit (sFU-40V).................................................................................... 73
7.7.2 Horizontal Fan Unit (sFU-40H)............................................................................... 74
8Verifying Installation..................................................................................................................... 75
8.1 Verifying Start-up Operations ............................................................................................... 75
8.2 Verifying Network Connections ............................................................................................ 75
8.3 Where to Go Next................................................................................................................. 75
8.4 Troubleshooting.................................................................................................................... 75
8.4.1 Solving Operation Problems................................................................................... 75
8.4.2 Identifying Startup Problems................................................................................... 76
9System Cabling ............................................................................................................................. 77
9.1 Cabling Guide Brackets Installation on Either Sides of the Chassis (Line Boards Cables). 77
9.1.1 Installing Cabling Guide Brackets........................................................................... 78
9.1.2 InfiniBand Cabling on Either Side of the Chassis (for Line Cables)....................... 79
9.2 Cabling Guide Brackets Installation under the Chassis (Fabric Board Cables)................... 80
9.2.1 Installing Cabling Guide Brackets for Switches with HyperScale Fabric Boards ... 81
9.2.2 Connecting the InfiniBand Cable between Fabric Boards...................................... 81
9.3 InfiniBand Cabling Guidelines .............................................................................................. 83
9.3.1 InfiniBand Cabling—Do’s and Don’ts ..................................................................... 84
9.4 Replacing Line Boards ......................................................................................................... 84
9.5 Connecting to the Management Ports.................................................................................. 84

Contents
6
Appendix A: Cabling Information and Specifications............................................................. 86
A.1 QSFP Cable.......................................................................................................................... 86
A.2 IB Port Cable Specifications................................................................................................. 87
A.3 1GbE Ports (Management)................................................................................................... 88
List of Figures
Figure 1: Grid Director 4700 with Standard Fabric Boards –Front....................................................... 11
Figure 2: Grid Director 4700 with HyperScale Fabric Boards –Front................................................... 11
Figure 3: Grid Director 4700 –Rear...................................................................................................... 12
Figure 4: 4700 Block Diagram............................................................................................................... 15
Figure 5: Grid Director 4700 Switching Topology.................................................................................. 21
Figure 6: Grid Director 4700 HyperScale Spine Switching Topology.................................................... 22
Figure 7: HyperScale –Mesh Configuration ......................................................................................... 23
Figure 8: HyperScale –Edge Configuration.......................................................................................... 23
Figure 9: sFU-40H Horizontal Fan Unit................................................................................................. 25
Figure 10: 4700 Line Board Slot Numbering......................................................................................... 26
Figure 11: 4700 Label (example)........................................................................................................... 38
Figure 12: Center of Gravity Data ......................................................................................................... 41
Figure 13: 4700 Switch Packed............................................................................................................. 46
Figure 14: 4700 Grounding Connection ................................................................................................ 49
Figure 15: Ground Warning Label......................................................................................................... 49
Figure 16: Grounding Holes for 4700.................................................................................................... 50
Figure 17: 4700 Fully Configured - N:1 Redundancy............................................................................ 53
Figure 18: Overview of Assembled Rail Kit........................................................................................... 56
Figure 19: Positioning the Insert in Front of the Right Bracket.............................................................. 58
Figure 20: Assembling the Rail Bracket ................................................................................................ 58
Figure 21: Right Rail Assembly (top) and Left Rail Assembly (bottom) ................................................ 58
Figure 22: U Alignment.......................................................................................................................... 59
Figure 23: Securing the 4700 Chassis for Transport - Regular Configuration.................................... 62
Figure 24: Securing the 4700 Chassis for Transport - HyperScale Configuration................................ 62
Figure 25: Installing the 19-In Long Bracket.......................................................................................... 63
Figure 26: Using Ejectors to Install a Line Board.................................................................................. 66
Figure 27: Using Ejectors to remove a Line Board................................................................................ 66
Figure 28: Using Ejectors to Install a Fabric Board............................................................................... 66
Figure 29: Using Ejectors to remove a Fabric Board ............................................................................ 67
Figure 30: Front Panel of the sLB-4018 Line Board ........................................................................... 69
Figure 31: Front Panel of the Management Board................................................................................ 70
Figure 32: Front Panel of the Power Supply Module ............................................................................ 71
Figure 33: sFU-40V Vertical Fan Unit Front Panel................................................................................ 73
Figure 34: 4700 Cable Dressing............................................................................................................ 80
Figure 35: HyperScale Cabling Guide Bracket for Switches with HyperScale Fabric Boards
(sFB-4700X2) ........................................................................................................................................ 80

Contents
7
Figure 36: Cable Management for Switches with HyperScale Fabric Boards ...................................... 81
Figure 37: Cabling Dressing between Switches Installed Back-to-Back............................................... 83
Figure 38: QSFP Connector-Dimensions.............................................................................................. 86
Figure 39: Straight-through Cables ....................................................................................................... 88
Figure 40: Cross-connect Cables.......................................................................................................... 88
List of Tables
Table 1: Module Configuration per Chassis .......................................................................................... 18
Table 2: Feature Summary Table.......................................................................................................... 19
Table 3: 4700 Basic Configuration ........................................................................................................ 19
Table 4: Spare Parts and Accessories.................................................................................................. 20
Table 5: Torque Settings for Module Assembly .................................................................................... 26
Table 6: Clearance Requirements......................................................................................................... 27
Table 7: 4700 Power Requirements...................................................................................................... 27
Table 8: Power Characteristics.............................................................................................................. 28
Table 9: Power Data by Module............................................................................................................ 28
Table 10: Module Power Data............................................................................................................... 29
Table 11: BTUs for QDR Sample Configurations.................................................................................. 29
Table 12: Maximum Heat Dissipation.................................................................................................... 30
Table 13: 4700 Airflow........................................................................................................................... 30
Table 14: 4700 Physical Characteristics ............................................................................................... 30
Table 15: HyperScale Physical Characteristics..................................................................................... 30
Table 16: Weights and Dimensions....................................................................................................... 30
Table 17: sLB-4018 Line Board LEDs................................................................................................... 32
Table 18: Fabric Board (sFB-4700) LEDs............................................................................................. 32
Table 19: HyperScale Fabric Board (sFB-4700-2X) LEDs.................................................................... 33
Table 20: Management Board (sMB-CM) LEDs.................................................................................... 33
Table 21: sFU-40H Horizontal Fan Unit LEDs ...................................................................................... 34
Table 22: sPSU-S Power Supply LEDs................................................................................................. 35
Table 23: 4700 Environmental Specifications ....................................................................................... 35
Table 24: Reliability (MTBF) Specifications........................................................................................... 35
Table 25: MTBF per Module Data......................................................................................................... 36
Table 26: Oscillator Speeds .................................................................................................................. 38
Table 27: Vibration ................................................................................................................................ 39
Table 28: Free Fall ................................................................................................................................ 40
Table 29: Export Information................................................................................................................. 40
Table 30: Center of Gravity Data........................................................................................................... 41
Table 31: Required Tools and Materials ............................................................................................... 47
Table 32: 4700 QDR N:1/N:N Redundancy .......................................................................................... 53
Table 33: 4X InfiniBand Port Cabling Specifications............................................................................. 87

Contents
8
Preface
About this Manual
This manual provides installation instructions for the Mellanox Grid DirectorTM 4700 40Gb/s
InfiniBand Switch, the product specifications, unpacking and installation information, unit
power up, and initiation and troubleshooting procedures.
Refer to the official and latest product release notes for last-minute updates.
Technical support may be obtained directly from:
Your regional distributor from whom this product was ordered
Your OEM customer representative
For further information and assistance, go to
http://www.mellanox.com/content/pages.php?pg=support_index.
Audience
The manual is intended primarily for system administrators who are authorized to install a
Grid Director 4700 switch.
It is assumed that the readers are familiar with the InfiniBand technology and terminology.
Related Documentation
For additional information, refer to the following documents:
Grid Director Family Getting Started Guide [LIT-00037]
Grid Director Family User Manual [DOC-00785]
Grid Director Family Release Notes [DOC-00962]
Regulatory and Compliance Reference Guide [DOC-00859]
Document Conventions
The following lists conventions used in this document.
NOTE: Identifies important information that contains helpful suggestions.

Contents
9
CAUTION: Alerts you to the risk of personal injury, system damage, or loss of data.
WARNING: Warns you that failure to take or avoid a specific action might result in
personal injury or a malfunction of the hardware or software. Be aware of the hazards
involved with electrical circuitry and be familiar with standard practices for preventing
accidents before you work on any equipment.
Typography
The following table describes typographical conventions in Mellanox documentation. All
terms refer to isolated terms within body text or regular table text unless otherwise mentioned
in the Notes column.
Term, Construct,
Text Block
Example
Notes
File name, pathname
/opt/ufm/conf/gv.cfg
Console session (code)
-> flashClear <CR>
Complete sample line or block.
Comprises both input and output.
The code can also be shaded.
Linux shell prompt
#
The "#"character stands for the
Linux shell prompt.
Mellanox CLI Guest Mode
Switch >
Mellanox CLI Guest Mode.
Mellanox CLI admin mode
Switch #
Mellanox CLI admin mode
String
< > or []
Strings in < > or [ ] are descriptions
of what will actually be shown on the
screen, for example, the contents of
<your ip> could be 192.168.1.1
Management GUI label, item
name
New Network,
New Environment
Management GUI labels and item
names appear in bold, whether or not
the name is explicitly displayed (for
example, buttons and icons).
User text entered into Manager,
e.g., to assign as the name of a
logical object
"Env1", "Network1"
Note the quotes. The text entered
does not include the quotes.
Chapter 1

Introduction
10
1 Introduction
This chapter provides a description of the Mellanox Grid Director 4700 chassis and its system
boards.
1.1 Mellanox Grid Director™ 4700 Overview
Mellanox’s fourth-generation Grid Director™ 4000 series of smart switches addresses the
growing size and complexity of clusters by providing high interconnect bandwidth, advanced
carrier-class management and a unique HyperScale™ stackable architecture.
The Grid Director™ 4700 is a high performance, ultra low latency and fully non-blocking
InfiniBand switch for high performance clusters. With configurations of up to 324 ports or
double-sided 648 ports of 40 Gb/s per port InfiniBand connectivity, the Grid Director 4700
delivers an impressive 51.8 Tb/s of non-blocking bandwidth with between 100 and 300
nanoseconds of port-to-port latency. Additionally the switch’s 648 ports can be divided
between two racks for weight distribution and greater ease of cabling.
The switch’s HyperScale architecture provides a unique inter-switch link capability for
stacking multiples of 324 ports to form highly scalable, cost effective, and low latency fabrics.
As a result, I/O bottlenecks are removed, allowing applications to operate at maximum
efficiency. As the industry’s largest QDR switching solution, the efficient Grid Director
4700’s smart design provides unprecedented levels of performance and makes it easy to build
clusters that can scale out to thousands of nodes.

Mellanox Grid Director 4700 Installation Manual
11
Figure 1: Grid Director 4700 with Standard Fabric Boards –Front
Figure 2: Grid Director 4700 with HyperScale Fabric Boards –Front

Introduction
12
Figure 3: Grid Director 4700 –Rear

Mellanox Grid Director 4700 Installation Manual
13
1.2 Main Features
Ultra-low latency: between 100 and 300 nanoseconds port-to-port
324 QDR (40Gb/s) auto-negotiating QSFP InfiniBand ports in a 19U switch
648 port option can be divided between two racks for weight distribution and greater ease
of cabling
Unique HyperScale™ architecture allows scaling to thousands of nodes with a single tier of
switches (under 400 nanoseconds of latency)
Available bandwidth: up to 51.8 Tbps of non-blocking bandwidth
Simple and fast device management
Fully managed by Mellanox Unified Fabric Manager (UFM)

Introduction
14
Support for longer and more varied cable options
Zero down time with no single point of failure and real-time fault notifications
QSFP ports supporting either copper or optical cables
Redundant, hot-swappable Management Boards, power supplies, and fans meet stringent
availability requirements
InfiniBand specification 1.2 compliant
1.3 Product Architecture
The Mellanox Grid Director 4700 is a modular, rack-mountable chassis for a wide range of
applications. Highlights of the Grid Director 4700 architecture include:
Up to eighteen (18) line boards with 18 QSFP InfiniBand QSFP external interfaces and 18
QSFP InfiniBand links for connectivity between the Line cards and the internal Switch
fabric
Supports up to nine vertical fabric boards
Supports one or two redundant hot swappable Management Boards (sMB) for fabric
management and chassis management; when two Management boards are installed, they
support failover capabilities in the event of failure of one of the boards
Up to 6 power supply units (sPSU-S) with N:1/N:N redundancy
Front-to-rear air cooling system consisting of a horizontal fan unit (sFU-40H)
and a vertical Fan Unit (sFU-40V) for the Grid Director 4700
Cabling for InfiniBand, management, and power located at rear of chassis
Local and remote management and provisioning using CLI (Command Line Interface).
1.3.1 Main Building Blocks
The Grid Director 4700 rear panel has 18 slots in which you can install the Line Boards
(sLB-4018).

Mellanox Grid Director 4700 Installation Manual
15
The following figure is a simplified functional block diagram of the Grid Director 4700
components.
Figure 4: 4700 Block Diagram
The main building blocks of the Grid Director 4700 chassis are listed and described in the
following sections.
Grid Director Midplane
The Grid Director includes a passive Midplane that connects the InfiniBand differential
signals, control signals and power between the different boards in the chassis. (Fabric Board
signals are routed from the front of the chassis to the Midplane; Line Boards are routed from
the rear of the chassis to the Midplane.) The Midplane is not field serviceable.
Line Boards
The rear side of the Grid Director 4700 can host up to up to eighteen 18-Port QDR (40Gbps)
Line Boards (sLB-4018). Each line board has 18 external QSFP interfaces, and 18 internal
4X InfiniBand interfaces for connectivity between the Line cards and the Switch fabric. Line
boards are installed at the rear of the chassis in a horizontal position.
Fabric Boards
The Grid Director 4700 supports up to nine Fabric Boards (sFB) that are installed at the front
of the chassis.
324-port chassis: The sFB-4700 QDR Fabric Board is installed in a standard 324-port chassis,
in a vertical position. Each sFB-4700 has 36 internal QSFP InfiniBand ports.
648-port chassis: The HyperScale Fabric Board is installed in a 648-port chassis in a vertical
position. Each HyperScale Fabric Board has 36 internal QSFP InfiniBand ports and 12
external 12x InfiniBand ports with CXP connectors.
The Fabric Boards determine the total bandwidth of the switching Clos topology within the
chassis.
Grid Director Power Backplane
The power supply unit (sPSU-S) and horizontal fan unit (sFU-40H) modules are connected to
the system through the backplane, which provides the mechanical, control and electrical
interface to the controllable AC/DC 48V power supplies. It allows electric current flow,

Introduction
16
supplies power to the Midplane via a special power harness, and provides connectivity to the
Midplane and the fan units (sFU-40H).
Air Flow Cooling System
The Grid Director 4700 uses front-to-back air flow cooling and provides full internal
redundancy, monitored fans. Fan speed is dynamically controlled as a function of temperature.
The air cooling system includes a horizontal fan unit (sFU-40H), containing eight fans and A
vertical fan unit (sFU-40V) containing five fans.
LED indicators on the horizontal fan unit sFU-40H provide an on-site report on the operating
conditions of the air cooling system modules, and provide an alarm indicator in the event of a
high temperature condition.
InfiniBand Switch Silicon Firmware
Each Line Board and Fabric Board incorporates InfiniBand 36-port switch (ASIC) devices
with locally attached memory containing firmware that controls basic functionality of the
devices. The firmware controls low level link and physical layers functionality and may be
upgraded via Ethernet or RS-232 on the Management Board or in-band via any of the
InfiniBand Line Board ports.
InfiniBand products are RoHS-6 compliant.
Management
The Grid Director 4700 provides a command line interface (CLI) interface for flexible access
the switch comprehensive management capabilities.
Device Management (DM)
Control a device (e.g., a switch) or a device cluster
Remote management, upgrades, logs/alarms
Management Board
The Grid Director supports either one or two hot-swappable Management Boards.
Power Supplies (sPSU-S)
Mellanox Grid Director switches support multiple power supply modules (sPSU-S)
configured in redundant, hot-swap, current sharing mode for N:1 or N:N redundancy,
depending on the configuration, and suitable for 100-240 VAC mains voltage (auto-sense).
The Grid Director supports up to six power supply modules.
1.3.2 Reset Buttons
In the rare event of a 4700 system lock-up, the Reset button can be used to reset the system.
The system has two Reset buttons: one is located on the front panel of the sFU-40H fan
assembly horizontal fan unit, a second one is on the sMB-CM Management Board front panel.
Management Reset
To reset the Grid Director 4700, push any of the reset buttons using a thick wire or tip of a pen.
Pressing the Reset button for one second resets the sMB-CM Management Board only;
pressing the Reset button for at least six seconds resets the entire chassis.

Mellanox Grid Director 4700 Installation Manual
17
1.4 Management
The Grid Director 4700 includes smart device management that provides a simple interface for
deploying, troubleshooting, maintaining and upgrading the switch. With a simple to use CLI
interface, routine tasks such as monitoring the switch operation or upgrading software and
firmware are made simple.
Chapter 2

System Configurations
18
2 System Configurations
2.1 Configuration Summary
This section summarizes the configuration for the Grid Director 4700 modules.
2.1.1 Module Configuration per Chassis
The following table provides a comprehensive list of modules per chassis, as well as the
maximum number of modules that can be installed in a 4700 system.
Some of the components are provided with the basic configuration chassis. The cabling
guide bracket kit (KIT-00022) for HyperScale Fabric Board (chassis with 648 ports) is
optional.
Table 1: Module Configuration per Chassis
Module
Name
Description
Number of
Modules per
chassis
Line Board
sLB-4018
Includes 18 QSFP ports
Up to 18
Fabric Board
sFB-4700
Includes 36 internal ports
Up to 9
HyperScale Fabric
Board
sFB-4700-2X
Includes 36 internal and 12 external CXP
(12X) ports
Up to 9
Power Supply
sPSU-S
AC/DC Power supply
Up to 6
Vertical Fan Unit
sFU-40V
Vertical Fan Unit
1
Horizontal Fan Unit
sFU-40H
Horizontal Fan Unit
1
Management Board
sMB-CM
Chassis Management Board
Up to 2
Cable Management
Right
N/A
Right cabling guide brackets for cable
management
1 (right)
Cable Management
Left
N/A
Left cabling guide brackets for cable
management
1 (left)
Optional: cabling guide
bracket kit for
HyperScale Fabric
Board
N/A
Optional: kit for 648 ports (must be
ordered separately). Includes HyperScale
cabling guide bracket and long 19"
brackets
1
2.1.1.1 Summary of Cabling Management and Rail Kit
Standard chassis configuration (324 ports):
Cable management (standard)
Rail kit
19" bracket attached to the chassis
HyperScale Fabric Board chassis configuration (648 ports):

Mellanox Grid Director 4700 Installation Manual
19
Cable management (standard)
Rail kit
19" bracket attached to the chassis
KIT-00022 includes 3 parts (must be purchased separately for the 648-port
configuration):
1 HyperScale cabling guide bracket
2x bracket long
8 screws
2.1.2 Specifications Summary
The following table summarizes the Grid Director 4700 specifications.
Table 2: Feature Summary Table
Feature
4700
Rack Height
19U
HyperScale Rack Height
20U
Rack Width
19"
Switch Type
Director
Backplane Bandwidth
51.8 Tbps
Line Board Slots
18
Fabric Board Slots
9
Max # of InfiniBand Ports
324 40 Gbps InfiniBand Ports
Optional 648 40 Gbps InfiniBand Ports
Interface Type
QSFP 40 Gbps (QDR) QSFP ports with support for optical
adapters and cables
Redundant Components
Power Supplies (sPSU-S)
Management Boards (sMB-CM)
Embedded Chassis Management (CLI)
Yes
2.2 4700 Configuration and Part Numbers
The following table details the basic configuration of the 4700 chassis.
Table 3: 4700 Basic Configuration
Product Description
Product/Module
Name
P/N
Grid Director 4700 18-Slot QDR Basic Configuration
Grid Director 4700
VLT-30040
sFB-4700 QDR Fabric Board
sFB-4700
VLT-30041
sFB-4700-2X QDR Fabric Board
sFB-4700-2X
VLT-30042

System Configurations
20
Product Description
Product/Module
Name
P/N
sLB-4018 18-Port QDR Line Board
sLB-4018
VLT-30043
sMB-CM Chassis Management Board
sMB-CM
VLT-30044
sPSU-S 1.4KW AC Power Supply Module
sPSU-S
VLT-30046
sFU-40H Horizontal Fan Unit
sFU-40H
VLT-30047-F
sFU-40V Vertical Fan Unit
sFU-40V
VLT-30048-F
The above parts can be ordered separately. The following table provides the list of additional
spare parts and accessories.
Table 4: Spare Parts and Accessories
Description
Comment
P/N
Cable Management Right
Delivered with the
switch
KIT-00023
Cable Management Left
KIT-00024
Cabling guide bracket kit for HyperScale Fabric Board. It
includes the following parts:
2 x 19" Long bracket (MEC-00576)
Cable management for Spine in 648 configuration
(MEC-00593)
Screws for assembly (380J30011)
Set must be ordered per
chassis
KIT-00022
2.3 Switch Internal Topology
High performance computing clusters typically utilize Clos networks, sometimes referred to
as "Fat Tree" or Constant Bisectional Bandwidth (CBB) networks, to construct large node
count non-blocking configurations. A Clos network is a switch topology in which integrated
non-blocking switch chips (crossbars) with a relatively low number of ports are used to build a
non-blocking switch topology supporting a much larger number of endpoints.
The Grid Director 4700 features an internal Clos network with Line Boards as the edge (leaf)
switches and Fabric Boards as the core (spine) switches. Each Line Board switch chip is
connected to each switch chip on the Fabric Boards through one or more InfiniBand
connections.
The path from one external port to another node may require three switch hops:
Hop 1 at a Line Board
Hop 2 at a Fabric Board
Hop 3 at another Line Board.
2.3.1 HyperScale Fabric Boards
HyperScale™ is an interconnect technology, enabling large scale fabrics at reduced latency,
cost and complexity. Mellanox HyperScale fabric cards have double the capacity of standard
fabric cards, as well as external (12x) ports. This combination enables new, modular
configurations that allow scaling to large fabric sizes with the minimum amount of ports and