Nvidia Mellanox ConnectX-5 User manual

Exported on Sep/19/2022 09:36 AM







NVIDIA ConnectX-5 InfiniBand/VPI Adapter
Cards User Manual



2
Table of Contents
Introduction..................................................................................... 6
Product Overview ....................................................................................6
Features and Benefits ...............................................................................7
Operating Systems/Distributions ..................................................................9
Connectivity ..........................................................................................9
Interfaces ......................................................................................10
InfiniBand Interface ............................................................................... 10
Ethernet QSFP28 Interface ....................................................................... 10
PCI Express Interface.............................................................................. 10
LED Interface ....................................................................................... 10
Thermal Sensors.................................................................................... 11
Hardware Installation ........................................................................12
Safety Warnings .................................................................................... 12
Installation Procedure Overview................................................................. 12
System Requirements ............................................................................. 12
Hardware Requirements ...................................................................... 12
Airflow Requirements ......................................................................... 13
Software Requirements ....................................................................... 14
Unpacking the Product ............................................................................ 14
Bracket Replacement Instructions .............................................................. 15
Installation Instructions........................................................................... 15
Cables and Modules................................................................................ 17
Identifying the Card in Your System............................................................. 18
Uninstalling the Card.............................................................................. 19
Safety Precautions ............................................................................. 19
Card Removal................................................................................... 19
Driver Installation.............................................................................21
Linux Driver Installation .......................................................................... 21
Prerequisites.................................................................................... 21
Downloading NVIDIA OFED .................................................................... 21
Installing NVIDIA OFED ........................................................................ 22
Installing MLNX_OFED Using YUM............................................................ 27

3
Installing MLNX_OFED Using apt-get Tool .................................................. 30
Updating Firmware After Installation....................................................... 31
UEFI Secure Boot............................................................................... 32
Performance Tuning ........................................................................... 34
Windows Driver Installation ...................................................................... 34
Software Requirements ....................................................................... 34
Downloading NVIDIA WinOF-2 Driver ........................................................ 35
Installing NVIDIA WinOF-2 Driver ............................................................ 35
Uninstalling NVIDIA WinOF-2 Driver ......................................................... 42
Extracting Files Without Running Installation ............................................. 43
Firmware Upgrade ............................................................................. 45
Troubleshooting ...............................................................................46
General Tips ........................................................................................ 46
Gathering Information - Linux Environment ................................................... 47
Gathering Information - Windows Environment ............................................... 47
Updating Adapter Firmware .................................................................48
Specifications..................................................................................49
MCX555A-ECAT Specifications .................................................................... 49
MCX556A-EC[A/U]T Specifications .............................................................. 50
MCX556A-ECAT/MCX556A-ECUT Specifications ................................................ 50
MCX556A-EDAT Specifications .................................................................... 51
Airflow Specifications ............................................................................. 53
LED Specifications ................................................................................. 53
Board Mechanical Drawings and Dimensions................................................... 53
Adapter Card Mechanical Drawing (Component Side).................................... 54
Bracket Mechanical Drawing ................................................................. 54
Finding the GUID/MAC and Serial Number on the Adapter Card.......................56
MCX555A-ECAT board label example:........................................................... 56
MCX556A-ECAT board label example:........................................................... 56
MCX556A-ECUT board label example: .......................................................... 57
MCX556A-EDAT board label example:........................................................... 57
Document Revision History ..................................................................58


4
•
•
About This Manual
This User Manual describes NVIDIA® ConnectX®-5 and ConnectX®-5Ex VPI Single and DualQSFP28
port PCI Express x16 adapter cards. It provides details as to the interfaces of the
board,specifications, required software and firmware for operating the board, and relevant
documentation.
Ordering Part Numbers
The table below provides the ordering part numbers (OPN) for the available ConnectX-5 VPI adapter
cards.
IC in Use OPN Marketing Description
ConnectX®-
5
MCX555A-
ECAT
ConnectX®-5 VPI adapter card, EDR IB (100Gb/s) and 100GbE, single-port QSFP28,
PCIe 3.0 x16, tall bracket
MCX556A-
ECAT
ConnectX®-5 VPI adapter card, EDR IB (100Gb/s) and 100GbE, dual-port QSFP28,
PCIe 3.0 x16, tall bracket
MCX556A-
ECUT
ConnectX®-5 VPI adapter card, EDR IB (100Gb/s) and 100GbE, dual-port QSFP28,
PCIe3.0 x16, UEFI enabled, tall bracket
ConnectX®-
5 Ex
MCX556A-
EDAT
ConnectX®-5 Ex VPI adapter card, EDR IB (100Gb/s) and 100GbE, dual-port
QSFP28, PCIe 4.0 x16, tall bracket
Intended Audience
This manual is intended for the installer and user of these cards.The manual assumes basic
familiarity with InfiniBand and Ethernet network and architecturespecifications.
Technical Support
Customers who purchased NVIDIAproducts directly from NVIDIA are invited to contact usthrough
the following methods:
URL:https://www.nvidia.com> Support
E-mail:[email protected]
Customers who purchased NVIDIA Global Support Services, please see your contract for details
regarding Technical Support.
Customers who purchased NVIDIA products through an NVIDIA-approved reseller should first seek
assistance through their reseller.
Related Documentation
MLNX_OFED for Linux User Manual and
Release Notes
User Manual describing OFED features, performance, band
diagnostic, tools content, and configuration. SeeMLNX_OFED for
Linux Documentation.
WinOF-2 for WindowsUser Manualand
Release Notes
User Manual describing WinOF-2 features, performance, Ethernet
diagnostic, tools content, and configuration. SeeWinOF-2 for
Windows Documentation.

5
NVIDIA VMware for Ethernet User Manual User Manual and release notes describing the various components
of the NVIDIA ConnectX® NATIVE ESXi stack. SeeVMware® ESXi
Drivers Documentation.
NVIDIA Firmware Utility (mlxup) User
Manual and Release Notes
NVIDIA firmware update and query utility used to update the
firmware. Refer toFirmware Utility (mlxup) Documentation.
NVIDIA Firmware Tools (MFT) User Manual
User Manual describing the set of MFT firmware management
tools for a single node. SeeMFT User Manual.
InfiniBand Architecture Specification
Release 1.2.1, Vol 2 - Release 1.3
InfiniBand Specifications
IEEE Std 802.3 Specification IEEE Ethernet Specifications
PCI Express Specifications Industry Standard PCI Express Base and Card Electromechanical
Specifications. Refer toPCI-SIG Specifications.
LinkX Interconnect Solutions
LinkX InfiniBand cables and transceivers are designed to
maximize the performance of High-Performance Computing
networks, requiring high-bandwidth, low-latency connections
between compute nodes and switch nodes.NVIDIA offers one of
the industry’s broadest portfolio ofQDR/FDR10 (40Gb/s), FDR
(56Gb/s), EDR/HDR100 (100Gb/s), HDR (200Gb/s)and NDR
(400Gb/s) cables, includingDirect Attach Copper cables (DACs),
copper splitter cables, Active Optical Cables (AOCs) and
transceivers in a wide range of lengths from 0.5m to 10km.In
addition to meeting IBTA standards, NVIDIAtests every product in
an end-to-end environment ensuring a Bit Error Rate of less than
1E-15.Read more atLinkX Cables and Transceivers.
Document Conventions
When discussing memory sizes, MB and MBytes are used in this document to mean size in mega
Bytes. The use of Mb or Mbits (small b) indicates size in mega bits. IB is used in this documentto
mean InfiniBand. In this document PCIe is used to mean PCI Express.

6
Introduction
Product Overview
This is the User Guide for VPI adapter cards based on the ConnectX®-5integrated circuit device.
These adapters connectivity provide the highest performing and mostflexible interconnect solution
for PCI Express Gen 3.0/4.0 servers used in Enterprise Data Centers,High-Performance Computing,
and Embedded environments.
The following table provides the ordering part number, port speed, number of ports, and PCI Express
speed. Each adapter comes with two bracket heights - short and tall.
ConnectX-5 Ex InfiniBand/VPI Adapter Cards
Model ConnectX-5 Ex InfiniBand/VPI Adapter Cards
Part Number MCX556A-EDAT
Data Rate
InfiniBand: SDR/DDR/QDR/FDR/EDR
Ethernet: 10/25/40/50/100 Gb/s
Network Connector Type
Dual-port QSFP28
PCI Express Connectors
PCIe Gen 3.0/4.0 x16 (a), (b)
SERDES @ 8.0GT/s / 16.0GT/s
Dimensions 2.71 in. x 5.6 in. (68.90mm x 142.24 mm) – low profile
RoHS RoHS Compliant
Adapter IC Part Number
MT28808A0-FCCF-EV
Device ID (decimal) 4121 for Physical Function (PF) and 4122 for Virtual Function (VF)
a. PCIe 4.0 x16 bus can supply a maximum bandwidth of 256Gb/s (=16 *16GT/s, including overhead),
and can support 200Gb/s when both network ports of the card run at 100Gb/s.
b.This card has been tested and certified with PCIe 3.0 servers. PCIe 4.0 interface will be tested
when servers with Gen 4.0 support become available.
ConnectX-5 InfiniBand/VPI Adapter Cards
Model ConnectX-5 InfiniBand/VPI Adapter Cards
Part Number
MCX555A-ECAT MCX556A-ECAT MCX556A-ECUT
Data Rate InfiniBand: SDR/DDR/QDR/FDR/EDR
Ethernet: 10/25/40/50/100 Gb/s
Network Connector Type Single-port QSFP28 Dual-port QSFP28 Dual-port QSFP28
PCI Express Connectors PCIe Gen 3.0 x16; SerDes @ 8.0GT/s

7
Model ConnectX-5 InfiniBand/VPI Adapter Cards
Dimensions 2.71 in. x 5.6 in. (68.90mm x 142.24 mm) – low profile
RoHS RoHS Compliant
Adapter IC Part Number MT27808A0-FCCF-EV
UEFI - - Enabled
Device ID (decimal) 4119 for Physical Function (PF) and 4120 for Virtual Function (VF)
For more detailed information seeSpecifications.
Features and Benefits
Feature Description
PCI
Express
(PCIe)
Uses PCIe Gen 3.0 (8GT/s) and Gen 4.0 (16GT/s) through an x16 edge connector. Gen 1.1 and 2.0
compatible.
EDR
InfiniBand
A standard InfiniBand data rate, where each lane of a 4X port runs a bit rate of 25.78125Gb/s
with a 64b/66b encoding, resulting in an effective bandwidth of 100Gb/s.
100Gb/s
Virtual
ProtocolIn
terconnect
(VPI)Adap
ter
ConnectX-5 offers the highest throughput VPI adapter, supporting EDR 100Gb/s InfiniBand and
100Gb/s Ethernet and enabling any standard networking, clustering, or storage to operate
seamlessly over any converged network leveraging a consolidated software stack.
InfiniBand
Architectu
re
Specificati
on v1.3
compliant
ConnectX-5 delivers low latency, high bandwidth, and computing efficiency for performance-
driven server and storage clustering applications. ConnectX-5 is InfiniBand Architecture
Specification v1.3 compliant.
This section describes hardware features and capabilities. Please refer to the relevant
driver and/or firmware release notes for feature availability.


8
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Feature Description
Up to 100
Gigabit
Ethernet
NVIDIA adapters comply with the following IEEE 802.3 standards:
100GbE/ 50GbE / 40GbE / 25GbE / 10GbE / 1GbE
IEEE 802.3bj, 802.3bm 100 Gigabit Ethernet
IEEE 802.3by, Ethernet Consortium25, 50 Gigabit Ethernet, supporting all FEC modes
IEEE 802.3ba 40 Gigabit Ethernet
IEEE 802.3ae 10 Gigabit Ethernet
IEEE 802.3ap based auto-negotiation and KR startup
Proprietary Ethernet protocols (20/40GBASE-R2, 50GBASE-R4)
IEEE 802.3ad, 802.1AX Link Aggregation
IEEE 802.1Q, 802.1P VLAN tags and priority
IEEE 802.1Qau (QCN)
Congestion Notification
IEEE 802.1Qaz (ETS)
IEEE 802.1Qbb (PFC)
IEEE 802.1Qbg
IEEE 1588v2
Jumbo frame support (9.6KB)
Memory SPI Quad - includes 128Mbit SPI Quad Flash device (W25Q128FVSIG device by ST
Microelectronics).
FRU EEPROM - Stores the parameters and personality of the card. The EEPROM capacity is
128Kbit. FRU I2C address is (0x50) and is accessible through the PCIe SMBus. Note:
Address 0x58 is reserved.)
Overlay
Networks
In order to better scale their networks, data center operators often create overlay networks that
carry traffic from individual virtual machines over logical tunnels in encapsulated formats such
as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from
the hardware offloading engines, placing higher loads on the host CPU. ConnectX-5 effectively
addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that
encapsulate and de-capsulate the overlay protocol.
RDMA and
RDMA over
Converged
Ethernet
(RoCE)
ConnectX-5, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged
Ethernet) technology, delivers low-latency and high performance over Band and Ethernet
networks. Leveraging data center bridging (DCB) capabilities as well as ConnectX-5 advanced
congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services
over Layer 2 and Layer 3 networks.
NVIDIA
PeerDirect
™
PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary
internal data copies between components on the PCIe bus (for example, from GPU to CPU), and
therefore significantly reduces application run time. ConnectX-5 advanced acceleration
technology enables higher cluster efficiency and scalability to tens of thousands of nodes.
CPU
Offload
Adapter functionality enabling reduced CPU overhead allowing more available CPU for
computation tasks.
Open
VSwitch
(OVS)
offload
using
ASAP2
• Flexible match-action flow tables
• Tunneling encapsulation/decapsulation
Quality of
Service
(QoS)
Support for port-based Quality of Service enabling various application requirements for latency
and SLA.

9
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Feature Description
Hardware-
based I/O
Virtualizat
ion
ConnectX-5 provides dedicated adapter resources and guaranteed isolation and protection for
virtual machines within the server.
Storage
Accelerati
on
A consolidated compute and storage network achieves significant cost-performance advantages
over multi-fabric networks. Standard block and file access protocols can leverage InfiniBand
RDMA for high-performance storage access.
NVMe over Fabric offloads for the target machine
Erasure Coding
T10-DIF Signature Handover
SR-IOV ConnectX-5 SR-IOV technology provides dedicated adapter resources and guaranteed isolation
and protection for virtual machines (VM) within the server.
High-
Performan
ce
Accelerati
ons
Tag Matching and Rendezvous Offloads
Adaptive Routing on Reliable Transport
Burst Buffer Offloads for Background Checkpointing
UEFI
UEFI is a standard firmware interface designed to replace BIOS. NVIDIA UEFI Network driver
allows boot over network via PXE (Preboot eXecution Environment). This network driver allows
remote boot over InfiniBand or Ethernet, or Boot over iSCSI (Bo-iSCSI) in UEFI mode, and also
supports the SecureBoot standard. The UEFI Network driver allows IT managers the flexibility to
deploy servers with a single adapter card into InfiniBand or Ethernet networks while also
enabling booting from LAN or remote storage targets. In addition to boot capabilities, NVIDIA
UEFI Network driver provides firmware management and diagnostic protocols compliant with the
UEFI specification.
For further information, refer to theNVIDIA PreBoot Drivers User Manual.
Supported in MCX556A-ECUT.
Operating Systems/Distributions
RHEL/CentOS
Windows
FreeBSD
VMware
OpenFabrics Enterprise Distribution (OFED)
OpenFabrics Windows Distribution (WinOF-2)
Connectivity
Interoperable with 1/10/25/40/50/100 Gb/s Ethernet switches
Passive copper cable with ESD protection
Powered connectors for optical and active cable support

10
•
•
•
•
•
•
Interfaces
InfiniBand Interface
The network ports of the ConnectX®-5 adapter cards are compliant with the InfiniBand
ArchitectureSpecification, Release 1.3. InfiniBand traffic is transmitted through the cards' QSFP28
connectors.
Ethernet QSFP28 Interface
The network ports of the ConnectX®-5 adapter card are compliant with the IEEE 802.3
Ethernetstandards listed in Features and Benefits.Ethernet traffic is transmitted through thecards'
QSFP28 connectors.
PCI Express Interface
The ConnectX®-5 adapter card supports PCI Express Gen 3.0/4.0 (1.1 and 2.0 compatible)through
an x16 edge connector. The device can be either a master initiating the PCI Express busoperations,
or a slave responding to PCI bus operations.The following lists PCIe interface features:
PCIe Gen4.0 and 3.0 compliant, 2.0 and 1.1 compatible
2.5, 5.0, or 8.0, or 16.0 GT/s link rate x16
Auto-negotiates to x16, x8, x4, x2, or x1
Support for MSI/MSI-X mechanisms
LED Interface
There is one bi-color (yellow/green) I/O LED per port to indicate link status located on the adapter
card.
Physical and Logical Link Indications - Ethernet Mode:
State LED Behavior
Beacon command for locating the card Blinks yellow at 1Hz
Error Blinks yellow at 4Hz until error is resolved. The reason for
the error may be:
Failure of I2C access to the port
Over-current conditions of the port
Link Activity Blinks green reflecting the link speed
Link Up Constant green light

11
•
•
Physical and Logical Link Indications - InfiniBand Mode:
State LED Behavior
Beacon command for locating the card Blinks yellow at 1Hz
Error Blinks yellow at 4Hz until error is resolved. The reason for
the error may be:
Failure of I2C access to the port
Over-current conditions of the port
Link Activity Blinks green reflecting the link speed
Link Up Constant green light
Thermal Sensors
Unable to render include or excerpt-include. Could not retrieve page.

12
Hardware Installation
Installation and initialization of ConnectX-5 adapter cards require attention to the mechanical
attributes, power specification, and precautions for electronic equipment.
Safety Warnings
Please observe all safety warnings to avoidinjury and prevent damage to system components. Note
that not all warnings are relevant to all models.
Unable to render include or excerpt-include. Could not retrieve page.
Installation Procedure Overview
The installation procedure of ConnectX-5 adapter cards involve the following steps:
Step Procedure
1Check the system’s hardware and software requirements.
2 Pay attention to the airflow consideration within the host system.
3 Unpack the product.
4 (Optional) Replace the full-height mounting bracket with the supplied short
bracket.
5 Install the adapter card in the system.
6 Connect cables or modules to the card.
7 Identify ConnectX-5 adapter card in the system.
System Requirements
Hardware Requirements
Safety warnings are provided here in the English language. For safety warnings in other
languages, refer to the Adapter Installation Safety Instructions document available on
NVIDIA's website.

Unless otherwise specified, NVIDIA products are designed to work in an environmentally
controlled data center with low levels of gaseous and dust (particulate) contamination.


13
A system with a PCI Express slot of corresponding bus width is required for installing the card.
Airflow Requirements
ConnectX-5 adapter cards are offered with two airflow patterns: from the heatsink to the network
ports, and vice versa, as shown below.
Please refer to the "Specifications" chapter for airflow numbers for each specific card model.
Airflowfromthe heatsinkto the network ports:
 
Airflowfrom the network portsto the heatsink:

The operation environment should meet severity level G1 as per ISA 71.04 for gaseous
contamination and ISO 14644-1 class 8 for cleanliness level.
For proper operation and performance, please make sure to use a PCIe slot with a
corresponding bus width and that can supply sufficient power to your card. Refer to the
Specifications section of the manual for more power requirements.

All cards in the system should be planned with the same airflow direction.


14
•
•
1.
a.
b.
2.
Software Requirements
See Operating Systems/Distributions section under the Introduction section.
Software Stacks - NVIDIA OpenFabric software package MLNX_OFED for Linux, WinOF-2 for
Windows, and VMware. See the Driver Installation section.
Unpacking the Product
Unpack the adapter package and place them on an antistatic surface, and verify you have
received the following items:
ConnectX-5 Adapter Card, with a tall bracket assembled on the card.
Short bracket (accessory)
Check the parts for visible damage that may have occurred during shipping.
•
•
•
•
The adapter is being installed in a system that operates with voltages that can be lethal.
Before opening the case of the system, observe the following precautions to avoid injury
and prevent damage to system components.
Remove any metallic objects from your hands and wrists.
Make sure to use only insulated tools.
Verify that the system is powered off and is unplugged.
It is strongly recommended to use an ESD strap or other antistatic devices.

Please note that if the card is removed hastily from the antistatic bag, the plastic
ziplock may harm the EMI fingers on the networking connector. Carefully remove the
card from the antistatic bag to avoid damaging the EMI fingers.


15
3.
•
•
1.
2.
3.
1.
2.
1.
Shut down your system if active:
Turn off the power to the system, and disconnect the power cord. Refer to the system
documentation for instructions. Before you install the ConnectX-5 card, make sure that the
system is disconnected from power.
Bracket Replacement Instructions
The card is usually shipped with an assembled high-profile bracket. If this form factor is suitable for
your requirements, you can skip the remainder of this section and move toInstallation Instructions.
If you need to replace the high-profile bracket with the short bracket that is included in the
shipping box, please follow the instructions in this section.
To replace the bracket you will need the following parts:
The new brackets of the proper height
The 2 screws saved from the removal of the bracket
Removing the Existing Bracket
Using a torque driver, remove the two screws holding the bracket in place.
Separate the bracket from the ConnectX-5 card.
Save the two screws.
Installing the New Bracket
Place the bracket onto the card until the screw holes line up.
Screw on the bracket using the screws saved from the bracket removal procedure above.
Installation Instructions
This section provides detailed instructions on how to install your adapter card in a system.
Before installing the card, make sure that the system is off and the power cord is not
connected to the server. Please follow proper electrical grounding procedures.
Due to risk of damaging the EMI gasket, it is not recommended to replace the bracket more
than three times.

Be careful not to put stress on the LEDs on the adapter card.

Do not force the bracket onto the adapter card.

Use a torque driver to apply up to 2 lbs-in torque on the screws.

Please note that the following figures are for illustration purposes only.


16
2.
3.
4.
5.
Open the system case.
Place the adapter in an available PCI Express slot.
Applying even pressure at both corners of the card, insert the adapter card into the PCI
Express slot until firmly seated.
When the adapter is properly seated, the port connectors are aligned with the slot opening,
and the adapter faceplate is visible against the system chassis.

17
6.
1.
2.
a.
b.
c.
d.
3.
4.
5.
6.
Secure the adapter with the adapter clip or screw.
Cables and Modules
To obtain the list of supported NVIDIA cables for your adapter, please refer to theCables Reference
Tableathttp://www.nvidia.com/products/interconnect/cables-configurator.php.
Cable Installation
All cables can be inserted or removed with the unit powered on.
To insert a cable, press the connector into the port receptacle until the connector is firmly
seated.
Support the weight of the cable before connecting the cable to the adapter card. Do
this by using a cable holder or tying the cable to the rack.
Determine the correct orientation of the connector to the card before inserting the
connector. Do not try and insert the connector upside down. This may damage the
adapter card.
Insert the connector into the adapter card. Be careful to insert the connector straight
into the cage. Do not apply any torque, up or down, to the connector cage in the
adapter card.
Make sure that the connector locks in place.
After inserting a cable into a port, the Green LED indicator will light when the physical
connection is established (that is, when the unit is powered on and a cable is plugged into
the port with the other end of the connector plugged into a functioning port). SeeAdapter
Card LED Operations.
After plugging in a cable, lock the connector using the latching mechanism particular to the
cable vendor. hen data is being transferred the Green LED will blink. SeeAdapter Card LED
Operations under the Interfaces section.
Care should be taken as not to impede the air exhaust flow through the ventilation holes. Use
cable lengths which allow for routing horizontally around to the side of the chassis before
bending upward or downward in the rack.
To remove a cable, disengage the locks and slowly pull the connector away from the port
receptacle. LED indicator will turn off when the cable is unseated.
To uninstall the adapter card, see Uninstalling the Card.

When installing cables make sure that the latches engage.

Always install and remove cables by pushing or pulling the cable and
connector in a straight line with the card.


18
1.
2.
3.
4.
5.
Identifying the Card in Your System
On Linux
Get the device location on the PCI bus by running lspci and locating lines with the string“Mellanox
Technologies”:
lspci |grep -i Mellanox
Network controller: Mellanox Technologies MT28800 Family [ConnectX-5]
On Windows
Open Device Manager on the server. ClickStart=>Run, and then enterdevmgmt.msc.
ExpandSystem Devicesand locate your NVIDIA ConnectX-5 adapter card.
Right click the mouse on your adapter's row and selectPropertiesto display the adapter card
properties window.
Click theDetailstab and selectHardware Ids(Windows 2012/R2/2016) from
thePropertypull-down menu.
PCI Device (Example)

In theValuedisplay box, check the fields VEN and DEV (fields are separated by ‘&’). In the
display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to
0x15B3 – this is the Vendor ID of NVIDIA; and DEV is equal to 1018 (for ConnectX-5) – this is a
valid NVIDIA PCI Device ID.

19
1.
2.
3.
1.
2.
3.
4.
Uninstalling the Card
Safety Precautions
The adapter is installed in a system that operates with voltages that can be lethal. Before
uninstalling the adapter card, please observe the following precautions to avoid injury and prevent
damage to system components.
Remove any metallic objects from your hands and wrists.
It is strongly recommended to use an ESD strap or other antistatic devices.
Turn off the system and disconnect the power cord from the server.
Card Removal
Please note that the following images are for illustration purposes only.
Verify that the system is powered off and unplugged.
Wait 30 seconds.
To remove the card, disengage the retention mechanisms on the bracket (clips or screws).
Holding the adapter card from its center, gently pull the adapter card out of the PCI Express
slot.
If the PCI device does not have a NVIDIA adapter ID, return to Step 2 to check
another device.

The list of NVIDIA PCI Device IDs can be found in the PCI ID repository at http://
pci-ids.ucw.cz/read/PC/15b3.

Please note that the following images are for illustration purposes only.


20
5. When the port connectors reach the top of the chassis window, gently pull the adapter card
in parallel to the motherboard.
