Chelsio Communications Chelsio T5 User manual

RHEL6.7 19pef3.12segmenUe mo

Chelsio T5/T4 Unified Wire for Linux ii
This document and related products are distributed under licenses restricting their use, copying,
distribution, and reverse-engineering.
No part of this document may be reproduced in any form or by any means without prior written permission
by Chelsio Communications.
All third party trademarks are copyright of their respective owners.
THIS DOCUMENTATION IS PROVIDED “AS IS” AND WITHOUT ANY EXPRESS OR IMPLIED
WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
THE USE OF THE SOFTWARE AND ANY ASSOCIATED MATERIALS (COLLECTIVELY THE
“SOFTWARE”) IS SUBJECT TO THE SOFTWARE LICENSE TERMS OF CHELSIO
COMMUNICATIONS, INC.
Sales
For all sales inquiries please send email to sa[email protected]
Support
For all support related questions please send email to [email protected]
Copyright © 2016.Chelsio Communications. All Rights Reserved.
Chelsio ® is a registered trademark of Chelsio Communications.
All other marks and names mentioned herein may be trademarks of their respective companies.
Chelsio Communications (Headquarters)
209 North Fair Oaks Avenue,
Sunnyvale, CA 94085
U.S.A
www.chelsio.com
Tel: [phone]
Fax: [phone]
Chelsio (India) Private Limited
Subramanya Arcade, Floor 3, Tower B
No. 12, Bannerghatta Road,
Bangalore-560029
Karnataka,
India
Tel: [phone]
Fax: [phone]
Chelsio KK (Japan)
Regus Place Canada
Akasaka 7-3-37
Minato-ku, Tokyo,
Japan 107-0052
Tel: [phone]

Chelsio T5/T4 Unified Wire for Linux iii
Document History
Version
Revision Date
1.0.0
12/08/2011
1.0.1
01/09/2013
1.0.2
01/27/2013
1.0.3
03/26/2013
1.0.4
04/12/2013
1.0.5
06/20/2013
1.0.6
08/17/2013
1.0.7
10/22/2013
1.0.8
03/08/2013
1.0.9
05/15/2013
1.1.0
07/26/2013
1.1.1
08/14/2013
1.1.2
12/06/2013
1.1.3
12/19/2013
1.1.4
03/13/2014
1.1.5
05/02/2014
1.1.6
06/30/2014
1.1.7
10/22/2014
1.1.8
11/04/2014
1.1.9
02/05/2015
1.2.0
03/04/2015
1.2.1
03/25/2015
1.2.2
06/03/2015
1.2.3
08/05/2015
1.2.4
02/29/2016

Chelsio T5/T4 Unified Wire for Linux iv
TABLE OF CONTENTS
I. CHELSIO UNIFIED WIRE 12
1. Introduction 13
1.1. Features 13
1.2. Hardware Requirements 14
1.3. Software Requirements 14
1.4. Package Contents 15
2. Hardware Installation 18
3. Software/Driver Installation 21
3.1. Pre-requisites 22
3.2. Installing Chelsio Unified Wire from source 22
3.3. Installing Chelsio Unified Wire from RPM 33
3.4. Firmware update 35
4. Software/Driver Uninstallation 37
4.1. Uninstalling Chelsio Unified Wire from source 37
4.2. Uninstalling Chelsio Unified Wire from RPM 41
5. Configuring Chelsio Network Interfaces 43
5.1. Configuring 40G adapters 43
5.2. Configuring network-scripts 45
5.3. Creating network-scripts 45
5.4. Checking Link 46
6. Performance Tuning 47
7. Software/Driver Update 48
II. NETWORK (NIC/TOE) 49
1. Introduction 50
1.1. Hardware Requirements 50
1.2. Software Requirements 51
2. Software/Driver Loading 52
2.1. Loading in NIC mode (without full offload support) 52
2.2. Loading in TOE mode (with full offload support) 52
3. Software/Driver Unloading 53
3.1. Unloading the NIC driver 53
3.2. Unloading the TOE driver 53
4. Software/Driver Configuration and Fine-tuning 55
4.1. Instantiate Virtual Functions (SR-IOV) 55
4.2. Enabling Busy waiting 55
4.3. Performance Tuning 56
III. VIRTUAL FUNCTION NETWORK (VNIC) 62

Chelsio T5/T4 Unified Wire for Linux v
1. Introduction 63
1.1. Hardware Requirements 63
1.2. Software Requirements 64
2. Software/Driver Loading 65
2.1. Instantiate Virtual Functions 65
2.2. Loading the driver 65
3. Software/Driver Unloading 66
3.1. Unloading the driver 66
IV. IWARP (RDMA) 67
1. Introduction 68
1.1. Hardware Requirements 68
1.2. Software Requirements 68
2. Software/Driver Loading 70
2.1. Loading iWARP driver 70
3. Software/Driver Unloading 71
4. Software/Driver Configuration and Fine-tuning 72
4.1. Testing connectivity with ping and rping 72
4.2. Enabling various MPIs 73
4.3. Setting up NFS-RDMA 80
4.4. Performance Tuning 82
V. RDMA BLOCK DEVICE DRIVER (RBD) 83
1. Introduction 84
1.1. Hardware Requirements 84
1.2. Software Requirements 84
2. Software/Driver Loading 85
3. Software/Driver Unloading 86
4. Software/Driver Configuration and Fine-tuning 87
VI. WD-UDP 88
1. Introduction 89
1.1. Hardware Requirements 89
1.2. Software Requirements 89
2. Software/Driver Loading 91
3. Software/Driver Unloading 92
4. Software/Driver Configuration and Fine-tuning 93
4.1. Accelerating UDP Socket communications 93
VII. ISCSI PDU OFFLOAD TARGET 99
1. Introduction 100

Chelsio T5/T4 Unified Wire for Linux vi
1.1. Features 100
1.2. Hardware Requirements 101
1.3. Software Requirements 102
2. Software/Driver Loading 104
2.1. Latest iSCSI Software Stack Driver Software 104
3. Software/Driver Unloading 106
4. Software/Driver Configuration and Fine-tuning 107
4.1. Command Line Tools 107
4.2. iSCSI Configuration File 107
4.3. A Quick Start Guide for Target 108
4.4. The iSCSI Configuration File 110
4.5. Challenge-Handshake Authenticate Protocol (CHAP) 121
4.6. Target Access Control List (ACL) Configuration 123
4.7. Target Storage Device Configuration 125
4.8. Target Redirection Support 127
4.9. The command line interface tools “iscsictl” & “chisns” 128
4.10. Rules of Target Reload (i.e. “on the fly” changes) 133
4.11. System Wide Parameters 135
4.12. Performance Tuning 136
VIII. ISCSI PDU OFFLOAD INITIATOR 137
1. Introduction 138
1.1. Hardware Requirements 138
1.2. Software Requirements 139
2. Software/Driver Loading 140
3. Software/Driver Unloading 141
4. Software/Driver Configuration and Fine-tuning 142
4.1. Accelerating open-iSCSI Initiator 142
4.2. Auto login from cxgb4i initiator at OS bootup 144
IX. DATA CENTER BRIDGING (DCB) 146
1. Introduction 147
1.1. Hardware Requirements 147
1.2. Software Requirements 147
2. Software/Driver Loading 149
3. Software/Driver Unloading 150
4. Software/Driver Configuration and Fine-tuning 151
4.1. Configuring Cisco Nexus 5010 switch 151
4.2. Configuring the Brocade 8000 switch 154
5. Running NIC & iSCSI Traffic together with DCBx 156

Chelsio T5/T4 Unified Wire for Linux vii
X. FCOE PDU OFFLOAD TARGET 157
1. Introduction 158
1.1. Hardware Requirements 158
1.2. Software Requirements 158
2. Software/Driver Loading 159
3. Software/Driver Unloading 160
4. Software/Driver Configuration and Fine-tuning 161
4.1. Configuring Cisco Nexus 5010 switch 161
4.2. Collecting port information 164
4.3. Configuring LUNs on Target 167
4.4. Configuring Persistent Target 169
4.5. Verifying initiators connected to the target 170
4.6. Removing LUNs 171
4.7. Performance Tuning 171
XI. FCOE FULL OFFLOAD INITIATOR 174
1. Introduction 175
1.1. Hardware Requirements 175
1.2. Software Requirements 175
2. Software/Driver Loading 177
3. Software/Driver Unloading 178
4. Software/Driver Configuration and Fine-tuning 179
4.1. Configuring Cisco Nexus 5010 and Brocade switch 179
4.2. FCoE fabric discovery verification 179
4.3. Formatting the LUNs and Mounting the Filesystem 183
4.4. Creating Filesystem 184
4.5. Mounting the formatted LUN 185
XII. OFFLOAD BONDING DRIVER 186
1. Introduction 187
1.1. Hardware Requirements 187
1.2. Software Requirements 187
2. Software/Driver Loading 189
3. Software/Driver Unloading 190
4. Software/Driver Configuration and Fine-tuning 191
4.1. Offloading TCP traffic over a bonded interface 191
XIII. OFFLOAD MULTI-ADAPTER FAILOVER (MAFO) 192
1. Introduction 193
1.1. Hardware Requirements 193
1.2. Software Requirements 194

Chelsio T5/T4 Unified Wire for Linux viii
2. Software/Driver Loading 195
3. Software/Driver Unloading 196
4. Software/Driver Configuration and Fine-tuning 197
4.1. Offloading TCP traffic over a bonded interface 197
XIV. UDP SEGMENTATION OFFLOAD AND PACING 198
1. Introduction 199
1.1. Hardware Requirements 199
1.2. Software Requirements 200
2. Software/Driver Loading 201
3. Software/Driver Unloading 202
4. Software/Driver Configuration and Fine-tuning 203
4.1. Modifying the application 203
4.2. Configuring UDP Pacing 205
XV. OFFLOAD IPV6 DRIVER 207
1. Introduction 208
1.1. Hardware Requirements 208
1.2. Software Requirements 208
2. Software/Driver Loading 210
3. Software/Driver Unloading 211
3.1. Unloading the NIC driver 211
3.2. Unloading the TOE driver 211
XVI. BYPASS DRIVER 212
1. Introduction 213
1.1. Features 213
1.2. Hardware Requirements 214
1.3. Software Requirements 215
2. Software/Driver Loading 216
3. Software/Driver Unloading 217
4. Software/Driver Configuration and Fine-tuning 218
4.1. Starting ba server 218
4.2. Bypass API (CLI) 218
XVII. WD SNIFFING AND TRACING 224
1. Theory of Operation 225
1.1. Hardware Requirements 226
1.2. Software Requirements 227
2. Installation and Usage 228

Chelsio T5/T4 Unified Wire for Linux ix
2.1. Installing basic support 228
2.2. Using Sniffer (wd_sniffer) 228
2.3. Using Tracer (wd_tcpdump_trace) 228
XVIII.CLASSIFICATION AND FILTERING 230
1. Introduction 231
1.1. Hardware Requirements 231
1.2. Software Requirements 232
2. Usage 233
2.1. Configuration 233
2.2. Creating Filter Rules 234
2.3. Listing Filter Rules 235
2.4. Removing Filter Rules 235
2.5. Layer 3 example 236
2.6. Layer 2 example 239
3. Hash/DDR Filters 243
3.1. Creating Filter Rules 243
3.2. Listing Filter Rules 244
3.3. Removing Filter Rules 245
3.4. Swap MAC feature 245
3.5. Hit Counters 246
XIX. TRAFFIC MANAGEMENT 248
1. Introduction 249
1.1. Hardware Requirements 249
1.2. Software Requirements 250
2. Software/Driver Loading 251
3. Software/Driver Unloading 252
4. Software/Driver Configuration and Fine-tuning 253
4.1. Traffic Management Rules 253
4.2. Configuring Traffic Management 255
5. Usage 258
5.1. Non-Offloaded Connections 258
5.2. Offloaded Connections 258
5.3. Offloaded Connections with Modified Application 259
XX. UNIFIED WIRE MANAGER (UM) 260
1. Introduction 261
1.1. Features 261
1.2. Reference Architecture 262
1.3. Unified Wire Manager Components 262

Chelsio T5/T4 Unified Wire for Linux x
1.4. Authentication and encryption 263
2. Hardware and Software 264
2.1. Supported Adapters 264
2.2. Platform/Component Matrix 265
2.3. Platform/Driver Matrix 265
3. Installing Unified Wire Manager 266
4. Verifying UM components status 267
4.1. Verifying Management Agent 267
4.2. Verifying Management Client 268
4.3. Verifying Management Station 268
5. Management Agent 269
5.1. Communication 269
5.2. Configuration 269
5.3. Service configuration 269
5.4. Firewall 270
6. CLI client 271
6.1. CLI Help system 271
6.2. Client conflict resolution 271
7. Web GUI client 272
7.1. Management Station 272
7.2. Accessing Web Management Interface 273
7.3. Layout and Navigation 276
7.4. Home page 277
7.5. System page 290
7.6. Network page 302
7.7. Storage Page 331
7.8. Hardware Features 364
8. Uninstalling Unified Wire Manager 380
8.1. Uninstalling Management Agent 380
8.2. Uninstalling Management Client 380
8.3. Uninstalling Management Station 381
XXI. UNIFIED BOOT 382
1. Introduction 383
1.1. Hardware Requirements 383
1.2. Software Requirements 385
2. Flashing firmware and option ROM 386
2.1. Preparing USB flash drive 386
2.2. Legacy 387
2.3. uEFI 390
3. Configuring PXE Server 395

Chelsio T5/T4 Unified Wire for Linux xi
4. PXE boot process 396
4.1. Legacy PXE boot 396
4.2. uEFI PXE Boot 401
5. FCoE boot process 407
5.1. Legacy FCoE boot 407
5.2. uEFI FCoE Boot 416
6. iSCSI boot process 426
6.1. Legacy iSCSI boot 426
6.2. uEFI iSCSI Boot 439
7. Creating Driver Update Disk (DUD) 451
7.1. Creating DUD for RedHat Enterprise Linux 451
7.2. Creating DUD for Suse Enterprise Linux 451
8. OS Installation 453
8.1. Installation using Chelsio NIC DUD (PXE only) 453
8.2. Installation on FCoE LUN 464
8.3. Installation on iSCSI LUN 484
XXII. LUSTRE FILE SYSTEM 501
1. Introduction 502
1.1. Hardware Requirements 502
1.2. Software Requirements 503
2. Creating/Configuring Lustre File System 504
XXIII.APPENDIX A 506
1. Troubleshooting 507
2. Chelsio End-User License Agreement (EULA) 509

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 12
I.Chelsio Unified Wire

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 13
1.Introduction
Thank you for choosing Chelsio T5/T4 Unified Wire adapters. These high speed, single chip,
single firmware cards provide enterprises and data centers with high performance solutions for
various Network and Storage related requirements.
The Terminator 5 (T5) is Chelsio’s next generation of highly integrated, hyper-virtualized
40/10GbE controllers. The T5 is built around a programmable protocol-processing engine, with
full offload of a complete Unified Wire solution comprising NIC, TOE, iWARP RDMA, iSCSI,
FCoE and NAT support. It scales true 40Gb line rate operation from a single TCP connection to
thousands of connections, and allows simultaneous low latency and high bandwidth operation
thanks to multiple physical channels through the ASIC.
The T4 adapters can fully offload TCP, UDP, iSCSI, iWARP and FCoE over a single Unified
Wire. The adapters also fully support SR-IOV, EVB/VNTag, DCB, Traffic Management and
Filtering.
Ideal for all data, storage and high performance clustering applications, the T5/T4 Adapters
enable a unified fabric over a single wire by simultaneously running all unmodified IP sockets,
Fibre Channel and InfiniBand applications over Ethernet at line rate.
Designed for deployment in virtualized data centers, cloud service installations and high
performance computing environments, Chelsio T5/T4 adapters bring a new level of performance
metrics and functional capabilities to the computer networking industry.
Chelsio Unified Wire software comes in two formats: Source code and RPM package forms.
Installing from source requires compiling the package to generate the necessary binaries. You
can choose this method when you are using a custom-built kernel. You can also install the
package using the interactive GUI installer. In other cases, download the RPM package specific
to your operating system and follow the steps mentioned to install the package. Please note that
the OFED software required to install Chelsio iWARP driver comes bundled in both source as
well as RPM packages.
This document describes the installation, use and maintenance of the software and its
various components.
1.1. Features
The Chelsio Unified Wire Package uses a single command to install various drivers and utilities.
It consists of the following software:
Network (NIC/TOE)
Virtual Function Network (vNIC)
iWARP (RDMA)

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 14
RDMA Block Device driver (RBD)
WD-UDP *
iSCSI PDU Offload Target
iSCSI PDU Offload Initiator
Data Center Bridiging (DCB) *
FCoE PDU Offload Target *
FCoE full offload Initiator *
Offload Bonding driver
Offload Multi-Adapter Failover(MAFO)
UDP Segmentation Offload and Pacing
Offload IPv6 driver
Bypass driver
Classification and Filtering feature
Traffic Management feature (TM)
Unified Wire Manager (UM)
Unified Boot Software^
Lustre File System
Utility Tools (cop, cxgbtool, t4_perftune, benchmark tools, sniffer & tracer)
libs (iWARP and WD-UDP libraries)
For detailed instructions on loading, unloading and configuring the drivers/tools please refer to
their respective sections.
* This product is not supported with the current release. Support will be added in upcoming
releases.
^ FCoE and iSCSI Boot are not supported with the current release. Support will be added
in upcoming releases.
1.2. Hardware Requirements
The Chelsio Unified Wire software supports Chelsio T5 and T4 Series of Unified Wire Adapters.
To know more about the list of adapters supported by each driver, please refer to their
respective sections.
1.3. Software Requirements
The Chelsio Unified Wire software has been developed to run on 64-bit Linux based platforms
and therefore it is a base requirement for running the driver. To know more about the complete
list of operating systems supported by each driver, please refer to their respective sections.

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 15
1.4. Package Contents
1.4.1. Source Package
The Chelsio Unified Wire source package consists of the following files/directories:
debrules: This directory contains packaging specification files required for building Debian
packages.
docs: This directory contains support documents - README, Release Notes and User’s
Guide (this document) for the software.
libs: This directory is for libraries required to install the WD-UDP and iWARP drivers. The
libibverbs library has implementation of RDMA verbs which will be used by iWARP
applications for data transfers. The librdmacm library works as an RDMA connection
manager. The libcxgb4 library works as an interface between the above mentioned generic
libraries and Chelsio iWARP driver. The libcxgb4_sock library is a LD_PRELOAD-able
library that accelerates UDP Socket communications transparently and without
recompilation of the user application.
lustre: This directory contains patch needed to be applied to the lustre kernel.
OFED: This directory contains supported OFED packages.
RPM-Manager: This directory contains support scripts used for cluster deployment.
scripts: Support scripts used by the Unified Wire Installer.
specs: The packaging specification files required for building RPM packages.
src: Source code for different drivers.
support: This directory contains source files for the dialog utility.
tools:
autoconf-2.63: This directory contains the source for autoconf v2.63 needed for WD-
UDP and iWARP libraries.
ba_server: Management and configuration tools for bypass adapters.
benchmarks: This directory contains various benchmarking tools to measure
throughput and latency of various networks.
chelsio_adapter_config: This directory contains scripts and binaries needed to
configure Chelsio 40G Adapters.
cop: The cop tool compiles offload policies into a simple program form that can be
loaded into the kernel and interpreted. These offload policies are used to determine
the settings to be used for various connections. The connections to which the
settings are applied are based on matching filter specifications. Please find more
details on this tool in its manual page (run man cop command).
cudbg:Chelsio Unified Debug tool which facilitates collection and viewing of various
debug entities like register dump, Devlog, CIM LA, etc.
cxgbtool: The cxgbtool queries or sets various aspects of Chelsio network interface
cards. It complements standard tools used to configure network settings and
provides functionality not available through such tools. Please find more details on
this tool in its manual page (run man cxgbtool command).
To use cxbtool for FCoE Initiator driver, use [root@host~]# cxgbtool stor -h
Note

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 16
rdma_tools: This directory contains iWARP benchmarking tools.
t4_sniffer: This directory contains sniffer tracing and filtering libraries. See WD
Sniffing and Tracing chapter for more information.
um: This directory contains Unified Wire Manager RPMs for different distributions
and Management Station configuration files.
90-rdma.rules: This file contains udev rules needed for running RDMA applications
as a non-root user.
chdebug: This script collects operating system environment details and debug
information which can be sent to the support team, to troubleshoot Chelsio
hardware/software related issues.
chiscsi_set_affinity.sh: This shell script is used for mapping iSCSI Worker threads
to different CPUs.
chsetup: The chsetup tool loads NIC, TOE and iWARP drivers, and creates WD-
UDP configuration file.
chstatus: This utility provides status information on any Chelsio NIC in the system.
t4_latencytune.sh: Script used for latency tuning of Chelsio Adapters.
t4_perftune.sh: This shell script is to tune the system for higher performance. It
achieves it through modifying the IRQ-CPU binding. This script can also be used to
change Tx coalescing settings.
t4-forward.sh: RFC2544 Forward test tuning script.
uname_r: This file is used by chstatus script to verify if the Linux platform is
supported or not.
wdload: UDP acceleration tool.
wdunload: Used to unload all the loaded Chelsio drivers.
install.py, dialog.py: Python scripts needed for the GUI installer.
EULA: Chelsio’s End User License Agreement
install.log: File containing installation summary.
Makefile: The Makefile for building and installing from the source.
sample_machinefile: Sample file used during iWARP installation on cluster nodes.
Uboot: There are two sub-directories in the Uboot directory: OptionROM and LinuxDUD.
The OptionROM directory contains Unified Boot Option ROM image (cubt4.bin), uEFI driver
(ChelsioUD.efi), default boot configuration file (bootcfg) and a legacy flash utility (cfut4.exe),
which can be used to flash the option ROM onto Chelsio's T5 and T4 based Converged
Network Adapters (CNAs).
The LinuxDUD directory contains image (.img) files required to update drivers for Linux
distributions.
1.4.2. RPM package
The Chelsio Unified Wire RPM package consists of the following:
config: This directory contains T5/T4 firmware configuration files.

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 17
docs: This directory contains support documents i.e. README, Release Notes and User’s
Guide (this document) for the software.
DRIVER-RPMS: RPM packages of Chelsio drivers.
OFED-RPMS: OFED RPM packages required to install iWARP driver.
scripts: Support scripts used by the Unified Wire Installer.
EULA: Chelsio’s End User License Agreement.
install.py: Python script that installs the RPM package. See Software/Driver Installation
section for more information.
uninstall.py: Python script that uninstalls the RPM package. See Software/Driver
Uninstallation section for more information.
Uboot: There are two sub-directories in the Uboot directory: OptionROM and LinuxDUD.
The OptionROM directory contains Unified Boot Option ROM image (cubt4.bin), uEFI driver
(ChelsioUD.efi), default boot configuration file (bootcfg) and a legacy flash utility (cfut4.exe),
which can be used to flash the option ROM onto Chelsio's T5 and T4 based Converged
Network Adapters (CNAs).
The LinuxDUD directory contains image (.img) files required to update drivers for Linux
distributions.

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 18
2.Hardware Installation
Follow these steps to install Chelsio Adapter in your system:
1. Shutdown/power off your system.
2. Power off all remaining peripherals attached to your system.
3. Unpack the Chelsio adapter and place it on an anti-static surface.
4. Remove the system case cover according to the system manufacturer’s instructions.
5. Remove the PCI filler plate from the slot where you will install the Ethernet adapter.
6. For maximum performance, it is highly recommended to install the adapter into a PCIe
x8/x16 slot.
7. Holding the Chelsio adapter by the edges, align the edge connector with the PCI connector
on the motherboard. Apply even pressure on both edges until the card is firmly seated. It
may be necessary to remove the SFP (transceiver) modules prior to inserting the adapter.
8. Secure the Chelsio adapter with a screw, or other securing mechanism, as described by the
system manufacturer’s instructions. Replace the case cover.
9. After securing the card, ensure that the card is still fully seated in the PCIE x8 slot as
sometimes the process of securing the card causes the card to become unseated.
10. Connect a fiber cable, multi-mode for short range (SR) optics or single-mode for long range
(LR) optics, to the 40/10Gb Ethernet adapter or regular Ethernet cable for the 1Gb Ethernet
adapter.
11. Power on your system.
12. Run update-pciids command to download the current version of PCI ID list
[root@host~]# update-pciids
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 198k 100 198k 0 0 491k 0 --:--:-- --:--:-- --:--:-- 626k
Done.
13. Verify if the adapter was installed successfully by using the lspci command
a. For T5 adapters :
[root@host~]# lspci |grep -i Chelsio
07:00.0 Ethernet controller: Chelsio Communications Inc T520-LL-CR Unified
Wire Ethernet Controller
07:00.1 Ethernet controller: Chelsio Communications Inc T520-LL-CR Unified
Wire Ethernet Controller
07:00.2 Ethernet controller: Chelsio Communications Inc T520-LL-CR Unified
Wire Ethernet Controller
07:00.3 Ethernet controller: Chelsio Communications Inc T520-LL-CR Unified
Wire Ethernet Controller
07:00.4 Ethernet controller: Chelsio Communications Inc T520-LL-CR Unified

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 19
Wire Ethernet Controller
07:00.5 SCSI storage controller: Chelsio Communications Inc T520-LL-CR
Unified Wire Storage Controller
07:00.6 Fibre Channel: Chelsio Communications Inc T520-LL-CR Unified Wire
Storage Controller
b. And for T4 adapters :
[root@host~]# lspci | grep –i Chelsio
03:00.0 Ethernet controller: Chelsio Communications Inc T420-CR Unified Wire
Ethernet Controller
03:00.1 Ethernet controller: Chelsio Communications Inc T420-CR Unified Wire
Ethernet Controller
03:00.2 Ethernet controller: Chelsio Communications Inc T420-CR Unified Wire
Ethernet Controller
03:00.3 Ethernet controller: Chelsio Communications Inc T420-CR Unified Wire
Ethernet Controller
03:00.4 Ethernet controller: Chelsio Communications Inc T420-CR Unified Wire
Ethernet Controller
03:00.5 SCSI storage controller: Chelsio Communications Inc T420-CR Unified
Wire Storage Controller
03:00.6 Fibre Channel: Chelsio Communications Inc T420-CR Unified Wire
Storage Controller
03:00.7 Ethernet controller: Chelsio Communications Inc Device 0000
For Chelsio T5/T4 adapters, the physical functions are currently assigned as:
Physical functions 0 - 3: for the SR-IOV functions of the adapter
Physical function 4: for all NIC functions of the adapter
Physical function 5: for iSCSI
Physical function 6: for FCoE
Physical function 7: Currently not assigned
Once Unified Wire package is installed and loaded, examine the output of dmesg to see if the
card is discovered.
For T5 adapters:
eth2: Chelsio T520-LL rev 1 1000/10GBASE-SFP RNIC MSI-X, Offload capable
0000:07:00.4: S/N: RE12130097, P/N: 11011675004

Chapter I. Chelsio Unified Wire
Chelsio T5/T4 Unified Wire for Linux Page 20
And, for T4 adapters:
eth0: Chelsio T420-CR rev 2 1000/10GBASE-SFP RNIC MSI-X, Offload capable
0000:04:00.4: S/N: PT18111226, P/N: 110112140D0
The above outputs indicate the hardware configuration of the adapters as well as the Serial
numbers.
Network device names for Chelsio’s physical ports are assigned using the following
convention: the port farthest from the motherboard will appear as the first network
interface. However, for T5 40G and T420-BT adapters, the association of physical
Ethernet ports and their corresponding network device names is opposite. For
these adapters, the port nearest to the motherboard will appear as the first network
interface.
Note