Cavium FastLinQ 45000 Series User manual

BC0154501-00 P
User’s Guide
Converged Network Adapters and
Intelligent Ethernet Adapters
FastLinQ 45000 Series

ii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Document Revision History
Revision A, April 19, 2016
Revision B, April 26, 2016
Revision C, May 11, 2016
Revision D, August 29, 2016
Revision E, August 30, 2016
Revision F, September 30, 2016
Revision G, March 10, 2017
Revision H, June 26, 2017
Revision J, November 21, 2017
Revision K, February 7, 2018
Revision L, May 22, 2018
Revision M, June 18, 2018
Revision N, August 24, 2018
Revision P, January 17, 2019
Changes Sections Affected
Updated the Cavium logos, preface content, and
copyright information.
Front cover, Preface, and back page
Removed references to Windows Nano Server. “System Requirements” on page 7 (Table 2-2)
Chapter 3 Driver Installation
Chapter 4 Firmware Upgrade Utility
Chapter 14 Windows Server 2016
Before Table 3-4, added “Cavium has certified
qedrntv for ESXi 6.5 and ESXi 6.7, which are bun-
dled along with qedentv as one package in vCG
listings.”
“VMware Drivers and Driver Packages” on
page 24
Updated the Linux firmware update procedure. “Upgrading Adapter Firmware on Linux” on
page 31
Updated the Windows firmware update procedure. “Upgrading Adapter Firmware on Windows” on
page 31
Added L2B Firmware Version as a read-only item
on the Firmware Information page.
“Displaying Firmware Information” on page 38
For device-level configuration, added the Port
Mode parameter to the bullet list.
“Configuring Device-level Parameters” on page 38

iii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
For port-level configuration:
Updated Figure 5-7.
In Step 4, updated the RoCE Priority descrip-
tion.
In Step 10, added iWARP+RoCE as a new
option.
“Configuring Port-level Parameters” on page 41
Added a footnote referencing the QConvergeCon-
sole Help system for procedures for changing the
adapter port mode in QConvergeConsole GUI.
“Changing the Adapter Port Mode” on page 53
Moved iSCSI boot from SAN information from the
iSCSI Configuration chapter to a new chapter.
Chapter 6 Boot from SAN Configuration
Moved FCoE boot from SAN information from the
FCoE Configuration chapter to a new chapter.
Chapter 6 Boot from SAN Configuration
In the VMware bullet, expanded the information to
explain the limitations of iSCSI boot in the ESXi
OS.
“Supported Operating Systems for iSCSI Boot” on
page 60
Following Step 1 of the To install SLES 12 SP3
and later procedure, added this note: “Although
they are not necessarily required for iSCSI boot
from SAN for SUSE, Cavium recommends that
you also complete Step 2 through Step 15 of the
Configuring iSCSI Boot from SAN for RHEL 7.5
and Later procedure”
“Configuring iSCSI Boot from SAN for SLES 12
SP3 and Later” on page 91
In Step 4 of the To install RHEL 7.4 and later pro-
cedure, changed the command to inst.dd
modprobe.blacklist=qed,qede,qedr,qedi
,qedf.
“Configuring FCoE Boot from SAN for RHEL 7.4
and Later” on page 120
In Table 7-1 for OFED:
Removed support for RHEL 7.4, SLES 12 SP2.
Added support for RHEL 7.6, SLES 12 SP4.
Replaced CentOS 7.4 with CentOS 7.6.
Changed Ubuntu 16.04 LTS support for RoCE
v2 to include “(4.5 + kernel)”.
“Supported Operating Systems and OFED” on
page 127
In Table 7-2, replaced the RDMA Mode property
with the NetworkDirect Technology prop-
erty.
“Configuring RoCE on the Adapter for Windows
Server” on page 135

iv BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Updated the following iWARP procedures:
In Step 1 c of the To enable iWARP on the
Windows host and verify RDMA procedure,
changed the parameter to NetworkDirect
Technology (was RDMA Mode).
In Step 4 b of the To verify iWARP traffic pro-
cedure, changed the command to netstat
-xan (was net -xan).
“Configuring iWARP on Windows” on page 169
In the To configure the NFS server procedure:
Added a new Step 1 that describes how to
create an nfs-server directory and grant
permission.
Replaced Step 4 for loading the service.
“Configuring NFS-RDMA” on page 175
In the To configure the NFS client procedure:
Added a new Step 1 that describes how to
create an nfs-client directory and grant
permission.
In Step 3, updated the command used for NFS
Version 4.
“Configuring NFS-RDMA” on page 175
Following Step 12, added a fourth bullet to the
note: “Switch dependent teaming (IEEE 802.3ad
LACP and Generic/Static Link Aggregation (Trunk-
ing) cannot use a switch independent partitioned
virtual adapter. IEEE standards require Switch
Dependent Teaming (IEEE 802.3ad LACP and
Generic/Static Link Aggregation (Trunking)) mode
to work per the entire port instead of just the MAC
address (fraction of a port) granularity.”
“Configuring Microsoft Initiator to Use Cavium’s
iSCSI Offload” on page 197
Added new section addressing an issue with iSCSI
offload on RHEL 7.6.
“” on page 208
In the note, added the following: “For NDKPI
Mode-2, you must enable Network Direct Function-
ality of the physical interface.”
“Configuring RoCE Interfaces with Hyper-V” on
page 241

v BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
In the To create a Hyper-V virtual switch with an
RDMA NIC procedure:
Changed the section title to “...RDMA NIC” (was
“...RDMA Virtual NIC”.
Changed procedure title (was To create a
Hyper-V virtual switch with an RDMA virtual
NIC),
Added a new Step 1: “On all physical interfaces,
set the value of the NetworkDirect Functional-
ity parameter to Enabled.”
“Creating a Hyper-V Virtual Switch with an RDMA
NIC” on page 242
In the To add host virtual NICs procedure:
In Step 2, corrected the last digit of the com-
mand to “S” (was “5”).
Changed the note’s first bullet, second sen-
tence to: “The same VLAN ID must be assigned
to ports on the switch” (was: “The same VLAN
ID must be assigned to all the interfaces, and
on the switch”).
“Adding a vLAN ID to Host Virtual NIC” on
page 243
Added terms and definitions. Glossary

vi BC0154501-00 P
Table of Contents
Preface
Supported Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
What Is in This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Technical Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Downloading Updates and Documentation . . . . . . . . . . . . . . . . . . . . . xxiii
Knowledgebase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Laser Safety—FDA Notice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv
Agency Certification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
EMI and EMC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
KCC: Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
VCCI: Class A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
Product Safety Compliance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
1 Product Overview
Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Adapter Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
QLogic Control Suite CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
QLogic QConvergeConsole GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
QLogic QConvergeConsole vCenter Plug-in. . . . . . . . . . . . . . . . . . . . 4
QConvergeConsole PowerKit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
FastLinQ ESXCLI VMware Plug-in . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Adapter Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Physical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Standards Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Hardware Installation
System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Safety Precautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Preinstallation Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Installing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

vii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
3 Driver Installation
Installing Linux Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Installing the Linux Drivers Without RDMA . . . . . . . . . . . . . . . . . . . . . 13
Removing the Linux Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Installing Linux Drivers Using the src RPM Package . . . . . . . . . 15
Installing Linux Drivers Using the kmp/kmod RPM Package . . . 16
Installing Ubuntu Linux Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . 16
Installing Linux Drivers Using the TAR File. . . . . . . . . . . . . . . . . 16
Installing the Linux Drivers with RDMA . . . . . . . . . . . . . . . . . . . . . . . . 17
Linux Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Linux Driver Operation Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Linux Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Importing a Public Key for Secure Boot. . . . . . . . . . . . . . . . . . . . . . . . 19
Installing Windows Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Installing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Removing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Managing Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Setting Power Management Options. . . . . . . . . . . . . . . . . . . . . . . . . . 23
Installing VMware Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
VMware Drivers and Driver Packages. . . . . . . . . . . . . . . . . . . . . . . . . 24
Installing VMware Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
VMware NIC Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . 26
VMware Driver Parameter Defaults. . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Removing the VMware Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
FCoE Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
iSCSI Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Firmware Upgrade Utility
Image Verification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Upgrading Adapter Firmware on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Upgrading Adapter Firmware on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . 31
Converting a 100G Adapter to Four-port 25G Adapter. . . . . . . . . . . . . . . . . 32
5 Adapter Preboot Configuration
Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Displaying Firmware Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Configuring Device-level Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Configuring Port-level Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Configuring FCoE Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

viii BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Configuring Partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Changing the Adapter Port Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Converting a 100G Adapter to Four-port 25G Adapter . . . . . . . . . . . . 53
Converting a 25G Adapter to Single-port 100G Adapter . . . . . . . . . . . 54
Converting a 40G Adapter to a Four-port 10G Adapter. . . . . . . . . . . . 55
Converting a 10G Adapter to a Four-port 40G Adapter. . . . . . . . . . . . 56
6 Boot from SAN Configuration
iSCSI Boot from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
iSCSI Out-of-Box and Inbox Support. . . . . . . . . . . . . . . . . . . . . . . . . . 59
iSCSI Preboot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Supported Operating Systems for iSCSI Boot . . . . . . . . . . . . . . 60
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 61
Selecting the iSCSI UEFI Boot Protocol. . . . . . . . . . . . . . . . . . . 64
Configuring the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Configuring iSCSI Boot Parameters . . . . . . . . . . . . . . . . . . . . . . 65
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 65
Configuring iSCSI Boot Options . . . . . . . . . . . . . . . . . . . . . . . . . 67
Configuring the DHCP Server to Support iSCSI Boot . . . . . . . . 77
Configuring iSCSI Boot from SAN on Windows . . . . . . . . . . . . . . . . . 81
Before You Begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Selecting the Preferred iSCSI Boot Mode . . . . . . . . . . . . . . . . . 82
Configuring iSCSI General Parameters . . . . . . . . . . . . . . . . . . . 82
Configuring the iSCSI Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Configuring the iSCSI Targets . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Detecting the iSCSI LUN and Injecting the QLogic Drivers . . . .84
Configuring iSCSI Boot from SAN on Linux . . . . . . . . . . . . . . . . . . . . 86
Configuring iSCSI Boot from SAN for RHEL 7.4 . . . . . . . . . . . . 86
Configuring iSCSI Boot from SAN for RHEL 7.5 and Later . . . . 90
Configuring iSCSI Boot from SAN for SLES 12 SP3
and Later . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Configuring iSCSI Boot from SAN for Other
Linux Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Configuring iSCSI Boot from SAN on VMware . . . . . . . . . . . . . . . . . . 107
FCoE Boot from SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
FCoE Out-of-Box and Inbox Support. . . . . . . . . . . . . . . . . . . . . . . . . . 107
FCoE Preboot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Specifying the BIOS Boot Protocol. . . . . . . . . . . . . . . . . . . . . . . 108
Configuring Adapter UEFI Boot Mode . . . . . . . . . . . . . . . . . . . . 109

ix BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring FCoE Boot from SAN on Windows . . . . . . . . . . . . . . . . . 114
Windows Server 2012 R2 and 2016 FCoE Boot Installation . . . 114
Configuring FCoE on Windows . . . . . . . . . . . . . . . . . . . . . . . . . 115
FCoE Crash Dump on Windows. . . . . . . . . . . . . . . . . . . . . . . . . 115
Injecting (Slipstreaming) Adapter Drivers into Windows
Image Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Configuring FCoE Boot from SAN on Linux . . . . . . . . . . . . . . . . . . . . 117
Prerequisites for Linux FCoE Boot from SAN. . . . . . . . . . . . . . . 117
Configuring Linux FCoE Boot from SAN . . . . . . . . . . . . . . . . . . 118
Configuring FCoE Boot from SAN on VMware . . . . . . . . . . . . . . . . . . 123
Injecting (Slipstreaming) ESXi Adapter Drivers into
Image Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Installing the Customized ESXi ISO . . . . . . . . . . . . . . . . . . . . . . 124
7 RoCE Configuration
Supported Operating Systems and OFED . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Planning for RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Preparing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Preparing the Ethernet Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Configuring the Cisco Nexus 6000 Ethernet Switch . . . . . . . . . . . . . . 129
Configuring the Dell Z9100 Ethernet Switch . . . . . . . . . . . . . . . . . . . . 131
Configuring the Arista 7060X Ethernet Switch. . . . . . . . . . . . . . . . . . . 133
Configuring RoCE on the Adapter for Windows Server . . . . . . . . . . . . . . . . 135
Viewing RDMA Counters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Configuring RoCE on the Adapter for Linux . . . . . . . . . . . . . . . . . . . . . . . . . 143
RoCE Configuration for RHEL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
RoCE Configuration for SLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
RoCE Configuration for Ubuntu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Verifying the RoCE Configuration on Linux . . . . . . . . . . . . . . . . . . . . . 147
vLAN Interfaces and GID Index Values. . . . . . . . . . . . . . . . . . . . . . . . 150
RoCE v2 Configuration for Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Identifying the RoCE v2 GID Index or Address . . . . . . . . . . . . . 151
Verifying the RoCE v1 or RoCE v2 GID Index and Address
from sys and class Parameters . . . . . . . . . . . . . . . . . . . . . . . . 151
Verifying the RoCE v1 or RoCE v2 Function Through
perftest Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Configuring RoCE on the Adapter for VMware ESX . . . . . . . . . . . . . . . . . . 156
Configuring RDMA Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Configuring MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
RoCE Mode and Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Configuring a Paravirtual RDMA Device (PVRDMA). . . . . . . . . . . . . . 159

x BC0154501-00 P
User’s Guide—Converged Network Adapters and Intelligent Ethernet Adapters
FastLinQ 45000 Series
Configuring DCQCN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
DCQCN Terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
DCQCN Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
DCB-related Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Global Settings on RDMA Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Setting vLAN Priority on RDMA Traffic . . . . . . . . . . . . . . . . . . . . 164
Setting ECN on RDMA Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Setting DSCP on RDMA Traffic . . . . . . . . . . . . . . . . . . . . . . . . . 165
Configuring DSCP-PFC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Enabling DCQCN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Configuring CNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
DCQCN Algorithm Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
MAC Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Script Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
8 iWARP Configuration
Preparing the Adapter for iWARP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Configuring iWARP on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Configuring iWARP on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Installing the Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Configuring iWARP and RoCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Detecting the Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Supported iWARP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Running Perftest for iWARP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Configuring NFS-RDMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
iWARP RDMA-Core Support on SLES 12 SP3 and RHEL 7.4 . . . . . . 177
9 iSER Configuration
Before You Begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Configuring iSER for RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Configuring iSER for SLES 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Configuring iSER for Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Configuring LIO as a Target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Configuring the Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Optimizing Linux Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Configuring CPUs to Maximum Performance Mode . . . . . . . . . . . . . . 191
Configuring Kernel sysctl Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Configuring IRQ Affinity Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Configuring Block Device Staging . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192