Cisco Nexus 9000 Series User manual

Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director
Configuration Guide, Release 9.x
First Published: 2015-07-02
Last Modified: 2018-11-05
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of
the UNIX operating system. All rights reserved. Copyright ©1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
This product includes cryptographic software written by Eric Young ([email protected]).
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (https://www.openssl.org/)
This product includes software written by Tim Hudson ([email protected]).
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:
https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (1110R)
©2018 Cisco Systems, Inc. All rights reserved.

CONTENTS
Preface vii
PREFACE
Audience vii
Document Conventions vii
Related Documentation for Cisco Nexus 9000 Series Switches viii
Documentation Feedback viii
Obtaining Documentation and Submitting a Service Request viii
New and Changed Information 1
CHAPTER 1
New and Changed Information 1
Configuring ITD 3
CHAPTER 2
About ITD 3
Deployment Modes 5
One-Arm Deployment Mode 5
One-Arm Deployment Mode with vPC 5
Sandwich Deployment Mode 6
Server Load-Balancing Deployment Mode 7
Device Groups 8
Multiple Device Groups in an ITD Service 8
VRF Support 9
Router ACLs 9
Include and Exclude ACLs 10
Virtual IP Address Filtering 10
Port Number-Based Filtering 10
Hot-Standby 10
Multiple Ingress Interfaces 11
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
iii

System Health Monitoring 11
Health of an Interface Connected to a Node 11
Peer Synchronization 11
Failaction Reassignment 12
No Failaction Reassignment 12
No Failaction Reassignment with a Probe Configured 12
No Failaction Reassignment without a Probe Configured 12
Licensing Requirements for ITD 12
Guidelines and Limitations for ITD 13
ITD Support Summary 14
Default Settings for ITD 15
Configuring ITD 16
Enabling ITD 16
Configuring a Device Group 16
Configuring an ITD Service 18
Assigning an ACL to an ITD Service 21
Nondisruptively Adding or Deleting Nodes 23
Nondisruptively Adding or Deleting ACEs in Include or Exclude ACLs 25
Verifying the ITD Configuration 25
Configuration Examples for ITD 28
Configuration Example: One-Arm Deployment Mode 43
Configuration Example: One-Arm Deployment Mode with vPC 44
Configuration Example: Sandwich Deployment Mode 46
Configuration Example: Server Load-Balancing Deployment Mode 47
Configuration Example: ITD as WCCP Replacement (Web-Proxy Deployment Mode) 48
Configuration Example: Peer Synchronization for Sandwich Mode 50
Configuration Example: Firewall on a Stick 52
ITD Services 52
ASA VLANs 52
Flow Symmetry 53
Link Failures 54
Configuration Example 55
Configuration Example: Firewall in Dual-Switch Sandwich Mode with vPCs 59
Configuration Example: Firewall in Layer 3 Clustering 61
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
iv
Contents

Related Documents 65
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
v
Contents

Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
vi
Contents

Preface
This preface includes the following sections:
•Audience, on page vii
•Document Conventions, on page vii
•Related Documentation for Cisco Nexus 9000 Series Switches, on page viii
•Documentation Feedback, on page viii
•Obtaining Documentation and Submitting a Service Request, on page viii
Audience
This publication is for network administrators who install, configure, and maintain Cisco Nexus switches.
Document Conventions
Command descriptions use the following conventions:
DescriptionConvention
Bold text indicates the commands and keywords that you enter literally
as shown.
bold
Italic text indicates arguments for which the user supplies the values.
Italic
Square brackets enclose an optional element (keyword or argument).[x]
Square brackets enclosing keywords or arguments separated by a vertical
bar indicate an optional choice.
[x | y]
Braces enclosing keywords or arguments separated by a vertical bar
indicate a required choice.
{x | y}
Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.
[x {y | z}]
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
vii

DescriptionConvention
Indicates a variable for which you supply values, in context where italics
cannot be used.
variable
A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.
string
Examples use the following conventions:
DescriptionConvention
Terminal sessions and information the switch displays are in screen font.screen font
Information you must enter is in boldface screen font.boldface screen font
Arguments for which you supply values are in italic screen font.
italic screen font
Nonprinting characters, such as passwords, are in angle brackets.< >
Default responses to system prompts are in square brackets.[ ]
An exclamation point (!) or a pound sign (#) at the beginning of a line
of code indicates a comment line.
!, #
Related Documentation for Cisco Nexus 9000 Series Switches
The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL:
http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to [email protected]. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service
request, and gathering additional information, see What's New in Cisco Product Documentation at:
http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.
Subscribe to What's New in Cisco Product Documentation, which lists all new and revised Cisco technical
documentation as an RSS feed and delivers content directly to your desktop using a reader application. The
RSS feeds are a free service.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
viii
Preface
Related Documentation for Cisco Nexus 9000 Series Switches

CHAPTER 1
New and Changed Information
This chapter provides release-specific information for each new and changed feature in the Cisco Nexus 9000
Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x.
•New and Changed Information, on page 1
New and Changed Information
This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS Intelligent Traffic
Director Configuration Guide, Release 9.x and tells you where they are documented.
Where DocumentedChanged in
Release
DescriptionFeature
Configuring ITD, on page
3
9.2 (1)Added support for the Cisco Nexus C9364C,
C9336C-FX2,C93240YC-FX2 switches.
ITD
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
1

Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
2
New and Changed Information
New and Changed Information

CHAPTER 2
Configuring ITD
This chapter describes how to configure the Intelligent Traffic Director (ITD) on the Cisco NX-OS device.
•About ITD, on page 3
•Licensing Requirements for ITD, on page 12
•Guidelines and Limitations for ITD, on page 13
•ITD Support Summary, on page 14
•Default Settings for ITD, on page 15
•Configuring ITD, on page 16
•Verifying the ITD Configuration, on page 25
•Configuration Examples for ITD, on page 28
•Related Documents, on page 65
About ITD
Intelligent Traffic Director (ITD) is an intelligent, hardware-based, multi-terabit solution that allows you to
build a scalable architecture for Layer 3 and Layer 4 traffic distribution, load balancing, and redirection.
Benefits of ITD:
• Multi-terabit solution at line rate
• Transparency to end device and stateless protocol benefits
• Reduced complexities and architecture scaling for alternative features like Web Cache Communication
Protocol (WCCP) and policy-based routing
• Simplified provisioning and ease of deployment
• Legacy service appliances can co-exist with new ones
• Removes the requirement for an expensive external load balancer
• No certification, integration, or qualification needed between the devices and the Cisco NX-OS switch
• Order of magnitude OPEX savings : reduction in configuration, and ease of deployment
• CAPEX savings : No service module or external L3/L4 load-balancer needed. Every Nexus port can be
used as load-balancer
ITD features:
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
3

1. Hardware based multi-terabit/s L3/L4 load-balancing at wire-speed
2. Zero latency load-balancing
3. Redirect line-rate traffic to any devices, for example web cache engines, Web Accelerator Engines
(WAE), video-caches, etc
4. Capability to create clusters of devices, for example, Firewalls, Intrusion Prevention System (IPS), or
Web Application Firewall (WAF), Hadoop cluster
5. IP-stickiness
6. Resilient (like resilient ECMP), Consistent hash
7. Virtual IP based L4 load-balancing
8. Weighted load-balancing and Failaction are supported among nodes
9. Load-balances to large number of devices/servers
10. ACL along with redirection and load balancing simultaneously
11. Bi-directional flow-coherency. Traffic from A–>B and B–>A goes to same node
12. The servers/appliances don’t have to be directly connected to Nexus switch
13. Monitoring the health of servers/appliances with IP SLA-based probes
14. N + M redundancy (N number of nodes and M number of hot-standbys)
15. Automatic failure handling of servers/appliances
16. VRF support, vPC support
17. Supports both IPv4 and IPv6 (all platforms do not support IPv6)
18. The feature does not add any load to the supervisor CPU
19. Handles unlimited number of flows
20. Nondisruptive node addition or deletion
21. Simultaneous redirection and load balancing
22. Rate sharing across multiple ITD services in the same switch
Use case examples:
• Load-balance to cluster of Firewalls.
• Scale IPS, IDS and WAF by load-balancing to standalone devices
• Scale the NFV solution by load-balancing to low cost VM/container based NFV
• Scale the WAAS / WAE solution. Traffic redirection mechanism for the Wide Area Application Services
(WAAS) or Web Accelerator Engine (WAE) solution
• Scale the VDS-TC (video-caching) solution
• Scale Layer-7 load-balancers, by distributing traffic to L7 LBs
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
4
Configuring ITD
About ITD

• Replaces ECMP or the port channel to avoid rehashing . ITD is resilient, and doesn’t cause re-hashing
on node add/delete/failure
• Server load balancing in DSR (Direct Server Return) mode
•Scales up NG intrusion prevention systems (IPSs) and web application firewalls (WAFs) by load balancing
to standalone devices
• Load balances to Layer 5 through Layer 7 load balancers
Deployment Modes
One-Arm Deployment Mode
You can connect servers to the switch in one-arm deployment mode. In this topology, the server is not in the
direct path of client or server traffic, which enables you to plug a server into the network with no changes to
the existing topology or network.
Figure 1: One-Arm Deployment Mode
One-Arm Deployment Mode with vPC
ITD supports an appliance pool connected to a virtual port channel (vPC). The ITD service runs on each
switch, and ITD programs each switch to provide flow-coherent traffic passing through the nodes.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
5
Configuring ITD
Deployment Modes

Figure 2: One-Arm Deployment Mode with vPC
Sandwich Deployment Mode
The sandwich deployment mode uses two switches to provide stateful handling of traffic.
The main requirement in this mode is that both the forward and reverse traffic of a flow must go through the
same appliance. Examples include firewalls and load balancer deployments, where traffic between the client
and the server must flow through the same appliance.
The key features are:
•An ITD service for each network segment, one for the outside network and another for the inside network.
• A source IP address load-balancing scheme where the ITD service operates on the interface that connects
to the outside world in an ingress direction.
• A destination IP address load-balancing scheme where the ITD service operates on the interface that
connects to the servers in the ingress direction.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
6
Configuring ITD
Sandwich Deployment Mode

Figure 3: Sandwich Deployment Mode
Server Load-Balancing Deployment Mode
The ITD service can be configured to host a virtual IP (VIP) on the switch. Internet traffic destined for the
VIP will be load balanced to the active nodes. The ITD service is not a stateful load balancer.
You need to configure the ITD service manually and in a similar manner on each switch.
Note
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
7
Configuring ITD
Server Load-Balancing Deployment Mode

Figure 4: ITD Load Distribution with VIP
Device Groups
Nodes can be a physical server, virtual server, or a service appliance where traffic can be load balanced. These
nodes are grouped together under a device group, and this device group can be mapped to a service.
ITD supports device groups. When you configure a device group, you can specify the following:
• The device group's nodes
• The device group's probe
You can configure probes at the device-group level or at the node level. With node-level probing, each node
can be configured with its own probe, allowing for further customization per node. Node-level probes are
useful in scenarios where each node needs to be monitored differently for failure conditions.
Multiple Device Groups in an ITD Service
Beginning with Cisco NX-OS Release 7.0(3)I3(1), multiple device groups are supported in an ITD service
(as shown in the figure below). An ITD service generates a single route map with different sequences that
point to different device groups.
Each device group represents different types of traffic requiring different services but arriving on the same
ingress interface. Traffic on the interface is redirected to the appropriate device group based on the virtual IP
address. Supporting multiple device groups per ITD service on the same interface allows ITD to scale.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
8
Configuring ITD
Device Groups

Figure 5: Multiple Device Groups in an ITD Service
For a configuration example showing how to configure multiple device groups in an ITD service, see
Configuration Examples for ITD, on page 28.
For the number of device groups supported, see the Cisco Nexus 9000 Series NX-OS Verified Scalability
Guide for your release.
VRF Support
The ITD service can be configured in the default VRF as well as in non-default VRFs.
Ingress interfaces and device-group nodes must all belong to the same VRF for the ITD service to redirect
traffic. You must ensure that all ingress interfaces and node members of the associated device group are
reachable in the configured VRF.
Router ACLs
The switch supports router access control lists (RACLs) with ITD.
You can configure ITD and an RACL on the same ingress interface. The resulting RACL, which is downloaded
to the TCAM, is a cross product of the ACL generated by ITD and the user-configured RACL. The permit
and deny statements configured on the RACL are combined with the ACL permits and redirect entries created
by ITD. This functionality helps you to filter and load distribute selected traffic.
ITD statistics do not function if you configure an RACL on an ITD ingress interface.
Note
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
9
Configuring ITD
VRF Support

Include and Exclude ACLs
The include ACL feature allows you to assign an access control list (ACL) to an ITD service. For each access
control entry (ACE) with the permit method in the ACL, this feature filters the unwanted traffic and generates
IP access lists and a route map to load-balance the permitted traffic.
You can configure an exclude ACL to specify the traffic that you want ITD to exclude from the ITD load
balancer. Traffic selected by the exclude ACL is RIB routed and bypasses ITD. An exclude ACL can filter
based on both source and destination fields. The exclude ACL precedes the virtual IP address.
Virtual IP Address Filtering
A virtual IP address can be used to filter traffic for ITD. A virtual IP address and subnet mask combination
for traffic filtering is supported for the destination field only.
Port Number-Based Filtering
Port numbering can be used to filter traffic for ITD. The following methods are supported to filter traffic based
on Layer 4 ports (for example, port 80):
• Matching destination ports
Any source or destination IP address with destination port 80 is matched. (For example: The virtual IP
address is configured as 0.0.0.0 0.0.0.0 tcp 80.)
• Matching source ports
Any port other than 80 bypasses ITD, and port 80 is redirected. (For example: The exclude ACL is
configured as permit tcp any neq 80 any.)
• Matching multiple port numbers
Multiple virtual IP address lines in ITD can be configured, one for each port.
Hot-Standby
The hot-standby feature reconfigures the switch to look for an operational hot-standby node and select the
first available hot-standby node to replace the failed node. ITD reconfigures the switch to redirect the traffic
segment that was originally headed toward the failed node to the hot-standby node. The service does not
impose any fixed mapping of hot-standby nodes to active nodes.
When the failed node becomes operational again, it is reinstated as an active node. The traffic from the acting
hot-standby node is redirected back to the original node, and the hot-standby node reverts to the pool of
standby nodes.
When multiple nodes fail, traffic destined to all failed nodes gets redirected to the first available hot-standby
node.
The hot-standby node can be configured only at the node level . At the node level, the hot-standby node
receives traffic only if its associated active node fails.
ITD supports N + M redundancy where M nodes can act as hot-standby nodes for N active nodes.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
10
Configuring ITD
Include and Exclude ACLs

Multiple Ingress Interfaces
You can configure the ITD service to apply traffic redirection policies on multiple ingress interfaces. This
feature allows you to use a single ITD service to redirect traffic arriving on different interfaces to a group of
nodes.
Beginning with Cisco NX-OS Release 7.0(3)I7(3), the same ingress interface can be included in two ITD
services, allowing one IPv4 ITD service and one IPv6 ITD service.
Including the same ingress interface in both IPv4 and IPv6 ITD services allows both IPv4 and IPv6 traffic to
arrive on the same ingress interface. An IPv4 ITD policy is applied to redirect the IPv4 traffic, and an IPv6
ITD policy is applied to redirect the IPv6 traffic.
Make sure that the same ingress interface is not referenced in more than one IPv4 ITD service or more than
one IPv6 ITD service. The system does not automatically enforce it and it is not supported.
Note
System Health Monitoring
ITD monitors health of the nodes and applications running on those nodes periodically to detect any failures
and to handle the failure scenarios.
ICMP, TCP, UDP, DNS and HTTP probes are supported.
Health of an Interface Connected to a Node
Beginning with Cisco NX-OS Release 7.0(3)I3(1), ITD leverages the IP service level agreement (IP SLA)
feature to periodically probe each node. In earlier releases, ITD uses the Internet Control Message Protocol
(ICMP) to periodically probe each node. The probes are sent at a 10-second frequency by default and can be
configured down to 1 second. They are sent simultaneously to all nodes. You can configure the probe as part
of the pool group configuration.
A probe is declared to have failed after retrying three times. At this point, the node state becomes “Failed,”
and its status becomes “PROBE_FAILED.”
Node Failure Handling
Upon marking a node as down, the ITD performs the following tasks automatically to minimize traffic
disruption and to redistribute the traffic to remaining operational nodes:
• Determines if a standby node is configured to take over from the failed node.
• If the standby node is operational, it is identified the node as a candidate node for traffic handling.
• Redefines the standby node as active for traffic handling, if an operational standby node is available
• Programs automatically to reassign traffic from the failed node to the newly active standby node.
Peer Synchronization
The peer synchronization feature synchronizes the node health status across two ITD peer services in sandwich
mode. It is useful in preventing traffic loss if a link on one of the ITD peer services goes down.
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
11
Configuring ITD
Multiple Ingress Interfaces

Each ITD service probes its peer service periodically to detect any failure. A ping is sent every second to the
ITD peer service. If a reply is not received, it is retried three times. The frequency and retry count are not
configurable.
Failaction Reassignment
Failaction for ITD enables traffic on the failed nodes to be reassigned to the first available active node. Once
the failed node comes back, it automatically resumesserving the connections. The failactioncommand enables
this feature.
When the node is down, the traffic bucket associated with the node is reassigned to the first active node found
in the configured set of nodes. If the newly reassigned node also fails, traffic isreassigned to the next available
active node. Once the failed node becomes active again, traffic is diverted back to the new node and resumes
serving connections
If all the nodes are down, the packets are routed automatically.
You must configure a probe under an ITD device group before enabling the failaction feature.
Note
No Failaction Reassignment
When failaction node reassignment is not configured, there are two possible scenarios:
No Failaction Reassignment with a Probe Configured
The ITD probe can detect the node failure or the lack of service reachability. If the node fails, the traffic is
routed and does not get reassigned, as failaction is not configured. Once the node recovers, the recovered
node starts to handle the traffic.
No Failaction Reassignment without a Probe Configured
Without a probe configuration, ITD cannot detect the node failure. When the node is down, ITD does not
reassign or redirect the traffic to an active node.
Licensing Requirements for ITD
License RequirementProduct
ITD requires a Network Services license. For a
complete explanation of the Cisco NX-OS licensing
scheme and how to obtain and apply licenses, see the
Cisco NX-OS Licensing Guide.
Cisco NX-OS
Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide, Release 9.x
12
Configuring ITD
Failaction Reassignment