Cisco UCS 6400 Series Manual

Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
First Published: 2018-08-14
Last Modified: 2020-02-28
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

©2018– 2020 Cisco Systems, Inc. All rights reserved.

CONTENTS
Product Overview 1
CHAPTER 1
Cisco UCS Fabric Interconnect Overview 1
Cisco UCS 6454 Fabric Interconnect 1
Cisco UCS 64108 Fabric Interconnect 3
Ports on the Cisco UCS Fabric Interconnects 5
Port Speeds and Types 6
Power Supplies 7
Fan Modules 8
Front Panel Ports and LEDs 9
Network Management Port LEDs 11
L1 and L2 Port LEDs 11
Rear Panel System Environment LED 12
Rear Panel Port LEDs 12
Supported Transceivers 12
Installing the Cisco UCS Fabric Interconnect 13
CHAPTER 2
Preparing for Installation 13
Considerations and Warnings 13
Airflow Considerations 14
Fabric Interconnect Weight 14
Installation Guidelines 14
Cabinet and Rack Requirements 15
General Requirements for Cabinets and Racks 16
Requirements Specific to Perforated Cabinets 16
Unpacking and Inspecting the Cisco UCS Fabric Interconnect 16
Required Equipment 17
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
iii

Installing the Cisco UCS FI in a Cabinet or Rack 18
Establishing System Ground 21
Proper Grounding Practices 22
Grounding the Fabric Interconnect 23
Starting the System 24
Connecting the Cisco UCS Fabric Interconnect 27
CHAPTER 3
Preparing for Network Connections 27
Connecting to the Console Port 27
Connecting the Management Port 29
Connecting to an SFP28 Ethernet or Fibre Channel Port 29
Installing or Removing Cables into SFP Transceivers 29
Installing a Transceiver 29
Removing a Transceiver 30
Installing or Removing Cables into SFP28 Trancsceivers 31
Installing a Cable into an SFP28 Transceiver 31
Removing a Cable from a Transceiver 31
Maintaining SFP28 Transceivers and Fiber-Optic Cables 32
Replacing Components 33
CHAPTER 4
Preparing a Fabric Interconnect for Removal 33
Shutting Down a Fabric Interconnect 34
Removing a Cisco UCS Fabric Interconnect From a Rack 37
Repacking the Cisco UCS Fabric Interconnect for Return Shipment 38
Technical Specifications 39
CHAPTER 5
System Specifications 39
Power Specifications 40
Transceiver Specifications 42
Environmental Conditions and Power Requirements Specification for SFP Transceivers 42
General Specifications for Cisco Fibre Channel SFP Transceivers 42
Cable and Port Specifications 45
CHAPTER 6
Accessory Kit for the Cisco UCS Fabric Interconnect 45
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
iv
Contents

Console Cable 46
Console Port 46
Supported AC Power Cords and Plugs 47
Argentina 47
Australia and New Zealand 48
Peoples Republic of China 48
Europe 49
India, South Africa, and United Arab Emirates 49
Israel 50
Italy 50
North America 51
Switzerland 52
Taiwan 52
United Kingdom 53
Cabinet Jumper Power Cords 53
Site Planning and Maintenance Records 57
CHAPTER 7
Site Preparation Checklist 57
Contact and Site Information 59
Chassis and Module Information 60
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
v
Contents

Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
vi
Contents

CHAPTER 1
Product Overview
•Cisco UCS Fabric Interconnect Overview, on page 1
•Cisco UCS 6454 Fabric Interconnect, on page 1
•Cisco UCS 64108 Fabric Interconnect, on page 3
•Ports on the Cisco UCS Fabric Interconnects, on page 5
•Port Speeds and Types, on page 6
•Power Supplies, on page 7
•Fan Modules, on page 8
•Front Panel Ports and LEDs, on page 9
•Network Management Port LEDs, on page 11
•L1 and L2 Port LEDs, on page 11
•Rear Panel System Environment LED , on page 12
•Rear Panel Port LEDs, on page 12
•Supported Transceivers, on page 12
Cisco UCS Fabric Interconnect Overview
The Cisco UCS Fabric Interconnects provide both network connectivity and management capabilities to the
Cisco UCS system. The fabric interconnect provides Ethernet and Fibre Channel to the servers in the system.
The servers connect to the fabric interconnect, and then to the LAN or SAN.
Each fabric interconnect runs Cisco UCS Manager software to fully manage all Cisco UCS elements. High
availability redundancy can be achieved when a fabric interconnect is connected to another fabric interconnect
through the L1 or L2 port on each device.
Cisco UCS 6454 Fabric Interconnect
The Cisco UCS 6454 Fabric Interconnect (FI) is a 1 RU top-of-rack switch that mounts in a standard 19-inch
rack such as the Cisco R Series rack.
The Cisco UCS 6454 Fabric Interconnect has 48 10/25 Gb SFP28 ports (16 unified ports) and 6 40/100 Gb
QSFP28 ports. Each 40/100 Gb port can break out into 4 x 10/25 Gb uplink ports. The sixteen unified ports
support 10/25 GbE or 8/16/32G Fibre Channel speeds.
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
1

The Cisco UCS 6454 Fabric Interconnect supported 8 unified ports (ports 1 - 8) with Cisco UCS Manager
4.0(1) and 4.0(2), but with release 4.0(4) and later it supports 16 unified ports (ports 1 - 16).
Note
The Cisco UCS 6454 Fabric Interconnect supports:
• maximum of 8 FCoE port channels
• or 4 SAN Ports
• or maximum of 8 SAN port channels and FCoE port channels (4 each)
The Cisco UCS 6454 Fabric Interconnect also has one network management port, one console port for setting
the initial configuration, and one USB port for saving or loading configurations. The FI also includes L1/L2
ports for connecting two fabric interconnects for high availability.
The Cisco UCS 6454 Fabric Interconnect also contains a CPU board that consists of:
• Intel Xeon D-1528 v4 Processor, 1.6 GHz
• 64 GB of RAM
• 8 MB of NVRAM (4 x NVRAM chips)
• 128 GB SSD (bootflash)
Figure 1: Cisco UCS 6454 Fabric Interconnect Rear View
Ports 17-44 (10/25 Gbps Ethernet or FCoE)
When using Cisco UCS Manager
releases earlier than 4.0(4), ports
9-44 are 10/25 Gbps Ethernet or
FCoE.
Note
2Ports 1-16 (Unified Ports 10/25 Gbps
Ethernet or FCoE or 8/16/32 Gbps Fibre
Channel)
When using Cisco UCS
Manager releases earlier than
4.0(4), only ports 1-8 are
Unified Ports.
Note
1
Uplink Ports 49-54 (40/100 Gbps Ethernet
or FCoE)
Each of these ports can be 4 x 10/25 Gbps
Ethernet or FCoE uplink ports when using
an appropriate breakout cable.
4Ports 45-48 (1/10/25 Gbps Ethernet or
FCoE)
3
For additional information about the port groups, see Port Speeds and Types, on page 6.
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
2
Product Overview
Cisco UCS 6454 Fabric Interconnect

The Cisco UCS 6454 Fabric Interconnect chassis has two power supplies and four fans. Two of the fans
provide front to rear airflow.
Figure 2: Cisco UCS 6454 Fabric Interconnect Front View
Fans 1 through 4, numbered left to right, when
facing the front of the chassis
2Power supply and power cord connector1
L1 port, L2 port, RJ45, console, USB port,
and LEDs
3
Cisco UCS 64108 Fabric Interconnect
The Cisco UCS 64108 Fabric Interconnect (FI) is a 2-RU top-of-rack switch that mounts in a standard 19-inch
rack such as the Cisco R Series rack. This high-density FI is an ideal upgrade from the high-density Cisco
UCS 6296 Fabric Interconnect.
The high-density Cisco UCS 64108 Fabric Interconnect has 96 10/25 Gb SFP28 ports and 12 40/100 Gb
QSFP28 ports. Each 40/100 Gb port can break out into 4 x 10/25 Gb uplink ports. Sixteen ports (1-16) are
unified ports that support 10/25 GbE or 8/16/32G Fibre Channel speeds.The Cisco UCS 64108 Fabric
Interconnect supports:
• Maximum of 8 FCoE port channels
• Or 4 SAN Ports
• Or a maximum of 8 SAN port channels and FCoE port channels (4 each)
The Cisco UCS 64108 Fabric Interconnect also has one network management port, one RS-232 serial console
port for setting the initial configuration, and one USB port for saving or loading configurations. The FI also
includes L1/L2 ports for connecting two fabric interconnects in a high-availability configuration.
The Cisco UCS 64108 Fabric Interconnect also contains a CPU board that consists of:
• Intel Xeon Processor, 6 core
• 64 GB of RAM
• 8 MB of NVRAM (4 x NVRAM chips)
• 128 GB SSD (bootflash)
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
3
Product Overview
Cisco UCS 64108 Fabric Interconnect

Figure 3: Cisco UCS 64108 Fabric Interconnect Rear View
Ports 17-88
10/25 Gbps Ethernet or FCoE
2Ports 1 - 16
Unified ports:
• 10/25 Gbps Ethernet or FCoE
• 8/16/32 Gbps Fibre Channel
1
Uplink Ports 97-108 (40/100 Gbps Ethernet
or FCoE)
Each of these ports can be 4 x 10/25 Gbps
Ethernet or FCoE uplink ports when using a
breakout cable.
4Ports 89-96
1/10/25 Gbps Ethernet or FCoE
3
System status LED6System environment (fan fault) LED5
Beacon LED7
The Cisco UCS 64108 Fabric Interconnect has two power supplies (redundant as 1+1) and three fans (redundant
as 2+1).
Figure 4: Cisco UCS 64108 Fabric Interconnect Front View
RS-232 serial console port (RJ-45 connector)2Cooling fans
(hot swappable, 2+1 redundancy)
1
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
4
Product Overview
Cisco UCS 64108 Fabric Interconnect

USB port4Network management port (RJ-45 connector)3
Power supplies
Two identical AC or DC PSUs, hot-swappable,
1+1 redundancy)
6Grounding pad for two-hole grounding lug
(under protective label)
5
Beacon LED8L1/L2 high-availabity ports (RJ-45 connector)7
--System status LED9
Ports on the Cisco UCS Fabric Interconnects
The ports on the fabric interconnects can be configured to carry either Ethernet or Fibre Channel traffic. You
can configure only ports 1-16 to carry Fibre Channel traffic. The ports cannot be used by a Cisco UCS domain
until you configure them.
When you configure a port on a Fabric Interconnect, the administrative state is automatically set to enabled.
If the port is connected to another device, this may cause traffic disruption. The port can be disabled and
enabled after it has been configured.
Note
The following table summarizes the Cisco UCS Fabric Interconnects.
Cisco UCS 6454 FI
54-Port Fabric InterconnectDescription
1-RUForm factor
48 10/25G interfacesNumber of fixed 10 GB Interfaces
16
This FI supported 8 unified ports (ports 1 - 8) with
Cisco UCS Manager 4.0(1) and 4.0(2), but with
Release 4.0(4) and later it supports 16 unified ports
(ports 1 - 16).
Number of Unified Ports
Ports 1-16Unified Port Range
10/25 Gbps or 8/16/32-Gbps FCUnified Port Speeds
6 40/100 Gigabit portsNumber of 40-Gbps ports
UCS 2204, UCS 2208, UCS 2408Compatibility with the IOM
Cisco Nexus 2232PP
Cisco Nexus 2232TM-E
Compatibility with the FEX
NoneExpansion Slots
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
5
Product Overview
Ports on the Cisco UCS Fabric Interconnects

Cisco UCS 6454 FI
4Fan Modules
2 (AC/DC/HVDC available)Power Supplies
Port Speeds and Types
Ports on the fabric interconnects are numbered and grouped according to their function. The ports are numbered
top to bottom and left to right. The following figures show the port numbering and define port speeds and the
types of ports that can be configured. For more information on how to configure the port modes, refer to
"Configuring Port Modes for a Fabric Interconnect" in the Cisco UCS Network Management Guide, Release
4.0.
Figure 5: Rear View of Cisco UCS 6454 FI, Port Numbers
Ports 17–44.
Each port can operate as 10G/25G Ethernet.
Port types in 10G/25G mode:
• FCoE uplink port
• Server port
• Appliance port (the FI must be in
Ethernet-End-Host mode)
• Monitor port
When using Cisco UCS
Manager releases earlier than
4.0(4), ports 9-44 are 10/25
Gbps Ethernet or FCoE. (Only
ports 1-8 were Unified Ports in
earlier releases.)
Note
2Ports 1–16.
Unified Ports can operate as 10/25
Gbps Ethernet or FCoE; or 8/16/32
Gbps Fibre Channel.
Port type in 8G/16G/32G FC mode: FC
uplink port
Port types in 10G/25G mode:
• FCoE uplink port
• Server port
• Appliance port (the FI must be in
Ethernet-End-Host mode)
• Monitor port
When using Cisco UCS
Manager releases earlier
than 4.0(4), only ports 1-8
are Unified Ports.
Note
1
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
6
Product Overview
Port Speeds and Types

Uplink Ports 49–54.
Each port can operate as 40G/100G
Ethernet or FCoE. With a breakout cable,
each of these ports can operate as 4 x 10G
or 4 x 25G Ethernet or FCoE ports.
Port types:
• Uplink port
• FCoE uplink port
• Monitor port
4Ports 45–48.
Each port can operate as 1G/10G/25G
Ethernet or FCoE port.
3
Power Supplies
The fabric interconnect has two power supplies that are accessible from the front of the chassis. The two
power supplies support 1+1 redundancy.
Table 1: Power Supply Models
WattageSourceFabric InterconnectCisco PID
650 W AC110 to 240 VACCisco UCS 6454UCS-PSU-6332-AC
(The Cisco UCS 6454 FI
shares the same AC power
supply and ordering PID
with the Cisco UCS 6300
Series.)
1200 W AC90 to 264 VACCisco UCS 64108UCS-PSU-64108-AC
930 W DC-48 VDC• Cisco UCS 6454
• Cisco UCS 64108
UCS-PSU-6332-DC
(The Cisco UCS 6400
Series shares the same DC
power supply and
ordering PID with the
Cisco UCS 6300 Series.)
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
7
Product Overview
Power Supplies

Figure 6: Power Supply LEDs (650 W AC Shown)
Power supplies have two LEDs: one for power status and one for a failure condition.
Power LED2Fault or Error LED1
DescriptionStateLED
Power supply is on and functioning properlySolid greenPower LED
3.3 V voltage standby (VSB) is on but the power supply is not
powering up the Fabric Interconnect
Blinking green
There is no AC power to the power supplyOff
Power supply failure that indicates an over voltage, over current,
or over temperature
Solid amberFault/error LED
AC power is present, 3.3 VSB is on, and the power supply is
off
Blinking amber
Normal operationOff
Fan Modules
The Cisco UCS 6454 FI and the Cisco UCS 64108 FI use different fan modules.
•Cisco UCS 6454 FI: four fan modules. The fans support 3+1 redundancy. All fans are hot swappable
but only one fan can be removed at a time.
•Cisco UCS 64108 FI: three fan modules. The fans support 2+1 redundancy. All fans are hot swappable
but only one fan can be removed at a time.
FanFabric Interconnect
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
8
Product Overview
Fan Modules

UCS-FAN-6332
(The Cisco UCS 6332 FI and Cisco UCS 6454 FI use
the same fan modules and ordering PID.)
Cisco UCS 6454
UCS-FAN-64108Cisco UCS 64108
Front Panel Ports and LEDs
The FIs in the Cisco UCS 6400 Series have slightly different front panel port and LED placement, but the
icons and functions are the same. See the following figures for placement on your FI. Also see the LED
state-definition table below.
When connecting L1 and L2 ports between FIs, the maximum supported length of Ethernet CAT5e or CAT6
cable is 100 meters.
Note
Figure 7: Cisco UCS 6454 FI Front Panel Ports and LEDs
L2 high availability port2L1 high availability port1
RS-232 serial console port (RJ-45 connector)4RJ45 Network Management Port3
Beacon LED6USB port
The USB port can be used for booting or
downloading scripts.
5
--System Status LED7
___
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
9
Product Overview
Front Panel Ports and LEDs

Figure 8: Cisco UCS 64108 FI Front Panel Ports and LEDs
System Status LED2Beacon LED1
RS-232 serial console port (RJ-45 connector)4L1 and L2 high availability ports3
Grounding pad for two-hole grounding lug
(under protective label)
6Network Management Port (RJ-45)5
--USB port
The USB port can be used for booting or
downloading scripts.
7
The definition of states of the beacon and system status LEDs are as follows:
DescriptionStateColorFunctionLocationLED
Chassis
selected
Solid OnBlueIndicate selected
chassis
Front and rearBeacon
LED
Chassis not
selected
Off
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
10
Product Overview
Front Panel Ports and LEDs

DescriptionStateColorFunctionLocationLED
Normal
operation
Solid onGreenIndicate system
power/health at bootup
and run time
Front and rearSystem
Status LED
System
powered off
Off
System
fault
OnAmber
Power shut
down by
software
Solid onRed
Secure boot
validation
has failed
Blinking
Network Management Port LEDs
The states of the LEDs of the management port on the front panel are listed below.
DescriptionLED StateLED Position
No linkOffLeft
Physical linkSolid green
No activity
OffRight
ActivityBlinking green
L1 and L2 Port LEDs
The states of the LEDs on the L1 and L2 front-panel ports are listed below.
DescriptionLED StateLED Position
No linkOffLeft
Physical linkSolid greenLeft
No activityOffRight
ActivityBlinking greenRight
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
11
Product Overview
Network Management Port LEDs

Rear Panel System Environment LED
The states of the LEDs on the system environment LED on the rear panel are listed below. The system
environment LED indicates a fan fault or alarm.
DescriptionLED State
Minor fan alarm (one fan is missing or there is a failure).Solid amber
Major fan alarm (two or more fans are missing or have failed, or there is a fan direction
mismatch).
Solid red
Rear Panel Port LEDs
The states of the LEDs on the rear panel ports are listed below.
DescriptionLED State
Enabled, but SFP not inserted
Administrative down (software shutdown)
Yellow
Enabled and link is upGreen
Enabled, but link is not connectedOff
Power On Self Test (POST) failed
Port beacon enabled
Blinking yellow
Supported Transceivers
For a complete list of the supported transceivers and cables with ordering PIDs, refer to the Cisco UCS Fabric
Interconnect 6454 Data Sheet.
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
12
Product Overview
Rear Panel System Environment LED

CHAPTER 2
Installing the Cisco UCS Fabric Interconnect
•Preparing for Installation, on page 13
•Cabinet and Rack Requirements, on page 15
•Establishing System Ground, on page 21
Preparing for Installation
Considerations and Warnings
Before you install, operate, or service the system, read the Regulatory Compliance and Safety Information
for Cisco UCS for important safety information.
Note
IMPORTANT SAFETY INSTRUCTIONS This warning symbol means danger. You are in a situation that
could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical
circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided
at the end of each warning to locate its translation in the translated safety warnings that accompanied this
device. Statement 1071
Warning
SAVE THESE INSTRUCTIONS
This unit is intended for installation in restricted access areas. A restricted access area can be accessed only
through the use of a special tool, lock and key, or other means of security. Statement 1017
Warning
Only trained and qualified personnel must be allowed to install, replace, or service this equipment.
Statement 1030
Warning
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
13

Each new fabric interconnect requires a license. For information on licensing, see the Configuration Guide
for the version of Cisco UCS Manager that you are using. The configuration guides are available at the
following URL: Cisco UCS Manager Configuration Guides
Note
Airflow Considerations
To ensure proper airflow, follow these guidelines:
• Maintain ambient airflow throughout the data center to ensure normal operation.
• Consider the heat dissipation of all equipment when determining air conditioning requirements. When
evaluating airflow requirements, take into consideration that hot air generated by equipment at the bottom
of the rack can be drawn in the intake ports of the equipment above.
• Ensure that exhaust airflow is unobstructed.
Fabric Interconnect Weight
We recommend that you use two people or a mechanical lift when lifting the system.
• The Cisco UCS 6454 FI weighs 22.24 lbs (10.10 kg)
• The Cisco UCS 64108 FI weighs 35.86 lbs (16.27 kg)
Caution
When lifting the system, follow these guidelines:
• Disconnect all power and external cables before lifting the system.
• Ensure that your footing is solid and that the weight of the system is evenly distributed between your
feet.
• Lift the system slowly, keeping your back straight. Lift with your legs, not with your back. Bend at the
knees, not at the waist.
Installation Guidelines
When installing the Cisco UCS Fabric Interconnect, follow these guidelines:
• Prepare the site as described in the Site Preparation Checklist.
• Plan your site configuration and prepare the site before installing the fabric interconnect. lists the
recommended site planning tasks.
• Record the information listed in Site Preparation Checklist as you install and configure the fabric
interconnect.
• Ensure that there is adequate space around the fabric interconnect to allow for servicing and for adequate
airflow. The Site Preparation Checklist lists airflow requirements.
Cisco UCS 6400 Series Fabric Interconnect Hardware Installation Guide
14
Installing the Cisco UCS Fabric Interconnect
Airflow Considerations