Cisco SE-CL-L3 User manual

Cisco Application Services Engine Hardware Installation Guide
First Published: 2019-10-15
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

©2019 Cisco Systems, Inc. All rights reserved.

CONTENTS
Overview 1
CHAPTER 1
Overview 1
External Features 1
Serviceable Component Locations 4
Summary of Server Features 6
Installing the Server 9
CHAPTER 2
Preparing for Installation 9
Installation Warnings and Guidelines 9
Rack Requirements 11
Installing the Server in a Rack 12
Installing the Cable Management Arm (Optional) 14
Reversing the Cable Management Arm (Optional) 15
Initial Server Setup 16
Connecting to the Server Locally For Setup 17
Connecting to the Server Remotely For Setup 18
Updating the BIOS and Cisco IMC Firmware 19
Accessing the System BIOS 19
Smart Access Serial 20
Maintaining the Server 21
CHAPTER 3
Status LEDs and Buttons 21
Front-Panel LEDs 21
Rear-Panel LEDs 24
Internal Diagnostic LEDs 25
Preparing For Component Installation 26
Cisco Application Services Engine Hardware Installation Guide
iii

Required Equipment For Service Procedures 26
Shutting Down and Removing Power From the Server 27
Shutting Down Using the Power Button 27
Shutting Down Using The Cisco IMC GUI 27
Shutting Down Using The Cisco IMC CLI 28
Removing the Server Top Cover 28
Serial Number Location 29
Hot Swap vs Hot Plug 29
Removing and Replacing Components 30
Serviceable Component Locations 30
Replacing SAS/SATA Hard Drives or Solid State Drives 32
SAS/SATA Drive Population Guidelines 32
4K Sector Format SAS/SATA Drives Considerations 33
Replacing a SAS/SATA Drive 33
Replacing a Front-Loading NVMe SSD 34
Front-Loading NVMe SSD Population Guidelines 34
Front-Loading NVME SSD Requirements and Restrictions 35
Enabling Hot-Plug Support in the System BIOS 35
Replacing a Front-Loading NVMe SSD 36
Installing a PCIe Cable For Front-Loading NVMe SSDs 37
Replacing Fan Modules 38
Replacing CPUs and Heatsinks 39
CPU Configuration Rules 39
Tools Required For CPU Replacement 39
Replacing a CPU and Heatsink 40
Moving an M5 Generation CPU 46
Replacing Memory DIMMs 51
DIMM Population Rules and Memory Performance Guidelines 51
Replacing DIMMs 53
Replacing a Mini-Storage Module 53
Replacing a Mini-Storage Module Carrier 54
Replacing an M.2 SSD in a Mini-Storage Carrier For M.2 55
Replacing an Internal USB Drive 56
Replacing a USB Drive 56
Cisco Application Services Engine Hardware Installation Guide
iv
Contents

Enabling or Disabling the Internal USB Port 57
Replacing the RTC Battery 58
Replacing Power Supplies 59
Replacing AC Power Supplies 59
Replacing a PCIe Card 60
PCIe Slot Specifications 60
Replacing a PCIe Card 61
Cisco Virtual Interface Card (VIC) Considerations 64
Replacing an mLOM Card 65
Replacing an mRAID Riser (Riser 3) 66
Replacing a SAS Storage Controller Card (RAID or HBA) in Riser 3 68
Storage Controller Card Firmware Compatibility 68
Replacing a SAS Storage Controller Card (RAID or HBA) 69
Replacing a Boot-Optimized M.2 RAID Controller Module 70
Cisco Boot-Optimized M.2 RAID Controller Considerations 70
Replacing a Cisco Boot-Optimized M.2 RAID Controller 71
Replacing the Supercap (RAID Backup) 73
Replacing a SATA Interposer Card 75
Replacing a Chassis Intrusion Switch 76
Installing a Trusted Platform Module (TPM) 77
TPM Considerations 77
Service Headers and Jumpers 78
Using the Clear CMOS Header (J38, Pins 9 - 10) 79
Using the BIOS Recovery Header (J38, Pins 11 - 12) 80
Procedure 1: Reboot With recovery.cap File 80
Procedure 2: Use BIOS Recovery Header and bios.cap Recovery File 81
Using the Clear Password Header (J38, Pins 13 - 14) 81
Using the Boot Alternate Cisco IMC Image Header (J39, Pins 1 - 2) 82
Using the Reset Cisco IMC Password to Default Header (J39, Pins 3 - 4) 83
Using the Reset Cisco IMC to Defaults Header (J39, Pins 5 - 6) 84
Server Specifications 85
APPENDIX A
Server Specifications 85
Physical Specifications 85
Cisco Application Services Engine Hardware Installation Guide
v
Contents

Environmental Specifications 85
Power Specifications 86
1050 W AC Power Supply 86
Power Cord Specifications 87
Storage Controller Considerations 91
APPENDIX B
Supported Storage Controllers and Cables 91
Storage Controller Card Firmware Compatibility 92
RAID Backup (Supercap) 92
Write-Cache Policy for Cisco 12G SAS Modular RAID Controller 92
Mixing Drive Types in RAID Groups 93
RAID Controller Migration 93
Storage Controller and Backplane Connectors 94
Embedded SATA RAID Controller 96
Embbeded SATA RAID Requirements 97
Embedded SATA RAID Controller Considerations 97
Embedded SATA RAID: Two SATA Controllers 98
Enabling SATA Mode For the Embedded Controllers 98
Accessing the Software RAID Configuration Utility 99
For More RAID Utility Information 99
Cisco Application Services Engine Hardware Installation Guide
vi
Contents

CHAPTER 1
Overview
•Overview, on page 1
•External Features, on page 1
•Serviceable Component Locations, on page 4
•Summary of Server Features, on page 6
Overview
Cisco Application Services Engine provides a common platform for deploying Cisco Data Center applications.
These applications provide real time analytics, visibility and assurance for policy and infrastructure.
The Cisco Application Services Engine server is required for installing and hosting the Cisco Application
Services Engine application.
The server is orderable in the following version:
• SE-CL-L3 — Small form-factor (SFF) drives, with 10-drive backplane. Supports up to 10 2.5-inch
SAS/SATA drives. Drive bays 1 and 2 support NVMe SSDs.
The Cisco Application Services Engine software is pre-installed on the server. It is detected after the server
is connected, installed and powered on. Cisco Application Services Engine app allows you to fully enable the
capabilities of the Cisco Application Services Engine.
Cisco Application Services Engine app can be downloaded from the Cisco DC App Center. See the Cisco
Application Services Engine for more information on downloading and using the app.
Note
External Features
This topic shows the external features of the server versions.
Cisco SE-CL-L3 (SFF Drives) Front Panel Features
The following figure shows the front panel features of the small form-factor drive versions of the server.
For definitions of LED states, see Front-Panel LEDs, on page 21.
Cisco Application Services Engine Hardware Installation Guide
1

Figure 1: Cisco SE-CL-L3 (SFF Drives) Front Panel
Fan status LED7Drive bays 1 – 10 support SAS/SATA hard disk
drives (HDDs) and solid state drives (SSDs)
1
Network link activity LED8• SE-CL-L3 : Drive bays 1 and 2 support
NVMe PCIe SSDs.
2
Temperature status LED9Power button/power status LED3
Pull-out asset tag10Unit identification button/LED4
KVM connector
(used with KVM cable that provides one DB-15
VGA, one DB-9 serial, and two USB connectors)
11System status LED5
-Power supply status LED6
SE-CL-L3 (LFF Drives) Front Panel Features
The following figure shows the front panel features of the large form-factor drive version of the server.
For definitions of LED states, see Front-Panel LEDs, on page 21.
Cisco Application Services Engine Hardware Installation Guide
2
Overview
External Features

Figure 2: SE-CL-L3 (LFF Drives) Front Panel
Temperature status LED7Drive bays 1 – 4 support SAS/SATA HDDs and
SSDs
1
Power supply status LED8Drive bays 1 and 2 support NVMe PCIe SSDs.
A size-converter drive sled is required if 2.5-inch
SSDs are used.
2
Network link activity LED9Power button/power status LED3
KVM connector
(used with KVM cable that provides one DB-15
VGA, one DB-9 serial, and two USB connectors)
10Unit identification button/LED4
Pull-out asset tag11System health LED5
-Fan status LED6
SE-CL-L3 Rear Panel Features
The rear panel features are the same for all versions of the server.
For definitions of LED states, see Rear-Panel LEDs, on page 24.
Cisco Application Services Engine Hardware Installation Guide
3
Overview
External Features

Figure 3: SE-CL-L3 Rear Panel
Rear unit identification button/LED7Modular LAN-on-motherboard (mLOM) card bay
(x16 PCIe lane)
1
Power supplies (two, redundant as 1+1)8USB 3.0 ports (two)2
PCIe riser 2/slot 2 (x16 lane)
Includes PCIe cable connectors for front-loading
NVMe SSDs (x8 lane)
9Dual 1-Gb/10-Gb Ethernet ports (LAN1 and
LAN2)
The dual LAN ports can support 1 Gbps and 10
Gbps, depending on the link partner capability.
3
PCIe riser 1/slot 1 (x16 lane)10VGA video port (DB-15 connector)4
Threaded holes for dual-hole grounding lug111-Gb Ethernet dedicated management port5
-Serial port (RJ-45 connector)6
Serviceable Component Locations
This topic shows the locations of the field-replaceable components and service-related items. The view in the
following figure shows the server with the top cover removed.
Cisco Application Services Engine Hardware Installation Guide
4
Overview
Serviceable Component Locations

Figure 4: SE-CL-L3 , Serviceable Component Locations
Power supplies (hot-swappable when redundant as
1+1)
10Front-loading drive bays 1–10 support SAS/SATA
drives.
• SE-CL-L3 : Drive bays 1 and 2 support
NVMe PCIe SSDs.
1
Trusted platform module (TPM) socket on
motherboard (not visible in this view)
11Cooling fan modules (seven, hot-swappable)2
PCIe riser 2/slot 2 (half-height, x16 lane)
Includes PCIe cable connectors for front-loading
NVMe SSDs (x8 lane)
12Supercap unit mounting bracket (RAID backup)3
PCIe riser 1/slot 1 (full-height, x16 lane)
Includes socket for Micro-SD card
13DIMM sockets on motherboard (12 per CPU)4
Modular LOM (mLOM) card bay on chassis floor
(x16 PCIe lane), not visible in this view
14CPUs and heatsinks (up to two)5
Cisco Application Services Engine Hardware Installation Guide
5
Overview
Serviceable Component Locations

Modular RAID (mRAID) riser, can optionally be a
riser that supports either:
• Hardware RAID controller card
• Interposer card for embedded SATA RAID
15Mini-storage module socket. Options:
• SD card module with two SD card slots
• M.2 module with slots for either two SATA
M.2 drives or two NVMe M.2 drives
• Cisco Boot-Optimized M.2 RAID Controller
(module with two slots for SATA M.2 drives,
plus an integrated SATA RAID controller that
can control the two M.2 drives in a RAID 1
array)
6
PCIe cable connectors for front-loading NVMe SSDs
on PCIe riser 2
16Chassis intrusion switch (optional)7
Micro-SD card socket on PCIe riser 117Internal USB 3.0 port on motherboard8
-RTC battery, vertical socket9
Summary of Server Features
The following table lists a summary of server features.
DescriptionFeature
One rack-unit (1RU) chassisChassis
Up to two CPUs from the Intel Xeon Processor Scalable Family.
This includes CPUs from the following series:
• Intel Xeon Silver 4XXX Processors
Central Processor
24 DDR4 DIMM sockets on the motherboard (12 each CPU)Memory
Multi-bit error protection is supportedMulti-bit error protection
BMC, running Cisco Integrated Management Controller (Cisco
IMC) firmware.
Depending on your Cisco IMC settings, Cisco IMC can be
accessed through the 1-Gb dedicated management port, the
1-Gb/10-Gb Ethernet LAN ports, or a Cisco virtual interface card.
Baseboard management
Cisco Application Services Engine Hardware Installation Guide
6
Overview
Summary of Server Features

DescriptionFeature
Rear panel:
• One 1-Gb Ethernet dedicated management port (RJ-45
connector)
• Two 1-Gb/10-Gb BASE-T Ethernet LAN ports (RJ-45
connectors)
The dual LAN ports can support 1 Gbps and 10 Gbps,
depending on the link partner capability.
• One RS-232 serial port (RJ-45 connector)
• One VGA video connector port (DB-15 connector)
• Two USB 3.0 ports
Front panel:
• One front-panel keyboard/video/mouse (KVM) connector
that is used with the KVM cable, which provides two USB
2.0, one VGA, and one DB-9 serial connector.
Network and management I/O
One dedicated socket (x16 PCIe lane) that can be used to add an
mLOM card for additional rear-panel connectivity.
Modular LOM
One power supply:
• AC power supplies 1050 W AC each
Power
The advanced configuration and power interface (ACPI) 4.0
standard is supported.
ACPI
Seven hot-swappable fan modules for front-to-rear cooling.Cooling
Two horizontal PCIe expansion slots on a PCIe riser assembly.
See PCIe Slot Specifications, on page 60 for specifications of the
slots.
PCIe I/O
The PCIe bus slots in this server support the InfiniBand
architecture.
InfiniBand
The server is orderable in three different versions, each with a
different front panel/drive-backplane configuration.
• SE-CL-L3 , Small form-factor (SFF) drives, with 10-drive
backplane. Supports up to 10 2.5-inch SAS/SATA drives.
Drive bays 1 and 2 support NVMe SSDs.
Storage, front-panel
Cisco Application Services Engine Hardware Installation Guide
7
Overview
Summary of Server Features

DescriptionFeature
The server has these internal storage options:
• One USB port on the motherboard.
• One micro-SD card socket on PCIe riser 1.
• Mini-storage module socket, optionally with either:
• SD card module. Supports up to two SD cards.
•M.2 SSD module. Supports either two SATA M.2 SSDs
or two NVMe M.2 SSDs.
• Cisco Boot-Optimized M.2 RAID Controller (module
with two slots for SATA M.2 drives, plus an integrated
SATA RAID controller that can control the two SATA
M.2 drives in a RAID 1 array)
Storage, internal
The server has a dedicated internal mRAID riser that supports
one of the following storage-controller options:
• A PCIe-style Cisco modular RAID controller card
(SAS/SATA).
• A PCIe-style interposer card for the server’s embedded
SATA RAID controller.
For a detailed list of storage controller options, see Supported
Storage Controllers and Cables, on page 91.
Storage management
The server has a mounting bracket near the cooling fans for the
supercap unit that is used with the Cisco modular RAID controller
card.
RAID backup
Integrated VGA video.Integrated video
Cisco Application Services Engine Hardware Installation Guide
8
Overview
Summary of Server Features

CHAPTER 2
Installing the Server
•Preparing for Installation, on page 9
•Installing the Server in a Rack, on page 12
•Installing the Cable Management Arm (Optional), on page 14
•Reversing the Cable Management Arm (Optional), on page 15
•Initial Server Setup, on page 16
•Connecting to the Server Locally For Setup, on page 17
•Connecting to the Server Remotely For Setup, on page 18
•Updating the BIOS and Cisco IMC Firmware, on page 19
•Accessing the System BIOS, on page 19
•Smart Access Serial, on page 20
Preparing for Installation
This section contains the following topics:
Installation Warnings and Guidelines
Before you install, operate, or service a server, review the Regulatory Compliance and Safety Information for
important safety information.
Note
IMPORTANT SAFETY INSTRUCTIONS
This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work
on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard
practices for preventing accidents. Use the statement number provided at the end of each warning to locate
its translation in the translated safety warnings that accompanied this device.
Statement 1071
Warning
Cisco Application Services Engine Hardware Installation Guide
9

To prevent the system from overheating, do not operate it in an area that exceeds the maximum recommended
ambient temperature of: 35° C (95° F).
Statement 1047
Warning
The plug-socket combination must be accessible at all times, because it serves as the main disconnecting
device.
Statement 1019
Warning
This product relies on the building’s installation for short-circuit (overcurrent) protection. Ensure that the
protective device is rated not greater than: 250 V, 15 A.
Statement 1005
Warning
Installation of the equipment must comply with local and national electrical codes.
Statement 1074
Warning
This unit is intended for installation in restricted access areas. A restricted access area can be accessed only
through the use of a special tool, lock, and key, or other means of security.
Statement 1017
Warning
To ensure proper airflow it is necessary to rack the servers using rail kits. Physically placing the units on top
of one another or “stacking” without the use of the rail kits blocks the air vents on top of the servers, which
could result in overheating, higher fan speeds, and higher power consumption. We recommend that you mount
your servers on rail kits when you are installing them into the rack because these rails provide the minimal
spacing required between the servers. No additional spacing between the servers is required when you mount
the units using rail kits.
Caution
Avoid uninterruptible power supply (UPS) types that use ferroresonant technology. These UPS types can
become unstable with systems such as the Cisco UCS, which can have substantial current draw fluctuations
from fluctuating data traffic patterns.
Caution
When you are installing a server, use the following guidelines:
• Plan your site configuration and prepare the site before installing the server.
Cisco Application Services Engine Hardware Installation Guide
10
Installing the Server
Installation Warnings and Guidelines

• Ensure that there is adequate space around the server to allow for accessing the server and for adequate
airflow. The airflow in this server is from front to back.
•Ensure that the air-conditioning meets the thermal requirements listed in the Environmental Specifications,
on page 85.
• Ensure that the cabinet or rack meets the requirements listed in the Rack Requirements, on page 11.
• Ensure that the site power meets the power requirements listed in the Power Specifications, on page 86.
If available, you can use an uninterruptible power supply (UPS) to protect against power failures.
Rack Requirements
The rack must be of the following type:
• A standard 19-in. (48.3-cm) wide, four-post EIA rack, with mounting posts that conform to English
universal hole spacing, per section 1 of ANSI/EIA-310-D-1992.
•The rack-post holes can be square 0.38-inch (9.6 mm), round 0.28-inch (7.1 mm), #12-24 UNC, or #10-32
UNC when you use the Cisco-supplied slide rails.
• The minimum vertical rack space per server must be one rack unit (RU), equal to 1.75 in. (44.45 mm).
Supported Cisco Slide Rail Kits
The server supports the following rail kit:
• Cisco part UCSC-RAILB-M4= (ball-bearing slide rail kit)
Rack Installation Tools Required
The slide rails sold by Cisco Systems for this server do not require tools for installation.
Slide Rail and Cable Management Arm Dimensions
The slide rails for this server have an adjustment range of 24 to 36 inches (610 to 914 mm).
The optional cable management arm (CMA) adds additional length requirements:
• The additional distance from the rear of the server to the rear of the CMA is 5.4 inches (137.4 mm).
• The total length of the server including the CMA is 35.2 inches (894 mm).
Cisco Application Services Engine Hardware Installation Guide
11
Installing the Server
Rack Requirements

Installing the Server in a Rack
To prevent bodily injury when mounting or servicing this unit in a rack, you must take special precautions to
ensure that the system remains stable. The following guidelines are provided to ensure your safety:
This unit should be mounted at the bottom of the rack if it is the only unit in the rack.
When mounting this unit in a partially filled rack, load the rack from the bottom to the top with the heaviest
component at the bottom of the rack.
If the rack is provided with stabilizing devices, install the stabilizers before mounting or servicing the unit in
the rack.
Statement 1006
Warning
Step 1 Attach the inner rails to the sides of the server:
a) Align an inner rail with one side of the server so that the three keyed slots in the rail align with the three pegs on the
side of the server.
b) Set the keyed slots over the pegs, and then slide the rail toward the front to lock it in place on the pegs. The front slot
has a metal clip that locks over the front peg.
c) Install the second inner rail to the opposite side of the server.
Figure 5: Attaching the Inner Rail to the Side of the Server
Locking clip on front of inner rail2Front of server1
Step 2 Open the front securing plate on both slide-rail assemblies. The front end of the slide-rail assembly has a spring-loaded
securing plate that must be open before you can insert the mounting pegs into the rack-post holes.
On the outside of the assembly, push the green-arrow button toward the rear to open the securing plate.
Cisco Application Services Engine Hardware Installation Guide
12
Installing the Server
Installing the Server in a Rack

Figure 6: Front Securing Mechanism, Inside of Front End
Securing plate shown pulled back to the open
position
3Front mounting pegs1
-Rack post between mounting pegs and opened
securing plate
2
Step 3 Install the outer slide rails into the rack:
a) Align one slide-rail assembly front end with the front rack-post holes that you want to use.
The slide rail front-end wraps around the outside of the rack post and the mounting pegs enter the rack-post holes
from the outside-front.
The rack post must be between the mounting pegs and the open securing plate.
Note
b) Push the mounting pegs into the rack-post holes from the outside-front.
c) Press the securing plate release button, marked PUSH. The spring-loaded securing plate closes to lock the pegs in
place.
d) Adjust the slide-rail length, and then push the rear mounting pegs into the corresponding rear rack-post holes. The
slide rail must be level front-to-rear.
The rear mounting pegs enter the rear rack-post holes from the inside of the rack post.
e) Attach the second slide-rail assembly to the opposite side of the rack. Ensure that the two slide-rail assemblies are at
the same height and are level front-to-back.
f) Pull the inner slide rails on each assembly out toward the rack front until they hit the internal stops and lock in place.
Step 4 Insert the server into the slide rails:
This server can weigh up to 60 pounds (27 kilograms) when fully loaded with components. We recommend
that you use a minimum of two people or a mechanical lift when lifting the server. Attempting this procedure
alone could result in personal injury or equipment damage.
Caution
a) Align the rear ends of the inner rails that are attached to the server sides with the front ends of the empty slide rails
on the rack.
b) Push the inner rails into the slide rails on the rack until they stop at the internal stops.
c) Slide the inner-rail release clip toward the rear on both inner rails, and then continue pushing the server into the rack
until its front slam-latches engage with the rack posts.
Cisco Application Services Engine Hardware Installation Guide
13
Installing the Server
Installing the Server in a Rack

Figure 7: Inner-Rail Release Clip
Outer slide rail attached to rack post3Inner-rail release clip1
-Inner rail attached to server and inserted into
outer slide rail
2
Step 5 (Optional) Secure the server in the rack more permanently by using the two screws that are provided with the slide rails.
Perform this step if you plan to move the rack with servers installed.
With the server fully pushed into the slide rails, open a hinged slam latch lever on the front of the server and insert a
screw through the hole that is under the lever. The screw threads into the static part of the rail on the rack post and prevents
the server from being pulled out. Repeat for the opposite slam latch.
Installing the Cable Management Arm (Optional)
The cable management arm (CMA) is reversible left-to-right. To reverse the CMA, see Reversing the Cable
Management Arm (Optional), on page 15 before installation.
Note
Step 1 With the server pushed fully into the rack, slide the CMA tab of the CMA arm that is farthest from the server onto the
end of the stationary slide rail that is attached to the rack post. Slide the tab over the end of the rail until it clicks and
locks.
Cisco Application Services Engine Hardware Installation Guide
14
Installing the Server
Installing the Cable Management Arm (Optional)