Cisco UCS C890 M5 User manual

Cisco UCS C890 M5 Rack Server Service Guide
First Published: July 19, 2021
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OFANY KIND,EXPRESS
OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITHTHE
PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT
YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public
domain version ofthe UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.CISCO
AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERSHAVE
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document arenot intended to be actual addresses and phone numbers. Any examples, command
display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone
numbers in illustrative content is unintentional and coincidental.
All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online
version for the latest version.Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on
the Cisco website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco
trademarks, go to this URL: https://www.cisco.com/c/en/us/about/legal/trademarks.html. Third-party trademarks mentioned are the property of their
respective owners. The use of the word partner does not imply apartnership relationship between Cisco and any other company. (1721R)
© 2020–2021 Cisco Systems, Inc. All rights reserved.

Contents
Page 3
Contents
Cisco UCS C890 M5 Rack Server Service Guide .......................................1
1Preface...........................................................................................5
2Overview........................................................................................6
2.1Unpacking the System...........................................................................6
2.2 System Specifications............................................................................6
2.3 Server Chassis Features ........................................................................8
2.3.1 Control Panel/CPU Module Indicators..................................8
2.3.2 Front Features..................................................................... 10
2.3.3 Rear Features...................................................................... 11
2.3.4 CPU Board........................................................................... 12
2.3.5 C890-M5-BMC Layout ........................................................ 14
2.3.6 Major Components on the C890-M5-BMC Card ................ 16
2.3.7 C890-M5 PCIE Card............................................................. 18
2.3.8 Location of the C890-M5-PCIEBOARD= Cards................... 18
2.3.9 Major Components on the C890-M5-PCIEBOARD Card..... 20
2.3.10 C890-M5-RISER-A Card ....................................................... 21
2.3.11 C890-M5-RISER-B Card....................................................... 22
2.3.12 C890-M5-HDD-BOARD Card ............................................... 23
2.3.13 C890-M5-BPLANE Midplane............................................... 24
2.3.14 Location of the C890-M5-BPLANE= Midplane................... 29
2.3.15 Major Components on the C890-M5-BPLANE= Midplane . 30
2.4 Major Components on the Rear Side of the C890-M5-BPLANE=
Midplane ............................................................................................ 32
2.5 Major Reverse Side Components of the C890-M5-BPLANE= Midplane
............................................................................................................ 32
2.5.1 C890-M5-BPLANE-D= Midplane ......................................... 33
2.6 Major Front Side Components of the C890-M5-BPLANE-D= Midplane
............................................................................................................ 33
2.6.1 System Block Diagram........................................................ 34
3Server Installation ........................................................................35
3.1 Overview ............................................................................................ 35
3.2 Preparing for Setup ........................................................................... 35
3.3 Choosing a Setup Location................................................................ 35
3.4 Rack Precautions ................................................................................ 35
3.5 Server Precautions ............................................................................. 36
3.6 Rack Mounting Considerations .......................................................... 36
3.6.1 Ambient Operating Temperature....................................... 36
3.7 Installing the System into a Rack ....................................................... 38
3.7.1 Rack Mounting Hardware ................................................... 38
4Maintenance and Component Installation ....................................41
4.1 Removing Power................................................................................ 41
4.2 Accessing the System ........................................................................ 41
4.3 CPU Modules...................................................................................... 42

Contents
Page 4
4.3.1 Removing/Installing a CPU Module................................... 42
Memory Support and Installation..................................................... 44
4.3.2 ESD Precautions.................................................................. 44
4.3.3 Precautions ......................................................................... 44
4.3.4 Introduction to Intel® Optane DC Persistent Memory....... 44
4.3.5 Memory Support................................................................. 44
4.3.6 DIMM Population Guidelines for Optimal Performance.... 48
4.3.7 Adding a Hot-Plug PCI-E Card ............................................ 59
4.3.8 CPU Module........................................................................ 60
4.3.9 Onboard Battery................................................................. 65
4.3.10 Chassis Components........................................................... 67
5Board Connections .......................................................................79
5.1 UCS C890 M5 Rack Server Connectors and LEDs.............................. 79
5.2 C890-M5-BMC= Connections, Jumpers and LEDs.............................. 80
5.3 C890-M5-HDD-BOARD= Connections and Jumpers.......................... 88
5.4 C890-M5-BPLANE= Connections........................................................ 89
6Software ......................................................................................91
6.1 Driver Installation............................................................................... 92
6.2 IPMI .................................................................................................... 94
7UEFI BIOS .....................................................................................96
7.1 Introduction........................................................................................ 96
7.2 Main Setup ......................................................................................... 97
7.3 Advanced Setup Configurations......................................................... 99
7.4 Event Logs......................................................................................... 151
7.5 IPMI .................................................................................................. 153
7.6 Security Settings............................................................................... 156
7.7 Boot Settings .................................................................................... 160
7.8 Save & Exit........................................................................................ 164
8BIOS PCIe Configuration..............................................................166
Appendix A BIOS Error Codes .........................................................177
A.1 BIOS Error Beep (POST) Codes ....................................................... 177
A.1 Additional BIOS POST Codes........................................................... 178
Appendix B Networking Options with Four CPUs............................179
Appendix C BIOS PCIe Configuration ..............................................181

Contents
Page 5
1Preface
This manual is written for professional system integrators, partner engineers and field
consultants. It provides information for the installation and use of the Cisco UCS C890
M5 Rack Server. Installation and maintenance should be performed by experienced
technicians only.
Refer to the Cisco UCS C890 M5 Rack Server specifications page on our website for
updates on supported memory, processors and operating systems (Cisco UCS C890
M5 Rack Server Data Sheet).

Contents
Page 6
2Overview
The Cisco UCS C890 M5 Rack Server, with eight sockets, is designed for workloads
that demand high-reliability, intensive compute operations with best-in-industry
management costs. This server delivers an impressive 605,730 SAPS by incorporating
eight Intel® Xeon® Platinum 8268, 8276, or 8280L processors. This server meets the
critical workload demands for SAPS/4 HANA, BW/4 HANA, Epic Caché databases, and
other massive virtualization workload consolidation efforts packed into a Seven-Unit
(7U) form factor.
2.1 Unpacking the System
Inspect the box the Cisco UCS C890 M5 Rack Server was shipped in and note if it was
damaged in any way. If any equipment appears damaged, file a damage claim with
the carrier who delivered it.
Decide on a suitable location for the rack unit that will hold the server. It should be
situated in a clean, dust-free area that is well ventilated. Avoid areas where heat,
electrical noise and electromagnetic fields are generated. It will also require a
grounded AC power outlet nearby.
2.2 System Specifications
The following table provides you with an overview of the main features and
specifications of the Cisco UCS C890 M5 Rack Server.
System Specifications
Motherboard
C890-M5-CPU-BOARD=
Chassis
C890-M5-CHASSIS=
CPU
The Cisco UCS C890 M5 Server includes eight C890-M5-CPU boards, and supports up to eight second-
generation Intel® Xeon series processors, which offers three full-width Intel UltraPath Interconnect (UPI) links
with the data
transfer rates of up to 10.4 GT/s in each direction among eight CPUs.
Socket Type
Socket P Type
Memory

Contents
Page 7
Each CPU board supports up to 3TB DCPMM or up to 3TB 3DS Load Reduced (3DS LRDIMM)/Load
Reduced (LRDIMM)/3DS Registered (RDIMM)/Registered (RDIMM) DDR4 memory of 2933*/MHz in 12
DIMM slots. It also supports up to 4.5TB DCPMM & DDR4 combined memory of up to 2933MHz in 12
slots per CPU board. With eight C890-M5 boards built-in, the C890-M5 Platform provides up to 24TB of
DDR4 memory or up to 36TB of Intel Optane DC PMem& DDR4 combined memory in 96 DIMM slots..
Note: 1. For your system memory to work properly, use memory modules of the same type and the same
speed on the motherboard. For more information on CPU, memory support, refer to the X11OPi board
section in this chapter. 2. Memory speed support depends on the processors used on the motherboard. 3.
2933 MHz memory and DCPMM memory are supported by Second Generation Intel® Xeon® Platinum
processors only. 4. Populating DDR4 memory modules in a two-DIMMs per-channel (2DPC) configuration
on this motherboard will affect memory bandwidth and performance.
Note 2: Supported DIMM memory sizes are:
• DDR4: 32GB, 64GB, 128 GB, and 256 GB
• Intel Optane DC PMem: 128 GB, 256 GB and 512 GB
Chipset
Intel C621 chipset
Expansion Slots
CPU modules: 16 FHFL PCI-e 3.0 x16 slots, two PCI-E 3.0x16 slots (CPU Slot1/Slot2) on each C890-M5-
CPU-BOARD= board.
PCIE modules: five FHHL PCI-e 3.0 x16 slots, one PCI-E 3.0 x16 slot on each C890-M5-PCIEBOARD=
board.
Storage module: two (2) PCI-E 3.0x8 in x16 slots (Slot1/Slot2) on the C890-M5-
HDD-BOARD= board. PCH module: two M.2 PCI-e 3.0 x4 slots (Slot1/Slot2) on
the C890-M5-AOC-1025 board.
Note: FHHL = full-height, half-length; FHFL = full-height, full-length; LP = low-profile
I/O Module Slot
One SIOM PCI-E 3.0x16 slot on the C890-M5-AOC-1025 board
Hard Drives
Storage module: Default setup is 2 hot-swap 2.5” HDDs to on-board SATA 3.0 ports with RAID
card options.
Total capacity is sixteen hot-swap 2.5” HDDs and six 3.5” or eight 2.5” internal fixed HDDs
PCH module: two SATA 3.0 ports for SATA DOMs
System Specifications
Graphics
ASPEED AST2500 BMC video controller via the BMC on the C890-M5-BMC card
Power
Five 1600 Watts Titanium Level power supplies
Form Factor
7U rackmount
Dimensions
(WxHxD) 17.63 x 12.2 x 28.87-in. / 310 x 448 x 733-cm

Contents
Page 8
Note: For IPMI configuration instructions, refer to the "Cisco UCS C-Series Integrated
Management Controller GUI Configuration Guide”.
2.3 Server Chassis Features
2.3.1 Control Panel/CPU Module Indicators
The switches and LEDs located on the control panel are described below.
Control Panel/CPU Module LED
Indicators
Item
Feature
Description
1
Power LED
Indicates power is being supplied to the system power supply. This LED is green when
the system is operating and amber when in standby mode.
2
Information
LED
See table below.
3
Power Button
Used to apply or remove power to the system. Turning off system power with this button
removes the main power but maintains standby power. Toperform most maintenance
tasks, you should also unplug the AC power cords before servicing.
4
CPU
Module
Connection
LED
When illuminated, this green LED indicates a CPU module is properly installed and
functioning.
5
Memory Error
LEDs
This array of LEDs is provided to indicate a memory slot on the CPU board that has failed. If
illuminated red, the designations (A1, A2, etc.) correspond to the failed slot on the CPU
board.
Information LED

Contents
Page 9
Status
Description
Continuously on and red
An overheat condition has occurred (this may be caused by cable congestion).
Blinking red (1Hz or 1/sec.)
Fan failure, check for an inoperative fan.
Blinking red (.25Hz or 4/sec.)
Indicates a power supply module has failed.

Contents
Page 10
2.3.2 Front Features
The Cisco UCS C890 M5 Rack Server is a 7U chassis.
Front Chassis
Features
Item
Feature
Description
1
CPU Modules
Eight CPU modules (designated CPU1-CPU8) for CPU, memory, NVMe,
PCI-E card and GPU.
2
Storage Module
Storage module for sixteen 2.5" hot-swap HDDs, optional RAID cards, and
optional fixed HDDs
3
Control Panel
Control panel with LEDs and buttons
4
Service Tag
Hidden tag for the information label

Contents
Page 11
2.3.3 Rear Features
The illustration below shows the features included on the rear of the chassis.
Item
Feature
Description
1-5
PCI-E Modules
Hot-plug* PCI-E modules 1 through 5 (#1 = PCIE module 1, etc.)
6-13
Hot-swap Fans
9-cm counter-rotating fans (#6 = FAN1, etc.)
14
PCH Module
PCH module for system PCH chip, IPMI port, SATA-DOM, M.2,
TPM, RAID key. Also includes a SIOM from CPU3 (default 4x
10G RJ45 with SFP LAN ports)
15-19
Power Supplies
5x Titanium Level 1600W power supplies (#15 = PS1, etc.)
20
Attention Button
Used to remove or apply power to the PCIE module
21
PCI-E Power LED
Green LED indicating power is present on the PCI-E module.
22
Attention LED
Amber LED indicating an operational problem on the PCI-E
module.
23
I/O Ports
VGA, COM and two USB ports in KVM port
*Must be supported by the OS.

Contents
Page 12
2.3.4 CPU Board
This section covers details about the CPU board.
Location of the C890-M5 Board
Major Components on the C890-M5 CPU Board

Contents
Page 13
Connectors on the C890-M5-CPU-BOARD
Connector
Description
J3/J6
High speed interconnect
J2/J4
High speed interconnect
J7
PCI-E 3.0 x16 slot supported by the CPU (Slot1)
J8
PCI-E 3.0 x16 slot supported by the CPU (Slot2)
J9/J10
CPU card guides (to be used to attach the CPU cards to the midplane board)
JLED1
Memory error LED connector
JPWR3/JPWR4/JPWR5
GPU power connectors 3/4/5
PWR1/PWR2
12-pin CPU power connectors 1/2
NVME0-NVME3
Non-Volatile Memory Express (NVME) connectors 1/2/3/4
DIMMA1-F2
DIMM memory modules (DIMMA1/A2/B1/B2/C1/C2/D1/D2/E1/E2/F1/F2)

Contents
Page 14
2.3.4.2 Processor and Memory Support on the C890-M5 CPU Board
CPU & Memory Support
Component
Description
CPU
Second-generation Intel® Xeon® Platinum 8XXX processor with UPI up to 10.4GT/s
DIMM Slots:
DIMMA1/A2,
DIMMB1/B2,
DIMMC1/C2,
DIMMD1/D2,
DIMME1/E2,
DIMMF1/F2
1. Each C890-M5-CPU-BOARD=board supports up to 3 TB Intel Optane PMem
memory of up 2933 MHz in 12 slots.
2. Each C890-M5-CPU-BOARD=board supports up to 2933 MHz Intel Optane DC
PMem and DDR4 combined memory in 12 slots.
3. With eight (8) C890-M5-CPU-BOARD= boards built-in, this platform supports up
to 24 TB of DDR4 memory or up to 36 TB of DCPMM & DDR4 combined memory
in 96 DIMM slots.
Notes:
1. Memory speed support depends on the CPUs used in the motherboard.
2. Refer to the memory configuration & population tables to maximize memory
performance.
2.3.5 C890-M5-BMC Layout
Your system comes with the C890-M5-BMC= card, which is to be installed on the
C890-M5-BPLANE= midplane. This section provides the information on the Intel C621
chipset add- on module.

Contents
Page 15
Location of the C890-M5-BMC= Card

Contents
Page 16
2.3.6 Major Components on the C890-M5-BMC Card
This section provides detailed information of the major components on the C890-M5-
BMC card.
C890-M5-BMC I/O Controllers
Note: M.2 modules come with 3 sizes in length. Horizontal lines marked "2260",
"2280", "22110" on the layout above indicate the locations where M.2 modules
should be installed depending on the sizes in length.

Contents
Page 17
2.3.6.1 Quick Reference to the Components on the C890-M5-BMC
Jumper
Description
Default Setting
JBT1
Clear CMOS
JPME1
ME Manufacturing Mode Select
Pins 1-2 (Normal)
JWD1
Watch Dog Timer Enable
Pins 1-2 (Reset)
Connector
Description
Battery (BT1)
Onboard CMOS Battery (See Chapter 3 for used battery disposal)
I-SATA3
Powered SATA 3.0 connector with a power pin built in. (SuperDOM) supported by Intel PCH.
I-SATA4
Powered SATA 3.0 connector with a power pin built in. (SuperDOM) supported by Intel PCH.
IPMI LAN
IPMI-dedicated LAN (J9) supported by the Baseboard Management Controller (BMC).
J1
M.2 connector used for SATA 3.0 or PCI-E 3.0 x4 support (Mounting hole locations: SWR1/SWR2/
SWR3/SWR4).
J2
M.2 connector used for SATA 3.0 or PCI-E 3.0 x4 support (Mounting hole locations: SWR5/SWR6/
SWR7/SWR8).
J5
Guide pin
J6
Power connector
JP1
DVD ROM power header (optional)
JSD1/JSD2
Power Connectors 1/2 for I-SATA DOM (Disk_on_Module).
JSIOM1
SIOM (Super I/O Module) PCI-E 3.0 x16 slot supported by CPU3 for use of a proprietary add-on
card
JTPM1
TPM (Trusted Platform Module)/Port 80 header.
JUSB1
Type A USB 3.0 header with support of USB 3.0 Port1, USB 2.0 Port2
JUSB2
USB 2.0 header with support of USB 5/USB 6
JUIDB1
UID (Unit Identification) button.
VROC (JRK1)
Intel VROC RAID key header for NVMe SSD
LED
Description
BMC_HB_LED1
BMC Heartbeat LED indicator (Green Blinking: BMC Normal)
LED1
Unit Identifier LED indicator (Blue: Unit Identified)

Contents
Page 18
2.3.7 C890-M5 PCIE Card
This section provides the information on the C890-M5-PCIEBOARD= add-on card.
2.3.8 Location of the C890-M5-PCIEBOARD= Cards
C890-M5-
PCIEBOARD (PCI-E
Card)
(5 Max.)

Contents
Page 19
Five (5)
C890-M5-PCIEBOARD on the
Rear side of Midplane

Contents
Page 20
2.3.9 Major Components on the C890-M5-PCIEBOARD Card
C890-M5-PCIEBOARD Card Layout (Front)
C890-M5-PCIEBOARD Card Layout (Reverse Side)
Major Components on the C890-M5-PCIEBOARD
Location
Description
Detailed Description
Front Side
J2
PCI-E Slot1: PCI-E 3.0 x16 slot
Reverse Side
SW1
Hot-plug attention button
Reverse Side
D1
Hot-plug attention LED (Orange: Hot-plug support needs attention)