xFusion Digital Technologies FusionServer XH321C V6 User manual

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
Issue
03
Date
2022-09-10
xFusion Digital Technologies Co., Ltd.

2022-09-10
ii
Copyrights © xFusion Digital Technologies Co., Ltd. 2022. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior
written consent of xFusion Digital Technologies Co., Ltd.
Trademarks and Permissions
and other xFusion trademarks are trademarks of xFusion Digital Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
In this document, "xFusion" is used to refer to "xFusion Digital Technologies Co., Ltd." for concise
description and easy understanding, which does not mean that "xFusion" may have any other meaning.
Any "xFusion" mentioned or described hereof may not be understood as any meaning other than "xFusion
Digital Technologies Co., Ltd.", and xFusion Digital Technology Co., Ltd. shall not bear any liability resulting
from the use of "xFusion".
The purchased products, services and features are stipulated by the contract made between xFusion and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
xFusion Digital Technologies Co., Ltd.
Address:
9th Floor, Building 1, Zensun Boya Square, Longzihu Wisdom Island
Zhengdong New District 450046
Zhengzhou, Henan Province
People's Republic of China
Website:
https://www.xfusion.com

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
Contents
2022-09-10
iii
Contents
About This Document.....................................................................................................................v
1 Overview.........................................................................................................................................7
2 Product Features ............................................................................................................................8
3 Physical Structure .......................................................................................................................11
4 Logical Structure .........................................................................................................................13
5 Hardware Description................................................................................................................15
5.1 Front Panel..................................................................................................................................................................15
5.1.1 Appearance..............................................................................................................................................................15
5.1.2 Indicators and Buttons .............................................................................................................................................16
5.1.3 Ports.........................................................................................................................................................................19
5.1.4 Installation Positions................................................................................................................................................20
5.2 Processors...................................................................................................................................................................21
5.3 Memory ......................................................................................................................................................................22
5.3.1 DDR4 Memory........................................................................................................................................................22
5.3.1.1 Memory Identifier.................................................................................................................................................22
5.3.1.2 Memory Subsystem Architecture..........................................................................................................................23
5.3.1.3 Memory Compatibility .........................................................................................................................................23
5.3.1.4 DIMM Installation Rules......................................................................................................................................25
5.3.1.5 Memory Installation Positions..............................................................................................................................25
5.3.1.6 Memory Protection Technologies .........................................................................................................................27
5.3.1.7 Memory Protection Technologies .........................................................................................................................28
5.4 Storage........................................................................................................................................................................28
5.4.1 Drive Configurations ...............................................................................................................................................28
5.4.2 Drive Numbering.....................................................................................................................................................29
5.4.3 RAID Controller Card .............................................................................................................................................29
5.5 Network ......................................................................................................................................................................29
5.5.1 OCP 3.0 Network Adapter .......................................................................................................................................29
5.6 I/O Expansion.............................................................................................................................................................30
5.6.1 PCIe Cards...............................................................................................................................................................30
5.6.2 PCIe Slots ................................................................................................................................................................30
5.6.3 PCIe Slot Description ..............................................................................................................................................31

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
Contents
2022-09-10
iv
5.7 PSUs ...........................................................................................................................................................................32
5.8 Boards.........................................................................................................................................................................33
5.8.1 Mainboard................................................................................................................................................................33
6 Product Specifications................................................................................................................35
6.1 Technical Specifications .............................................................................................................................................35
6.2 Environmental Specifications.....................................................................................................................................38
6.3 Physical Specifications ...............................................................................................................................................39
7 Software and Hardware Compatibility ..................................................................................41
8 Safety Instructions ......................................................................................................................42
8.1 Security.......................................................................................................................................................................42
8.2 Maintenance and Warranty .........................................................................................................................................45
9 System Management ..................................................................................................................46
10 Certifications..............................................................................................................................48
A Appendix .....................................................................................................................................50
B Glossary........................................................................................................................................57
C Acronyms and Abbreviations..................................................................................................60

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
About This Document
2022-09-10
v
About This Document
Purpose
This document describes the FusionServer XH321C V6 server node in terms of features,
structure, specifications, and component hardware and software compatibility.
Intended Audience
This document is intended for pre-sales engineers.
Symbol Conventions
The symbols that may be found in this document are defined as follows.
Symbol
Description
Indicates a hazard with a high level of risk which, if not
avoided, will result in death or serious injury.
Indicates a hazard with a medium level of risk which, if not
avoided, could result in death or serious injury.
Indicates a hazard with a low level of risk which, if not
avoided, could result in minor or moderate injury.
Indicates a potentially hazardous situation which, if not
avoided, could result in equipment damage, data loss,
performance deterioration, or unanticipated results.
NOTICE is used to address practices not related to personal
injury.
Supplements the important information in the main text.
NOTE is used to address information not related to personal
injury, equipment damage, and environment deterioration.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
About This Document
2022-09-10
vi
Change History
Issue
Date
Description
03
2022-09-10
Updated 6.2 Environmental Specifications.
02
2022-03-18
Updated 6.1 Technical Specifications and A.4 Sensor
List.
01
2021-12-28
This issue is the first official release.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
1 Overview
2022-09-10
7
1 Overview
FusionServer XH321C V6 is a liquid-cooled server node designed for X6000 V6 servers. An
X6000 V6 server can house up to four XH321C V6 server nodes in a 2 U chassis.
The XH321C V6 delivers supreme performance and high storage density in limited space
through innovative design. It is easy to manage and maintain.
XH321C V6 supports two SATA M.2 SSDs, a maximum of 16 DDR4 DIMMs, one PCIe card,
one RAID controller card, and one OCP card.
XH321C V6 is ideal for data center, cloud computing, Big Data, and Internet applications.
The XH321C V6 is a node with liquid cooling for processors, memory modules, and VRD.
⚫Provides liquid cold plates for processors, memory modules, and VRD.
⚫Used with the liquid-cooled cabinet.
⚫When used with the liquid-cooled cabinet, an optional liquid-cooled door can be used to
provide cooling for PCIe cards, OCP standard cards, and other components.
Figure 1-1 XH321C V6

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
2 Product Features
2022-09-10
8
2 Product Features
Performance and Scalability
⚫Powered by the third-generation Intel®Xeon®Scalable Ice Lake processors, the server
provides up to 40 cores, 3.6 GHz frequency, a 60 MB L3 cache, and up to three 11.2
GT/s UPI links between the processors, which deliver supreme processing performance.
−It supports up to two processors with 80 cores and 160 threads to maximize the
concurrent execution of multithreaded applications.
−The L2 cache capacity is increased. Each core exclusively occupies 1.25 MB L2
cache, and at least 1.5 MB L3 cache.
−Intel Turbo Boost Technology 2.0 allows processor cores to run faster than the
frequency specified in the Thermal Design Power (TDP) configuration if they are
operating below power, current, and temperature specification limits.
−Intel Hyper-Threading Technology enables each processor core to run up to two
threads, improving parallel computation capability.
−The hardware-assisted Intel® Virtualization Technology (Intel® VT) allows
operating system (OS) to better use hardware to address virtualization workloads.
−Intel DL Boost (VNNI) is supported to improve the performance of deep learning
applications.
−Intel® Advanced Vector Extensions 512 (IntelAVX-512) significantly accelerates
floating-point performance for computing-intensive applications.
⚫The server supports a maximum of 16 DDR4 ECC 3200 MT/s DIMMs. The DDR4 ECC
DIMMs support registered DIMMs (RDIMM) and load-reduced DIMMs (LRDIMMs),
which provide high speed and availability. A server supports a maximum memory
capacity of 2048 GB and a maximum memory bandwidth of 400 GB/s in theory.
⚫It supports 2 M.2 SSDs. The SSDs deliver higher I/O performance than HDDs. An SSD
supports approximately 100 times more I/O operations per second (IOPS) than a typical
HDD.
⚫With Intel integrated I/O, the third-generation Intel®Xeon®Scalable processors integrate
the PCIe 4.0 controller to shorten I/O latency and improve overall system performance.
⚫The server supports one half-height half-length PCIe 4.0 standard card.
⚫One slot dedicated for the OCP 3.0 network adapter.
Availability and Serviceability
⚫Carrier-class components with process expertise ensure high system reliability and
availability.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
2 Product Features
2022-09-10
9
⚫The server supports hot-swappable SAS/SATA/NVMe U.2 drives. SAS/SATA drives
support RAID 0, 1, 5, 6, 10, 50, and 60, depending on the RAID controller card used.
⚫The server provides simplified O&M and efficient troubleshooting through the UID/HLY
indicators on the front panel and iBMC WebUI.
⚫The panel provides iBMC direct connect management ports to support local iBMC O&M,
improving O&M efficiency.
⚫Each node provides one hot-swappable PSU. The entire chassis supports a maximum of
four hot-swappable PSUs in 1+1 or 2+2 redundancy.
⚫SSDs offer better reliability than HDDs, prolonging system uptime.
⚫The built-in iBMC monitors system parameters in real time, triggers alarms, and
performs recovery actions to minimize the system downtime.
⚫For details about the warranty policy, visit Warranty.
Manageability and Security
⚫The built-in iBMC monitors server operating status and provides remote management.
⚫A password is required for accessing the BIOS, ensuring system boot and management
security.
⚫The NC-SI feature allows a network port to serve as a management port and a service
port. The NC-SI feature is disabled by default and can be enabled through the iBMC or
BIOS.
⚫The integrated Unified Extensible Firmware Interface (UEFI) improves setup,
configuration, and update efficiency and simplifies fault handling.
⚫Aggregated management of node management ports
⚫Intel Execute Disable Bit (EDB) function prevents certain types of malicious buffer
overflow attacks when working with a supported OS.
⚫The Intel Converged Boot Guard & Trusted Execution Technology (Intel CBnT)
prevents malicious software attacks based on hardware, prevents the firmware from
being maliciously modified, and prevents the execution of unauthorized boot blocks. It
also allows applications to run in their own independent space without being affected by
other software running in the system, thereby enhancing security.
⚫The trusted platform module (TPM) and trusted cryptography module (TCM) provide
advanced encryption functions, such as digital signatures and remote authentication.
⚫The secure boot based on the chip RoT implements level-by-level firmware verification
starting from the hardware RoT and builds a complete secure boot chain.
⚫The following requirements in NIST SP 800-147B are met:
−The BIOS firmware digital signature update mechanism is supported. During the
upgrade, the digital signature is verified to prevent unauthorized BIOS firmware
upgrade.
−The flash security protection mechanism is supported to prevent unauthorized
modification of the flash memory in the OS.
The service port with NC-SI enabled supports the following configuration:
⚫Configuring any port on the OCP 3.0 network adapter or other network adapters that support the
NC-SI feature as the service port with NC-SI enabled.
⚫Enabling, disabling, and setting a virtual local area network (VLAN) ID for this port. The VLAN ID
is 0and disabled by default.
⚫Configuring IPv4 addresses (IPv4 address, subnet mask, and gateway) and IPv6 addresses (IPv6
address, prefix length, and gateway) for this port.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
2 Product Features
2022-09-10
10
Energy Efficiency
⚫The server supports 80 Plus Platinum/Titanium PSUs of different energy efficiency
levels. The PSU efficiency reaches 96% at 50% load.
⚫Compared with the last-generation product, third-generation Intel®Xeon®Scalable
processor greatly improves the performance.
⚫Active/standby power supply and HVDC power supply are supported, improving the
efficiency of the power supply system.
⚫Efficient voltage regulator-down (VRD) power supplies for boards minimize the energy
loss from DC/DC power conversion.
⚫Intel Intelligent Power Capability allows the processor logic unit to be powered on or off
based on requirements.
⚫Staggered spin-up of drives reduces the server boot power consumption.
⚫The server supports intelligent frequency scaling of processors for energy conservation
and consumption reduction.
⚫SSDs consume 80% less power than HDDs.
⚫The quadrilateral ventilation holes on the node panel provide higher ventilation density
than round holes, increasing the system cooling efficiency.
⚫The low-voltage third-generation Intel®Xeon®Scalable Ice Lake processors consume
less energy, ideally suited for data centers and telecommunications environments
constrained by power and thermal limitations.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
3 Physical Structure
2022-09-10
11
3 Physical Structure
Figure 3-1 XH321C V6 physical structure
1
Screw-in RAID controller
card
2
Cable management arm
3
Liquid cold plate
4
TPM/TCM
5
Memory modules
6
Processors
7
Mainboard
8
Coin battery
9
Server node case
10
I/O board

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
3 Physical Structure
2022-09-10
12
11
OCP 3.0 network adapter
12
Management network port
board
13
PCIe riser module
14
PSU
15
M.2 SSD
16
M.2 riser card

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
4 Logical Structure
2022-09-10
13
4 Logical Structure
Figure 4-1 XH321C V6 logical structure
⚫Scenario 1: CPU 1 and CPU 2 are connected to the pass-through PCIe riser card through PCIe x8
connectors respectively.
⚫Scenario 2: CPU 2 is connected to the pass-through PCIe riser card through a PCIe x16 connector.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
4 Logical Structure
2022-09-10
14
⚫The server supports one or two third-generation Intel®Xeon®Scalable Ice Lake
processors.
⚫The server supports up to 16 memory modules.
⚫The CPUs (processors) interconnect with each other through three UPI links at a speed
of up to 11.2 GT/s.
⚫CPU1 supports one OCP 3.0 network adapter.
⚫The PCIe riser module connects to the processors through PCIe buses to provide one
PCIe slot.
⚫The screw-in RAID controller card connects to the mainboard through a high-speed
connector, and the mainboard is connected to drives through the middle module to
support different local storage configurations.
⚫The LBG-R Platform Controller Hub (PCH) is integrated on the mainboard to support
the USB 3.0 port.
⚫The BMC integrates the graphics card, video compression, and virtual media
components to provide device management functions, such as power control, slot ID
acquisition, power supply detection, and KVM over IP.
⚫The BMC management chip integrated on the mainboard supports a video graphics array
(VGA) port, a management network port, and a serial port.
The VGAport is on the right mounting ear of the X6000 V6 chassis. For details, see "Ports" in the
FusionServer X6000 V6 Server User Guide.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
15
5 Hardware Description
5.1 Front Panel
5.2 Processors
5.3 Memory
5.4 Storage
5.5 Network
5.6 I/O Expansion
5.7 PSUs
5.8 Boards
5.1 Front Panel
5.1.1 Appearance
Figure 5-1 Front view
1
PCIe card 1
2
Node ejector lever
3
FlexIO card
4
Slide-out label plate (with an

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
16
NOTE
The FlexIO card slot supports
only OCP 3.0 network adapters.
SN label)
5
PSU
-
-
5.1.2 Indicators and Buttons
Indicator and Button Positions
Figure 5-2 Indicators and buttons on the front panel
1
Health status indicator
2
Power button/indicator
3
UID button/indicator
4
iBMC direct connect
management port indicator
5
Management network port
data transmission status
indicator
6
Connection status indicator of
the management network port
7
PSU indicator
-
-
Indicator and Button Description
Table 5-1 Indicators and buttons on the front panel
Silkscreen
Indicator/Butt
on
Description
Power
button/indicator
Power indicator:
⚫Off: The node is not powered on.
⚫Steady green: The node is in the power-on state.
⚫Blinking yellow: The iBMC is starting. The power
button is locked and cannot be pressed. The iBMC
is started in about 1 minute, and then the power
indicator is steady green.
⚫Steady yellow: The node is standby.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
17
Silkscreen
Indicator/Butt
on
Description
Power button:
⚫When the device is powered on, you can press this
button to gracefully shut down the OS.
NOTE
For different OSs, you may need to shut down the OS as
prompted.
⚫When the device is powered on, you can hold
down this button for 6 seconds to forcibly power
off the device.
⚫When the power indicator is steady yellow, you
can press this button to power on the device.
Health status
indicator
⚫Off: The device is powered off or is faulty.
⚫Blinking red at 1 Hz: A major alarm has been
generated on the system.
⚫Blinking red at 5 Hz: A critical alarm has been
generated on the system.
⚫Steady green: The device is operating properly.
UID
button/indicator
The UID button/indicator helps identify and locate a
device.
UID indicator:
⚫Off: The device is not being located.
⚫Blinking or steady blue: The device is being
located.
UID button:
⚫You can control the UID indicator status by
pressing the UID button or using the iBMC.
⚫You can press this button to turn on or off the
UID indicator.
⚫You can press and hold down this button for 4 to
6 seconds to reset the iBMC.
-
iBMC direct
connect
management
port indicator
Indicates the status when the iBMC direct connect
management port connects to a terminal (local PC or
Android mobile phone):
⚫Off: No terminal is connected.
⚫Blinking green at short intervals for 3 seconds and
then off: The port is disabled.
⚫Steady green: The terminal is connected.
-
Data
transmission
status indicator
for the
management
network port
⚫Off: No data is being transmitted.
⚫Blinking yellow: Data is being transmitted.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
18
Silkscreen
Indicator/Butt
on
Description
-
Connection
status indicator
of the
management
network port
⚫Off: The network port is not connected.
⚫Steady green: The network port is connected
properly.
-
PSU indicator
⚫Off: No power is supplied.
⚫Blinking green at 1 Hz:
−The input is normal, and the server is standby.
−The input is overvoltage or undervoltage.
−The PSU is in deep hibernation mode.
⚫Blinking green at 4 Hz: The firmware is being
upgraded online.
⚫Steady green: The power input and output are
normal.
⚫Steady orange: The input is normal but there is no
output.
NOTE
The possible causes of no power output are as follows:
⚫Power supply overtemperature protection
⚫Power output overcurrent or short-circuit
⚫Output overvoltage
⚫Short-circuit protection
⚫Device failure (excluding failure of all devices)
⚫Off: No power is supplied.
⚫Blinking green at 1 Hz:
−The input is normal, and the server is in the
standby state.
−The input is overvoltage or undervoltage.
−The PSU is in deep hibernation mode.
⚫Blinking green at 4 Hz: The firmware is being
upgraded online.
⚫Steady green: The power input and output are
normal.
⚫Steady orange: The input is normal but there is no
output.
NOTE
The possible causes of no power output are as follows:
⚫Power supply overtemperature protection
⚫Power output overcurrent or short-circuit
⚫Output overvoltage
⚫Short-circuit protection
⚫Device failure (excluding failure of all devices)
⚫Steady red: The output overcurrent occurs.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
19
5.1.3 Ports
Port Positions
Figure 5-3 Ports on the front panel
1
Water inlet
2
Water outlet
3
USB 3.0 Type-C port
4
iBMC direct connect
management port/debugging
serial port
5
Management network port
6
PSU socket
Port Description
Table 5-2 Ports on the front panel
Port
Type
Quantity
Description
Water inlet
-
1
Connects to a water pipe to supply
cold water.
Water outlet
-
1
Connects to a water pipe to
discharge hot water.
USB 3.0
Type-C port
USB Type-C
1
Used to connect to a USB 3.0
device.
NOTICE
⚫The maximum current is 1.2 A for
an external USB device.
⚫Before connecting an external
USB device, ensure that the USB
device functions properly.
Otherwise, it may adversely
impact the server.
⚫Supports one or two USB 3.0 port,
depending on the cable type.

FusionServer XH321C V6 Liquid-cooled Server Node
Technical White Paper
5 Hardware Description
2022-09-10
20
Port
Type
Quantity
Description
iBMC direct
connect
management
port/debugging
serial port
USB Type-C
1
⚫Uses a USB Type-C cable to
connect a local PC or mobile
phone to the server node iBMC
to monitor and manage the
system.
NOTE
Only local PCs running Windows 10
and mobile phones running Android
are supported.
⚫To log in to the iBMC from the
local PC, enter https://IP address
of the iBMC management network
port in the address box of the
browser on the local PC.
⚫When accessing the iBMC
through a mobile phone, you need
to use the mobile application
FusionMobile to access the iBMC.
For details, see the FusionMobile
User Guide.
⚫Connects to a Type-C-to-serial
port (RJ45) to implement the
debugging function.
Management
network port
RJ45
1
iBMC management network port,
which is used to manage the
server.
NOTE
The management network port is a
GE port that supports 100 Mbit/s and
1000 Mbit/s auto-negotiation.
PSU socket
-
1
Used to connect to a power
distribution unit (PDU) through a
power cable. You can select the
PSUs as required.
NOTE
When determining the PSUs, ensure
that the sum of rated power of all
PSUs is greater than that of the server.
5.1.4 Installation Positions
The XH321C V6 nodes are installed in the 4 slots on the rear of a 2 U X6000 V6 chassis,
which can house a maximum of 4 XH321C V6 nodes.