xFusion Digital Technologies FusionServer G5500 V6 User manual

FusionServer G5500 V6 Server
Technical White Paper
Issue
01
Date
2023-02-14
xFusion Digital Technologies Co., Ltd.

2023-02-14
ii
Copyrights © xFusion Digital Technologies Co., Ltd. 2023. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of xFusion Digital Technologies Co., Ltd.
Trademarks and Permissions
and other xFusion trademarks are trademarks of xFusion Digital Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
In this document, "xFusion" is used to refer to "xFusion Digital Technologies Co., Ltd." for concise
description and easy understanding, which does not mean that "xFusion" may have any other meaning. Any
"xFusion" mentioned or described hereof may not be understood as any meaning other than "xFusion Digital
Technologies Co., Ltd.", and xFusion Digital Technology Co., Ltd. shall not bear any liability resulting from
the use of "xFusion".
The purchased products, services and features are stipulated by the contract made between xFusion and
the customer. All or part of the products, services and features described in this document maynot be within
the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
xFusion Digital Technologies Co., Ltd.
Address:
9th Floor, Building 1, Zensun Boya Square, Longzihu Wisdom Island
Zhengdong New District 450046
Zhengzhou, Henan Province
People's Republic of China
Website:
https://www.xfusion.com

FusionServer G5500 V6 Server
Technical White Paper
Contents
2023-02-14
iii
Contents
About This Document........................................................................................................v
1 Product Overview ...........................................................................................................7
2 Product Features ............................................................................................................8
3 Physical Structure ........................................................................................................11
4 Logic Structure .............................................................................................................13
5 Hardware Description...................................................................................................17
5.1 Front Panel ................................................................................................................................................ 17
5.1.1 Appearance ............................................................................................................................................ 17
5.1.2 Indicators and Buttons............................................................................................................................ 19
5.1.3 Ports ....................................................................................................................................................... 21
5.2 Rear Panel................................................................................................................................................. 23
5.2.1 Appearance ............................................................................................................................................ 23
5.2.2 Indicators and Buttons............................................................................................................................ 24
5.2.3 Ports ....................................................................................................................................................... 26
5.3 Processors................................................................................................................................................. 27
5.4 Memory...................................................................................................................................................... 28
5.4.1 DDR4 Memory........................................................................................................................................ 28
5.4.1.1 Memory Identifier................................................................................................................................. 28
5.4.1.2 Memory Subsystem Architecture......................................................................................................... 30
5.4.1.3 Memory Compatibility.......................................................................................................................... 31
5.4.1.4 DIMM Installation Rules....................................................................................................................... 32
5.4.1.5 Memory Installation Positions.............................................................................................................. 33
5.4.1.6 Memory Protection Technologies ........................................................................................................ 37
5.5 Storage ...................................................................................................................................................... 37
5.5.1 Drive Configuration and Drive Numbering.............................................................................................. 37
5.5.1.1 24 x 3.5" drive configuration (20 x SAS/SATA + 4 x NVMe)................................................................ 37
5.5.1.2 24 x 3.5" drive configuration (16 x SAS/SATA+ 8 x NVMe)................................................................ 39
5.5.2 Drive Indicators....................................................................................................................................... 42
5.5.3 RAID Controller Card.............................................................................................................................. 44
5.6 Network...................................................................................................................................................... 44
5.6.1 OCP 3.0 Network Adapter ...................................................................................................................... 44

FusionServer G5500 V6 Server
Technical White Paper
Contents
2023-02-14
iv
5.7 I/O Expansion ............................................................................................................................................ 44
5.7.1 PCIe Cards............................................................................................................................................. 44
5.7.2 PCIe Slots............................................................................................................................................... 45
5.7.3 PCIe Slot Description ............................................................................................................................. 47
5.8 PSUs.......................................................................................................................................................... 49
5.9 Fan Modules.............................................................................................................................................. 50
5.10 Boards ..................................................................................................................................................... 52
5.10.1 Mainboard............................................................................................................................................. 52
5.10.2 Drive Backplane.................................................................................................................................... 54
5.10.3 Fan Board............................................................................................................................................. 56
5.10.4 PSU Backplane..................................................................................................................................... 57
5.10.5 PCIe Switch Board................................................................................................................................ 58
5.10.6 M.2 SSD Adapter Board ....................................................................................................................... 60
6 Product Specifications.................................................................................................61
6.1 Technical Specifications ............................................................................................................................ 61
6.2 Environmental Specifications .................................................................................................................... 65
6.3 Physical Specifications.............................................................................................................................. 66
7 Software and Hardware Compatibility ........................................................................69
8 Safety Instructions........................................................................................................70
8.1 Security...................................................................................................................................................... 70
8.2 Maintenance and Warranty........................................................................................................................ 73
9 System Management....................................................................................................74
10 Certifications...............................................................................................................76
A Appendix.......................................................................................................................77
B Glossary........................................................................................................................91
C Acronyms and Abbreviations......................................................................................94

FusionServer G5500 V6 Server
Technical White Paper
About This Document
2023-02-14
v
About This Document
Purpose
This document describes the appearance, features, performance parameters, and
hardware and software compatibility of the FusionServer G5500 V6, so that users can
have an in-depth and detailed understanding of FusionServer G5500 V6.
Intended Audience
This document is intended for presales engineers.
Symbol Conventions
The symbols that may be found in this document are defined as follows.
Symbol
Description
Indicates a hazard with a high level of risk which, if not
avoided, will result in death or serious injury.
Indicates a hazard with a medium level of risk which, if not
avoided, could result in death or serious injury.
Indicates a hazard with a low level of risk which, if not
avoided, could result in minor or moderate injury.
Indicates a potentially hazardous situation which, if not
avoided, could result in equipment damage, data loss,
performance deterioration, or unanticipated results.
NOTICE is used to address practices not related to
personal injury.
Supplements the important information in the main text.
NOTE is used to address information not related to
personal injury, equipment damage, and environment
deterioration.

FusionServer G5500 V6 Server
Technical White Paper
About This Document
2023-02-14
vi
Change History
Issue
Date
Description
01
2023-02-14
This issue is the first official release.

FusionServer G5500 V6 Server
Technical White Paper
1 Product Overview
2023-02-14
7
1 Product Overview
The FusionServer G5500 V6 server (G5500 V6 for short) is a GPU computing server
that is optimized in HPC as well as data training and inference for deep learning for
scenarios such as AI, HPC, cloud computing, and big data analysis. It supports
multiple types of heterogeneous processors and applies to both enterprise and public
cloud deployments.
The G5500 V6 is a 4U GPU server that features supreme performance, flexible
scalability, and balanced storage.
For details about the G5500 V6 nameplate, see A.4 Nameplate.
Figure 1-1 G5500 V6 G5500 V6 server with 24 x 3.5" drives (example)

FusionServer G5500 V6 Server
Technical White Paper
2 Product Features
2023-02-14
8
2 Product Features
Performance
⚫Powered by the third-generation Intel®Xeon®Scalable processors (Ice Lake),
each of which provides up to 40 cores, 3.6 GHz frequency, a 60 MB L3 cache,
and up to three 11.2 GT/s UPI links between the processors, the server delivers
supreme processing performance.
−It supports up to two processors with 80 cores and 160 threads to maximize
the concurrent execution of multithreaded applications.
−The L2 cache capacity is increased. Each core exclusively occupies 1.25 MB
L2 cache, and at least 1.5 MB L3 cache.
−Intel Turbo Boost Technology 2.0 allows processor cores to run faster than
the frequency specified in the Thermal Design Power (TDP) configuration if
they are operating below power, current, and temperature specification
limits.
−Intel hyper-threading technology enables each processor core to run up to
two threads, improving parallel computation capability.
−The hardware-assisted Intel®Virtualization Technology (Intel®VT) allows the
operating system (OS) to better use hardware to address virtualization
workloads.
−Intel®Advanced Vector Extensions 512 (IntelAVX-512) significantly
accelerates floating-point performance for computing-intensive applications.
−Intel DL Boost (VNNI) is supported to improve the performance of deep
learning applications.
−The Intel®SGX and Intel®TME security features provide fine-grained data
protection through application isolation in the memory, and defend against
physical attacks through full memory encryption.
⚫The G5500 V6 supports a maximum of 32 DDR4 ECC 3200 MT/s DIMMs. The
DDR4 ECC DIMMs support registered DIMMs (RDIMMs) or load-reduced DIMMs
(LRDIMMs). They provide high speed and availability.A server supports a
maximum memory capacity of 8192 GB and a maximum memory bandwidth of
400 GB/s in theory.
⚫Flexible drive configurations cater to a variety of business requirements and
ensure high elasticity and scalability of storage resources.
⚫The use of 12 Gbit/s SCSI (SAS) serial connection for internal storage provides
twice data transmission rate than the use of 6 Gbit/s SAS connection, maximizing
the performance of I/O-intensive applications.

FusionServer G5500 V6 Server
Technical White Paper
2 Product Features
2023-02-14
9
⚫With Intel integrated I/O, the third-generation Intel®Xeon®Scalable processors
integrate the PCIe 4.0 controller to shorten I/O latency and improve overall
system performance.
Scalability
⚫Flexible drive configurations cater to a variety of business requirements and
ensure high elasticity and scalability of storage resources.
⚫The server supports a maximum of 24 x 3.5" or 24 x 2.5" front drives.
⚫The server supports a maximum of 8 x 2.5" NVMe U.2 drives.
⚫The server supports up to 12 PCIe 4.0 x16 expansion slots.
⚫The server supports three OCP 3.0 network adapters, one of which supports
upgradable PCIe 4.0 x16 bandwidth. GE/10GE/25GE/100GE network adapters
can be configured flexibly. OCP 1/OCP 2 supports orderly hot swap.
The hot swap of the OCP NIC depends on the OS driver. Ensure that the system enters the OS
state and the OCP hot swap-related drivers have been loaded.
⚫Two M.2 modules can be configured.
Availability and Serviceability
⚫Carrier-class components with process expertise ensure high system reliability
and availability.
⚫The server supports hot-swappable SAS/SATA drives. SAS/SATA drives support
RAID 0, 1, 1E, 10, 5, 50, 6, and 60, depending on the RAID controller card used.
It also uses a supercapacitor to protect the RAID cache data against power
failures.
⚫The server provides simplified O&M and efficient troubleshooting through the
UID/HLY indicators on the front panel, fault diagnosis LED, and iBMC WebUI.
⚫The panel provides the iBMC direct connect management port to support local
iBMC O&M, improving O&M efficiency.
⚫The server provides four hot-swappable PSUs in 2+2 redundancy mode and six
hot-swappable fan modules in N+1 redundancy mode, improving system
availability.
⚫The onboard Intelligent Baseboard Management Controller (iBMC) can
continuously monitor system parameters, trigger alarms, and take recovery
measures to minimize shutdown.
⚫For more information about the warranty in the global market, see Warranty.
Manageability and Security
⚫The built-in iBMC monitors server operating status and provides remote
management.
⚫A password is required for accessing the BIOS, ensuring system boot and
management security.
⚫The Network Controller Sideband Interface (NC-SI) feature allows a network port
to simultaneously serve as a management port and a service port. The NC-SI
feature is disabled by default and can be enabled on the iBMC or BIOS.
The service port with NC-SI enabled supports the following configuration:

FusionServer G5500 V6 Server
Technical White Paper
2 Product Features
2023-02-14
10
⚫Configuring any network port on the OCP 3.0 network adapter or PCIe NIC (with NC-SI
enabled).
⚫Enabling, disabling, and setting a virtual local area network (VLAN) ID for this port. The VLAN
ID is 0and disabled by default.
⚫Configuring IPv4 addresses (IPv4 address, subnet mask, and default gateway) and IPv6
addresses (IPv6 address, prefix length) for this port.
⚫The integrated Unified Extensible Firmware Interface (UEFI) improves setup,
configuration, and update efficiency and simplifies fault clearance.
⚫The lockable server chassis panel ensures local data security.
⚫Chassis cover opening detection is supported to enhance physical security.
⚫Intel Execute Disable Bit (EDB) function prevents certain types of malicious buffer
overflow attacks when working with a supported OS.
⚫Intel®Trusted Execution Technology prevents malicious software attacks based
on hardware, prevents the firmware on the device from being maliciously
modified, and prevents unauthorized boot block execution.
⚫Intel®Software Guard Extensions (Intel®SGX) technology allows applications to
run in their own independent space without being affected by other software
running in the system, thereby enhancing security.
⚫The secure boot based on the chip RoT implements level-by-level firmware
verification starting from the hardware RoT and builds a complete secure boot
chain.
⚫The trusted platform module (TPM) provides advanced encryption functions,
such as digital signatures and remote authentication.
⚫The following requirements in NIST SP 800-147B are met:
−The BIOS firmware digital signature update mechanism is supported. During
the upgrade, the digital signature is verified to prevent unauthorized BIOS
firmware upgrade.
−The flash security protection mechanism is supported to prevent
unauthorized modification of the flash memory in the OS.
Energy Efficiency
⚫The server supports 80 PLUS Platinum/Titanium PSUs of different energy
efficiency levels. The PSU efficiency reaches 96% at 50% load.
⚫Active/standby power supply and HVDC power supply are supported, improving
the efficiency of the power supply system.
⚫Efficient voltage regulator-down (VRD) power supplies for boards minimize the
energy loss from DC-to-DC power conversion.
⚫Area-based, Proportional-Integral-Derivative (PID) intelligent fan speed
adjustment and intelligent CPU frequency scaling optimize heat dissipation and
reduce overall system power consumption.
⚫The improved thermal design with energy-efficient fans ensures optimal heat
dissipation and reduces system power consumption.
⚫The server is protected with power capping and power control measures.
⚫Staggered spinup of drives reduces the server boot power consumption.

FusionServer G5500 V6 Server
Technical White Paper
3 Physical Structure
2023-02-14
11
3 Physical Structure
Figure 3-1 Physical structure of a server with 24 x 3.5" drives (example)
1
PCIe switch I/O board
2
GPU
3
PSUs
4
Power adapter board
5
Chassis
6
GPU beam
7
Supercapacitor
8
Air duct
9
Cable management arm
(CMA)
10
M.2 SSD
11
PCIe RAID controller card
12
PCIe riser card
13
Intrusion sensor
14
Fan board

FusionServer G5500 V6 Server
Technical White Paper
3 Physical Structure
2023-02-14
12
15
24 x 3.5" drive backplane
16
Left mounting ear plate
17
3.5" drives
18
Fan modules
19
Right mounting ear plate
20
Fan module frame
21
CMA
22
CPU
23
CPU heat sink
24
Mainboard
25
Memory modules
26
TPM/TCM
27
BMC card
28
OCP 3.0 network adapter
29
NIC riser modules
-
-

FusionServer G5500 V6 Server
Technical White Paper
4 Logic Structure
2023-02-14
13
4 Logic Structure
Figure 4-1 Logic structure

FusionServer G5500 V6 Server
Technical White Paper
4 Logic Structure
2023-02-14
14
⚫The server supports one or two third-generation Intel®Xeon®Scalable
processors (Ice Lake).
⚫The server supports up to 32 memory modules.

FusionServer G5500 V6 Server
Technical White Paper
4 Logic Structure
2023-02-14
15
⚫The processors interconnect with each other through three UPI links at a speed
of up to 11.2 GT/s.
⚫The processors connect to the PCIe riser card through the PCIe buses, and the
riser card connects to the mainboard through cables. Therefore, the processors
can be flexibly configured as required.
⚫CPU 1 supports one OCP 3.0 network adapter, and CPU 2 supports two OCP 3.0
network adapters.
⚫The PCIe RAID controller card on the mainboard connects to CPU 1 through
PCIe buses, and connects to the drive backplane through SAS signal cables.A
variety of drive backplanes are provided to support different local storage
configurations.
⚫The LBG-R Platform Controller Hub (PCH) is integrated on the mainboard to
support five USB 3.0 ports.
⚫The BMC management chip integrated on the mainboard provides a video
graphics array (VGA) port, a management network port, and a serial port.
⚫The G5500 V6 supports multiple logical topologies, cascaded and balanced
topologies, and can be configured using the management software to quickly
adapt to different service scenarios and achieve optimal service performance.
The details are as follows:
−As shown in Figure 4-2, the balanced topology is applicable to large-scale
deep learning training.
Figure 4-2 Balanced topology
−As shown in Figure 4-3, the cascaded topology is applicable to small- and
medium-scale deep learning training and inference, public cloud, and HPC
scenarios.

FusionServer G5500 V6 Server
Technical White Paper
4 Logic Structure
2023-02-14
16
Figure 4-3 Cascaded topology

FusionServer G5500 V6 Server
Technical White Paper
5 Hardware Description
2023-02-14
17
5 Hardware Description
5.1 Front Panel
5.2 Rear Panel
5.3 Processors
5.4 Memory
5.5 Storage
5.6 Network
5.7 I/O Expansion
5.8 PSUs
5.9 Fan Modules
5.10 Boards
5.1 Front Panel
5.1.1 Appearance
⚫24 x 3.5" drive configuration (20 x SAS/SATA + 4 x NVMe)

FusionServer G5500 V6 Server
Technical White Paper
5 Hardware Description
2023-02-14
18
Figure 5-1 Front view
1
SATA/SAS drives
2
SAS/SATA/NVMe drives
3
Slide-out label plate (with
an SN label)
-
-
⚫24 x 3.5" drive configuration (16 x SAS/SATA + 8 x NVMe)
Figure 5-2 Front view
1
SATA/SAS drives
2
SAS/SATA/NVMe drives
3
Slide-out label plate (with
an SN label)
-
-

FusionServer G5500 V6 Server
Technical White Paper
5 Hardware Description
2023-02-14
19
5.1.2 Indicators and Buttons
Indicator and Button Positions
⚫24 x 3.5" drive configuration
Figure 5-3 Indicators and buttons on the front panel
1
Fault diagnosis LED
2
Power button/indicator
3
Health status indicator
4
UID button/indicator
5
FlexIO card 1 presence
indicator
6
FlexIO card 3 presence
indicator
7
iBMC direct connect
management port indicator
-
-
Indicator and Button Descriptions
Table 5-1 Indicatorsand buttons on thefront panel
Silkscreen
Indicator and
Button
Description
Fault
diagnosis LED
⚫---: The device is operating properly.
⚫Error code: A component is faulty.
For details about the error codes, see the
FusionServer G5500 Server iBMC Alarm
Handling.
Power
button/indicato
r
Power indicator:
⚫Off: The device is powered off.
⚫Steady green: The device is powered on.
⚫Blinking yellow: The iBMC is starting. The
power button is locked and cannot be
pressed. The iBMC is started in about 1
minute, and then the power indicator is steady

FusionServer G5500 V6 Server
Technical White Paper
5 Hardware Description
2023-02-14
20
Silkscreen
Indicator and
Button
Description
yellow.
⚫Steady yellow: The device is in the standby
state.
Power button:
⚫When the device is powered on, you can
press this button to gracefully shut down the
OS.
NOTE
For different OSs, you may need to shut down the OS
as prompted.
⚫When the device is powered on, you can hold
down this button for 6 seconds to forcibly
power off the device.
⚫When the power indicator is steady yellow,
you can press this button to power on the
server node.
Health status
indicator
⚫Off: The device is powered off or faulty.
⚫Blinking red at 1 Hz: A major alarm has been
generated on the system.
⚫Blinking red at 5 Hz: A critical alarm has been
generated on the system.
⚫Steady green: The device is operating
properly.
UID
button/indicato
r
The UID button/indicator helps identify and
locate a device.
UID indicator:
⚫Off: The device is not being located.
⚫Blinking or steady blue: The device is being
located.
UID button:
⚫You can control the UID indicator status by
pressing the UID button or using the iBMC.
⚫You can press this button to turn on or off the
UID indicator.
⚫You can press and hold down this button for 4
to 6 seconds to reset the iBMC.
OCP 3.0
network
adapter
presence
indicator
Indicates whether OCP 1 and OCP 3 of the OCP
3.0 network adapter is detected.
⚫Off: The OCP 3.0 network adapter is not in
position.
⚫Blinking green at 0.5 Hz: The OCP 3.0
network adapter is in position but is not
powered on.
⚫Blinking green at 2 Hz: The OCP 3.0 network