Broadcom Stingray PS225 User manual

Broadcom Confidential 5880X-PS225-UG101
December 12, 2018
Stingray™ PS225
Quick Start Guide
User Guide
Broadcom Confidential

Broadcom, the pulse logo, Connecting everything, Stingray, TruFlow, Avago Technologies, Avago, and the A logo are among
the trademarks of Broadcom and/or its affiliates in the United States, certain other countries, and/or the EU.
Copyright © 2018 Broadcom. All Rights Reserved.
The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, please visit www.broadcom.com.
Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability,
function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does
not assume any liability arising out of the application or use of this information, nor the application or use of any product or
circuit described herein, neither does it convey any license under its patent rights nor the rights of others.
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
3
PS225 User Guide Quick Start Guide
Table of Contents
1 Introduction ..................................................................................................................................................................4
1.1 Block Diagram......................................................................................................................................................4
1.2 Board Features ....................................................................................................................................................5
1.3 Package Contents................................................................................................................................................6
1.4 PS225 Card Variants ...........................................................................................................................................7
2 Checklist .......................................................................................................................................................................8
3 Initial Board Setup and Connectivity .......................................................................................................................10
3.1 PS225 Connector Locations ..............................................................................................................................10
3.2 Installing the PS225 ...........................................................................................................................................10
3.3 Setting Up the Serial Console (Optional)...........................................................................................................11
3.4 Accessing the PS225 from an x86 Host ............................................................................................................12
3.4.1 Installing L2 Driver on the x86 Host.........................................................................................................12
3.4.2 Assigning an IP Address to the First New Host Interface........................................................................13
3.4.3 Establishing an SSH Connection to the PS225.......................................................................................16
4 Upgrading the PS225 Software.................................................................................................................................17
5 Setting Up a Linux Distribution rootfs on the PS225..............................................................................................19
5.1 Running Ubuntu on a PS225 .............................................................................................................................19
5.1.1 Setting Up Ubuntu on a PS225................................................................................................................19
5.2 Running CentOS on a PS225............................................................................................................................25
5.2.1 Setting Up CentOS on the PS225 ...........................................................................................................25
Revision History............................................................................................................................................................ 30
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
4
PS225 User Guide Quick Start Guide
1 Introduction
Broadcom®PS225 SmartNIC adapters, based on Broadcom's latest data center SoC technology, offer groundbreaking
network and compute power in a compact package. The PS225 adapters are PCI Express low-profile, half-length adapters,
and provide two 25-Gigabit Ethernet (25GbE) ports along with eight 64-bit ARMv8 processors running at an unmatched
3.0 GHz clock rate. The adapters are available in a range of on-board DDR4 memory sizes. The PS225-H04, PS225-H08,
and PS225-H16 adapters include 4 GB, 8 GB, and 16 GB of DDR4 memory, respectively.
1.1 Block Diagram
Figure 1 shows the main functional blocks of the PS225 SmartNIC adapter cards.
Figure 1: PS225 Block Diagram
PCIe 3.0
CPU
Subsystem
Ethernet 25GbE SerDes
DDR4
Ch. 1
8L
72b
L3$
DDR
x16
DDR
x16 DDR x16
DDR
x16
DDR
x16
SPI
8 MB
SMBus
NC-SI 20-pin
connector
PCIe Edge Connector
BCM58802H DDR4
Ch. 0
DDR
x16
DDR
x16 DDR x16
DDR
x16
DDR
x16
eMMC
16 GB
UART 3.5 mm
connector
Accelerators
72b
VPD
FRU
SFP28
connector
SFP28
connector
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
5
PS225 User Guide Quick Start Guide
1.2 Board Features
PCIe Interface:
– x8 PCI Express 3.0 compliant
– Single-Root I/O Virtualization (SR-IOV) with up to 128 VFs
– Function Level Reset (FLR) support
Processing Subsystem:
– 8-core ARMv8 A72 64-bit processor subsystem at 3.0 GHz
– 16 MB total cache (8 MB L2 + 8 MB L3)
– Two channels (64-bit + ECC) of DDR4 memory at 2,400 MT/s
Hardware Accelerators:
– TruFlow™ configurable flow accelerator engine
– Line-rate Crypto engine with single-pass hashing and encryption
– RAID 5/6 engine
Network Interface:
– Dual-port SFP28 pluggable media interface, supporting 25GbE or 10GbE optical transceiver or direct-attach copper
(DAC) cables
– RDMA over Converged Ethernet (RoCE) v1 and v2
– Advanced congestion avoidance
– Virtual network termination – VXLAN, NVGRE, Geneve, GRE
– Multiqueue, NetQueue, and VMQ
– Tunnel-aware stateless offloads:
IPv4 and IPv6
TCP, UDP, and IP checksum
Large send offload (LSO)
Large receive offload (LRO)
TCP segmentation offload (TSO)
Receive-side scaling (RSS)
Transmit-side scaling (TSS)
VLAN insertion/removal
– DCB support: PFC, ETS, QCN, DCBx
– Jumbo frames (up to 9 KB)
– Network boot (PXE, UEFI)
Security and Manageability:
– Secure Boot
– Secure Key Storage
– ARM PKA Engine
– NC-SI over MCTP (SMBus, PCIe VDM)
– NC-SI over RMB (via separate connector)
Form Factor:
– PCI Express CEM specification for half-height, half-length adapters
– I/O brackets available in both low profile and full height
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
6
PS225 User Guide Quick Start Guide
1.3 Package Contents
The PS225 package contains the following components:
PS225 adapter with low-profile bracket installed
Full-height bracket
The PS225 card with a low-profile bracket installed is shown in Figure 2.
Figure 2: PS225 with Low-Profile Bracket
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
7
PS225 User Guide Quick Start Guide
1.4 PS225 Card Variants
The PS225 card comes in the following three variants based on the amount of DDR memory:
PS225-H04 (4 GB DDR4, part number BCM958802A8024C/BCM958802A8044C)
PS225-H08 (8 GB DDR4, part number BCM958802A8028C/BCM958802A8048C)
PS225-H16 (16 GB DDR4, part number BCM958802A8021C)
The back of the card contains a sticker that indicates the board type. Figure 3 shows an example of the 4G card label on a
preproduction adapter.
Figure 3: Example Label

Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
8
PS225 User Guide Quick Start Guide
2 Checklist
The following list of major steps must be completed to complete the setup of the PS225:
1. Install the PS225 card in a host machine (see Initial Board Setup and Connectivity).
2. Obtain the documentation listed in Table 1.
3. Install the latest binary/image package for the PS225. Upgrade if the PS225 card received has an old image.
NOTE: To check the image version information, refer to docSAFE (the Broadcom documentation portal) to download the
binary package required for the specific card type.
Package name format : NXS_SN-<release type>_<release version>_<card type>_bin.tar.gz
Example: NXS_SN-BM-Beta_1.1.8.0_H08GB_bin.tar
4. Become familiar with the following PS225 basic information:
– PS225 software version:
On PS225 Stingray (MAIA) console:
root@bcm958802a8048c:~# uname -a
Linux bcm958802a8048c 4.14.79+g5bb1623 #1 SMP Tue Nov 27 01:48:50 UTC 2018 aarch64
aarch64 aarch64 GNU/Linux
– PS225 board/card type (8 GB, 4 GB, 16 GB):
On PS225 Stingray (MAIA) console:
root@bcm958802a8048c:~# hostname
bcm958802a8048c
– Check the Nitro profile configuration.
NOTE: By default, the card is upgraded with 2 x 25G port mode and 8 + 8 PFs profile configuration with eight PFs
on the Stingray (MAIA) side and eight PFs on the host (x86) side.
Table 1: List of Reference Documentation
Document name Document number Description
SmartNIC Configuration Guide 5880X-PS225-AN1Xx Application note for configuring OVS+DPDK on the
PS225.
BCM5880X High Performance Ethernet System on Chip 5880X-DS1Xx BCM5880X Data Sheet
BCM5880X Hardware Design Guide 5880X-DG1Xx Hardware design guide
BCM5880X SmartNIC Solution 5880X-UG3Xx BCM5880X SmartNIC solution
PS225 SmartNIC Adapters Dual-Port 25 Gb/s PCI
Express 5880X-PS225-DS1Xx PS225 Advanced Data Sheet
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
9
PS225 User Guide Quick Start Guide
Supported Nitro profile configurations in the PS225 released image package are as follows:
stingray_4p.pkg
stingray_2p.pkg
stingray_1p.pkg
BCM958802A8048C.pkg
bcm958802a8048_8+8_pf_rdma.pkg
bcm958802a8048_2xAN.pkg
bcm958802a8048_2xAN_8+5_pf_rep.pkg
bcm958802a8048_2xAN_8+4_pf_rep.pkg
bcm958802a8048_2xAN_8+4_pf.pkg
bcm958802a8048_2xAN_8+3_pf.pkg
bcm958802a8048_2xAN_8+2_baremetal.pkg
bcm958802a8048_2x25g.pkg
bcm958802a8048_2x25g_8+5_pf_rep.pkg
bcm958802a8048_2x25g_8+4_pf_rep.pkg
bcm958802a8048_2x25g_8+4_pf.pkg
bcm958802a8048_2x25g_8+3_pf.pkg
bcm958802a8048_2x25g_8+2_baremetal.pkg
bcm958802a8048_2x10g.pkg
bcm958802a8048_2x10g_8+5_pf_rep.pkg
bcm958802a8048_2x10g_8+4_pf_rep.pkg
bcm958802a8048_2x10g_8+4_pf.pkg
bcm958802a8048_2x10g_8+3_pf.pkg
bcm958802a8048_2x10g_8+2_baremetal.pkg
bcm958802a8048_1xAN.pkg
bcm958802a8048_1x25g.pkg
bcm958802a8048_1x10g.pkg
Broadcom Confidential

Broadcom Confidential 5880X-PS225-UG101
10
PS225 User Guide Quick Start Guide
3 Initial Board Setup and Connectivity
This section describes the initial board setup and connectivity.
3.1 PS225 Connector Locations
There are two SFP28 ports and a UART console port for the ARM CPU subsystem located on the I/O panel of the board
(see Figure 4).
Figure 4: PS225 Connector Locations
NOTE: The UART console is accessible via the round 3.5 mm jack on the I/O panel. The interface uses TTL-level signaling
and can be accessed using a widely available USB-to-TTL cable of type TTL-232R-3V3-AJ (based on FTDI chip).
3.2 Installing the PS225
To install the PS225:
1. Power down the host system and remove the AC power.
2. Insert the PS225 card into a PCIe 3.0 slot (at least x8 size) in the host system.
3. Connect one end of the SFP cable to port 0 or port 1 on the PS225 (see Figure 5).
4. Connect the other end of the SFP cable to an Ethernet switch that supports 10G or 25G Ethernet (see Figure 5).
NOTE: Both Ethernet ports must be connected to link partners operating at the same speed: either both 25GbE or both
10GbE. The ports do not support mixing 25GbE and 10GbE links.
5. Optionally:
a. Connect the other port of the PS225 card to an Ethernet switch (see Figure 5).
b. Insert the 3.5 mm plug of a USB serial cable to the 3.5 mm socket on the front panel (see Figure 5).
ARM A72
UART Console Port 0 Port 1
Broadcom Confidential