CERN PXIeCOMe User manual

PXIeCOMe
Production Test Suite
User Manual
Revision 1.1

2
Revision Table
Revision
Date
Author
Comments
1.0
19/09/2022
INCAA Computers BV
Initial version
1.1
15/12/2022
INCAA Computers BV
Added BIOS update description and procedures for
the COMe-bTL6 with ATM as preferred BIOS.
Updated PXIeCOMe board image to a picture of
EDA-04509-V1-0.
Added a section regarding jumper SW1.

3
Table of contents
Revision Table........................................................................................................................... 2
Introduction................................................................................................................................ 5
STANDARDS AND DOCUMENTS............................................................................................. 6
COM Express......................................................................................................................... 6
PXI and PXI Express.............................................................................................................. 6
CompactPCI and CompactPCI Express ................................................................................. 6
SATA and mSATA.................................................................................................................. 6
INSTALLATION ......................................................................................................................... 7
Installing Linux CentOS Stream 8........................................................................................... 7
Installing remote desktop........................................................................................................ 8
Update kernel and packages.................................................................................................. 8
Installing development packages............................................................................................ 9
Installing Kontron GPIO and i2c ............................................................................................. 9
Installing Python....................................................................................................................10
Installing the PXIeCOMe PTS ...............................................................................................10
Installing Xilinx Vivado...........................................................................................................10
Installing BIOS files ...............................................................................................................11
CERN preferred COMe-bTL6 (with ATM)...........................................................................12
COMe-bTL6 (without ATM)................................................................................................12
COMe-bCL6.......................................................................................................................12
TEST: PRE-CHECKS ...............................................................................................................13
Assembly...............................................................................................................................13
Visual inspection ...................................................................................................................13
Short circuit check.................................................................................................................13
TEST: PREPERATION.............................................................................................................14
Auxiliary equipment...............................................................................................................14
Initial test setup .....................................................................................................................14
Lithium battery and mSATA SSD.......................................................................................14
PXIeCOMe in slot 1 and PXCT in slot 2.............................................................................15
PXCT in system timing slot................................................................................................15
LAN cable..........................................................................................................................16
USB mouse and keyboard.................................................................................................16
USB Serial to PXCT in system timing slot..........................................................................16
USB to Xilinx download cable ............................................................................................17
USB3.0 memory sticks.......................................................................................................17
SMB trigger cable to PXCT in slot 2...................................................................................17
Serial to PXCT in slot 2......................................................................................................18
DisplayPort ........................................................................................................................18
TEST: BIOS UPDATES ............................................................................................................19
BIOS update..........................................................................................................................19
Updating CERN preferred COMe-bTL6 (with ATM)...............................................................19
Updating COMe-bTL6 (without ATM) ....................................................................................19

4
Updating COMe-bCL6...........................................................................................................20
Changing BIOS .....................................................................................................................21
THE TESTS..............................................................................................................................22
Reset Test.............................................................................................................................23
PTS Tests .............................................................................................................................23
Test00 –Setup ..................................................................................................................24
Test01 –Check SATA/mSATA and LEDs..........................................................................24
Test02 –Check DisplayPort...............................................................................................24
Test03 –Check LAN..........................................................................................................24
Test04 –Check USB .........................................................................................................25
Test05 –Check JTAG........................................................................................................26
Test06 –Check Serial (COM)............................................................................................27
Test07 –Check GPIO........................................................................................................27
Test08 –Check I2C and SMBus........................................................................................27
Test09 –Check PCI Express.............................................................................................27
Test10 –Check PXI...........................................................................................................27
Test11 –Check TRIG ........................................................................................................27
End of the tests..................................................................................................................28
Test wrap up..........................................................................................................................28
mSATA SSD......................................................................................................................28
Lithium battery...................................................................................................................28
Log files retrieval.......................................................................................................................29

5
Introduction
This document describes the methods for the automatic testing, using the Production Test Suite
(PTS), of the PXIeCOMe (PXIe controller COM Express based carrier). It consists of the
instruction for the installation of Linux (CentOS Stream 8) which is used as the test environment,
including all necessary tools and modules. And also describes the tests which are intended to
check the functional operation of all the interfaces and devices as part of the PXIeCOMe v0
prototypes.
Although the PXIeCOMe is designed as carrier board for any COM Express basic type 6
processor boards, during the tests the PXIeCOMe is equipped with a Kontron COMe-bCL6. The
complete assembly of the PXIeCOMe carrier board with the attached COMe-bCL6, heatsink
and front is referred to as the PXIeCOMe.
More information regarding the PXIeCOMe can be found here:
https://www.ohwr.org/project/pxie-ctl-comexpress/wikis/home
•Compliance with COM Express basic Pin-out type 6
•Compliance with PXIe standard for 3U system slot
•PCIe lane designed to meet PCIe GEN 3 specification
•16x PCIe lanes routed to the 4 link configuration PXIe backplane connector
•PXI trigger controller (Xilinx XC7A50T-1FTG256C FPGA)
•Possibility to install a full size mSATA SSD (supports mini mSATA)
•1x RS232 port DSUB9 connector
•4x USB 2.0
•2x USB 3.0
•1x 10/100/1000 Ethernet LAN
•1x DisplayPort
•1x SMB PXI trigger line

6
STANDARDS AND DOCUMENTS
COM Express
The COM Express specification defines a family of Computer On Module (COM) single board
computers. The requested COM Express Basic form factor and type 6 pin out have been
defined in PIGMG COM.0 R2.0 and R3.0.
The standard is available via https://www.picmg.org/.
The freely available COM Express design Guide Rev 2.0 is also available at this site:
https://www.picmg.org/wp-content/uploads/PICMG_COMDG_2.0-RELEASED-2013-12-061.pdf
PXI and PXI Express
The PXI and PXI Express specifications are created by the PXI Systems Alliance and are
defined in “PXI-1 Hardware Specification Rev. 2.3” and “PXI-5 PXI Express Hardware
Specification Rev. 1.1”.
The standards are available for download at http://www.pxisa.org/.
The PXI standard defines the interface for the extension of CompactPCI for instrumentation
applications. The PXI Express standard adds an extra set of features to be used within
CompactPCI Express.
CompactPCI and CompactPCI Express
The CompactPCI (PICMG 2.0 R3.0) and CompactPCI Express (PICMG EXP.0 R2.0) standards
are, like COM Express, created by PIGMG and are available via https://www.picmg.org/.
SATA and mSATA
The specification of SATA and the mSATA module (Serial ATA Revision 3.4) is created by the
Serial ATA International Organization and is available via https://sata-io.org/. The mSATA
module is part of the specification as of ‘Serial ATA Revision 3.1’.

7
INSTALLATION
Installing Linux CentOS Stream 8
The mSATA SSD, which will be used to test the PXIeCOMe, should be prepared by installing
Linux CentOS Stream 8 on it. For this a validated PXIeCOMe board can be use, but when no
validated PXIeCOMe boards are available a COM Express type 6 carrier board can also be
used. When the carrier board isn’t mSATA capable (like with the Kontron COMe Eval Carrier
T6) the mSATA SSD should be connected via a SATA to mSATA converter, or a USB to
mSATA converter.
1. Download CentOS Stream 8 DVD ISO version, like CentOS-Stream-8-x86_64-20220816-
dvd1.iso
2. Use a program like balenaEtcher (Portable) to create a bootable image of this ISO on a USB
stick.
3. Prepare the COM Express carrier so that is can be used for the installation of Linux. So
attach power supply, keyboard, mouse, display and the network.
4. Insert the mSATA SSD in the mSATA connector, or use a SATA to mSATA converter, or a
USB to mSATA converter to make the SSD part of the system.
5. Connect the CentOS Stream 8 ISO USB stick to a USB 3.0 port of the COM Express carrier
and boot the system.
6. Change the BIOS setup of the COMe-bCL6 or COMe-bTL6 so that the USB stick is used as
boot device.
7. The CentOS Stream 8 installation process is booted and now some configuration should be
made
•Welcome
•Select a language like English and English (United States).
•System
•Installation Destination: select the mSATA SSD and reclaim space when asked.
•KDUMP: do not change
•Network & Host Name: Enable the Ethernet connection and change the Host Name
to an appropriate value, like PXIeCOMe-test.
•Security Policy: do not change
•Localization
•Keyboard: select an appropriate keyboard
•Language Support: select the appropriate language support.
•Time & Date: select the appropriate region, and enable ‘Network Time’.
•Software
•Installation Source: do not change (Local media).
•Software Selection: Workstation.
•User settings
•Root Password: do not change
•User Creation: enter the user name ‘user’ and the password ‘baraka’ and check
both ‘Make this user administrator’ and ‘Require a password to use this account’.

8
8. Begin installation…
9. Reboot the system
10. Accept the license agreement
11. Finish the configuration…
12. Login…
Installing remote desktop
To access the installation remotely it is possible to control the PXIeCOMe from a different
Windows PC.
See also: https://www.enlinux.com/enable-remote-desktop-on-centos/
$ sudo yum install epel-release
$ sudo yum install xrdp -y
$ sudo systemctl enable xrdp
$ sudo systemctl start xrdp
$ sudo nano /etc/xrdp/xrdp.ini
Add at the end of the file the following line: exec gnome-session
Ctrl-O (Write Out)
Ctrl-X (Exit)
$ sudo systemctl restart xrdp
Change the firewall to provide access using remote desktop
$ sudo firewall-cmd --new-zone=xrdp -–permanent
$ sudo firewall-cmd --zone=xrdp –-add-port=3389/tcp –-permanent
Make it accessible from all ip-addresses with the 192.168.x.x range:
$ sudo firewall-cmd --zone=xrdp –-add-source=192.168.0.0/16 –-permanent
$ sudo firewall-cmd –-reload
The remote desktop access only works when no login session is active on the main display.
Update kernel and packages
To update the kernel and the installed packages to the latest version of CentOS Stream 8 the
following instructions were used:
$ sudo yum update -y kernel
$ sudo yum update

9
After restart of the system the new kernel can be used.
Installing development packages
To be able to make, compile and install the drivers several other packages need to be installed
first.
$ sudo yum install make
$ sudo yum install gcc
$ sudo yum install kernel-devel
$ sudo yum install elfutils-libelf-devel
Installing Kontron GPIO and i2c
To be able to control the COM Express specific GPIO and I2C interfaces the Kontron Linux PLD
Driver needs to be downloaded from the Customer Section at www.kontron.com. Version r34
was downloaded (kempld-drivers.34.tar.gz), which is applicable for many versions of Linux,
and so the driver for the right Linux kernel version has to be used.
$ uname -r
4.18.0-408.el8.x86_64
And so the source from the kempld-drivers.34/kempld-modules-linux_v4.18 has to be used.
When compiling the sources for the kempld drivers for the v4.18 kernel as is, will result in some
compile errors. To prevent these errors the source has to be updated by simply copying the
kempld-core.c file from a more recent kernel version implementation of the driver.
Now go to the kempld-modules-linux_v4.18 directory
$ cd Downloads/kempld-drivers.34/kempld-modules-linux_v4.18
$ cp ../kempld-modules-linux_v5.9/kempld-core.c .
And make and install the drivers:
$ sudo make
$ sudo make install
During the make install some SSL errors could be reported, but these can be ignored.
After restart of the system the kempld drivers are available and can be used.
To be able to access the I2C and SMBus ports the following package need to be installed.

10
$ sudo yum install i2c-tools
Installing Python
Since the PTS uses python2 it should be installed since CentOS comes standard with python3
only.
$ sudo yum install python2
$ sudo alternatives --set python /usr/bin/python2
$ sudo pip2 install pyserial
Installing the PXIeCOMe PTS
The PXIeCOMe PTS should be placed in the home directory of the user.
The pxiecome_pts_v1.0.tar.gz should be extracted in the home directory of the user.
$ tar -zxvf pxiecome_pts_v1.0.tar.gz
$ cd ~/pts
$ cdmod +x pxiecome.sh
It can now be started using:
$ ./pxiecome.sh
Installing Xilinx Vivado
For some of the tests an installation of Xilinx Vivado (Labtools only) is needed. This will be used
to test and program the FPGA of the PXIeCOMe.
Download the Vivado Labtools from the Xilinx website. For the PTS the
Xilinx_Vivado_Lab_Lin_2021.1_610_2318.tar.gz is used. This file is downloaded in the
~/Downloads directory. Extract this tar.gz-file here, which can be done using the Files
application and right-clicking on the tar.gz-file and select Extract Here.
Open a Terminal window and start the installation (do not use sudo):
$ sudo mkdir /home/Xilinx
$ sudo chmod 777 /home/Xilinx
$ sudo ln -s /home/Xilinx /opt/Xilinx
$ cd Downloads/Xilinx_Vivado_Lab_Lin_2021.1_610_2318

11
$ ./xsetup
•During installation disable the WebTalk feature:
•Accept the license agreements.
•Select the local installation directory within the home directory.:
•Install the tools.
$ cd
/opt/Xilinx/Vivado_Lab/2021.1/data/xicom/cable_drivers/lin64/install_script/install_d
rivers
$ sudo ./install_drivers
$ sudo ln -s /lib64/libtinfo.so.6.1 /lib64/libtinfo.so.5
In this way the Vivado Labtools and the download cable drivers have been installed.
•Restart the system.
Installing BIOS files
The PCI Express backplanes are defined in two types of configurations (See also PXI-5 PXI
Express Hardware Specification Rev. 1.1 section 4.10.2).
•2 link configuration: 1x PCIe x8 (8 lanes) link plus 1x PCIe x16 (16 lanes) link.
•4 link configuration: 4x PCIs x4 (4 lanes) link.
The PXIeCOMe is designed to interface with a PXI Express backplane using the 4 link
configuration only.
The COMe-bCL6 and the COMe-BTL6 are 4 link configuration capable but needs a BIOS
update to BCL6R113_2x4.bin for the COMe-bCL6 and BTL6R901_2x4.bin or BTL6R110_2x4.bin

12
for the COMe-bTL6. This BIOS file defines the PCIe lanes 0 to 7 as two 4 lane links. This BIOS
file can be found in the Kontron Customer Section on their site or will be provided by CERN.
CERN preferred COMe-bTL6 (with ATM)
To prepare the update of the BIOS of the COMe-bTL6 to BTL6R901_2x4.bin or do the following:
•The BTLR901.zip file contains the BTL6R901_2x4 directory. Copy this directory with content to
the root directory of both USB 3.0 sticks which will be used during the PTS tests.
COMe-bTL6 (without ATM)
To prepare the update of the BIOS of the COMe-bTL6 to BTL6R110_2x4.bin do the following:
•The BTLR110.zip file contains the BTL6R110_2x4 directory. Copy this directory with content to
the root directory of both USB 3.0 sticks which will be used during the PTS tests.
COMe-bCL6
To prepare the update of the BIOS of the COMe-bCL6 to BCL6R113_2x4.bin do the following:
•The BCLR113.zip file contains the BCL6R113_2x4 directory. Copy this directory with content to
the root directory of both USB 3.0 sticks which will be used during the PTS tests.

13
TEST: PRE-CHECKS
Before starting the test procedures, it is needed to wear an antistatic wrist band to avoid
electrostatic issues when handling the boards and the cables.
Assembly
The PXIeCOMe will be subject to the tests in a fully assembled state including a Kontron
COMe-bCL6 or COMe-bTL6 board.
Visual inspection
No special test setup is required since the tests to be carried out are visual inspections
and analysis of the board.
•Check the module for completeness.
•Check the module for the solder quality.
•Check the mounting of the mezzanine, front, COMe module and heatsink.
Short circuit check
Every power supply test point should be checked to verify there are no short circuits present
between these power supply rails and GND.
TP4:+5V
TP3: +3.3V
TP6: GND
TP5: +12V
TP14: +1.0V
TP13: +1.8V
TP8: +5.0V (Aux)
TP9: +3.3V (Aux)

14
TEST: PREPERATION
Auxiliary equipment
To be able to perform the tests a set of auxiliary cables and equipment is needed, which
consists of the following list:
•NI PXIe-1075, 18-Slot PXI Express chassis.
•INCAA Computers CPE-PXCT-19916, PXi Communication Tester, for Peripheral Slot.
•INCAA Computers CPE-PXCT-19917, PXi Communication Tester, for System Timing Slot.
•A display with a DisplayPort interface.
•Full size mSATA SSD containing the CentOS Stream 8 and PTS installation.
•USB mouse
•USB keyboard
•2x USB 3.0 capable USB stick, like a SanDisk Ultra Flair, or equivalent sized USB stick.
These should contain the BIOS update files.
•Xilinx Platform cable USB II (or equivalent) to be controlled using Xilinx Vivado (Labtools).
•USB-Serial adapter, like a Eminent EM1016, Prolific PL2303, or equivalent.
•2x RS232 DSub9 male to female cable.
•A CAT-5A or CAT-6 Ethernet cable attached to the LAN and internet.
•DisplayPort cable
•SMB to SMB coaxial cable.
Initial test setup
Power supply jumper
Remove jumper SW1 if present.
When jumper SW1 is placed the power supply of the PXIe chassis is switch on independent of
the presence or state of the COMe module.
Jumper SW1

15
Lithium battery and mSATA SSD
Insert the lithium battery in its socket and place the mSATA in its socket.
PXIeCOMe in slot 1 and PXCT in slot 2
Insert the PXIeCOMe in slot 1 of the
NI PXIe-1075 chassis.
Insert the CPE-PXCT-19916 (for
Peripheral Slot) in slot 2.
PXCT in system timing slot
Insert the CPE-PXCT-19917 (for
System Timing Slot) in the system
timing slot, which is slot 10 of the
PXIe-1075.
mSATA SSD
3V Lithium battery

16
LAN cable
Insert the LAN cable in the RJ45
socket indicated with LAN.
USB mouse and keyboard
Insert the USB mouse and USB
keyboard in the two USB2.0
connectors next to the RJ45 socket
and indicated with USB.
USB Serial to PXCT in system timing slot
Insert the USB Serial
adapter cable in the
PXIeCOMe USB port, and
connect it via a serial cable
to the PXCT in the system
timing slot.

17
USB to Xilinx download cable
Insert the Xilinx download cable in the USB
port and connect the flat cable, through the
opening in the PXCT, to the J3 (JTAG) on the
PXIeCOMe board.
USB3.0 memory sticks
Insert the two USB3.0 memory
sticks in the slots indicated with USB
(SS).
SMB trigger cable to PXCT in slot 2
Insert the SMB to SMB cable between
the PXIeCOMe TRIG port and the
PXCT TRIG port.

18
Serial to PXCT in slot 2
Insert the serial cable in the
PXIeCOMe, and connect it to the
PXCT in slot 2.
DisplayPort
Insert the DisplayPort cable, which
should be connected to the display, in
the port indicated with DP.

19
TEST: BIOS UPDATES
BIOS update
The BIOS of every COMe module (Kontron COMe-bCL6 or Kontron COMe-bTL6) which is
assembled on a PXIeCOMe carrier and shipped as part of the PXIeCOMe assembly should be
updated.
Updating CERN preferred COMe-bTL6 (with ATM)
To update the BIOS to BTL6R901_2x4.bin do the following:
Start the system and select the System setup to enter the BIOS environment. When the system
starts in EFI shell mode type exit to leave the shell and select System setup to enter the BIOS
environment.
•Under ChipSet | PCH-IO Configuration | Security Configuration change the BIOS Lock
to Disabled.
•Under Advanced | PCH-FW Configuration | Firmware Update Configuration change the
ME FW Image Re-flash to Enabled.
•Under Boot change the Boot Option #1 to UEFI: Built-in EFI Shell.
•Under Save & Exit select Save Changes and Exit so the system will boot to the EFI shell.
•Check to see if the USB stick can be accessed as fs0 or fs1. So use
> fs0:
or
> fs1:
and
> ls
to see which disk contains the BTL6R901_2x4 directory.
•And enter this directory.
> cd BTL6R901_2x4
•Update the BIOS using
> flash.nsh
•Perform a shutdown of the system
> reset -s
•Turn the power supply of the PXIe chassis off, so that also the standby power is turned off.
This is needed to use the new BIOS when powered on.
Updating COMe-bTL6 (without ATM)
To update the BIOS to BTL6R110_2x4.bin do the following:
Start the system and select the System setup to enter the BIOS environment. When the system
starts in EFI shell mode type exit to leave the shell and select System setup to enter the BIOS
environment.
•Under ChipSet | PCH-IO Configuration | Security Configuration change the BIOS Lock
to Disabled.

20
•Under Advanced | PCH-FW Configuration | Firmware Update Configuration change the
ME FW Image Re-flash to Enabled.
•Under Boot change the Boot Option #1 to UEFI: Built-in EFI Shell.
•Under Save & Exit select Save Changes and Exit so the system will boot to the EFI shell.
•Check to see if the USB stick can be accessed as fs0 or fs1. So use
> fs0:
or
> fs1:
and
> ls
to see which disk contains the BTL6R110_2x4 directory.
•And enter this directory.
> cd BTL6R110_2x4
•Update the BIOS using
> flash.nsh
•Perform a shutdown of the system
> reset -s
Turn the power supply of the PXIe chassis off, so that also the standby power is turned off. This
is needed to use the new BIOS when powered on.
Updating COMe-bCL6
To update the BIOS to BCL6R113_2x4.bin do the following:
•Start the system and select the System setup to enter the BIOS environment. When the
system starts in EFI shell mode type exit to leave the shell and select System setup to enter
the BIOS environment.
•Under ChipSet | PCH-IO Configuration | Security Configuration change the BIOS Lock
to disabled.
•Under Boot change the Boot Option #1 to UEFI: Built-in EFI Shell.
•Under Save & Exit select Save Changes and Exit so the system will boot to the EFI shell.
•Check to see if the USB stick can be accessed as fs0 or fs1. So use
> fs0:
or
> fs1:
and
> ls
to see which disk contains the BCL6R113_2x4 directory.
•And enter this directory.
> cd BCL6R113_2x4
•Update the BIOS using
> flash.nsh
•Perform a shutdown of the system
> reset -s
•Turn the power supply of the PXIe chassis off, so that also the standby power is turned off.
This is needed to use the new BIOS when powered on.