Cubix Xpander Desktop Elite User manual

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 1
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Xpander Desktop Elite
Gen 3 User Guide
Xpander Desktop Elite (see the image at the
right) is a portable PCI Express (PCIe)
expansion enclosure that enables connection of
graphics processing units (GPUs) or other
controllers to a host computer. The host must
provide one PCIe x16 expansion slot available
for the host interface controller (HIC).
Install the x16 HIC and cable in a PCIe Gen 3
x16 slot to provide a theoretical 128Gbps
connection to Desktop Elite with four slots.
Desktop Elite provides four PCIe Gen 3 x16
slots on one mainboard for double-wide GPUs
that are NVIDIA® Compute Unified Device
Architecture (CUDA) or OpenCL-enabled.
Desktop Elite supports other PCIe adapters as
well. It supports applications that exploit
multiple GPUs, including rendering graphics or
high-performance computing (HPC).
Desktop Elite standard version features a 1,500Watt power supply, 4 x 8P + 8P auxiliary power
connectors and a fan that cools 4 x double-wide, actively-cooled GPUs. Optional versions
feature either a 1,600Watt power supply or a 1,200Watt power supply with 4 x 8P + 6P auxiliary
power connectors and a fan that cools 4 x double-wide, actively-cooled GPUs or other PCIe
cards. Adjustable fan settings (see page 8) allow quieter operation if fewer GPUs or single-wide
GPUs or other PCIe cards are installed.
Xpander Desktop Elite Technical Specifications
Expansion slots, x16 std
PCI Express x16 mechanical / electrical on 1 Main Interface Boards (MIB)
Number of slots / MIB
4 slots / 1 MIB
Form factor support
4 double-wide, full-length, full-height controllers / adapters
PCIe device support
PCIe cards must adhere to PCIe specifications for power and dimensions:
maximum 250Watts and 4.38in (111.15mm) H x 10.51in (267.00mm) L
Connecting cable
External PCI Express x16, 136-pin
Connector duty cycle
250 mating cycles maximum per connector
Length, standard
2.00meters (78.74inches or 6.56feet)
Length, optional
1.00meters (39.37inches or 3.28feet)
PCIe x16 HIC
Install in a PCIe x16 slot in the host computer
In host PCIe Gen 3 slot
Provides theoretical 128Gbps data transfer
Front panel LCD
Main menu = left button; sub-menu = right button (See detail on page 7.)
Cubix Corporation
MIB assembly and revision; management firmware revision
Power supply status
Standard main power; optional redundant power
Temperature status
3 sensor readouts in degrees Centigrade (C)
Fan status
Fan status, voltage percent (%) speed
PCIe link status
x16 cable connected; 8.0Gbps x16 link speed
x16 cable
Connected or not connected status

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 2
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Slot 1 - 4 x16 transfer
rate, theoretical
PCIe Gen 3, 8.0Gbps x16 = 128.0Gbps
LCD panel backlight
Toggle on / off using right button
Rear panel
Power input receptacle, PCIe cables and cooling fan
Fan profile settings
Use Xpander internal jumpers JP7 and JP8 (see page 8)
Cooling fan profiles
1 x fan runs at 112cfm (cubic feet per minute) maximum
Performance
2 x double-wide GPUs in slots 1 and 4, plus 1 –2 x single-slot PCIe cards
Quiet
1 x double-wide GPU in slot 1 or 4 x single-wide GPUs
High Performance
2 –3 x double-wide GPUs, plus 1 –2 x single-slot PCIe cards
GPU
4 x double-wide GPUs
AC power supply
Standard version
Single-phase, auto-ranging, 100 - 240VAC, 15-7Amps, 50/60Hz; total
output power is not to exceed 1,500Watts
High-power option
Single-phase, auto-ranging, 115 - 240VAC, 17-10Amps, 50/60Hz; total
output power is not to exceed 1,600Watts
Low-power option
Single-phase, auto-ranging, 100 - 240VAC, 10-5Amps, 50/60Hz; total
output power is not to exceed 1,200Watts
Standard aux power
4 sets of connectors, each with 8pins + 8pins
Optional aux power
4 sets of connectors, each with 8pins + 6pins
Operating environment
0° to 30° Celsius (32° to 86° Fahrenheit) temperature
5% to 80% non-condensing humidity
Dimensions (no handle)
10.85" H x 8.50" W x 16.17" D (27.56cm H x 21.59cm W x 41.07cm D)
Dimensions (with handle)
12.60" H x 8.50" W x 16.17" D (32.00cm H x 21.59cm W x 41.07cm D)
Weight
23.00lbs (10.43kg)
Warranty / period
Parts & labor return to manufacturer / 2 years
Ext. warranty period
Optional 3 years
Included
Desktop Elite HE standard version includes a
cooling fan, a 1,500W power supply with 4 x
8Pin + 8Pin (8P + 8P) auxiliary (aux) power
cables and a power cord as well as Main
Interface Board (MIB), one PCIe x16 HIC and
one external PCIe x16 cable, 1meter long.
Desktop Elite includes installation media
because installation requires no driver or other
software. The only drivers you install are those
for the devices you install in the unit. Xpander
includes a two-year warranty.
Options
The optional LE version features a 1,200W
power supply with 4 x 8P + 6P aux power
cables. The standard external PCIe cable length
is 1meter. Optional 2m PCIe cable is also
available. Cubix offers an optional third-year
warranty on Xpander products.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 3
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Quick Start without Detail
This quick start is for users who have installed GPUs in Cubix Xpander products. If you have not
installed Xpander before, proceed to the Start-up / Test Procedure with Detail below.
Install a GPU in a host system PCIe x16 slot and install device drivers. Once you have verified
operation, repeat the process for other PCIe cards. After verifying operation for each type of
PCIe card, move the cards to Xpander, install the HIC in the same slot in the host computer and
connect Xpander to the HIC using the cable provided. Connect power to Xpander and power on
the host computer. Boot the host computer. If it is plug-n-play, the operating system (OS) will
detect the new hardware configuration and install device drivers.
Start-up / Test Procedure with Detail
Check your host computer motherboard vendor’s Website for device driver and firmware
updates, especially those that may affect multiple GPUs. During this procedure, place the
Xpander Desktop Elite on a stable surface. Use a Philips #2 screwdriver. Do not install graphics
or other controllers until instructed to do so.
1. Remove AC power to the host computer and to Xpander.
2. Install the x16 HIC in a PCIe x16 slot within the host computer. In Apple® Mac Pro, install
the x16 HIC in PCIe slot 2.
3. Remove the screws holding the top cover,
remove it and set it aside.
4. Connect the external PCIe cable coming from
the Xpander Desktop Elite into the HIC in the
host computer. Make sure the cable
connector seats firmly in the HIC port. You
will feel the cable connector click into place.
See the image of a properly-seated
connector in an Apple Mac Pro at the right.
5. Connect a power cord to the receptacle at
the enclosure rear.
6. Check the connection LED near the HIC
L-bracket in the host. If you have properly
seated the HIC in the host PCIe slot, you will
see the LED near the HIC L-bracket light
steady green.
7. Power on the host computer. Xpander
Desktop Elite also powers on.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 4
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
8. Confirm that exterior Link OK LED, the HIC Link LED, the interior Cable
(P0) LED are lit green and the unit fans are spinning.
Xpander is functioning properly once the Cable (P0)
LED lights green.
9. Switch off AC power to the host computer and disconnect power to Desktop Elite. Leave the
external PCIe cable to the host computer port connected.
10. Remove as many screws as necessary to install GPUs or other PCIe devices. For example,
if you have 2 x double-wide
GPUs, remove 4 x screws.
After installation, these
screws will hold down end
brackets of GPUs or other
PCIe devices. See the
image at right. If you have
an empty slot, install the 2 x
blank end-brackets provided
within the packaging.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 5
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
11. Install the graphics or other controller(s) in the
Xpander PCIe slot(s) and connect PCIe auxiliary
power cables (see the image at the right). Desktop
Elite with a 1200Watt provides four sets of aux power
connectors with 8P + 6P. Optional 8P + 8P aux
power connectors are available. Seat each set of
connectors properly in the GPU devices so the host
computer system and operating system (OS) can
detect and use the devices.
12. Connect GPU video port(s), if present, to monitors. This step is especially important for
Linux, because X Windows (X) will crash if you have not connected video ports to monitors.
If you cannot connect monitors to installed GPU video ports, you can edit the Device section
of /etc/X11/xorg.conf to define which GPU X should use as the primary display device. For
example:
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "Quadro M6000"
BusID "PCI:8:0:0"
EndSection
To determine the BusID for the GPU you wish to use as the primary display, run the scanpci
tool discussed in Step 14.
13. Connect power to Desktop Elite and power up the computer to confirm that the appropriate
slot link LEDs (P1, P5, P7 and P9 in the image above) are lit green, which confirms that you
have installed the controller(s) correctly in each slot.
14. For instance, in a PC running Microsoft® Windows®, check Windows Device Manager.
Within Windows, you can also run GPU-Z, a utility available for free download here. In an
Apple Mac Pro, run the System Profiler and look for GPUs under Graphics/Displays. To
open System Profiler, choose Apple Menu > About This Mac, and then click More Info. In a
PC running Linux, run the following:
root# > XFree86 –scanpci
Read more about scanpci here.
15. If all the Cable Link and slot LEDs are lit green but your OS does not detect the GPUs, shut
down the OS gracefully, power down GPU-Xpander and remove / replace the GPUs. Make
sure that you seat each GPU firmly in its slot and that you connect all auxiliary power
connectors properly. Switch on GPU-Xpander and boot up the host computer.
16. Install the appropriate graphics or other controller driver(s) using the vendor’s procedure.
Apple Mac OS X may not support natively all NVIDIA® GPU products.
Warning
Before installing or removing cards from the Xpander, shut down the host, disconnect AC
power from Xpander and wait for the 5V Standby LED to power off.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 6
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
17. Power down the host computer gracefully, which powers down Desktop Elite. Leave the
power cords and the external PCIe cable to the host computer port connected.
18. Replace the Xpander top cover with the vents away from the fan and secure the screws
holding the top cover in place. Use all of the screws included in the bag.
19. Switch on the host computer and both will boot and run.
Normal Power On/Off Procedure
1. With Xpander connected to the host computer and running, shut down the host normally.
2. With Xpander connected to the host computer and with power connected, switch on the
connected host; both will boot up.
3. To run the host PC or Mac Pro without Xpander connected, power down the host, remove
power from Xpander and boot the host. When you are ready to run Xpander again, power
down the host, connect power to Xpander and boot the host.
Front Panel LCD
Xpander Desktop Elite front panel
LCD provides enclosure details
and status. Beneath the LCD are
two buttons: left button scrolls
through main-menu items and
right button scrolls through sub-
menu items.
When you first connect power,
Xpander comes up in standby
mode. The LCD displays Cubix on
the top line and Corporation on
the bottom line. This is the Start
menu. Access all other main
menu items by pressing the left
button.
The upper right corner of the LCD
shows a flashing asterisk *. This
indicates that the on-board Micro-
Controller Unit (MCU) is running
properly. If the asterisk is not flashing or the LCD is not responding, reset the MCU by pressing
and holding the left button for 3 seconds while Xpander is running. A corresponding HB (for
heartbeat) LED on the MIB also shows the MCU is working.
When a fault occurs, the LCD will jump to that fault and the LCD backlight will flash on / off.
Press the left button to acknowledge the fault and the LCD will stop flashing and enable normal
Warning
Replace the cover as soon as possible to allow for proper cooling of controllers installed in
the unit and secure the cover in place with all of the screws provided. Do not run Xpander
without the cover for more than a few minutes.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 7
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
operation. The LCD will continue to display the fault at the applicable sub-menu until you clear
the fault.
Start Menu
Menu Sub-menu Sub-menu Sub-menu
Main menu: Cubix Corporation
Sub-menu: PCB Assembly
•Board Number
•EEPROM Revision
•Board Revision
Sub-menu: MCU (Micro-Controller Unit) Firmware
•Release Number
•Release Date
Sub-menu: Cubix Serial Number
Power Supply Status
Menu Sub-menu Sub-menu
Menu: Power Supply Status
Sub-Menu: Power Supply Status:
•PWR Standby = Main power is off; system is in standby
•PWR ON-Good = Main power is on; +3.3V, +5V, +12, -12V are within spec.
•PWR ON-FAULT = Main power is on; +3.3V, +5V, +12V, or -12V is out of spec.
Temperature Status
Menu Sub-menu
Menu: Temperature Status
Sub-Menu: Display the temperature of the 3 temperature sensors in degrees Celsius.
Note: If a temperature sensor fails, FLT (fault) appears under the failed sensor.
Fan Status
Menu Sub-menu Sub-menu
Menu: Fan Status
Sub-Menu: Indicates fan status for fan 1
Cubix
Corporation
c
Cubix
A09041 Rev 01 01
c
MCU Firmware
Rel 01 06-20-12
c
Power Supply
PWR ON-Good
c
Power Supply
Status
c
Temperature
Status
c
Tmp1 Tmp2 Tmp3
24C 25C 26C
c
Fan
Status
c
Fan 1
Running 7V 70%
c
Fan 2
Running 7V 70%
c
Serial Number
10190000
c
Power Supply
RPS ON-Good
c
IC Temperature
65C
c

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 8
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Note: Fan speed is controlled by adjusting fan voltage based on sensed system temperature. Fan voltage
adjusts from 6V min (25deg C or lower) at 50% to 12V max (50deg C or higher) at 100%. Fan voltage is
the same for all fans.
Status: PWR OFF is 0V at 00%; system is in standby.
Status: Running aaV at pp%; fan is good; aa = fan voltage and pp = fan power percent (%).
Status: STOPPED is 12V at 100%; fan fault: fan voltage is set to full on.
Status: NO FAN aaV at pp%; Xpander can be configured with 1, 2, 3 or 4 fans. No fan means fan
is not installed.
PCIe Link Status
Menu Sub-menu Sub-menu Sub-menu
Menu: PCIe Link Status
Sub-Menu: Indicates x16 PCIe cable installation status
•Connected x16 cable is properly connected to the HIC
•NOT-Connected x16 cable is not properly connected to the HIC
Sub-Menu: x16 PCIe cable link speed and link width. Speed and width depend on the host capability.
•Speed: 2.5Gb/s, 5.0Gb/s or 8.0Gb/s
•Width: x1, x2, x4, x8, x16 and “NO Link” when the host did not successfully link or Xpander is
in standby power.
Sub-Menu: Link speed and link width for slots 1-4.
•Speed: 2.5Gb/s, 5.0Gb/s or 8.0Gb/s
•Slot 1 - 4 width: x1, x2, x4, x8, and “NO Link” when the slot is empty, unsuccessful link, or
Xpander is in standby power.
LCD Backlight Control
Press right button to toggle the LCD backlight On / Off.
Fan Speed Control
Shunts JP7 and JP8 are used to change the fan profile to reduce fan noise. For both JP7 and
JP8, pin 1 is farthest from the board edge and pin 3 is closest to the board edge.
Fan Profile
JP7
JP8
*Tmax
*Tmin
Performance Mode
1-2
1-2
55C
20C
Quiet Mode
1-2
2-3
65C
30C
High Performance Mode
2-3
1-2
40C
5C
**GPU Mode
2-3
2-3
80C/40C
45C/5C
* Tmax - Fan Voltage ~12V, Tmin - Fan Voltage ~6V
**GPU Mode: Tmax: GPU temp = 80C, if no GPU is found Tmax = 40C (on-board temp sensors)
Tmin: GPU temp = 45C, if no GPU is found Tmin = 5C (on-board temp sensors)
GPUs supported: Quadro K6000, M6000 and P6000
Fan Profile Guidelines. GPU Mode supports 4 x double-wide GPUs like NVIDIA Quadro or GTX
Series. High-Performance Mode supports 2 –3 double-wide GPUs. Performance Mode
supports 2 x double-wide GPUs. Quiet Mode supports 1 x double-wide GPU or 4 x single-wide
GPUs. See Technical Specifications on page 2.
PCIe Link
Status
c
PCIe X16 Cable
Connected
c
PCIe Cable
8.0Gb/s X16
c
PCIe Slot n
8.0Gb/s X16
c

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 9
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Shipping Procedure
Before shipping Desktop Elite, remove any installed graphics or other controller(s) using the
following procedure:
1. Power down and switch off the host computer.
2. Remove the power cord from the rear power receptacle on Xpander.
3. Disconnect GPU-Xpander from the HIC.
4. Using a Philips #2 screwdriver, remove the screws securing the top cover.
5. Loosen the captured screws holding down the end brackets and remove the graphics or
other controllers.
6. Replace the hold-down bracket and tighten the captured screws.
7. Pack the graphics cards within the boxes in which you received them.
8. Replace the top cover and secure it in place using 4 screws. Put the remaining screws in the
plastic bag included.
9. Within the original packaging, pack the Xpander, the extra screws for the top cover, the
power cord, the host adapter and the boxed graphics cards. Include a copy of these
instructions.
Troubleshooting Procedure
Issue: The front-panel LCD asterisk does not flash.
Resolution: Reset the LCD’s MCU by pressing and holding the left button for 3 seconds while
Xpander is running.
Issue: The HB LED inside Xpander flashes green.
Resolution: This is normal; it indicates that the on-board MCU is running properly.
Issue: Cable LED inside Xpander does not light green.
Resolution: Check the following:
•Confirm that the HIC is firmly seated in a PCIe slot within the host computer.
•Confirm that the cable connector is firmly seated HIC connector port.
Issue: Cable LED inside Xpander flashes green even though I have a PCIe x16 HIC
and cable.
Resolution: The internal Cable LED should light steady green if the HIC is in a PCIe Gen 3.0
slot. If the Cable LED flashes, it is transferring data at a reduced bandwidth
because you installed the PCIe x16 HIC in a Gen 1 or Gen 2 slot or a PCIe x8
(electrical) slot. The HIC operates at full bandwidth when installed in a PCIe Gen
3 slot.
Issue: Slot LED inside Xpander does not light green.
Resolution: Shut down the OS gracefully, power down Xpander and reseat the GPUs in each
PCIe slot. Make sure that you seat each GPU firmly in its slot and that you
connect all aux power connectors firmly.
Issue: Slot LED inside Xpander flashes green even though I installed a PCIe Gen 3 x16
GPU or card.
Warning
Disconnect power to Xpander before connecting or disconnecting the HIC.

Cubix Corporation 2800 Lockheed Way Carson City, NV 89706-0719 Page 10
Sales 800.829.0550 http://www.cubix.com [e-mail] Xpander-desktop-elite-g3 20190201
Resolution: Slot LEDs should light steady green if a PCIe Gen 3 x16 GPU or other card is
installed. If they are flashing, it may be because the GPUs in the slots do not
have a processing load. Apply a processing load on the GPUs. If the slot LEDs
still flash green, check the vendor’s link for a driver update.
Issue: All Cable and Slot LEDs are lit green but your OS does not detect the GPUs.
Resolution: Shut down the OS gracefully, power down Xpander and reseat the GPUs in each
PCIe slot. Make sure that you seat each GPU firmly in its slot and that you
connect all aux power connectors firmly.
Issue: The front-panel LCD screen backlight is flashing on/off, which indicates a fault.
Resolution: Acknowledge the fault by pressing the Menu button, which will stop the LCD
flashing and the LCD will remain at the fault.
Issue: After acknowledging a fault, the front-panel LCD screen backlight continues
flashing on/off, which indicates two or more faults. If multiple faults have
occurred, when you acknowledge the first fault, the LCD will jump to the next
fault and the LCD backlight will once again flash on/off. This continues until you
clear all faults.
Resolution: Acknowledge the fault by pressing the Menu button, which will either stop the
LCD flashing and the LCD will remain at the fault, or the LCD will jump to the
next fault and the LCD backlight will once again flash on/off.
Issue: The front-panel LCD screen either does not respond or is blank even though
Xpander is operating.
Resolution: Press and hold the MIB Select button to reset the Micro-Controller Unit (MCU).
Issue: The front-panel LCD screen either does not respond or is blank for a particular
MIB even though Xpander and the MIB are both operating.
Resolution: Press and hold the Menu (left) button for 3 seconds to reset the Micro-Controller
Unit (MCU). If the Xpander is running, the Xpander fans will turn on full speed for
10 seconds and then slow down to the operating speed set by the MIB local
temperatures.