Dolphin IXS600 User manual

Dolphin PCI Express IXS600 Switch
IXS600 8 ports PCI Express x8 switch users guide
This guide is only valid for revisions IXS600-HI and newer
Version 2.05
Date: 20th April, 2016

IXS600 Users Guide –Dolphin Interconnect Solutions Page 1
Table of Contents
Table of Contents.......................................................................................................................................................................1
Overview....................................................................................................................................................................................4
Revision history..........................................................................................................................................................................5
Hardware revisions................................................................................................................................................................5
Management software revisions...........................................................................................................................................5
Configuration examples.............................................................................................................................................................6
8 node PCIe Gen2 and Gen3 NTB Cluster..............................................................................................................................6
20 node PCIe Gen2 NTB Cluster ............................................................................................................................................6
Transparent PCIe Gen3 expansion –up to 7 Expansion chassis connected to one server ...................................................6
Quick install guide......................................................................................................................................................................7
IXS600 operating modes .......................................................................................................................................................7
Cable connections .................................................................................................................................................................7
Transparent mode port usage...............................................................................................................................................7
NTB mode port usage............................................................................................................................................................8
Cascading switches................................................................................................................................................................8
Partitioning –mixed transparent and NTB usage .................................................................................................................8
PCI Express Gen3 Switch - IXS600..............................................................................................................................................9
IXS600 high level specification ..............................................................................................................................................9
Cables ..................................................................................................................................................................................10
Transition cable to lower lane width...................................................................................................................................10
Management processor ......................................................................................................................................................10
Use cases summary and settings.........................................................................................................................................11
LEDs .........................................................................................................................................................................................12
Management processor LEDs..............................................................................................................................................12
Switch status LEDs...............................................................................................................................................................12
Front and rear panel link LEDs ............................................................................................................................................13
Installation ...............................................................................................................................................................................14
Safety instructions...............................................................................................................................................................14
Installing the Switch in a rack..............................................................................................................................................14
Installing and removing the cable .......................................................................................................................................14
Connecting multiple switches..................................................................................................................................................15
Connecting up to 14 nodes using two switches ..................................................................................................................15
Connecting up to 20 nodes using three switches................................................................................................................16
Connecting up to 56 nodes for reflective memory .............................................................................................................16

IXS600 Users Guide –Dolphin Interconnect Solutions Page 2
Web GUI...................................................................................................................................................................................17
Authentication.....................................................................................................................................................................17
Switch IP configuration........................................................................................................................................................19
Configuring the proper time zone...................................................................................................................................20
Network configuration without the web interface.........................................................................................................21
Switch Monitoring ...............................................................................................................................................................23
Firmware management & upgrade .....................................................................................................................................24
Web GUI..........................................................................................................................................................................24
USB flash drive ................................................................................................................................................................25
The USB upgrade procedure ..............................................................................................................................................25
Link Watchdog.....................................................................................................................................................................27
Management processor logs...............................................................................................................................................28
Software installation................................................................................................................................................................29
Support ....................................................................................................................................................................................29
Compliance ..............................................................................................................................................................................30
EMC compliance..................................................................................................................................................................30
Safety compliance ...............................................................................................................................................................30
RoHS compliance.................................................................................................................................................................30
Flammability standard.........................................................................................................................................................30
WEEE Notice........................................................................................................................................................................30
Limited warranty .....................................................................................................................................................................31
Warranty period..................................................................................................................................................................31
Coverage..............................................................................................................................................................................31
Service procedure ...............................................................................................................................................................31
Limitations...........................................................................................................................................................................31
DISCLAIMER
DOLPHIN INTERCONNECT SOLUTIONS RESERVES THE RIGHT TO MAKE CHANGES WITHOUT FURTHER NOTICE TO ANY OF
ITS PRODUCTS TO IMPROVE RELIABILITY, FUNCTION, OR DESIGN. DOLPHIN INTERCONNECT SOLUTIONS DOES NOT
ASSUME ANY LIABILITY ARISING OUT OF THE APPLICATION OR USE OF ANY PRODUCT.
LIFE SUPPORT POLICY
DOLPHIN INTERCONNECT SOLUTIONS’ PRODUCTS ARE NOT AUTHORIZED FOR USE AS CRITICAL COMPONENTS IN LIFE
SUPPORT DEVICES.

IXS600 Users Guide –Dolphin Interconnect Solutions Page 3
ENVIRONMENTAL POLICY
Dolphin is minimizing the amount of printed documentation and software CDs in its shipments, please download
additional documentation and software from www.dolphinics.com/support/.

IXS600 Users Guide –Dolphin Interconnect Solutions Page 4
Overview
This document describes the Dolphin IXS600 PCI Express switch used in combination with other Dolphin PCI Express
produces. Dolphin is currently provides the following PCI Express products:
•PXH810 PCI Express Gen3 x8 NTB iPass cable adapter card
•PXH812 PCI Express Gen3 x8 Transparent Host and Target iPass cable adapter card
•PXH830 PCI Express Gen3 x16 NTB MiniSAS-HD cable adapter card
•PXH832 PCI Express Gen3 x16 Transparent Host and Target MiniSAS-HD cable adapter card
•IXH610 PCI Express Gen2 x8 NTB and Host iPass cable adapter card
•IXH611 PCI Express Gen2 x8 Host and Target iPass cable adapter card
•IXS620 XMC PCI Express Gen2 x8 NTB, Host and Target adapter card
•IXS600 8 port PCI Express Gen3 x8 rack mountable iPass cable switch
•IXE60X PCI Express expansion products (contact Dolphin for details)
•IXC0M 50cm PCIe Gen3 x8 Copper Cable
•IXC1M 1 Meter PCIe Gen3 x8 Copper Cable
•IXC2M 2 Meter PCIe Gen3 x8 Copper Cable
•IXC3M 3 Meter PCIe Gen3 x8 Copper Cable
•IXC5M 5 Meter PCIe Gen3 x8 Copper Cable
•IXC7M 7 Meter PCIe Gen2 x8 Copper Cable
The IXS600 switch can be used to connect to any of the above iPass cable based products or to any compliant existing PCI
Express downstream device having a standard PCI Express x8 iPass connector (PCI Express Gen 1 or Gen2 auto detect).
The IXS600-Hx revision and newer is PCI Express Gen3 ready but requires a firmware update to operate at Gen3 speeds
between IXS600 switches. The IXS600-HN revision is required to support the new PXH810 card at Gen3 speed.
All Dolphin PCI Express NTB adapter cards support the complete suite of Dolphin PCI Express Software including Dolphin
SuperSockets™, optimized TCP/IP drivers, and SISCI Embedded software. They are also compliant with the PXImc software
specification.
Dolphin SuperSockets is a Berkeley Sockets compliant library that provides a socket latency below 1 microsecond, and
close to the wire speed streaming bandwidth for networked applications. SuperSockets™is currently available on Linux,
RedHawk and Windows. It is a 100% transparent plug-and-play solution for commercial and embedded applications.
More on SuperSockets is located at http://www.dolphinics.com/products/dolphin-supersockets.html
Dolphin’s optimized TCP/IP driver enables PCI Express to be used as a traditional 10G Ethernet / 40G Ethernet
replacement such as for NFS sharing and legacy networking that does not need the low latency provided by SuperSockets.
The TCP/IP driver supports gateway functionality.
The SISCI software provides a well-defined, easy to use shared memory programming API for PCI Express over cable. More
on SISCI is located at http://www.dolphinics.com/products/embedded-sisci-developers-kit.html
More information about the software provided for the Dolphin PCI Express adapter cards is located at
http://www.dolphinics.com/products/dolphin_pci_express_software.html.

IXS600 Users Guide –Dolphin Interconnect Solutions Page 5
Revision history
The IXS600 switch has been undergoing a number of changes to adopt to new requirements and improved performance
made available through new revisions of the PCIe chipset.
Hardware revisions
The following table gives a general overview of the hardware capabilities.
Switch revision
Capabilities
IXS600-Bx
•PCI Express Gen2 only
•No cascading of switches.
IXS600-BE
•Minor service level update
•Current Gen2 revision
IXS600-Hx
•Introduced updated IDT PCIe Gen3 chipset.
•A firmware upgrade is required to interconnect switches at PCIe Gen3 speed.
•Enables cascading of switches to support 20/56 nodes using IXH610/IXH620 adapter cards.
IXS600-HN
•Improved power supply to support Gen3 PXH810 cards on all ports
•Current Gen3 revision, full support for all Gen2 and Gen3 IXH and PXH iPass based cards
from Dolphin.
Management software revisions
The following table gives a general overview of the management software capabilities. All versions contain various bug
fixes, security patches, and stability improvements.
Revision
Capabilities
1.0.0 - 1.3.1
•Initial functionality
1.3.2
•Added optional management web interface, disabled by default.
1.3.3
•Web interface enabled by default
•File upload support from web interface
1.3.5
•Select active EEPROM from web interface
1.3.6
•Support for web interface upgrades
•Added support for USB stick status dump and management
•Automatic fan speed control
1.3.7
•Added web interface authentication
•Changed back to constant fan speed due to dynamic speed issues
1.3.8
•Fixed EEPROM upgrade issues, that caused slow upgrades of empty EEPROMs
•Implemented the reset button on the management page
1.3.9
•Added support for DHCP. Default off.
•Added link watchdog with management page.
•Removed unnecessary reset on boot, which caused transparent issues
•Included Gen 3 EEPROMs and configured Gen 3 as default
(for HN revisions and newer)
1.3.9.1
•Fixed a BE revision (and older) bug where EEPROM is not loaded on boot.
1.3.9.2
•Stability fixes
•Added NOCLOCK transp EEPROMs supporting no clock on upstream port

IXS600 Users Guide –Dolphin Interconnect Solutions Page 6
Configuration examples
The IXS600 switch can be used as a standalone switch, for larger transparent PCIe expansion systems or PCIe cluster
systems using multiple switches. Some examples can be found below.
8 node PCIe Gen2 and Gen3 NTB Cluster
Each node has a Dolphin Express PXH810, IXH610 or IXH620 adapter.
Up to eight systems can be connected to the Dolphin IXS600 switch
using the default firmware configuration with the front panel
configuration switch set to M1.
The IXS600-HN revision is required to support the PXH810 Gen3
card on all ports.
20 node PCIe Gen2 NTB Cluster
Three IXS600 switches are connected together to create a 20 node
PCI Express cluster. Two switches can be used to build a 14 node
system. In reflective memory mode, you can connect up to 56
nodes. The correct firmware must be loaded on the switches to
establish this configuration. See page 15 for more details.
Transparent PCIe Gen3 expansion –up to 7 Expansion chassis connected to one
server
A Dolphin Gen2 or Gen3 adapter card in upstream transparent
mode is connected to switch port 8. Any standardized PCI Express
x8 downstream device or expansion chassis is connected to switch
port 1-7. No special device driver is required for the IXH6xx, PXH812
adapter card or the IXS600 switch Any device in the PCI Express IO
system will operate using its standard device driver. With the
default firmware configuration, the front-panel configuration switch
must be set to M2 (PCI Express Gen2) or M3 (PCI Express Gen3).

IXS600 Users Guide –Dolphin Interconnect Solutions Page 7
Quick install guide
The IXS600 can be mounted in a 19-inch rack or used in a standalone setup. Rack mount ears are included in the package
and can be mounted on both sides of the switch depending on user requirements. Rubber feet are included in the package
and should be applied beneath the switch to avoid scratches.
Note: Static electricity from your clothes or work environment can damage your PCI Express adapter card or
your PC. Always wear a grounded antistatic wrist strap while opening the PC and when the card is removed
from the anti-static bag.
IXS600 operating modes
The Dolphin PCI Express IXS600 switch has two main operational modes: Transparent and Non Transparent Bridge (NTB)
mode. The switch operating mode is controlled by the available EEPROMs, which can be selected with the small switch
knob on the front panel. A thin non-electrically leading tool (e.g. a toothpick) should be used to move the switch knob. An
enlarged picture of the selector is shown in picture 1 below:
Picture 1: IXS600 configuration switch mode selector
Configuration switch setting
Factory default functionality
M1
NTB mode –(Dolphin software) –TOP switch configuration
M2
Transparent mode, Gen2
M3
Transparent mode, Gen3
The IXS600 switch must be power cycled to enable a full reconfiguration after moving the configuration switch selector. It
is possible to change the functionality of each configuration switch setting, see page 24 for further details. Please be
careful when reprogramming the EEPROMs.
Cable connections
The IXS600 supports x8 iPass™ PCI Express standard cables as well as PCI Epress x8 to x4 iPass™ Transition cables. A single
cable per swicth port is used to connect to other devices having a x8 or x4 iPass connector; Standard transparent I/O
devices, IXE60x expansion boxes, or PCs with an IXH610, IXH611 or IXH620 cards.
To install the cable, match the male portion on the x8 cable with the female connector on the IXS600 switch. Use even
pressure to insert the connector until it locks and is secure.
Transparent mode port usage
In default transparent mode, Port 8 is the upstream port. Port 1-7 is downstream.

IXS600 Users Guide –Dolphin Interconnect Solutions Page 8
NTB mode port usage
Any port can be used in any order, no restrictions. Dolphins NTB software and adapter cards will auto detect ports and
automatically connect at the highest speed possible. Customers writing their own low-level drivers need to contact
Dolphin for further technical information.
Cascading switches
The IXS600 (revisions Hx and higher) supports cascading multiple switches into larger PCI Express networks. In
combination with the IXH610 adapter, up to 20 nodes can communicate using Dolphins NTB driver. In reflective memory
configurations, up to 56 nodes can be interconnected. More details can be found on page 15.
Partitioning –mixed transparent and NTB usage
The switch supports partitioning into separate switch domains, enabling a single switch to be used as two or more
independent logical switch domains. The switch hardware prevents unintended data communication and reset /
configuration cycles to propagate between separate domains. This configuration can be easily enabled by loading the
partitioning firmware. Please contact Dolphin for more information and instructions.

IXS600 Users Guide –Dolphin Interconnect Solutions Page 9
PCI Express Gen3 Switch - IXS600
The IXS600 is a PCI Express Gen3 x8 switch available from Dolphin providing an easy to use, multi-functional solution.
IXS600 high level specification
•PCI Express Gen1 2.5 Gbps / Gen2 5.0 Gbps / Gen3 8.0 Gbps per lane signaling –8 lanes –up to 64 Gbps total
signaling pr port.
•Compliant with PCI Express Gen1 and Gen2 computers and IO systems, auto detection. Each port operates
independently at Gen1 or Gen2 speeds.
•Lane width auto-training to connected cable width (x8, x4, x2, x1)
•19-Inch 1U rack mountable chassis, optional rack mount ears.
•Physical dimensions 440mm x 300mm x 45mm (WxDxH).
•Supports NTB connections to other hosts and transparent connections to IO systems as a transparent P2P device.
•PCI Express Base Specification Rev 3.0
•PCI Express External Cabling Specification, Rev. 1.0
•Eight PCI Express Gen2 x8 iPass Cable connection.
•Link clock isolation. Automatic support for hosts running CFC or SSC mode.
•Support for hot plugging of the PCI Express cable (NTB mode, SISCI and SuperSockets)
•Built in management processor for monitoring and firmware management.
•Optional firmware upgrade and configuration via web interface or USB.
•RJ45 Ethernet (10/100) connector (optional use)
•Front and rear operational status and alert LEDs
•Three programmable 128 KB EEPROM for switch configuration data.
•No PCI Express power domain isolation.
•Partitioning
•Redundant fans
•Operating systems supported:
oNTB mode: Full SuperSockets, TCP/IP driver, Reflective Memory and SISCI compliance. Linux, RedHawk,
Windows, RTX and VxWorks support.
oTransparent mode: All (No drivers required)
•Power supply input 100 - 240VAC 50/60 Hz –2.2A
•Power Consumption: Max 80 Watt (including active optical transceivers).
•Operating temperature 0°C to 55°C, Relative Humidity 5% - 95% non-condensing
•Storage temperature -25°C to 70°C, Relative Humidity 10% - 90% non-condensing
•Weight : 4.0 Kilo
•JTAG programming and test
•RoHS compliant
•CE Mark
Picture 2: IXS600 front view