HP bh5700 User manual

HP bh5700 ATCA 14-Slot Blade Server
Ethernet Switch Blade
First Edition
Manufacturing Part Number: AD171-9603A
June 2006

Ethernet Switch Blade User's Guide release 3.2.2j page ii

Legal otices
The information in this document is subject to change without notice.
Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but
not limited to, the implied warranties of merchantability and fitness for a particular
purpose. Hewlett- Packard shall not be held liable for errors contained herein or direct,
indirect, special, incidental or consequential damages in connection with the furnishing,
performance, or use of this material.
Restricted Rights egend. Use, duplication or disclosure by the U.S. Government is subject
to restrictions as set forth in subparagraph (c) (1) (ii) of the Rights in Technical Data and
Computer Software clause at DFARS 252.227-7013 for DOD agencies, and subparagraphs
(c) (1) and (c) (2) of the Commercial Computer Software Restricted Rights clause at FAR
52.227-19 for other agencies.
Information in this document is provided in connection with Intel® products. No
license, express or implied, by estoppel or otherwise, to any intellectual property rights
is granted by this document. Except as provided in Intel’s Terms and Conditions of
Sale for such products, Intel assumes no liability whatsoever, and Intel disclaims any
express or implied warranty, relating to sale and/or use of Intel products including
liability or warranties relating to fitness for a particular purpose, merchantability, or
infringement of any patent, copyright or other intellectual property right. Intel
products are not intended for use in medical, life saving, or life sustaining applications.
Intel may make changes to specifications and product descriptions at any time, without
notice. HEW ETT-PACKARD COMPANY 3000 Hanover Street Palo Alto, California 94304
U.S.A.
Copyright Notice. Copyright ©2003 Hewlett-Packard Development Company, .P.
Reproduction, adaptation, or translation of this document without prior written
permission is prohibited, except as allowed under the copyright laws.
Additional Copyright Notices. AdvancedTCA® is a registered trademark of the PCI
Industrial Computer Manufacturers Group. inux® is a registered trademark of inus
Torvalds. ZNYX Networks, RAIN, RAINlink, OpenArchitect®, CarrierClass and HotSwap
are trademarks or registered trademarks of ZNYX Networks in the United States and/or
other countries. All other marks, trademarks or service marks are the property of their
respective owners.
Ethernet Switch Blade User's Guide release 3.2.2j page iii

About the Ethernet Switch Blade Manual
This manua inc udes everything you need to begin using the HP Ethernet Switch B ade with
OpenArchitect software, Re ease 3.2.2j.
Ethernet Switch Blade User's Guide release 3.2.2j page iv

Table of Contents
Chapter 1 Overview of the Ethernet Switch B ade ...........................................................17
High Performance Embedded Switching...................................................................... 17
Advanced TCA® Comp iant.........................................................................................17
OpenArchitect Switch Management............................................................................. 18
Extensib e Customization of Routing Po icies..............................................................18
Powerfu CarrierC ass Features.....................................................................................18
Ethernet Port Layout..................................................................................................... 18
Ethernet Switch B ade Port Configuration..................................................................19
Base switch Quick Reference.............................................................................. 19
Fabric Switch Quick Reference........................................................................... 19
OpenArchitect Switch Environment............................................................................. 20
OpenArchitect Software Structure................................................................................ 20
Chapter 2 Port Cab ing and LED Indicators...................................................................... 23
Connecting the Cab es...................................................................................................23
Conso e Port Cab ing................................................................................................23
Connecting to the Conso e Port................................................................................23
Out of Band Ports (OOB Ports).................................................................................... 24
LED Reference......................................................................................................... 24
Chapter 3 High Avai abi ity Networking...........................................................................27
Surviving Partner.......................................................................................................... 27
VRRP........................................................................................................................28
z md.......................................................................................................................... 28
Switch Rep acement and Reconfiguration............................................................... 29
zspconfig...................................................................................................................29
Examp e HA Switch Configuration..........................................................................30
Modifying zsp.conf on the Base switch...............................................................31
Modifying zsp_v an.conf on the Fabric Switch...................................................35
Configuring Surviving Partner......................................................................................42
Centra Authority.......................................................................................................... 43
Chapter 4 Fabric Switch Configuration ............................................................................ 46
Two switches, two conso es..........................................................................................46
Connecting to the Fabric Switch Conso e.....................................................................46
OpenArchitect Configuration Procedure.......................................................................46
Changing the She Prompt........................................................................................... 47
Defau t Configuration Scripts...................................................................................47
Examp e Configuration Scripts................................................................................ 47
Overview of OpenArchitect VLAN Interfaces.........................................................48
Tagging and Untagging VLANs..........................................................................48
Switch Port Interfaces..........................................................................................49
Layer 2 Switch Configuration.......................................................................................49
Using the S50 ayer2 Script.................................................................................. 50
Ethernet Switch Blade User's Guide release 3.2.2j page v

Rapid Spanning Tree................................................................................................ 50
To Enab e Rapid Spanning Tree:.........................................................................51
Port Path Cost...................................................................................................... 51
Layer 3 Switch Configuration............................................................................. 52
Using the S50 ayer3 Script.................................................................................. 52
Layer 3 Routing Protoco s with GateD ........................................................................54
Using the S55gatedRip1 Script................................................................................ 54
To Modify the GateD Scripts: ................................................................................. 56
C ass of Service (COS) ................................................................................................ 57
Egress Queues.......................................................................................................... 57
Ingress C assification................................................................................................57
Marking and Re-marking......................................................................................... 58
Schedu ing................................................................................................................ 58
ztmd Exp ained.................................................................................................... 58
zfi terd Exp ained..................................................................................................... 58
Running zfi terd........................................................................................................58
Restrictions on Imp ementation................................................................................59
Conf ict Reso ution.............................................................................................. 59
iptab es and fi tering............................................................................................ 60
Introduction..........................................................................................................60
Packet Wa k......................................................................................................... 61
Fi ter Ru es Specifications...................................................................................62
Specifying Source and Destination IP Addresses.................................................... 62
Specifying Protoco ............................................................................................. 62
Specifying an ICMP Message Type.................................................................... 62
Specifying TCP or UDP ports............................................................................. 63
Specifying TCP f ags...........................................................................................63
Specifying an Interface........................................................................................ 63
Fi ter Ru e Targets............................................................................................... 63
Supported Targets................................................................................................63
C assica Targets..................................................................................................63
ZNYX Targets..................................................................................................... 63
ZACTION Examp es........................................................................................... 64
Extensions to the defau t matches........................................................................64
tc and zqosd ............................................................................................................. 65
FIFO Queues (pfifo and bfifo discip ines)...........................................................65
PRIO and WRR queues....................................................................................... 67
The U32 Fi ter.......................................................................................................... 69
Combining Queuing Discip ines.............................................................................. 69
Hand e Semantics................................................................................................ 70
COPS: Common Open Po icy Service..........................................................................70
Protoco Architecture................................................................................................71
OpenArchitect PEP...................................................................................................71
Using pepd................................................................................................................72
Ethernet Switch Blade User's Guide release 3.2.2j page vi

Chapter 5 Fabric Switch Administration........................................................................... 73
Setting the Root Password............................................................................................ 73
Adding Additiona Users...............................................................................................73
Setting up a Defau t Route............................................................................................ 74
Name Service Reso ution..............................................................................................74
DHCP C ient Configuration..........................................................................................74
DHCP Server Configuration......................................................................................... 74
Network Time Protoco (NTP) C ient Configuration................................................... 75
Network Fi e System (NFS) C ient Configuration........................................................75
NFS Server Configuration.............................................................................................76
Connecting to the Switch Using FTP............................................................................77
ftpd Server Configuration............................................................................................. 77
Connecting to the Switch Using TFTP......................................................................... 77
TFTPD Server Configuration........................................................................................77
SNMP Agent................................................................................................................. 78
Supported MIBS.......................................................................................................78
Supported Traps........................................................................................................79
SNMP and OpenArchitect Interface Definitions......................................................80
ifStackTab e Entries.............................................................................................81
SNMP Configuration................................................................................................81
SNMP App ications..................................................................................................82
Port Mirroring............................................................................................................... 82
Link and LED Contro .................................................................................................. 83
Link Event Monitoring..................................................................................................83
Chapter 6 Fabric Switch Maintenance...............................................................................84
Overview of the OpenArchitect switch boot process....................................................84
Saving Changes.............................................................................................................86
Modifying Fi es and Updating the Switch.................................................................... 86
Recovering from a System Fai ure................................................................................86
System Boots with a Conso e Cab e............................................................................. 86
Booting with the –i option.............................................................................................87
System Hangs During Boot...........................................................................................88
Booting the Dup icate F ash Image...............................................................................88
Upgrading the OpenArchitect Image............................................................................ 88
Upgrading or Adding Fi es............................................................................................89
Exc uding Saving Fi es to F ash...............................................................................89
Upgrading the Switch Driver........................................................................................ 89
Using apt-get................................................................................................................. 90
Chapter 7 Base Switch Configuration................................................................................91
Two switches, two conso es..........................................................................................91
Connecting to the Base Switch Conso e....................................................................... 91
OpenArchitect Configuration Procedure..................................................................91
Changing the She Prompt.......................................................................................92
Defau t Configuration Scripts..............................................................................92
Ethernet Switch Blade User's Guide release 3.2.2j page vii

Examp e Configuration Scripts............................................................................92
Overview of OpenArchitect VLAN Interfaces....................................................93
Tagging and Untagging VLANs..........................................................................94
Switch Port Interfaces..........................................................................................94
Layer 2 Switch Configuration.................................................................................. 94
Using the S50 ayer2 Script.................................................................................. 96
Rapid Spanning Tree................................................................................................ 96
To Enab e Rapid Spanning Tree:.........................................................................96
Port Path Cost...................................................................................................... 97
Layer 3 Switch Configuration.................................................................................. 97
Using the S50 ayer3 Script.................................................................................. 98
Layer 3 Switch Using Mu tip e VLANs............................................................100
Using the S50mu tiv an Script...........................................................................100
To Modify the Layer 3 Mu tiv an Script ......................................................... 102
Modify the examp e script you copied into the /etc/rcZ.d directory. Adjust and
assign the number of IP addresses as app icab e. In the examp e be ow, the IP
address is changed for the interface in the ifconfig command ine of the script.
........................................................................................................................... 102
Layer 3 Routing Protoco s with GateD ................................................................. 102
Using the Provided S55gatedRip1 Script.......................................................... 102
To Modify the GateD Scripts: .......................................................................... 104
C ass of Service (COS) ..........................................................................................105
Egress Queues....................................................................................................105
Ingress C assification.........................................................................................105
Marking and Re-marking...................................................................................106
Schedu ing......................................................................................................... 106
zcos......................................................................................................................... 106
zfi terd.....................................................................................................................106
ztmd................................................................................................................... 106
Running zfi terd................................................................................................. 107
Restrictions on Imp ementation.........................................................................107
Conf ict Reso ution............................................................................................ 107
iptab es and fi tering............................................................................................... 108
Introduction........................................................................................................109
Packet Wa k....................................................................................................... 110
Fi ter Ru es Specifications.................................................................................110
Specifying Source and Destination IP Addresses..............................................110
Specifying Protoco ........................................................................................... 110
Specifying an ICMP Message Type.................................................................. 110
Specifying TCP or UDP ports........................................................................... 111
Specifying TCP f ags.........................................................................................111
Specifying an Interface...................................................................................... 111
Fi ter Ru e Targets............................................................................................. 111
Supported Targets..............................................................................................111
Ethernet Switch Blade User's Guide release 3.2.2j page viii

C assica Targets................................................................................................111
ZNYX Targets................................................................................................... 112
ZACTION Examp es......................................................................................... 112
Extensions to the defau t matches......................................................................113
tc: Traffic Contro ..................................................................................................113
Strict Priority Qdisc................................................................................................113
Weighted Round Robin Qdisc................................................................................114
FIFO Queues (pfifo and bfifo discip ines).........................................................114
Fifo Qdiscs..............................................................................................................115
Using Fi ters to Direct Packets to a COS Queue....................................................115
Protoco ip.............................................................................................................. 115
Protoco arp............................................................................................................ 116
Protoco a ..............................................................................................................116
Matching Specific Ingress Ports.............................................................................116
Advanced Fi tering – Po icing................................................................................117
Examp es............................................................................................................118
Po icing Actions..................................................................................................... 118
u32 match se ectors used in fi ters.........................................................................119
zqosd.......................................................................................................................120
PRIO and WRR queues..................................................................................... 121
The U32 Fi ter........................................................................................................ 123
Combining Queuing Discip ines............................................................................ 124
Hand e Semantics.............................................................................................. 124
COPS: Common Open Po icy Service................................................................... 124
Protoco Architecture.........................................................................................125
OpenArchitect PEP............................................................................................126
Using pepd......................................................................................................... 126
Chapter 8 Base Switch Administration............................................................................128
Setting the Root Password......................................................................................128
Adding Additiona Users........................................................................................128
Setting up a Defau t Route..................................................................................... 129
Name Service Reso ution....................................................................................... 129
DHCP C ient Configuration................................................................................... 129
DHCP Server Configuration...................................................................................129
Network Time Protoco (NTP) C ient Configuration.............................................130
Network Fi e System (NFS) C ient Configuration.................................................130
NFS Server Configuration......................................................................................131
Connecting to the Switch Using FTP..................................................................... 131
ftpd Server Configuration.......................................................................................132
Connecting to the Switch Using TFTP...................................................................132
TFTPD Server Configuration................................................................................. 132
SNMP Agent.......................................................................................................... 132
Supported MIBS................................................................................................ 132
Supported Traps.................................................................................................134
Ethernet Switch Blade User's Guide release 3.2.2j page ix

SNMP and OpenArchitect Interface Definitions............................................... 134
ifStackTab e Entries...........................................................................................135
SNMP Configuration......................................................................................... 135
SNMP App ications........................................................................................... 136
Port Mirroring.........................................................................................................136
Link and LED Contro ............................................................................................137
Link Event Monitoring........................................................................................... 137
Chapter 9 Base Switch Maintenance............................................................................... 138
Overview of the OpenArchitect switch boot process............................................. 138
Saving Changes...................................................................................................... 140
Modifying Fi es and Updating the Switch..............................................................140
Recovering from a System Fai ure......................................................................... 140
System Boots with a Conso e Cab e.......................................................................140
Booting with the –i option......................................................................................141
System Hangs During Boot.................................................................................... 142
Booting the Dup icate F ash Image........................................................................ 142
Upgrading the OpenArchitect Image......................................................................142
Upgrading or Adding Fi es.....................................................................................143
Exc uding Saving Fi es to F ash.............................................................................143
Upgrading the Switch Driver..................................................................................143
Using apt-get.......................................................................................................... 144
Chapter 10 Connecting to the Ethernet Switch B ade..................................................... 145
Base Interface Hub System:........................................................................................ 145
Ethernet Interfaces: ................................................................................................145
Management Interfaces: ........................................................................................ 145
Fabric Interface Hub System: .....................................................................................146
Ethernet Interfaces: ................................................................................................146
Management Interfaces: ........................................................................................ 146
Connecting to the Base Interface................................................................................ 146
Base Interface Seria Port Connection....................................................................146
Base Interface Out-of-Band Ethernet Connection .................................................147
Connecting to the Fabric Interface .............................................................................148
Fabric Interface Seria Port Connection ................................................................ 148
Fabric Interface Out of Band Ethernet Connection ...............................................149
Chapter 11 Diagnosing a Fai ed Ethernet Switch B ade Activation ..............................150
Accessing the ShMM.................................................................................................. 152
Verifying Communications Between the ShMM and Switch................................ 152
Critica Thresho d Error Reported..................................................................... 152
Ana yzing Mstate information for the switch............................................................. 153
Checking the ekey Status From the She f Manager.................................................... 153
Chapter 12 Troub eshooting a Fai ed OpenArchitect Load.............................................155
Recovering from a System Fai ure .............................................................................157
Booting Without the Over ay Fi e...............................................................................158
Ethernet Switch Blade User's Guide release 3.2.2j page x

Booting the Dup icate F ash Image ............................................................................159
Chapter 13 Network Configuration Prob ems ............................................................... 160
Interface Overview......................................................................................................160
Physica Interfaces..................................................................................................160
Defau t Base Interface Configuration.....................................................................161
24 port, Layer 2 Switching, sing e VLAN.........................................................161
Defau t Fabric Interface Configuration.................................................................. 163
Editing the S50 ayer2 script can change the Ethernet Switch B ade Fabric Interface
defau t configuration. The S50Layer2 script and inc uded examp e scripts
(/etc/rcZ.d/examp es) can be used as temp ates to create custom scripts. The defau t
S50 ayer2 script configures the switch according y:..............................................163
Configuration Troub eshooting...................................................................................165
Determining ekey status for a specific s ot................................................................. 165
Querying Base Interface ekey Status......................................................................167
Querying Fabric Interface ekey Status................................................................... 168
Network Connectivity Troub eshooting......................................................................170
No Connection........................................................................................................170
Diminished Network Throughput...........................................................................170
Connecting to Devices with Fixed Port Speeds ......................................................... 170
Externa Fau t LED..................................................................................................... 170
Network Tests............................................................................................................. 171
Ping Test ................................................................................................................171
Traceroute Test.......................................................................................................172
Chapter 14 Iso ating Hardware Fai ures.......................................................................... 173
Hardware Subsystem...................................................................................................176
Testing the F ashROMs...............................................................................................177
Testing the Switch Fabric............................................................................................178
Link Status for a sing e port................................................................................... 178
Link Status for a range of ports.............................................................................. 178
Testing the onboard RAM...........................................................................................179
Testing the Contro Processor..................................................................................... 180
Hardware Fau t....................................................................................................... 181
Software Error................................................................................................... 181
Chapter 15 High Avai abi ity Troub eshooting............................................................... 183
Spontaneous Fai over Activity....................................................................................183
Unexpected Fai -back Activity...............................................................................183
Chapter 16 Switch Firmware Overview.......................................................................... 184
Checking the switch firmware version........................................................................184
3.1 Fabric Interface............................................................................................185
Updating the Switch Firmware................................................................................... 186
BootLoader Firmware Upgrade:.............................................................................186
OpenArchitect Firmware Upgrade:........................................................................ 186
IPMC Firmware Upgrade:......................................................................................187
Ethernet Switch Blade User's Guide release 3.2.2j page xi

Chapter 17 Restoring the Factory Defau t Configuration................................................188
Chapter 18 Before Ca ing Support..................................................................................189
Appendix A Fabric Switch Command Man Pages........................................................ 191
vrrpconfig ...................................................................................................................192
vrrpd ........................................................................................................................... 194
zbootcfg ......................................................................................................................197
zconfig ........................................................................................................................199
zcos .............................................................................................................................207
zdog ............................................................................................................................211
zfi terd ........................................................................................................................ 213
zf ash........................................................................................................................... 214
z 2, z 2mc, z 3host, z 3net, zv an................................................................................ 216
zgvrpd .........................................................................................................................219
z 2d .............................................................................................................................221
z 3d .............................................................................................................................223
z c ............................................................................................................................... 225
z md ............................................................................................................................228
z ogrotate ....................................................................................................................230
zmirror ........................................................................................................................231
zmnt.............................................................................................................................233
zpeer ........................................................................................................................... 235
zqosd .......................................................................................................................... 238
zrc ...............................................................................................................................240
zreg..............................................................................................................................241
zr d ............................................................................................................................. 243
zsnoopd ...................................................................................................................... 244
zspconfig .................................................................................................................... 246
zstack ..........................................................................................................................253
ztats............................................................................................................................. 258
zsync............................................................................................................................259
ztmd ............................................................................................................................261
brct (8) ........................................................................................................................263
Appendix B Base Switch Command Man Pages...........................................................266
vrrpconfig ...................................................................................................................267
vrrpd ........................................................................................................................... 269
zbootcfg ......................................................................................................................272
zconfig ........................................................................................................................274
zcos .............................................................................................................................282
zdog ............................................................................................................................286
zffpcounter ................................................................................................................. 288
zfi terd......................................................................................................................... 292
zf ash........................................................................................................................... 293
zgmrpd........................................................................................................................ 295
Ethernet Switch Blade User's Guide release 3.2.2j page xii

zgr................................................................................................................................297
zgvrpd..........................................................................................................................300
z 2d..............................................................................................................................302
z 3d..............................................................................................................................304
z c ............................................................................................................................... 306
z md ............................................................................................................................308
z ogrotate ....................................................................................................................310
zmirror ........................................................................................................................311
zmnt.............................................................................................................................314
zpeer ........................................................................................................................... 316
zqosd........................................................................................................................... 319
zrc ...............................................................................................................................321
zreg..............................................................................................................................322
zr d ............................................................................................................................. 324
zsnoopd ...................................................................................................................... 325
zspconfig .................................................................................................................... 328
zstack ..........................................................................................................................336
ztats............................................................................................................................. 340
zsync............................................................................................................................341
ztmd.............................................................................................................................343
brct (8).........................................................................................................................345
Appendix C Inte igent P atform Management Interface ..............................................348
ISwitch-ShMC Interaction.......................................................................................... 348
Periphera Management Contro er Functiona Support............................................. 349
Sensor Reading Examp e........................................................................................350
Structure of Standard IPMI Commands: From BMC to PMC....................................352
Structure of Standard IPMI Responses: From PMC to BMC..................................... 352
Event Generator ........................................................................................................ 353
IPMB Event message format............................................................................. 353
IPMI Event Message Definitions.......................................................................353
Fie d Rep aceab e Unit Inventory Device.............................................................. 353
IPMB Override/Loca Status - Event Data 3 for the IPMB ink........................354
Table of Figures
Figure 1.1: Fabric Switch E ements...................................................................................20
Figure 1.2: OpenArchitect Software Structure.................................................................. 22
Figure 2.1: LED Reference................................................................................................ 25
Figure 3.1: Host HA Architecture......................................................................................27
Figure 4.1: Fabric VLANs................................................................................................. 48
Figure 4.2: Firewa F ow ................................................................................................ 61
Figure 4.3: COPS Network Architecture........................................................................... 70
Figure 6.1: ROM Devices in Open Architect.................................................................... 84
Figure 6.2: Boot F ow Chart.............................................................................................. 85
Ethernet Switch Blade User's Guide release 3.2.2j page xiii

Figure 6.3: Init Script F ow................................................................................................86
Figure 7.1: Mu tip e VLANs..............................................................................................94
Figure 7.2: Layer 2 Switch ................................................................................................95
Figure 7.3: Layer 3 Switch ................................................................................................99
Figure 7.4: Mu tip e VLAN Configuration......................................................................101
Figure 7.5: Firewa F ow ............................................................................................... 109
Figure 7.6: COPS Network Architecture........................................................................ 125
Figure 9.1: ROM Devices in OpenArchitect................................................................... 138
Figure 9.2: Booting up Process F ow..............................................................................139
Figure 9.3: Init Script F ow..............................................................................................140
Figure 10.1: Fabric and Base .......................................................................................... 145
Figure 10.2: Base Interface Seria Port............................................................................ 147
Figure 10.3: Fabric Interface Seria Ports........................................................................ 148
Figure 11.1: Ethernet Switch B ade Activation States.....................................................150
Figure 12.1: OpenArchitect Boot Process....................................................................... 156
Figure 12.2: ROM Devices in OpenArchitect................................................................. 157
Figure 18.1: ROM Devices in OpenArchitect................................................................. 190
Index of Tables
Tab e 5.1: Supported MIBs................................................................................................79
Tab e 5.2: Supported Traps................................................................................................80
Tab e 5.3: Link and SNMP Status..................................................................................... 81
Tab e 7.1: Port Path Cost................................................................................................... 97
Tab e 7.2: Po icing Actions..............................................................................................119
Tab e 7.3: U Match Se ectors...........................................................................................120
Tab e 8.1: Supported MIBs..............................................................................................134
Tab e 8.2: Supported Traps..............................................................................................134
Tab e 8.3: Physica Link Status on Base Switch..............................................................135
Tab e 11.1: Troub eshooting States................................................................................. 152
Tab e 13.1: Ethernet Switch B ade Backp ane Interfaces (zre Ports)..............................160
Tab e 13.2: Additiona Interfaces.................................................................................... 161
Tab e C.1.: IPMI M States............................................................................................... 349
Tab e C.2: PMC Contro er Support................................................................................ 349
Tab e C.3: GetSensorReading..........................................................................................350
Tab e C.4: GetSensorResonse..........................................................................................351
Tab e C.5: Standard IPMI Commands.............................................................................352
Tab e C.6: Standard IPMI Responses.............................................................................. 352
Tab e C.7: Event Message Format...................................................................................353
Tab e C.8: SEEPROM Space...........................................................................................354
Tab e C.9.: IPMB Override Status Data.......................................................................... 355
Ethernet Switch Blade User's Guide release 3.2.2j page xiv

Ethernet Switch Blade User's Guide release 3.2.2j page 1

Ethernet Switch Blade User's Guide release 3.2.2j page 16

Chapter 1 Overview of the Ethernet Switch Blade
The Ethernet Switch B ade is a 72-port AdvancedTCA® Hub and providing Gigabit Ethernet.
Up to 14 ATCA node boards may be addressed via the PICMG 3.0 Base Interface and via the
ATCA PICMG 3.1 fabric . The Base and Fabric switching domains are kept tota y separate, both
on the physica ayer and the software ayer. The Ethernet Switch B ade provides a tight y
integrated modu ar switching p atform that enab es high-density so utions.
The Ethernet Switch B ade is actua y two separate switches, one for the Base ports and one for
the fabric ports. There are two OpenArchitect® operating system images, one for each switch,
a owing the maximum in separation between the contro signa ing and the data. The modu ar
design provides great f exibi ity and contro .
Ethernet Switch B ades can support a 10 Gigabit Ethernet Inter-Switch Link (ISL) for the Fabric
Interfaces, and a Gigabit Ethernet ISL for the Base Interface switches. Depending on the version
of OpenArchitect used, the ISL for the Fabric Interface switches may be operated at 10 Gigabits
per second and provide stacking features.
Linux-based OpenArchitect 3 runs on the embedded processors, providing a comprehensive
package for the management of Layer 2 and Layer 3 packet switching. VLAN management and
Layer 2-7 packet c assification are a so inc uded with a user-friend y interface. OpenArchitect can
be used with a variety of IP routing protoco s.
As part of Advanced TCA, the switch incorporates the PICMG 3.0 Inte igent P atform
Management Interface (IPMI) standard for Fie d Rep aceab e Unit FRU) management by the
She f Manager.
High Performance Embedded Switching
The Ethernet Switch B ade with OpenArchitect combines the performance of si icon-based
switching fabric with f exibi ity of software-managed routing po icies. It provides Base fabric
PICMC 3.0 (1 Gigabit Ethernet ) inks to each of the pay oad s ots, p us two to four PICMC 3.1
in-band GigE ports to each node card, and GigE inks to management ports and the second
switch. The Ethernet Switch B ade maintains the forwarding tab e on si icon, providing the
capabi ity to switch and route at fu ine rate performance on every port.
Advanced TCA® Compliant
The Advanced TCA® standard deve oped by the PCI Industria Computer Manufacturer Group
defines an embedded Ethernet environment for high avai abi ity chassis. This environment
inc udes two switch fabric s ots that create a dua star Ethernet network to the 14 Base node s ots.
P acing the Ethernet Switch B ade in a hub s ot provides embedded Ethernet services to each
node card of the chassis. A standard HA configuration is one Ethernet Switch B ade p aced in
each of the two hub s ots in a chassis for creation of a redundant, high avai abi ity system.
Ethernet Switch Blade User's Guide release 3.2.2j page 17

OpenArchitect Switch Management
The OpenArchitect software component – open source Linux, IP protoco stack, contro
app ications and the OA Engine – runs on two embedded PowerPC microprocessors.
OpenArchitect provides extensive managed IP routing protoco s and other open standards for
switch management. Examp es inc ude network services; Virtua Redundant Router Protoco ;
Routing Information Protoco ; Open Shortest Path First; Border Gateway Protoco ; Qua ity of
Service and C ass of Service; access contro ists; Simp e Network Management Protoco MIBs,
Common Open Po icy Services and web.
Extensible Customization of Routing Policies
The OpenArchitect software environment enab es rapid porting of other UNIX/Linux-based
protoco s, inc uding open source software conforming to RFCs and other standards. It a so
enab es the deve opment of app ication-specific protoco configuration scripts.
Powerful CarrierClass Features
The Ethernet Switch B ade has High Avai abi ity hardware features for advanced
te ecommunication app ications. The switch imp ements the PICMG 3.0 Fu Hotswap support.
This feature provides fie d rep aceab e capabi ities so a switch can fai and be rep aced without
impacting the operationa performance of a chassis.
The PICMG 3.0 Inte igent P atform Management Interface (IPMI) standard is a so supported.
IPMI uses message-based interfaces that monitor the physica hea th characteristics of the
Ethernet Switch B ade. The switch provides operationa status information to an IPMI
management app ication. End customers benefit with advanced notice of potentia prob ems.
The Ethernet Switch B ade a so imp ements the Media Dependent Interface ca ed Auto MDI-X.
Auto MDI-X a ows connections to any device, switches, hubs, or systems using a regu ar
straight-through or crossover Cat 5 cab e. The RJ-45 port wi auto detect and switch MDI/MDI-
X modes. This IEEE standard makes cab ing – especia y between switches – faster and ess error
prone.
E-Keying is supported by the Ethernet Switch B ade.
Ethernet Port Layout
The Ethernet Switch B ade has a tota of 72 switched Gigabit Ethernet ports. The base fabric is
connected via 24 Gigabit Ethernet ports and the data fabric is connected via 48 Gigabit Ethernet
ports. The Ethernet Switch B ade is actua y composed of two separate switches, one for Base
port activity and another for fabric port activity. The Base ports ( contro and signa ing) are
switched on the Base switch, and the fabric ports ( data ) are switched on the fabric switch, which
provides tota separation between system management or contro packets, and customer data
packets.
Ethernet Switch Blade User's Guide release 3.2.2j page 18

Ethernet Switch Blade Port Configuration
Base switch Quick Reference
She fManager1 zre22
She fManager2 zre13
ISL channe ( Base node2 ) zre23
Base nodes 3-14 zre0-11
Base nodes 15,16 zre 20-21
Front pane
zre12, zre14, zre15
Fabric Switch Quick Reference
slot zre numbers
3zre0-3
4zre4-7
5zre8-11
6zre12-15
7zre16-19
8zre24-27
9zre28-29
10 zre30-31
11 zre32-33
12 zre34-35
13 zre36-37
14 zre38, zre39
15 zre40-41
16 zre42-43
Inter-switch Link (ISL) zre51
Front pane zre20-23
Ethernet Switch Blade User's Guide release 3.2.2j page 19

You wi find the Ethernet Switch B ade has a straightforward insta ation and configuration.
UNIX or Linux system management ski s and some understanding of network protoco s wi be
required. Configure the Ethernet Switch B ades to your networking app ication before you
begin using the OpenArchitect switch.
OpenArchitect Switch Environment
The key e ements of the OpenArchitect environment inc ude two embedded Linux operating
systems, OpenArchitect-specific app ications and ibraries, p us, an innovative switch hardware
design.
OpenArchitect hardware is in many ways simi ar to typica switch architectures. The primary
difference in OpenArchitect is that the PCI bus that interfaces with the embedded processor and
the switch fabric is at a higher performance eve than a typica switch (see Figure 1.1: Fabric
Switch E ements). The use of PCI creates a pipe of significant bandwidth between the processor
and the switch fabric.
The embedded processors, running Linux and the OpenArchitect processes, contro the f ow of a
traffic by maintaining the switch forwarding tab es. These tab es define the f ow of the switch
traffic. Because they are on the switching chips, packets proceed at ine rate.
OpenArchitect Software Structure
OpenArchitect is based on an embedded Linux operating system and inc udes a number of ZNYX
Networks-supp ied modu es. The key e ement is the Linux routing tab e, which is crucia in a
Ethernet Switch Blade User's Guide release 3.2.2j page 20
Figure 1.1: Fabric Switch Elements