Silicon Graphics 10-Gigabit Ethernet Network Adapter User manual

SGI®10-Gigabit Ethernet Network
Adapter User’s Guide
007–4669–002

CONTRIBUTORS
Written by Lori Johnson
Illustrated by Dan Young and Chrystie Danzer
Engineering contributions Arthur Kepner, Edward Mascarenhas, Steve Modica, Alain Renaud, James Yarbrough
COPYRIGHT
© 2004, 2005 Silicon Graphics, Inc. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere
herein. No permission is granted to copy, distribute, or create derivative works from the contents of this electronic documentation in
any manner, in whole or in part, without the prior written permission of Silicon Graphics, Inc.
LIMITED RIGHTS LEGEND
The electronic (software) version of this document was developed at private expense; if acquired under an agreement with the US
government or any contractor thereto, it is acquired as “commercial computer software” subject to the provisions of its applicable
license agreement, as specified in (a) 48 CFR 12.212 of the FAR; or, if acquired for Department of Defense units, (b) 48 CFR 227-7202 of
the DoD FAR Supplement; or sections succeeding thereto. Contractor/manufacturer is Silicon Graphics, Inc., 1600 Amphitheatre Pkwy
2E, Mountain View, CA 94043-1351.
TRADEMARKS AND ATTRIBUTIONS
Silicon Graphics, SGI, the SGI logo, Altix, IRIX, Onyx, Origin, and Silicon Graphics Tezro are registered trademarks and Silicon
Graphics Prism is a trademark of Silicon Graphics, Inc., in the United States and/or other countries worldwide.
Intel is a registered trademark of Intel Corporation. Linux is a registered trademark of Linus Torvalds, used with permission by Silicon
Graphics, Inc. UNIX is a registered trademark of The Open Group in the United States and other countries.
FCC WARNING
This equipment has been tested and found compliant with the limits for a Class A digital device, pursuant to Part 15 of the FCC rules.
These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a
commercial environment. This equipment generates, uses, and can radiate radio frequency energy and if not installed and used in
accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a
residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at personal
expense.
ATTENTION
This product requires the use of external shielded cables in order to maintain compliance pursuant to Part 15 of the FCC Rules.
VDE 0871/6.78
This equipment has been tested to and is in compliance with the Level A limits per VDE 0871.
EUROPEAN UNION STATEMENT
This device complies with the European Directives listed on the “Declaration of Conformity” which is included with each product. The
CE mark insignia displayed on the device is an indication of conformity to the aforementioned European requirements.
TUV
geprufte
Sicherheit
R
NRTL/C

International Special Committee on Radio Interference (CISPR)
This equipment has been tested to and is in compliance with the Class A limits per CISPR publication 22.
Canadian Department of Communications Statement
This digital apparatus does not exceed the Class A limits for radio noise emissions from digital apparatus as set out in the Radio
Interference Regulations of the Canadian Department of Communications.
Attention
Cet appareil numérique n’émet pas de perturbations radioélectriques dépassant les normes applicables aux appareils numériques de
Classe A préscrites dans le Règlement sur les interferences radioélectriques etabli par le Ministère des Communications du Canada.
Japanese Compliance Statement
Compliance Statement in Chinese

New Features
This guide contains the new Chapter 4, "Performance Tuning" on page 15.
007–4669–002 v

Record of Revision
Version Description
001 May 2004
Initial publication
002 August 2005
Revision
007–4669–002 vii

Contents
About this Guide ..................... xi
Important Information . . . ................... xi
Scope of this Guide ....................... xii
Related Publications . . . ................... xii
Obtaining Publications . . . ................... xiii
Conventions . ........................ xiii
Product Support ........................ xiv
Reader Comments ....................... xiv
1. Features and Capabilities of the SGI 10-Gbit Ethernet Network Adapter 1
Features . . ........................ 2
Cabling . . ........................ 2
Configuration Limits . . . ................... 3
Tools . . . ........................ 3
2. Connecting the Adapter to a Network . . . .......... 5
Installing the Adapter . . . ................... 5
Connecting to the Network . ................... 5
3. Operating the Adapter . ................. 7
Verifying Functionality . . . ................... 7
Using LEDs to Determine Functionality . . .............. 7
Verifying Adapter Recognition .................. 8
IRIX Adapter Recognition Verification . .............. 8
Linux Adapter Recognition Verification . .............. 9
007–4669–002 ix

Contents
Enabling the Adapter . . ................... 10
Verifying that the Adapter is Properly Configured and Enabled ........ 10
IRIX Adapter Configuration Verification . .............. 10
Linux Adapter Configuration Verification .............. 11
Resetting the Adapter . . . ................... 11
Configuration ........................ 12
IRIX Configuration . . . ................... 12
Linux Configuration . . . ................... 12
MTU Size Settings ....................... 12
4. Performance Tuning . . ................. 15
IRIX Performance Tuning . . ................... 15
Jumbo Frames and IRIX . ................... 15
Read/Write Size and IRIX . ................... 16
Socket Buffer Size and IRIX ................... 16
Multibuffer Mode and IRIX ................... 16
SGI ProPack 3 Performance Tuning .................. 17
Socket Read and Write Buffer Sizes for SGI ProPack 3 . . ......... 17
Jumbo Frames and SGI ProPack 3 . . . .............. 17
Read/Write Size and SGI ProPack 3 . . . .............. 17
TCP/IP Socket Buffer Size and SGI ProPack 3 .............. 18
Glossary . . . ..................... 19
Index . ......................... 21
x 007–4669–002

About this Guide
This guide describes the SGI 10-Gigabit (Gbit) Ethernet network adapter. This guide
shows you how to connect the adapter to an Ethernet network and explains how to
operate the adapter.
The SGI 10-Gbit Ethernet network adapter operates under the versions of the IRIX
and SGI ProPack for Linux operating systems, as specified in the release notes.
You can use the SGI 10-Gbit Ethernet network adapter in addition to your current
adapter.
This guide assumes that you have general knowledge of Ethernet networks and the
system in which the adapter is installed.
Important Information
Warning: Never look into the end of a fiber optic cable to confirm that light is being
emitted (or for any other reason).
Do not use any type of magnifying device, such as a microscope, eye loupe, or
magnifying glass. Such activity causes cause a permanent burn on the retina of the
eye. Optical signal cannot be determined by looking into the fiber end.
Most fiber optic laser wavelengths (1300 nm and 1550 nm) are invisible to the eye and
cause permanent eye damage. Shorter wavelength lasers (for example, 780 nm) are
visible and can cause significant eye damage.
Use only an optical power meter to verify light output.
007–4669–002 xi

About this Guide
Scope of this Guide
This guide is written to facilitate installation of the adapter and does not cover
detailed points of network configuration. It contains the following chapters:
• Chapter 1, "Features and Capabilities of the SGI 10-Gbit Ethernet Network
Adapter", summarizes features, cabling, configuration limits, and tools.
• Chapter 2, "Connecting the Adapter to a Network", shows you how to connect the
adapter to your network.
• Chapter 3, "Operating the Adapter", explains how to verify installation of the
adapter and software, how to reset the adapter, how to set parameters to improve
performance, and how to set configuration parameters.
Related Publications
This guide is part of a document set that fully supports the installation, operation,
and service of the adapter. For more information about installing and servicing the
adapter, see the user’s guide for the system in which the adapter is installed.
Also see the following:
•IRIX Admin: Networking and Mail
•Linux Configuration and Operations Guide
•The Network Administrators’ Guide
• The Intel Corporation 10 Gigabit Ethernet Technology Overview white paper:
http://www.intel.com/network/connectivity/resources/doc_library/white_papers/pro10gbe_lr_sa_wp.pdf
• IRIX man pages that are part of the 10-Gbit Ethernet product:
–xg(7M)
–xgcmd(1M)
• Standard Linux man pages that are useful for any Ethernet device:
–ethtool(8)
–ifconfig(8)
xii 007–4669–002

SGI®10-Gigabit Ethernet Network Adapter User’s Guide
–ip(8)
Obtaining Publications
You can obtain SGI documentation as follows:
• See the SGI Technical Publications Library at http://docs.sgi.com. Various formats
are available. This library contains the most recent and most comprehensive set of
online books, release notes, man pages, and other information.
• If it is installed on your SGI system, you can use InfoSearch, an online tool that
provides a more limited set of online books, release notes, and man pages. With
an IRIX system, enter infosearch at a command line or select Help >
InfoSearch from the Toolchest.
• On IRIX systems, you can view release notes by entering either grelnotes or
relnotes at a command line.
• On Linux systems, you can view release notes on your system by accessing the
README.txt file for the product. This is usually located in the
/usr/share/doc/productname directory, although file locations may vary.
• You can view man pages by typing man title at a command line.
Conventions
The following conventions are used throughout this document:
Convention Meaning
command This fixed-space font denotes literal items such as
commands, files, routines, path names, signals,
messages, and programming language structures.
variable Italic typeface denotes variable entries and words or
concepts being defined.
user input This bold, fixed-space font denotes literal items that the
user enters in interactive sessions. (Output is shown in
nonbold, fixed-space font.)
007–4669–002 xiii

About this Guide
[ ] Brackets enclose optional portions of a command or
directive line.
... Ellipses indicate that a preceding element can be
repeated.
Product Support
SGI provides a comprehensive product support and maintenance program for its
products:
• If you are in North America, contact the Technical Assistance Center at +1 800 800
4SGI or contact your authorized service provider.
• If you are outside North America, contact the SGI subsidiary or authorized
distributor in your country.
Reader Comments
If you have comments about the technical accuracy, content, or organization of this
publication, contact SGI. Be sure to include the title and document number of the
publication with your comments. (Online, the document number is located in the
front matter of the publication. In printed publications, the document number is
located at the bottom of each page.)
You can contact SGI in any of the following ways:
• Send e-mail to the following address:
[email protected]
• Use the Feedback option on the Technical Publications Library Web page:
http://docs.sgi.com
• Contact your customer service representative and ask that an incident be filed in
the SGI incident tracking system.
xiv 007–4669–002

SGI®10-Gigabit Ethernet Network Adapter User’s Guide
• Send mail to the following address:
Technical Publications
SGI
1500 Crittenden Lane, M/S 535
Mountain View, California 94043–1351
SGI values your comments and will respond to them promptly.
007–4669–002 xv

Chapter 1
Features and Capabilities of the SGI 10-Gbit
Ethernet Network Adapter
The 10-Gbit Ethernet technology is an extension of Gigabit Ethernet (1000-Base-T)
technology that allows over-the-wire speeds up to 10 Gbits per second (Gbps), which
is theoretically ten times the rate of existing technology. Given the limitations of the
bus and the CPUs, realistic performance with the SGI 10-Gbit Ethernet network
adapter will not exceed 4.5–6 Gbps.
The 10-Gbit Ethernet technology is targeted at backbone networks and interserver
connectivity. It provides an upgrade path for high-end workstations that require more
bandwidth than Gigabit Ethernet can provide.
The adapter is supported in the following systems:
• Silicon Graphics Prism
• SGI Altix 3000
• SGI Altix 350
• SGI Origin 350
• Silicon Graphics Onyx 4
• Silicon Graphics Onyx 350
• SGI Origin 3000 with IX brick or PX brick
• Silicon Graphics Onyx 3000 with IX brick or PX brick
• Silicon Graphics Tezro
This chapter includes the following sections:
• "Features" on page 2
• "Cabling" on page 2
• "Configuration Limits" on page 3
• "Tools" on page 3
007–4669–002 1

1: Features and Capabilities of the SGI 10-Gbit Ethernet Network Adapter
Features
The adapter includes the following features:
• Full-duplex Gigabit Ethernet interface as defined in the IEEE P802.3ae approved
standard
• Jumbo frame support (up to 9600 bytes)
• 16 DMA channels (8 for reading, 8 for writing)
• Single LC fiber connection
• Support for PCI 64-bit 66-MHz and PCIX 66/100/133-MHz
• Universal dual-voltage signaling (3.3 V and 5 V)
• Compliance with PCI Local Bus Specification (Revision 2.3) and PCI-X Addendum
to the PCI Local Bus Specification (Revision 1.0a) standards
• Optical cabling 850–nm (10GBASE-SR)
Cabling
The adapter is an LC connector and uses a 10GBASE-SR transceiver at 850–nm. It is
connected to the network using a multimode fiber (MMF) cable of 50-micron. The
cable (which is not included in the shipment) must have a quality in the range from
400-MHz * km to 2000-MHz * km, depending on its length, as shown in Table 1-1.
Table 1-1 10-Gbit Cable Standards
Diameter (Microns) Modal Bandwidth (MHz * km) Range (Meters)
50 400 2 to 66
50 500 2 to 82
50 2000 2 to 300
2 007–4669–002

SGI®10-Gigabit Ethernet Network Adapter User’s Guide
Configuration Limits
The number of the 10-Gbit Ethernet network adapters supported varies by system.
Consult with your SGI representative to determine the currently supported maximum
for your configuration.
Tools
The SGI 10-Gbit Ethernet network adapter product contains the following IRIX tools:
•xg(7F) driver
•xgcmd(1M) diagnostic tool to interrogate and control both the driver itself and the
interfaces under its control
The following standard Linux commands are useful with any Ethernet device:
•ethtool(8)
•ifconfig(8)
•ip(8)
For more information, see the man page associated with each tool.
007–4669–002 3