Compaq BL10e - HP ProLiant - 512 MB RAM User manual

Abstract.............................................................................................................................................. 2
Transition from parallel to serial SCSI protocol ........................................................................................ 2
SAS terminology.................................................................................................................................. 2
SAS technology................................................................................................................................... 3
SAS devices .................................................................................................................................... 3
Initiators ...................................................................................................................................... 4
Expanders ................................................................................................................................... 4
Targets........................................................................................................................................ 4
Differential signaling ........................................................................................................................ 5
SAS protocol evolution ......................................................................................................................... 6
SAS-1 ............................................................................................................................................. 6
SAS-2 ............................................................................................................................................. 6
SAS-2.1 .......................................................................................................................................... 6
Active cables ............................................................................................................................... 6
Storage power management.......................................................................................................... 7
SAS/SATA interoperability ................................................................................................................... 8
Cabling and connectors ....................................................................................................................... 9
Mini SAS 4x cable connectors and receptacles ................................................................................... 9
Mini SAS 8x cable connectors......................................................................................................... 11
SAS topologies.................................................................................................................................. 11
Internal...................................................................................................................................... 11
External..................................................................................................................................... 13
Multi-node clusters ...................................................................................................................... 14
Zoning ............................................................................................................................................. 14
SAS performance .............................................................................................................................. 16
For more information.......................................................................................................................... 18
Call to action .................................................................................................................................... 18
Serial Attached SCSI storage technology
2nd Edition

2
Abstract
This technology brief describes Serial Attached SCSI-2 (SAS-2) technology, the evolution of the SAS
standard, interoperability with Serial Advanced Technology Attachment (SATA) devices,
enhancements to SAS-2 cabling and connectors, and SAS topologies and zoning.
Transition from parallel to serial SCSI protocol
The original SCSI standard was developed in 1981 to provide a common interface that could be
used across all peripheral platforms and system applications, such as Redundant Array of
Independent Disks (RAID) storage. Since that time, there have been seven generations of the parallel
SCSI protocol. Each generation doubled the bandwidth of the previous one, primarily by doubling the
bus clock frequency. But as the bus frequency was increased with each new generation, so did the
negative impact of bus contention, signal degradation, and signal skew—slight signal delays from
one wire trace to the next. After the development of Ultra320 SCSI with a bandwidth of 320 MB/s
per channel, further bandwidth improvements to parallel SCSI could not occur without developing
new and expensive technologies.
In 2001, HP (Compaq), IBM, LSI Logic, Maxtor, and Seagate founded the Serial Attached SCSI
Working Group to define the rules for exchanging information between SCSI devices using a serial
interconnect (SAS). SAS was later transferred to the InterNational Committee for Information
Technology Standards (INCITS) T10 to become an American (ANSI) and international (ISO/IEC)
standard.
SAS inherits its command set from parallel SCSI, frame formats and full duplex communication from
Fibre Channel, and it uses the SATA interface for compatibility and investment protection. The SAS
architecture solves the parallel SCSI problems of bus contention, clock skew, and signal degradation
at higher signaling rates, thereby providing performance headroom to meet enterprise storage needs
for years to come.
SAS terminology
Table 1 provides a list of terms related to SAS technology to aid in understanding the concepts
described in this paper.
Table 1. SAS terminology
Term
Definition
ANSI
American National Standards Institute
Domain
An I/O system consisting of a set of SAS devices that communicate with one another by
means of a service delivery subsystem
Enterprise-class devices
SAS drives that provide maximum reliability, highest performance, scalability, and error
management for use with unconstrained I/O workloads in mission-critical applications
Entry level drives
SATA drives with the lowest unit cost that provide a basic level of reliability and
performance for non-mission-critical environments
Expander
A device that functions as a switch to attach one or more initiators to one or more targets
Initiator
A device containing SSP, STP, and/or SMP initiator ports in a SAS domain

3
ISO/IEC
International Organization for Standardization/International Electrotechnical Commission
Midline devices
SAS and SATA drives that provide larger capacity, greater reliability, improved resistance
to rotational and operational vibration than Entry level drives, making them better suited
for use in multi-drive configurations.
Phy
The mechanism that contains a transceiver which electrically interfaces to a physical link.
Phy is a common abbreviation for the physical layer of the OSI model.
Physical link
Two differential signal pairs, one pair in each direction, that connect two physical phys
SAS address
The identifier of an initiator port, a target port, or an expander device
Serial ATA Tunneling
Protocol (STP)
A protocol used to communicate with SATA drives
Serial Management
Protocol (SMP)
A protocol used to communicate with SAS expanders
Serial SCSI Protocol
(SSP)
A protocol used to communicate with SAS drives
Service delivery
subsystem
The part of a SAS I/O system that transmits information between a SAS initiator port and a
SAS target port
Subtractive routing
A routing technique used when a device is not able to find other devices in the same sub-
branch. This will pass the request to a different branch altogether.
Table routing
Table routing is used for identifying devices connected to the expanders connected to a
device's own phy.
Target
An end device such as a SAS hard disk drive, SATA hard disk drive, or SAS tape drive
Training
The process of adapting equalization circuitry in a receiver device to an incoming
transmission pattern
Virtual phy
Contains a vendor-specific interface to another virtual phy
Wide link
A group of physical links that attaches a wide port to another wide port
Wide port
A port that contains more than one phy
Zone group
A set of phys in a zone that all have the same access permission
Zoned portion of a
service delivery
subsystem (ZPSDS)
A group of zoning expander devices that cooperate to control access between phys
SAS technology
SAS is a point-to-point architecture that transfers data to and from SCSI storage devices by using
serial communication (one bit at a time). SAS devices and the differential signaling method they use to
achieve reliable, high-speed serial communication are described in this section.
SAS devices
There are three types of SAS devices: initiators, targets, and expanders. An initiator device is a host
bus adaptor (HBA), or controller. The initiator is attached to one or more targets—SAS hard disk
drives, SATA hard disk drives, and SAS tape drives—to form a SAS domain. Expanders are low-cost,
high-speed switches that scale the number of targets attached to an initiator, thereby creating a larger

4
SAS domain (Figure 1). Each SAS device has a unique worldwide name (SAS address) assigned at
manufacturing to simplify its identification in a domain
Initiators
SAS initiators have multiple ports for connection to internal and/or external targets. Each initiator port
can have a single physical link (a narrow port) or 2, 4, or 8 physical links (a wide port). SAS initiator
ports can be connected to separate domains for fail-over redundancy.
Expanders
Expanders establish connections between initiators, targets, and other expanders by receiving
commands and data in one port and routing them to another port based on the SAS address of the
target. Expanders use three routing methods—direct, table, and subtractive. An expander uses direct
routing to forward commands and data to targets directly attached to the expander. An expander
uses table routing to forward commands and data to another expander. When an expander receives
an address that it does not recognize, it uses subtractive routing to forward the commands and data
to another expander that does recognize the address.
Targets
SAS hard drives (enterprise-class and midline devices) have two narrow ports. Each port can reside in
a different SAS domain to provide fail-over redundancy and load balancing. SAS hard drives
leverage a common electrical and physical connection interface with SATA hard drives. However,
SATA hard drives, including solid state drives, have a single narrow port.
Figure 1. Example of SAS devices in a domain. The number of initiators and targets allowed in a domain is
limited only by the size of the expanders’ routing tables.

5
Differential signaling
All SAS devices have connection points called ports. One or more transceiver mechanisms, called
phys, are located in the port of each SAS device. A physical link, consisting of two wire pairs,
connects the transmitter of each phy in one device’s port to the receiver of a phy in another device’s
port. The SAS interface allows the combination of multiple physical links to create two (2x), 3x, 4x, or
8x connections per port for scalable bandwidth. A port that has one phy is described as ―narrow‖
while a port with two to four phys is described as ―wide.‖
SAS uses differential signaling to transfer data over a physical link (Figure 2), which reduces the
effects of capacitance, inductance, and noise experienced by parallel SCSI at higher speeds. SAS
communication is full duplex, which means that each phy can send and receive information
simultaneously over the two wire pairs.
Figure 2. In differential signaling, positive minus negative equals 1500–900 = 600mV or 900–1500 = -600mV.
The physical link rates for SAS and SATA technologies are listed in Table 2.
Table 2. Physical link rates per direction
Physical link rate
Generation
Bandwidth
4x bandwidth
1.5 Gbps
SAS-1, SAS-1.1, SATA Revision
1.0
150 MB/s
600 MB/s
3 Gbps
SAS-1, SAS-1.1, SATA Revision
2.0
300 MB/s
1200 MB/s
6 Gbps
SAS-2, SAS-2.1, SATA Revision
3.0
600 MB/s
2400 MB/s

6
SAS protocol evolution
SAS-1
The speed of the first-generation SAS (SAS-1) link is 3.0 gigabits per second (Gb/s). The original SAS
standard defined two classes of expanders: edge expanders and fanout expanders. An edge
expander attaches directly to targets, or to another edge expander, which reduces its complexity (and
cost) by constraining the size of its routing table. Edge expanders can also be connected together to
form an edge expander device set, allowing the set to address up to 128 devices. A fanout expander
allows multiple edge expanders or edge expander sets to communicate with each other; therefore, it
has to maintain a more extensive routing table.
SAS-2
The second-generation SAS (SAS-2) link speed doubles the physical link rate to 6.0 Gb/s. SAS-2
eliminates the distinction between fanout and edge expanders by replacing them with self-configuring
expanders. SAS-2 also enables zoning for enhanced resource deployment flexibility, security, and
data traffic management. SAS-2 is backward compatible with SAS-1.
Each SAS device (initiator, target, or expander) may support one or more SAS communication
speeds. If any two linked devices support multiple speeds, the highest speed will be used if
performance is critical or a slower speed may be used if reliability is more important. During this
speed negotiation process, linked SAS devices determine their mutually supported SAS speed(s), and
other speed-related options, such as transmission amplitude, slew rate, de-emphasis, and spread
spectrum clocking.
After the speed negotiation process is complete, the SAS-2 standard provides a time interval, or
training window, during which the linked devices exchange predefined signals to test the link using
the selected speed(s) and speed-related options.
SAS-2.1
The enhancement to the SAS-2 standard, SAS-2.1, will define active cables, storage power
management, and additional connectors (see ―Cabling and connectors‖). Also, SAS-2.1 splits out the
protocol layer into a separate standard, SAS Protocol Layer (SPL).
Active cables
To help reduce cable weight, improve cable management, and improve airflow, the SAS 2.1
standard implements thinner cables with active circuitry. Active circuitry includes built-in drivers and
repeaters, along with an equalizer. The equalizer removes inter-symbol interference (ISI), a form of
signal distortion. The drivers and repeaters reduce the signal-to-noise ratio (SNR) by boosting the
received signal and by reducing near-end crosstalk (NEXT). NEXT occurs when two wires are close
enough for the signal traveling in one wire to interfere with the signal traveling in the other. Cable
designers reduced this phenomenon by placing a low-power equalizing filter inside the cable to
compensate for the dielectric and conductor losses. Economical, low-power, low-latency, active cables
enable high data transfer rates using thinner wire gauges over longer cable runs (Figure 3).

7
Figure 3. SAS 1.1 at 1.5 Gbps and 3.0 Gbps in untrained mode has a 6-m limit. SAS-2 in trained mode raises
the limit to 10 m at all speeds. At 6 Gbps, active copper and optical cables extend the distance to 20 m.
Storage power management
SAS-2.1 devices will be able to turn off SAS physical links when they are idle. Each SAS transceiver
consumes about 200 mW. Therefore, the total power savings from a dual-ported drive with two
transceivers and a controller (or attached SAS expander) with two transceivers is a little less than
1 W. Table 3 includes an example of power savings for a 9-W small form factor (SFF) SAS drive.
Future SAS and SATA disk drives will include more standardized power management features.
Today's drives support the Active and Stopped states. The Stopped state is rarely used due to its long
recovery time.
Table 3. Power management state example for a 9-W SFF SAS drive
State
Description
Commands processed
Power savings
Recovery time
Active
Fully active
Yes
None
None
Idle
Stop clocks to idle
circuitry
Yes
~1 W
< 1 ms
Park heads
Retract the heads from
the media, reducing
drag
Automatic state
change
~2 W
1 sec
Low-rpm standby
Retract heads, spin
down to a lower rpm
Automatic state
change
~4 W
7 sec
Standby
Retract heads, spin
down to 0 rpm, cache
still powered
Automatic state
change
~7 W
15-20 sec
Stopped
Same as Standby
No
~7 W
15-20 sec
Sleep
Everything off except
wakeup circuitry
No
~9 W
30 sec

8
SAS/SATA interoperability
Using the SAS interface, SAS drives, SATA drives, or a mix of both SAS and SATA drives will function
in the same storage enclosure. With this broad range of storage solutions (Figure 4), IT managers can
choose storage devices based on reliability, performance, and cost. Because the SAS architecture
features a proven SCSI command set, advanced command queuing, and advanced verification/error
correction, SAS is the ideal solution for mission-critical enterprise storage applications.
SAS supports three protocols to handle communications with various devices:
Serial Management Protocol (SMP) –manages the point-to-point topology of expanders and
enclosures
Serial SCSI Protocol (SSP) –facilitates communication with SAS devices and existing SCSI software
SATA Tunneling Protocol (STP) –allows SAS controllers to communicate with SATA devices through
expanders
Figure 4. The SAS architecture provides flexible solutions ranging from entry level NAS in desktop
environments to mission-critical RAID systems that complement Fibre Channel technology.

9
Cabling and connectors
Mini SAS 4x cable connectors and receptacles
Mini SAS 4x connectors and receptacles (Figure 5) are replacing SAS 4x connectors and receptacles
in new designs. Mini SAS connectors contain ground pins that can be used for power in active
cables.
Figure 5. The internal Mini SAS 4i connector (left) and receptacle (right) are replacing SAS 4x
connectors and receptacles.
Figure 6 shows the Mini SAS 4x (external) cable plug connector and receptacle. Mini SAS 4x cable
connectors attach to end devices and enclosure universal ports, but they are specific to enclosure out
and in ports. The SAS-2 receptacles are universal, thus they do not have table routing or subtractive
routing restrictions. The receptacles also have a reverse key (not illustrated) to accept cables longer
than 6 m as well as shorter SAS-1 cables 6 m or less.
Figure 6. The external Mini SAS 4x connector is specific to enclosure in and out ports, but the receptacle is
universal.

10
Figure 7 shows the type and location of icons that identify the connectors that attach to an end
device, an enclosure ―out‖port, an enclosure ―in‖port, and an enclosure universal port.
Figure 7. External Mini SAS 4x cable connector icons identify compatible devices and ports.
Cable connectors and receptacles are keyed to correspond to in ports (subtractive routing), out ports
(table routing), and ports for direct-attached end devices (direct routing). Figure 8 shows the three
keying methods.
Figure 8. Three key slot positions correspond to enclosure in and out ports and ports for direct-attached devices.

11
Mini SAS 8x cable connectors
SAS-2.1 adds Mini SAS 8x internal and external connectors. The Mini SAS 8i HD (internal) connector
will be a hybrid that combines two serial general purpose input/output (SGPIO) busses. The Mini SAS
8x HD (external) connector will support both passive and active cables.
SAS topologies
SAS makes it possible for manufacturers to create highly scalable topologies—internal, external, or
both—giving customers the flexibility to design and deploy a range of solutions. Using SATA
Tunneling Protocol (STP), SAS controllers communicate with SATA devices through expanders, which
is the key to SATA scalability in the SAS domain.
Internal
Figure 9 describes a topology that can be used for internal RAID systems incorporating SAS or SATA
drives. Each drive has a point-to-point connection to the controller. The controller can support a
maximum of eight drives. For internal configurations of this type, the external port is disabled if more
than four drives are connected.
Figure 9. Two internal ports on the controller provide redundancy for storage applications.

12
Figure 10 shows an alternate internal topology for RAID systems using SAS or SATA drives. The full
height HP Smart SAS Expander Card supports more than 8 internal hard disk drives on select ProLiant
servers when connected to a Smart Array P410 Controller (SA-P410) or Smart Array P410i Controller
(SA-P410i). The SAS expander card supports up to 24 internal drive bays, and it has a Mini SAS 4x
port for connection for tape.
Figure 10. An internal port of the SAS controller can be used to provide an alternate topology for RAID systems
using an HP SAS Expander Card and SAS or SATA drives.

13
External
Figure 11 shows a topology for connecting one external port on the two-port controller to a storage
enclosure containing up to ten SFF SAS or SATA drives. This single controller port incorporates four
lanes for a total maximum throughput of 2400 MB/s. The storage enclosure contains an internal 36-
port expander that supports cascading an additional enclosure in a 1+1 configuration containing up
to 25 SFF SAS or SATA drives.
Figure 11. The external port of the controller can connect to an enclosure with up to 25 SFF SAS or SATA drives,
while supporting the cascading of an additional enclosure.

14
Multi-node clusters
A multi-node cluster using SAS provides an alternative to clustered Fibre Channel local loop
topologies. This highly scalable SAS architecture enables topologies that provide high performance
and high availability with no single point of failure. Figure 12 shows a multi-node cluster application
using a SAS RAID HBA controller. This SAS topology can also be implemented using an ASIC
(application-specific integrated circuit) embedded on the motherboard.
Figure 12. Topologies for multi-node cluster applications using a SAS RAID controller provide high performance
and high availability.
Zoning
The number of devices (initiators, targets, expanders, and/or virtual devices) allowed in a given
domain is limited only by the size of the expander routing tables. But managing such a large number
of devices can be very complicated. Therefore, zoning was introduced into the SAS-2 standard for
efficiency (traffic management) and security. With SAS-2, large physical topologies can be broken
into logical groups. This grouping allows access within and between zone groups being controlled. A
group of zoning-enabled expanders that cooperate to control access between phys is known as a
zoned portion of a service delivery system (ZPSDS).
There can be 128 or 256 zone groups numbered from 0 to 127 or 0 to 255, respectively. Zone
groups 0 through 8 are pre-defined and cannot be changed by the user. Devices in zone group 0 can
only access devices in zone group 1, while devices in zone group 1 are allowed access to all zone
groups. For example, a system administrator can use zone group 0 for a new (unassigned) device
that is added to a ZPSDS. At the same time, the administrator can use zone group 1 for topology
discovery and zone management.
Permission tables in SAS expanders control zoning. This means that an end device does not require
any special features to operate within a zoned SAS domain, which makes legacy SAS and SATA
devices compatible. An end device in a zone can only ―see‖ other end devices in the domain as
permitted by the zoning expander(s). Figure 13 shows a SAS domain with a ZPSDS containing three
zoning expanders in addition to one expander device without zoning enabled.

15
Figure 13. A group of zoning-enabled SAS expanders (ZPSDS) can be configured to allow only certain end
devices to see each other.
Zoning methodology
The SAS-2 standard permits zoning, a secure zoning method that uses the unique ID number of each
expander phy. In zoning, each port of a zoning expander is assigned to a zone group, and any
device attached to one of the ports becomes part of that respective zone group. By default, all devices
within a zone group can interact with each other. The permission table in the expander controls
access between devices in different zone groups. If an attached device changes, the expander may
be configured to set the zone group to 0 (no access), which allows an address-resolved-like policy to
be implemented. For example, if a particular SAS device address needs to have certain permissions
and the device is moved to a different expander in the fabric, then the zone manager can reprogram
the zone group at the new location.
Zoning is ideal for small topologies, server blade enclosures, and clustering applications.
Zone management
A zone manager is responsible for configuring each zone. As shown in Figure 14 (top), the zone
manager can control a zone by using an end device that has a SAS port connected to one of the
zoning expanders. The zone manager can also control a zone through a sideband interface (such as
Ethernet) on one or more zoning expanders (Figure 14 bottom).

16
Figure 14. The zone manager can be attached to an end device (top) or directly to one or more expanders
through a sideband interface (bottom).
The zone manager assigns zone groups to all zoning expander phys, and it assigns all phys in a
wide port to the same zone group. The zone manager stores the zoning assignment of each expander
phy along with SAS addresses in the zoning expander’s route table.
Inside a particular ZPSDS, the zone manager assigns each zoning expander phy attached to another
zoning expander phy to zone group 1. Basically, phys in zone group 1 have access to phys in all
zone groups. The zone manager assigns each zoning expander phy on the boundary of the ZPSDS to
a zone group other than group 1. The ZPSDS boundary is defined by designating expander phys as
―not trusted‖ when connected to end devices outside the zone.
Each zoning expander device also contains a zone permission table that controls whether a
connection is allowed between phys in different zone groups.
SAS performance
Enterprise-class SAS hard disk drives (HDDs) must provide maximum performance 24x7 and under
continuous I/O workload in high-vibration environments. An important factor in HDD performance is
seek time, which is the time from when a read or write action is initiated until the data transfer from or
to the disk actually begins. The smaller platters of SFF SAS drives inherently yield lower seek times,
which is an advantage in file servers with frequent random accesses.

17
In enterprise server environments, SFF SAS drives excel in performance and reliability.1Since SFF
drives require only 70% of the space and half the power of 3.5-inch large form factor (LFF) SAS
drives, higher drive densities per-U is possible without a significant increase in power consumption.
Higher drive densities provide better overall performance, greater reliability, and lower operating
costs.
1For more information, refer to technology brief ―Performance factors for HP ProLiant Serial Attached Storage (SAS)‖ at
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01460725/c01460725.pdf.

For more information
For additional information, refer to the resources listed below.
Resource description
Web address
Redundancy in enterprise storage networks
using dual-domain SAS configurations
http://h20000.www2.hp.com/bc/docs/support/SupportManu
al/c01451157/c01451157.pdf
Performance factors for HP ProLiant Serial
Attached Storage (SAS)
http://h20000.www2.hp.com/bc/docs/support/SupportManu
al/c01460725/c01460725.pdf
ProLiant storage papers and audio podcasts
http://h18004.www1.hp.com/products/servers/technology/w
hitepapers/proliant-storage.html
Drive technology overview, 2nd Edition
http://h20000.www2.hp.com/bc/docs/support/SupportManu
al/c01071496/c01071496.pdf
Call to action
Send comments about this paper to [email protected].
© 2009 Hewlett-Packard Development Company, L.P. The information contained
herein is subject to change without notice. The only warranties for HP products and
services are set forth in the express warranty statements accompanying such
products and services. Nothing herein should be construed as constituting an
additional warranty. HP shall not be liable for technical or editorial errors or
omissions contained herein.
TC090603TB, June 2009