Sun Microsystems StorEdge 3000 Series Instructions for use

Sun Microsystems, Inc.
www.sun.com
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Sun StorEdge™3000 Family
Best Practices Manual
Sun StorEdge 3510 FC and 3511 SATA Arrays
Part No. 816-7325-22
March 2007, Revision A

Copyright © 2002–2007 Dot Hill Systems Corporation and others, 2200 Faraday Avenue, Suite 100, Carlsbad, California 92008, USA.
All rights reserved.
Sun Microsystems, Inc. and Dot Hill Systems Corporation may have intellectual property rights relating to technology embodied in this product
or document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at
http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and other countries.
This product or document is distributed under licenses restricting its use, copying distribution, and decompilation. No part of this product or
document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.
Third-party software is copyrighted and licensed from Sun suppliers.
Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in
the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd.
Sun, Sun Microsystems, the Sun logo, Sun StorEdge, AnswerBook2, docs.sun.com, and Solaris are trademarks or registered trademarks of Sun
Microsystems, Inc. in the U.S. and in other countries.
U.S. Government Rights—Commercial use. Government users are subject to the Sun Microsystems, Inc. standard license agreement and
applicable provisions of the FAR and its supplements.
DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT,
ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
Copyright © 2002–2007 Dot Hill Systems Corporation et d'autres, 2200 Faraday Avenue, Suite 100, Carlsbad, Californie 92009, Etats-Unis.
Tous droits réservés.
Sun Microsystems, Inc. et Dot Hill Systems Corporation peuvent avoir les droits de propriété intellectuels relatants à la technologie incorporée
dans le produit qui est décrit dans ce document. En particulier, et sans la limitation, ces droits de propriété intellectuels peuvent inclure un ou
plus des brevets américains énumérés à http://www.sun.com/patents et un ou les brevets plus supplémentaires ou les applications de brevet
en attente dans les Etats-Unis et dans les autres pays.
Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la
décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans
l'autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.
Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des
fournisseurs de Sun.
Des parties de ce produit pourront être dérivées des systèmes Berkeley BSD licenciés par l’Université de Californie. UNIX est une marque
déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.
Sun, Sun Microsystems, le logo Sun, Sun StorEdge, AnswerBook2, docs.sun.com, et Solaris sont des marques de fabrique ou des marques
déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.
LA DOCUMENTATION EST FOURNIE “EN L’ÉTAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES
OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT
TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L'APTITUDE A UNE UTILISATION PARTICULIERE OU A
L’ABSENCE DE CONTREFAÇON.

iii
Contents
Preface xi
1. Introduction 1
Executive Summary 1
RAS Features of Sun StorEdge 3000 Family Arrays 1
Additional RAS Features You Can Use With FC and SATA Arrays 2
Differences Between Sun StorEdge 3510 FC Array and Sun StorEdge 3511 SATA
Array 4
2. Underlying Concepts and Practices 7
Fibre Channel Protocols 7
Supported RAID Levels 8
Logical Drives 8
Maximum Drive Configurations per Array 9
Maximum Number of Disks and Maximum Usable Capacity per Logical Drive
10
Cache Optimization 12
Configuring an Array’s RCCOM Channel 14
Using Four DRV + RCCOM Channels 14
Using Channels 4 and 5 as RCCOM Channels 14
Array Management Tools 15

iv Sun StorEdge 3000 Family Best Practices Manual •March 2007
Saving and Restoring Configuration Information 16
3. Planning Your Storage Architecture 17
Direct-Attached Storage 18
Storage Area Networking 19
Scaling Capacity 21
First Steps in Designing a Solution 22
Designing a Storage Solution for an Existing Environment 22
Designing a New Storage Solution 22
General Configuration Considerations 23
4. DAS Configurations 25
Single-Controller DAS Configurations 25
Single-Controller DAS Tips and Techniques 26
Single-Controller DAS Setup Details 27
Dual-Controller Multipath DAS Configurations 29
Dual-Controller Multipath DAS Tips and Techniques 30
Sample Dual-Controller Multipath DAS Setup Details 31
5. SAN Configurations 35
Full-Duplex SAN Configurations 35
Full-Duplex SAN Tips and Techniques 36
Sample Full-Duplex SAN Setup Details 37
Redundant Point-to-Point SAN Configurations 39
Redundant Point-to-Point Tips and Techniques 40
Redundant Point-to-Point Setup Details 40
6. Using Multiple Expansion Units in High Capacity Configurations 43
Sun StorEdge 3510 FC Array High-Capacity Configurations 43
Limitations 44

Contents v
Connecting One Expansion Unit 45
Connecting Two Expansion Units 46
Connecting Three Expansion Units 47
Connecting Four Expansion Units 48
Connecting Five Expansion Units 49
Connecting Six Expansion Units 51
Connecting Seven Expansion Units 53
Connecting Eight Expansion Units 55
7. Sun StorEdge 3511 SATA Array High-Capacity Configurations 57
Connecting One Expansion Unit 58
Connecting Two Expansion Units 59
Connecting Three Expansion Units 60
Connecting Four Expansion Units 62
Connecting Five Expansion Units 64
8. Combining Sun StorEdge 3510 FC Arrays and Sun StorEdge 3511 SATA
Expansion Units 67
Restrictions on the Use of Expansion-Unit Combinations 67
Connecting Two FC Expansion Units and Three SATA Expansion Units to a Sun
StorEdge 3510 RAID Array 68

vi Sun StorEdge 3000 Family Best Practices Manual •March 2007

vii
Figures
FIGURE 3-1 DAS and SAN Storage Architectures 17
FIGURE 3-2 Two DAS Configurations 18
FIGURE 4-1 Three Single-Controller DAS Configurations 26
FIGURE 4-2 Sample Single-Controller Sun StorEdge 3510 FC DAS Connections 27
FIGURE 4-3 Sample Single-Controller Sun StorEdge 3511 SATA DAS Connections 28
FIGURE 4-4 Sample Dual-Controller Multipath DAS Configurations 29
FIGURE 4-5 Sample Dual-Controller Multipath Sun StorEdge 3510 DAS Connections 31
FIGURE 4-6 Sample Dual-Controller Multipath Sun StorEdge 3511 DAS Connections 32
FIGURE 5-1 Typical Full-Fabric SAN Configuration 35
FIGURE 5-2 Sample Sun StorEdge 3510 FC Full-Duplex SAN Connections 37
FIGURE 5-3 Sample Sun StorEdge 3511 SATA Full-Duplex SAN Connections 38
FIGURE 5-4 Sample Redundant Point-to-Point SAN Configuration 39
FIGURE 5-5 Sample Redundant Point-to-Point Sun StorEdge 3510 SAN Connections 41
FIGURE 5-6 Sample Redundant Point-to-Point Sun StorEdge 3511 SAN Connections 41
FIGURE 6-1 Sun StorEdge 3510 FC Array Configuration with One Expansion Unit 45
FIGURE 6-2 Sun StorEdge 3510 FC Array Configuration with Two Expansion Units 46
FIGURE 6-3 Sun StorEdge 3510 FC Array Configuration with Three Expansion Units 47
FIGURE 6-4 Sun StorEdge 3510 FC Array Configuration with Four Expansion Units 48
FIGURE 6-5 Sun StorEdge 3510 FC Array Configuration with Five Expansion Units 50
FIGURE 6-6 Sun StorEdge 3510 FC Array Configuration with Six Expansion Units 52

viii Sun StorEdge 3000 Family Best Practices Manual •March 2007
FIGURE 6-7 Sun StorEdge 3510 FC Array Configuration with Seven Expansion Units 54
FIGURE 6-8 Sun StorEdge 3510 FC Array Configuration with Eight Expansion Units 56
FIGURE 7-1 Sun StorEdge 3511 SATA Array Configuration with One Expansion Unit 58
FIGURE 7-2 Sun StorEdge 3511 SATA Array Configuration with Two Expansion Units 59
FIGURE 7-3 Sun StorEdge 3511 SATA Array Configuration with Three Expansion Units 61
FIGURE 7-4 Sun StorEdge 3511 SATA Array Configuration with Four Expansion Units 63
FIGURE 7-5 Sun StorEdge 3511 SATA Array Configuration with Five Expansion Units 65
FIGURE 8-1 Sun StorEdge 3510 FC Array with Two FC Expansion Units and Three SATA Expansion
Units 69

ix
Tables
TABLE 2-1 Maximum Number of Disks per Logical Drive 9
TABLE 2-2 Maximum Number of Supported Physical and Logical Drives, Partitions, and LUN
Assignments 10
TABLE 2-3 Actual Capacities per Drive 11
TABLE 2-4 Maximum Usable Storage Capacity Determined by RAID Level 11
TABLE 2-5 Default Stripe Size Per Optimization Mode (Kbyte) 13

xSun StorEdge 3000 Family Best Practices Manual •March 2007

xi
Preface
This manual gives an overview of Sun StorEdge™3510 Fibre Channel (FC) arrays
and Sun StorEdge 3511 SATA arrays, and presents several sample storage solutions
for entry-level, mid-range, and enterprise servers.
Caution – You should read the Sun StorEdge 3000 Family Safety, Regulatory, and
Compliance Manual before beginning any procedure in this manual.
How This Book Is Organized
This book includes the following topics:
Chapter 1 provides an overview of the Sun StorEdge 3510 FC array and Sun
StorEdge 3511 SATA array.
Chapter 2 provides an overview of important concepts and practices that underlie
the configurations you can use.
Chapter 3 helps you determine which direct-attached storage (DAS) and storage area
network (SAN) configurations are most appropriate to your requirements.
Chapter 4 presents several sample DAS configurations you can use.
Chapter 5 presents several sample SAN configurations you can use.
Chapter 6 describes high-capacity configurations involving multiple expansion units
and presents sample Sun StorEdge 3510 FC configurations that maximize reliability,
availability, and serviceability (RAS).
Chapter 7 presents sample high-capacity Sun StorEdge 3511 SATA configurations
that maximize reliability, availability, and serviceability.

xii Sun StorEdge 3000 Family Best Practices Manual •March 2007
Chapter 8 describes a sample high-capacity configuration that includes both Sun
StorEdge 3510 FC expansion units and Sun StorEdge 3511 SATA expansion units
connected to a Sun StorEdge 3510 FC RAID array
in a way that maximizes reliability,
availability, and serviceability.
Using UNIX Commands
This document might not contain information on basic UNIX®commands and
procedures such as shutting down the system, booting the system, and configuring
devices. Refer to the following for this information:
■Software documentation that you received with your system
■Solaris™ operating environment documentation, which is at
http://docs.sun.com
Shell Prompts
Shell Prompt
C shell machine-name%
C shell superuser machine-name#
Bourne shell and Korn shell $
Bourne shell and Korn shell superuser #

Preface xiii
Typographic Conventions
Related Documentation
Typeface*
*. The settings on your browser might differ from these settings.
Meaning Examples
AaBbCc123 The names of commands, files,
and directories; on-screen
computer output
Edit your.login.login file.
Use ls -a to list all files.
% You have mail.
AaBbCc123 What you type, when contrasted
with on-screen computer output
% su
Password:
AaBbCc123 Book titles, new words or terms,
words to be emphasized.
Replace command-line variables
with real names or values.
Read Chapter 6 in the User’s Guide.
These are called class options.
You must be superuser to do this.
To delete a file, type rm filename.
Title Part Number
Sun StorEdge 3510 FC Array and Sun StorEdge 3511 SATA Array Release Notes 817-6597
Sun StorEdge 3000 Family Installation, Operation, and Service Manual for the Sun
StorEdge 3510 FC Array and Sun StorEdge 3511 SATA Array
816-7300
Sun StorEdge 3000 RAID Firmware 4.2 User’s Guide 817-3711
Sun StorEdge 3000 Family Configuration Service 2.4 User’s Guide 817-3337
Sun StorEdge 3000 Family Diagnostic Reporter 2.4 User’s Guide 817-3338
Sun StorEdge 3000 Family Software Installation Manual 817-3764
Sun StorEdge 3000 Family CLI 2.4 User’s Guide 817-4951
Sun StorEdge 3000 Family Rack Installation Guide for 2U Arrays 817-3629
Sun StorEdge 3000 Family FRU Installation Guide 816-7326

xiv Sun StorEdge 3000 Family Best Practices Manual •March 2007
Accessing Sun Documentation
All Sun StorEdge 3000 family documentation is available online in both PDF and
HTML format at the following location:
http://www.sun.com/products-n-
solutions/hardware/docs/Network_Storage_Solutions/Workgroup/
You can view, print, or purchase a broad selection of Sun documentation at:
http://www.sun.com/documentation
Contacting Sun Technical Support
For late-breaking news and troubleshooting tips, review the Release Notes at the
locations shown in “Accessing Sun Documentation” on page xiv.
If you have technical questions about this product that are not answered in the
documentation, go to:
http://www.sun.com/service/contacting
To initiate or check on a USA-only service request, contact Sun support at:
800-USA-4SUN
To obtain international technical support, contact the sales office of each country at:
http://www.sun.com/service/contacting/sales.html
Sun StorEdge 3000 Family RAID Controller Firmware Migration Guide 819-6573
Sun StorEdge 3000 Family Safety, Regulatory, and Compliance Manual 816-7930
Title (Continued) Part Number

Preface xv
508 Accessibility Features
Sun StorEdge 3000 family documentation is available in Section 508-compliant
HTML files that can be used with assistive technology programs for visually
impaired personnel. These files are provided on the Documentation CD for your
product as well as on the websites identified in the previous “Accessing Sun
Documentation” section. Additionally, the software and firmware applications
provide keyboard navigation and shortcuts that are documented in the user's guides.
Sun Welcomes Your Comments
Sun is interested in improving its documentation and welcomes your comments and
suggestions. You can submit your comments by going to:
http://www.sun.com/hwdocs/feedback
Please include the title and part number of your document with your feedback:
Sun StorEdge 3000 Family Best Practices Manual, part number 816-7325-22.

xvi Sun StorEdge 3000 Family Best Practices Manual •March 2007

1
CHAPTER 1
Introduction
This Best Practices Manual describes the use of Sun StorEdge 3510 FC arrays and
Sun StorEdge 3511 SATA arrays, as well as the use of their corresponding expansion
units. It complements the Sun StorEdge 3000 Family Installation, Operation, and
Service Manual for these products.
This manual gives a high level overview of Sun StorEdge 3510 FC arrays and Sun
StorEdge 3511 SATA arrays, and presents several sample storage solutions for entry-
level, mid-range, and enterprise servers. Use these solutions as-is or tailor them to fit
your exact needs. Examples of customization opportunities include adding disks,
enclosures and software, or even combining configurations. Choosing the solution
that best matches your particular environment will provide the best results.
Executive Summary
Before reading the detailed information in this document, review this high-level
overview of factors that affect the reliability, availability, and serviceability (RAS) of
not only your storage arrays but your system as a whole.
RAS Features of Sun StorEdge 3000 Family Arrays
The following subjects are described in more detail in this document:
■Dual-controller configurations
Use dual-controller arrays to avoid a single point of failure. A dual-controller FC
array features a default active-to-active controller configuration. This
configuration improved application availability because, in the unlikely event of a
controller failure, the array automatically fails over to a second controller,
resulting in no interruption of data flow.

2Sun StorEdge 3000 Family Best Practices Manual •March 2007
■Logical drives
The best practice for creating logical drives is to add them evenly across the
primary and secondary controllers. With at least one logical drive assigned to
each controller, both controllers are active. This configuration is known as an
active-active controller configuration and allows maximum use of a dual-
controller array's resources.
■Power redundancy
To ensure power redundancy, connect the two power modules to two separate
circuits, such as one commercial circuit and one UPS.
■FC cabling to both controllers
When attaching FC cables for LUNs configured with redundant paths, make sure
one cable is connected to a channel on the upper controller and the other cable is
connected to a different channel on the lower controller. Then, if multipathing
software is configured on the host, a controller can be hot-swapped in the event of
failure without losing access to the LUN.
■Cache optimization
The RAID array’s cache optimization mode determines the cache block size used
by the controller for all logical drives:
■For sequential optimization, the cache block size is 128 Kbyte.
■For random optimization, the cache block size is 32 Kbyte.
■User-configurable stripe size
When you create a logical drive, you can replace the default stripe size with one
that better suits your application.
■For sequential optimization, available stripe size choices include 16 Kbyte, 32
Kbyte, 64 Kbyte, 128 Kbyte, and 256 Kbyte.
■For random optimization, available stripe size choices include 4 Kbyte, 8
Kbyte, 16 Kbyte, 32 Kbyte, 64 Kbyte, 128 Kbyte, and 256 Kbyte.
■Cable to expansion units so there is no single point of failure
Representative sample cabling diagrams are presented in this document for a
number of expansion unit configurations.
Additional RAS Features You Can Use With FC
and SATA Arrays
Even with all the Sun StorEdge 3000 Family’s built-in RAS features and the best
practices described in this document, single points of failure can still occur. For
instance, some rare hardware failures—a midplane or bus failure, for
instance—could conceivably disable both controllers of an active-active

Chapter 1 Introduction 3
configuration. It is prudent to use other RAS best practices, such as traditional
backup-and-restore strategies and host-based multipathing and clustering, to ensure
network reliability. Sun Microsystems makes available a wide variety of white
papers and product documentation that provide specific information to help you
plan or upgrade the architecture of your systems.
■The Solaris 10 operating system provides built-in multipathing support for FC
storage devices. Consider upgrading to the Solaris 10 operating system in order to
take advantage of this multipath support. This functionality is described in the
Solaris Fibre Channel and Storage Multipathing Administration Guide:
http://docs-pdf.sun.com/819-0139/819-0139.pdf
Earlier versions of the Solaris operating system do not contain this built-in
multipathing support. Sun SAN Foundation Suite software, including Sun
StorEdge Traffic Manager, provides some of this functionality. For more
information, refer to the Sun StorEdge SAN Foundation Software 4.4 Guide to
Documentation:
http://docs-pdf.sun.com/817-3670-12/817-3670-12.pdf
■A feature article on Sun's BigAdmin System Administration Portal, “Backup,
Restore, and Disaster Recovery on a System With Zones Installed” describes
useful security procedures that take advantage of features in the Solaris operating
system and in third-party backup software, including some useful links to other
system administration documentation:
http://www.sun.com/bigadmin/features/articles/backup_zones.html
■Another useful feature article, “Installing and Configuring Sun Cluster 3.1 09/04
Software for High-Availability Applications” provides an excellent overview of
strategies “to design an environment where services can migrate between
individual servers so that a reboot of a single machine never causes a service
downtime. Sun's solution to architecting these highly available services is
combining the software product, Sun Cluster, with the appropriately sized
servers, shared storage, and network topology.” It also describes the interaction of
Sun Cluster environments with Volume Manager software, and contains an
excellent list of related links:
http://www.sun.com/bigadmin/features/articles/install_cluster.
html
■A useful IDC white paper sponsored by Sun Microsystems, “Meeting Business
Objectives Through Unified and Simplified Storage,” describes a wide spectrum
of present and emerging best practices, including tiered storage; unified data
replication; and automated provisioning, expansion, and data protection tasks
across heterogeneous storage systems. While the specific business cases
referenced in this article include some software features currently found in
higher-end arrays than the Sun StorEdge 3000 family, this functionality is rapidly
migrating to workgroup storage arrays:
http://www.sun.com/storage/white-papers/idc_virtualization.pdf

4Sun StorEdge 3000 Family Best Practices Manual •March 2007
■A more basic article for less experienced system administrators who are setting
up file servers, “Introduction to Creating a File Server,” provides a useful
overview of mirroring boot disks and using advanced RAID levels to distribute
data across multiple physical disks in an array:
http://www.sun.com/bigadmin/features/articles/file_server.html
■Performance issues and strategies are described in considerable detail in “Storage
System Bottlenecks and Their Solutions:”
http://www.sun.com/storage/white-papers/storage_bottlenecks.pdf
The objective of this white paper is to enable readers to:
■increase end-to-end performance by taking advantage of multi-threaded I/Os
to the storage system controller and its drives
■deliver predictable storage system performance by matching the storage
system configuration to workload requirements
■increase storage system throughput by distributing a workload across the
storage system
Differences Between Sun StorEdge 3510
FC Array and Sun StorEdge 3511 SATA
Array
The Sun StorEdge 3510 FC array is a next-generation Fibre Channel storage system
designed to provide direct attached storage (DAS) to entry-level, mid-range, and
enterprise servers, or to serve as the disk storage within a storage area network
(SAN). This solution features powerful performance and reliability, availability and
serviceability features using modern FC technology. As a result, the Sun StorEdge
3510 FC array is ideal for performance-sensitive applications and for environments
with many entry-level, mid-range, and enterprise servers, such as:
■Internet
■Messaging
■Database
■Technical
■Imaging
The Sun StorEdge 3511 SATA array is a Fibre Channel array that uses Serial ATA
drives. These arrays are best suited for inexpensive secondary storage applications
that are not mission-critical, where higher capacity drives are needed, and where
lower performance and less-than 7/24 availability is an option. This includes near-
line applications such as: