HGST Ultrastar Data60 H4060-J User manual

User Guide
Ultrastar®Data60
Regulatory Model: H4060-J
December 2018
Rev. 1.7
1ET1101
Long Live Data ™| www.hgst.com

Contents
Revision History..............................................................................................7
Copyright........................................................................................................9
Chapter 1 Overview..................................................................................10
1.1 Ultrastar®Data60 Description..........................................................10
1.2 System Architecture Overview.........................................................11
1.3 System Level Block Diagram...........................................................12
1.4 Ultrastar Data60 Specification Summary.........................................12
1.5 Ultrastar Data60 Layout...................................................................14
1.6 List of Customer Replaceable Units (CRUs)....................................15
1.7 LEDs................................................................................................16
1.7.1 Front and Rear IO LEDs...............................................................16
1.7.2 IOM LEDs.....................................................................................18
1.7.3 IOM Fan LED................................................................................19
1.7.4 PSU LED......................................................................................20
1.7.5 Rear Fan LED...............................................................................21
1.7.6 Drive Assembly LED....................................................................22
1.8 Supported SKUs..............................................................................22
1.9 List of Compatible Drives.................................................................23
Chapter 2 System Management Overview.............................................32
2.1 Firmware Features Overview...........................................................32
2.2 OOBM Management Overview........................................................33
2.3 SCSI Enclosure Services Page 02...................................................33
Chapter 3 Component Overviews...........................................................34
3.1 Chassis Description.........................................................................34
3.1.1 Chassis Specifications.................................................................34
ii
Contents
User Guide

3.1.2 Chassis Layout.............................................................................35
3.2 IOM Description...............................................................................36
3.2.1 IOM Specifications.......................................................................36
3.2.2 IOM Layout...................................................................................37
3.2.3 IOM Blank Description.................................................................38
3.3 PSU Description...............................................................................38
3.3.1 PSU Specifications.......................................................................39
3.3.2 PSU Layout..................................................................................40
3.4 Rear Fan Description.......................................................................41
3.4.1 Rear Fan Specifications...............................................................41
3.4.2 Rear Fan Layout...........................................................................42
3.5 IOM Fan Description........................................................................43
3.5.1 IOM Fan Specifications................................................................43
3.5.2 IOM Fan Layout............................................................................44
3.6 Rails Description..............................................................................45
3.6.1 Rails Specifications......................................................................45
3.6.2 Rails Layout..................................................................................46
3.6.3 Rear Cover Alignment Bracket Description..................................47
3.7 CMA Description..............................................................................47
3.7.1 CMA Specifications......................................................................48
3.7.2 CMA Layout..................................................................................49
3.7.3 CMA Cable Tray...........................................................................50
3.8 Drive Assembly Description.............................................................51
3.8.1 Drive Assembly Specifications.....................................................51
3.8.2 Drive Assembly Layout.................................................................52
3.8.3 2.5" Drive Carrier Description.......................................................53
3.8.3.1 Operating the 2.5" Drive Carrier...............................................53
3.8.4 Drive Blank Description................................................................55
Chapter 4 Part Replacement...................................................................57
4.1 Part Replacement Service Window.................................................57
iii
Contents
User Guide

4.2 IOM Replacement............................................................................57
4.3 PSU Replacement............................................................................60
4.4 Rear Fan Replacement....................................................................64
4.5 IOM Fan Replacement.....................................................................66
4.6 Drive Assembly Replacement..........................................................68
4.7 CMA Replacement...........................................................................71
4.8 Rails Replacement...........................................................................81
4.9 Chassis Replacement....................................................................110
4.10 Special Considerations for Cable Routing ...................................142
Chapter 5 System Management............................................................146
5.1 Firmware Upgrade.........................................................................146
5.1.1 Downloading Firmware from the Support Portal........................146
5.1.2 Linux Upgrade Preparation........................................................148
5.1.3 Linux Upgrade to New Firmware................................................150
5.1.4 Non-Automatic Firmware Activation in Linux..............................152
5.1.5 Windows Upgrade Preparation...................................................153
5.1.6 Windows Upgrade to New Firmware..........................................155
5.1.7 Non-Automatic Firmware Activation in Windows........................157
5.1.8 Upgrading Firmware with OOBM...............................................161
5.2 Zoning............................................................................................162
5.2.1 Before Zoning.............................................................................162
5.2.2 Predefined Zoning Configurations..............................................162
5.2.3 sg_senddiag Command.............................................................168
5.2.4 Enabling Zoning using Linux......................................................169
5.2.5 Disabling Zoning using Linux.....................................................172
5.2.6 Enabling Zoning using Windows................................................176
5.2.7 Disabling Zoning using Windows...............................................179
5.3 Partially Populated Enclosures.......................................................183
5.3.1 Partial Population Configurations...............................................183
5.3.2 Installing Drives..........................................................................186
iv
Contents
User Guide

5.4 Daisy Chaining...............................................................................188
5.4.1 Daisy Chaining Configurations...................................................188
5.4.2 One Host Cable Configurations..................................................191
5.4.3 Two Host Cable Configurations..................................................195
5.4.4 Cabling for Daisy Chaining.........................................................199
Chapter 6 Safety.....................................................................................204
6.1 Electrostatic Discharge..................................................................204
6.2 Optimizing Location........................................................................204
6.3 Power Connections........................................................................204
6.4 Power Cords...................................................................................205
6.5 Rackmountable Systems...............................................................205
6.6 Safety and Service.........................................................................205
6.7 Safety Warnings and Cautions.......................................................206
Chapter 7 Disclaimers............................................................................207
7.1 Restricted Access Location............................................................207
7.2 Safety Compliance.........................................................................207
7.3 Electromagnetic Compatibility (EMC) Class A Compliance ..........207
7.4 Country Certifications....................................................................208
Chapter 8 Regulatory Statements.........................................................209
8.1 Europe (CE Declaration of Conformity)..........................................209
8.2 FCC Class A Notice.......................................................................209
8.3 ICES-003 Class A Notice—Avis NMB-003, Classe A....................209
8.4 Japanese Compliance Statement, Class A ITE.............................209
8.5 Taiwan Warning Label Statement, Class A ITE.............................210
Chapter 9 Points of Contact..................................................................211
v
Contents
User Guide

Appendix A Site and System Operation Requirements............................212
A.1 Power Requirements.....................................................................212
A.2 Supported Operating Systems......................................................212
A.3 List of Supported Server Components..........................................212
A.4 Ultrastar Data60 Rack Requirements............................................214
A.4.1 Compatible Hardware Configuration..........................................215
A.5 ESD...............................................................................................216
A.6 Enclosure Cooling..........................................................................217
A.7 Host Connectivity...........................................................................217
vi
Contents
User Guide

Revision History
CommentRevisionDate
Initial ReleaseRevision 1.0November2017
Revision 1.1November2017 • The crossbar on the CMA was changed. See CMA
Description on page 47.
• Removed Lowline power specs from Detailed Power
Specifications.
• Changed required rack depth, see Required Rack Depth.
• Changed typical power consumption, see Typical Power
Consumption.
• Updated LED Flash Patterns, see LEDs on page 16.
Revision 1.2December2017 • Added active cable support
• Added firmware upgrade section. See Firmware Upgrade
on page 146.
• Added a note regarding single IOMs in SATA TLAs. See
the list of features.
• Updated the Non-Op altitude specification. See
Non-Operational Altitude.
Updated the product nameRevision 1.21January 2018
Revision 1.3January 2018 • Updated information on the 2.5" drive carrier option. See
2.5" Drive Carrier Description on page 53.
• Added torque requirements for all screws used in the
enclosure.
• Updated the IOM replacement section to account for the
possibility of a firmware mismatch. See IOM Replacement
on page 57.
•Updated the drive assembly installation instructions to clarify
the orientation of the drive assemblies. Drive Assembly
Replacement on page 68.
Revision 1.4April 2018 • Updated Compatible Drives List. See List of Compatible
Drives on page 23.
• Updated the Rack Requirements. See Ultrastar Data60
Rack Requirements on page 214.
•UpdatedtheFirmware Upgrades. See: Firmware Upgrade
on page 146
Added the Part Replacement Service Window. See: Part
Replacement Service Window on page 57
Revision 1.5May 2018
Revision 1.6June 2018 • Updated Compatible Drives List. See List of Compatible
Drives on page 23
• Updated the Firmware Upgrade section. See Firmware
Upgrade on page 146
• Updated the Firmware Download section. See
Downloading Firmware from the Support Portal on page
146
7
Revision History
User Guide

CommentRevisionDate • Updated the System Architecture Overview section. See
System Architecture Overview on page 11
• Updated the Daisy Chaining section. See Daisy Chaining
on page 188
• Added the Rear Cover Alignment Bracket Description.
SeeRear Cover Alignment Bracket Description on page
47
Revision 1.7November2018 • Updated the images in the Daisy Chaining section. See
Daisy Chaining on page 188
•Updated List of CRUs. See List of Customer Replaceable
Units (CRUs) on page 15
• Updated Compatible Drives List. See List of Compatible
Drives on page 23
• Updated the Host Connectivity section. See: Host
Connectivity on page 217
8
Revision History
User Guide

Copyright
The following paragraph does not apply to the United Kingdom or any country where such provisions
are inconsistent with local law: HGST a Western Digital company PROVIDES THIS PUBLICATION "AS
IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Some states do not allow disclaimer or express or implied warranties in certain transactions,
therefore, this statement may not apply to you.
This publication could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. HGST may
make improvements or changes in any products or programs described in this publication at any time.
It is possible that this publication may contain reference to, or information about, HGST products (machines
andprograms),programming, or services that are not announced in yourcountry.Suchreferences or information
must not be construed to mean that HGST intends to announce such HGST products, programming, or
services in your country.
Technical information about this product is available by contacting your local HGST representative or on the
Internet at: support.hgst.com
HGST may have patents or pending patent applications covering subject matter in this document. The furnishing
of this document does not give you any license to these patents.
Copyright ©2017-2018 Western Digital Corporation or its affiliates.
Western Digital
5601 Great Oaks Parkway
San Jose, CA 95119
Long Live Data™is a trademark of HGST, Inc. and its affiliates in the United States and/or other countries.
HGST trademarks are authorized for use in countries and jurisdictions in which HGST has the right to use,
market and advertise the brands.
Other product names are trademarks or registered trademarks of their respective owners.
One MB is equal to one million bytes, one GB is equal to one billion bytes, one TB equals 1,000GB (one trillion
bytes) and one PB equals 1,000TB when referring to storage capacity. Usable capacity will vary from the raw
capacity due to object storage methodologies and other factors.
References in this publication to HGST products, programs or services do not imply that HGST intends to
make these available in all countries in which HGST operates.
Product information is provided for information purposes only and does not constitute a warranty.
Information is true as of the date of publication and is subject to change. Actual results may vary. This publication
is for general guidance only. Photographs may show design models.
9
Copyright
User Guide

Chapter
1
1 Overview
This section provides a high level overview of the features of the Ultrastar Data60.
1.1 Ultrastar®Data60 Description
Figure 1: Ultrastar Data60
The Ultrastar®Data60 is a 4U form factor, high availability, high density, rack-mounted storage enclosure that
is capable of hosting up to 60 SAS or SATA drives. The maximum data storage capacity of the Ultrastar
Data60 is 840 TB using 14TB HGST Ultrastar®HC530 drives . (For a full list of compatible drives and total
storage capacities, see the List of Compatible Drives on page 23.) The enclosure runs on an input voltage
of 200 - 240 VAC and consumes ~1000W of power under typical conditions. It requires a maximum of ~1250W
at full load.
Note: Max and Typical Power Consumption values represent the output power to the system. Input
power will vary depending on the PSU efficiency and load sharing between PSUs
It is designed to fit within a 4U rack space and requires 900mm (35.43in.) usable rack space, frame to frame.
A fully loaded system will add 79.4 kg. / 175 lbs. of static load when fully loaded with drives.
• 4U Storage Enclosure
• Supports up to 60 Drives
• Can support 3.5” drives and 2.5” SSD drives (2.5" requires an adapter) in the 60 available drive bays.
• Up to 12W per drive slot for the 60 data storage drives (Cannot exceed 85A on the 5V rail)
• House and control four (4) N+1 redundant 80mm rear fans
• House and control a dual rotor 40mm internal IOM Fan
• Controlled by two (2) redundant I/O Modules
• Powered by two (2) redundant 1600W PSUs
• Full high availability with independent dual paths to all HDDs
• Toolless replacement of all Customer Replaceable Units (CRUs)
• Fits within a standard EIA-310 rack including all necessary cable management
• Supports up to 4m passive SAS cables and also supports active cables
• Supports Highline Input Power
1SATA based models will only include 1 IOM
10
1 Overview
User Guide 1.1 Ultrastar®Data60 Description

1.2 System Architecture Overview
The Ultrastar Data60 IOM uses a cascaded expander design to allow for connection to all 60 drives. A 48-port
primary expander connects with the six host ports, has a x3 link to the other IOM for IOM-IOM communication
and syncing, and also has a x10 SAS link to each secondary expander. Each secondary expander then
connects with fifty-one (51) drives.
The out-of-band management microprocessor provides an Ethernet connection using a Redfish/RESTful API
to access the various enclosure services. All the SES enclosure information can be obtained through the
out-of-band management port. Major use cases for this feature include obtaining storage subsystem health
information, locating enclosure components using the IDENT LEDs, and updating firmware.
The system FPGAs control and report the states of the system fans, enclosure LEDs,connector LEDs, drive
LEDs, and T10 drive power disable signals on the 60 data storage drives.
Note: To use T10 power disable, the drives installed must also support this
feature.
The I2C architecture is designed to support only one single master on any given bus. The primary expander
will be the master on each I2C bus. There are eight I2C buses used in the Ultrastar Data60 enclosure. The
number of devices on each bus are balanced to allow communication to peripheral devices and not overload
any one bus. The devices connected on the I2C buses include the enclosure VPDs, temp sensors, baseboard
FPGAs, and SAS connectors among others.
11
1 Overview
User Guide 1.2 System Architecture Overview

1.3 System Level Block Diagram
The following image shows the system block diagram for the Ultrastar Data60. .
Figure 2: System Block Diagram
1.4 Ultrastar Data60 Specification Summary
Table 1: Environmental Specification Summary
OperationalNon-OperationalSpecication
5°C to 35°C-40°C to 70°CTemperature
20°C per hour maximum30°C / HrTemperature Gradient
1°C per 125m above 950m1°C per 300m above 3000mTemperature De-rating
8-90% Non-Condensing8-90% Non-CondensingRelative Humidity
30% per hour maximum30% per hour maximumRelative Humidity Gradient
12
1 Overview
User Guide 1.3 System Level Block Diagram

OperationalNon-OperationalSpecication
-300m to 3048m / -984 ft. to 10,000 ft.-300m to 12,000m / -984 ft. to 39,370 ftAltitude
Table 2: Electrical Specifications
ValueSpecication
~1250WMax Power Consumption
~1000WTypical Power Consumption
Note: Max and Typical Power Consumption
values represent the output power to the
system. Input power will vary depending on
thePSUefficiencyandloadsharingbetween
PSUs
200 - 240 VACInput Voltage
C14PSU Connector Type
80 PLUS PlatinumPSU Efciency
AC line inrush current shall not exceed 40A peak for up
to one-quarter of the AC cycle, after which, the input
Inrush Current Maximum (per PSU)
current should be no more than the specied maximum
input current.
Caution: The Ultrastar Data60 can only be plugged into highline. If the unit is plugged into lowline, the
PSU will report a "Critical" state when status pages are queried using SES. In this case, the enclosure
will power up, but the drives will not. The enclosure will remain in low-power mode.
Table 3: Mechanical Specifications
OperationalNon-OperationalSpecication
5G, 0 - peak, 11ms half sine; 3
positive and 3 negative pulses in each
10G, 0 - peak,11ms half sine; 3
positive and 3 negative pulses in each
axis Shock
Shock
axis- minimum 6 seconds between
shocks to allow for write/read
recovery
0.10G,0 - peak swept sine; 5 -500Hz;
1 complete sweep @ 1/2octave per
minute
0.75G, 0 - peak swept sine; 5 -500Hz;
1 complete sweep @ 1/2 octave per
minute
Vibration
79.4 kg. / 175 lbs.Weight
447mm x 890mm x 175mm / 17.67in. x 35.04in. x 6.89in.Dimensions
712 mm / 28.03in.Length without CMA
13
1 Overview
User Guide 1.4 Ultrastar Data60 Specification Summary

OperationalNon-OperationalSpecication
450mm (17.72in.) with 465mm (18.31in.) ± 1.5mm nominal hole spacing.
See EIA-310 Rack Standard
Required Rack Width
900mm (35.43in.) usable rack space, frame to frameRequired Rack Depth
4URack Units (U)
24 in. - 32 in.Vertical Rack Rail Spacing
Table 4: Performance Specifications
ValueSpecication
60Number of Drive Slots
12Gbps SAS / 6Gbps SATAData Transfer Rates
840 TB using 14TB HGST Ultrastar®HC530 drivesMax Raw Data Storage Capacity
12 xMini-SAS HD ( 6 per IOM)
2x10/100/1G Ethernet
SAS Ports
1.5 Ultrastar Data60 Layout
Figure 3: Front and Rear Product Layout
6
2
115
3 4
Table 5: Front and Rear Component Identification
ComponentNumber
Enclosure Handles1
CMAs2
CMA Tray3
Rear Fans4
14
1 Overview
User Guide 1.5 Ultrastar Data60 Layout

ComponentNumber
PSUs5
Chassis Cover6
Rear Alignment Brackets7
Rails8
The following is an image of the layout of the major system components inside the Ultrastar Data60.
Figure 4: Component Layout
1.6 List of Customer Replaceable Units (CRUs)
The following table lists the replaceable components and their part numbers.
Table 6: List of Replaceable Components
Part NumberComponent
1EX0442Ultrastar Data60 Chassis with a single IOM and PSU
1EX0443Ultrastar Data60 Chassis with IOMs and PSUs
1EX0430IOM
1EX0431IOM Blank
1EX0434PSU 1600W
1EX0433Rear Fan
1EX0432IOM Fan
1EX0435Rails
1EX1174CMA
1EX1119CMA Cable Tray
1EX04383.5 in. Drive Carrier, Qty=1
15
1 Overview
User Guide 1.6 List of Customer Replaceable Units (CRUs)

Part NumberComponent
1EX04393.5 in. to 2.5 in. conversion Drive Carrier, Qty=1
1EX04293.5 in. Drive Blank, Qty=1
1EX1530Power Cable for PDU, C13-C14, 18AWG, 3m, Qty=1
1EX1161Dual Port 1/10/25GbE Unied Wire Adapter
1EX1533HD Mini-SAS to HD Mini-SAS, 3m, Qty=2
1.7 LEDs
1.7.1 Front and Rear IO LEDs
The Ultrastar Data60 has a number of LEDs on the exterior of the enclosure that display various system
statuses. There are three on the front and three on the rear that mirror each other and provide general status.
This allows the status of the enclosure to be visible from either side of the rack.
Figure 5: Front LEDs
12 3
From the rear there are the three enclosure status LEDs and LEDs on the Ethernet and SAS ports.
Table 7: Front LED Identification
BehaviorColorLED NameNumber
Blink @ 1 Hz – Blinks only when Identication has been
activated. Will blink when any component is identied.
BlueIdentify1
Blink @ 1 Hz –Enclosure has a fault
AmberFault2
Off – Enclosure has no fault
16
1 Overview
User Guide 1.7 LEDs

BehaviorColorLED NameNumber
Solid - Powered OnGreenPower3
Figure 6: Rear IO LEDs
Table 8: Rear LED Identification
BehaviorColorLED NameNumber
Solid – SAS Cable Connected
GreenSAS Link
Status
1
Off – SAS Cable Not Connected
Blink @ 1 Hz – SAS connection fault
AmberSAS Fault
Status
2
Off – No SAS connection fault
Blink @ 1 Hz – Blinks only when Identication has been
activated. Will blink when any component is identied.
BlueIdentication3
Blink @ 1 Hz –Enclosure has a fault
AmberFault4
Off – Enclosure has no fault
Solid - Powered OnGreenPower5
Off - Operating at 10 Mbps
Green Solid - Operating at 100 Mbps
Amber Solid - Operating at 1Gpbs
Green/AmberEthernet
Connectors
Link/Activity
6
Off - No Connection
Solid - Connected
Blink - Activity
GreenEthernet
Connector
Speed
7
17
1 Overview
User Guide 1.7 LEDs

1.7.2 IOM LEDs
The IOM has three LEDs, one each for power, fault, and identification.
Figure 7: IOM LEDs
Table 9: IOM LED Identification
BehaviorColorLED Name
Blink @ 1 Hz – Blinks only when
IOM Identication has been activated.
Off - Not being identied
BlueIOM Identication
Solid - IOM has Fault
Off - IOM is functioning normally
AmberIOM Fault
Solid - IOM is on
Off - IOM is off
GreenIOM Power
18
1 Overview
User Guide 1.7 LEDs

1.7.3 IOM Fan LED
The IOM Fan has a single LED that has three distinct states, one each for a fault condition, identification, and
power off.
Figure 8: IOM Fan LED
1
Table 10: IOM Fan LED Identification
BehaviorColorLED Name
Solid – IOM Fan is reporting faults
Blinking @ 1 Hz – IOM Fan is being
AmberIOM Fan LED
identied
Off – IOM Fan is on and reporting no
faults
19
1 Overview
User Guide 1.7 LEDs

1.7.4 PSU LED
The PSU has a single multi-function LED. See the table below for a detailed functional description.
Figure 9: PSU LEDs
1
Table 11: PSU LED Identification
BehaviorColorLED Name
Solid – PSU is on and reporting no faults
Blinking @ 2Hz – PSU in rmware update mode
Off – PSU is disconnected from power
GreenPSU Multi-function LED
Solid – PSU is disconnected from power or critical fault
causing a shutdown failure
Amber
Blinking @ 0.5Hz – PSU reporting warnings
Off – PSU is reporting no faults
20
1 Overview
User Guide 1.7 LEDs