IBM 3000 6.0.2 User manual

IBM Elastic Storage System 3000
6.0.2
Service Guide
IBM
SC28-3187-01

Note
Before using this information and the product it supports, read the information in “Notices” on page
61.
This edition applies to version 6 release 0 modication 2 of the following product and to all subsequent releases and
modications until otherwise indicated in new editions:
• IBM Spectrum® Scale Data Management Edition for IBM® ESS (product number 5765-DME)
• IBM Spectrum Scale Data Access Edition for IBM ESS (product number 5765-DAE)
IBM welcomes your comments; see the topic “How to submit your comments” on page xi. When you send information
to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without
incurring any obligation to you.
© Copyright International Business Machines Corporation 2019, 2021.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM Corp.

Contents
Figures.................................................................................................................. v
Tables................................................................................................................. vii
About this information.......................................................................................... ix
Who should read this information...............................................................................................................ix
IBM Elastic Storage System information units........................................................................................... ix
Related information.....................................................................................................................................ix
Conventions used in this information.......................................................................................................... x
How to submit your comments...................................................................................................................xi
Chapter 1. Events.................................................................................................. 1
Array events................................................................................................................................................. 1
Canister events.............................................................................................................................................2
Enclosure events..........................................................................................................................................4
Physical disk events..................................................................................................................................... 8
Recovery group events.............................................................................................................................. 11
Server events............................................................................................................................................. 11
Virtual disk events......................................................................................................................................16
Chapter 2. Servicing ............................................................................................19
Removing and replacing a drive................................................................................................................ 19
Removing and replacing a drive blank ......................................................................................................23
Removing and replacing a power supply unit........................................................................................... 23
Removing and replacing a power interposer............................................................................................ 26
Miscellaneous equipment specication (MES) instructions.....................................................................28
ESS 3000 storage drives MES upgrade................................................................................................28
ESS 3000 adapter MES upgrade: EC64 and EC67.............................................................................. 30
ESS 3000 adapter MES upgrade: AJP1............................................................................................... 37
ESS 3000 memory MES upgrade......................................................................................................... 43
ESS 3000 storage drives concurrent MES upgrade.............................................................................47
Example: Manually restarting GPFS on the ESS 3000 canisters..............................................................52
Chapter 3. Part Listings........................................................................................55
CRU part number list..................................................................................................................................55
FRU part number list..................................................................................................................................55
Cable part number list............................................................................................................................... 56
Accessibility features for the system.................................................................... 59
Accessibility features.................................................................................................................................59
Keyboard navigation.................................................................................................................................. 59
IBM and accessibility.................................................................................................................................59
Notices................................................................................................................61
Trademarks................................................................................................................................................ 62
Glossary..............................................................................................................63
iii

Index.................................................................................................................. 71
Index.................................................................................................................. 71
iv

Figures
1. Unlocking the drive and release latch........................................................................................................ 21
2. Removing the drive......................................................................................................................................21
3. Inserting the new drive............................................................................................................................... 22
4. Completing the drive installation................................................................................................................22
5. Correct drive blank orientation................................................................................................................... 23
6. Details of Power Supply Units in the management GUI.............................................................................24
7. Features of a power supply unit..................................................................................................................25
8. Removing the power supply unit ................................................................................................................26
9. Sliding out the power interposer................................................................................................................ 27
10. Removing a power interposer...................................................................................................................27
11. Inserting the new power interposer.........................................................................................................28
12. Ethernet ports on canister 1 (upper canister)..........................................................................................31
13. Ethernet ports on canister 2 (lower canister).......................................................................................... 31
14. Ethernet ports on canister 1 (upper canister)..........................................................................................38
15. Ethernet ports on canister 2 (lower canister).......................................................................................... 38
v

vi

Tables
1. Conventions................................................................................................................................................... x
2. Events for the Array component................................................................................................................... 1
3. Events for the Canister component.............................................................................................................. 2
4. Events for the Enclosure component............................................................................................................4
5. Events for the physical disk component....................................................................................................... 8
6. Events for the Recovery group component................................................................................................ 11
7. Server events...............................................................................................................................................11
8. Events for the virtual disk component........................................................................................................16
9. CRU Part Numbers...................................................................................................................................... 55
10. FRU Part Numbers.................................................................................................................................... 55
11. Cable Part Numbers..................................................................................................................................56
vii

viii

About this information
Who should read this information
This information is intended for administrators of IBM Elastic Storage® System (ESS) that includes IBM
Spectrum Scale RAID.
IBM Elastic Storage System information units
IBM Elastic Storage System (ESS) 3000 documentation consists of the following information units.
Information unit Type of information Intended users
Hardware Planning
and Installation
Guide
This unit provides ESS 3000 information
including technical overview, planning,
installing, troubleshooting, and cabling.
System administrators and IBM
support team
Quick Deployment
Guide
This unit provides ESS information including the
software stack, deploying, upgrading, and best
practices.
System administrators,
analysts, installers, planners,
and programmers of IBM
Spectrum Scale clusters who
are very experienced with the
operating systems on which
each IBM Spectrum Scale
cluster is based
Service Guide This unit provides ESS 3000 information
including events, servicing, and parts listings.
System administrators and IBM
support team
Problem
Determination Guide
This unit provides ESS 3000 information
including setting up call home, replacing
servers, issues, maintenance procedures, and
troubleshooting.
System administrators and IBM
support team
Command Reference This unit provides information about ESS
commands and scripts.
System administrators and IBM
support team
IBM Spectrum Scale
RAID:
Administration
This unit provides IBM Spectrum Scale RAID
information including administering, monitoring,
commands, and scripts.
• System administrators of IBM
Spectrum Scale systems
• Application programmers
who are experienced with
IBM Spectrum Scale systems
and familiar with the
terminology and concepts in
the XDSM standard
Related information
Related information
For information about:
• IBM Spectrum Scale, see:
http://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html
• mmvdisk command, see mmvdisk documentation.
© Copyright IBM Corp. 2019, 2021 ix

• DCS3700 storage enclosure, see:
–System Storage® DCS3700 Quick Start Guide, GA32-0960-04:
https://www-01.ibm.com/support/docview.wss?uid=ssg1S7005178
–IBM System Storage DCS3700 Storage Subsystem and DCS3700 Storage Subsystem with Performance
Module Controllers: Installation, User's, and Maintenance Guide, GA32-0959-07:
http://www.ibm.com/support/docview.wss?uid=ssg1S7004920
• For information about the IBM Power Systems EXP24S I/O Drawer (FC 5887), see IBM Knowledge
Center :
http://www.ibm.com/support/knowledgecenter/8247-22L/p8ham/p8ham_5887_kickoff.htm
• Detailed information about the IBM Spectrum Scale installation toolkit, see Using the installation toolkit
to perform installation tasks: Explanations and examples.
• CES HDFS, see Adding CES HDFS nodes into the centralized le system.
• Installation toolkit ESS support, see ESS awareness with the installation toolkit.
• IBM POWER8® servers, see IBM Knowledge Center:
http://www.ibm.com/support/knowledgecenter/POWER8/p8hdx/POWER8welcome.htm
• Extreme Cluster/Cloud Administration Toolkit (xCAT), go to the xCAT website :
http://xcat.org/
– xCAT 2.16.1 Release Notes®
For the latest support information about IBM Spectrum Scale RAID, see the IBM Spectrum Scale RAID
FAQ in IBM Knowledge Center:
http://www.ibm.com/support/knowledgecenter/SSYSP8/gnrfaq.html
Conventions used in this information
Table 1 on page x describes the typographic conventions used in this information. UNIX le name
conventions are used throughout this information.
Table 1. Conventions
Convention Usage
bold Bold words or characters represent system elements that you must use literally,
such as commands, flags, values, and selected menu options.
Depending on the context, bold typeface sometimes represents path names,
directories, or le names.
bold
underlined
bold underlined keywords are defaults. These take effect if you do not specify a
different keyword.
constant width Examples and information that the system displays appear in constant-width
typeface.
Depending on the context, constant-width typeface sometimes represents path
names, directories, or le names.
italic Italic words or characters represent variable values that you must supply.
Italics are also used for information unit titles, for the rst use of a glossary term,
and for general emphasis in text.
<key>Angle brackets (less-than and greater-than) enclose the name of a key on the
keyboard. For example, <Enter> refers to the key on your terminal or workstation
that is labeled with the word Enter.
x About this information

Table 1. Conventions (continued)
Convention Usage
\In command examples, a backslash indicates that the command or coding example
continues on the next line. For example:
mkcondition -r IBM.FileSystem -e "PercentTotUsed > 90" \
-E "PercentTotUsed < 85" -m p "FileSystem space used"
{item}Braces enclose a list from which you must choose an item in format and syntax
descriptions.
[item]Brackets enclose optional items in format and syntax descriptions.
<Ctrl-x>The notation <Ctrl-x> indicates a control character sequence. For example,
<Ctrl-c> means that you hold down the control key while pressing <c>.
item... Ellipses indicate that you can repeat the preceding item one or more times.
|In synopsis statements, vertical lines separate a list of choices. In other words, a
vertical line means Or.
In the left margin of the document, vertical lines indicate technical changes to the
information.
How to submit your comments
To contact the IBM Spectrum Scale development organization, send your comments to the following
email address:
[email protected]
About this information xi

xii IBM Elastic Storage System 3000: Service Guide

Chapter 1. Events
The recorded events are stored in the local database on each node. The user can get a list of recorded
events by using the mmhealth node eventlog command. Users can use the mmhealth node show
or mmhealth cluster show commands to display the active events in the node and cluster
respectively.
The recorded events can also be displayed through the GUI.
When you upgrade to IBM Spectrum Scale 5.0.5.3 or a later version, the nodes where no sqlite3
package is installed have their RAS event logs converted to a new database format to prevent known
issues. The old RAS event log is emptied automatically. You can verify that the event log is emptied either
by using the mmhealth node eventlog command or in the IBM Spectrum Scale GUI.
Note: The event logs are updated only the rst time IBM Spectrum Scale is upgraded to version 5.0.5.3 or
higher.
The following sections list the RAS events that are applicable to various components of the IBM Spectrum
Scale system:
Array events
The following table lists the events that are created for the Array component.
Table 2. Events for the Array component
Event Event Type Severity Message Description Cause User Action
gnr_array_found INFO_ADD_ENTITY INFO GNR declustered
array {0} was
found.
A GNR
declustered
array listed in the
IBM Spectrum
Scale
conguration
was detected.
N/A
gnr_array_needsser
vice
STATE_CHANGE WARNIN
G
GNR declustered
array {0} needs
service.
The declustered
array state needs
service.
N/A N/A
gnr_array_ok STATE_CHANGE INFO GNR declustered
array {0} is ok.
The declustered
array state is ok.
N/A N/A
gnr_array_unknown STATE_CHANGE WARNIN
G
GNR declustered
array {0} is in
unknown state.
The declustered
array state is
unknown.
N/A N/A
gnr_array_vanished INFO_DELETE_ENTI
TY
INFO GNR declustered
array {0} has
vanished.
A GNR
declustered
array listed in the
IBM Spectrum
Scale
conguration
was not
detected.
A GNR declustered array, listed in the
IBM Spectrum Scaleconguration as
mounted before, is not found. This
could be a valid situation
Run the
mmlsrecoverygrou
p command to verify
that all the expected
GNR declustered
arrays exist.
© Copyright IBM Corp. 2019, 2021 1

Canister events
The following table lists the events that are created for the Canister component.
Table 3. Events for the Canister component
Event Event Type Severity Message Description Cause User Action
bootdrive_mirror_degraded STATE_CHANGE WARNING The bootdrive's
mirroring is
degraded.
The bootdrive's
mirroring is
degraded.
The
tsplatformstat
-a command returns
a DEGRADED value
for at least one
partition.
N/A
bootdrive_installed STATE_CHANGE INFO The bootdrive
attached to port {0}
is available.
The bootdrive is
available.
The
tsplatformstat
-a command returns
the bootdrives as
expected.
N/A
bootdrive_mirror_failed STATE_CHANGE ERROR The bootdrive's
mirroring is failed.
The bootdrive's
mirroring is failed.
The
tsplatformstat
-a command returns
a FAILED value for
at least one partition.
N/A
bootdrive_mirror_ok STATE_CHANGE INFO The bootdrive's
mirroring is OK.
The bootdrive's
mirroring is OK.
The
tsplatformstat
-a command returns
optimal for all
partitions.
N/A
bootdrive_mirror
_uncongured
STATE_CHANGE WARNING The bootdrive's
mirroring is
uncongured.
The bootdrive's
mirroring is
uncongured.
The
tsplatformstat
-a command returns
uncongured for
mirroring.
N/A
bootdrive_missing STATE_CHANGE ERROR The bootdrive on
port {0} is missing or
dead.
One bootdrive is
missing or dead.
Redundancy is not
given anymore.
The
tsplatformstat
-a command returns
only one instead of
two bootdrives. Two
drives are expected
to ensure
redundancy.
Inspect that the drive is
correctly installed on the
referenced port. Else
insert or replace the drive.
bootdrive_smart_failed STATE_CHANGE ERROR The smart
assessment of
bootdrive {0}
attached to port {1}
does not return OK.
The bootdrive's
smart assessment
does not return OK.
The
tsplatformstat
-a command does
not return a PASSED
value in the
selfAssessment
eld for the
bootdrive.
Verify the smart status of
the bootdrive using
tsplatformstat
command or smartctl.
bootdrive_smart_ok STATE_CHANGE INFO The smart
assessment of
bootdrive {0}
attached to port {1}
returns OK.
The bootdrive's
smart assessment
returns OK.
The
tsplatformstat
-a command returns
a PASSED in the
selfAssessment
eld for the
bootdrive.
N/A
can_fan_failed STATE_CHANGE WARNING Fan {0} is failed. The fan state is
failed.
The
mmlsenclosure
command reports
the fan as failed.
Check the fan status by
using the
mmlsenclosure
command. Replace the
fan module in the
canister.
can_fan_ok STATE_CHANGE INFO Fan {0} is OK. The fan state is OK. The
mmlsenclosure
command reports
the fan as working.
N/A
can_temp_bus_failed STATE_CHANGE WARNING Temperature sensor
{0} I2C bus is failed.
The temperature
sensor I2C bus
failed.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command.
2 IBM Elastic Storage System 3000: Service Guide

Table 3. Events for the Canister component (continued)
Event Event Type Severity Message Description Cause User Action
can_temp_high_critical STATE_CHANGE WARNING Temperature sensor
{0} measured a high
temperature value.
The temperature
exceeded the actual
high critical
threshold value for at
least one sensor.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command.
can_temp_high_warn STATE_CHANGE WARNING Temperature sensor
{0} measured a high
temperature value.
The temperature
exceeded the actual
high warning
threshold value for at
least one sensor.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command.
can_temp_low_critical STATE_CHANGE WARNING Temperature sensor
{0} measured a low
temperature value.
The temperature has
fallen below the
actual low critical
threshold value for at
least one sensor.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command.
can_temp_low_warn STATE_CHANGE WARNING Temperature sensor
{0} measured a low
temperature value.
The temperature has
fallen below the
actual low warning
threshold value for at
least one sensor.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command.
can_temp_sensor_failed STATE_CHANGE WARNING Temperature sensor
{0} is failed.
The temperature
sensor state is failed.
The
mmlsenclosure
command reports
the temperature
sensor with a failure.
Check the temperature
status by using the
mmlsenclosure
command. Replace the
canister.
can_temp_sensor_ok STATE_CHANGE INFO Temperature sensor
{0} is OK.
The temperature
sensor state is OK.
N/A N/A
canister_failed STATE_CHANGE ERROR Canister {0} is failed. The canister is
reporting a failed
hardware state. This
might be caused by a
failure of an
underlying
component. For
example, the fan.
The
mmlsenclosure
command reports
the canister as failed.
Check for detailed error
events of canister
components by using the
mmhealth command.
Inspect the output of
mmlsenclosure all -
L command for the
referenced canister.
canister_ok STATE_CHANGE INFO Canister {0} is OK. The canister state is
OK.
The
mmlsenclosure
command reports
the canister as failed.
N/A
cpu_inspection_failed STATE_CHANGE ERROR The inspection of the
CPU slots found a
mismatch
Number of populated
CPU slots, number of
enabled CPUs,
number of CPU
cores, number of
CPU threads or CPU
speed is not as
expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned an
InspectionPasse
d unequal to
True value.
Check for specic events
related to CPUs by using
the mmhealth command.
Inspect the output of the
ess3kplt command for
details.
cpu_inspection_passed STATE_CHANGE INFO The CPUs of the
canister are OK.
The CPU speed and
number of populated
CPU slots is as
expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned an
InspectionPasse
d equal to True
value.
N/A
cpu_speed_ok STATE_CHANGE INFO The CPU speed is
OK.
The speed of all
CPUs is as expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned no speed
errors.
N/A
cpu_speed_wrong STATE_CHANGE ERROR One or more CPUs
have an
unsupported speed.
The speed of one or
more CPUs is not as
expected. This
conguration is not
supported.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned one or more
speed errors.
Inspect the output of the
ess3kplt command to
see which CPUs have an
unsupported speed.
Chapter 1. Events 3

Table 3. Events for the Canister component (continued)
Event Event Type Severity Message Description Cause User Action
dimm_inspection_failed STATE_CHANGE ERROR The inspection of the
memory dimm slots
found a failure.
The capacity, speed,
or number of
populated dimm
slots is not as
expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned an
InspectionPasse
d unequal to
True value.
Check for specic events
related to dimms by using
the mmhealth command.
Inspect the output of the
ess3kplt command for
details.
dimm_inspection_passed STATE_CHANGE INFO The memory dimms
of the canister is OK.
The capacity, speed,
and number of
populated dimm
slots is as expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned an
InspectionPasse
d equal to True
value.
N/A
dimm_size_ok STATE_CHANGE INFO All installed memory
dimms have the
expected capacity.
The capacity of all
populated memory
dimm slots is as
expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned no capacity
errors.
N/A
dimm_size_wrong STATE_CHANGE ERROR One or more
memory dimm
modules have an
unsupported
capacity.
The capacity of one
or more memory
dimm slots is not as
expected. This
conguration is not
supported.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned some
capacity errors.
Inspect the output of the
ess3kplt command to
see which memory dimm
slots have an
unsupported capacity and
replace those dimm
modules.
dimm_speed_ok STATE_CHANGE INFO All installed memory
dimms have the
expected speed.
The speed of all
populated memory
dimm slots is as
expected.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned no speed
errors.
N/A
dimm_speed_wrong STATE_CHANGE ERROR One or more
memory dimm
modules have an
unsupported speed.
The speed of one or
more memory dimm
slots is not as
expected. This
conguration is not
supported.
The /opt/ibm/gss
/tools/bin/
ess3kplt command
returned some speed
errors.
Inspect the output of the
ess3kplt command to
see which memory dimm
slots have an
unsupported speed and
replace those dimm
modules.
pair_canister_missing STATE_CHANGE WARNING Pair canister {0} is
missing or dead.
Could not get the
state of the pair
canister. It might be
missing or dead.
The
mmlsenclosure
command reports
only one canister
instead of two.
Check for detailed error
events of the referenced
canister node by using the
mmhealth command.
Inspect the output of the
mmlsenclosure all -
L command for the
referenced canister.
pair_canister_visible STATE_CHANGE INFO Pair canister {0} is
visible.
Successfully get the
state of the pair
canister.
The
mmlsenclosure
command reports
both canisters.
N/A
Enclosure events
The following table lists the events that are created for the Enclosure component.
Table 4. Events for the Enclosure component
Event Event Type Severity Message Description Cause User Action
adapter_bios_notavail STATE_CHANGE WARNIN
G
The bios level of
adapter {0} is not
available.
The bios level of
the adapter is not
available.
N/A Check the installed
BIOS level using the
mmlsfirmware
command.
adapter_bios_ok STATE_CHANGE INFO The BIOS level of
adapter {0} is
correct.
The BIOS level of
the adapter is
correct.
N/A N/A
4 IBM Elastic Storage System 3000: Service Guide

Table 4. Events for the Enclosure component (continued)
Event Event Type Severity Message Description Cause User Action
adapter_bios_wrong STATE_CHANGE WARNIN
G
The bios level of
adapter {0} is
wrong.
The bios level of
the adapter is
wrong.
N/A Check the installed
BIOS level using the
mmlsfirmware
command.
adapter_rmware_notavail STATE_CHANGE WARNIN
G
The rmware level
of adapter {0} is
not available.
The rmware level
of the adapter is
not available.
N/A Check the installed
BIOS level using the
mmlsfirmware
command.
adapter_rmware_ok STATE_CHANGE INFO The rmware level
of adapter {0} is
correct.
The rmware level
of the adapter is
correct.
N/A N/A
adapter_rmware_wrong STATE_CHANGE WARNIN
G
The rmware level
of adapter {0} is
wrong.
The rmware level
of the adapter is
wrong.
N/A Check the installed
BIOS level using the
mmlsfirmware
command.
current_failed STATE_CHANGE ERROR currentSensor {0}
failed.
The currentSensor
state is failed.
N/A N/A
current_ok STATE_CHANGE INFO currentSensor {0}
is ok.
The currentSensor
state is ok.
N/A N/A
current_warn STATE_CHANGE WARNIN
G
currentSensor {0}
is degraded.
The currentSensor
state is degraded.
N/A N/A
dcm_drawer_open STATE_CHANGE WARNIN
G
DCM {0} drawer is
open.
The DCM drawer is
open.
N/A N/A
dcm_failed STATE_CHANGE WARNIN
G
DCM {0} is failed. The DCM state is
failed.
N/A N/A
dcm_not_available STATE_CHANGE WARNIN
G
DCM {0} is not
available.
The DCM is not
installed or not
responding.
N/A N/A
dcm_ok STATE_CHANGE INFO DCM {id[1]} is ok. The DCM state is
ok.
N/A N/A
drawer_failed STATE_CHANGE ERROR drawer {0} is failed. The drawer state is
failed.
N/A N/A
drawer_ok STATE_CHANGE INFO drawer {0} is ok. The drawer state is
ok.
N/A N/A
drive_rmware_notavail STATE_CHANGE WARNIN
G
The rmware level
of drive {0} is not
available.
The rmware level
of the drive is not
available.
N/A Check the installed
rmware level using
the mmlsfirmware
command.
drive_rmware_ok STATE_CHANGE INFO The rmware level
of drive {0} is
correct.
The rmware level
of the drive is
correct.
N/A N/A
drive_rmware_wrong STATE_CHANGE WARNIN
G
The rmware level
of drive {0} is
wrong.
The rmware level
of the drive is
wrong.
N/A Check the installed
rmware level using
the mmlsfirmware
command.
enclosure_data STATE_CHANGE INFO Enclosure data
found.
Successfully
queried the
enclosure details.
The
mmlsenclosure
all -L -Y
command reports
enclosure data.
N/A
enclosure_rmware_notavail STATE_CHANGE WARNIN
G
The rmware level
of enclosure {0} is
not available.
The rmware level
of the enclosure is
not available.
N/A Check the installed
rmware level using
the mmlsfirmware
command.
enclosure_rmware_ok STATE_CHANGE INFO The rmware level
of enclosure {0} is
correct.
The rmware level
of the enclosure is
correct.
N/A N/A
Chapter 1. Events 5

Table 4. Events for the Enclosure component (continued)
Event Event Type Severity Message Description Cause User Action
enclosure_rmware_unknown STATE_CHANGE WARNIN
G
The rmware level
of enclosure {0} is
unknown.
The SAS card is
unable to read
enclosure
rmware.
The SAS card does
not report the
enclosure
rmware.
Check the SAS
connectivity from
node to enclosure.
Use the
mmlsrecoverygrou
p rg_name -L --
pdisk command to
verify if all the paths
to pdisk are
available. Check the
SAS connectivity
using a combination
of the
mmgetpdisktopolo
gy and the
topsummary
command. If there is
an issue with the SAS
HBA or SAS Cable,
reboot the node to
see if this resolves
the issue. If not
contact your IBM
representative.
enclosure_rmware_wrong STATE_CHANGE WARNIN
G
The rmware level
of enclosure {0} is
wrong.
The rmware level
of the enclosure is
wrong.
N/A Check the installed
rmware level using
mmlsfirmware
command.
enclosure_found INFO_ADD_ENTITY INFO Enclosure {0} was
found.
A GNR enclosure
listed in the IBM
Spectrum Scale
conguration was
detected.
N/A N/A
enclosure_needsservice STATE_CHANGE WARNIN
G
Enclosure {0}
needs service.
The enclosure
needs service.
N/A N/A
enclosure_ok STATE_CHANGE INFO Enclosure {0} is ok. The enclosure state
is ok.
N/A N/A
enclosure_unknown STATE_CHANGE WARNIN
G
Enclosure state {0}
is unknown.
The enclosure state
is unknown.
N/A N/A
enclosure_vanished INFO_DELETE_ENTIT
Y
INFO Enclosure {0} has
vanished.
A GNR enclosure
listed in the IBM
Spectrum Scale
conguration was
not detected.
A GNR enclosure,
listed in the IBM
Spectrum Scale
conguration as
mounted before, is
not found. This
could be a valid
situation.
Run the
mmlsenclosure
command to verify
that all expected
enclosures exist.
esm_absent STATE_CHANGE WARNIN
G
ESM {0} is absent. The ESM state is
not installed .
N/A N/A
esm_failed STATE_CHANGE WARNIN
G
ESM {0} is failed. The ESM state is
failed.
N/A N/A
esm_ok STATE_CHANGE INFO ESM {0} is ok. The ESM state is
ok.
N/A N/A
expander_absent STATE_CHANGE WARNIN
G
expander {0} is
absent.
The expander is
absent.
N/A N/A
expander_failed STATE_CHANGE ERROR expander {0} is
failed.
The expander state
is failed.
N/A N/A
expander_ok STATE_CHANGE INFO expander {0} is ok. The expander state
is ok.
N/A N/A
fan_failed STATE_CHANGE WARNIN
G
Fan {0} is failed. The fan state is
failed.
N/A N/A
fan_ok STATE_CHANGE INFO Fan {0} is ok. The fan state is ok. N/A N/A
fan_speed_high STATE_CHANGE WARNIN
G
Fan {0} speed is
too high
The fan speed is
out of the tolerance
range
N/A Check the enclosure
cooling module LEDs
for fan faults.
6 IBM Elastic Storage System 3000: Service Guide

Table 4. Events for the Enclosure component (continued)
Event Event Type Severity Message Description Cause User Action
fan_speed_low STATE_CHANGE WARNIN
G
Fan {0} speed is
too low
The fan speed is
out of the tolerance
range
N/A Check the enclosure
cooling module LEDs
for fan faults.
no_enclosure_data STATE_CHANGE WARNIN
G
Enclosure data and
state information
cannot be queried.
Cannot query the
enclosure details.
State reporting for
all enclosures and
canisters will be
incorrect.
The
mmlsenclosure
all -L -Y
command fails to
report any
enclosure data.
Run the
mmlsenclosure
command to check
for errors. Use the
lsmod command to
verify that the
pemsmod is loaded.
power_high_current STATE_CHANGE WARNIN
G
Power supply {0}
reports high
current.
The DC power
supply current is
greater than the
threshold.
N/A N/A
power_high_voltage STATE_CHANGE WARNIN
G
Power supply {0}
reports high
voltage.
The DC power
supply voltage is
greater than the
threshold.
N/A N/A
power_no_power STATE_CHANGE WARNIN
G
Power supply {0}
has no power.
Power supply has
no input AC power.
The power supply
may be turned off
or disconnected
from the AC supply.
N/A N/A
power_supply_absent STATE_CHANGE WARNIN
G
Power supply {0} is
missing.
The power supply
is missing
N/A N/A
power_supply_failed STATE_CHANGE WARNIN
G
Power supply {0} is
failed.
The power supply
state is failed.
N/A N/A
power_supply_off STATE_CHANGE WARNIN
G
Power supply {0} is
off.
The power supply
is not providing
power.
N/A N/A
power_supply_ok STATE_CHANGE INFO Power supply {0} is
ok.
The power supply
state is ok.
N/A N/A
power_switched_off STATE_CHANGE WARNIN
G
Power supply {0} is
switched off.
The requested on
bit is off, indicating
that the power
supply has not
been manually
turned on or been
requested to turn
on by setting the
requested on bit.
N/A N/A
sideplane_failed STATE_CHANGE ERROR sideplane {0}
failed.
The sideplane state
is failed.
N/A N/A
sideplane_ok STATE_CHANGE INFO sideplane {0} is ok. The sideplane state
is ok.
N/A N/A
temp_bus_failed STATE_CHANGE WARNIN
G
Temperature
sensor {0} I2C bus
is failed.
The temperature
sensor I2C bus has
failed.
N/A N/A
temp_high_critical STATE_CHANGE WARNIN
G
Temperature
sensor {0}
measured a high
temperature value.
The temperature
has exceeded the
actual high critical
threshold value for
at least one sensor.
N/A N/A
temp_high_warn STATE_CHANGE WARNIN
G
Temperature
sensor {0}
measured a high
temperature value.
The temperature
has exceeded the
actual high warning
threshold value for
at least one sensor.
N/A N/A
temp_low_critical STATE_CHANGE WARNIN
G
Temperature
sensor {0}
measured a low
temperature value.
The temperature
has fallen below
the actual low
critical threshold
value for at least
one sensor.
N/A N/A
Chapter 1. Events 7

Table 4. Events for the Enclosure component (continued)
Event Event Type Severity Message Description Cause User Action
temp_low_warn STATE_CHANGE WARNIN
G
Temperature
sensor {0}
measured a low
temperature value.
The temperature
has fallen below
the actual low
warning threshold
value for at least
one sensor.
N/A N/A
temp_sensor_failed STATE_CHANGE WARNIN
G
Temperature
sensor {0} is failed.
The temperature
sensor state is
failed.
N/A N/A
temp_sensor_ok STATE_CHANGE INFO Temperature
sensor {0} is ok.
The temperature
sensor state is ok.
N/A N/A
voltage_bus_failed STATE_CHANGE WARNIN
G
Voltage sensor {0}
I2C bus is failed.
The voltage sensor
I2C bus has failed.
N/A N/A
voltage_high_critical STATE_CHANGE WARNIN
G
Voltage sensor {0}
measured a high
voltage value.
The voltage has
exceeded the
actual high critical
threshold value for
at least one sensor.
N/A N/A
voltage_high_warn STATE_CHANGE WARNIN
G
Voltage sensor {0}
measured a high
voltage value.
The voltage has
exceeded the
actual high warning
threshold value for
at least one sensor.
N/A N/A
voltage_low_critical STATE_CHANGE WARNIN
G
Voltage sensor {0}
measured a low
voltage value.
The voltage has
fallen below the
actual low critical
threshold value for
at least one sensor.
N/A N/A
voltage_low_warn STATE_CHANGE WARNIN
G
Voltage sensor {0}
measured a low
voltage value.
The voltage has
fallen below the
actual low warning
threshold value for
at least one sensor.
N/A N/A
voltage_sensor_failed STATE_CHANGE WARNIN
G
Voltage sensor {0}
is failed.
The voltage sensor
state is failed.
N/A N/A
voltage_sensor_ok STATE_CHANGE INFO Voltage sensor {0}
is ok.
The voltage sensor
state is ok.
N/A N/A
Physical disk events
The following table lists the events that are created for the Physical disk component.
Table 5. Events for the physical disk component
Event Event Type Severity Message Description Cause User Action
gnr_nvram_degraded STATE_CHANGE WARNING The NVDIMM of the
pdisk {0} is degraded.
The NVRAM drive of
the disk is in
degraded state.
The tslsnvramstatus
command shows
degraded state for the
NVRAM drive of the disk.
N/A
gnr_nvram_disarmed STATE_CHANGE ERROR The NVDIMM of the
pdisk {0} is disarmed.
NVDIMM is unable to
preserve future
content.
The tslsnvramstatus
command reports
disarmed failure
condition for the NVRAM
drive of the disk.
Identify the
NVDIMM cards or
BPM, which
encountered the
errors from FSP log
or call home data,
and replace the
faulty NVDIMM
cards, BPM or both
as soon as possible.
8 IBM Elastic Storage System 3000: Service Guide