NetApp FAS9500 User manual

Maintain
ONTAP Systems
NetApp
July 19, 2022
This PDF was generated from https://docs.netapp.com/us-en/ontap-
systems/fas9500/add_io_module.html on July 19, 2022. Always check docs.netapp.com for the latest.

Table of Contents
Maintain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê1
Add an I/O module - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê1
Boot media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê4
Hot-swap a caching module - FAS9500. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê20
Chassis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê22
Controller module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê35
Replace a DIMM - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê52
Replace the Destage Control Power Module containing the NVRAM11 battery - FAS9500 . . . . . . . . . . . . . Ê60
Swap out a fan - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê62
Replace an I/O module - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê63
Replace an LED USB module - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê65
Replace the NVRAM module and/or NVRAM DIMMs - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê66
Swap out a power supply - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê74
Replace the real-time clock battery - FAS9500 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê76

Maintain
Add an I/O module - FAS9500
You can add an I/O module to your system by either replacing a NIC or storage adapter
with a new one in a fully-populated system, or by adding a new NIC or storage adapter
into an empty chassis slot in your system.
Before you begin
•Check the NetApp Hardware Universe to make sure that the new I/O module is compatible with your
system and version of ONTAP you’re running.
•If multiple slots are available, check the slot priorities in NetApp Hardware Universe and use the best one
available for your I/O module.
•To non-disruptively add an I/O module, you must take over the target controller, remove the slot blanking
cover in the target slot or remove an existing I/O module, add the new or replacement I/O module, and then
giveback the target controller.
•Make sure that all other components are functioning properly.
Option 1: Add the I/O module to a system with open slots
You can add an I/O module into an empty module slot in your system as either a NIC or a storage module for
the SAS storage shelves.
1. Shutdown controller A:
a. Disable automatic giveback: storage failover modify -node local -auto-giveback
false
b. Take over the target node: storage failover takeover -ofnode target_node_name
The console connection shows that the node drops to the LOADER prompt when the take over is
complete.
2. If you are not already grounded, properly ground yourself.
3. Remove the target slot blanking cover:
a. Depress the lettered and numbered cam button.
b. Rotate the cam latch down until it is the open position.
c. Remove the blanking cover.
4. Install the I/O module:
a. Align the I/O module with the edges of the slot.
b. Slide the I/O module into the slot until the lettered and numbered I/O cam latch begins to engage with
the I/O cam pin.
c. Push the I/O cam latch all the way up to lock the module in place.
5. If the replacement I/O module is a NIC, cable the module to the data switches.
Make sure that any unused I/O slots have blanks installed to prevent possible thermal
issues.
1

6. Reboot the node from the LOADER prompt: bye
7. Give back the node from the partner node. storage failover giveback -ofnode
target_node_name
8. Enable automatic giveback if it was disabled: storage failover modify -node local -auto
-giveback true
9. If you are using slots 3 and/or 7 for networking, use the storage port modify -node <node name>
-port <port name> -mode network command to convert the slot for networking use.
10. Repeat these steps for controller B.
11. If you installed a storage I/O module, install and cable your SAS shelves, as described in Hot-adding a
SAS shelf.
Option 2: Add an I/O module in a system with no open slots
You must remove one or more existing NIC or storage modules in your system in order to install one or more
I/O modules into your fully-populated system.
1. If you are:
Replacing a… Then…
NIC I/O module with the same the
same number of ports
The LIFs will automatically migrate when its controller module is shut
down.
NIC I/O module with fewer ports Permanently reassign the affected LIFs to a different home port. See
Migrating a LIF for information about using System Manager to
permanently move the LIFs.
NIC I/O module with a storage I/O
module
Use System Manager to permanently migrate the LIFs to different
home ports, as described in Migrating a LIF.
2. Shut down controller A:
a. Disable automatic giveback: storage failover modify -node local -auto-giveback
false
b. Take over the target node: storage failover takeover -ofnode target_node_name
The console connection shows that the node drops to the LOADER prompt when the take over is
complete.
3. If you are not already grounded, properly ground yourself.
4. Unplug any cabling on the target I/O module.
5. Remove the target I/O module from the chassis:
a. Depress the lettered and numbered cam button.
The cam button moves away from the chassis.
b. Rotate the cam latch down until it is in a horizontal position.
2

The I/O module disengages from the chassis and moves about 1/2 inch out of the I/O slot.
c. Remove the I/O module from the chassis by pulling on the pull tabs on the sides of the module face.
Make sure that you keep track of which slot the I/O module was in.
Animation - Replacing an I/O module
Lettered and numbered I/O cam latch
I/O cam latch completely unlocked
6. Install the I/O module into the target slot:
a. Align the I/O module with the edges of the slot.
b. Slide the I/O module into the slot until the lettered and numbered I/O cam latch begins to engage with
the I/O cam pin.
c. Push the I/O cam latch all the way up to lock the module in place.
7. Repeat the remove and install steps to replace additional modules for controller A.
8. If the replacement I/O module is a NIC, cable the module or modules to the data switches.
9. Reboot the node from the LOADER prompt: bye
10. Give back the node from the partner node. storage failover giveback -ofnode
target_node_name
11. Enable automatic giveback if it was disabled: storage failover modify -node local -auto
-giveback true
12. If you added:
If I/O module is a… Then…
NIC module in slots 3 or 7, Use the storage port modify -node *<node name> -port
*<port name> -mode network command for each port.
3

If I/O module is a… Then…
Storage module Install and cable your SAS shelves, as described in Hot-adding a SAS
shelf.
13. Repeat these steps for controller B.
Boot media
Replace the boot media - FAS9500
The boot media stores a primary and secondary set of system (boot image) files that the system uses when it
boots. Depending on your network configuration, you can perform either a nondisruptive or disruptive
replacement.
You must have a USB flash drive, formatted to FAT32, with the appropriate amount of storage to hold the
image_xxx.tgz.
You also must copy the image_xxx.tgz file to the USB flash drive for later use in this procedure.
•The nondisruptive and disruptive methods for replacing a boot media both require you to restore the var
file system:
◦For nondisruptive replacement, the HA pair does not require connection to a network to restore the var
file system. The HA pair in a single chassis has an internal e0S connection, which is used to transfer
var config between them.
◦For disruptive replacement, you do not need a network connection to restore the var file system, but
the process requires two reboots.
•You must replace the failed component with a replacement FRU component you received from your
provider.
•It is important that you apply the commands in these steps on the correct node:
◦The impaired node is the node on which you are performing maintenance.
◦The healthy node is the HA partner of the impaired node.
Pre-shutdown checks for onboard encryption keys - FAS9500
Prior to shutting down the impaired controller and checking the status of the onboard encryption keys, you
must check the status of the impaired controller, disable automatic giveback, and check the version of ONTAP
that is running.
If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a healthy
controller shows false for eligibility and health, you must correct the issue before shutting down the impaired
controller; see the NetApp Encryption overview with the CLI.
Steps
1. Check the status of the impaired controller:
◦If the impaired controller is at the login prompt, log in as admin.
◦If the impaired controller is at the LOADER prompt and is part of HA configuration, log in as admin on
the healthy controller.
4

◦If the impaired controller is in a standalone configuration and at LOADER prompt, contact
mysupport.netapp.com.
2. If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message:
system node autosupport invoke -node * -type all -message
MAINT=number_of_hours_downh
The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*>
system node autosupport invoke -node * -type all -message MAINT=2h
3. Check the version of ONTAP the system is running on the impaired controller if up, or on the partner
controller if the impaired controller is down, using the version -v command:
◦If <lno-DARE> or <1Ono-DARE> is displayed in the command output, the system does not support
NVE, proceed to shut down the controller.
ONTAP 9.6 and later
Before shutting down the impaired controller, you need to verify whether the system has either NetApp Volume
Encryption (NVE) or NetApp Storage Encryption (NSE) enabled. If so, you need to verify the configuration.
1. Verify whether NVE is in use for any volumes in the cluster: volume show -is-encrypted true
If any volumes are listed in the output, NVE is configured and you need to verify the NVE configuration. If
no volumes are listed, check whether NSE is configured and in use.
2. Verify whether NSE is configured and in use: storage encryption disk show
◦If the command output lists the drive details with Mode & Key ID information, NSE is configured and
you need to verify the NSE configuration and in use.
◦If no disks are shown, NSE is not configured.
◦If NVE and NSE are not configured, no drives are protected with NSE keys, it’s safe to shut down the
impaired controller.
Verify NVE configuration
1. Display the key IDs of the authentication keys that are stored on the key management servers: security
key-manager key-query
After the ONTAP 9.6 release, you may have additional key manager types. The types are KMIP,
AKV, and GCP. The process for confirming these types is the same as confirming external or
onboard key manager types.
•If the Key Manager type displays external and the Restored column displays yes, it’s safe to shut
down the impaired controller.
•If the Key Manager type displays onboard and the Restored column displays yes, you need to
complete some additional steps.
•If the Key Manager type displays external and the Restored column displays anything other than
yes, you need to complete some additional steps.
•If the Key Manager type displays onboard and the Restored column displays anything other than yes,
you need to complete some additional steps.
5

1. If the Key Manager type displays onboard and the Restored column displays yes, manually back
up the OKM information:
a. Go to advanced privilege mode and enter ywhen prompted to continue: set -priv advanced
b. Enter the command to display the key management information: security key-manager
onboard show-backup
c. Copy the contents of the backup information to a separate file or your log file. You’ll need it in
disaster scenarios where you might need to manually recover OKM.
d. Return to admin mode: set -priv admin
e. Shut down the impaired controller.
2. If the Key Manager type displays external and the Restored column displays anything other than
yes:
a. Restore the external key management authentication keys to all nodes in the cluster: security
key-manager external restore
If the command fails, contact NetApp Support.
mysupport.netapp.com
b. Verify that the Restored column equals yes for all authentication keys: security key-
manager key-query
c. Shut down the impaired controller.
3. If the Key Manager type displays onboard and the Restored column displays anything other than
yes:
a. Enter the onboard security key-manager sync command: security key-manager onboard
sync
Enter the customer’s onboard key management passphrase at the prompt. If the
passphrase cannot be provided, contact NetApp Support. mysupport.netapp.com
b. Verify the Restored column shows yes for all authentication keys: security key-manager
key-query
c. Verify that the Key Manager type shows onboard, and then manually back up the OKM
information.
d. Go to advanced privilege mode and enter ywhen prompted to continue: set -priv advanced
e. Enter the command to display the key management backup information: security key-
manager onboard show-backup
f. Copy the contents of the backup information to a separate file or your log file. You’ll need it in
disaster scenarios where you might need to manually recover OKM.
g. Return to admin mode: set -priv admin
h. You can safely shut down the controller.
Verify NSE configuration
1. Display the key IDs of the authentication keys that are stored on the key management servers: security
6

key-manager key-query -key-type NSE-AK
After the ONTAP 9.6 release, you may have additional key manager types. The types are KMIP,
AKV, and GCP. The process for confirming these types is the same as confirming external or
onboard key manager types.
•If the Key Manager type displays external and the Restored column displays yes, it’s safe to shut
down the impaired controller.
•If the Key Manager type displays onboard and the Restored column displays yes, you need to
complete some additional steps.
•If the Key Manager type displays external and the Restored column displays anything other than
yes, you need to complete some additional steps.
•If the Key Manager type displays external and the Restored column displays anything other than
yes, you need to complete some additional steps.
1. If the Key Manager type displays onboard and the Restored column displays yes, manually back
up the OKM information:
a. Go to advanced privilege mode and enter ywhen prompted to continue: set -priv advanced
b. Enter the command to display the key management information: security key-manager
onboard show-backup
c. Copy the contents of the backup information to a separate file or your log file. You’ll need it in
disaster scenarios where you might need to manually recover OKM.
d. Return to admin mode: set -priv admin
e. You can safely shut down the controller.
2. If the Key Manager type displays external and the Restored column displays anything other than
yes:
a. Enter the onboard security key-manager sync command: security key-manager external
sync
If the command fails, contact NetApp Support.
mysupport.netapp.com
b. Verify that the Restored column equals yes for all authentication keys: security key-
manager key-query
c. You can safely shut down the controller.
3. If the Key Manager type displays onboard and the Restored column displays anything other than
yes:
a. Enter the onboard security key-manager sync command: security key-manager onboard
sync
Enter the customer’s onboard key management passphrase at the prompt. If the passphrase
cannot be provided, contact NetApp Support.
mysupport.netapp.com
7

b. Verify the Restored column shows yes for all authentication keys: security key-manager
key-query
c. Verify that the Key Manager type shows onboard, and then manually back up the OKM
information.
d. Go to advanced privilege mode and enter ywhen prompted to continue: set -priv advanced
e. Enter the command to display the key management backup information: security key-
manager onboard show-backup
f. Copy the contents of the backup information to a separate file or your log file. You’ll need it in
disaster scenarios where you might need to manually recover OKM.
g. Return to admin mode: set -priv admin
h. You can safely shut down the controller.
Shut down the impaired controller - FAS9500
Shut down or take over the impaired controller using one of the following options.
After completing the NVE or NSE tasks, you need to complete the shutdown of the impaired node.
To shut down the impaired controller, you must determine the status of the controller and, if necessary, take
over the controller so that the healthy controller continues to serve data from the impaired controller storage.
About this task
•If you are using NetApp Storage Encryption, you must have reset the MSID using the instructions in the
“Returning SEDs to unprotected mode” section of the ONTAP 9 NetApp Encryption Power Guide.
ONTAP 9 NetApp Encryption Power Guide
•If you have a SAN system, you must have checked event messages (event log show) for impaired
controller SCSI blade.
Each SCSI-blade process should be in quorum with the other nodes in the cluster. Any issues must be
resolved before you proceed with the replacement.
•If you have a cluster with more than two nodes, it must be in quorum. If the cluster is not in quorum or a
healthy controller shows false for eligibility and health, you must correct the issue before shutting down the
impaired controller; see the Administration overview with the CLI.
•If you have a MetroCluster configuration, you must have confirmed that the MetroCluster Configuration
State is configured and that the nodes are in an enabled and normal state (metrocluster node show).
Steps
1. If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message:
system node autosupport invoke -node * -type all -message
MAINT=number_of_hours_downh
The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*>
system node autosupport invoke -node * -type all -message MAINT=2h
2. Disable automatic giveback from the console of the healthy controller: storage failover modify
–node local -auto-giveback false
8

3. Take the impaired controller to the LOADER prompt:
If the impaired controller is
displaying…
Then…
The LOADER prompt Go to Remove controller module.
Waiting for giveback… Press Ctrl-C, and then respond ywhen prompted.
System prompt or password
prompt (enter system password)
Take over or halt the impaired controller from the healthy controller:
storage failover takeover -ofnode
impaired_node_name
When the impaired controller shows Waiting for giveback…, press
Ctrl-C, and then respond y.
Remove the controller, replace the boot media, and transfer the boot image -
FAS9500
You must remove and open the controller module, locate and replace the boot media in
the controller, and then transfer the image to the replacement boot media.
Step 1: Remove the controller module
To access components inside the controller, you must first remove the controller module from the system and
then remove the cover on the controller module.
Steps
1. If you are not already grounded, properly ground yourself.
2. Unplug the cables from the impaired controller module, and keep track of where the cables were
connected.
3. Slide the terra cotta button on the cam handle downward until it unlocks.
Animation - Remove controller module
9

A1
B1
A2
A3
A4
A5
A6
A7
A8
A9
A10
A11
AB
2
1
Cam handle release button
Cam handle
4. Rotate the cam handle so that it completely disengages the controller module from the chassis, and then
slide the controller module out of the chassis.
Make sure that you support the bottom of the controller module as you slide it out of the chassis.
5. Place the controller module lid-side up on a stable, flat surface, press the blue button on the cover, slide
the cover to the back of the controller module, and then swing the cover up and lift it off of the controller
module.
1
10

Controller module cover locking button
Step 2: Replace the boot media
You must locate the boot media in the controller and follow the directions to replace it.
Steps
1. Lift the black air duct at the back of the controller module and then locate the boot media using the
following illustration or the FRU map on the controller module:
Animation - Replace boot media
1
2
Press release tab
Boot media
2. Press the blue button on the boot media housing to release the boot media from its housing, and then
gently pull it straight out of the boot media socket.
Do not twist or pull the boot media straight up, because this could damage the socket or the
boot media.
3. Align the edges of the replacement boot media with the boot media socket, and then gently push it into the
socket.
4. Check the boot media to make sure that it is seated squarely and completely in the socket.
If necessary, remove the boot media and reseat it into the socket.
5. Push the boot media down to engage the locking button on the boot media housing.
11

6. Reinstall the controller module lid by aligning the pins on the lid with the slots on the motherboard carrier,
and then slide the lid into place.
Step 3: Transfer the boot image to the boot media
You can install the system image to the replacement boot media using a USB flash drive with the image
installed on it. However, you must restore the var file system during this procedure.
Before you begin
•You must have a USB flash drive, formatted to FAT32, with at least 4GB capacity.
•A copy of the same image version of ONTAP as what the impaired controller was running. You can
download the appropriate image from the Downloads section on the NetApp Support Site
◦If NVE is enabled, download the image with NetApp Volume Encryption, as indicated in the download
button.
◦If NVE is not enabled, download the image without NetApp Volume Encryption, as indicated in the
download button.
•If your system is a stand-alone system you do not need a network connection, but you must perform an
additional reboot when restoring the var file system.
Steps
1. Align the end of the controller module with the opening in the chassis, and then gently push the controller
module halfway into the system.
2. Recable the controller module, as needed.
3. Insert the USB flash drive into the USB slot on the controller module.
Make sure that you install the USB flash drive in the slot labeled for USB devices, and not in the USB
console port.
4. Push the controller module all the way into the system, making sure that the cam handle clears the USB
flash drive, firmly push the cam handle to finish seating the controller module, and then push the cam
handle to the closed position.
The node begins to boot as soon as it is completely installed into the chassis.
5. Interrupt the boot process to stop at the LOADER prompt by pressing Ctrl-C when you see Starting
AUTOBOOT press Ctrl-C to abort….
If you miss this message, press Ctrl-C, select the option to boot to Maintenance mode, and then halt the
node to boot to LOADER.
6. Although the environment variables and bootargs are retained, you should check that all required boot
environment variables and bootargs are properly set for your system type and configuration using the
printenv bootarg name command and correct any errors using the setenv variable-name
<value> command.
a. Check the boot environment variables:
▪bootarg.init.boot_clustered
▪partner-sysid
▪bootarg.init.flash_optimized for AFF
▪bootarg.init.san_optimized for AFF
12

▪bootarg.init.switchless_cluster.enable
b. If External Key Manager is enabled, check the bootarg values, listed in the kenv ASUP output:
▪bootarg.storageencryption.support <value>
▪bootarg.keymanager.support <value>
▪kmip.init.interface <value>
▪kmip.init.ipaddr <value>
▪kmip.init.netmask <value>
▪kmip.init.gateway <value>
c. If Onboard Key Manager is enabled, check the bootarg values, listed in the kenv ASUP output:
▪bootarg.storageencryption.support <value>
▪bootarg.keymanager.support <value>
▪bootarg.onboard_keymanager <value>
d. Save the environment variables you changed with the savenv command
e. Confirm your changes using the printenv variable-name command.
7. Set your network connection type at the LOADER prompt:
◦If you are configuring DHCP: ifconfig e0a -auto
The target port you configure is the target port you use to communicate with the
impaired node from the healthy node during var file system restore with a network
connection. You can also use the e0M port in this command.
◦If you are configuring manual connections: ifconfig e0a -addr=filer_addr -mask=netmask
-gw=gateway-dns=dns_addr-domain=dns_domain
▪filer_addr is the IP address of the storage system.
▪netmask is the network mask of the management network that is connected to the HA partner.
▪gateway is the gateway for the network.
▪dns_addr is the IP address of a name server on your network.
▪dns_domain is the Domain Name System (DNS) domain name.
If you use this optional parameter, you do not need a fully qualified domain name in the netboot
server URL. You need only the server’s host name.
Other parameters might be necessary for your interface. You can enter help ifconfig at the
firmware prompt for details.
8. If the controller is in a stretch or fabric-attached MetroCluster, you must restore the FC adapter
configuration:
a. Boot to Maintenance mode: boot_ontap maint
b. Set the MetroCluster ports as initiators: ucadmin modify -m fc -t iniitator adapter_name
c. Halt to return to Maintenance mode: halt
13

The changes will be implemented when the system is booted.
Boot the recovery image - FAS9500
You must boot the ONTAP image from the USB drive, restore the file system, and verify
the environmental variables.
1. From the LOADER prompt, boot the recovery image from the USB flash drive: boot_recovery
The image is downloaded from the USB flash drive.
2. When prompted, either enter the name of the image or accept the default image displayed inside the
brackets on your screen.
3. Restore the var file system:
If your system has… Then…
A network connection a. Press ywhen prompted to restore the backup configuration.
b. Press ywhen prompted to overwrite
/etc/ssh/ssh_host_ecdsa_key.
c. Press ywhen prompted to confirm if the restore backup was
successful.
d. Press Ywhen prompted to the restored configuration copy.
e. Set the healthy node to advanced privilege level: set
-privilege advanced
f. Run the restore backup command: system node restore-
backup -node local -target-address
impaired_node_IP_address
g. Return the node to admin level: set -privilege admin
h. Press ywhen prompted to use the restored configuration.
i. Press ywhen prompted to reboot the node.
No network connection a. Press nwhen prompted to restore the backup configuration.
b. Reboot the system when prompted by the system.
c. Select the Update flash from backup config (sync flash) option
from the displayed menu.
If you are prompted to continue with the update, press y.
14

If your system has… Then…
No network connection and is in a
MetroCluster IP configuration
a. Press nwhen prompted to restore the backup configuration.
b. Reboot the system when prompted by the system.
c. Wait for the iSCSI storage connections to connect.
You can proceed after you see the following messages:
date-and-time [node-
name:iscsi.session.stateChanged:notice]:
iSCSI session state is changed to Connected
for the target iSCSI-target (type:
dr_auxiliary, address: ip-address).
date-and-time [node-
name:iscsi.session.stateChanged:notice]:
iSCSI session state is changed to Connected
for the target iSCSI-target (type:
dr_partner, address: ip-address).
date-and-time [node-
name:iscsi.session.stateChanged:notice]:
iSCSI session state is changed to Connected
for the target iSCSI-target (type:
dr_auxiliary, address: ip-address).
date-and-time [node-
name:iscsi.session.stateChanged:notice]:
iSCSI session state is changed to Connected
for the target iSCSI-target (type:
dr_partner, address: ip-address).
d. Select the Update flash from backup config (sync flash) option
from the displayed menu.
If you are prompted to continue with the update, press y.
4. Ensure that the environmental variables are set as expected:
a. Take the node to the LOADER prompt.
b. Check the environment variable settings with the printenv command.
c. If an environment variable is not set as expected, modify it with the
setenvenvironment_variable_name changed_value command.
d. Save your changes using the saveenv command.
5. The next depends on your system configuration:
◦If your system has onboard keymanager, NSE or NVE configured, go to Post boot media replacement
steps for OKM, NSE, and NVE
15

◦If your system does not have onboard keymanager, NSE or NVE configured, complete the steps in this
section.
6. From the LOADER prompt, enter the boot_ontap command.
If you see… Then…
The login prompt Go to the next step.
Waiting for giveback… a. Log into the partner node.
b. Confirm the target node is ready for giveback with the storage
failover show command.
7. Connect the console cable to the partner node.
8. Give back the node using the storage failover giveback -fromnode local command.
9. At the cluster prompt, check the logical interfaces with the net int -is-home false command.
If any interfaces are listed as "false", revert those interfaces back to their home port using the net int
revert command.
10. Move the console cable to the repaired node and run the version -v command to check the ONTAP
versions.
11. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.
Post boot media replacement steps for OKM, NSE, and NVE - FAS9500
Once environment variables are checked, you must complete steps specific to restore
Onboard Key Manager (OKM), NetApp Storage Encryption (NSE) and NetApp Volume
Encryption (NVE).
1. Determine which section you should use to restore your OKM, NSE, or NVE configurations: If NSE or NVE
are enabled along with Onboard Key Manager you must restore settings you captured at the beginning of
this procedure.
◦If NSE or NVE are enabled and Onboard Key Manager is enabled, go to Restore NVE or NSE when
Onboard Key Manager is enabled.
◦If NSE or NVE are enabled for ONTAP 9.6, go to Restore NSE/NVE on systems running ONTAP 9.6
and later.
Restore NVE or NSE when Onboard Key Manager is enabled
1. Connect the console cable to the target node.
2. Use the boot_ontap command at the LOADER prompt to boot the node.
3. Check the console output:
16

If the console displays… Then…
The LOADER prompt Boot the node to the boot menu: boot_ontap menu
Waiting for giveback…. a. Enter Ctrl-C at the prompt
b. At the message: Do you wish to halt this node rather than wait
[y/n]? , enter: y
c. At the LOADER prompt, enter the boot_ontap menu command.
4. At the Boot Menu, enter the hidden command, recover_onboard_keymanager, and reply yat the
prompt.
5. Enter the passphrase for the onboard key manager you obtained from the customer at the beginning of this
procedure.
6. When prompted to enter the backup data, paste the backup data you captured at the beginning of this
section, when asked. Paste the output of security key-manager backup show OR security key-
manager onboard show-backup command.
The data is output from either security key-manager backup show or security key-
manager onboard show-backup` command.
Example of backup data:
Enter the backup data:
--------------------------BEGIN BACKUP--------------------------
TmV0QXBwIEtleSBCbG9iAAEAAAAEAAAAcAEAAAAAAADuD+byAAAAACEAAAAAAAAA
QAAAAAAAAABvOlH0AAAAAMh7qDLRyH1DBz12piVdy9ATSFMT0C0TlYFss4PDjTaV
dzRYkLd1PhQLxAWJwOIyqSr8qY1SEBgm1IWgE5DLRqkiAAAAAAAAACgAAAAAAAAA
3WTh7gAAAAAAAAAAAAAAAAIAAAAAAAgAZJEIWvdeHr5RCAvHGclo+wAAAAAAAAAA
IgAAAAAAAAAoAAAAAAAAAEOTcR0AAAAAAAAAAAAAAAACAAAAAAAJAGr3tJA/
LRzUQRHwv+1aWvAAAAAAAAAAACQAAAAAAAAAgAAAAAAAAACdhTcvAAAAAJ1PXeBf
ml4NBsSyV1B4jc4A7cvWEFY6lLG6hc6tbKLAHZuvfQ4rIbYAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA . . . .
H4nPQM0nrDRYRa9SCv8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAA
---------------------------END BACKUP---------------------------
7. At the Boot Menu select the option for Normal Boot.
The system boots to Waiting for giveback… prompt.
8. Move the console cable to the partner node and log in as admin.
9. Confirm the target node is ready for giveback with the storage failover show command.
10. Give back only the CFO aggregates with the storage failover giveback -fromnode local
17

-only-cfo-aggregates true command.
◦If the command fails because of a failed disk, physically disengage the failed disk, but leave the disk in
the slot until a replacement is received.
◦If the command fails because of an open CIFS session, check with the customer how to close out CIFS
sessions.
Terminating CIFS can cause loss of data.
◦If the command fails because the partner "not ready", wait 5 minutes for the NVRAMs to synchronize.
◦If the command fails because of an NDMP, SnapMirror, or SnapVault process, disable the process. See
the appropriate content for more information.
11. Once the giveback completes, check the failover and giveback status with the storage failover show
and storage failover show-giveback commands.
Only the CFO aggregates (root aggregate and CFO style data aggregates) will be shown.
12. If you are running ONTAP 9.6 or later, run the security key-manager onboard sync:
a. Run the security key-manager onboard sync command and then enter the passphrase when
prompted.
b. Enter the security key-manager key-query command to see a detailed view of all keys stored
in the onboard key manager and verify that the Restored column = yes/true for all authentication
keys.
If the Restored column = anything other than yes/true, contact Customer Support.
c. Wait 10 minutes for the key to synchronize across the cluster.
13. Move the console cable to the partner node.
14. Give back the target node using the storage failover giveback -fromnode local command.
15. Check the giveback status, three minutes after it reports complete, using the storage failover show
command.
If giveback is not complete after 20 minutes, contact Customer Support.
16. At the clustershell prompt, enter the net int show -is-home false command to list the logical
interfaces that are not on their home node and port.
If any interfaces are listed as false, revert those interfaces back to their home port using the net int
revert command.
17. Move the console cable to the target node and run the version -v command to check the ONTAP
versions.
18. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.
Restore NSE/NVE on systems running ONTAP 9.6 and later
1. Connect the console cable to the target node.
18