IBM 57D7 User manual

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
IBM Power Systems 57D7/57B4 Adapter Write Cache
1 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Contents
1 Feature Description 3
2 Software and Hardware Requirements 4
2.0.1 Supported Systems ...................................... 4
2.0.2 Supported Storage Adapter Backplanes .......................... 4
2.0.3 Supported Linux Operating System Releases & Configurations ............. 4
2.0.4 Minimum SAS RAID Adapter Firmware Level ...................... 5
2.0.5 Minimum Linux iprutils Package Level ........................... 5
3 Installation Instructions 6
3.1 Linux Software Installation Instructions .............................. 6
3.1.1 YUM Repository ....................................... 6
3.1.1.1 Red Hat ...................................... 6
3.1.1.2 SUSE ........................................ 6
3.1.2 Manual Installation ..................................... 7
3.2 Adapter Firmware Installation Instructions ............................ 8
4 Feature Enablement 12
4.1 Curses Interface ............................................ 12
4.2 Command Line Interface ....................................... 14
5 Support 15
2 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Chapter 1
Feature Description
This document describes how to enable and use the new volatile adapter write cache feature for the integrated
SAS controllers in the #EL3T and #EL3V SAS backplanes on IBM Power 8247-21L and 8247-22L systems,
the integrated SAS controllers in the #EJ0N and #EJ0S SAS backplanes on the IBM Power 8247-42L,
and the PCIe3 x8 SAS RAID adapter low profile Quad Port 6Gb (#EJ0J and #EJ0M). Once the required
adapter firmware and Linux software prerequisites are installed, this feature is enabled by default, but can
be manually disabled on a per RAID array basis. This feature enables a volatile write cache in the SAS
RAID adapter into which data is written. This data is then later flushed to the underlying disks. This
allows the RAID adapter to combine multiple host writes into fewer writes to the disks. It also enables the
RAID adapter to issue stripe writes to RAID 5 or 6 arrays, when a full parity stripe of data is available in
the cache, which can eliminate read-xor write overhead that may occur when a write cache is not available.
This can provide a noticeable performance advantage for some write workloads.
The adapter firmware which implements this volatile write caching feature includes support for a SYNCHRONIZE
CACHE mechanism. This allows journaled filesystems in the Linux operating system to flush any previously
written data prior to updating the filesystem journal. This ensures the filesystem data stays coherent with
respect to the journal in the case of an unexpected power loss of the system. For use cases which access
the RAID array block devices directly and bypass any filesystems in Linux, it may be desireable to dis-
able this feature, particularly if the software has not been written to issue SYNCHRONIZE CACHE commands.
Instructions on how to disable this feature are described later in this document.
3 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Chapter 2
Software and Hardware Requirements
The required hardware, firmware, and software levels for this feature are described below.
2.0.1 Supported Systems
•IBM Power 8247-21L
•IBM Power 8247-22L
•IBM Power 8247-42L
2.0.2 Supported Storage Adapter Backplanes
•#EL3T: PCIe3 x8 SAS RAID internal adapter 6Gb
•#EL3V: PCIe3 x8 SAS RAID internal adapter 6Gb (Split Backplane)
•#EJ0N: PCIe3 x8 SAS RAID internal adapter 6Gb (4U)
•#EJ0S: PCIe3 x8 SAS RAID internal adapter 6Gb (4U) (Split Backplane)
•#EJ0J: PCIe3 x8 SAS RAID adapter Quad Port 6Gb1
•#EJ0M: PCIe3 x8 SAS RAID adapter low profile Quad Port 6Gb1
2.0.3 Supported Linux Operating System Releases & Configurations
•Bare Metal Systems (indicated by #EC16):
–Ubuntu Server 16.04, or later
•PowerKVM 3.1 Systems (indicated by #EC40) or PowerKVM 2.1 (#EC20)
–Not supported
•Other environments, such as PowerVM (not PowerKVM, not Bare Metal)
–Red Hat Enterprise Linux 6.5, or later
–Red Hat Enterprise Linux 7.2, little endian, or later
–Red Hat Enterprise Linux 7.2, big endian, or later
1Single adapter configurations only, feature not supported for dual adapter, HA configurations
4 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
–SUSE Linux Enterprise Server 11, Service Pack 4, or later
–SUSE Linux Enterprise Server 12, Service Pack 1, or later
–Ubuntu Server 16.04, or later
2.0.4 Minimum SAS RAID Adapter Firmware Level
•14511900
2.0.5 Minimum Linux iprutils Package Level
•2.4.11
5 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Chapter 3
Installation Instructions
3.1 Linux Software Installation Instructions
3.1.1 YUM Repository
Installing via the IBM Power Tools YUM repository is the preferred method for obtaining the required
2.4.11 version of the iprutils package. Use this method if your system has connectivity to the external
network and is able to download files direct from the internet. If you are using Ubuntu 16.04, the requisite
version of iprutils should be able to be installed using apt-get, like any other package that is part of the
Ubuntu distribution. In this case, you can skip ahead to section 3.2 on updating the adapter firmware.
First, configure the IBM Power Tools YUM repository. Go to the following page for instructions on how to
download the configuration RPM file and how to configure the repository: http://www14.software.ibm.
com/webapp/set2/sas/f/lopdiags/yum.html.
Once the repository is configured, upgrade the iprutils package to the 2.4.11 version from the newly
configured repository. During this process, you will be prompted to accept the GPG key for the newly added
YUM repository. Please allow this new GPG key to be accepted, otherwise the package installation will be
aborted.
3.1.1.1 Red Hat
# yum update iprutils
3.1.1.2 SUSE
# zypper update iprutils
The currently installed version of iprutils can be verified with the following command.
# iprconfig --version
iprconfig: 2.4.11.1
6 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Ensure the iprutils daemons are enabled to start on the next boot. Please note, that the iprinit daemon
must be enabled and running for this write cache feature to get enabled at boot time.
# chkconfig iprinit on
# chkconfig iprupdate on
# chkconfig iprdump on
3.1.2 Manual Installation
Use this method if either the YUM repository method fails or your system is unable to download files directly
from the internet. Download the iprutils package directly from one of the following locations, depending
on your OS level, and transfer to the server.
RHEL 6.5, or later
http://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/RHEL/6/ppc64/iprutils-2.4.11-1.
41.1.ppc64.rpm
RHEL 7.2 Little Endian
http://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/RHEL/7/ppc64le/iprutils-2.4.
11-1.1.ppc64le.rpm
RHEL 7.2 Big Endian
http://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/RHEL/7/ppc64/iprutils-2.4.11-1.
41.1.ppc64.rpm
SLES 11 SP4
http://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/SLES/11/ppc64/iprutils-2.4.11-1.
41.1.ppc64.rpm
SLES 12 SP1
http://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/SLES/12/ppc64le/iprutils-2.4.
11-1.41.1.ppc64le.rpm
Next, remove the old iprutils package and then install the 2.4.11 version by running the following com-
mands on the server to which you transferred the iprutils rpm package.
# rpm -e iprutils
# rpm -ivh iprutils-2.4.11-1.41.1.ppc64.rpm
The currently installed version of iprutils can be verified with the following command.
# iprconfig --version
iprconfig: 2.4.11.1
7 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Ensure the iprutils daemons are enabled to start on the next boot. Please note, that the iprinit daemon
must be enabled and running for this write cache feature to get enabled at boot time.
# chkconfig iprinit on
# chkconfig iprupdate on
# chkconfig iprdump on
3.2 Adapter Firmware Installation Instructions
The required 14511900 version of SAS adapter firmware to enable this feature can be downloaded from IBM
Fix Central at the following URL: http://www.ibm.com/support/fixcentral/main/quickorder?parent=
Power%2BIO%2BFirmware&product=ibm/io/featurecode/EJ0T&release=all&platform=all&function=all&source=
fc. The pci.1014034A.51-14511900-1.Linux.noarch.rpm file should be downloaded. Once downloaded,
install to your Linux system with the rpm command.
# rpm -ivh pci.1014034A.51-14511900-1.Linux.noarch.rpm
To begin the process of installing the new adapter firmware, first login as root on the target server, then
run iprconfig.
You should see a screen similar to this:
IBM Power RAID Configuration Utility
Select one of the following:
1. Display hardware status
2. Work with disk arrays
3. Work with disk unit recovery
4. Work with configuration options
5. Work with microcode updates
6. Devices Statistics
7. Analyze log
Selection:
e=Exit
Select option 5. Work with microcode updates, and press Enter.
8 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Select option 2. Download latest microcode to all devices, and press Enter.
Work with microcode updates
Select one of the following:
1. Download microcode
2. Download latest microcode to all devices
Selection:
e=Exit q=Cancel
You should then see a screen similar to the following screen, showing one or more adapters to be updated
with the pci.1014034A.51.14511900 firmware image.
Confirm Microcode Download
The following devices are going to be updated.
ATTENTION: System performance may be
affected during the microcode download process. Download may take some time.
Press Enter to continue.
q=Cancel to return and change your choice.
Name Version Date Image File
------ --------- --------- -----------------------------
sg6 14511900 05/12/16 /lib/firmware/pci.1014034A.51.14511900
sg7 14511900 05/12/16 /lib/firmware/pci.1014034A.51.14511900
q=Cancel
Press Enter and follow the prompts. Once the operation is completed, return to the main menu by pressing
’e’ if necessary.
9 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
The current version of the adapter firmware can be confirmed by selecting option 1. Display hardware
status. This should present a screen similar to this:
Display Hardware Status
Type option, press Enter.
1=Display hardware resource information details
OPT Name PCI/SCSI Location Description Status
--- ------ ------------------------- ---------------------------- ------------
0003:70:00.0/0: PCI-E SAS RAID Adapter Operational
sda 0003:70:00.0/0:2:0:0 RAID 0 Array Optimized
0003:70:00.0/0:0:0:0 RAID 0 Array Member Active
sdb 0003:70:00.0/0:2:1:0 RAID 10 Array Optimized
0003:70:00.0/0:0:1:0 RAID 10 Array Member Active
0003:70:00.0/0:0:2:0 RAID 10 Array Member Active
sdc 0003:70:00.0/0:2:2:0 RAID 5 Array Optimized
0003:70:00.0/0:0:3:0 RAID 5 Array Member Active
0003:70:00.0/0:0:4:0 RAID 5 Array Member Active
0003:70:00.0/0:0:5:0 RAID 5 Array Member Active
More...
e=Exit q=Cancel r=Refresh t=Toggle f=PageDn b=PageUp
Using the arrow keys, place the cursor next to the PCI-E SAS RAID Adapter, type 1, and press Enter.
10 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
This should display a screen similar to the screen below.
IOA Hardware Resource Information Details
Manufacturer..............:IBM
Machine Type and Model . . . . . . . . . : 57D7001SISIOA
Firmware Version . . . . . . . . . . . . : 14511900
Serial Number. . . . . . . . . . . . . . : 00423088
Part Number. . . . . . . . . . . . . . . : 0000000E2845
Plant of Manufacturer. . . . . . . . . . : 00UE
WriteCacheSize............:256MB
DRAMSize................:512MB
Resource Name. . . . . . . . . . . . . . : /dev/sg12
Physical location
PCI Address. . . . . . . . . . . . . . . : 0003:70:00.0
ResourcePath..............:FE
SCSIHostNumber............:0
Platform Location. . . . . . . . . . . . : U78CB.001.WZS00E2-P1-C14
Cache Protection . . . . . . . . . . . . : Synchronize Cache
Press Enter to Continue
e=Exit q=Cancel
The Firmware Version field should show 14511900. You should now see a Cache Protection field showing
a value of Synchronize Cache. Note, you may need to page down using ’f’ in order to see this value.
11 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Chapter 4
Feature Enablement
4.1 Curses Interface
By default, the write cache should now be set to Write Back, indicating the adapter volatile write cache
is enabled. This can be confirmed by selecting option 4. Work with configuration options from the
main menu, then pressing Enter.
At this point you should see a screen like the following.
IBM Power RAID Configuration Utility
Select one of the following:
1. Work with SCSI bus configuration
2. Work with driver configuration
3. Work with disk configuration
4. Work with adapter configuration
Selection:
e=Exit q=Cancel
Next, select option 3. Work with disk configuration from the menu and press Enter.
12 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
You should now see a screen similar to the following.
Change Disk Configuration
Type option, press Enter.
1=Change Disk Configuration
OPT Name PCI/SCSI Location Description Status
--- ------ ------------------------- ---------------------------- -------------
sda 0003:70:00.0/0:2:0:0 RAID 0 Array Optimized
0003:70:00.0/0:0:0:0 RAID 0 Array Member Active
sdb 0003:70:00.0/0:2:1:0 RAID 10 Array Optimized
0003:70:00.0/0:0:1:0 RAID 10 Array Member Active
0003:70:00.0/0:0:2:0 RAID 10 Array Member Active
sdc 0003:70:00.0/0:2:2:0 RAID 5 Array Optimized
0003:70:00.0/0:0:3:0 RAID 5 Array Member Active
0003:70:00.0/0:0:4:0 RAID 5 Array Member Active
0003:70:00.0/0:0:5:0 RAID 5 Array Member Active
e=Exit q=Cancel r=Refresh t=Toggle
Here, you will see all the physical disks as well as the logical RAID array devices as well. In this example,
we will choose the first RAID array, sda. Using the arrow keys, navigate to the desired RAID array, select
the device by entering ’1’, then press Enter.
Change Configuration of Disk
Current Disk configurations are shown. To change setting hit ’c’ for options
menu. Highlight desired option then hit Enter.
c=Change Setting
QueueDepth...............:4
Device Write Cache Policy. . . . . . . . : Write Back
Or leave blank and press Enter to cancel
e=Exit q=Cancel
Here we can see the Device Write Cache Policy is set to Write Back, as expected. If it is desired to
disable the volatile adapter write cache for a specific RAID array, on this same screen, using the arrow keys,
select the Device Write Cache Policy field, then type ’c’ to display the possible options. Then use the
arrow keys to select the desired caching mode and press Enter. Press Enter again to confirm the change
13 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
4.2 Command Line Interface
Alternatively, the iprconfig command line interface can be used to display and change the current caching
mode.
To list the current hardware RAID arrays current configured on the system use the following com-
mand:
# iprconfig -c show-arrays
Name PCI/SCSI Location Description Status
------ ------------------------- ------------------------- -----------------
sda 0003:70:00.0/0:2:0:0 RAID 0 Array Optimized
sdb 0003:70:00.0/0:2:1:0 RAID 10 Array Optimized
sdc 0003:70:00.0/0:2:2:0 RAID 5 Array Optimized
To display the current caching mode for the first RAID array, we do the following:
# iprconfig -c query-write-cache-policy sda
write back
Here we can see the write cache policy is set to write back, indicating the volatile adapter write cache is
enabled for this array.
To change this setting, you can do the following.
# iprconfig -c set-write-cache-policy sda writethrough
# iprconfig -c query-write-cache-policy sda
write through
Now let’s change it back.
# iprconfig -c set-write-cache-policy sda writeback
# iprconfig -c query-write-cache-policy sda
write back
14 of 15

IBM Power Systems 57D7/57B4 Adapter Write Cache
Version 1.3 dated May 19, 2016
Chapter 5
Support
Problems this feature should be reported using the normal IBM Support process by opening a hardware
support ticket.
15 of 15