Virident FlashMAX II User manual

Virident FlashMAX II
User’s Guide for VMware ESXi
© 2012- 13 Virident Systems

Virident FlashMAX II User’s Guide, Version 3.1.1 - i -
Table of Contents
1Introduction .......................................................................................................................................... 1
2System Requirements ........................................................................................................................... 1
3Hardware Installation ........................................................................................................................... 1
3.1 Bracket Installation ....................................................................................................................... 2
3.2 PCI Slot Selection .......................................................................................................................... 3
4System BIOS Settings ............................................................................................................................ 3
4.1 Fan Speed Settings in BIOS ........................................................................................................... 3
4.2 Processor Settings in BIOS ............................................................................................................ 4
5Software Installation............................................................................................................................. 5
5.1 Device Driver Installation.............................................................................................................. 6
5.1.1 Install Virident Driver VIBs Using Offline Bundle ................................................................... 6
5.1.2 Install Virident Driver Using VMware vSphere Update Manager (VUM)............................... 6
5.1.3 Reboot................................................................................................................................... 8
5.1.4 Confirm driver installation after reboot................................................................................ 8
5.2 Ensuring FlashMAX II is Operational............................................................................................. 8
5.3 Firmware Upgrade ........................................................................................................................ 8
5.3.1 FlashMAX II Firmware Upgrade Procedure........................................................................... 9
6Configuring FlashMAX II Datastore using vSphere Client ...................................................................10
7FlashMAX II Configuration & Management Utilities...........................................................................15
7.1 Esxcli vgc config........................................................................................................................... 15
7.2 Esxcli vgc monitor ....................................................................................................................... 17
7.3 Esxcli vgc beacon.........................................................................................................................18
7.4 Using vgc-diags to collect log bundle..........................................................................................19
7.5 Using vgc-secure-erase for purging the drive data.....................................................................20
8Multiple Virident FlashMAX II Devices................................................................................................ 23
9Using FlashMAX II Devices in Pass-Through Mode ............................................................................. 23
10 Optimizing Performance .................................................................................................................23

Virident FlashMAX II User’s Guide, Version 3.1.1 - ii -
10.1 Usable space vs. write performance........................................................................................... 23
10.2 Partitioning and write performance ...........................................................................................23
10.3 Setup recommendations to utilize full performance of FlashMAX II with a single Guest VM. ..24
10.3.1 After adding 4 vmdks using VM “Edit Settings” will see VM properties hardware tab
shows as below ...................................................................................................................................24
11 Troubleshooting on ESXi Systems...................................................................................................26
11.1 Device not detected on ESXi .......................................................................................................26
11.2 Firmware Mismatch or Device not Initialized............................................................................. 26
11.3 Thermal Shutdown...................................................................................................................... 27
11.4 Device is Degraded or in Read-Only mode .................................................................................27
11.5 Searching for errors in the logs...................................................................................................27
12 Onboard diagnostic LEDs ................................................................................................................ 28
13 Contacting Virident Support ...........................................................................................................28

Virident FlashMAX II User’s Guide - Version 3.1.1 - iii -
VIRIDENT
SOFTWARE END USER LICENCE AGREEMENT
This is a legal agreement between you, the end user,
("You") and Virident Systems Inc. (“Virident”). BY
INSTALLING THIS SOFTWARE OR CLICKING ON THE
“ACCEPT” BUTTON YOU ARE AGREEING TO BE BOUND BY
THE TERMS OF THIS AGREEMENT. Certain of the software
delivered may be open source software and is licensed
pursuant to the applicable open source licenses specified in
the documentation. This Agreement applies to the
software that is proprietary to Virident (the “Program(s)”).
1. License
Virident grants you a non-exclusive, non-transferable
license to use one copy of the Program(s) for each server
product for which you have paid the applicable license fee,
for internal business purposes, in accordance with the user
documentation, and in accordance with the terms of this
Agreement. You may make one back-up copy of the
Program, provided that such copy shall be subject to the
terms of this Agreement. You may not use the Program(s)
with any equipment other than the equipment specified by
Virident. You may not loan, rent, lease, license, sublicense
or distribute the Program(s) or any copies.
3. Copyright and other intellectual property rights
All intellectual property rights in the Program(s) and user
documentation are owned by Virident and are protected by
copyright and other intellectual property laws,
international treaty provisions and applicable national
laws. All rights not expressly granted in this Agreement are
reserved by Virident.
4. Further restrictions
You must not reverse engineer or decompile the
Program(s) except to the extent permitted by applicable
law despite any contractual prohibition. If you wish to
obtain interface information relating to the Program, to
achieve interoperability, a request should be made in
writing to Virident.
5. No warranties
ALL PROGRAMS ARE PROVIDED “AS IS” WITHOUT
WARRANTY OF ANY KIND. NO CONDITIONS, WARRANTIES
OR REPRESENTATIONS SHALL BE DEEMED TO HAVE BEEN
GIVEN BY VIRIDENT IN RESPECT OF THE PROGRAM OR
DOCUMENTATION. VIRIDENT DOES NOT WARRANT OR
ENTER INTO ANY TERM TO THE EFFECT THAT ANY
PROGRAM WILL BE FREE FROM DEFECTS OR THAT ITS
OPERATION WILL BE ERROR-FREE. ALL IMPLIED
WARRANTIES, INCLUDING THE IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR PURPOSE,
NONINFRINGEMENT AND SATISFACTORY QUALITY ARE
EXPRESSLY DISCLAIMED. If you have purchased technical
support then errors will be addressed in accordance with
the applicable technical support policies.
6. Limitation of Liability
VIRIDENT'S TOTAL AGGREGATE LIABILITY FOR ANY AND
ALL CLAIMS SHALL NOT EXCEED THE AMOUNTS PAID FOR
THE PROGRAMS DURING THE SIX-MONTH PERIOD PRIOR
TO THE DATE ON WHICH THE CLAIM AROSE. IN NO EVENT
SHALL VIRIDENT BE LIABLE TO ANYONE FOR SPECIAL,
COLLATERAL, INDIRECT, EXEMPLARY, INCIDENTAL OR
CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING
COSTS OF COVER, COSTS OF REWORK, RETESTING OR
REMOVAL AND REINSTALLATION OF PRODUCTS, LOSS OF
GOODWILL, LOSS OF PROFITS, OR LOSS OF USE) IN
CONNECTION WITH THIS AGREEMENT ON ANY THEORY OF
LIABILITY AND WHETHER OR NOT VIRIDENT HAS BEEN
ADVISED OF THE POSSIBLITY OF SUCH DAMAGES
7. Termination
Your license to use the Program(s) automatically
terminates if You fail to comply with any provision of this
agreement.
8. Assignment
You may assign all of Your rights to use the Program(a) to
another person or entity provided that: (a) You transfer
this agreement, the physical medium in which the Program
is made available, user documentation and all copies of the
Program(s) and user documentation together with the
equipment on which it is installed; (b) You retain no copies
of the Program(s), including copies stored on any medium;
and (c) You remain liable under this Agreement for any
breach of its terms by the assignee. Otherwise, the
Programs may not be assigned without the consent of
Virident.
9. Statutory rights

Virident FlashMAX II User’s Guide - Version 3.1.1 - iv -
If You purchased the copy of the Program(s) covered by
this Agreement as a consumer, the exclusions and
limitations contained in this Agreement do not affect any
statutory rights to which You are entitled and which may
not by law be excluded or limited.
10. Certain Applications
Virident’s Products are not designed and not warranted to
be suitable for use in military applications and/or military
environments, life support applications or other critical
applications which may involve potential risk of death,
personal injury or severe property or environmental
damage Use of Products in such applications is fully at the
risk of Buyer.
11. U.S. Government License Rights
If You are an entity of the United States Government you
receive the Programs with restricted rights. In no event
shall the Government acquire greater than
RESTRICTED/LIMITED RIGHTS. At a minimum, use,
duplication, or disclosure by the Government is subject to
restrictions as set forth in FAR §52.227-14 Alternates I, II
and III (JUN 1987); FAR §52.227-19 (JUN 1987) and/or FAR
§12.211/12.212 (Commercial Technical Data/Computer
Software); and DFARS §252.227-7015 (NOV 1995)
(Technical Data) and/or DFARS §227.7202 (Computer
Software) as applicable. If Products are sold under a US
Government contract, Virident rejects provisions or clauses
required to be passed on to sellers pursuant to such
contract and such provisions or clauses shall not be
deemed to be binding on Virident unless accepted in
writing by an authorized representative.
12. Miscellaneous
A. Export Control: Virident's obligation to provide the
Products, documentation, or any media in which any of the
foregoing is contained, as well as any other technical
assistance shall be subject in all respects to such United
States laws and regulations as shall form time to time
governing the license and delivery of technology and
products abroad by persons subject to the jurisdiction of
the United States, including the Export Administration Act
of 1979, as amended, any successors or legislation, and the
Export Administration Regulations and other United States
laws and regulations governing exports in effect from time
to time.
B. Governing Law and Jurisdiction: This Agreement is
deemed to have been entered into in the State of
California, and its interpretation, construction, and the
remedies for its enforcement or breach are to be applied
pursuant to and in accordance with the laws of the State of
California, excluding its conflict of law provisions. The UN
Convention on the International Sale of Goods does not
apply to this Agreement. The parties agree that in the
event any action, suit or proceeding is brought by either
party to enforce this Agreement, then such action, suit or
proceeding may be brought only in a United States District
Court for the Northern District of California, or, if that court
does not have or elect to exercise its jurisdiction, then in
the appropriate state court in the State of California,
county of Santa Clara.
C. Contingency: Virident is not liable, either wholly or in
part, for nonperformance or a delay in performance due to
force majeure or contingencies or causes beyond the
reasonable control of Virident, including but not limited to,
shortage of labor, fuel, raw material or machinery or
technical or yield failure where Virident has exercised
ordinary care in the prevention thereof. Production and
deliveries may be allocated by Virident in a reasonable
manner in the event of shortage of goods.
D. Partial Invalidity: If any provision of this Agreement is
held to be invalid or unenforceable, the remainder of this
Agreement shall continue in full force and effect so long as
neither party is denied material economic benefit
hereunder and will be interpreted to reflect the original
intent of the parties.
E. Waiver. A waiver by Virident of any term or condition of
this Agreement in any instance shall not be deemed or
construed to be a future waiver of such term or condition.
F. Construction; Language. As used in this Agreement, the
words "include" and "including" and variations thereof, will
not be deemed to be terms of limitation, but rather will be
deemed to be followed by the words "without limitation."
The headings in this Agreement will not be referred to in
connection with the construction or interpretation of this
Agreement. This Agreement is in the English language
only, which language shall be controlling in all respects,
and all notices under this Agreement shall be in the English
language.

Virident FlashMAX II User’s Guide - Version 3.1.1
1
1Introduction
The Virident FlashMAX™line of solid state storage devices is a revolutionary product family that utilizes the
industry standard PCI Express bus interface, along with an innovative hardware and software architecture
to deliver highest sustained application performance over its life time and enterprise-class reliability.
Virident FlashMAX II enables enterprises to tackle performance-intensive applications such as databases,
business analytics, virtualization and high-performance computing applications. This document explains
how to install, configure, and manage Virident FlashMAX II devices with software version 3.1.1 on ESXi
5.0/5.1 based systems.
2System Requirements
For optimal performance the system hardware needs to meet the following requirements:
Model
PCI Express
Airflow
Processor
Memory
FlashMAX II
550GB and 1.1TB
Gen 2 x8
electrical
150 LFM @
45C inlet
Multicore Intel Xeon®
5500 or AMD
Opteron™
2GB for 550GB
3GB for 1.1TB
FlashMAX II
2.2TB
Gen 2 x8
electrical
200 LFM @
45C inlet
Multicore Intel Xeon®
5500 or AMD
Opteron™
6GB
Please ensure that you have ample memory for other applications to run in addition to the minimum
memory required by each installed Virident device.
3Hardware Installation
Please observe Electrostatic Discharge (ESD) precautions while installing or handling
FlashMAX II devices. ESD can damage server and/or device components.
Make sure to power down the system before installing FlashMAX II devices. Disconnect
the server from the AC power to ensure no dangerous voltages are present.

Virident FlashMAX II User’s Guide - Version 3.1.1
2
3.1 Bracket Installation
If necessary, install the appropriate bracket on the FlashMAX II device.
Please use only brackets supplied by Virident with the device. Also, do not remove the
heatsinks during the bracket replacement process as doing so could incur damage to the
device and void the warranty.
Single Board: If your single board device has nuts like the one in the picture below then, follow the single
board instructions. If your single board device does not have any nuts, please follow the double board
instructions.
To replace brackets for a single board device, please remove only the two screws closest to the bracket
(see picture below). Please slide the bracket in horizontally to line up with the screw holes. You will need
an ‘m 2.5’ screw driver and an ‘m 2.5’ nut driver. The screws need to be tightened to 4lb-in (0.45Nm).
Double Board: To replace brackets for a double board device, or a single board device without retaining
nuts, please remove only the two screws nearest the bracket (see picture below). Please slide the bracket
in horizontally to line up with the screw holes. You will need an ‘m 2.5’ screw driver. The screws need to be
tightened to 4lb-in (0.45Nm).
Please remove only these two
screws to replace brackets

Virident FlashMAX II User’s Guide - Version 3.1.1
3
3.2 PCI Slot Selection
FlashMAX II devices need to be installed in x8 or x16 PCIe Gen 2 slots. If no PCIe x8/x16 slots are available,
x4 slots can be used. However, using an x4 slot may result in up to 50% reduction in performance.
For optimal performance and cooling on specific servers, please go to the online support
knowledge base at http://support.virident.com/supportweb/kbase.
Some systems have PCIe slots with x8 physical connectors but only provide x4 electrical
connectivity. From a performance point of view these slots behave like an x4 slot. Check
your system documentation for PCIe slot specifications.
Some systems have PCIe slots with x8 physical connectors but only provide x4 electrical
connectivity. From a performance point of view these slots behave like an x4 slot. Check
your system documentation for PCIe slot specifications.
4System BIOS Settings
After a system restart, prior to booting the operating system, enter the system BIOS setup to verify several
important settings. Note that each BIOS often has its own nomenclature for these settings, so the exact
wording of the configuration parameters may not match the examples. Not all BIOS/s will have all of these
options.
4.1 Fan Speed Settings in BIOS
If your system has BIOS settings for configuring system fan speeds, select options that maximize the
airflow.
Please remove only these two
screws to replace brackets.

Virident FlashMAX II User’s Guide - Version 3.1.1
4
On HP systems: BIOS -> Advanced Options -> Thermal Configuration -> Increased/Maximum Cooling
On Dell systems: System Settings -> iDRAC Settings -> Thermal
Maximum Performance
Fan Speed Offset
High Fan Speed Offset
4.2 Processor Settings in BIOS
The following settings are often found under the “Advanced: Processor/Chipset Control” BIOS menus.
Disable the following optional settings:
Processor Performance States / C States / C-State Tech / EIST
C1E States / C1 Enhanced States
(Link) Active State Power Management
Enable the following on Intel™ based systems:
Hyperthreading / Logical Processors
Turbo Mode
Finally, if there are performance profiles (System Power Modes) available, set them to “Maximum
Performance.”
Intel has identified an issue in Xeon E5-2600 series processors that may result in severe drops
of PCIe and FlashMAX II bandwidth when running workloads that have low CPU consumption.
To avoid this issue, C2 and C1E processor power states must be disabled. For details, please
see Erratum BT160 in the below document:
http://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-family-
spec-update.pdf

Virident FlashMAX II User’s Guide - Version 3.1.1
5
5 Software Installation
This section covers the procedure for installing the device drivers and updating the firmware for FlashMAX
II devices on an ESXi 5.0 or ESXi 5.1 system.
All the ESXCLI commands listed in this section can be executed in one of the following methods.
1. Directly on the ESXi host after logging in via SSH (SSH service must be enabled)
Syntax:
Example:
2. Remotely using the vSphere Remote CLI utility (VCLI)
http://www.vmware.com/support/developer/vcli/
Note that if any command uses a filename, the file should be present on the target system and
not on the system where VCLI is executed.
Syntax:
Example:
~# esxcli <command name space> <command arguments>
~# esxcli software vib list
~# esxcli --server=<ESX host name/IP Address> --username=<root>
--password=<password> <command name space> <command arguments>
~# esxcli --server='lab-m6' --username='root' –
password='password' software vib list

Virident FlashMAX II User’s Guide - Version 3.1.1
6
5.1 Device Driver Installation
Download the device driver offline bundle from Virident Support website before proceeding. Ensure that
the device is seen by the operating system.
5.1.1 Install Virident Driver VIBs Using Offline Bundle
Note that full path to the driver offline bundle must be provided.
5.1.2 Install Virident Driver Using VMware vSphere Update Manager (VUM)
1. Configure update manager and install the required plugin for vSphere client. Import the
downloaded device driver bundle into the update manager Patch Repository as show below.
~ # lspci | grep -i virident
000:131:00.0 Memory controller: Virident Systems, Inc. FlashMAX II [vgca]
# esxcli software vib install -d /tmp/VSI-FlashMAXII-ESXi5x-
3.1.1.51450.C6.zip
Installation Result
Message: The update completed successfully, but the system needs to be
rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: Virident_bootbank_block-vgc_3.1.1 51450.C6-
1OEM.500.0.0.472560, Virident_bootbank_esxcli-vgc-utils_3.1.1.51450.C6-
5.0.0.464360
VIBs Removed:
VIBs Skipped:

Virident FlashMAX II User’s Guide - Version 3.1.1
7
2. Create a new baseline with type as Host Extension and include the driver from the repository.
3. Attach proper baseline to host or cluster and stage/remediate.

Virident FlashMAX II User’s Guide - Version 3.1.1
8
For additional vSphere Update Manager details please refer to the following VMware KB article
http://kb.vmware.com/kb/2004501
5.1.3 Reboot
Reboot the host for the driver installation to take effect. Please ensure that all the Guest VMs are gracefully
shutdown before the reboot.
5.1.4 Confirm driver installation after reboot
5.2 Ensuring FlashMAX II is Operational
Check if Driver Modules are loaded using the vmkload_mod command:
To verify the other operational parameters, such as FlashMAX II device health, use the esxcli vgc
monitor command documented in this Guide. In addition, please review the section on configuring
datastore on ESXi systems as well.
5.3 Firmware Upgrade
In some cases your FlashMAX II device may require firmware upgrade. Use the vgc-monitor utility to check
whether or not the firmware is up to date.
If the firmware is up to date, the following is displayed:
~# reboot
~ ~ # esxcli software vib list | grep -i virident
~ # esxcli software vib list | grep -i virident
block-vgc 3.1.1.51450.C6-1OEM.500.0.0.472560 Virident
VMwareCertified 2013-01-25
esxcli-vgc-utils 3.1.1.51450.C6-5.0.0.464360 Virident
PartnerSupported 2013-01-25
~ # vmkload_mod -l | grep vgc
vgc 1 1788

Virident FlashMAX II User’s Guide - Version 3.1.1
9
The following message is displayed if the FlashMAX II device has older firmware
5.3.1 FlashMAX II Firmware Upgrade Procedure
Use the vgc-update utility to upgrade FlashMAX II Firmware as follows:
~ # esxcli vgc monitor drive
vgc-monitor: 3.1.1(51450.C6)
Driver Uptime: 0:06
Card_Name Num_Partitions Card_Type Status
/dev/vgca 1 VIR-M2-LP-2200-2A Good
Partition Usable_Capacity RAID
/dev/vgca0 2222 GB enabled
~ # esxcli vgc monitor drive
vgc-monitor: 3.1.1(51450.C6)
/dev/vgca: Could not get information about FlashMAX drive /dev/vgca: No
such device (rc=19)
Unclaimed Drives:
0000:86:00.0 FLASH memory: Virident Systems Inc. FlashMAX Drive (rev
00) (0x1a78:0x0040, 0x1a78:0x0040)

Virident FlashMAX II User’s Guide - Version 3.1.1
10
Host system Power Cycle is mandatory for successful firmware upgrade
6Configuring FlashMAX II Datastore using vSphere Client
Add the FlashMAX II device as a VMware datastore using the vSphere client.
Select Configuration-> Storage-> Add Storage…
~# cd /opt/vgc/bin/
~# sh vgc-update.sh
WARNING! This will update FlashMAX firmware.
This is sensitive operation and may potentially cause data loss.
All data on all FlashMAX drives must be backed up before the firmware
update.
PLEASE MAKE SURE THAT POWER IS NOT DISCONNECTED DURING THE UPDATE.
PLEASE DO NOT INTERRUPT THE UPDATE PROCESS ONCE IT IS STARTED.
Do you want to continue? (Yes/No) : yes
Firmware upgrade starts within few seconds, Please wait...
Initialize /dev/vgcmgmta-zion_secondary_image
Erasing 128 Kibyte @ 1fe0000 -- 100% complete.
Upgrading Firmware-- 100 % complete.
Verifying Firmware -- 100 % complete.
Initialize /dev/vgcmgmta-zion_image
Erasing 128 Kibyte @ 1fe0000 -- 100% complete.
Upgrading Firmware-- 100 % complete.
Verifying Firmware -- 100 % complete.
#################################
# #
# PLEASE POWER CYCLE SYSTEM NOW #
# #
#################################

Virident FlashMAX II User’s Guide - Version 3.1.1
11

Virident FlashMAX II User’s Guide - Version 3.1.1
12

Virident FlashMAX II User’s Guide - Version 3.1.1
13

Virident FlashMAX II User’s Guide - Version 3.1.1
14

Virident FlashMAX II User’s Guide - Version 3.1.1
15
7FlashMAX II Configuration & Management Utilities
A suite of CLI utilities is delivered with the device driver to allow end-users to configure, manage and
monitor the FlashMAX II devices.
7.1 Esxcli vgc config
Most administrative interactions with the Virident FlashMAX II device are handled by the esxcli vgc config
command. This utility can show the current partitioning and write acceleration modes on an existing
device.
The usage information for esxcli vgc config is listed below. Users can set the number of physical partitions
(“-n XX”) and a write performance profile (“-m YYY”) of the Virident FlashMAX II devices.
esxcli vgc config -h
Usage: esxcli vgc config {cmd} [cmd options]
Available Commands:
drive drive related options
list List all configuration
partition Partition related options
reset Reset Configurations to Factory default
esxcli vgc config list
Description:
Lists available devices configuration
Syntax:
Usage: esxcli vgc config list [cmd options]
Description:
list List all configuration
Cmd options:
-d|--drive=<str> drive name /dev/vgc[a-h].
-p|--partition=<str> partition name /dev/vgc[a-h][0-1]
Example:
# esxcli vgc config list
vgc-config: 3.1.1(51450.C6)
Current Configuration:
/dev/vgca 1 partition(s)
/dev/vgca0 mode=maxperformance sector-size=512 raid=enabled