CLOUDIAN HyperStore User manual

USING HYPERSTORE WITH OBJECT LOCK 1
QUICKSTART GUIDE
Using Object Lock with
Cloudian HyperStore

USING HYPERSTORE WITH OBJECT LOCK 2
Table of Contents
HYPERSTORE INTRODUCTION ........................................................................................ 3
OBJECT LOCK FEATURE OVERVIEW ............................................................................. 3
ENABLING OBJECT LOCK IN HYPERSTORE.................................................................. 4
HYPERSTORE PREREQUISTES............................................................................... 5
DEPLOY OR UPGRADE HYPERSTORE TO 7.2 ....................................................... 5
ENABLING OBJECT LOCK ....................................................................................... 5
DISABLING THE ROOT USER PASSWORD ............................................................ 6
DEPLOY AND CONFIGURE AWS CLI ............................................................................... 7
DEPLOY THE AWS CLI ON LINUX............................................................................ 7
Prerequisites..................................................................................................... 7
AWS CLI Installation on Linux........................................................................... 7
CONFIGURE THE AWS CLI....................................................................................... 7
USING THE AWS CLI TO ENABLE OBJECT LOCK ......................................................... 9
CONCLUSION ....................................................................................................................10

USING HYPERSTORE WITH OBJECT LOCK 3
HYPERSTORE INTRODUCTION
Cloudian HyperStore® is a scale-out object storage system designed to manage massive amounts of
unstructured data. It is a Software Defined Storage (SDS) platform which runs on any standard x64 server
platform. This dramatically reduces the cost for datacenter storage while still providing limitless scalability,
extreme availability and unprecedented reliability. Cloudian HyperStore, with native S3 compatibility,
enables data centers to provide highly cost-effective on-premise unstructured data storage repositories.
Cloudian HyperStore is built on standard hardware that can span across the enterprise as well as out into
public cloud environments.
Cloudian HyperStore is available as a stand-alone software or fully integrated with hardware as a Cloudian
HyperStore appliance. It easily scales to limitless capacities and offers multi-datacenter storage.
HyperStore also has fully automated data tiering to all major public clouds, including AWS, Azure and
Google Cloud Platform. It fully supports S3 applications and has flexible security options. HyperStore
deployment models include on-premises storage, distributed storage, storage-as-a-service or even other
combinations as illustrated below.
OBJECT LOCK FEATURE OVERVIEW
With the 7.2 release, HyperStore can now implement WORM (Write Once Read Many) protection for stored
objects by supporting the standard AWS S3 Object Lock functionality. To use the Object Lock within
HyperStore, you must have a separate Object Lock license. It is also required to enable the HyperStore
Shell (HSH) and disable the root account password on all of the HyperStore nodes. For more information
see the Prerequisites section in 4.3.7.2 within the online user guide.
For an excellent overview of the Object Lock feature itself, please see the following introduction published
by Amazon Web Services (AWS):
•https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html
•https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock.html
•https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-managing.html
Once Object Lock is enabled in HyperStore, then the HyperStore S3 Service supports all the standard
AWS S3 API methods and headers associated with the S3 Object Lock feature. Using these standard S3
API methods and headers, third party S3 client applications can:
•Enable Object Lock on new buckets as those buckets are created. Note that Object Lock can only
be enabled on newly created buckets, as part of the operation that creates the bucket just as AWS
supports. Note that Object Lock cannot be enabled on buckets that already exist.
•Enabling Object Lock as a bucket is created automatically enables Versioning on the bucket.
Object Lock can only be used in combination with Versioning enabled.

USING HYPERSTORE WITH OBJECT LOCK 4
Enabling Object Lock on a bucket as the bucket is created does not by itself have the effect of locking
objects that are subsequently stored in that bucket. It only makes it possible to lock such objects, using the
methods described as follows.
1. For an Object Lock enabled bucket, optionally set a bucket default Object Lock configuration that
will apply to all objects that are subsequently created in the bucket. The default Object Lock
configuration specifies a Retention time period that will be applied to objects that are subsequently
created in the bucket. Each object's retention period starts when the object is created in the bucket
(and for objects with multiple versions, each object version's retention period starts when that
object version is created). The default Object Lock configuration also specifies which of two modes
the Object Lock is implemented in:
•Governance mode, which allows privileged users to change the retention period or
delete objects before their retention period completes.
•Compliance mode, which does not allow any user to change the retention period
or delete objects before their retention period completes.
2. For an Object Lock enabled bucket, optionally set Object Lock attributes on individual objects,
either as the objects are created in the bucket or after the objects have been created in the bucket.
The Object Lock set on an object can be either or both of:
•Retention, in Governance mode or Compliance mode
•Legal Hold, which applies for an indefinite period until explicitly released. While
objects are in Legal Hold, no user can delete them.
These per-object Object Lock attributes override the bucket's default Object Lock configuration, if a default
configuration has been set.
Please note that as of the 7.2 release, all users must use a third party S3 client application that
supports the standard S3 APIs pertaining to object locking. In the current HyperStore release, the CMC
does not support setting up object locking on buckets or objects. It is also not possible to use the HSH to
create Object Lock enabled buckets or manage Object Lock bucket policies at the command line. This
guide will demonstrate how to create and configure bucket level Object Lock capabilities using the third-
party AWS CLI product supplied by Amazon. Please note that not all third-party S3 applications support
the Object Lock feature at this time so be sure to choose an application that does.
ENABLING OBJECT LOCK IN HYPERSTORE
This section will briefly describe how to enable Object Lock within HyperStore. If more information is
needed, all of the details to enable Object Lock within HyperStore can be found in the online user guide by
logging in as the default admin user and clicking on Help as illustrated below.

USING HYPERSTORE WITH OBJECT LOCK 5
HYPERSTORE PREREQUISTES
To setup and use Object Lock in HyperStore on newly created buckets, the following prerequisites must be
completed first:
•Deploy or upgrade HyperStore to version 7.2 on a minimum of 3 nodes
•Apply a HyperStore license that supports the Object Lock feature. To view the current license, see
the Cloudian Management Console (CMC) Cluster Information page. The "Object Lock License"
field will indicate either "Enabled" or "Disabled". If it shows Disabled, contact Cloudian customer
support for an upgraded license key.
•Enable HyperStore Shell (HSH) The HSH shell is a new feature in the 7.2 release that provides a
secure command line interface for management tasks.
•Have the root user account password disabled. Once disabled, root password access to the
HyperStore will require Cloudian Support for assistance.
A 3rd party S3 application that supports S3 API for object locking is also required. In this guide, the AWS
CLI will be used. Only newly created buckets support Object Lock. All new buckets that support Object
Lock will also automatically have versioning enabled. Existing buckets cannot have object lock enabled.
Be aware that other additional configuration changes In HyperStore may be needed in addition to Object
Lock. Many third-party S3 applications may require IAM, SSL enabled for secure https S3 endpoints and
other features that are not enabled within HyperStore by default.
DEPLOY OR UPGRADE HYPERSTORE TO 7.2
The first step is to deploy or upgrade HyperStore to 7.2. If doing a new deployment, a minimum of 3 nodes
is required. Please follow the instructions in the HyperStore Installation Guide.
ENABLING OBJECT LOCK
To enable Object Lock, do the following steps.
1. Apply a license file that supports the Object Lock feature from the Cluster Information page in the
CMC. First click on Choose File.
a. Select an Object Lock enabled license file from the directory and file list that is displayed.
b. Once selected and uploaded, click on Update License. After a few minutes, the Object
Lock License will show “Enabled”. A manual refresh of the browser may be required.

USING HYPERSTORE WITH OBJECT LOCK 6
2. Enable the HSH and disable the root account password. What follows is a brief example using the
default admin user. In practice, other user accounts will be used. For details see section 4.3.1.2,
Enabling the HSH and Managing HSH Users in the 7.2 administrators guide.
a. Log into the Puppet Master node as the root user.
b. Check to confirm that the HSH is currently disabled. The command that will be used is
hsctl, a new node management tool that remains mostly behind the scenes in HyperStore
7.2 but will be more prominent in future releases.
[root@hst72]# hsctl config get hsh.enabled
False
c. Set hsh.enabled to true.
[root@hst72]# hsctl config set hsh.enabled=true
d. Push the configuration change out to the cluster.
[root@hst72]# hsctl config apply hsh
e. Confirm that HSH is now enabled.
[root@hst72]# hsctl config get hsh.enabled
True
3. HSH is now enabled in your system, but no users are yet able to log into it. To provision the default
admin user (and others) for HSH do the following steps:
a. After installing or upgrading to HyperStore 7.2, log into the CMC as the "admin" user
b. Change the "admin" user's password. This password change causes the system to create
a corresponding HSH user.
4. Once an HSH user has been created, that user can use SSH to log into any HyperStore node.
Upon login, the user's shell will be the HyperStore shell. The prompt will appear as follows:
5. For the default admin user, the prompt will appear as follows:
sa_admin@hyperstore1$
6. You can confirm that you are in the HyperStore shell by typing help:
sa_admin@hyperstore1$ help
DISABLING THE ROOT USER PASSWORD
To disable root password access to all HyperStore nodes:
1. As root, log into the Puppet Master node and then change into the staging directory (/opt/cloudian-
staging/<version>).
2. Launch the HyperStore installer.
./cloudianInstall.sh
3. In the installer main menu enter 4 for "Advanced Configuration Options", then at the next menu
enter m for "Disable the root password").
4. Follow the prompts to disable the root password.
After exiting the installer, log out from the node. Then try to log back in as root, using the root password --
the login attempt should fail. Then log in as sa_admin or another HSH user, with that user's password. The
login should succeed, and you should be in the HyperStore shell.

USING HYPERSTORE WITH OBJECT LOCK 7
DEPLOY AND CONFIGURE AWS CLI
Once HyperStore is ready for Object Lock, a third-party application is required to create new buckets with
Object Lock enabled and to set new or change Object Lock retention policies.
In this section, the AWS CLI will be used with Object Lock. First the AWS CLI will be deployed and
configured using a CentOS Linux server. The AWS CLI can also be installed on Windows systems,
however that won’t be covered here. For details on installation, see https://cloudacademy.com/blog/how-
to-use-aws-cli/.
Once the AWS CLI is deployed and configured for use with HyperStore, there will be a few example
commands on how to create a new bucket in HyperStore with Object Lock enabled, create a new
GOVERANCE policy on the new bucket and set the default retention time.
Note: The S3 endpoint must be resolvable by name. IP addresses cannot be used. Either have the IP
address to name mapping in the /etc/hosts file or in the DNS server. To test, ping the S3 endpoint. For
example:
$ ping s3-reg01.cloudiantme.local
PING s3-reg01.cloudiantme.local (10.50.125.88) 56(84) bytes of data.
64 bytes from s3-reg01.cloudiantme.local (10.50.125.88): icmp_seq=1 ttl=64 time=0.355 ms
DEPLOY THE AWS CLI ON LINUX
Prerequisites
To deploy the AWS CLI, at least Python 2 version 2.6.5+ or Python 3 version 3.3+ is required to be
installed. To verify the current python version, run the following command:
$ python --version
AWS CLI Installation on Linux
The recommendation for installing the AWS CLI is to use the bundled installer provided by AWS. The
bundled installer includes all dependencies required for the installation.
1. To begin the installation run the following command:
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-
bundle.zip"
2. Next, you must unzip the downloaded package from step 1:
$ unzip awscli-bundle.zip
3. Once the package in unzipped, you can run the installation:
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Using the -b option allows all users to use the AWS CLI from any directory, meaning you will not need to
specify the install directory in the user’s $PATH variable.
CONFIGURE THE AWS CLI
Once you have installed the AWS CLI, you now need to configure it to be able to connect to a HyperStore
user account. To do so, enter the following from your command prompt:
$ aws configure
Through aws configure, the AWS CLI will prompt you for 4 pieces of information. The S3 HyperStore user
account access key ID and secret access key serve as the account credentials. The other information that
is needed is the region name and output format. It is possible to generate new credentials within
HyperStore Identity and Access Management (IAM) also if desired.

USING HYPERSTORE WITH OBJECT LOCK 8
The user account credentials are found here
Input the Access Key ID and Secret Key when prompted form the aws configure command as shown
$ aws configure
AWS Access Key ID [****************6d60]:
AWS Secret Access Key [****************gXhM]:
Default region name [us-east-1]:
Default output format [json]:
The default region name simply defines the Region where S3 requests will be sent to. The region is
defined during the Cloudian HyperStore installation procedure and can be found in the CMC on the Cluster
Information page or the user credentials page.
The default output format specifies how the results are formatted. Values that can be used here include:
•json
•text
•table
The S3 access key ID and AWS secret access key are both used to authenticate the Cloudian HyperStore
account. This authorizes the HyperStore user to carry out specific tasks and functions as defined by the
permissions level. The access key ID is made up of 20 random uppercase alphanumeric characters, such
as the one displayed within the CMC.The secret access key is made up of 40 random upper and
lowercase alphanumeric and non-alphanumeric characters. This key can also be accessed via the CMC

USING HYPERSTORE WITH OBJECT LOCK 9
USING THE AWS CLI TO ENABLE OBJECT LOCK
Once configured, the AWS CLI can be used to access HyperStore and create new buckets with Object
Lock enabled using the S3 API command set. The example that follows shows a few of many commands
that can be used to manage buckets and objects. A good introduction to using the S3 API with the AWS
CLI can be found here: https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-
apicommands.html. For a complete reference see: https://docs.aws.amazon.com/cli/latest/reference/s3api/
1. First test that the AWS CLI is set up properly by listing existing buckets for the user account
configured earlier. This user has 3 buckets.
$ aws s3 --endpoint-url=http://s3-reg01.cloudiantme.local ls
2019-11-15 16:23:46 b01
2019-11-15 16:29:09 b02
2019-11-15 16:32:26 b03
2. Create a new bucket with object lock enabled using the S3 API command set.
$ aws s3api create-bucket --bucket oblck01 --object-lock-enabled-for-bucket
--endpoint-url=http://s3-reg01.cloudiantme.local
{
"Location": "http://oblck01.s3-reg01.cloudiantme.local"
}
3. List the buckets to verify it was created properly. Note that oblck01 now appears in the list.
$ aws s3 --endpoint-url=http://s3-reg01.cloudiantme.local ls
2019-11-15 16:23:46 b01
2019-11-15 16:29:09 b02
2019-11-15 16:32:26 b03
2019-11-18 16:11:48 oblck01
4. Verify that Object Lock is enabled on the new bucket
$ aws s3api get-object-lock-configuration --bucket oblck01 --endpoint-
url=http://s3-reg01.cloudiantme.local
{
"ObjectLockConfiguration": {
"ObjectLockEnabled": "Enabled"
}
}
5. A JSON file can be used to set the Object Lock policy and default retention time for objects in the
bucket. The example JSON file sets the bucket to GOVERNANCE mode and a retention of 1 day.
$ aws s3api put-object-lock-configuration --bucket b02 --object-lock-
configuration file://object-lock-config-g-1day.json

1
6. View the Object Lock policy to verify that it is set correctly after applying the Object Lock policy.
$ aws s3api get-object-lock-configuration --bucket b02 --endpoint-
url=http://s3-reg01.cloudiantme.local
{
"ObjectLockConfiguration": {
"ObjectLockEnabled": "Enabled",
"Rule": {
"DefaultRetention": {
"Mode": "GOVERNANCE",
"Days": 1
}
}
}
}
CONCLUSION
This quick start guide illustrated how to configure and use the S3 Object Lock feature on HyperStore using
the AWS CLI third-party S3 application. A few examples on using the AWS CLI with the S3 API command
set where shown. Other third-party applications that support the S3 Object Lock feature can also be used.
As of the publishing of this document, Object Lock is still fairly new, so not all S3 applications will
automatically support Object Lock. Cloudian HyperStore has native S3 compatibility so any AWS S3
application that works with Object Lock, will very likely have seemless integration with HyperStore.
CLOUDIAN, INC.
177 Bovet Road, Suite 450,San Mateo, CA94402
Tel: [phone] | cloudian.com
©2020 Cloudian, Inc. Cloudian, the Cloudian logo, HyperFile, HyperScale, and HyperStore are registered trademarks or trademarks ofCloudian,
Inc. All other trademarks areproperty of their respective holders.
QUICK-OBJLK-0120