Object Storage Clients - Configuration

Standard client tools can be used to browse objects in VPSA Object Storage. This Appendix will help configuring Object Storage Client Tools to work against VPSA Object Storage. In order to access the VPSA Object Storage the client tool must be configured with the user’s authentication credentials.

The VPSA Object Storage support two API interfaces:

  1. Openstack Swift API

  2. AWS S3 API

The Needed parameters can be found in the Object Storage User Information page. Information about the user currently logged in to the Object Storage GUI is displayed by clicking the user name on the GUI upper right corner.

image86

Openstack Swift Interface

Cloudberry Explorer for OpenStack (v3 authentication)

Use the logged-in User Information properties to set the authentication fields of Cloudberry Explorer

image81

CyberDuck

Cyberduck version: 7.7.1 (33788)

Cyberduck client support “Openstack Swift (Keystone 3)” API interface.

Use the logged-in User Information properties to set the authentication field of CyberDuck client.

  1. Server - the VPSA Object Storage v3 Auth Endpoint.

  2. Port - 5000

  3. Project:Domain:Username - <VPSA Object Storage Account>:default:<VPSA Object Storage Username>

cyber-duck-config


cURL (swift API)

cURL can be used for Object Storage operations. The connectiviy information is available in the User Information view.

In this example, we will use the API Token and Account URL in order to create a new container:

$ curl -H "x-auth-token: <user_token>" -X PUT <account_url>/test-bucket1/
$ URL=https://vsa-00000001-mycloud-01.zadara.com:443/v1/AUTH_123456789
$ TOKEN=<MYAPI TOKEN>
$ curl -H "x-auth-token: $TOKEN" -X PUT $URL/test-bucket1/

Important

By default, the API token is valid for 24 hours. the preferred option to identify/renew the API token via an API call is to use a Swift command and not the VPSA Object Storage command indicated in the Zadara VPSA Object Storage REST API User Guide here: http://zios-api.zadarastorage.com .


The following example describes how to get the token programmatically using the Swift API:

$ curl -i -H "Content-Type: application/json" \
-d '{ "auth": \
{ "identity": { "methods": ["password"], "password": \
{ "user": {"name": "<USERNAME>", "domain": { "id": "default" }, \
"password": "<USER PASSWORD>" }} }, "scope": { "project": \
{ "name": "<ACCOUNT_NAME>", "domain": { "id": "default" } } } } }' \
"https://vsa-00000001-mycloud-01.zadara.com:5000/v3/auth/tokens" ;

and use the returned token for the subsequent API calls.

HTTP/1.1 201 Created
Date: Thu, 19 Nov 2020 16:05:28 GMT
Server: Apache/2.4.29 (Ubuntu)
Content-Length: 1114
X-Subject-Token: gAAAAABftpfIAiuo2tRZZP8VVtomU1knVG7xNUONV4b2u....

Additional examples of using the Openstack Swift API can be found at the Openstack Swift API documentation

AWS S3 Compatible clients

Supported S3 APIs

The VPSA Object Storage is utilizing Openstack Swift’s S3 Middleware. As S3 is an AWS product, It includes some features that are AWS oriented and are outside of the scope of Zadara’s Object Storage offering.

The list of supported S3 operations can be found in the S3/Swift REST API Comparison Matrix.

Zadara have added a specific support for:

Authentication information

For Object Storage connectivity, it is required to gather the following information from the VPSA Object Storage management UI:

  1. VPSA Object Storage Endpoint

  2. VPSA Object Storage region.

  3. S3 API Access Key/Secret Key

In the VPSA Object Storage GUI, navigate to the User Information section (top right corner, by clicking the logged in username).

s3-auth-details

S3 Browser

S3 Browser can be used to administrate and perform object operations against Zadara’s VPSA Object Storage. The account information in S3 Browser should be configured according to the following example (S3 Compatible Storage):

s3-browser-create-account

Once the Endpoint and authentication details are configured properly, click on the Advanced S3-compatible storage settings

In the advanced settings select the following:

  1. Signature version - Signature V4

  2. Addressing model - Path style

  3. Override storage regions - specify the VPSA Object Storage region name; the format is Region Name=<region name>.

Close and save the account information.

Note

S3 Browser client is hard-coded to use us-east-1 as the default region, In order to use Object Storage v4 signatures, ensure the same region value is configured in your VPSA Object Storage or override the default S3Browser region name in the Advanced Settings options.

S3cmd

The credentials can be retrieved from the VPSA Object Storage logged in “User Information” properties.

/etc/.s3cfg

[default]
access_key = <S3 Access Key>
secret_key = <S3 Secret Key>
host_base = vsa-00000001-cloud-01.zadara.com
host_bucket = vsa-00000001-cloud-01.zadara.com
use_https = True

Note

  • access_key is the user S3 Access Key

  • secret_key is the user S3 Secret Key

  • host_base is the HTTPS path to the VPSA Object Storage being accessed

AWS Command Line Interface

Update the default/create new profile for the VPSA Object Storage within aws configuration file.

~/.aws/config

[profile zadara]
s3 =
    signature_version = s3v4

Note

It is possible to use both AWS v4/v2 signatures with S3-compatible storage such as Zadara VPSA Object Storage.

~/.aws/credentials

[zadara]
aws_access_key_id = <S3 Access Key>
aws_secret_access_key = <S3 Secret Key>

The credentials can be retrieved from the VPSA Object Storage logged in “User Information” properties.

Example of usage:

$ aws s3 --profile=zadara --endpoint-url=https://vsa-00000001-cloud-01.zadara.com --region=US ls s3://zadara-test

2018-04-01 19:00 mytestfile1
2018-04-01 19:10 mytestfile2
2018-04-01 19:20 mytestfile3

Note

  • profile is the name of the credentials and config profile specified above (in this case, “zadara”)

  • endpoint-url is the HTTPS path to the VPSA Object Storage being accessed

  • region should match the Region defined in the VPSA Object Storage settings page

boto3 python library

Update the default/create new profile for the VPSA Object Storage within aws configuration file.

~/.aws/config

[profile zadara]
s3 =
    signature_version = s3v4

Note

It is possible to use both AWS v4/v2 signatures with S3-compatible storage such as Zadara VPSA Object Storage.

~/.aws/credentials

[zadara]
aws_access_key_id = <S3 Access Key>
aws_secret_access_key = <S3 Secret Key>

The credentials can be retrieved from the VPSA Object Storage logged in “User Information” properties.

In your python code:

#!/usr/bin/env python

import boto3

session = boto3.session.Session(profile_name='zadara')

s3_client = session.client(
    service_name='s3',
    region_name='US',
    endpoint_url='https://vsa-00000001-cloud-01.zadara.com',
)

print('Buckets')
print(s3_client.list_buckets())

print('')

print('Objects')
print(s3_client.list_objects(Bucket='test'))

Note

  • profile_name is the name of the credentials and config profile specified above (in this case, “zadara”)

  • endpoint_url is the HTTPS path to the VPSA Object Storage being accessed

  • region should match the Region defined in the VPSA Object Storage settings page

AWS S3 Java SDK (aws-java-sdk)

AWS Provides a comprehensive S3 Java SDK that can be used with Zadara’s VPSA Object Storage. Getting started guide is available in Zadara’s Support Knowledge Base article - How to use AWS S3 Java SDK with VPSA Object Storage.

AWS S3 PHP SDK (aws-sdk-php)

AWS Provides a comprehensive S3 PHP SDK that can be used with Zadara’s VPSA Object Storage. Getting started guide is available in Zadara’s Support Knowledge Base article - How to use AWS S3 PHP SDK with VPSA Object Storage.

AWS S3 JavaScript SDK (aws-sdk)

AWS Provides a comprehensive S3 JavaScript SDK that can be used with Zadara’s VPSA Object Storage. Getting started guide is available in Zadara’s Support Knowledge Base article - How to use AWS S3 JavaScript SDK with VPSA Object Storage.