Backup to Object Storage

Zadara VPSA provides built in backup and restore capabilities to AWS S3, Google Cloud Storage, Zadara VPSA Object Storage or any other S3 compatible object storage. The backup process involves transporting VPSA Snapshots to the remote Object Storage for safe keeping.

Connecting to Remote Object Storage

In order to back up your data to Object Storage you need to connect the VPSA to the Object Storage bucket (container). To do this you will need the following information:

  • Bucket/Container name

  • Access key ID

  • Secret access key

Note

  • In order to keep the data backed up ready for restore, the remote Object Storage bucket must not have any life-cycle policy (such as archiving to Glacier) as all backup objects are required for immediate restore.

  • For AWS-S3 the minimal S3 permissions required for the remote Object Storage bucket keys:

    • GetLifecycleConfiguration

    • GetObject

    • PutObject

    • List*

    • DeleteObject

Since public object storage, such as AWS S3, is on a public network and your VPSA is within your private cloud or local network, there are 2 options:

  • Connect via a public IP address (see Assigning Public IPs for assigning a public IP address)

  • Connect via a proxy server in your VPC that has access to the Internet

To connect to Remote Object Storage:

image108

  • Open the VPSA GUI > Remote Object Storage and click the Connect button.

  • Select between AWS S3, Google Cloud Storage, VPSA Object Storage or Custom (S3 Compatible Object Storage).

  • Enter the bucket/container name, access key and secret key.

  • Select the connection method – via public IP, or the local management network.

  • If needed set-up a proxy server and provide the proxy IP address and port, as well as login credentials.

Note

For details about setting up the proxy server see this article: Setup Backup To S3 (B2S3) Through a Proxy In Your AWS VPC

  • In case the target Object Storage type is AWS S3, the following options are available:

    • Region - the target bucket AWS region (mandatory)

    • Ignore Lifecycle Policies - Could be checked in case Lifecycle cannot be disabled on the target bucket. (not recommended)

    • Use KMS Key ID - default KMS managed private key ID to be used for SSE (Server-Side Encryption). (optional)

  • Press Submit

Viewing Remote Object Storage properties

The Remote Object Storages details are shown in the following South Panel tabs:

Properties

Each Remote Object Storage includes the following properties:

Property

Description

ID

An internally assigned unique ID

Type

AWS S3, Google Cloud Storage, VPSA Object Storage or Custom

Endpoint

Location (region) of the object storage

Connect Via

The network used for the backup data transfer (Public IP or Management Network)

Bucket

The name of the S3 bucket used to store the backup data

Proxy IP

IP address of the proxy server

Proxy Port

Port used for the proxy connection (typically 3128)

KMS Key

(AWS S3) The KMS Key ID used for SSE

Allow Lifcycle Policies

Whether Lifecycle Policies are ignored for the target Bucket

Backup Jobs Tab - List of all backup jobs using the selected Remote Object Storage

Restore Jobs Tab - List of all restore jobs using the selected Remote Object Storage

Logs Tab - List of event log messages related to that Remote Object Storage

Creating New Backups

In order to create a Backup for a given Volume, you must first have the Remote Object Storage connected as explained here Connecting to Remote Object Storage

To create a Backup:

image109

  • Open the VPSA GUI > Backup to Object Storage and click the Create button.

  • Give the new Backup Job a name

  • Select the Volume to be backed up

  • Select the Remote Object Storage to be used

  • Select a Snapshot Policy. Snapshots created by the selected Policy are stored in the Object Storage bucket

Note

Snapshot Policies used for backup purposes are the same Snapshots used locally within the VPSA.

  • (AWS S3 Only) Select the SSE (Server-Side Encryption) - AES256, KMS(Default KMS Key), KMS Key ID(User defined KMS Access ID) (AWS S3 Only)

  • (AWS S3 Only) Select Storage Class for backup data placement. Besides S3 standard storage class Backups can be also sent to S3 Intelligent Tiering or S3 Infrequent Access storage class.

    Note

    S3 Storage classes can optimize overall S3 costs for specific data types and retention policies. Please consult AWS documentation and consider your backup retention policy before selecting a storage class.

  • Check the Compress Data box if you want to compress the data in flight. This may save on the traffic fees

  • Press Submit

Monitoring Backups

Remote Object Storage Backups can be managed and monitored from the VPSA GUI.

Open the VPSA GUI > Backup to Object Storage page. It lists all of the jobs that have been configured. From this page you can perform the following actions on each backup job (regardless of the parameters given when the Backup Job was created):

  • Delete the Backup Job

  • Pause / Resume

  • Enable / Disable compression

  • Rate Limit - Limit the backup job bandwidth (MB/s)

  • Change the Snapshot Policy of the Backup Job

  • Add a comment to a backup job

  • Change a backup job target S3 storage class (AWS S3 Only)

Note

If target S3 Storage class settings is modified for a specific backup job the new class will be applied on backups taken after this changed was performed. Previously created backups copies will not be modified.

image110

The Backup Job details are shown in the following South Panel tabs:

image111

Properties

Each job includes the following properties:

Property

Description

ID

An internally assigned unique ID.

Name

Name that was given at creation time

Comment

User free text comment. Can be used for labels, reminders etc…

Status

Current job status: Idle / Running

SSE

(AWS S3 Only) Server side encryption type

Storage Class

(AWS S3 Only) S3 target storage class for backup copies

KMS Key ID

(AWS S3 Only) AWS KMS key ID (for SSC with KMS Key ID)

Snapshot Policy

The Snapshot Policy used by this job.

RPO

Time stamp of the most recent successfully backed up Snapshot.

Compression

Compression enabled: Yes / No

Created

Creation time stamp.

Modified

Last modify time stamp.

Source Volume

Name of the protected Volume.

Destination Type

Type of the Remote Object Storage.

Account

Account on the Remote Object Storage.

End Point

Location of the Remote Object Storage.

Bucket

Bucket in the Remote Object Storage where the backups are kept.

Local Snapshots

The Local Snapshots tab lists the point-in-time Snapshots of this Volume that were created for backup purposes by the selected job.

The following Properties are provided per Local Snapshot:

Attribute

Description

ID

Snapshot ID

Name

Display Name.

TimeStamp

Snapshot creation time stamp

Status

Normal/Pending Deletion/Deletion

Object Storage Snapshots

The Object Storage Snapshots tab lists the point-in-time Snapshots of this Volume as stored in the Remote Object Storage. These snapshots were created by the selected job.

The following Properties are provided per Object Storage Snapshot:

Attribute

Description

ID

Snapshot ID

Name

Display Name.

TimeStamp

Snapshot creation time stamp.

Status

Normal\Pending Deletion\Deleting

Metering - The Metering Charts provide live metering and statistics of the IO workload associated with the selected Backup Job.

The following charts are displayed:

Chart

Description

Bandwidth (MB/s)

Total throughput (in MB) of backup data transferred to the Remote Object Storage.

IO Time (ms)

Average response time IO commands issued by the Backup Job during the selected interval.

Logs – The Logs tab displays a list of event log messages related to that Backup Job.

Restore

In order to restore a Volume from a Snapshot in Remote Object Storage, open the VPSA GUI > Restore from Object Storage page and click Create. In the dialog that opens select the Remote Object Storage, and navigate to the bucket (VPSA / Volume / Snapshot) to restore from. Click Next.

image112

Note

Since listing of large buckets may be time consuming there is an option to specify the full path of the snapshot to restore from (if known). The path should be given in the following format:

<cloud_name.cloud_uid/vpsa_name.vpsa_id/volume_name.volume_id/object_snapshot_name>

image112a

The Restore Job creates a new Volume from the selected Snapshot. Restore supports three modes of operation:

Restore – This mode is useful for creating a full copy of the Volume from the Snapshot, to be used for offline processing. In this mode there is no need to wait for all of the data to be transferred back. The new Volume can be immediately attached to the Host. If the Host needs data that is not yet restored the system will get it on demand.

Clone – This mode is useful for restoring a small amount of data (a few files) without needing to copy the entire Volume capacity from the Object Storage. Again, the new volume can be immediately attached to the host, but data is only transferred on demand.

Import Seed – This mode is useful for restoring data from a given point-in-time, subsequently enable synchronization via Mirroring. In this mode a full capacity Volume is created, but you have to wait until all of the Volume’s capacity is restored before you can use it.

image113

To create a new Restore Job:

  • Give the new Volume a name.

  • Select the restore mode.

  • If you want the new Volume to be encrypted check the Encrypted box.

  • Select a Pool to contain the new Volume.

  • Press Submit.

A Restore job is then generated and begins working according to the selected mode. You may switch between Restore and Clone mode while the job is running by clicking the Switch to… button. This button toggles depending on its current status.