Backup to Object Storage

Backup to Object Storage

Zadara VPSA provides built in backup and restore capabilities to AWS S3, Google Cloud Storage, Zadara VPSA Object Storage or any other S3 compatible object storage. The backup process involves transporting VPSA Snapshots to the remote Object Storage for safe keeping.

Connecting to Remote Object Storage

In order to back up your data to Object Storage you need to connect the VPSA to the Object Storage bucket (container). To do this you will need the following information:

  • Bucket/Container name
  • Access key ID
  • Secret access key

Note

  • In order to keep the data backed up ready for restore, the remote Object Storage bucket must not have any life-cycle policy (such as archiving to Glacier) as all backup objects are required for immediate restore.
  • For AWS-S3 the minimal S3 permissions required for the remote Object Storage bucket keys:
    • GetLifecycleConfiguration
    • GetObject
    • PutObject
    • List*
    • DeleteObject

Since public object storage, such as AWS S3, is on a public network and your VPSA is within your private cloud or local network, there are 2 options:

  • Connect via a public IP address (see Assigning Public IPs for assigning a public IP address)
  • Connect via a proxy server in your VPC that has access to the Internet

To connect to Remote Object Storage:

image108

  • Open the VPSA GUI > Remote Object Storage and click the Connect button.
  • Select between AWS S3, Google Cloud Storage, VPSA Object Storage or Custom (S3 Compatible Object Storage).
  • Enter the bucket/container name, access key and secret key.
  • Select the connection method – via public IP, or the local management network.
  • If needed set-up a proxy server and provide the proxy IP address and port, as well as login credentials.

Note

For details about setting up the proxy server see this article: Setup Backup To S3 (B2S3) Through a Proxy In Your AWS VPC

  • In case the target Object Storage type is AWS S3, the following options are available:
    • Region - the target bucket AWS region (mandatory)
    • Ignore Lifecycle Policies - Could be checked in case Lifecycle cannot be disabled on the target bucket. (not recommended)
    • Use KMS Key ID - default KMS managed private key ID to be used for SSE (Storage Service Encryption). (optional)
  • Press Submit

Viewing Remote Object Storage properties

The Remote Object Storages details are shown in the following South Panel tabs:

Properties

Each Remote Object Storage includes the following properties:

Property Description
ID An internally assigned unique ID
Type AWS S3, Google Cloud Storage, VPSA Object Storage or Custom
Endpoint Location (region) of the object storage
Connect Via The network used for the backup data transfer (Public IP or Management Network)
Bucket The name of the S3 bucket used to store the backup data
Proxy IP IP address of the proxy server
Proxy Port Port used for the proxy connection (typically 3128)
KMS Key (AWS S3) The KMS Key ID used for SSE
Allow Lifcycle Policies Whether Lifecycle Policies are ignored for the target Bucket

Backup Jobs Tab - List of all backup jobs using the selected Remote Object Storage

Restore Jobs Tab - List of all restore jobs using the selected Remote Object Storage

Logs Tab - List of event log messages related to that Remote Object Storage

Creating New Backups

In order to create a Backup for a given Volume, you must first have the Remote Object Storage connected as explained here Connecting to Remote Object Storage

To create a Backup:

image109

  • Open the VPSA GUI > Backup to Object Storage and click the Create button.
  • Give the new Backup Job a name
  • Select the Volume to be backed up
  • Select the Remote Object Storage to be used
  • Select a Snapshot Policy. Snapshots created by the selected Policy are stored in the Object Storage bucket

Note

Snapshot Policies used for backup purposes are the same Snapshots used locally within the VPSA.

  • Select the SSE (Storage Service Encryption) - AES256, KMS(Default KMS Key), KMS Key ID(User defined KMS Access ID) (AWS S3 Only)
  • Check the Compress Data box if you want to compress the data in flight. This may save on the traffic fees
  • Press Submit

Monitoring Backups

Remote Object Storage Backups can be managed and monitored from the VPSA GUI.

Open the VPSA GUI > Backup to Object Storage page. It lists all of the jobs that have been configured. From this page you can perform the following actions on each backup job (regardless of the parameters given when the Backup Job was created):

  • Delete the Backup Job
  • Pause / Resume
  • Enable / Disable compression
  • Rate Limit - Limit the backup job bandwidth (MB/s)
  • Change the Snapshot Policy of the Backup Job

image110

The Backup Job details are shown in the following South Panel tabs:

image111

Properties

Each job includes the following properties:

Property Description
ID An internally assigned unique ID.
Name Name that was given at creation time
Comment User free text comment. Can be used for labels, reminders etc...
Status Current job status: Idle / Running
Snapshot Policy The Snapshot Policy used by this job.
RPO Time stamp of the most recent successfully backed up Snapshot.
Compression Compression enabled: Yes / No
Created Creation time stamp.
Modified Last modify time stamp.
Source Volume Name of the protected Volume.
Destination Type Type of the Remote Object Storage.
Account Account on the Remote Object Storage.
End Point Location of the Remote Object Storage.
Bucket Bucket in the Remote Object Storage where the backups are kept.

Local Snapshots

The Local Snapshots tab lists the point-in-time Snapshots of this Volume that were created for backup purposes by the selected job.

The following Properties are provided per Local Snapshot:

Attribute Description
ID Snapshot ID
Name Display Name.
TimeStamp Snapshot creation time stamp
Status Normal/Pending Deletion/Deletion

Object Storage Snapshots

The Object Storage Snapshots tab lists the point-in-time Snapshots of this Volume as stored in the Remote Object Storage. These snapshots were created by the selected job.

The following Properties are provided per Object Storage Snapshot:

Attribute Description
ID Snapshot ID
Name Display Name.
TimeStamp Snapshot creation time stamp.
Status Normal\Pending Deletion\Deleting

Metering - The Metering Charts provide live metering and statistics of the IO workload associated with the selected Backup Job.

The following charts are displayed:

Chart Description
Bandwidth (MB/s) Total throughput (in MB) of backup data transferred to the Remote Object Storage.
IO Time (ms) Average response time IO commands issued by the Backup Job during the selected interval.

Logs – The Logs tab displays a list of event log messages related to that Backup Job.

Restore

In order to restore a Volume from a Snapshot in Remote Object Storage, open the VPSA GUI > Restore from Object Storage page and click Create. In the dialog that opens select the Remote Object Storage, and navigate to the bucket (VPSA / Volume / Snapshot) to restore from. Click Next.

image112

Note

Since listing of large buckets may be time consuming there is an option to specify the full path of the snapshot to restore from (if known). The path should be given in the following format:

<cloud_name.cloud_uid/vpsa_name.vpsa_id/volume_name.volume_id/object_snapshot_name>

image112a

The Restore Job creates a new Volume from the selected Snapshot. Restore supports three modes of operation:

Restore – This mode is useful for creating a full copy of the Volume from the Snapshot, to be used for offline processing. In this mode there is no need to wait for all of the data to be transferred back. The new Volume can be immediately attached to the Host. If the Host needs data that is not yet restored the system will get it on demand.

Clone – This mode is useful for restoring a small amount of data (a few files) without needing to copy the entire Volume capacity from the Object Storage. Again, the new volume can be immediately attached to the host, but data is only transferred on demand.

Import Seed – This mode is useful for restoring data from a given point-in-time, subsequently enable synchronization via Mirroring. In this mode a full capacity Volume is created, but you have to wait until all of the Volume’s capacity is restored before you can use it.

image113

To create a new Restore Job:

  • Give the new Volume a name.
  • Select the restore mode.
  • If you want the new Volume to be encrypted check the Encrypted box.
  • Select a Pool to contain the new Volume.
  • Press Submit.

A Restore job is then generated and begins working according to the selected mode. You may switch between Restore and Clone mode while the job is running by clicking the Switch to… button. This button toggles depending on its current status.