Backup to Object Storage¶
Zadara VPSA provides built in backup and restore capabilities to AWS S3, Google Cloud Storage, Zadara VPSA Object Storage or any other S3 compatible object storage. The backup process involves transporting VPSA Snapshots to the remote Object Storage for safe keeping.
Connecting to Remote Object Storage¶
In order to back up your data to Object Storage you need to connect the VPSA to the Object Storage bucket (container). To do this you will need the following information:
Bucket/Container name
Access key ID
Secret access key
Note
In order to keep the data backed up ready for restore, the remote Object Storage bucket must not have any life-cycle policy (such as archiving to Glacier) as all backup objects are required for immediate restore.
For AWS-S3 the minimal S3 permissions required for the remote Object Storage bucket keys:
GetLifecycleConfiguration
GetObject
PutObject
List*
DeleteObject
Since public object storage, such as AWS S3, is on a public network and your VPSA is within your private cloud or local network, there are 2 options:
Connect via a public IP address (see Assigning Public IPs for assigning a public IP address)
Connect via a proxy server in your VPC that has access to the Internet
To connect to Remote Object Storage:
Open the and click the Connect button.
Select between AWS S3, Google Cloud Storage, VPSA Object Storage or Custom (S3 Compatible Object Storage).
Enter the bucket/container name, access key and secret key.
Select the connection method – via public IP, or the local management network.
If needed set-up a proxy server and provide the proxy IP address and port, as well as login credentials.
Note
For details about setting up the proxy server see this article: Setup Backup To S3 (B2S3) Through a Proxy In Your AWS VPC
In case the target Object Storage type is AWS S3, the following options are available:
Region - the target bucket AWS region (mandatory)
Ignore Lifecycle Policies - Could be checked in case Lifecycle cannot be disabled on the target bucket. (not recommended)
Use KMS Key ID - default KMS managed private key ID to be used for SSE (Server-Side Encryption). (optional)
Press Submit
Viewing Remote Object Storage properties¶
The Remote Object Storages details are shown in the following South Panel tabs:
Properties
Each Remote Object Storage includes the following properties:
Property |
Description |
|---|---|
ID |
An internally assigned unique ID |
Type |
AWS S3, Google Cloud Storage, VPSA Object Storage or Custom |
Endpoint |
Location (region) of the object storage |
Connect Via |
The network used for the backup data transfer (Public IP or Management Network) |
Bucket |
The name of the S3 bucket used to store the backup data |
Proxy IP |
IP address of the proxy server |
Proxy Port |
Port used for the proxy connection (typically 3128) |
KMS Key |
(AWS S3) The KMS Key ID used for SSE |
Allow Lifcycle Policies |
Whether Lifecycle Policies are ignored for the target Bucket |
Backup Jobs Tab - List of all backup jobs using the selected Remote Object Storage
Restore Jobs Tab - List of all restore jobs using the selected Remote Object Storage
Logs Tab - List of event log messages related to that Remote Object Storage
Creating New Backups¶
In order to create a Backup for a given Volume, you must first have the Remote Object Storage connected as explained here Connecting to Remote Object Storage
To create a Backup:
Open the and click the Create button.
Give the new Backup Job a name
Select the Volume to be backed up
Select the Remote Object Storage to be used
Select a Snapshot Policy. Snapshots created by the selected Policy are stored in the Object Storage bucket
Note
Snapshot Policies used for backup purposes are the same Snapshots used locally within the VPSA.
(AWS S3 Only) Select the SSE (Server-Side Encryption) - AES256, KMS(Default KMS Key), KMS Key ID(User defined KMS Access ID) (AWS S3 Only)
(AWS S3 Only) Select Storage Class for backup data placement. Besides S3 standard storage class Backups can be also sent to S3 Intelligent Tiering or S3 Infrequent Access storage class.
Note
S3 Storage classes can optimize overall S3 costs for specific data types and retention policies. Please consult AWS documentation and consider your backup retention policy before selecting a storage class.
Check the Compress Data box if you want to compress the data in flight. This may save on the traffic fees
Press Submit
Monitoring Backups¶
Remote Object Storage Backups can be managed and monitored from the VPSA GUI.
Open the page. It lists all of the jobs that have been configured. From this page you can perform the following actions on each backup job (regardless of the parameters given when the Backup Job was created):
Delete the Backup Job
Pause / Resume
Enable / Disable compression
Rate Limit - Limit the backup job bandwidth (MB/s)
Change the Snapshot Policy of the Backup Job
Add a comment to a backup job
Change a backup job target S3 storage class (AWS S3 Only)
Note
If target S3 Storage class settings is modified for a specific backup job the new class will be applied on backups taken after this changed was performed. Previously created backups copies will not be modified.
The Backup Job details are shown in the following South Panel tabs:
Properties
Each job includes the following properties:
Property |
Description |
|---|---|
ID |
An internally assigned unique ID. |
Name |
Name that was given at creation time |
Comment |
User free text comment. Can be used for labels, reminders etc… |
Status |
Current job status: Idle / Running |
SSE |
(AWS S3 Only) Server side encryption type |
Storage Class |
(AWS S3 Only) S3 target storage class for backup copies |
KMS Key ID |
(AWS S3 Only) AWS KMS key ID (for SSC with KMS Key ID) |
Snapshot Policy |
The Snapshot Policy used by this job. |
RPO |
Time stamp of the most recent successfully backed up Snapshot. |
Compression |
Compression enabled: Yes / No |
Created |
Creation time stamp. |
Modified |
Last modify time stamp. |
Source Volume |
Name of the protected Volume. |
Destination Type |
Type of the Remote Object Storage. |
Account |
Account on the Remote Object Storage. |
End Point |
Location of the Remote Object Storage. |
Bucket |
Bucket in the Remote Object Storage where the backups are kept. |
Local Snapshots
The Local Snapshots tab lists the point-in-time Snapshots of this Volume that were created for backup purposes by the selected job.
The following Properties are provided per Local Snapshot:
Attribute |
Description |
|---|---|
ID |
Snapshot ID |
Name |
Display Name. |
TimeStamp |
Snapshot creation time stamp |
Status |
Normal/Pending Deletion/Deletion |
Object Storage Snapshots
The Object Storage Snapshots tab lists the point-in-time Snapshots of this Volume as stored in the Remote Object Storage. These snapshots were created by the selected job.
The following Properties are provided per Object Storage Snapshot:
Attribute |
Description |
|---|---|
ID |
Snapshot ID |
Name |
Display Name. |
TimeStamp |
Snapshot creation time stamp. |
Status |
Normal\Pending Deletion\Deleting |
Metering - The Metering Charts provide live metering and statistics of the IO workload associated with the selected Backup Job.
The following charts are displayed:
Chart |
Description |
|---|---|
Bandwidth (MB/s) |
Total throughput (in MB) of backup data transferred to the Remote Object Storage. |
IO Time (ms) |
Average response time IO commands issued by the Backup Job during the selected interval. |
Logs – The Logs tab displays a list of event log messages related to that Backup Job.
Restore¶
In order to restore a Volume from a Snapshot in Remote Object Storage, open the page and click Create. In the dialog that opens select the Remote Object Storage, and navigate to the bucket (VPSA / Volume / Snapshot) to restore from. Click Next.
Note
Since listing of large buckets may be time consuming there is an option to specify the full path of the snapshot to restore from (if known). The path should be given in the following format:
<cloud_name.cloud_uid/vpsa_name.vpsa_id/volume_name.volume_id/object_snapshot_name>
The Restore Job creates a new Volume from the selected Snapshot. Restore supports three modes of operation:
Restore – This mode is useful for creating a full copy of the Volume from the Snapshot, to be used for offline processing. In this mode there is no need to wait for all of the data to be transferred back. The new Volume can be immediately attached to the Host. If the Host needs data that is not yet restored the system will get it on demand.
Clone – This mode is useful for restoring a small amount of data (a few files) without needing to copy the entire Volume capacity from the Object Storage. Again, the new volume can be immediately attached to the host, but data is only transferred on demand.
Import Seed – This mode is useful for restoring data from a given point-in-time, subsequently enable synchronization via Mirroring. In this mode a full capacity Volume is created, but you have to wait until all of the Volume’s capacity is restored before you can use it.
To create a new Restore Job:
Give the new Volume a name.
Select the restore mode.
If you want the new Volume to be encrypted check the Encrypted box.
Select a Pool to contain the new Volume.
Press Submit.
A Restore job is then generated and begins working according to the selected mode. You may switch between Restore and Clone mode while the job is running by clicking the Switch to… button. This button toggles depending on its current status.