Configuring Storage Pools

Configuring Storage Pools

Understanding Storage Pools

Storage Pools are virtual entities that manage storage provisioning from the aggregated capacity of one or more RAID Groups pooled into a single construct with some QoS attributes.

Volumes are thinly provisioned, allocating capacity from the Pool only when needed. The Pool has an underlying block virtualization layer which maps virtual address space to physically allocated Pool space and manages sharing of Pool physical chunks between Volumes, Snapshots and Clones.

Snapshots and Clones consume zero capacity when they are created because they share the same data chunks as the originating Volume. Anytime you actually modify the data in the Volume, or in one of the Clones, the data chunk is copied-on-write (COW) from the source in order to apply the new data write to a new pool region without affecting the data set of any other objects that share the same data chunk.

The Pool’s attributes define the way Volumes, Snapshots and Clones are provisioned.

Creating and Managing Pools

Creating a Pool

To create a new Storage Pool press either the Create button on the Pools page or the Create Pool button on the RAID Groups page. You will see the following dialog appear:

image20

Select the Pool attributes:

  • Display Name – You can modify this anytime later.
  • Raid Group(s) selection – Check the box(es) of one or more RAID Groups from which protected storage capacity will be allocated for this Pool.
  • Capacity – The Pool’s physical capacity shown in GB. By default the capacity is the aggregated capacities of all the selected RAID Groups, but you do not have to allocate full RAID Groups. If you define a capacity smaller than is available in the selected RAID groups the capacity will be evenly distributed between the RAID Groups.

Note

The actual usable capacity of the Pools is a little less than the requested size, as the system reserves some space for the Pool’s metadata (typically up to 100GB).

  • Type – The VPSA supports Transactional, Repository and Archive Pool types. These Pool types use different chunk sizes for the mapping of virtual LBAs to Physical Drive addresses. The following table describes the tradeoffs for each type and the recommended use cases:
  Transactional Pool Repository Pool Archive Pool
Chunk size 256KB 1MB 2MB
Pros
  • Faster COW operation
  • Space efficiency on Random writes to Snapshots
  • Smaller metadata size
  • Sequential workload performance is similar to transactional
  • Allows large pools
  • Sequential workload performance is the same
Cons Increased metadata size
  • Slower COW operation
  • Less space efficient
  • Slower with frequent data modifications
  • Limited Snapshots frequncy (1 hour min)
Use Case Transactional Workload with Snapshots
  • Repository type workload.
  • Large Pools
  • Many snapshots to keep
  • Relatively static data
  • Archive type workloads
  • very large pools/volume (< 100TB)
Limit Transactional Pools have a maximum size of 20TB
  • Repository Pools have a maximum size of 100TB
  • Archive Pools have a maximum size of 100TB
  • Cached – Check this box to use SSD to Cache Server’s reads and writes.
    • All Pools that are marked as “Cached” share the VPSA Cache.
    • Flash cache usually improves the performance of volumes based on HDD’s pools. However it depends on the specific workload and the size of the cache vs. the size of the active data set.
    • If the Pool consists of SSD drives this option will be disabled.
  • Striped – This check box is enabled only when you select two or more RAID Groups. Striping over RAID-1, RAID-5 and RAID-6 creates RAID-10, RAID-50 and RAID-60 configurations respectively. Use striping to improve performance of random workloads since the IOs will be distributed and all drives will share the workload.

Expanding Pool Capacity

To Expand the Pool press the Expand button on the Pools page.

image21

You can use capacity from any RAID Group to expand a Pool. If the RAID Group from which the new capacity is added doesn’t match the protection type or drive type of the existing capacity you’ll see a warning message pop up asking you to confirm the mismatch. Keep in mind that continuing with the mismatched types may impact the pool performance and protection QoS.

It is possible to enable Caching on non-cached Pools.

One use case for leveraging this capability is to enable caching only after the initial copy of the data into the VPSA. The initial copy typically generates a sequential write IO workload, where non-cached Pools are most efficient. Once the initial copy is completed enable caching on the Pool if you expect a more random type of IO workload.

Disabling SSD cache on a pool

image22

By default every pool is cached by the VPSA’s SSD cache, but it is also possible to disable caching on cached Pools which will remove this feature. The Enable Cache/Disable Cache buttons toggle depending on the current caching state of the Pool.

Viewing Pool properties

The Pools details are shown in the following South Panel tabs:

Properties

Each Pool includes the following properties:

Property Description
ID An internally assigned unique ID.
Name User assigned name. Can be modified anytime.
Comment User free text comment. Can be used for labels, reminders or any other purpose
Status
  • Normal
  • Creating
  • Deleting
  • Partial/Failed – At least one of the underlying RAID groups has failed, or the Pool metadata cannot be initialized at Start Of the Day.
Capacity Total available capacity for user data & system metadata.
Available Capacity Available (free) capacity to be used for User data. VPSA reserves 2% of the total Pool capacity for system metadata. If the VPSA needs more capacity for the metadata (very rare scenario), it will be consumed from the available capacity.
Metadata Capacity Metadata Capacity
Free Capacity State
  • Normal
  • Alert
  • Protected
  • Emergency

See Managing Pool Capacity Alerts for more details.

Mode
  • Simple – There are one or more concatenated RAID Groups.
  • Stripe – There are two or more striped RAID Groups.
  • Mixed – There are two or more concatenated and striped RAID Groups.
Type
  • Transactional Workloads
  • Repository Storage
  • Archival Storage
Stripe Size Applicable only for Pools of Striped mode (i.e. when data is striped between 2 or more RAID groups). The Stripe size is always 64KB.
Cached Yes/No – Indicates whether the Pool utilizes SSD for read/write caching
Cache COW Writes Yes/No – Indicates whether flash cache is used for internal snapshots Copy-On-Write Operations.Enabled by default. Disable only on rare cases where frequent snapshots cause extreme load of metadata operations. Consult Zadara support.
Raid Group(s) RAID Group name, or “Multiple (X)” where X denotes the number of RAID Groups in the Pool.
Created Date & time when the object was created.
Modified Date & time when the object was last modified.

RAID Groups

In the RAID Groups View This tab lists the RAID Groups allocated to the selected Pool. Each RAID Group includes the following information:

  • Name
  • Protection (RAID-1, RAID-5or RAID-6)
  • Status
  • Contributed Capacity

In the Segments View This tab shows the structure of pool made of concatinated or striped segments

image136

Volumes and Dest Volumes

These two tabs display the provisioned Volumes and the Provisioned Remote Mirroring Destination Volumes. Please note that the Dest Volumes are not displayed in the main Volumes page since most operations are not applicable to them. Displaying the list of the Dest Volumes in the Pools South Panel provides a complete picture of the Objects that consume capacity from the Pool. Each Volume includes the following information:

  • Name
  • Capacity (virtual, not provisioned)
  • Status
  • Data Type (Block or File-System)

Recycle Bin

By default when you delete a volume it moves to a Pool’s Recycle Bin for 7 days until it is permanently deleted. From the Recycle Bin an administrator can purge (permanently delete) or restore a volume.

Logs

Displays all event logs associated with this Pool.

Metering

The Metering Charts provide live metering of the IO workload associated with the selected Pool.

The charts display the metering data as it was captured in the past 20 “intervals”. An interval length can be set to one of the following: 1 Second, 10 seconds, 1 Minute, 10 Minutes, or 1 Hour. The Auto button lets you see continuously-updating live metering info (refreshed every 3 seconds).

Pool Metering includes the following charts:

Chart Description
IOPs The number of read and write SCSI commands issued to the Pool, per second.
Bandwidth (MB/s) Total throughput (in MB) of read and write SCSI commands issued to the Pool, per second.
IO Time (ms) Average response time of all read and write SCSI commands issued to the Pool, per selected interval .

Capacity Alerts

The Capacity Alerts tab lists the configurable attributes of the Pool Protection Mechanism. See Managing Pool Capacity Alerts for more details. You can modify the following attributes:

  • Alert Mode Threshold - “Alert me when it is estimated that the Pool will be at full capacity in X Minutes.”
    • Default Value: 360 minutes
  • Protection Mode Threshold - “Do not allow new Volumes, Shares, or Snapshots to be created when it is estimated that the Pool will be at full capacity in X Minutes.”
    • Default Value: 60 minutes
  • Calculation Window - “Calculate the estimated time until the Pool is full based on new capacity usage in the previous X minutes.”
    • Default Value: 60 minutes
  • Emergency Mode Threshold - “Delete snapshots, starting from the oldest, when there is less than the following capacity left in the Pool”
    • Default Value: 5 GB

Performance Alerts

The Performance Alerts tab lists the Pool’s ability to send alerts when performance drops below expectations. See Managing Pool Performance Alerts for more details.

Managing Pool Capacity Alerts

The VPSA’s efficient and sophisticated storage provisioning infrastructure maximizes storage utilization, while providing key enterprise-grade data management functions. As a result, you can quite easily over-provision a Pool with Volumes, Snapshots and Clones, hence requiring a Pool Protection Mechanism to alert and protect when free Pool space is low.

The VPSA Pool Protection Mechanism is mostly time-based. The goal is to provide you sufficient time to fix the low free space situation by either deleting unused Volumes/Snapshots/Clones or by expanding the Pool’s available capacity (a very simple and quick process due to the elasticity of the VPSA and the Zadara Storage Cloud).

The VPSA measures the rate at which the Pool’s free space is consumed and calculates the estimated time left before running out of free space.

The following user-configurable parameters impact alerts and operations that are performed as part of the Pool Protection Mechanism:

  • Alert Threshold – The estimated time (in minutes) before running out of free space. When triggered an online support ticket is submitted and an email is sent to the VPSA user. When crossing this threshold the Free Capacity State changes to “Alert” and the available capacity will be shown in Yellow. A secondary “reminder” ticket and an email will be generated when only half of this threshold’s estimated time is left.
    • Default: 360 minutes (6 hours)
    • Minimum: 1 minute (0 means disable this alert)
  • Protection Threshold – The estimated time (in minutes) before running out of free space. When triggered the VPSA starts blocking the creation of new Volumes, Snapshots and Clones in that Pool. A support ticket and email are also generated. When crossing this threshold, the Free Capacity State changes to “Protect” and the available capacity will be shown in Red.
    • Default: 60 minutes (1 hour)
    • Minimum: 1 minute (0 means disable this alert)
  • Alert Interval - The size of the window (in minutes) that is used to calculate the rate at which free space is consumed. The smaller the window is the more this rate is impacted by intermediate changes in capacity allocations, which can result from changes in workload characteristics and/or the creation/deletion of new Snapshots and Clones.
    • Default: 60 minutes (1 hours)
    • Minimum: 1 minute
  • Emergency Threshold – When the Pool’s free capacity drops below this fixed threshold (in GB), the VPSA starts freeing Pool capacity by deleting older snapshots. The VPSA will delete one snapshot at a time, starting with the oldest snapshot, until it exceeds the Emergency threshold (i.e. when free capacity is greater than the threshold). A support ticket and email are also generated. When this threshold is crossed the Free Capacity State changes to “Emergency” and the available capacity will be shown in Red.
    • Default: 5 GB
    • Minimum: 1 GB

Managing Pool Performance Alerts

A VPSA administrator has the option to set Pool Performance Alerts in addition to the default Pool Capacity Alerts. Performance Alerts are available for:

Read IOPS Limit – Creates an alert when the average read IOPS, during the past minute, for a Pool exceeds a user-specified threshold.

Read Throughput Limit - Creates an alert when, during the past minute, the average read MB/s for a Pool exceeds a user-specified threshold.

Read Latency Limit – Creates an alert when, during the past minute, the average read latency for a Pool exceeds a user-specified threshold.

Write IOPS Limit – Creates an alert when, during the past minute, the average write IOPS for a Pool exceeds a user-specified threshold.

Write Throughput Limit - Creates an alert when, during the past minute, the average write MB/s for a Pool exceeds a user-specified threshold.

Write Latency Limit – Creates an alert when, during the past minute, the average write latency for a Pool exceeds a user-specified threshold.