I’ve been recently working with HPE 3PAR arrays for quite some time and thought of put up some basics, I thought it might be a good idea to write a blog related some of the concepts built around the 3PAR architecture and 3PAR beginners and administrators can make use of this. Either way I’m not an expert and still learning myself so ease up on me if I make any mistake.
Layers in 3PAR Storage
Like any array there are multiple layers from host to storage till destination of requested data. 3PAR StoreServ array is comprised of the following logical data layers
- Physical Disks.
- Logical Disks
- Common Provisioning Groups
- Virtual Volumes
A physical disk is a hard drive (spinning media or Solid-State Drive) located in an HPE 3PAR StoreServ drive enclosure or cage. a physical disk located inside of your 3PAR array, encompassing all types of disk within the array
Cage is nothing but a shelf where all physical disks are put together, Cage is legacy term which is also known as Drive Enclosure or Disk Shelf.
Chunklets are automatically created by the HPE 3PAR OS, and they are used to create logical disks. A chunklet is assigned to only one logical disk. Physical disks are divided into chunklets. Each chunklet occupies physically contiguous space on an FC, NL or SSD disk. Each chunklet is 1GB in size and occupies contiguous space on a physical disk. Chunklets are local to that physical disk only and cannot span to others.
Logical disks are essentially a grouping of chunklets which are arranged as rows of like RAID sets. LD’s will ensure that each chunklet which resides in a RAID set is physically located on different physical disks. We don’t directly create LD’s on the 3PAR – they are generated during the creation of a CPG.
CPGs (Common Provisioning Groups)
A CPG is simply a pool of Logical
Disks that provide the means for a Virtual Volume (explained next) to consume
space. A CPG allows up to 65,536 virtual volumes to share a CPG’s assigned
resources. You can create fully provisioned virtual volumes (FPVVs), Thinly
Deduped Virtual Volumes (TDVVs) and
Compressed VVs that draw space from a CPG’s logical disks. When we deploy a CPG, we do not actually use any of the space in our pooled logical disks until a virtual volume is created.
VVs (Virtual volumes)
VVs (Virtual volumes) draw their resources from the LDs in CPGs and are exported as LUNs (Logical Unit Numbers) to hosts. Virtual volumes are the only data layer visible to the hosts. Just like most arrays Virtual Volumes can be provisioned either thick or thin – with a thin provisioned Virtual Volume only instructing its associated CPG to draw space from the logical disks as space is needed. CPGs can create logical disks as needed to handle the increased demand for capacity up until the user-defined size limit of the CPG is reached.
So, working backwards we can come to somewhat of the following
- A LUN / datastore is located on a Virtual Volume.
- A Virtual Volume draws its’ space from a Common Provisioning Group (CPG).
- A Common Provisioning Group is any given number of Logical Disks joined together to form some sort of contiguous space.
- A Logical Disk is simply a collection of chunklets which are joined together in rows in order to produce a certain RAID set (1,5,6, etc).
- A Chunklet is a 1GB piece (chunk) of any given physical disk within the array.
- A physical disk is…well, a physical disk.
Now we will discuss more on VVs (Virtual volumes)
To access a LUN from a host we need to export the volume in VVs. Volumes are exported by creating VV-LUN pairings (VLUNs) on the system. When you create VLUNs, the system produces both VLUN templates that establish export rules, and active VLUNs that the host sees as LUNs as attached disk devices. A VLUN will be created for each path available to the host for each VV exported.
FPVVs (Fully provisioned virtual volumes)
A FPVV virtual volume is a volume that uses logical disks that belong to a CPG. Unlike TPVVs or TDVVs, fully provisioned virtual volumes have a set amount of user space that is allocated for user data. The fully provisioned volume size is allocated and consumed at the time of provisioning, size limits range from 256 MB to 64 TB (Compressed and Dedupe VVs have a maximum size of 16 TB). The volume size can be increased at any time (provided free space is available) up to the maximum 64 TB size without any downtime however, the VV size cannot be decreased below the initial allocation.
TPVVs (Thinly provisioned virtual volumes)
A TPVV is a volume that uses
logical disks that belong to a CPG. TPVVs, TDVVs, or DECO VVs (deduped and
compressed) associated with the same CPG draw space from those CPGs’ LDs
as needed, allocating space on demand in 16 KiB increments for each VV. As the
volumes that draw
space from the CPGs’ LDs (including compressed and deduped) require additional storage the HPE 3PAR OS automatically creates additional logical disks or expands the size of existing LDs and adds them to the CPG until the CPG reaches a user-defined growth limit, if one has been set,
or the system runs out of space.
TDVVs (Thinly deduped provisioned virtual volumes)
In addition to the features and functionality
of TPVVs; TDVVs go through an additional process before allocating space on a
disk. All data writes with a block size 16 KiB or greater have a unique
hash generated and the resulting value is compared to a hash lookup table to
determine if the
data has already been stored. If the data is a duplicate of existing data, an entry is added to the destination volume’s lookup table, but no data is written. Otherwise it is written to disk. For more information on this, refer to the HPE 3PAR Thin Technologies white paper located here.
Clones / Physical Copies
A clone duplicates all the data from a base volume to a destination volume. The base volume is the original volume that is copied to the destination volume. The clone on the destination volume remains available if the original base volume becomes unavailable. A clone requires the destination volume have usable capacity equal to or greater than the usable capacity of the base volume being cloned. As of HPE 3PAR OS 3.2.1 the clone can be exported immediately after creation, while the data copy continues in the background.
Snapshots / Virtual Copy
Unlike a clone, which is a block for block duplicate of an entire volume, snapshots preserve a bitmap of a VV at a point in time. Updates to VVs are written to SD (Snap Data) space and the bitmap (Snap Admin space) of the VV. Snapshots for FPVVs, TPVVs, dedupe compressed and DECO clones and other snapshots are created using copy-on-write techniques available(aka Virtual Copy) snapshots for TDVVs are created using ROW (Redirect On Write). Hundreds of snapshots of each virtual volume can be created assuming that there is enough storage space available. It is worth noting that snapshots do not consume any space unless data on the base volume has been updated and the original data copied to the SD (Snap Data) space. Changed data is copied only once regardless of the number of snapshots taken. Snapshots are particularly useful for test/dev environments as they can be created in seconds and exported while not effecting production data.
File Provisioning Group
A File Provisioning Group (FPG) is an instance of the HPE intellectual property Adaptive File System. It controls how files are stored and retrieved. Each FPG is transparently constructed from one or multiple Virtual Volumes (VVs) and is the unit for replication and disaster recovery for File Persona Software. There are up to 16 FPGs supported on a node pair.
Virtual File Server
A Virtual File Server (VFS) is conceptually like a server; as such it presents virtual IP addresses to clients, participates in User Authentication Services and can have properties for such characteristics as user/group Quota Management and Antivirus policies.
File Stores are the slice of a VFS and FPG where snapshots are taken, capacity Quota Management can be performed, and Antivirus Scan Services policies customized.
File Shares are what provide data access to clients via SMB, NFS, and the Object Access API, subject to the share permissions applied to them.
If you have any comments, please drop me a line.
I hope this article was informative, and don’t forget to buy me a coffee if you found this worth reading.