Chapter 7. Developing Storage Strategies
One of the more challenging aspects of setting up Ceph Storage clusters and Ceph Object Gateways for production use is defining effective storage strategies. Storage strategies involve the following factors:
See the Red Hat Ceph Storage 4 Storage Strategies guide for general guidance on storage strategies and command-line usage.
7.1. Developing CRUSH Hierarchies
When deploying a Ceph cluster and an Object Gateway, typically the object gateway will have a default zone group and zone. The Ceph storage cluster will have default pools, which in turn will use a CRUSH map with a default CRUSH hierarchy and a default CRUSH rule.
The default rbd
pool may use the default CRUSH rule. DO NOT delete the default rule or hierarchy if Ceph clients have used them to store client data.
For general details on CRUSH hierarchies, see the CRUSH Administration section of the Storage Strategies guide.
Production gateways typically use a custom realm, zone group and zone named according to the use and geographic location of the gateways. Additionally, the Ceph cluster will have a CRUSH map that has multiple CRUSH hierarchies.
-
Service Pools: At least one CRUSH hierarchy will be for service pools and potentially for data. The service pools include
.rgw.root
and the service pools associated with the zone. Service pools typically fall under a single CRUSH hierarchy, and use replication for data durability. A data pool may also use the CRUSH hierarchy, but the pool will usually be configured with erasure coding for data durability. - Index: At least one CRUSH hierarchy SHOULD be for the index pool, where the CRUSH hierarchy maps to high performance media such as SSD or NVMe drives. Bucket indices can be a performance bottleneck. It is strongly recommended to use SSD or NVMe drives in this CRUSH hierarchy. To economize, create partitions for indices on SSDs or NVMe drives used for OSD journals. Additionally, an index should be configured with bucket sharding. See Creating an Index Pool and supporting links for details.
- Placement Pools: The placement pools for each placement target include the bucket index, the data bucket and the bucket extras. These pools may fall under separate CRUSH hierarchies. Since Ceph Object Gateway can support multiple storage policies, the bucket pools of the storage policies may be associated with different CRUSH hierarchies, reflecting different use cases such as IOPS-optimized, throughput-optimized, and capacity-optimized respectively. The bucket index pool SHOULD use its own CRUSH hierarchy to map the bucket index pool to higher performance storage media such as SSD or NVMe drives.
7.1.1. Creating CRUSH Roots
From the command line on the administration node, create CRUSH roots in the CRUSH map for each CRUSH hierarchy. There MUST be at least one CRUSH hierarchy for service pools that may also potentially serve data storage pools. There SHOULD be at least one CRUSH hierarchy for the bucket index pool, mapped to high performance storage media such as SSDs or NVMe drives.
For details on CRUSH hierarchies, see the CRUSH Hierarchies section in the Red Hat Ceph Storage Storage Strategies Guide 4.
To manually edit a CRUSH map, see the Editing a CRUSH Map section in the Red Hat Ceph Storage Storage Strategies Guide 4.
In the following examples, the hosts named data0
, data1
and data2
use extended logical names such as data0-sas-ssd
, data0-index
and so forth in the CRUSH map, because there are multiple CRUSH hierarchies pointing to the same physical hosts.
A typical CRUSH root might represent nodes with SAS drives and SSDs for journals. For example:
## # SAS-SSD ROOT DECLARATION ## root sas-ssd { id -1 # do not change unnecessarily # weight 0.000 alg straw hash 0 # rjenkins1 item data2-sas-ssd weight 4.000 item data1-sas-ssd weight 4.000 item data0-sas-ssd weight 4.000 }
A CRUSH root for bucket indexes SHOULD represent high performance media such as SSD or NVMe drives. Consider creating partitions on SSD or NVMe media that store OSD journals. For example:
## # INDEX ROOT DECLARATION ## root index { id -2 # do not change unnecessarily # weight 0.000 alg straw hash 0 # rjenkins1 item data2-index weight 1.000 item data1-index weight 1.000 item data0-index weight 1.000 }
7.1.2. Using Logical Host Names in a CRUSH Map
In RHCS 3 and later releases, CRUSH supports the notion of a storage device "class," which is not supported in RHCS 2 and earlier releases. In RHCS 3 clusters with hosts or nodes that contain multiple classes of storage device, such as NVMe, SSD or HDD, use a single CRUSH hierarchy with device classes to distinguish different classes of storage device. This eliminates the need to use logical host names. In RHCS 2 and earlier releases, use multiple CRUSH hierarchies, one for each class of device, and logical host names to distinguish the hosts or nodes in the CRUSH hierarchy.
In the CRUSH map, host names must be unique and used only once. When the host serves multiple CRUSH hierarchies and use cases, a CRUSH map may use logical host names instead of the actual host name in order to ensure the host name is only used once. For example, a node may have multiple classes of drives such as SSDs, SAS drives with SSD journals, and SATA drives with co-located journals. To create multiple CRUSH hierarchies for the same host in RHCS 2 and earlier releases, the hierarchies will need to use logical host names in lieu of the actual host names so the bucket names are unique within the CRUSH hierarchy. For example, if the host name is data2
, the CRUSH hierarchy might use logical names such as data2-sas-ssd
and data2-index
. For example:
host data2-sas-ssd { id -11 # do not change unnecessarily # weight 0.000 alg straw hash 0 # rjenkins1 item osd.0 weight 1.000 item osd.1 weight 1.000 item osd.2 weight 1.000 item osd.3 weight 1.000 }
In the foregoing example, the host data2
uses the logical name data2-sas-ssd
to map the SAS drives with journals on SSDs into one hierarchy. The OSD IDs osd.0
through osd.3
in the forgoing example represent SAS drives using SSD journals in a high throughput hardware configuration. These OSD IDs differ from the OSD ID in the following example.
In the following example, the host data2
uses the logical name data2-index
to map the SSD drive for a bucket index into a second hierarchy. The OSD ID osd.4
in the following example represents an SSD drive or other high speed storage media used exclusively for a bucket index pool.
host data2-index { id -21 # do not change unnecessarily # weight 0.000 alg straw hash 0 # rjenkins1 item osd.4 weight 1.000 }
When using logical host names, ensure that one of the following settings is present in the Ceph configuration file to prevent the OSD startup scripts from using the actual host names upon startup and thereby failing to locate data in CRUSH map.
When the CRUSH map uses logical host names, as in the foregoing examples, prevent the OSD startup scripts from identifying the hosts according to their actual host names at initialization. In the [global]
section of the Ceph configuration file, add the following setting:
osd_crush_update_on_start = false
An alternative method of defining a logical host name is to define the location of the CRUSH map for each OSD in the [osd.<ID>]
sections of the Ceph configuration file. This will override any locations the OSD startup script defines. From the foregoing examples, the entries might look like the following:
[osd.0] osd crush location = "host=data2-sas-ssd" [osd.1] osd crush location = "host=data2-sas-ssd" [osd.2] osd crush location = "host=data2-sas-ssd" [osd.3] osd crush location = "host=data2-sas-ssd" [osd.4] osd crush location = "host=data2-index"
If one of the foregoing approaches isn’t used when a CRUSH map uses logical host names rather than actual host names, on restart, the Ceph Storage Cluster will assume that the OSDs map to the actual host names, and the actual host names will not be found in the CRUSH map, and Ceph Storage Cluster clients will not find the OSDs and their data.
7.1.3. Creating CRUSH Rules
Like the default CRUSH hierarchy, the CRUSH map also contains a default CRUSH rule.
The default rbd
pool may use this rule. DO NOT delete the default rule if other pools have used it to store customer data.
For general details on CRUSH rules, see the CRUSH Rules section in the Storage Strategies guide for Red Hat Ceph Storage 4. To manually edit a CRUSH map, see the Editing a CRUSH Map section in the Storage Strategies guide for Red Hat Ceph Storage 4.
For each CRUSH hierarchy, create a CRUSH rule. The following example illustrates a rule for the CRUSH hierarchy that will store the service pools, including .rgw.root
. In this example, the root sas-ssd
serves as the main CRUSH hierarchy. It uses the name rgw-service
to distinguish itself from the default rule. The step take sas-ssd
line tells the pool to use the sas-ssd
root created in Creating CRUSH Roots, whose child buckets contain OSDs with SAS drives and high performance storage media such as SSD or NVMe drives for journals in a high throughput hardware configuration. The type rack
portion of step chooseleaf
is the failure domain. In the following example, it is a rack.
## # SERVICE RULE DECLARATION ## rule rgw-service { type replicated min_size 1 max_size 10 step take sas-ssd step chooseleaf firstn 0 type rack step emit }
In the foregoing example, if data gets replicated three times, there should be at least three racks in the cluster containing a similar number of OSD nodes.
The type replicated
setting has NOTHING to do with data durability, the number of replicas or the erasure coding. Only replicated
is supported.
The following example illustrates a rule for the CRUSH hierarchy that will store the data pool. In this example, the root sas-ssd
serves as the main CRUSH hierarchy—the same CRUSH hierarchy as the service rule. It uses rgw-throughput
to distinguish itself from the default rule and rgw-service
. The step take sas-ssd
line tells the pool to use the sas-ssd
root created in Creating CRUSH Roots, whose child buckets contain OSDs with SAS drives and high performance storage media such as SSD or NVMe drives in a high throughput hardware configuration. The type host
portion of step chooseleaf
is the failure domain. In the following example, it is a host. Notice that the rule uses the same CRUSH hierarchy, but a different failure domain.
## # THROUGHPUT RULE DECLARATION ## rule rgw-throughput { type replicated min_size 1 max_size 10 step take sas-ssd step chooseleaf firstn 0 type host step emit }
In the foregoing example, if the pool uses erasure coding with a larger number of data and encoding chunks than the default, there should be at least as many racks in the cluster containing a similar number of OSD nodes to facilitate the erasure coding chunks. For smaller clusters, this may not be practical, so the foregoing example uses host
as the CRUSH failure domain.
The following example illustrates a rule for the CRUSH hierarchy that will store the index pool. In this example, the root index
serves as the main CRUSH hierarchy. It uses rgw-index
to distinguish itself from rgw-service
and rgw-throughput
. The step take index
line tells the pool to use the index
root created in Creating CRUSH Roots, whose child buckets contain high performance storage media such as SSD or NVMe drives or partitions on SSD or NVMe drives that also store OSD journals. The type rack
portion of step chooseleaf
is the failure domain. In the following example, it is a rack.
## # INDEX RULE DECLARATION ## rule rgw-index { type replicated min_size 1 max_size 10 step take index step chooseleaf firstn 0 type rack step emit }
7.2. Creating the Root Pool
The Ceph Object Gateway configuration gets stored in a pool named .rgw.root
, including realms, zone groups and zones. By convention, its name is not prepended with the zone name.
.rgw.root
If the Ceph Storage Cluster is running, create an .rgw.root
pool using the new rule. See the Ceph Placement Groups (PGs) per Pool Calculator and the Placement Groups chapter in the Storage Strategies Guide for details on the number of PGs. See the Create a Pool section in the Storage Strategies Guide for details on creating a pool.
In this instance, the pool will use replicated
and NOT erasure
for data durability. For example:
# ceph osd pool create .rgw.root 32 32 replicated sas-ssd
For service pools, including .rgw.root
, the suggested PG count from the Ceph Placement Groups (PGs) per Pool Calculator is substantially less than the target PGs per OSD. Also, ensure the number of OSDs is set in step 3 of the calculator.
Once this pool gets created, the Ceph Object Gateway can store its configuration data in the pool.
7.3. Creating a Realm
The Ceph Storage pools supporting the Ceph Object Gateway apply to a zone within a zone group. By default, Ceph Object Gateway will define a default zone group and zone.
For the master zone group and zone, Red Hat recommends creating a new realm, zone group and zone. Then, delete the default zone and its pools if they were already generated. Use Configuring a Master Zone as a best practice, because this configures the cluster for Multi Site operation.
- Create a realm. See Realms for additional details.
- Create a master zone group. See Zone Groups for additional details on zone groups.
- Create a master zone. See Zones for additional details on zones.
-
Delete the default zone group and zone. You MAY delete default pools if they were created, and are not storing client data. DO NOT delete the
.rgw.root
pool. - Create a System User.
- Update the period.
- Update the Ceph Configuration file.
This procedure omits the step of starting the gateway, since the gateway may create the pools manually. To specify specific CRUSH rules and data durability methods, create the pools manually.
By setting up a new realm, zone group and zone, the cluster is now prepared for expansion to a multi site cluster where there are multiple zones within the zone group. This means that the cluster can be expanded and configured for failover, and disaster recovery. See Expanding the Cluster with Multi Site for additional details.
In Red Hat Ceph Storage 2, multi site configurations are active-active by default. When deploying a multi site cluster, the zones and their underlying Ceph storage clusters may be in different geographic regions. Since each zone has a deep copy of each object in the same namespace, users can access the copy from the zone that is physically the closest to them, reducing latency. However, the cluster may be configured in active-passive mode if the secondary zones are intended only for failover and disaster recovery.
Using a zone group with multiple zones is supported. Using multiple zone groups is a technology preview only, and is not supported in production.
7.4. Creating Service Pools
The Ceph Object Gateway uses many pools for various service functions, and a separate set of placement pools for storing bucket indexes, data and other information.
Since it is computationally expensive to peer a pool’s placement groups, Red Hat generally recommends that the Ceph Object Gateway’s service pools use substantially fewer placement groups than data storage pools.
The service pools store objects related to service control, garbage collection, logging, user information, usage, etc. By convention, these pool names have the zone name prepended to the pool name.
Since Red Hat Ceph Storage 4.1, garbage collection uses the log pool with regular RADOS objects instead of OMAP. In the future, more features will store metadata on the log pool. Therefore, it is highly recommended to use NVMe/SSD OSDs for the log pool.
-
.<zone-name>.rgw.control
: The control pool. -
.<zone-name>.log
: The log pool contains logs of all bucket/container and object actions such as create, read, update and delete. -
.<zone-name>.rgw.buckets.index
: This pool stores index of the buckes. -
.<zone-name>.rgw.buckets.data
: This pool stores data of the buckets. -
.<zone-name>.rgw.meta
: The metadata pool storesuser_keys
and other critical metadata. -
.<zone-name>.meta:users.uid
: The user ID pool contains a map of unique user IDs. -
.<zone-name>.meta:users.keys
: The keys pool contains access keys and secret keys for each user ID. -
.<zone-name>.meta:users.email
: The email pool contains email addresses associated to a user ID. -
.<zone-name>.meta:users.swift
: The Swift pool contains the Swift subuser information for a user ID.
Execute the Get a Zone procedure to see the pool names.
# radosgw-admin zone get [--rgw-zone=<zone>]
When radosgw-admin
creates a zone, the pool names SHOULD be prepended with the zone name. For example, a zone named us-west
SHOULD have pool names that look something like this:
{ "domain_root": ".rgw.root", "control_pool": ".us-west.rgw.control", "gc_pool": ".us-west.rgw.gc", "log_pool": ".us-west.log", "intent_log_pool": ".us-west.intent-log", "usage_log_pool": ".us-west.usage", "user_keys_pool": ".us-west.users.keys", "user_email_pool": ".us-west.users.email", "user_swift_pool": ".us-west.users.swift", "user_uid_pool": ".us-west.users.uid", "system_key": { "access_key": "", "secret_key": ""}, "placement_pools": [ { "key": "default-placement", "val": { "index_pool": ".us-west.rgw.buckets.index", "data_pool": ".us-west.rgw.buckets", "data_extra_pool": ".us-west.rgw.buckets.non-ec" "index_type": 0 } } ] }
Beginning with control_pool
and ending with user_uid_pool
, create the pools using the pool names in the zone, provided the zone name is prepended to the pool name. Following the previous examples, pool creation might look something like this:
# ceph osd pool create .us-west.rgw.control 32 32 replicated rgw-service ... # ceph osd pool create .us-west.users.uid 32 32 replicated rgw-service
From previous examples, the rgw-service
rule represents a CRUSH hierarchy of SAS drives with SSD journals and rack
as the CRUSH failure domain. See Creating CRUSH Roots, and Creating CRUSH Rules for preceding examples.
See the Ceph Placement Groups (PGs) per Pool Calculator and the Placement Groups chapter in the Storage Strategies guide for details on the number of PGs. See the Create a Pool section in the Storage Strategies guide for details on creating a pool.
For service pools the suggested PG count from the calculator is substantially less than the target PGs per OSD. Ensure that step 3 of the calculator specifies the correct number of OSDs.
Generally, the .rgw.root
pool and the service pools should use the same CRUSH hierarchy and use at least node
as the failure domain in the CRUSH rule. Like the .rgw.root
pool, the service pools should use replicated
for data durability, NOT erasure
.
7.5. Creating Data Placement Strategies
The Ceph Object Gateway has a default storage policy called default-placement
. If the cluster has only one storage policy, the default-placement
policy will suffice. This default placement policy is referenced from the zone group configuration and defined in the zone configuration.
See Storage Policies section in the Red Hat Ceph Storage 4 Ceph Object Gateway Guide for Red Hat Enterprise Linux for additional details.
For clusters that support multiple use cases, such as IOPS-optimized, throughput-optimized or capacity-optimized, a set of placement targets in the zone group configuration and a set of placement pools in the zone configuration represent each storage policy.
The examples in the following sections illustrate how to create a storage policy and make it the default policy. This example also assumes the default policy will use a throughput-optimized hardware profile. Topics include:
7.5.1. Creating an Index Pool
By default, Ceph Object Gateway maps a bucket’s objects to an index, which enables a gateway client to request a list of objects in a bucket among other things. While common use cases may involve quotas where users have a bucket and a limited number of objects per bucket, buckets can store innumerable objects. When buckets store millions of objects or more, index performance benefits substantially from using high performance storage media such as SSD or NVMe drives to store its data. Additionally, bucket sharding also dramatically improves performance.
See the Ceph Placement Groups (PGs) per Pool Calculator and the Placement Groups chapter in the Storage Strategies guide for details on the number of PGs. See the Create a Pool section in the Storage Strategies guide for details on creating a pool.
The PG per Pool Calculator recommends a smaller number of PGs per pool for the index pool; however, the PG count is approximately twice the number of PGs as the service pools.
Red Hat does not support HDD devices for index pools. For more information on supported configurations, see the Red Hat Ceph Storage: Supported configurations article.
To create an index pool, execute ceph osd pool create
with the pool name, the number of PGs and PGPs, the replicated
data durability method, and the name of the rule.
If buckets will store more than 100k objects, configure bucket sharding to ensure that index performance doesn’t degrade as the number of objects in the bucket increases. See the Configuring Bucket Sharding section in the Ceph Object Gateway Guide Configuration and Administration Guide. Also see the Bucket Index Resharding section in the Ceph Object Gateway Guide Configuration and Administration Guide for details on resharding a bucket if the original configuration is no longer suitable.
7.5.2. Creating a Data Pool
The data pool is where Ceph Object Gateway stores the object data for a particular storage policy. The data pool should have a full complement of PGs, not the reduced number of PGs for service pools. The data pool SHOULD consider using erasure coding, as it is substantially more efficient than replication and can significantly reduce the capacity requirements while maintaining data durability.
To use erasure coding, create an erasure code profile. See the Erasure Code Profiles section in the Storage Strategies Guide for more details.
Choosing the correct profile is important because you cannot change the profile after you create the pool. To modify a profile, you must create a new pool with a different profile and migrate the objects from the old pool to the new pool.
The default configuration is two data chunks and one encoding chunk, which means only one OSD can be lost. For higher resiliency, consider a larger number of data and encoding chunks. For example, some large very scale systems use 8 data chunks and 3 encoding chunks, which allows three OSDs to fail without losing data.
Each data and encoding chunk SHOULD get stored on a different node or host at a minimum. For smaller clusters, this makes using rack
impractical as the minimum CRUSH failure domain when using a larger number of data and encoding chunks. Consequently, it is common for the data pool to use a separate CRUSH hierarchy with host
as the minimum CRUSH failure domain. Red Hat recommends host
as the minimum failure domain. If erasure code chunks get stored on OSDs within the same host, a host failure such as a failed journal or network card could lead to data loss.
To create a data pool, execute ceph osd pool create
with the pool name, the number of PGs and PGPs, the erasure
data durability method, the erasure code profile and the name of the rule.
7.5.3. Creating a Data Extra Pool
The data_extra_pool
is for data that cannot use erasure coding. For example, multi-part uploads allow uploading a large object such as a movie in multiple parts. These parts must first be stored without erasure coding. Erasure coding will apply to the whole object, not the partial uploads.
The PG per Pool Calculator recommends a smaller number of PGs per pool for the data_extra_pool
; however, the PG count is approximately twice the number of PGs as the service pools and the same as the bucket index pool.
To create a data extra pool, execute ceph osd pool create
with the pool name, the number of PGs and PGPs, the replicated
data durability method, and the name of the rule. For example:
# ceph osd pool create .us-west.rgw.buckets.non-ec 64 64 replicated rgw-service
7.5.4. Configuring Placement Targets in a Zone Group
Once the pools are created, create the placement target in the zone group. To retrieve the zone group, execute the following to output the zone group configuration to a file called zonegroup.json
:
# radosgw-admin zonegroup get [--rgw-zonegroup=<zonegroup>] > zonegroup.json
The file contents will look something like this:
{ "id": "90b28698-e7c3-462c-a42d-4aa780d24eda", "name": "us", "api_name": "us", "is_master": "true", "endpoints": [ "http:\/\/rgw1:80" ], "hostnames": [], "hostnames_s3website": [], "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e", "zones": [ { "id": "9248cab2-afe7-43d8-a661-a40bf316665e", "name": "us-east", "endpoints": [ "http:\/\/rgw1" ], "log_meta": "true", "log_data": "true", "bucket_index_max_shards": 0, "read_only": "false" }, { "id": "d1024e59-7d28-49d1-8222-af101965a939", "name": "us-west", "endpoints": [ "http:\/\/rgw2:80" ], "log_meta": "false", "log_data": "true", "bucket_index_max_shards": 0, "read_only": "false" } ], "placement_targets": [ { "name": "default-placement", "tags": [] } ], "default_placement": "default-placement", "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe" }
The placement_targets
section will list each storage policy. By default, it will contain a placement target called default-placement
. The default placement target is identified immediately after the placement_targets
section.
Assuming a placement target called throughput-optimized
, with throughput-optimized
as the default target, the placement_targets
section and the default_placement
setting of the zone group configuration should be modified to something like this:
{ ... "placement_targets": [ { "name": "throughput-optimized", "tags": [] } ], "default_placement": "throughput-optimized", ... }
Finally, set the zone group configuration with the settings from the modified zonegroup.json
file; then, update the period. For example:
# radosgw-admin zonegroup set [--rgw-zonegroup=<zonegroup>] --infile zonegroup.json # radosgw-admin period update --commit
7.5.5. Configuring Placement Pools in a Zone
Once the zone group has the new throughput-optimized
placement target, map the placement pools for throughput-optimized
in the zone configuration. This step will replace the mapping for default-placement
to its associated pools with a throughput-optimized
set of placement pools.
Execute the Get a Zone procedure to see the pool names.
# radosgw-admin zone get [--rgw-zone=<zone>] > zone.json
Assuming a zone named us-west
, the file contents will look something like this:
{ "domain_root": ".rgw.root", "control_pool": ".us-west.rgw.control", "gc_pool": ".us-west.rgw.gc", "log_pool": ".us-west.log", "intent_log_pool": ".us-west.intent-log", "usage_log_pool": ".us-west.usage", "user_keys_pool": ".us-west.users.keys", "user_email_pool": ".us-west.users.email", "user_swift_pool": ".us-west.users.swift", "user_uid_pool": ".us-west.users.uid", "system_key": { "access_key": "", "secret_key": ""}, "placement_pools": [ { "key": "default-placement", "val": { "index_pool": ".us-west.rgw.buckets.index", "data_pool": ".us-west.rgw.buckets", "data_extra_pool": ".us-west.rgw.buckets.non-ec" "index_type": 0 } } ] }
The placement_pools
section of the zone configuration defines sets of placement pools. Each set of placement pools defines a storage policy. Modify the file to remove the default-placement
entry, and replace it with a throughput-optimized
entry with the pools created in the preceding steps. For example:
{ ... "placement_pools": [ { "key": "throughput-optimized", "val": { "index_pool": ".us-west.rgw.buckets.index", "data_pool": ".us-west.rgw.buckets.throughput"} "data_extra_pool": ".us-west.rgw.buckets.non-ec", "index_type": 0 } ] }
Finally, set the zone configuration with the settings from the modified zone.json
file; then, update the period. For example:
# radosgw-admin zone set --rgw-zone={zone-name} --infile zone.json # radosgw-admin period update --commit
The index_pool
points to the index pool and CRUSH hierarchy with SSDs or other high-performance storage, the data_pool
points to a pool with a full complement of PGs, and a CRUSH hierarchy of high-throughput host bus adapters, SAS drives and SSDs for journals.
7.5.6. Data Placement Summary
When processing client requests, the Ceph Object Gateway will use the new throughput-optimized
target as the default storage policy. Use this procedure to establish the same target in different zones and zone groups in a multi-site configuration, replacing the zone name for the pools as appropriate.
Use this procedure to establish additional storage policies. The naming for each target and set of placement pools is arbitrary. It could be fast
, streaming
, cold-storage
or any other suitable name. However, each set must have a corresponding entry under placement_targets
in the zone group, and one of the targets MUST be referenced in the default_placement
setting; and, the zone must have a corresponding set of pools configured for each policy.
Client requests will always use the default target, unless the client request specifies X-Storage-Policy
and a different target. See Create a Container for an object gateway client usage example.