How to use GPT partitioned table for big disk (>2TB) on overcloud ?
Environment
Red Hat OpenStack Platform 8.0
Issue
- After deployed overcloud on the nodes with 6TB disk, we only see 1.6TB partition was created and 4TB was left unused on the compute / controller nodes.
- We also found that the node used MBR partition table and this seems to be the reason why there is no partition greater than 2TB.
- Does Red Hat provide GPT-partitioned overcloud image?
Here's output from compute node:
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 1.5T 54G 1.5T 4% /
devtmpfs 126G 0 126G 0% /dev
tmpfs 126G 39M 126G 1% /dev/shm
tmpfs 126G 4.0M 126G 1% /run
tmpfs 126G 0 126G 0% /sys/fs/cgroup
tmpfs 26G 0 26G 0% /run/user/990
tmpfs 26G 0 26G 0% /run/user/991
192.168.228.208:/nsvol01/glance_images 56G 920M 55G 2% /var/lib/glance/images
tmpfs 26G 0 26G 0% /run/user/1000
$ sudo fdisk /dev/sda
...
Command (m for help): p
Disk /dev/sda: 5998.2 GB, 5998190264320 bytes, 11715215360 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0001810e
Device Boot Start End Blocks Id System
/dev/sda1 2048 4095 1024 83 Linux
/dev/sda2 * 4096 3125273877 1562634891 83 Linux
Resolution
- Currently, we cannot use UEFI boot mode and GPT paritioned disk for over 2TB disk on overcloud with Director.
- BIOS boot mode will use MBR partition and UEFI boot mode will use GPT, but Director currently does not support UEFI boot mode.
- Due to growpart Bug which destroy partition table, we cannot use GPT even if we could use UEFI.
(We also confirmed internally another method (changing partition type by setting flavor for the ironic) didn't work too.
Root Cause
1) According to the doc, Ironic will decide partition type based on boot mode ( bios / uefi ) as follows.
The disk label can be configured in two ways; when Ironic is used with the Compute service or in standalone mode. The following bullet points and sections will describe both methods:
- When no disk label is provided Ironic will configure it according to the boot mode; bios boot mode will use msdos and uefi boot mode will use gpt.
- Only one disk label - either msdos or gpt - can be configured for the node.
And in the code, we found that ironic will use GPT when boot with UEFI too.
- /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py
254 def make_partitions(dev, root_mb, swap_mb, ephemeral_mb,
255 configdrive_mb, node_uuid, commit=True,
256 boot_option="netboot", boot_mode="bios"):
...
284 # For uefi localboot, switch partition table to gpt and create the efi
285 # system partition as the first partition.
286 if boot_mode == "uefi" and boot_option == "local":
287 dp = disk_partitioner.DiskPartitioner(dev, disk_label="gpt")
288 part_num = dp.add_partition(CONF.deploy.efi_system_partition_size,
289 fs_type='fat32',
290 bootable=True)
291 part_dict['efi system partition'] = part_template % part_num
292 else:
293 dp = disk_partitioner.DiskPartitioner(dev)
294
- /usr/lib/python2.7/site-packages/ironic/common/disk_partitioner.py
52 class DiskPartitioner(object):
53
54 def __init__(self, device, disk_label='msdos', alignment='optimal'):
2) But according to Private Bug 1290272 - [OSP 8.0 Bug]: osp-director doesn't work with UEFI boot loader. Director does not support UEFI boot mode.
IMPORTANT
The director currently does not support UEFI boot mode.
3) Also, we confirmed internally that we cannot change partition type by setting flavor for the ironic.
ironic node-update <uuid> replace properties/capabilities='profile:compute,boot_option:local,disk_label:gpt'
openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="compute" --property "capabilities:disk_label"="gpt" Compute
4) And so for GPT-partitioned overcloud image, if we create and use it , we may hit following growpart bug when running growpart from cloud-init on booting bare-metal image after deploying by ironic.
This bug during expanding filesystem over 2TB, should be fixed in the upstream revision 250 and we're working on following Bugzilla.
Reference for related patch and bug for ironic:
ironic
- Switch default partition type to gpt
ironic-lib
- Switch default partition type to gpt
Upstream launchpad Bug
- Can't create partitions with size > 2TB
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments