How to use GPT partitioned table for big disk (>2TB) on overcloud ?

Solution Verified - Updated -

Environment

Red Hat OpenStack Platform 8.0

Issue

  • After deployed overcloud on the nodes with 6TB disk, we only see 1.6TB partition was created and 4TB was left unused on the compute / controller nodes.
  • We also found that the node used MBR partition table and this seems to be the reason why there is no partition greater than 2TB.
  • Does Red Hat provide GPT-partitioned overcloud image?

Here's output from compute node:

$ df -h
Filesystem                              Size  Used Avail Use% Mounted on
/dev/sda2                               1.5T   54G  1.5T   4% /
devtmpfs                                126G     0  126G   0% /dev
tmpfs                                   126G   39M  126G   1% /dev/shm
tmpfs                                   126G  4.0M  126G   1% /run
tmpfs                                   126G     0  126G   0% /sys/fs/cgroup
tmpfs                                    26G     0   26G   0% /run/user/990
tmpfs                                    26G     0   26G   0% /run/user/991
192.168.228.208:/nsvol01/glance_images   56G  920M   55G   2% /var/lib/glance/images
tmpfs                                    26G     0   26G   0% /run/user/1000

$ sudo fdisk /dev/sda
...
Command (m for help): p

Disk /dev/sda: 5998.2 GB, 5998190264320 bytes, 11715215360 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0001810e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048        4095        1024   83  Linux
/dev/sda2   *        4096  3125273877  1562634891   83  Linux

Resolution

  • Currently, we cannot use UEFI boot mode and GPT paritioned disk for over 2TB disk on overcloud with Director.
    • BIOS boot mode will use MBR partition and UEFI boot mode will use GPT, but Director currently does not support UEFI boot mode.
    • Due to growpart Bug which destroy partition table, we cannot use GPT even if we could use UEFI.
      (We also confirmed internally another method (changing partition type by setting flavor for the ironic) didn't work too.

Root Cause

1) According to the doc, Ironic will decide partition type based on boot mode ( bios / uefi ) as follows.

The disk label can be configured in two ways; when Ironic is used with the Compute service or in standalone mode. The following bullet points and sections will describe both methods:

  - When no disk label is provided Ironic will configure it according to the boot mode; bios boot mode will use msdos and uefi boot mode will use gpt.
  - Only one disk label - either msdos or gpt - can be configured for the node.

And in the code, we found that ironic will use GPT when boot with UEFI too.

  • /usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py
 254 def make_partitions(dev, root_mb, swap_mb, ephemeral_mb,
 255                     configdrive_mb, node_uuid, commit=True,
 256                     boot_option="netboot", boot_mode="bios"):
...
 284     # For uefi localboot, switch partition table to gpt and create the efi
 285     # system partition as the first partition.
 286     if boot_mode == "uefi" and boot_option == "local":
 287         dp = disk_partitioner.DiskPartitioner(dev, disk_label="gpt")
 288         part_num = dp.add_partition(CONF.deploy.efi_system_partition_size,
 289                                     fs_type='fat32',
 290                                     bootable=True)
 291         part_dict['efi system partition'] = part_template % part_num
 292     else:
 293         dp = disk_partitioner.DiskPartitioner(dev)
 294
  • /usr/lib/python2.7/site-packages/ironic/common/disk_partitioner.py
 52 class DiskPartitioner(object):
 53
 54     def __init__(self, device, disk_label='msdos', alignment='optimal'):

2) But according to Private Bug 1290272 - [OSP 8.0 Bug]: osp-director doesn't work with UEFI boot loader. Director does not support UEFI boot mode.

IMPORTANT

The director currently does not support UEFI boot mode.

3) Also, we confirmed internally that we cannot change partition type by setting flavor for the ironic.

ironic node-update <uuid> replace properties/capabilities='profile:compute,boot_option:local,disk_label:gpt'

openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="compute" --property "capabilities:disk_label"="gpt" Compute

4) And so for GPT-partitioned overcloud image, if we create and use it , we may hit following growpart bug when running growpart from cloud-init on booting bare-metal image after deploying by ironic.

This bug during expanding filesystem over 2TB, should be fixed in the upstream revision 250 and we're working on following Bugzilla.

Reference for related patch and bug for ironic:

ironic
- Switch default partition type to gpt

ironic-lib
- Switch default partition type to gpt

Upstream launchpad Bug
- Can't create partitions with size > 2TB

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.