RHEL 7.2 | req. assist with using btrfs on a raid 50, 16TB array, one volume only.

Latest response

I am seeking advice on potentially using btrfs for an ftp server that has a hardware raid 50 (seven raid 3s with a stripe, and a global hot spare).

We will only have one volume, we will never add additional volumes.

We recognize that btrfs is a technology preview, and the risks involved. We found the documentation at access.redhat.com/documentation...., yet the first (and only) thing it says to do is at paragraph 3.1 to do the following:

# mkfs.btrfs /dev/device

And there is no mention of using parted prior to make the partition in that documentation for btrfs.

I'm kickstarting the system right now, and plan on adding the 16TB array afterwards.

Has anyone ever added a btrfs hardware raid with only one volume (btrfs allows sub-volumes, but we don't need that for this specific use). The ftp server gets a lot of traffic in terms of files being written to, and then the files are sent elsewhere via scp. The files are generally "bigger" with the least sizes ranging in the kilobytes, and larger files as big as 64GB, give or take 10GB or so-ish, along with files smaller than that generally ranging in the GB sizes.

After looking at the documentation I linked to above, I'd appreciate any practical help if partitioning needs to be done prior (I'd personally be surprised if it didn't), and we'd probably use parted first, but the documentation mentions +nothing+ about that. We then just want to make one huge volume taking the entire raid50 with no sub-volumes and no other volumes to be added later, ever.

Thanks

Responses

Creation of volume/partition looks to be assumed knowledge.

You can place the filesystem in whatever sort of container you like, whether that container is a raw block device, a partition, LVM LV, software RAID, or any combination thereof. The choice is yours.

I'd use LVM either in a partition or on the raw device. The choice depends on your RAID chunk size (parted and LVM align to 1 MiB by default) and whether another admin is likely to see no partition in fdisk -l and assume the device is unused then start making their own partitions and overwriting existing data. A partition can be a good safety net against this.

I don't know much about btrfs, but if you aren't storing everything in the same directory, your use case sounds ideal for XFS. That would also then be fully supported.

LVM and XFS should detect the underlying RAID geometry automatically.

Jamie,

I've been reading "a lot" on this, and it seems there's a number of sources that say one does -=not=- have to partition first when using btrfs, but they can make the file system directly, then make a pool, or a volume.

Jamie, thanks for the input.

We have to date been using XFS for RHEL 7.2 however we heard the speed on btrfs was superior for the types of files we have. In short, after reviewing the material in the links below, we will probably go with XFS (and if we use btrfs, we'll have another "like" server using it)

We will have a number of sub-directories, but we have read various sources that btrfs is faster in some specific conditions (such as ours). However, I listened to this set of slides at http://marc.merlins.org/linux/talks/2015/Btrfs-LCA2015/Btrfs.pdf and the corresponding youtube presentation is here https://www.youtube.com/watch?v=6DplcPrQjvA and the material here seems to infer it may not be a good fit for us... we have two such servers, maybe we will have one with btrfs and one with XFS. The sad thing is there seems to be some certain care and feeding that is kinda mandatory.

Thanks much for the info and clarification. If anyone has any other experience or guidance on btrfs, please advise. thanks

We may go ahead and do this, but without metadata. Why? Because the data written to this volume can be recovered elsewhere in a day, and we're really interested in speed.

We would appreciate any input from those using btrfs,

Thanks,

Can you confirm whether the workload in question is single threaded or multi-threaded? I think probably the latter from the description, but that has a bearing on the answer, most specifically on performance where all the data is in a single btrfs sub-volume. Looking at the slides I think there is some old/inaccurate information in there - I didn't look at the presentation though, so maybe the clarifies some of those points.

While it is true that older LVM snaphots were slower than they might have been, this issue has been fixed more recently. We would be happy to give some guidance on that if that would be useful. I suspect that the LCA presentation refers to the older snapshots, but it is not clear and also doesn't mention other critical details such as the workload and what kind of device was in use.

The short answer is however, that I would highly recommend XFS and LVM in combination for your use case.

Thanks Steven,

Multi-threaded processes. I think after discussion with folks here (and your input above), we're going to take your advice and go with LVM with XFS. We do have one (just one) of our customers running btrfs on just one server for one unique partition, but we'll wait until btrfs matures some more.

Thanks much!

Jamie, thank you too!

No worries!

Thanks - the reason that I asked specifically about whether the workload was multi-threaded is that there is a known performance issue with btrfs in the case that multiple threads are all writing to the same subvolume. This stems from the single tree/single lock design of btrfs and XFS will give significantly better performance as the thread count increases. It can be avoided on btrfs by giving each thread its own subvolume, but this is often not convenient for the use case. This is also conditional upon the underlying device being of high enough performance to see the effect, but with a RAID and/or striping this is quite likely to be the case.

I did also notice that the original message mentions RAID3 which may not be the highest performance configuration for your use case. Do let us know if you'd like some guidence on that and I can ask the storage team to respond.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.