Why does RHEL5 copy files with a buffer of only 512 bytes when the filesystem can handle 8MB chunks?

Solution Unverified - Updated -

Issue

It was found out that the process of copying a file from one location to another on the same shared file system GPFS (set up using RDMA) does take a significantly long time. This issue looks to be related to our cluster components running under Red Hat Enterprise Linux 5. The following example consists of a 2 GB file, copied within the same folder:

ls -lh /gpfs3/trash/largefile
-rw-r--r-- 1 root root 2.0G Mar 14 12:08 /gpfs3/trash/largefile

Red Hat Enterprise Linux 5:

[root@hpc-rhel5 ~]# time cp /gpfs3/trash/largefile /gpfs3/trash/largefile2

real    0m25.323s
user    0m0.633s
sys     0m24.325s

Red Hat Enterprise Linux 6

[root@hpc-rhel6 ~]# time cp /gpfs3/trash/largefile /gpfs3/trash/largefile2

real    0m0.666s
user    0m0.001s
sys     0m0.632s

When coping data on attached local drives using cp, the copy duration is quite identical on both systems. Additionally, e.g. using dd to "copy" data does not seem to be affected.

Environment

  • Red Hat Enterprise Linux 5.7
  • Mellanox Infiniband hardware (OFED-1.5.3-4.0.22 drivers)
  • IBM GPFS shared filesystem with support for 8MB blocks

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content