ext3 file fragmentation causes performance issue

Latest response

I'm sure I'll take a beating for this, but I'll ask anyway. I have files on filesystems that are getting badly fragmented and are causing serious performance problems. Heres an example of a 659MB file:

 

-rw-r--r-- 1 root     root  659906342 Oct 19 20:13 marktest
 

root@etvfdpa1:/apps/opt/apb # filefrag marktest
marktest: 237 extents found, perfection would be 6 extents
 

I copied this file to another filesystem and it looks like this:

 

marktest: 6 extents found
 

That copy took about 3 minutes because the source file was so badly fragmented.

 

I then copied the same file that was in the filesystem with 6 extents to a third filesystem and it only took 2 seconds. The resulting file also has 6 extents.

 

This is killing the performance of our application. When we copy the files that our aplication reads to a fresh clean filesystem and read the records from those files, our application runs in 30 minutes as opposed to 3 hours on a filesystem that has the same files but are fragmented.

 

I have read everything in google about ext3 filesystem do not need to be defragmented, but I have real evedence that this is not the case.

 

How do I defragment these files? A simple copy to the same filesystem does not work. We also cannot bring the filesystem down and fsck it or whatever because we are constantly processing files all day.

 

Any help would be greatly appreciated as this issue is killing us.

 

Mark

Responses

I've never used it, but it looks like there is a tool called Shake that will provide this functionality at the user-level. The attached link is Ubuntu specific, but I'm sure there's a RHEL/CentOS/EL build someplace.
http://www.webupd8.org/2009/10/defragmenting-linux-ext3-filesystems.html