Issue

We upgraded 393 servers from ksh-20120801-10.el6 to ksh-20120801-10.el6_5.4. After that upgrade, one of the servers hung, forcing a reboot. RCA on the system found that the issue was that hundreds of ksh scripts were hanging, eventually rendering the system unusable.

We've been able to re-create the issue with the following ksh script:

#!/usr/bin/ksh

FILE_DIR=kshhang.data
mkdir $FILE_DIR
for x in {1..12000}
do
    touch $FILE_DIR/some.longish.file.name.$x
done

cd $FILE_DIR                   
>ALL_LRNs.dat                      
ls some.longish.file.name.* >.files.tmp         
for FILE in `cat .files.tmp`
do                      
cat $FILE >> ALL_LRNs.dat         
rm $FILE                   
done

We also found this happens even with the newest ksh from RHN.