Issue

Customer noticed some performance issues on IPA Server.
After some investigation noticed device sda has lots of excessive I/O.

• sda has excessive io %util:

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.81    0.00    4.48   94.70    0.00    0.00

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00    16.90    0.00  566.60     0.00  4668.02     8.24     1.40    2.46   1.76  99.57
sdb               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdc               0.00     1.63    0.00    0.61     0.00    17.92    29.33     0.00    7.00   2.33   0.14
VolGroupboot-LogVolroot     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVolsw02     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupdata1-LogVoldirsrv     0.00     0.00    0.00    2.24     0.00    17.92     8.00     0.02    7.00   0.64   0.14
VolGroupbackup-LogVolbackup     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVoltmp     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVolopt     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVolhome     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVolvar     0.00     0.00    0.00  583.50     0.00  4668.02     8.00     1.50    2.58   1.71  99.71
VolGroupboot-LogVolaudit     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
VolGroupboot-LogVolusr     0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

• noticed there were lots of IPA replication storm:

NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): Skipping update operation with no message_id (uniqueid 90335001-d6db11e2-95cbb106-b5189ab3, CSN 533c6ef2000300110000):
agmt="cn=example.com" (host003:389) - load=13 rec=1147 csn=533c6ef3000000110000
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: Sending modify operation (dn="uid=user1,cn=users,cn=accounts,dc=example,dc=com" csn=533c6ef3000000110000)
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: modifys operation (dn="uid=user1,cn=users,cn=accounts,dc=example,dc=com" csn=533c6ef3000000110000) not sent - empty
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: Consumer successfully sent operation with csn 533c6ef3000000110000
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): Skipping update operation with no message_id (uniqueid 90335001-d6db11e2-95cbb106-b5189ab3, CSN 533c6ef3000000110000):
agmt="cn=example.com" (host003:389) - load=13 rec=1148 csn=533c6ef5000000110000
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: Sending modify operation (dn="uid=user1,cn=users,cn=accounts,dc=example,dc=com" csn=533c6ef5000000110000)
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: modifys operation (dn="uid=user1,cn=users,cn=accounts,dc=example,dc=com" csn=533c6ef5000000110000) not sent - empty
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: Consumer successfully sent operation with csn 533c6ef5000000110000
 NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): Skipping update operation with no message_id (uniqueid 90335001-d6db11e2-95cbb106-b5189ab3, CSN 533c6ef5000000110000):
agmt="cn=example.com" (host003:389) - load=13 rec=1149 csn=533c6efc000000110000
NSMMReplicationPlugin - agmt="cn=example.com" (host003:389): replay_update: Sending modify operation (dn="uid=user1,cn=users,cn=accounts,dc=example,dc=com" csn=533c6efc000000110000)