ping and ibnetdiscover not working with realtime kernel after opensm server is restarted

Issue

We experienced some issue with our Infiniband infrastructure after the restart of our primary opensm server:

• Some servers were not able to ping each other via IPoIB network while the ibping command worked fine.
• The same servers were not able to execute ibnetdiscover command because it hanged.
• The problem was not extended to all servers and we solved the issue via reboot of the affected nodes.
• sometimes while switching between different opensm masters a random opensm client crashes and dumps a vmcore