NTP Drift, How To Fix?

So I have a mixed environment of RHEL 5/6 servers and on some of them, there is ntp drift. The time drift varies for all of them.

These are all VMs. All of them are patched up and I have iptables rules to allow for 123 on input/output.

For one of them, when I run various ntp commands, such as:

ntpstat 
Unable to talk to NTP daemon.  Is it running? 

./ntpd status 
ntpd dead but pid file exist 

ntpdate -u (ntp server address here) 
18 Nov 09:4700 ntpdate (11967):  step time server 143.83.220.1 offset --197.500050 sec 

The final command forced it to work, however I will check it later this week and there will be drift.

Also what is the general rule for having the hardware clock match the system time?

thanks