I have a dell server running windows (unfortunately) that is experiencing daily or every other day clock drifts of about -211665ms, and its weirdly almost exactly the same amount each time. I have a gps time server and most of the log events correspond with when the sync happens. As far as i can tell the gateway isn’t running out of memory, and there is no anti virus or vulnerability scanning on this machine.
Does anyone have any idea what may be causing this? I’ve heard that it could be hardware related, but do not know how one would begin diagnosing that.
One or more hardware clocks are out of sync with each other. The OS is reporting to Java "hey, your time jumped backwards ~200 seconds", and Java is dutifully responding. The 'clock drift' warning in Ignition is just a symptom of that underlying problem.
I understand that, but how would one begin to determine the underlying problem? I’ve used these GPS time servers before without issue, so i am leaning towards it being on the server end. The most recent sync was 10am today and i can already see the os time is 3 seconds off.
I thought the cmos battery only would affect the time at startup? This server very rarely sees a restart. I have a redundant gateway that also syncs with the same GPS time server and that one appears to be also generating the clock drift warnings but a positive amount. I am now more confused.
I completely agree, but our IT is VERY resistant to anything not Microsoft, though i have been pushing the issue. I am more concerned by the amount of the drift and that it may be an indicator of potential hardware failure. I am trying to get these servers replaced soon in any event.
Just guessing about why your clocks might be drifting so quickly.
Windows time corrections are warps, so if you are drifting a lot between the syncs then what you're seeing is expected. Speed up the syncs, try alternate NTP service (linked), or I guess try to figure out why time is drifting out of sync so quickly
According to Stack Overflow, any time updates from the windows end should be in the logs:
You can read the Windows Event Logs and look for changes to system time. (System time changes are one of the system events that are automatically logged.) For example, I just tweaked my system time by a few seconds and the following appeared in the System Event log:
Information 10/21/2011 11:16:26 AM Kernel-General 1 None
The system time has changed to 2011-10-21T16:16:26.000000000Z from 2011-10-21T16:16:26.000000000Z.
They do show in the log, though interestingly there are 2 time change events at the same time (it doesn’t appear the event log is more granular than to the second though). The first one was a miniscule shift and the second was approximately 3.5 minutes which corresponds with the clock drift warning in ignitions log.
Maybe increasing the poll rate for the NTP server would make the warps much smaller, VMware recommendation is 900 seconds (15min) because of the nature of VMs, so maybe on a bare metal server that would help?
When VMs are involved, there can be conflicts between hypervisor and guest if both are running time sync. (In the Linux world, it is common to have the hypervisor track an upstream time source and provide a clock driver to the guest that transparently locks them together.)
