Hi Team,
We are suddenly facing an issue where the Ignition server is using 100% CPU.
The server log shows the following message:
“Clock drift, degraded performance, or pause-the-world detected. Max allowed deviation = 1000ms.”
Has anyone encountered this issue before or have an idea how to resolve it? The entire plant operation is currently stopped.
Thanks in advance for your help.
I would contact official IA support.
I had that issue last week, and by going to Status > Running Scripts I could see that there was some script somewhere that was running for seconds and minutes, and just kept piling up instances of it. Since it wasn’t a scheduled task I couldn’t tell where it was running from. I was able to kill the scripts, but not keep them from continuing to pile up until I restarted the server.
Something I had done recently during development caused an endless loop or something, but I never did figure out what, since the problem went away after the restart.
The message is the result of your server issues. as @Ryan_Deardorff said, start checking active scripts, and other things going on in your Gateway like Database connections, threads, etc. Does the wrapper log have any more information to help you?
This if you need immediate support. Support isn't guaranteed here since it's a community forum. I wish you luck!!
Script instances piling up is usually some kind of abuse of system.util.invokeAsynchronous(), particularly if CPU goes to 100%.
I never use that. I do use Timers, though.
Eeewwww! (jython stdlib)
Can you elaborate? I only user Timers in the view, when I can’t figure a better way to get something like .focus() to happen at the right time. Not ideal, I know, but better than sleep().
Actually, no, I would say a short sleep is better than a jython Timer. Perspective will create and clean up threads around sleeps without jython's dangerous threading getting involved. (Can leak interpreters, possibly worse.)
Good to know, but I thought that since sleep() blocks its thread from completion, it was the worse choice. I don’t use them a lot, and only as a last resort, but I haven’t had issues with Timers, that I knew of anyway–it’s entirely possible that what I encountered last week was caused by one.
Sleep is a horror for Vision, because Swing requires all GUI activity to happen on the foreground thread. Perspective doesn't have that problem, and will dynamically add threads to continue forward progress. So a short sleep in a Perspective event isn't that ugly. (Short = ~1s or less.)
For posterity, because I just looked it up:
Jython's time.sleep is exactly equivalent to java.lang.Thread.sleep(), in that it's literally a native Java method that dispatches to Thread.sleep internally:
(I was curious whether using time.sleep might cause issues down the road when we migrate to a version of Java with virtual threads and possibly change Perspective's default dispatch facility to use virtual threads)
public static void sleep(double secs) {
if (secs == (double)0.0F) {
Thread.yield();
} else {
try {
Thread.sleep((long)(secs * (double)1000.0F));
} catch (InterruptedException var3) {
throw new PyException(Py.KeyboardInterrupt, "interrupted sleep");
}
}
}
It's the Timer class's thread creation that I'd worry about.
Jython's Timer extends Jython's Thread which extends java.lang.Thread, so it's at least not doing anything hideously terrible.
The wait is ultimately done by a Jython _Event that's backed by a org.python.modules._threading.Condition instance which internally contains java.concurrent lock/condition objects. I still wouldn't recommend using it, but mostly from the "leaking interpreters when you pass anonymous closures in" angle, less from a concurrency safety standpoint. 