We are having an issue where the ignition gateway is shutting down unexpectedly, and repeatedly.
I have looked at the console, and it looks like something is triggering a gateway/project shutdown script.
There are multiple errors occuring, but I cannot determine what is causing the shutdown to occur.
I have attached the logs. We are having to reboot the machine running the ignition gateway multiple times per day.
Dan
logs (5).bin.gz (504 KB)
Can you zip up your wrapper.log files (in the logs directory of wherever Ignition is installed) and attach those as well?
I see some instances where the JVM hung and got restarted by the service wrapper - this could be a deadlock or OutOfMemory problem. A few others appear to be machine restarts, which can only happen externally.
A good place to start would be upgrading to the latest 7.5.x release. It looks like you’re running 7.5.3 currently.
The manual restarts were performed due to the unusability of the system. It would get hung up on something, and then would lock up.
Can the upgrade be applied live, or does it require downtime? We can’t have the plant down without scheduling it way in advance.
Can you give me a definitive answer on what is causing the gateway to shutdown/fault/lockup?
Dan
[quote=“dfenter”]The manual restarts were performed due to the unusability of the system. It would get hung up on something, and then would lock up.
Can the upgrade be applied live, or does it require downtime? We can’t have the plant down without scheduling it way in advance.
Can you give me a definitive answer on what is causing the gateway to shutdown/fault/lockup?
Dan[/quote]
No, there’s no indication as to what’s causing it. All I can tell is that the JVM stops responding to the service wrapper’s keep-alive ping for some reason. You could try using the GCU to get a thread dump from the gateway before you restart it next time but there’s no guarantee it will work if the JVM isn’t responding. It could also be memory related - how much memory is Ignition configured to use (you can find this in data/ignition.conf, wrapper.java.maxmemory property)?
You won’t be able to upgrade without downtime, but it sounds like you’re already experiencing downtime anyway… assuming nothing goes wrong, an upgrade is just as quick as restarting the gateway/server.