We recently moved from Windows to Linux - ever since the transition, the Kernal's OOM has been shutting down Java about every day due to the system running out of memory.
Something to take note is that Ignition's gateway webpage for memory usage does not match what we are getting with TOP
Any ideas?
That's because top is showing you java's total process memory, and the gateway is showing you the java heap.
Java never releases heap memory back to the OS. It is vital in production systems that ignition.conf specify the same value for initial and max heap, so Ignition can claim it all up front.
It is also vital that any other processes on that system not use so much memory that the OS runs out. Java will be the biggest user in most systems, so it gets picked as the "problem" child.
Is there any reason why the Java Memory would be higher than the Max Heap setting? (we've been experiencing this).
Yes. Java's native code is loaded into non-heap memory, and various subsystems (like networking) also have some non-heap memory overhead. Any 3rd party JNI libraries will also add non-heap overhead. Typical is 5-10% extra.
You seem to have substantial non-heap memory usage. You may have a leaky JNI library in your system. What 3rd-party modules or JARs are you using?
We upgraded the MQTT Engine module to the latest version and that seems to have resolved the Memory Leak. Thanks @pturmel
