High memory usage when opening screens – slow heap release and occasional Gateway restarts

Ignition
, ,
1

Hi everyone,

I’m experiencing an issue with high memory usage on an Ignition 8.1.x Gateway, running on Windows Server 2019.

The server has 16 GB of RAM, with 12 GB allocated as the maximum heap for Ignition, while the initial heap is set to 4 GB.

When users open certain screens (Vision / Perspective), the Gateway memory usage increases significantly, which is expected. However, after closing those screens, the memory is not released or takes a long time to decrease (in some cases more than 30 minutes), and does not return to a stable baseline.

With continued system usage and repeated opening of these screens by multiple users, the heap usage keeps growing over time, eventually leading to Gateway restarts caused by OutOfMemoryError in some cases.

Environment Details

  • Operating System: Windows Server 2019

  • Total server memory: 16 GB RAM

  • Ignition maximum heap size: 12 GB

  • Ignition initial heap size: 4 GB

  • Ignition version: 8.1.x

  • Java: Java 11 (bundled with Ignition)

  • Garbage Collector: default configuration (G1GC, to be confirmed)

  • Memory increase is mainly triggered by opening specific screens

  • Screens include:

    • Bindings to Named Queries

    • Historical data queries

    • Components with relatively large datasets

I understand that the JVM does not immediately release memory and that Garbage Collection is demand-driven. However, in this case, the heap does not appear to stabilize, and instead continues to grow until it causes Gateway restarts.

Questions

  1. Is allocating 12 GB of maximum heap to Ignition on a server with 16 GB of RAM appropriate, or is there a more recommended value?

  2. Would it be advisable to increase the initial heap size (currently 4 GB) to improve Garbage Collector behavior and reduce frequent heap expansions?

  3. Are there any recommended JVM flags or Garbage Collector tuning options to improve memory management in Ignition?

  4. What are the best practices to avoid memory retention related to screens, bindings, queries, or scripting?

  5. Is it recommended to enable heap dumps on OutOfMemoryError to diagnose this behavior?

  6. Are there any differences in memory management between Ignition 8.1 and Ignition 8.3 that would justify an upgrade?

  7. Would migrating to a different Java version improve Garbage Collector performance or memory handling?

  8. In cases of excessive memory usage, beyond simply allocating more resources, would implementing Gateway Redundancy help distribute query load and memory consumption, considering that this system is primarily used for monitoring purposes?

Thanks in advance for any guidance or shared experiences.

2

1 - 12GB of ram on a system with 16GB seems appropriate, assuming this system runs ONLY Ignition and nothing else (DB should be on separate, co-located hardware)

2 - Initial heap and max heap should be set to the same value, since Java doesn't return memory to the host os after allocating

3 - G1GC should be enough for most deployments. There are exceptions to this, others are more knowledgable than I regarding this.

4 - There are quite a few, one of the big ones is don't store custom classes in system.util.getGlobals when scripting. Browse/Search the forum, there are many posts regarding memory issues.

5 - This can help point you towards the culprit, I would say its worth enabling

6 - Someone more versed will need to answer this

7 - I doubt it. Memory leaks are almost always due to configuration/programmer error, unless you hit a real specific edge case. Scripting tends to be the main culprit.

8 - This will accomplish nothing in this regard, redundancy does not split load, it just allows fail-over if the primary server fails. Its main purpose is downtime reduction.

1 Like
3

Gateway redundancy is not load balancing. It’s failover. You only have 1 active gateway at a given time. Load balancing is done with a load balancer and multiple full Ignition gateways.

I haven’t had memory problems like what you describe.

I would definitely look at the performance on your named queries. A lot of the time my named queries are just calling a stored procedure in a database. Then I optimize the stored procedure so I know I am getting optimized execution plans and the named query is just an interface to the stored procedure.

Historical data queries can benefit from adding an index on the date stamp field. If your code is spamming historical lookups it can create problems.

Take a look at the diagnostics tools on the gateway. Monitor active scripts and see if you’re capping out the main thread. You can spin off slower-running routines into their own thread to avoid locking that up. I’m wondering if you have processes waiting in line for the main thread.

1 Like