Ignition Service Stop

Hello guys,
I currently have a problem with the use of Ignition. Where the use of memory is suddenly high until it causes Services to stop.
I have tried submitting tickets and troubleshooting in meetings to find a solution to this problem.
Together with Inductive, we are troubleshooting from checking the Wrapper Log and VMWare Configuration. From these results, no significant problems were found, then directed to check the processes in Ignition such as the use of Tags, Scripts, Expressions and others.
Performance conditions when Ignition occurs problems, as follows:

When an Issue occurs, Waiting Threads can reach more than 100,000 with the following Running Script conditions:

After that I tried to find a temporary solution by increasing RAM and Maximum Memory Usage in Configuration.
Before: RAM : 32Gb, Max Memory: 20Gb
After : RAM : 48Gb, Max Memory : 32Gb.
The addition did not produce any results.

I also attached Wrapper Log, Ignition.Conf..
From the above conditions, can you help me to take action that can be done. Because this issue has been going on for 1 month and the information why Running Script runs can be too high to consume large RAM.
wrapper.zip (187.0 KB)

Plain threads like that do not get created by Ignition itself. Something in your own code in Ignition is doing that. Or perhaps in a third-party module.

The wrapper log is not what needs to be inspected. It is your entire codebase in your Ignition project(s). You probably won't get that for free, on this forum or elsewhere.

2 Likes

Thank you for your advance,
After I break down all of the projects, I notice several projects using time.sleep function in their script. And I realized in the forum you answered that if that function can impact it, it will freeze the user interface.

Your memory is increasing over time. It should look like a sawtooth pattern.

Increasing RAM is just increasing the time before a crash.

If your VM has 48 GB, I recommend setting ignition.conf to 16 GB for both initial and max. Your memory is circa 8GB before the leak starts.

There is a server sizing guide that can help with this.