Hi,
Can someone help me, our Ignition Master Server keeps on crashing and based on the performance, there are times where it uses almost 90% of CPU and memory.
Thanks.
What are you doing when the memory usage skyrockets? Are you allowing large time spans on process charts? Especially in Perspective?
Or running complex reports (with similar large data loads)?
How many users does this gateway have, and which UI technologies are involved? (Vision vs. Perspective.)
Do you have scheduled scripts doing bulk work at these times?
(Side note: 8GB of RAM for Ignition is on the small side for typical systems now-a-days.)
What are you doing when the memory usage skyrockets? Are you allowing large time spans on process charts? Especially in Perspective?
- during the time it happened, I am not doing anything that could trigger the memory to spike, for process charts we normally have minute and 10 second sampling time for our process tags.
Or running complex reports (with similar large data loads)?
- our reports usually just runs every Monday for weekly reports.
How many users does this gateway have, and which UI technologies are involved? (Vision vs. Perspective.)
- we normally have 25-30 perspective sessions opened up at he same time
Do you have scheduled scripts doing bulk work at these times?
- we have no running scripts at the moment
You probably want to open a support ticket. Those crashes probably created a dump file that would be worth analyzing.
1 Like
Is your gateway hosted on a virtual machine? If it is, is the VM host dedicating a fixed resource pool to the Ignition VM? Are you sure it is?
If you are certain it's not a particular Ignition activity leading to the spike, another possible culprit is an otherwise overtaxed system struggling to manage Ignition and putting Ignition on a lower priority than some other more aggressive workload.
I'll definitely echo the suggestion to contact support.
1 Like
Hi,
Yes we are running the Ignition server on a VM host. I'll create a ticket right now.
Thanks.
Is it possible to change the role of the back up server into master server role and the current master will be the back up server.
Yes, but not without downtime. A backup license cannot run as a master, so you have to move both licenses. And make both role changes while not licensed. Have support look over your shoulder for that, too, if it is critical.
(Or use a third system during the shuffle.)
do you mind if I send you the thread dump file or wrapper log files now so you can also assess what is wrong with our system?
I don't provide free help in private, generally. You can attach files here if you are comfortable with the content in public. (And I'm bailing out for the day now--I'm in Eastern Time.)
1 Like
The resource monitor will always show Ignition using its allowed memory, plus some overhead. Ignition's memory usage graph is entirely internal to java--java doesn't give any "free" memory back to the OS. (That is, the resource monitor in the OS is useless.)
Why is Chrome running on the gateway server? (A production gateway should never be used as a client or designer.)
Google Chrome was just opened up to check the gateway status.
Google Chrome was just opened up to check the gateway status.
Almost all of Ignition configuration and monitoring can be done via the gateway configuration web pages on a remote client.