# High CPU Usage - Diagnosing

**URL:** <https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954>\
**Category:** Ignition\
**Created:** [May 8, 2018, 7:30am UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954 "2018-05-08T07:30:58Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [December 12, 2018, 3:27pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/21 "2018-12-12T15:27:26Z")

</div>

Consider including the `-Xloggc:....` and `-XX:+PrintGCDetails` options.

---

<div class="post-metadata">

**Author:** ![oscar.salcedo](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/oscar.salcedo/32/11019_2.png) [@oscar.salcedo](https://forum.inductiveautomation.com/u/oscar.salcedo)\
**Post date:** [December 12, 2018, 4:21pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/22 "2018-12-12T16:21:28Z")

</div>

Got it, this is what it looks like now:

wrapper.java.additional.1=-XX:+UseG1GC  
wrapper.java.additional.2=-XX:MaxGCPauseMillis=100  
wrapper.java.additional.3=-Ddata.dir=data  
wrapper.java.additional.4=-Dorg.apache.catalina.loader.WebappClassLoader.ENABLE\_CLEAR\_REFERENCES=false  
wrapper.java.additional.3=-Xloggc:…/logs/javagc-%WRAPPER\_TIME\_YYYYMMDDHHIISS%.log  
wrapper.java.additional.5=-XX:+PrintGCDetails  
wrapper.java.additional.6=-XX:+PrintGCTimeStamps  
wrapper.java.additional.7=-XX:+PrintGCDateStamps

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [December 12, 2018, 4:38pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/23 "2018-12-12T16:38:09Z")

</div>

The “additional” numbers must be unique. After cutting and pasting the lines, renumber the ones that aren’t commented out.

---

<div class="post-metadata">

**Author:** ![oscar.salcedo](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/oscar.salcedo/32/11019_2.png) [@oscar.salcedo](https://forum.inductiveautomation.com/u/oscar.salcedo)\
**Post date:** [December 12, 2018, 6:28pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/24 "2018-12-12T18:28:30Z")

</div>

Boy am I dense today! Thanks, corrected it.  
A quick update: our ESX engineers moved 6 (out of 9 total) other VMs that were on our same host to a different one, and this happened:

We will have to define some parameters for them to isolate our VMs our guarantee resources. Is there a set of best practices for running Ignition in a virtualized environment?

Thanks!  
Oscar.

---

<div class="post-metadata">

**Author:** ![oscar.salcedo](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/oscar.salcedo/32/11019_2.png) [@oscar.salcedo](https://forum.inductiveautomation.com/u/oscar.salcedo)\
**Post date:** [December 14, 2018, 1:43pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/25 "2018-12-14T13:43:34Z")

</div>

An update on this and seeking some more help:

We added two vCPUs to our VM yesterday and restarted the Gateway after the updates to the ignition.conf to use the new Garbage Collector exclusively.

Here’s what our ignition.conf looks like:  
wrapper.java.additional.1=-XX:+UseG1GC  
wrapper.java.additional.2=-XX:MaxGCPauseMillis=100  
wrapper.java.additional.3=-Ddata.dir=data  
wrapper.java.additional.4=-Dorg.apache.catalina.loader.WebappClassLoader.ENABLE\_CLEAR\_REFERENCES=false  
wrapper.java.additional.5=-Xloggc:…/logs/javagc-%WRAPPER\_TIME\_YYYYMMDDHHIISS%.log  
wrapper.java.additional.6=-XX:+PrintGCDetails  
wrapper.java.additional.7=-XX:+PrintGCTimeStamps  
wrapper.java.additional.8=-XX:+PrintGCDateStamps

After adding the 2 vCPUs (6 vCPUs total), our CPU usage drop considerably:

 ![](https://us1.discourse-cdn.com/inductiveautomation/original/2X/f/fc9fdfc147926c70273bef8da3de77bdf560f6ca.png)

However, our memory profile is looking a bit odd, and we had 2 crashes overnight (we had to restart the gateway).  
Our Memory profile before the GC changes:  
 ![](https://us1.discourse-cdn.com/inductiveautomation/original/2X/8/8ccd8626e2c2ff6bd44f0c41b006bdd79cfc0757.png)

and after:  
 ![](https://us1.discourse-cdn.com/inductiveautomation/original/2X/3/38a006bfc41c48b6ecb54236be83f98857f6dd03.png)

We did not get a change to grab a Thread Dump, but the logs indicate issues connecting to devices (OPCUA) and memory full events for alarms.  
It also looks like this line:  
wrapper.java.additional.5=-Xloggc:…/logs/javagc-%WRAPPER\_TIME\_YYYYMMDDHHIISS%.log  
is not working as I don’t see any java logs in the logs folder? Maybe this could be causing some overhead?

Thanks,

Oscar.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [December 14, 2018, 4:48pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/26 "2018-12-14T16:48:17Z")

</div>

> [@oscar.salcedo](#):
>
> We did not get a change to grab a Thread Dump, but the logs indicate issues connecting to devices (OPCUA) and memory full events for alarms.

So using G1GC with a pause target really smoothed out your memory profile. It's not clear to me why you have no GC log file. There's probably an error in the wrapper log at startup indicating the problem with that parameter. Consider using a full path to the log file.  
The memory full indication suggests that something is happening that spikes memory usage and 8GB of RAM isn't enough to handle it. Whether it is a bug causing a memory leak or a legitimate need for lots of memory (I'm looking at you, reporting!), I can't tell. There are probably clues in your wrapper log. Can you give the VM a bunch more memory (increasing the max in ignition.conf accordingly) to see if you can ride through the spike? If so, and the profile returns to the vicinity of 6GB afterwards, then I would suggest it is a legitimate load.

---

<div class="post-metadata">

**Author:** ![oscar.salcedo](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/oscar.salcedo/32/11019_2.png) [@oscar.salcedo](https://forum.inductiveautomation.com/u/oscar.salcedo)\
**Post date:** [December 14, 2018, 6:02pm UTC](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954/27 "2018-12-14T18:02:19Z")

</div>

Hi,  
I will try that; I’ll report back with the results of increasing the RAM.  
One of the common errors we saw in the logs was around alarm journaling; we disabled it around 9:30AM this morning and that helped the load. This is likely not the culprit, but most definitely a contributor.

You are spot on on the GC log file… I am trying using backslashes for the path next (…\logs\javagc…)  
INFO | jvm 1 | 2018/12/13 12:27:56 | Java HotSpot™ 64-Bit Server VM warning: Cannot open file …/logs/javagc-20181213122746.log due to No such file or directory

Oscar.

[Previous page](https://forum.inductiveautomation.com/t/high-cpu-usage-diagnosing/17954.md?page=1)
