Unexplained high CPU

Ignition
1

I’m running a fresh 8.1.7 install (in trial mode) on a debian VM with 2GB ram. With no project yet built. I only have MQTT Engine/Distributor and MySQL modules running and connected. Three times in the last week, my IT group reported to me that the VM CPU was pegged.

Ignition VM High CPU
Ignition VM High CPU1400×548 54.5 KB

It was reported to me on the 3rd time, and when looking into it, it was definitely the Ignition service loading down the VM. The CPU was at 100% and all memory was used. I looked through the running threads and didn’t see anything that jumped out at me, so I just restarted the service and things went back to normal.

Looking back over log files for something to pin it on, I find the following:

  1. The MQTT Engine was (or became) disconnect from the Distributor
  2. ClockDiftDetector warning in the log files
  3. BasicExecutionEngine error - Task gateway performance monitor threw an uncaught exception.

image
image1173×692 89.2 KB

I’m not sure any of that log info is relevant, but I’d like to get ahead of this before I start any development. It’s obviously a little disconcerting that this would happen before any project was built and/or running.

Is this a known issue in some way? If it happens again, what should I look for?

2

I can’t tell you what wrong at least from this picture, but uploading your wrapper.log file if you have it would be good so others can take a more in depth look at what is going on in your gateway.

3

Logs and a thread dump if it happens again…

4

Here’s a wrapper.log that would include entries during high CPU usage.

wrappper.log.tgz (302.6 KB)

How do I do a thread dump?

5

https://docs.inductiveautomation.com/display/DOC81/Diagnostics+-+Threads#DiagnosticsThreads-IndividualThreads

6

I’m guessing this is related to the MQTT modules some how, since you have nothing else going on with this gateway and the logs are full of MQTT warnings.

7

Okay. I figured as much.

I fixed the CirrusClient connect with retry failed due to Bad user name or password warning and the logs are now quiet. I’ll keep a watch on it.

Thanks!