# High CPU and Memory Usage

**URL:** <https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966>\
**Category:** Ignition\
**Created:** [January 16, 2025, 11:16am UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966 "2025-01-16T11:16:55Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![edward.fowler](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/edward.fowler/32/24179_2.png) [@edward.fowler](https://forum.inductiveautomation.com/u/edward.fowler)\
**Post date:** [January 16, 2025, 11:16am UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/1 "2025-01-16T11:16:55Z")

</div>

What's the best way to determine the cause of high CPU and memory usage?  
Thank you.

Ignition v 8.1.44  
Ubuntu 22.04.5 LTS

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/7/4/740c3ae4206dcc0b73dedda971544dacb21a57ad.png)

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/0/9/098a02c75e6c6bf6b61634ab7f81c500bca3aa05.png)

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [January 16, 2025, 1:26pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/2 "2025-01-16T13:26:36Z")

</div>

After the fact, with no prep, there's nothing to see.

I recommend running a timer event that uses java's MX bean infra to check CPU and memory usage every second, and grab a thread dump any time either is greater than some value ~95%.

You could also run a continuous profiler, but those tend to be fairly heavy-weight.

Edit: Here's what I use to take heap dumps in crises. You can use as is, or perhaps alter to only do thread dumps:

```python

# Copyright 2023 Automation Professionals, LLC <sales@automation-pros.com>
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
# SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
# OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
# OF SUCH DAMAGE.

from java.lang.management import ManagementFactory, MemoryMXBean
from com.sun.management import HotSpotDiagnosticMXBean

logger = system.util.getLogger(system.util.getProjectName() + '.' + __name__ )

mbServer = ManagementFactory.getPlatformMBeanServer()

diagMxBean = ManagementFactory.newPlatformMXBeanProxy(mbServer, "com.sun.management:type=HotSpotDiagnostic", HotSpotDiagnosticMXBean)

memMxBean = ManagementFactory.getMemoryMXBean()

def heapUsedNow():
	usage = memMxBean.getHeapMemoryUsage()
	if usage.max > 0:
		return 1.0 * usage.used / usage.max
	return 0.0
#
#
priorDumpTagPath = '[default]Diagnostics/HeapDumpInhibitTS'
dumpThresholdTagPath = '[default]Diagnostics/HeapDumpThreshold'
def checkMemForDump():
	now = system.date.now()
	inhibitTS, threshold = [x.value for x in system.tag.readBlocking([priorDumpTagPath, dumpThresholdTagPath])]
	if now.after(inhibitTS):
		# dump is allowed
		if heapUsedNow() > threshold:
			system.tag.writeBlocking([priorDumpTagPath], [system.date.addSeconds(now, 30)])
			fnBase = '/usr/share/ignition/diag_dump_' + system.date.format(now, 'yyyyMMdd_HHmmss')
			diagMxBean.dumpHeap(fnBase + '.hprof', False)
			system.file.writeFile(fnBase + '.threads', system.util.threadDump())
#

#
# kate: tab-width 4; indent-width 4; tab-indents on; dynamic-word-wrap off; indent-mode python; line-numbers on;
#

```

---

<div class="post-metadata">

**Author:** ![Steve\_Laubach](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/steve_laubach/32/60900_2.png) [@Steve\_Laubach](https://forum.inductiveautomation.com/u/Steve_Laubach)\
**Post date:** [January 16, 2025, 2:32pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/3 "2025-01-16T14:32:44Z")

</div>

That will likely come in handy down the road. Thanks Phil!

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [January 16, 2025, 3:18pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/4 "2025-01-16T15:18:30Z")

</div>

I should mention that taking a heap dump is itself extremely heavyweight, and produces a JVM stall all by itself. Don't enable the timer event that calls that check function unless you are actively tracking a problem. Think of it as exploratory surgery.

---

<div class="post-metadata">

**Author:** ![Steve\_Laubach](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/steve_laubach/32/60900_2.png) [@Steve\_Laubach](https://forum.inductiveautomation.com/u/Steve_Laubach)\
**Post date:** [January 16, 2025, 3:39pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/5 "2025-01-16T15:39:22Z")

</div>

Noted. Thanks again.

I saved a link to this post in case someone calls me in to troubleshoot this kind of problem. It happens from time to time.

---

<div class="post-metadata">

**Author:** ![avaughn](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/avaughn/32/38972_2.png) [@avaughn](https://forum.inductiveautomation.com/u/avaughn)\
**Post date:** [January 16, 2025, 3:59pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/6 "2025-01-16T15:59:51Z")

</div>

Starting with version 8.1.13, automatic thread dumps can also be enabled under Gateway Settings.

> **[Gateway Settings | Ignition User Manual](https://docs.inductiveautomation.com/docs/8.1/platform/gateway/config/gateway-settings#automatic-thread-dump)**
>
> Gateway Settings Property Reference

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [January 16, 2025, 4:06pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/7 "2025-01-16T16:06:43Z")

</div>

> [@avaughn](#):
>
> automatic thread dumps

Those help for pure performance problems, but performance problems accompanying memory exhaustion are nigh-impossible to figure out without a heap dump.

---

<div class="post-metadata">

**Author:** ![avaughn](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/avaughn/32/38972_2.png) [@avaughn](https://forum.inductiveautomation.com/u/avaughn)\
**Post date:** [January 16, 2025, 4:09pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/8 "2025-01-16T16:09:21Z")

</div>

Agreed! I probably should have clarified that it would only help troubleshoot high CPU usage.

---

<div class="post-metadata">

**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\
**Post date:** [January 16, 2025, 4:28pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/9 "2025-01-16T16:28:15Z")

</div>

I'll also point out for posterity that if you set a flag prior to launching the gateway, you can capture heap dumps in a Diagnostics Bundle:

> **[Status | Ignition User Manual](https://www.docs.inductiveautomation.com/docs/8.1/platform/gateway/status#diagnostic-bundle-export)**
>
> Your Gateway at a Glance

Doesn't help with an _immediate_ problem diagnosis, but can be a useful thing with a known-problematic system.

---

<div class="post-metadata">

**Author:** ![Chris\_Bingham](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@Chris\_Bingham](https://forum.inductiveautomation.com/u/Chris_Bingham)\
**Post date:** [January 16, 2025, 4:40pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/10 "2025-01-16T16:40:16Z")

</div>

Interesting. Thanks for sharing!

> [@pturmel](#):
>
> uses java's MX bean infra to check CPU and memory usage every second

I'm curious how this method differs from the `[System]Gateway/Performance/CPU Usage` (& similar) tags 🤔, and if negligible, whether you would recommend the Gateway Timer Script over a similar Gateway Tag Change script monitoring the aforementioned tag(s)?

---

<div class="post-metadata">

**Author:** ![edward.fowler](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/edward.fowler/32/24179_2.png) [@edward.fowler](https://forum.inductiveautomation.com/u/edward.fowler)\
**Post date:** [January 16, 2025, 4:40pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/11 "2025-01-16T16:40:48Z")

</div>

Thank you @pturmel , much appreciated.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [January 16, 2025, 4:54pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/12 "2025-01-16T16:54:13Z")

</div>

The MX bean is where the tag gets its value. So there's more overhead in the tag subsystem. I'd not use a tag event for this. Just use a timer.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [January 16, 2025, 4:56pm UTC](https://forum.inductiveautomation.com/t/high-cpu-and-memory-usage/100966/13 "2025-01-16T16:56:20Z")

</div>

> [@paul-griffith](#):
>
> set a flag prior to launching the gateway

Noted. My script can be deployed without a gateway restart, which can be a problem for production sites.