Perspective Workers High CPU Usage

Greetings,

My organization has an Ignition Gateway instance running version 8.0.3-rc2 (b2019072512).

The Java version is 11.0.3+7-LTS running on an Ubuntu Linux instance on amd64.

On average the entire Gateway runtime is eating up ~80% CPU usage. The significant offenders appear to be perspective-worker threads, as shown:

Our team will attempt to disable a potential resource intensive page and investigate further.

I don’t think this should be normal behavior and figuring out what specific actions are causing this is very important.

Taking a thread dump while those worker threads are in a high CPU state would be helpful.

dev-ignition1_thread_dump.txt (101.0 KB)

Yes, that was one thing I forgot to include! This state is more or less a constant, so there is no specific window where the CPU usage spikes - it’s constantly ~80% with + 20% -10% spikes.

Hmm, one possibility might be a circular property reference. We try to prevent you from doing that with direct property bindings, but there are still some circumstances where it’s possible. If you’re able to, try isolating it down to a certain view/page configuration/etc and start troubleshooting from there.

This definitely isn’t something we would expect - there’s always going to be some load on the gateway from running the Perspective backend, but it shouldn’t be nearly this heavy.

I’ll dig a little bit to see if there’s a circular property reference.

@PGriffith Your guess was correct! Took much digging, but it was a poorly implemented checkbox that caused my issues.

Would there ever be a push for a more in-depth diagnostics page to see which bindings / transforms / change scripts are eating up processor time?

Thank you for your help!

1 Like

Good to hear!
Our first priority would probably actually be to try to prevent you from doing such things in the first place - they’re obviously not intentional, just tend to happen over time. If they do happen, then they should (ideally) gracefully fail and give some kind of indication about why to the designer.

Long term, yes, the general Perspective debugging experience is something we’re planning to improve.

3 Likes

A post was split to a new topic: Diagnosing high CPU Perspective worker thread