Sorry, Nick. I overlooked your comment about posting items of our project. I can work on that today.
This is a view from my primary front-end Vision gateway. What is the difference between the connection with "ignition-IGNTAGGW" and "ignition-IGNTAGGW-Master?"
Attached is a search for system.alarm.queryStatus.
The first item is a script that I have disabled this morning and then monitored for about 45 minutes. I didn't see any appreciative improvement in the system while it was disabled.
What of those expression binding uses?
A gateway timer script that does one query to support many clients is a gateway timer script you want to keep.
I was curious on how many things hit up the two project scripts.
That timer script resides as a Client Event Script, which made it seem like an obvious potential culprit.
Yes, that would be more workload than a gateway script, but generally not as bad as bindings calling runScript()
. Those tend to proliferate, especially within templates.
I'd comment out all but the timer script (just add a return to the top line of the functions of the project scripts) and see what your performance is like.
The expression bindings appear to be inside a "Alarm Horn" template. But, any deeper search on that template's use turns up nothing.
Thanks for the suggestion. I performed this and monitored for 45 minutes with no change in performance.
I have shared exports of both my front end and back end gateways with IA tech support.
Hmm, well that's surprising, unless they're just not used in the windows at all I've seen clients grind to A halt when calling queryStatus within the clients to get alarm summary info, albeit its usually immediately apparent, not progressively more apparent over a period of time
I have continued to search and search, but no smoking gun, yet. Also, haven't heard anything back IA support that they have found anything.
The system continues to run poorly. I'm close to reverting back to an early October export of the gateways, but I'm not convinced that will solve anything. But, at the time of the exports, the system was at least usable and right now it is not.
My hope is to post an update here at some point when we finally figure out and resolve the issue.
If the backup was done under 8.1.22, but I'm now running 8.1.32, can I still revert to that backup?
Yes, you can't however restore your .32 GWBK into a .22 install, should you wish to go back to that version for some reason.
Yes. there appear to be several items that are not used in the project, which makes troubleshooting more difficult. Working with Travis, I've been pouring over the project as well and runScript, alarmQuery, and these other items aren't obvious culprits, by themselves anyway. One thing I wonder is if we hit some kind of limit on clients...or more along the lines of how the client is launched. many of the clients are single screens, or a dual screen where 1 window is opened on each screen. However, there are a handful of clients that have 1 or 2 50+ inch 4k monitors, with each monitor getting 4 windows. Currently, the main window actually opens in the background, meaning there are 9 active windows on some of these clients. Does anyone have experience with multi-window, multi-monitor clients and if they have experienced any performance impacts? Would a single client driving 2 4k tv's with this many windows pull enough over the gateway network that would impact all clients?
Your performance issues aren't (yet) anything to do with multi-window or multi-monitor. I saw some of your unresponsive EDT thread dumps provided to support, many of them showed the EDT (UI thread) blocked by bindings making network calls to the gateway, e.g. to read a tag value.
Kevin -
I just sent three more EDT files with an explanation of the action carried out prior to receiving the EDT messages. I figured that may help give some additional context.
Lots of single read/writeBlocking calls in a row maybe? Called from property change scripts?
An earlier post showed the number of "inserts/sec" of the Vision (front end) gateway's Alarm Query Provider Service. I'm assuming the values it's been showing (anywhere from 20 to 130) is higher than it should be. I've been trying to find the reason behind this, without success.
So, as a test, assuming those calls are part of the problem, if I pause the Alarm Query Provider Service via the Vision (front end) gateway's Outgoing Queues, should I see an appreciative improvement in client responsiveness?