Recently we added new tags to an existing system and now all our expression tag are taking extremely long to evaluate. They are on Tag Groups that execute every second and are mainly functions like .getbit() pointed towards an Int OPC tag in the same UDT. They seem to be taking 15-20 seconds to execute while our OPC tags are doing just fine.
I've tried making a test UDT that has one OPC tag and one exp. The OPC tags reads the datetime from the plc and the expression tag just looks at the opc tag. The expression tag has a tag group all to itself and the same behavior occur. Setting the tag to Fixed Rate or Event Driven doesn't result in any change.
The only thing I can think of is that this recent change understandably drove up our resource consumption on the host. We went from about 25% usage to 50%. Our VM still has a fair bit of overhead and we are getting no clock drift errors or other misc errors in the logs. Am at a bit of a loss, any help would be appreciated.
Look for pathological tag event scripts. Any that take more than single-digit milliseconds to execute should be rewritten and/or moved to project events. Also look for expression tags that use runScript to similarly execute long-running operations. Those should also be moved to project events. In any scripts, look for any .sleep() operations, or while loops that just spin waiting on some conditions. Where waiting is really required, rewrite such using a state machine called from a timer event to check timestamps and conditions.
Do you have a thread dump?
Iād say the most common cause is the use and abuse of runScript to call a script that queries alarm status or history and executes slowly.
1 Like
I've attached the thread dump. Looking through our new udt's we've added and nothing crazy in terms of expressions I don't believe. Your mention of alarms however has reminded me of something else we found. We have these three threads called tag-expressions-1,2,3 that take up a lot of resources and become blocked alot of the time. Also attached one of the threads below.
thread-dump-2023-07-10-082145.json (289.3 KB)
tagexpthread.json (5.0 KB)
This does show an expression using runScript to query alarm status. On all 3 available expression execution threads.
1 Like
One option is call system.alarm.queryStatus once and store the results in a top level variable or "global" document tag. Then reference this variable or tag as needed.
Don't call it from a tag though, call it from a gateway timer script