These times are atrocious for a system that should be updating every 800ms. I watched the threads status page and I don't even see this script being called. I imagine that is because it is executing so quickly....once it executed. It's like the valueChange isn't executing right away because the opc read isn't able to acquire the data at the 800ms poll rate. It doesn't see the value update right away and therefore the script isnt called.
Ooooo! Do you have many tag change event scripts? Any with long executions? Or any form of sleep() function calls? You might be backlogging the tag event thread pool. Instead of using a tag event (on the tag), try a gateway tag change event (in the project's scripting/events part of the designer tree). Those don't have the same kind of queuing issues.
(Set it to have a dedicated thread just to be sure.)
If that makes a difference (and I suspect it will), you will need to visit all of your tags to get rid of the pathological algorithms.
(If you are using sleep() anywhere in Ignition, you are usually screwing up.)
Oooh this is interesting. I didn't code the system initially so I can't attest to how many tags are using the valueChange event, but I do believe there are many. One thing I noticed is the tag values seem to stutter in the tag editor window. Like the tags are just straight up not being updated. Would the backlog of change events cause the opc connection to stop updating tags as well?
Generally not, but you are on a buggy version of Ignition. 
Try my experiment first.
Sure thing. I will follow up Monday with the results. Thank you and everyone else chiming in with advice.
Forgive me for what is probably a beginner mistake, but I tried to setup a gateway event and I'm having zero luck getting it to work.
I have the tag being processed as the normal gMES_Debug, i checked its path and its valid.
Here is the script I have executing.
I attempted to put the logger in just to see if it was actually being called. I'm not sure where the logger data can be viewed.
Gateway status page for gateway-scoped scripts.
Yeah that's where I was watching. Doesn't appear to be calling the script. Not sure why, the value is certainly changing.
Look for some other error coming from a syntax error or similar--those show up just once at the time you save the project.
Maybe this is my problem. At what point does the script become active? I assumed it was when I click ok on the gateway event menu. Is it only active once the project is saved?
Project script edits you've OK'd become active when project is saved. The gateway tag change event scripts you setup under project tree are part of the project (unlike tag event scripts defined directly on tags).
1 Like
This was the problem. I wasn't saving the project. I would attempt it without saving it and it didn't work so I disabled the script so as to not cause any unforeseen issues.
It is working now, I did the echo test 10 times...
These response times seem by far more consistent. However, the plant isn't running at capacity right now because its the weekend and therefore ignition isnt loaded down with its other tasks. I will run the test again tomorrow while I am there and we are at fully operation.
Are there any white papers or documentation on best practices on when you should use tag level valueChange scripts vs gateway level valueChange scripts?
Not a white paper, but this thread has some good info (don't stop at first post):
2 Likes
So I was able to move the scripts related to omron over to the gateway events and it helped tremendously. I am going to move the problematic scripts for the Siemens side tomorrow. I'm hoping to see the same drastic improvements.
I'm still having a hard time understanding why it was problematic using the tag events. By default, it looks like tags all each have their own queue (3 I believe). Was it a problem with the tag scripts queueing up in that, or a problem some where else? I'd like to better understand why this change helped so much.
The each have their own queue of length 5. But there are only three threads servicing all of the queues. If any event script bogs down, only two threads will be servicing everything else. If three event scripts bog down, nothing gets serviced.
Do note that if you move long-duration event scripts to the unbounded queues, you can still get in trouble. (Out of memory, in particular.) Plan on rewriting those algorithms.
Is there a way to view the script threads to see which script is causing the hang ups? That way I can target those scripts first.
Not really, no. Export all of your tags and search that for the literal sleep(.
After you've fixed those, add uses of java.lang.System.nanoTime() to capture start and end timestamps. Log the delta at the end of every event. Skip logging for anything shorter than five milliseconds.
You're the man! I really appreciate all of the advice you've provided. I will post with results.
So I've identified a script that is taking in excess of 30s to run. Sometimes takes over a minute. There are 4 tags that call the script. I frequently see the script occupying all 3 of the available threads. I'm thinking this may be the problem.
While I investigate reworking these scripts to make them more efficient, I was thinking about increasing the number of threads available for tag event processing. If I were to increase it from 3 to say 5, it would allow one thread to always be available even if the other 4 get backed up processing the lengthy script. This would be a temporary solution to help our lack of throughput through our process. Run with 5 threads to help get parts out while I rework the problematic script. Once the script is figured out I would change the number of threads back to 3.
My question with this is if I increase the thread count to 5 does it pull resources from else where that might create a problem in itself?