Gateway Timer Events vs. Threading?

[byrnep]

Hi all,

So having been a little let down by the behavior of Gateway Events in V8.0:

Gateway Timer Event Scripts in 8.0.6

So, we just upgraded from 7.9 to 8.0 and most things are working well. First things first, the online manual has a typo: https://docs.inductiveautomation.com/display/DOC80/Gateway+Event+Scripts The timer scripts example has a Hello World example that will not compile system.util.getLogger(“TestScript”).info.(“Hello”) should be system.util.getLogger(“TestScript”).info(“Hello”) To get to the more complicated issue, I’m seeing our Gateway timers executing far more rapidly than they should be.…

And lacking a solution for a few weeks I reverted to the following solution to get the 'trigger' property on a Sound Player to toggle on and off: I basically spawn a thread with an enclosed sleep timer, which on initiation sets a property on the SoundPlayer and on conclusion resets it. I have a propertyChanged script which tracks changes in numbers of alarms, as well as the current state of the thread which times out the alarm.

It works as well as the old gateway timer did, and given the divergences (in my opinion) between how Gateway Events work now and how the 8.0 manual says they ought, I'd prefer to leave my solution in place, since I 100% understand how it works and I control how or if its behavior changes. I'm really only looking for serious concerns about possible mismanagement of memory or other resources. This is the first time I've mucked around with threads in Python, and I haven't much experience with them in other languages either. So I can very easily see myself setting up another issue down the road.

Apologies for the unPythonic Grammar and the probably needlessly long code:

import time
from threading import Thread

def sleepFunc(secs,triggerState):
        #This function waits, then sets two Sound Player properties
	time.sleep(secs)
	event.source.trigger = triggerState
	event.source.ThreadClear = True

if event.propertyName == 'AlarmTotal':
       #if there are Alarms, turn on RingToggle and set the Sound Player on.
       #if there aren't, turn off RingToggle and trigger on the Sound Player
	if event.newValue != 0:
		event.source.RingToggle = True
	else:
		event.source.RingToggle = False
    event.source.trigger = bool(event.source.AlarmTotal)

elif (event.propertyName == 'trigger') or (event.propertyName == 'ThreadClear'):
        #Once 'Ring Toggle' is on the player will cycle 'trigger' on and off till it is removed
        #Each cycle will be spawned as a thread, and a new one will not start till the old one
        #concludes.
	if event.source.ThreadClear:
		event.source.ThreadClear = False
		if ( (event.source.AlarmTotal != 0) and (event.source.trigger) ):
			alarmThread = Thread(target = sleepfunc, args = (0.25,False) )
			alarmThread.start()
		elif ( (event.source.AlarmTotal != 0) and (not event.source.trigger) ):
			alarmThread = Thread(target = sleepfunc, args = (1.75,True) )
			alarmThread.start()
        if event.source.AlarmTotal = 0:
            event.source.trigger = False

[pturmel]

There’s nothing wrong with using java threads from jython in Ignition, as long as you take care to destroy and recreate them upon script module edits, and take care to only create the number of threads desired.

If the thread will run indefinitely, you need some form of object tracking in the persistent dictionary from system.util.getGlobals(). But:

Since gateway event script behavior hasn’t changed in v8 other than the addition of inheritance, and you had working scripts in v7.9, you probably don’t need to do any this. (Go back and look at Kevin’s comment about how you messed up your global project.)

[byrnep]

Thanks Phil,

Again, there’s nothing I did to the global project; I neither created it, nor assigned it inheritors, nor gave my gateway events to it. to be run simultaneously by all inheritors. The behavior may be standard, but it was a shift from the behavior in the previous version, one that if it was flagged I didn’t notice it. It also doesn’t seem to be documented in the current 8.0 manual discussion of how gateway events function.

While I’m aware of how to detangle my gateway events from my client events now, under the new 8.0 rubric, I’m leery of such changes to how things work coming through on future updates, and I’d prefer in that case to keep any unexpected behavior due to bugs in my own, self-maintained code. This was a minor issue, but it was a minor issue in our plant’s alarming project, and that’s a larger cause for concern.

I believe my current code, which only permits a thread to be created if the prior thread has terminated successfully (swapping the ThreadComplete flag on the component as last command) should limit myself to one thread at a time. Again I was concerned about committing some obvious ‘newb’ error that would eventually crash my gateway, but it sounds like we should be fine. There’s no indication that memory useage on the gateway has been anything but pancake flat for the last several days. More granular analysis, for windows 10 at least, seems to require installation of a program called ‘process explorer’. I’d like to keep the gateway as bare of extraneous software as possible however.

[nminchin]

@byrnep

process explorer

I don't know if you're aware, but this app is only a few kB it's also designed as a portable app, so no need to install

[nminchin]

@pturmel

as long as you take care to destroy and recreate them upon script module edits

Hi Phil, is the reason for this simply in case the code running in the thread has been modified, so it must be destroyed and re-started up again to start running the new definition?

[pturmel]

Big memory leak if you don't clean out threads running old code--the entire interpreter is stuck in memory (and running) until all of its threads die. (And all persistent objects that point at any old code objects are gone, too.)

[nminchin]

Ah, not good!

Although now I have more questions...
In this example here:

Managing multiple asynchronous threads

No, more like this: persist = system.util.persistent("someUniqueConstantName") oldFileWatcher = persist.pop("fw") persist['fw'] = FileWatcher(oldFileWatcher) def getWatcher(): return persist['fw'] Note the oldFileWatcher passed to the new one so the new one’s thread can tell it to shut down.

i.e.

persist = system.util.persistent("someUniqueConstantName")
oldFileWatcher = persist.pop("fw")
persist['fw'] = FileWatcher(oldFileWatcher)

def getWatcher():
    return persist['fw']

My understanding from this is that when the script engine starts up again after saving the project, the new engine gets the old engine's file watcher object in order to interrupt it, kill it, and then replace it. What I don't understand though is how two interpretters can be running side-by-side with the ability to talk to each other? i.e. how can "fw" in the new interpretter point to the old interpretter's object? To me, this is like the Designer's running Python code referencing the gateway's running Python code, which I didn't think was possible. e.g. like system.util.getGlobals('var1') being accessible as the same object from both Designer and Gateway scopes. I'm obviously missing something fundamental here...

[I think it's time to finally do a course that goes through the nitty gritty of Python...]

[pturmel]

The two interpreters are running in the same JVM, so not like designer vs. gateway scopes at all.

And jython is fully multi-threaded. Persistent dictionaries like system.util.getGlobals() and my alternatives are java objects, sharable across interpreters.

If you never start your own asynchronous threads, and never put things in persistent dictionaries, and never add listeners to gateway scope infrastructure, and never use infinite loops, interpreter cleanup is assured.

But that is quite the set of handcuffs.

(The nitty gritty details of python won't help--jython is a separate implementation.)

I should mention that when java infrastructure calls a jython code object that is implementing a java interface, the calling thread will adopt that code's declaring intepreter and drive on. If a new interpreter calls an old code object through persistence, then old code will run in the new interpreter and things get weird.

[nminchin]

Ok, that makes a lot more sense now, thanks!!

Actually one more thing. What are the issues with system.util.getGlobals()?

[pturmel]

It is actually the same dictionary that holds global names in the legacy-scoped gateway events (all of them together, across all projects). So, lots of opportunities for clashes. To avoid that:

• Make all gateway events one-liners calling project library scripts.

• Use a better persistent object storage tool. * Cough *

[nminchin]

In your https://www.automation-pros.com/ignition/queueUtil.py, is there anything here that needs to added to protect from this? (I presume not, but just thought I'd ask).

The managementMap in the global dict (queueUtilPersistence) is a dict whose keys contain LinkedBlockingQueue Java objects. The init method on the WrappedBlockingQueue, which adds new LBQ objects into the global dict, first checks if the key already exists, and if it does it just sets the class' _queue class variable's value to the existing queue (see arrow in screenshot). This is the part i'm not sure about. Wouldn't this then be using the old interpretter's version? or does it not matter because these are Java objects?

Is there any way to tell how many interpretter instances are running?

@pturmel

Use a better persistent object storage tool. * Cough *

What does your system.util.getGlobalVars() implementation solve/improve that limits getGlobals()? (in layman's terms if possible )

[pturmel]

@nminchin

tell how many interpretter instances

Not that I know of, at runtime. A heap dump would show something.

@nminchin

What does your

1. Independent of legacy scope,

2. Requires an application key string to separate uses.

3. Delivers a java ConcurrentHashMap instead of a python dictionary, to expose extra methods. Particularly .putIfAbsent() and .computeIfAbsent().