Perspective Project Script Persistance

Ignition v8.1.35

I have a project for Zebra mobile scanners. Each scanner has it's own respective UDT that it reads/writes to using the scannerNumber. The scannerNumber is set by an onActionPerformed event on a Begin button that is on the starting page of the project. This starting page is used to either get a :scannerNumber URL parameter if it were present, if not an entry field appears allowing the user to manually enter a number.
I was asked if I could get/display the scanners gps coords on a map, and I figured out how to do that.
I have a timer script that sends a message to all sessions, causing them to grab their current gps coords, and write them to the respective tag.

I would like to instead save the gps coords in a class instance and persist the scanners dictionary across script restarts.

Reasons being:

  • Having multiple sessions call system.tag.writeBlocking() once they receive the message from the gateway, does not sound efficient. There could be at least 20 scanners online, and possibly more in the future.

  • Allowing a new session to then create it's own Scanner instance automatically, essentially removes the need to have a UDT for each scanner and not have to create/delete tags manually or via scripting. Although it can be done.

Now, I made two classes that I create an instance of in Perspective Project Scripts.
The newScanner() function is called from the same onActionPerformed event on the Begin button.
I also have a removeScanner(scannerNumber) function that will delete the scanner from the scanners dictionary when a session is closed.

How could I (or should I) persist the scanners dictionary across script restarts?

class Gps:
	def __init__(self, lat, lng, tstamp):
		self.lat = lat
		self.lng = lng
		self.tstamp = tstamp

class Scanner:
	
	def __init__(self, session):
		self.session = session
		self.name = 'Scanner #{0}'.format(str(session.custom.scannerNumber))
		self.gps = Gps(session.props.geolocation.data.latitude, session.props.geolocation.data.longitude, session.props.geolocation.data.timestamp)
		
	def __str__(self):
		return 'Scanner# {0}, Session ID:{1}, IP: {2}'.format(self.session.custom.scannerNumber, self.session.props.id, self.session.props.address)
		
	def __eq__(self, scannerToCompare):
		if isinstance(scannerToCompare, Scanner):
			return self.session.custom.scannerNumber == scannerToCompare.session.custom.scannerNumber
		return False
	
	def getName(self):
		return self.name
		
	def getSession(self):
		return self.session
		
	def getScannerNumber(self):
		return self.session.custom.scannerNumber
		

scanners = {}

def newScanner(session):
    if session:
        new_scanner = Scanner(session)
        scanners[new_scanner.getScannerNumber()] = new_scanner
        refreshScannerTable()
        refreshScannerGps()

def removeScanner(scannerNumber):
    if scannerNumber in scanners:
        del scanners[scannerNumber]
        refreshScannerTable()
        refreshScannerGps()
        

def getScanner(scannerNumber = None):
    if scannerNumber is None:
        return list(scanners.values())
    else:
        return scanners.get(scannerNumber, None)

def updateScannerGps(scannerNumber, lat, lng, tstamp):
    if scannerNumber in scanners:
        scanner = scanners[scannerNumber]
        scanner.gps.lat = lat
        scanner.gps.lng = lng
        scanner.gps.tstamp = tstamp
        
        refreshScannerGps()
        
def getScannerGps():
    scannersData = []
    for scanner in scanners.values():
        scannerData = {
            "name": scanner.getName(),
            "properties": {},  
            "enabled": True,
            "lat": scanner.gps.lat,
            "lng": scanner.gps.lng,
            "opacity": 1,  
            "icon": {
                "path": "material/location_on",  
                "color": "#4190F7",  
                "rotate": 0,  
                "size": {
                    "width": 30,  
                    "height": 30  
                },
                "style": {
                    "classes": ""  
                }
            },
            "event": {
                "stopPropagation": False  
            },
            "tooltip": {
                "content": {
                    "text": scanner.getScannerNumber(),  
                    "view": {
                        "path": "",  
                        "params": {}  
                    }
                },
                "direction": "top",  
                "permanent": False,  
                "sticky": False,  
                "opacity": 0.7  
            },
            "popup": {
                "enabled": False,  
                "content": {
                    "text": scanner.getName(),  
                    "view": {
                        "path": "",  
                        "params": {}  
                    }
                },
                "width": {
                    "max": 300,  
                    "min": 50  
                },
                "height": {
                    "max": None  
                },
                "pan": {
                    "auto": True  
                },
                "closeButton": True,  
                "autoClose": True,  
                "closeOnEscapeKey": True,  
                "closeOnClick": None  
            }
        }
        scannersData.append(scannerData)
    return scannersData
   

def refreshScannerGps():
	local.util.sendMessageToSessions('refreshScannerGps', payload = None)

def getScannersTable():
    scannersData = []
    for scanner in scanners.values():
        data = {
            "scannerNumber": scanner.getScannerNumber(),
            "sessionId": scanner.getSession().props.id,
        }
        scannersData.append(data)
    return scannersData
    
def refreshScannerTable():
	local.util.sendMessageToSessions('refreshScannerTable', payload = None)

You cannot persist class instances across script restarts in Ignition. That leaks huge amounts of memory, as the instances hold onto the old code versions, which holds onto the entire script interpreter and all of its old code modules.

Store only native python or java collection types in persistent JVM objects. At any nesting depth.

1 Like

Thanks, Phil.

Now that you say that, I don't have to save the class instances. I just need the data to be saved.

Technically, I don't need classes at all. Just a dictionary containing each scanner's dictionary of values.

I can save that to getGlobals() and reinitialize my scanners dictionary after restart.
Any update to a scanner or new scanners coming online, will update the getGlobals() variable.

Are the variables in getGlobals() there forever? Is it a good idea to clean it up manually?

For the life of the JVM itself. Empty on gateway restart. When initializing, be sure to use python dictionaries' .setdefault() method to place items in a race-free way.

Be aware that script module initialization can run in parallel across projects and scopes--locking (also prepped with .setdefault()) is sometimes needed. Weird errors are possible (especially failed name lookups for things like _25_15 that appear to be line and character positions). Arguably, this is a bug, but appears to be intractable.

1 Like

Phil,
Would you agree that storing/retrieving the scanner data from a dictionary in project scripts would be better than reading/writing to tags?

I am not sure how to gauge when "too many is too many" when it comes to read/writing to tags.

Always? No. Mostly, yes. Particularly as a custom cache for a database.

Memory tags survive gateway restart, and are synchronized to a peer in a redundant peer. No JVM variable does that. But both disk and network traffic are involved, and there's a always a short window where a failure or failover could lose data.

Memory tags propagate naturally to all interested parties through bindings. The dictionary from system.util.getGlobals() cannot do that. The dictionaries from my own system.util.globalVarMap() can do that through the corresponding expression function. (Similar for the Perspective-specific session/page/view VarMaps.)

Only actual database operations have long-term persistence. If the life of the JVM suits your needs, a persistent dictionary has huge performance advantages over any other method.

Thanks again, Phil.
Updated code:

scanners = {}

def newScanner(session):
    if session:
    	newScanner = {
    		'session': session,
    		'scannerNumber': session.custom.scannerNumber,
    		'gps': {
    			'lat': session.props.geolocation.data.latitude,
    			'lng': session.props.geolocation.data.longitude,
    			'tstamp': session.props.geolocation.data.timestamp
    		}
    		
    	}
        
        scanners.setDefault(session.custom.scannerNumber, newScanner)
        refreshScannerTable()
        refreshScannerGps()

def removeScanner(scannerNumber):
    if scannerNumber in scanners:
        del scanners[scannerNumber]
        refreshScannerTable()
        refreshScannerGps()

def getScanner(scannerNumber = None):
    if scannerNumber is None:
        return list(scanners.values())
    else:
        return scanners.get(scannerNumber, None)

def updateScannerGps(scannerNumber, lat, lng, tstamp):
    if scannerNumber in scanners:
        scanner = scanners[scannerNumber]
        scanner['gps']['lat'] = lat
        scanner['gps']['lng'] = lng
        scanner['gps']['tstamp'] = tstamp
        
        refreshScannerGps()

The .setDefault() function is "thread safe" for adding a new scanner to the dict. What would be a "thread safe" method for removing scanners as their sessions shutdown/expire? As in my function removeScanner().

What about a session needing to update it's respective scanner's data?
Does the below hold true?

From: Python objects and thread safe - Ignition - Inductive Automation Forum

1 Like

I usually use .pop() to remove items in shared dictionaries. No error if something has already deleted it.

Yes, you are responsible for your own locking, if needed. Python objects being thread-safe just means python itself won't crash if two threads access the same object. Your algorithm might crash, though.

Gotcha.
The way I understand it now is that I don't have to use a lock in order to safely create/edit/delete scanners in my application, as I have seen used in other threads and examples.

A threading.Lock() would provide insurance on top of any inherent thread safe ability with the datatypes in my application.

Unless, you mean that the dict is "thread safe" in that Jython/Python just doesn't care if multiple threads access it at the same time.

This, really. Jython implements those base data types with java ConcurrentMap and ConcurrentList objects. Those will efficiently moderate changes to content from multiple threads, preventing its own crashes, but provide no other guarantee.

Thanks, Phil.

I appreciate you.