Looking for advice on system.alarm.queryStatus in Perspective apps

I'm working on a large Perspective app that is a migration from a large Wonderware InTouch app that has thousands of alarm groups with nested hierarchy and animations off both acknowledged and unacknowledged alarm and warning conditions. Basically, devices are solid color if they have an alarm or warning that is acknowledged and they blink if they have an unacknowledged condition.

I know that system.alarm.queryStatus has performance issues if you use it aggressively and they have alarm groups for over 900 different devices for both warnings and alarms. I don't think it will be scalable to use system.alarm.queryStatus on every device to animate based on current alarm conditions.

I'm working out my plan and thought it would be good to ask people with more experience optimizing the use of this method. I have only ever used it sparingly on smaller systems.

My current plan was to have warnings be "Medium" priority and alarms be "High" priority. Then I was going to have a global script that updates the alarm list for all alarms for each condition. I was going to use 4 calls to system.alarm.queryStatus to make independent lists using filters to separate warnings form alarms and acknowledged from unacknowledged.

I'm trying to think of a way I can do it without polling. Possibly with message handlers or something like that. I'm open to all suggestions. I just want to build the best possible solution instead of an "it works" solution.

Thanks

This topic might be helpful.

1 Like

Thanks, that's a good thread.

1 Like

After much reading of various people's threads on optimizing this kind of a script I wrote 2 "working" solutions. One of them performed terribly but it helped me flesh out my plan. Special thanks to @pturmel. So many helpful threads on optimization.

I thought it would be good to include the source for what I ended up doing. Feel free to critique it if I'm doing something subpar. This is my first stab at doing something like this.

I have 2 python lists declared as globals to track changes in active alarm state. This allows me to detect alarm and warning states clearing. The rest is just querying the active alarms and throwing them in their respective buckets. So far it seems to be running great on a 1s timer script with 20 devices. I will load test it when I have more devices implemented.

# This routine does the following:
#	Polls the active alarms and lists 
#	Compares to previous active alarms and list to determine what alarms and warnings have cleared
#	Builds tag write lists to update all effected UDTs  with a single tag write


# filtering out WWTP alarms because they don't follow the EMOS structure
	props = [("alarmGroup", "!=", "WWTP")]
	state = ["ActiveUnacked", "ActiveAcked"]
	
	alarms = system.alarm.queryStatus(state = state, any_properties = props)
	
	# active alarm and warning lists are compared to previous alarm and warning lists to see what alarm conditions need to be cleared
	# the other 4 buckets are used to determine the current state of each device (block writes below)
	retActiveAlarm = []		
	retActiveWarning = []	
	retAlarmUnacked = []	
	retAlarmAcked = []
	retWarningUnacked = []
	retWarningAcked = []
	
	alarmList = alarms.getDataset()
	
	# Sort the results into arrays of strings
	for row in range(alarmList.getRowCount()):
		rowString = alarmList.getValueAt(row, 1)
		
		# The path is the same regardless of what bucket the alarm goes in so I'm doing the parsing in a temp var
		path = rowString[rowString.find(":/tag:") + 6:rowString.find("/alm:") - 1]
		path = '[TagProvider]' + path.rsplit("/")[0] + '/' + path.rsplit("/")[1]
		
		# Warnings
		if alarmList.getValueAt(row, 5) == 2:	
			retActiveWarning.append(path)
			# Unacked
			if alarmList.getValueAt(row, 4) == 2:
				retWarningUnacked.append(path)
			else:
				retWarningAcked.append(path)
		else: # Alarms
			retActiveAlarm.append(path)
			# Unacked
			if alarmList.getValueAt(row, 4) == 2:
				retAlarmUnacked.append(path)
			else:
				retAlarmAcked.append(path)
	
	# Remove duplicates from lists (multiple alarms can be active per device)
	retActiveAlarm = list(set(retActiveAlarm))		
	retActiveWarning = list(set(retActiveWarning))
	retAlarmUnacked = list(set(retAlarmUnacked))	
	retAlarmAcked = list(set(retAlarmAcked))
	retWarningUnacked = list(set(retWarningUnacked))
	retWarningAcked = list(set(retWarningAcked))
	
	tagPaths = []
	values = []
	
	# Build tagpaths and values for things currently in alarm.  It's faster to write current state in a block than to check if it needs to change
	for basePath in retAlarmUnacked:
		tagPaths.append(basePath + '/Ind/Fault Unack')
		tagPaths.append(basePath + '/Ind/Fault')
		values.append(True)
		values.append(True)
		
	for basePath in retAlarmAcked:
		tagPaths.append(basePath + '/Ind/Fault Unack')
		values.append(False)
		tagPaths.append(basePath + '/Ind/Fault')
		values.append(True)
		
	for basePath in retWarningUnacked:
		tagPaths.append(basePath + '/Ind/Warning Unack')
		tagPaths.append(basePath + '/Ind/Warning')
		values.append(True)
		values.append(True)
		
	for basePath in retWarningAcked:
		tagPaths.append(basePath + '/Ind/Warning Unack')
		values.append(False)
		tagPaths.append(basePath + '/Ind/Warning')
		values.append(True)
		
	# get the list of cleared alarms and warnings
	clearedAlarms = [item for item in alarmScripts.gPrevAlarm if item not in retActiveAlarm]
	clearedWarnings = [item for item in alarmScripts.gPrevWarning if item not in retActiveWarning]
	
	# Build tagpaths and values for cleared alarms and warnings
	for basePath in clearedAlarms:
		tagPaths.append(basePath + '/Ind/Fault Unack')
		tagPaths.append(basePath + '/Ind/Fault')
		values.append(False)
		values.append(False)
		
	for basepath in clearedWarnings:
		tagPaths.append(basePath + '/Ind/Warning Unack')
		tagPaths.append(basePath + '/Ind/Warning')
		values.append(False)
		values.append(False)
	
	# Update the previous alarms and warnings global with current alarms and warnings
	alarmScripts.gPrevAlarm = retActiveAlarm
	alarmScripts.gPrevWarning = retActiveWarning
		
	system.tag.writeBlocking(tagPaths, values)

My global declarations look like this

global gPrevAlarm
global gPrevWarning

gPrevAlarm = []
gPrevWarning = []

I put an instance of my alarm state Indicator UDT a folder with other related UDTs for each device. The folder is named after the device. My UDT looks like this (some details removed for this example):

{
  "name": "DeviceAlarmIndicator",
  "parameters": {
    "AlarmGroup": {
      "dataType": "String"
    }
  },
  "tagType": "UdtType",
  "tags": [
    {
      "valueSource": "memory",
      "expression": "{[.]Alarm Update}[1]",
      "dataType": "Boolean",
      "name": "Fault",
      "value": false,
      "tagType": "AtomicTag"
    },
    {
      "valueSource": "memory",
      "expression": "{[.]Alarm Update}[2]",
      "dataType": "Boolean",
      "name": "Warning Unack",
      "value": false,
      "tagType": "AtomicTag"
    },
    {
      "valueSource": "memory",
      "expression": "{[.]Alarm Update}[3]",
      "dataType": "Boolean",
      "name": "Warning",
      "value": false,
      "tagType": "AtomicTag"
    },
    {
      "valueSource": "memory",
      "dataType": "String",
      "name": "Label",
      "value": "TT",
      "tagType": "AtomicTag"
    },
    {
      "valueSource": "memory",
      "expression": "{[.]Alarm Update}[0]",
      "dataType": "Boolean",
      "name": "Fault Unack",
      "value": false,
      "tagType": "AtomicTag"
    }
  ]
}

While I haven't gone through in detail what all you're doing, my initial concern is that doing bulk tag writes will spam your audit log if you're using the audit log to track tag writes. Whereas, an expression tag using runScript won't trigger the tag changes to be logged.

If you look at the scripts in the other linked thread, there's a generic UDT AlarmSummary tag that pulls the relevant info for a given folder, but could probably be modified on a per-device basis (and once 8.3 comes out, this will all get easier and most if not all of it will go away).

I'm not logging tag writes to gateway logs. Am I good to go as long as I'm not logging tag writes to gateway logs?

I might have been doing something wrong. I built a device-based one using that example and was having some performance issues. The tags were polling on 2s and I think the runScripts were competing. That concerned me because I will definitely have problems with 900+ devices if I'm having a problem with 20.

That 8.3 tag folder alarm stats feature would have been very handy for me. No doubt.

Edit: I meant to say audit logs not gateway logs.

The runScripts shouldn't be polling the alarms, but should only be pulling data out of the global variables created by the gateway timer script that updates periodically. They should be pretty fast, but since I was doing it at an area level vs each UDT, I'm not certain there wouldn't be an impact at that larger scale.

I'm fairly certain this was fixed in a later release.

I recall hearing it was going to be fixed, but didn't recall seeing it in a changelog unless I missed it.

I guess they forgot to include it, I haven't confirmed it's fixed, I'm still in v41.