Way to maintain column types when splitting a dataset?

Ignition
1

Have a large dataset full of alarm data that has headerds and datatypes like

headers = ['Alarm Occured At', 'Alarm Cleared At', 'Alarm Name', 'Username', 'AllData']
datatypes = [class java.util.Date, class java.util.Date, class java.lang.String, class java.lang.String, class com.inductiveautomation.ignition.common.alarming.BasicAlarmEvent]

I am attempting to measure downtime based on custom alarm categorization. I have the following code -

from com.inductiveautomation.ignition.common.alarming.config import CommonAlarmProperties

CATEGORIES = {
	'CAT_1': {'title':'Down Time Due to Machine'},
	'CAT_2': {'title':'Down Time Due to Other Reasons'},
	'CAT_3': {'title':'Category 3'},
	'CAT_4': {'title':'Category 4'},
	'CAT_5': {'title':'Category 5'},
	'CAT_6': {'title':'Category 6'},
	'CAT_7': {'title':'Category 7'},
	'CAT_8': {'title':'Category 8'},
	'CAT_9': {'title':'Category 9'},
	'CAT_10': {'title':'Category 10'}
}

ALARM_CATEGORIZATIONS = {'machine/PLC/Alarms/AIR_PressureLow': 'CAT_6',
 'machine/PLC/Alarms/GEN_NoBatch': 'CAT_8',
 'machine/PLC/Alarms/GEN_NoRecipe': 'CAT_8',
 'machine/PLC/Alarms/HOME_Failed': 'CAT_6',
 'machine/PLC/Alarms/HOME_Required': 'CAT_2',
 'machine/PLC/Alarms/IORackCommStatus0': 'CAT_1',
 'machine/PLC/Alarms/MC_MotorPosRstRequired': 'CAT_10',
 'machine/PLC/Alarms/MOD_StatusAlarm_1': 'CAT_9',
etc
....}

def generateDowntimeDataForBatch(startingDate, endingDate):
	"""
	Grabs all alarms from a specific batch and categeorizes downtime based on the above ALARM_CATEGORIZATIONS
	dictionary above.
	Args:
		batchNum: str, whats the batch?
	Returns:
		List of [Datasets]
	"""
#	startingDate, endingDate = reporting.batch.getBatchStartAndEndDate(batchNum)
	ds = reporting.common.getAlarmsDataSet(startingDate, endingDate, extended=True)
	# Step 1 - run through dataset and replace any Nones with batch start or end time when necessary
	for (row, (AlarmOccuredAt, AlarmClearedAt, AlarmName, UserLoggedIn, AllData)) in enumerate(system.dataset.toPyDataSet(ds)):
		if AlarmOccuredAt is None:
			# This alarm started before batch and was cleared during the batch, we use batch start time
			ds = system.dataset.updateRow(ds, row, {'Alarm Occured At':startingDate})
		if AlarmClearedAt is None:
			# this alarm started before end of batch and did not clear
			ds = system.dataset.updateRow(ds, row, {'Alarm Cleared At':endingDate})
	
	# Don't want to accidentally mess up our categories variable
	resultingData = CATEGORIES.copy()
	for categeory, categoryData in resultingData.items():
		categoryData['data'] = system.dataset.toDataSet(list(ds.getColumnNames()),[])
	# Now we take the full set of alarm data, look at how the alarm is categorized, and split them up
	for (AlarmOccuredAt, AlarmClearedAt, AlarmName, UserLoggedIn, AllData) in system.dataset.toPyDataSet(ds):
		# 1 - get tag path so we can get alarmCategory and what not
		itemPath = str(AllData.get(CommonAlarmProperties.ItemPath))
		# 2 - Get Categorization of said alarm
		try:
			alarmCategorization = ALARM_CATEGORIZATIONS[itemPath]
			resultingData[alarmCategorization]['data'] = system.dataset.addRow(resultingData[alarmCategorization]['data'], [AlarmOccuredAt, AlarmClearedAt, AlarmName, UserLoggedIn, AllData])
		except KeyError, e:
			print "Could not find %s in ALARM_CATEGORIZATIONS"%(itemPath)
	# 
	return ds, resultingData

And this almost worked. I do get a result like

{'CAT_1': {'data': Dataset [50R ⅹ 5C], 'title': 'Down Time Due to Machine'},
 'CAT_10': {'data': Dataset [19R ⅹ 5C], 'title': 'Category 10'},
 'CAT_2': {'data': Dataset [155R ⅹ 5C],
           'title': 'Down Time Due to Other Reasons'},
 'CAT_3': {'data': Dataset [29R ⅹ 5C], 'title': 'Category 3'},
 'CAT_4': {'data': Dataset [7R ⅹ 5C], 'title': 'Category 4'},
 'CAT_5': {'data': Dataset [15R ⅹ 5C], 'title': 'Category 5'},
 'CAT_6': {'data': Dataset [46R ⅹ 5C], 'title': 'Category 6'},
 'CAT_7': {'data': Dataset [5R ⅹ 5C], 'title': 'Category 7'},
 'CAT_8': {'data': Dataset [5R ⅹ 5C], 'title': 'Category 8'},
 'CAT_9': {'data': Dataset [205R ⅹ 5C], 'title': 'Category 9'}}

so it split it nicely. However, now I am writing the part where I go through each categories downtime to sum up time differentials between alarm occured and alarm cleared and I find that all of them are unicode, every single column is of type unicode and so I can’t subtract the dates without doing a system.date.parse, however depending on the requirements I may also need the last column to remain as BasicAlarmEvent if possible as well. Is there a way to accomplish this ? Perhaps I need to split my dataset a little bit different? Not sure where to go from here.

2

At this point, there should be a community repository for easily browsable community scripts.

def createDataset(headers, rows, types):
	from com.inductiveautomation.ignition.common.util import DatasetBuilder
	builder = DatasetBuilder.newBuilder()
	builder.colNames(headers)
	builder.colTypes(types)
	for row in rows:
		builder.addRow(row)
	return builder.build()
2 Likes
3

I always forget about Dataset Builder. Thank you.