Power Table configureCell intermittently returns the wrong result for up to the entire table

liam.mcglaughlin · August 3, 2020, 7:42pm

I have a power table that has an issue with the formatting. The table has a column with timestamps, 10 value columns, then 2 limit columns. The configure cell code (below) should reference the limit columns for the value columns. However, it will at times mark all of the value columns as being out of spec.

Something odd, that seems likely to be involved, the minimum and maximum "<>"s need to be reversed from what they should be to display properly when it is working.

Anecdotal observations:

It shows up sometimes when the window is first started, other times it starts fine.
It will sometimes show up after running fine for a while.
It eventually seems to resolve on its own, for a while.
I have at times been able to query times when it had happened and put that on the table and it showed up again. Other times I have tried and the table looked fine.

Here is the configureCell code, it’s loaded down trying to catch exceptions to figure out what is going on. I have not seen any exceptions with or without the error handling. It ‘thinks’ everything is working fine.

        from java.lang import Exception
	from traceback import format_exc as exc
	try:
		value = float(value)
	except:
		pass
	try:
		# timestamp, min, max columns
		if colIndex in [0, 11, 12]:
			return {'background': 'black','foreground' : 'white'}
		# empty
		elif value is None:
			return {'background': 'black','foreground' : 'white'}
		# zero values
		elif value == 0:
			return {'background': 'black','foreground' : 'grey'}
		# out of spec for min
		elif self.data.getValueAt(rowIndex, 11) > value:
			return {'background': 'red','foreground' : 'blue'}
		# out of spec for max
		elif self.data.getValueAt(rowIndex, 12) < value:
			return {'background': 'red','foreground' : 'blue'}
		# anything else
		else:
			return {'background': 'black','foreground' : 'white'}	
	except Exception, e:
		print('java error', exc())
		logr = system.util.getLogger("Lam mahlo chart cell config script")
		logr.info(exc())
		return {'background': 'orange','foreground' : 'green'}
	except:
		print('error', exc())
		logr = system.util.getLogger("Lam mahlo chart cell config script")
		logr.info(exc())
		return {'background': 'orange','foreground' : 'green'}

The columns when everything is within spec.

The columns when something is out of spec.

The columns when things have been going wrong. This is from a video where the formatting is beginning to show properly, from the bottom, one row at a time every few seconds.
Table Example 02
Screen captures of it today:

Sanderd17 · August 4, 2020, 6:57am

This looks like a timing issue to me. The code being called while the data is updated, or something like that.

Are you using special scripts (perhaps with async code or separate threads) to load the data?

In any case, I would log (or just print) the timestamp, value, min and max of the red cells. Then you can see what values were found during the running of the script, and check if those are still the same in the table.

Note that the configureCell function doesn’t run on all cells, but only those in view. So if data gets loaded in a way that it doesn’t get registered, scrolling would indeed rerun the configureCell, and correct the mistakes on cells that come into view.

liam.mcglaughlin · August 5, 2020, 4:14pm

The dataset is generated using a script. It is then written to a dataset tag that the table’s data is bound to. I’ve added logging as you’ve suggested and I’ll update when I see what that tells me. Thanks for the advice.

martinel3746 · August 5, 2020, 5:44pm

The orange is the result of an exception - I would remove the exception clause and discover what conditions are causing the exception(s).

Another thought is to initialize the return dictionary at the beginning and set member values along the way then return the dictionary at the end. That will allow you to set background based on one condition, foreground on another condition etc. and you can expand formatting to include things like bold or italics as additional visual indicators on a cell.

liam.mcglaughlin · August 5, 2020, 7:55pm

The background color in the cells is red, with blue text (it does look orange in the photograph), the orange background with green text was so I could see if there was an error. It hasn’t ever shown up in production.

Adding to the dictionary as you’d mentioned sounds like a good way to go about things. Though, it doesn’t seem like it would change anything regarding the issue I’m having.

I did have the thought about the oddness with the reversed '<>'s. I’m going to try adding a condition to show if they have swapped back to normal during that time. Because that would make sense in at least some of the instances. (As much as boolean operators intermittently reversing meaning could make sense.)

Sanderd17 · August 6, 2020, 8:48am

Can you show that part of the script? And how the script is triggered?

liam.mcglaughlin · August 6, 2020, 1:03pm

The update_mahlo_ds() is called from a client timer script every 10s, fixed delay, shared. Takes ~115ms.

ptag is a convenience wrapper class. It uses readBlocking and writeBlocking system.tag functions.

def ds2p(ds):
	"""Get a mutable python list of list s and  the headers from a dataset."""
	return [[col for col in row] for row in system.dataset.toPyDataSet(ds)], ds.getColumnNames
	
def dedupe(dset, check_col=0):
	"""Remove rows with duplicate values in the provided column from Python 2d list, in place.
	
	Keep the last row where the column value (default column index 0) is the same as the previous
	row.
	"""

	# this provides the normal index as it iterates over the list backwards
	for rn, row in reversed(list(enumerate(dset))):
		# if it's not the last index
		if rn == len(dset) - 1:
			pass
		# if the next higher index row has the same value in the column, delete this row
		elif dset[rn + 1][check_col] == row[check_col]:
			del(dset[rn])

def update_mahlo_ds():
	"""Updates the client dataset with the latest data. Called by client timer script."""
	
	from Shared.L import ptag
	from traceback import format_exc as exc
	from pprint import pprint
	
	# for timing the script
	st_time = system.date.now()
	try:
	
		######################################
		##   Get the Mahlo thickness Data   ##
		######################################
		
		# get this laminator's number
		lamnum = ptag("[client]ClientGlobals/Laminator Number").value()
		print(system.date.now(), 'mahlo ds update, lamnum ', lamnum)
		
		# put together the base tagpath for zone tags for this lam
		prototag = "[default]MAHLO/LAM{ln}/Zone".format(ln=lamnum)
		
		# generate the list of zone tags to query
		qtags = [prototag + str(x) for x in range(0, 10, 1)]
		
		# when to query to/from
		endTime = system.date.now()
		startTime = system.date.addMinutes(endTime, -30)

		# run the query
		mahlo_ds = system.tag.queryTagHistory(paths=qtags, startDate=startTime, endDate=endTime, returnSize=-1, returnFormat='Wide', noInterpolation=True)
		
		##########################################
		##   Add the target thickness columns   ##
		##########################################
		# unfortunately the thickness data above is needed as 'on change' where the targets
		# are needed with the last value, so they must be queried seperately and then combined
		
		# the tags to query
		targ_thicc_tags = ["[default]LAM{ln}/Set by Ignition/Recipe Tags/Current Min Spec Thickness".format(ln=lamnum), "[default]LAM{ln}/Set by Ignition/Recipe Tags/Current Max Spec Thickness".format(ln=lamnum)]
	
		# go further back for these since they could be the same for a long time
		startTime2 = system.date.addHours(endTime, -8)
		
		# run the query
		target_thicc_ds = system.tag.queryTagHistory(paths=targ_thicc_tags, startDate=startTime2, endDate=endTime, returnSize=0, aggregationMode="LastValue", returnFormat='Wide', noInterpolation=True)
		
		# make the min and max into python lists to add to the thickness dataset
		tds, thd = ds2p(target_thicc_ds)
		
		# as the data goes in on change and the tag values are actually being changed at different times
		# we get back a 'half changed' row which would likely cause issues with the table formatting
		# so lets get rid of duplicate rows
		dedupe(tds)
		
		mds, mhd = ds2p(mahlo_ds)
#		print('pre-append targets mds')
#		for row in mds[:30]:
#			print(row)
		
		# target thickness row number
		trn = 0
		for mrn, row in enumerate(mds):
			looking = True
			# if mahlo data tstamp is after target data tstamp
			while looking:
				if row[0] > tds[trn][0]:
					trn += 1
				
				# if target data ts is equal or after mahlo data tstamp
				else:
					# append the thickness data to the row
					mds[mrn].append(tds[trn][1])
					mds[mrn].append(tds[trn][2])
					looking = False
					
#		print('post append targets mds')
#		for row in mds[5:15]:
#			print(row)

		# the new headers to display in the table
		headers = ['Timestamp', 'Zone 0', 'Zone 1', 'Zone 2', 'Zone 3', 'Zone 4', 'Zone 5', 'Zone 6', 'Zone 7', 'Zone 8', 'Zone 9', 'min', 'max']
		
		# turn it back into a dataset
		final_ds = system.dataset.toDataSet(headers, mds)
		
		# write it to the dataset tag
		ptag("[client]ClientGlobals/Dataset Tags/Mahlo Dataset Tag").write(final_ds)	
		
	except Exception:
		from traceback import format_exc as exc
		msg = 'Mahlo dataset update script error: ' + exc()
		print(system.date.now(), msg)
		log = system.util.getLogger(system.util.getProjectName() + ' - ' + __name__)
		log.info(msg)
	
	finally:
		run_time = system.date.millisBetween(st_time, system.date.now())
		print(system.date.now(), 'mahlo ds update, lamnum ', lamnum, 'script ran in {} ms'.format(run_time))