Script to populate data is not finding the tag data

Ignition
,
1

I will do my best to describe what is happening. I am trying to figure out someone else's code here and I am quite a novice when it comes to Python scripting.
The person who wrote this originally is using a script to get a list of inverters from a custom property. Then it creates a list of tags and then queries the tags. When I look at the props where the script is bound, I can see the tag names but the values are null. I am hoping someone can help me in the right direction to start diagnosing this issue. I will paste the script and some screenshots that will hopefully be helpful. The table that is supposed to display looks like it has an overlay issue but it is not clear.

def transform(self, value, quality, timestamp):
	
	invList = value.instances
	
	#queryTags = ["ACTIVE_POWER","DC_POWER","DC_CURRENT_IN_A","AC_CURRENT","VAB","VBC","VCA","INVERTER_DC_VOLTAGE","STATUS_1","NORMALIZED_STATE","TOTAL_ALARM_COUNT","CURTAILMENT"]
	queryTags = ["GRID_POWER_P","DC_INPUT_POWER","DC_INPUT_CURRENT","GRID_CURRENT_AVG","GRID_VOLT_RS","GRID_VOLT_ST","GRID_VOLT_TR","DC_VOLTAGE_PV_VDC","STATUS_1","NORMALIZED_STATE","TOTAL_ALARM_COUNT","EVENT_STATE","CURTAILMENT","NUM_RUNNING_MODULES","NUM_OUT_OF_SERVICE_MODULES"]
	queryLen = len(queryTags)
	tags = []
	
	#build tag list
	for inv in invList:
		for q in queryTags:
			tags.append('Inverters/'+inv+'/'+q)
	
	#read values	
	results = system.tag.readBlocking(tags)
	retList = []
	
	#evaluate values
	def eval(val):
		if val == None:
			return ''
		return {'value':val.value,'style':{'classes':'tblErr' if str(val.getQuality()) != 'Good' else ''}}

#custom code to highlight NUM_RUNNING_MODULES column if value in cell is < 6
	def evalModules(val,num):
		if val == None:
			return ''
		return {'value':val.value,'style':{'classes':' tblErr' if str(val.getQuality()) != 'Good' else ('tblFault' if val.value != num else '')}}
#end of custom code to highlight NUM_RUNNING_MODULES column if value in cell is < 6

	#built results
	for i in range(len(invList)):
		ind = i * queryLen
			
		retList.append(
			{'Inverter':invList[i],
			'Active_Power': eval(results[ind]),
			'DC_POWER': eval(results[ind+1]),
			'DC_CURRENT_IN_A': eval(results[ind+2]),
			'AC_CURRENT': eval(results[ind+3]),
			'VAB': eval(results[ind+4]),
			'VBC': eval(results[ind+5]),
			'VCA': eval(results[ind+6]),
			'DC_VOLTAGE': eval(results[ind+7]),
			'STATUS_1': eval(results[ind+8]),
			'NORMALIZED_STATE': eval(results[ind+9]),
			'TOTAL_ALARM_COUNT_UNACK':eval(results[ind+10]),
			'EVENT_STATE':eval(results[ind+11]),
			'CURTAILMENT':eval(results[ind+12]),
			'NUM_RUNNING_MODULES':evalModules(results[ind+13],6),
			'NUM_OUT_OF_SERVICE_MODULES':evalModules(results[ind+14],0)
			})
	
	return retList

The table has a binding on the Prop "data" and as you can see from the image below, it gathers the inverter name but the tag data is null. I have verified the data is good in the tag browser.

image

Below is an image of what the table looks like currently. There is no indication as to what the red triangle with exclamation mark is indicating.

image
image1099×448 11.5 KB

Any help would be appreciated. Please be kind. I am still learning.

2

Can you hit the pencil icon below the post to edit it, select the block of code and then press the </> button. It will preserve the indentation and apply syntax highlighting.
Thanks.

3

I would add system.perspective.print(str(tags)) right before the #read values line and ensure the paths being read are actually what you think they are. I can come up with a lot of other ways to improve this code, but in the interest of expediency, that's where I'd start.

5

I will give that a try. I would be interested in your code improvement ideas when you have time. I have had to fix other areas of " code flexing" as I like to call it.

6

Where should I see the paths being read? I added the line but I don't see any difference in the output.

7

I tried your suggestion. It appears that indentation is present. Are you seeing what you expect to see now?

8

You can see the output in the console, either in the designer (ctrl + shift + c) or in a browser (ctrl + shift + i).

As for code improvement, I'll give it a shot when I have enough time, but the first thing I'd do is remove the comments that don't match what the code actually does :X

#custom code to highlight NUM_RUNNING_MODULES column if value in cell is < 6
	def evalModules(val,num):
		if val == None:
			return ''
		return {'value':val.value,'style':{'classes':' tblErr' if str(val.getQuality()) != 'Good' else ('tblFault' if val.value != num else '')}}
#end of custom code to highlight NUM_RUNNING_MODULES column if value in cell is < 6

I'd also avoid using eval as an identifier, as it's already a built-in python function.

9

@pascal.fragnoud , in the console, how do you duplicate the value that the script would be seeing from the custom parameter? IE, view.cusom.instances. This custom value contains the inverter list that changes depending on a drop down selection.

10

Alright, I'm taking a coffee break before SQL melts my brain. Let's see if we can rewrite this a bit - refactoring always chills me down.
Disclaimer: I might take things a bit too far. Stop where you're confortable, find your sweet spot between refactor and readability.

first, we can online the path tags build, so

tags = []
for inv in invList:
	for q in queryTags:
		tags.append('Inverters/'+inv+'/'+q)

becomes

tags = ["Inverters/{}/{}".format(inv, q) for inv in invList for q in queryTags]

I also prefer .format over concatenation, but use your favorite method

I'd also group the functionality of the 2 functions, which are close enough (it's not always a good idea, but this will be helpful later):

def make_item(val, num=None):
	if val is None:
		return ''
	if val.quality.notGood:
		style_class = 'tblErr'
	elif num is not None and val.value != num:
		style_class = 'tblFault'
	else:
		style_class = ''
	return {
		'value': val.value,
		'style': {
			'classes': style_class
		}
	}

This could probably be refactored as well. Maybe later.

Next step is to make the last loop is bit more user friendly. Using indices here is annoying.
To take chunks off of an iterable, you can use this:

from itertools import izip_longest as zipl
def chunks(iterable, size):
	return zipl(*[iter(iterable)] * size)

I have a version of this function in my project libraries, but if you don't plan on using it elsewhere, just define it there.
Here's an example of what it does:

x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
list(chunks(x, 3))
list(chunks(x, 4))

[ [1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12] ]
[ [1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12] ]

Gotta use list() here, because chunks itself is a generator.

So now we can do this:

for _ in chunks(results, len(queryTags)):
    build_the_return_list

In that loop, instead of building the dictionaries manually, we can first define all the item names, then zip them with the return values of the make_item function.

items_names = ['Inverter', 'Active_Power', 'DC_POWER', 'DC_CURRENT_IN_A', 'AC_CURRENT', 'VAB', 'VBC', 'VCA', 'DC_VOLTAGE', 'STATUS_1', 'NORMALIZED_STATE', 'TOTAL_ALARM_COUNT_UNACK', 'EVENT_STATE', 'CURTAILMENT']
retList = []
for inv in chunks(results, len(queryTags)):
	items = dict(zip(items_names, map(make_item, inv[:-2])))
	retList.append(items)

Okay so things start to become a bit weird here if you're not used to it.
we have a list with the items names. We need to make a list of the same length with the values we want to associate with those names.

map(function, list)

Will call function, using each item in list as the argument.
We use it on every item of a chunk, except the last 2, because we need to pass another parameter for those last 2 items.
We can do the same thing for them:

modules_names = ['NUM_RUNNING_MODULES', 'NUM_OUT_OF_SERVICE_MODULES']
for inv in chunks(results, len(queryTags)):
	modules = dict(zip(modules_names, map(make_item, inv[-2:], [6, 0])))

Will call make_item twice, passing it inv[-2], 6 then inv[-1], 0
To make a dictionary out of this, we zip the items names and the list returned by map, then pass this to dict.
Example, zip(['a', 'b', 'c'], [1, 2, 3]) will produce a list of tuples like this [('a', 1), ('b', 2), ('c', 3)],
then dict will transform those to {'a': 1, 'b': 2, 'c': 3}.

We can them join the 2 dictionaries with update:

retList = []
for inv in chunks(results, len(queryTags)):
	items = dict(zip(items_names, map(make_item, inv[:-2])))
	modules = dict(zip(modules_names, map(make_item, inv[-2:], [6, 0])))
	items.update(modules)
	retList.append(items)

We can make this a bit better, by building a list of values that will be passed as the second argument to make_elem:

check_list = [None] * (len(items_names)-2) + [6, 0]
for inv in chunks(results, len(queryTags)):
	item = zip(items_names, map(make_item, inv, check_list))

Which also means we can inline this so we don't have to append to a list:

return [
	dict(
		zip(items_names, map(make_item, inv, check_list))
	) for inv in chunks(results, len(queryTags))
]

Full script:

def transform(self, value, quality, timestamp):
	def chunks(iterable, size):
		return zip(*[iter(iterable)] * size)

	def make_item(val, num=None):
		if val is None:
			return ''
		if val.quality.notGood:
			style_class = 'tblErr'
		elif num is not None and val.value != num:
			style_class = 'tblFault'
		else:
			style_class = ''
		return {
			'value': val.value,
			'style': {
				'classes': style_class
			}
		}

	invList = value.instances
	queryTags = ["GRID_POWER_P","DC_INPUT_POWER","DC_INPUT_CURRENT","GRID_CURRENT_AVG","GRID_VOLT_RS","GRID_VOLT_ST","GRID_VOLT_TR","DC_VOLTAGE_PV_VDC","STATUS_1","NORMALIZED_STATE","TOTAL_ALARM_COUNT","EVENT_STATE","CURTAILMENT","NUM_RUNNING_MODULES","NUM_OUT_OF_SERVICE_MODULES"]
	tags = ["Inverters/{}/{}".format(inv, q) for inv in invList for q in queryTags]
	results = system.tag.readBlocking(tags)

	items_names = ['Inverter', 'Active_Power', 'DC_POWER', 'DC_CURRENT_IN_A', 'AC_CURRENT', 'VAB', 'VBC', 'VCA', 'DC_VOLTAGE', 'STATUS_1', 'NORMALIZED_STATE', 'TOTAL_ALARM_COUNT_UNACK', 'EVENT_STATE', 'CURTAILMENT', 'NUM_RUNNING_MODULES', 'NUM_OUT_OF_SERVICE_MODULES']
	check_list = [None] * (len(items_names)-2) + [6, 0]

	return [
		dict(
			zip(items_names, map(make_item, inv, check_list))
		) for inv in chunks(results, len(queryTags))
	]

huh, that coffee break was a bit long.
Also note that I didn't run any of this, so there are probably a few things to fix (typos, etc) before it's functional.

3 Likes
11

The simplest thing would be to print that list from the transform itself, I guess.
Or maybe I'm missing something ?

In a broader way, you can use a value change script on any property, to run code when the value of that property changes. You can print things from there.

Note that in the script in the post above, I left invList = value.instances as it is, expecting value to come from a structure binding, but if only the inverters list changes, a simple property binding on that custom property is enough, and you can use invList = value.

2 Likes
12

@pascal.fragnoud I like your suggestions. The short answer I discovered was the script was not invoking the default provider for whatever reason. If I add opening and closing square brackets in front of the path on line 14 tags.append("[]Inverters/"+inv+"/"+q), then it adds the default provider to the tag path and the values come in as expected.
I am going to take a lunch break and try to absorb your refactor. Thanks for the input.

13

I changed a little bit in the full script:
I was using izip_longest, because that's what I have in my library:

from itertools import izip_longest as zipl

def chunks(iterable, size, fval=None):
	return zipl(*[iter(iterable)]*size, fillvalue=fval)

But here you don't need it, as the chunks should always be the same length, so a simple zip is enough.

If you think you might chunk things again in the future, I suggest adding the full function in your library.

1 Like