Looking for advice on dynamic Pen creation in the Power Chart component

[Jacob_Barlow]

I have a client requesting the ability to add/remove set categories of tags on the Power Chart, I have a rough working method where I browse for all the tags of that type depending on the site selected, and append the JSON object with "name" and "source" parameterized using substrings from the tag paths. See below for code:

basePath = self.session.custom.basePath
pens = []
ambTemp = system.tag.browse(basePath, {'name':'AMBIENT_TEMP*', 'tagType':'AtomicTag', 'recursive': True})
for pyr in ambTemp:
	name = str(pyr['fullPath'])[38:]
	source = str(pyr['fullPath'])[5:]
	#ignore the weird quality check for now 
	#was trying to find a way of creating pens for tags with Bad quality without breaking the chart
	if str(pyr['value'].quality).find("Bad_") == -1:
		if str(pyr['value'].quality).find("Good") != -1:
			pens.append(
						{
						    "name": name,
						    "visible": True,
						    "enabled": True,
						    "selectable": True,
						    "axis": "Axis 1",
						    "plot": 0,
						    "display": {
						      "type": "line",
						      "interpolation": "curveLinear",
						      "breakLine": True,
						      "radius": 3,
						      "styles": {
						        "normal": {
						          "stroke": {
						            "color": "#DE7E41",
						            "width": 1,
						            "opacity": 0.8,
						            "dashArray": 0
						          },
						          "fill": {
						            "color": "#DE7E41",
						            "opacity": 0.8
						          }
						        },
						        "highlighted": {
						          "stroke": {
						            "color": "#DE7E41",
						            "width": 1,
						            "opacity": 1,
						            "dashArray": 0
						          },
						          "fill": {
						            "color": "#DE7E41",
						            "opacity": 1
						          }
						        },
						        "selected": {
						          "stroke": {
						            "color": "#DE7E41",
						            "width": 1,
						            "opacity": 1,
						            "dashArray": 0
						          },
						          "fill": {
						            "color": "#DE7E41",
						            "opacity": 1
						          }
						        },
						        "muted": {
						          "stroke": {
						            "color": "#DE7E41",
						            "width": 1,
						            "opacity": 0.4,
						            "dashArray": 0
						          },
						          "fill": {
						            "color": "#DE7E41",
						            "opacity": 0.4
						          }
						        }
						      }
						    },
						    "data": {
						      "source": "histprov:Stellar_ROC:/drv:default:default:/tag:HISTORIAN/Stellar_ROC/"+source,
						      "aggregateMode": "Average"
						    }
						  }
						)
self.parent.parent.getChild("PowerChart").props.pens = pens

Basically I have an onActionPerformed event on a button that will fire the tag browse, fill in the list with array objects, and set the chart's Pens property to that list of arrays. If I weren't building this out for multiple sites I'd just set custom properties with static lists and have that be that but it has to handle dynamic sets of tags. My problem right now is there are some categories that are returning 100+ results and looping through for all of that is really slow.

I'm wondering if there's a way to maybe store these lists of arrays to a set of memory tags in each site's tag folder and just have the button event set the chart's pens property equal to the tag? Or if there's an altogether better way to approach this whole thing.

[pascal.fragnoud]

@Jacob_Barlow

My problem right now is there are some categories that are returning 100+ results and looping through for all of that is really slow.

Looping through things isn't slow in itself. And 100+ iterations is nothing. The operations you might do for each iteration, on the other hand, could be slow. But you're not doing anything particularly expansive, so I'd say what's slow is the tag browse.

@Jacob_Barlow

I'm wondering if there's a way to maybe store these lists of arrays to a set of memory tags in each site's tag folder and just have the button event set the chart's pens property equal to the tag? Or if there's an altogether better way to approach this whole thing.

Storing a list of tag paths can speed things up. It doesn't need to be a tag, though. Phil might suggest a dictionary/list at the top level of your script library. Maybe a simple custom property on the powerchart could be enough.

A few things about your script:

• be careful with indentation, some lines are a mix of tabs and spaces.
• name = str(pyr['fullPath'])[38:] => name = pyr['name']
• In general, when extracting a part of a path, be careful with hardcoded slices (what if something in the path changes and the name doesn't start at the 38th character ?). Splitting on a fixed separator is usually a better idea, and better yet, the BasicTagPath type has some methods you can use to extract parts of the path: BasicTagPath
• if str(pyr['value'].quality).find("Bad_") == -1: => if pyr['value'].quality.isGood()
• if str(pyr['value'].quality).find("Good") != -1: => if pyr['value'].quality.isNotGood()

[pturmel]

@pascal.fragnoud

Phil might suggest a dictionary/list at the top level of your script library.

Yes, if scripting anyways. If avoiding scripting to avoid its startup penalty, the lookup() expression function with a dataset tag is a performant option.

@pascal.fragnoud

pyr['value'].quality.isGood()
pyr['value'].quality.isNotGood()

No, this:

pyr['value'].quality.good
not pyr['value'].quality.good

Use the NetBeans shortcuts everywhere applicable, not just .value or .quality, as they run faster and are more readable. Even better is to only check .good in the if clause and use an else: clause for not good.

[pascal.fragnoud]

I hesitated, but I still find quality.isGood() is...I don't know, there's something about being able to read "quality is good" that I really like. I doubt the performance hit is noticeable on such a small scale.

[pturmel]

@pascal.fragnoud

I doubt the performance hit is noticeable on such a small scale.

Yeah, but code gets copied and pasted into situations where the scale is no longer small. I like consistently applied rules of thumb, in this case for both the performance and the readability that comes with terse expressions.

[Jacob_Barlow]

@pascal.fragnoud

Looping through things isn't slow in itself. And 100+ iterations is nothing. The operations you might do for each iteration, on the other hand, could be slow. But you're not doing anything particularly expansive, so I'd say what's slow is the tag browse.

This particular example only yields 5-15 results, and takes roughly 2-3 seconds. I have, for instance, a category for tracker controllers (Grid-Scale Solar PV), and that browse yields somewhere north of 150 results, taking upwards of 2 minutes to fully render the pens.

@pascal.fragnoud

Storing a list of tag paths can speed things up. It doesn't need to be a tag, though. Phil might suggest a dictionary/list at the top level of your script library. Maybe a simple custom property on the powerchart could be enough.

If I store the lists of tags to be added to pens in a set of custom property on the chart, that would increase the load time of the component itself, right? The lists would change every time the selected site is changed. And eliminating the tag browse for the button action would help but I would still need a for loop to create the list of JSON arrays to add to the pen property?

@pascal.fragnoud

A few things about your script:

• be careful with indentation, some lines are a mix of tabs and spaces.
• name = str(pyr['fullPath'])[38:] => name = pyr['name']
• In general, when extracting a part of a path, be careful with hardcoded slices (what if something in the path changes and the name doesn't start at the 38th character ?). Splitting on a fixed separator is usually a better idea, and better yet, the BasicTagPath type has some methods you can use to extract parts of the path: BasicTagPath
• if str(pyr['value'].quality).find("Bad_") == -1: => if pyr['value'].quality.isGood()
• if str(pyr['value'].quality).find("Good") != -1: => if pyr['value'].quality.isNotGood()

• I think the tabs/spaces was an issue when I copy+pasted, not seeing that on my end
• the folder structure per site is identical so the goal was to use substrings based on tag type since they'll all be starting at the same place if they're in the same folder
• the quality check stuff is something I was testing, there seems to be an issue when I try to display any pens with "Bad" quality, there are some that are "Bad_OutofRange" that I was filtering out while troubleshooting.

[Jacob_Barlow]

@pturmel

Yes, if scripting anyways. If avoiding scripting to avoid its startup penalty, the lookup() expression function with a dataset tag is a performant option.

Would saving a list of pen configs to a dataset and pulling out the corresponding objects be possible? I could make a udt and save the dataset tags in each site's folder structure and just pull those out when the script fires if it's possible. I just wasn't sure where to even begin storing these JSON arrays in a coherent format.

[Jacob_Barlow]

Side note:

Do either of you know how I'm meant to adjust the vertical position of the Y-axis label for the PowerChart? The 'offset' property only adjusts horizontal position, and the Y-axes' labels are directly on the upper edge of the chart. See below:

[pturmel]

@Jacob_Barlow

Would saving a list of pen configs to a dataset and pulling out the corresponding objects be possible?

Getting multiple rows is tricky. I created an expression function just for this case (view(), part of the free Simulation Aids module). It is convenient, but since it actually invokes jython, it doesn't have the performance advantage of lookup().

@Jacob_Barlow

dataset tags in each site's folder structure

Yes, this is likely best, if you are already indirectly addressing tags in these folders, just indirectly address the dataset tag, too. You could create a maintenance script in your gateway that would browse the folders and update the datasets. If you like.

[Jacob_Barlow]

@pturmel

Yes, this is likely best, if you are already indirectly addressing tags in these folders, just indirectly address the dataset tag, too. You could create a maintenance script in your gateway that would browse the folders and update the datasets. If you like.

I've been avoiding doing stuff like that to reduce the total work to onboard each site but with the functionality & performance they're requesting I think it might be inevitable. Do you have a suggestion for how I can get the JSON arrays tucked into a dataset?
(I'm still really new to all this scripting stuff, first project, teaching myself as I go.)

[pturmel]

I mostly don't put json in datasets, actually. I recommend you bind the dataset as a private property of your view or session, and reference it in a transform between your history binding and your chart data, creating your columns property as a side effect.

[Jacob_Barlow]

I ended up taking the list of tags for each category and created a dataset with the name & source strings I wanted on a custom property for the PowerChart. The buttons then take the dataset and iterate through the JSON creation using the strings in the dataset. Not pretty but it is a bit faster.