Perspective property query binding transform executing when value is NoneType

I am working on some perspective views with some production data charts (using the free Embr Charts module) which get populated with data from a SQL database. To get this to work, I added a custom property with a query binding which returns a dataset and script transform to interpret and format the data for the charts. Since some of the charts are used on multiple views, I defined functions in gateway scripts which interpret and format the data, which are called by transforms on property bindings.

I’m having an issue on a couple of my charts where it seems like the transform script is running before the query is complete. At first I was thinking this was an issue with the pie chart from Embr Charts, but after adding some logging I think that isn’t the issue. I added some logging via system.perspective.sendMessage so that I can see what is happening from the client (behavior is different/harder to reproduce in designer). I am writing to the log when navigation changes to a different view. I also added a property change script on the property with the query binding and transform which logs currentValue.getQuality(). A section of this log is at the bottom of this post. What I am seeing is that when I navigate to the display titled “Defects” (which is having this issue) about 90% of the time, the property change script will run twice. The first time it will say that the quality is bad because 'NoneType' object has no attribute 'getRowCount'. One of the first things that is done in the function called by the transform script is getting the row count from the dataset that the query returns, so it seems like this is running before getting the data from the query. After about 290ms the property change script runs again and logs the quality as good. When this happens, the chart shows a warning icon with the text “Component Error” (tooltip says Cannot read properties of null (reading ‘length’). The other 10% of the time the property change script only shows up in the log a single time with “Good” quality and the chart displays properly.

I tried removing the transform so that the property binding is only running the query. Then I creating a second custom property with an expression structure binding (with “Wait On All” enabled) pointing to the first property and calling the transform script. I thought this might do essentially the same thing but force the second binding to wait for the dataset to be populated, but it did exactly the same thing.

Am I doing something incorrectly? Is the problem that I’m calling a function defined in the gateway scripts from a property binding?

01/21 08:25:40.743: Defect chart data quality: Good
01/21 08:25:40.670: Next view: Defects
01/21 08:25:39.077: Next view: Time
01/21 08:25:37.812: Defect chart data quality: Good
01/21 08:25:37.524: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:37.492: Next view: Defects
01/21 08:25:36.058: Next view: Time
01/21 08:25:35.014: Defect chart data quality: Good
01/21 08:25:34.721: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:34.693: Next view: Defects
01/21 08:25:33.243: Next view: Time
01/21 08:25:31.651: Defect chart data quality: Good
01/21 08:25:31.351: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:31.315: Next view: Defects
01/21 08:25:29.531: Next view: Time
01/21 08:25:28.297: Defect chart data quality: Good
01/21 08:25:28.008: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:27.982: Next view: Defects
01/21 08:25:26.034: Next view: Time
01/21 08:25:24.327: Defect chart data quality: Good
01/21 08:25:24.027: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:23.972: Next view: Defects
01/21 08:25:21.744: Next view: Time
01/21 08:25:18.579: Defect chart data quality: Good
01/21 08:25:18.290: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:18.252: Next view: Defects
01/21 08:25:16.355: Next view: Time
01/21 08:25:14.944: Defect chart data quality: Good
01/21 08:25:14.654: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:14.614: Next view: Defects
01/21 08:25:12.910: Next view: Time
01/21 08:25:10.812: Next view: PPM
01/21 08:25:09.069: Next view: Time
01/21 08:25:05.498: Defect chart data quality: Good
01/21 08:25:05.206: Defect chart data quality: Error_ScriptEval("Traceback (most recent call last):
  File "<transform>", line 2, in transform
  File "<module:BinData>", line 740, in unitDefectsChartsCombined
  File "<module:BinData>", line 848, in PieChart_Unit_Defects
AttributeError: 'NoneType' object has no attribute 'getRowCount'
")
01/21 08:25:05.179: Next view: Defects
01/21 08:25:04.246: Next view: Time
01/21 08:25:03.134: Next view: PPM
01/21 08:25:00.487: Next view: Time
01/21 08:24:54.733: Defect chart data quality: Good
01/21 08:24:54.680: Next view: Defects
01/21 08:24:52.494: Next view: Time

Is there error only occurring on page load? If so, it's initially giving the binding a "None" value, which is then trying to run thru the Transform. You need to either persist a "good" set of data, or tweak your transform so it recognizes the empty DS and stops trying to go further. Then once the actual binding fires, it should run thru correctly.

1 Like

Are you suggesting something like this?

if value is None:
    return dictOfBlankData
else:
    return gatewayScript(value)

Edit: I tried this and it actually made the component error appear on the bar chart as well as the pie chart. I also noticed that in a client when I resize the split container that the pie chart is in, the component error goes away and the chart is rendered properly. Almost like the property value is correct, but the chart isn’t redrawn. Calling refreshBinding on the query binding doesn’t seem to fix it.

Also worth noting, my transform script is returning a dictionary containing 4 dictionaries. The 4 dictionaries each contain the data for 1 of the 4 charts in the view. Only 1 of the 4 charts is having problems, the other 3 work every time.

Edit 2: I tried setting the property with the query binding and transform as persistent and saving it with a good value. The behavior remains unchanged. I also noticed when I open the GUI for configuring the binding, if I have binding preview enabled, initially under “Query” is shows “Dataset[1850 rows, 141 cols]”, then it briefly shows, “Error_Exception”, then it goes back to showing “Dataset[1850 rows, 141 cols]”. This seems very similar to what is happening when navigating to this view.

Is your binding using values from other fields/tags/props? If so, are those valid upon load?

Sorry for the delay, I had some time off and a few other items come up.

Yes, I am using a handful of properties from self.view.custom but all of them are valid. I was able to reproduce the issue in designer just now. The error on the property points to the first line of the function that the transform calls from the project library and states ‘NoneType’ object has no attribute ‘getRowCount’. The only variable that I run getRowCount on the is the dataset which is returned from the query binding, so I don’t think any of the properties being passed into the transform are the issue.

I seem to have gotten a higher chance to get the chart to render properly on the first try by disabling “Cache & Share” for the query binding but it still is not 100% of the time. When the component has the error, if I simply move the divider for the split container it is inside, the chart will then render properly.

Adding while value is None: pass at the start of the the transform (which I realize is a bad idea) before calling the script doesn’t change how the chart behaves in a client. It does prevent the tag change script which logs the tag quality to only run a single time and say that the quality is ‘Good’ every time, which normally only happens when the chart displays properly.

We're probably going to need to see your bindings / transforms.

As previously mentioned, this is happening on two pie charts and I’m using the Embr Charts module (link). The charts show similar data, but have different scripts which do the calculations and format the data properly. One pie chart gets an error when the getColumnNames() method is invoked on a dataset returned from the query binding. The other pie chart gets an error when the getRowCount() method is invoked on a dataset which comes from a property that is passed into the transform function. In both cases, this is happening the first time that a dataset method is invoked in the script.

Both scripts in question are actually calculating and formatting data for 4 different charts. 3 of those charts work every time (line chart, bar chart, and legend). Those other 3 charts work without issue but the pie chart is not working every time.

Query binding with transform:

Relevant function from BinData (Project Library) which throws an error when setting failReasons (first line) because data is NoneType instead of dataset:

def stationDefectsChartsCombined(data, minDate, maxDate, binSize, selectedEM, colorTheme):
	failReasons = [i for i in data.getColumnNames() if i.startswith('FailureData/EM' + str(selectedEM) + '_') or i.startswith('FailureData/EM0' + str(selectedEM) + '_') or i.startswith('FailureData/EM00' + str(selectedEM) + '_')]
	
	# Generate Pie Chart Data
	pieChartData = PieChart_EM_Defects(data,selectedEM,failReasons,colorTheme)
	
	# Generate Legend Data
	rejectReasons = {key[key.index('_')+1:]:key[key.index('_')+1:] for key in failReasons}
	legendData = PieChartLegend_Defects(pieChartData,labelMap=rejectReasons)
	
	#Generate Bar Chart Data
	barChartData = BarChart_EM_Defects(data,selectedEM,failReasons,binSize,colorTheme,minTimestamp=minDate,maxTimestamp=maxDate)
	
	count = 0
	for row in range(data.getRowCount()):
		if data.getValueAt(row,'bRanEM/' + str(selectedEM)) and not (data.getValueAt(row,'bQualityPart') or data.getValueAt(row,'bScrapOnly')):
			count += data.getValueAt(row,'iPartQuantity')
	
	yieldChartData = BinData(unit=binSize,
		dataSet=data, action='PartCountEM',
		minTimestamp=minDate,
		maxTimestamp=maxDate,
		selectedEM = selectedEM,
		countGoodBadorUgly='Yield')
		
	if sum(pieChartData['Values']) == 0.0:
		pieChartData = {'Percents': [1.0],'Values': [1.0],'Colors': ['#DDDDDD'],'Labels': ['No Defects']}
	
	return {'PieChart':pieChartData,
		'LegendData':legendData,
		'BarChart':barChartData,
		'Points':count,
		'YieldChart':yieldChartData}

How the returned dictionary from the script is bound to the 4 charts in the view:

I'm going to guess that the props your passing into the parameters of your query aren't all valid when it loads. Likely the ones that are a date.

Try changing those parameters so they're hard set to "valid" values instead of your props. See if that makes it reliable. If so, then it's a matter of coming up with a way to test for all valid values before you run the query.

I removed the bindings from all of the properties which are passed into the transform function, leaving them with valid values. I left the query binding in place. The behavior did not change.

Next, I made a new property called test1 and bound it with the same query binding but without the transform to populate it with valid data. Then I removed the binding so that the dataset does not change. Then I added a property test2 and gave it a property binding to test1 and added the transform script. Then I updated the pie chart to point at test2.PieChart instead of ChartData.PieChart. Now the chart will always load the same data, so it isn’t very useful, but it never shows up with the component error.

Lastly, I added back all of the property bindings that I initially removed. The chart still loads correctly every time, albeit with the same dataset which is no longer being queried.

Query bindings run asynchronously, so always return null/None at startup, shortly followed by the proper data. You have to handle this in your transform. (Return an empty list if value is None, perhaps.)

Changing my transform to the below did not change the behavior. As before, 3 of 4 charts that get data from this query binding & transform work without issue. Only the pie chart gets a component error. The tag change script that logs the quality of the property with the query binding is still logging twice when navigating to this view but both times it now says ‘Good’. Am I missing something? Could this be an issue with the Embr Chart pie chart?

def transform(self, value, quality, timestamp):
	if value is None:
		return {}
	else:
		return BinData.stationDefectsChartsCombined(value,
			self.view.custom.Chart.MinDate,
			self.view.custom.Chart.MaxDate,
			self.view.custom.Chart.Bin,
			self.view.custom.selectedEM,
			self.view.custom.Chart.ColorTheme)