[IGN-3902] Perspective Table: props.filter.results.data has a different structure than props.data

@cmallonee I’m running a script to summarize the results of a filter, which is working well. I want to run the same script on the table data when no filter is present (all data is present), but am running into an issue when there is table formatting.

I started with a switch to detect if there was text in the filter or not to choose the source of data
for a binding on “this.props.filter.text”:

	if value is None:
		dataSource = self.props.filter.results.data
	else:
		dataSource = self.props.data

but I quickly found out that getting data from props.filter.results.data is a list with formatting scrubbed from it, whereas “props.data” includes formatting.

My first thought is that props.filter.results.data should include all data that is being displayed. So even when filter string is None, all non-formatted data would be present.

Otherwise, I’m going to have to code a bunch of logic to handle detection if the props.data[0] object is the data or if it’s in {style=…, value=…} format. Then for each value here, I have to repeat the check if it’s just the data or if it’s in the {style=…, value=…} format for cell-level formatting.

I’m not seeing what you claim to be encountering, so I am going to need a concrete example (and/or some screenshots) to look into this more.

When I applied a formatting type of “none” to my population column, I found that the table honored that formatting decision, but both my props.data and props.filter.results.data[x] displayed identical objects for the filtered results.

props.data:

props.filter.results.data:

You can see from the table on the right that the formatting is in place and has no effect on the data, so it’s not clear to me what formatting you are seeing. In essence the “formatting” of the table is just a lens through which you’re seeing the data. The formatting does not actually modify the data in any way, and so the filter should always have the same content as props.data - albeit pared down to include only results which match your filter.

Below is a case where I use row formatting and cell formatting within the props.data field:
image

This is the filtered list:
image

So a metric summarizing the displayed list when filtered would be something like the following:

for row in props.filter.results.data:
   if (row['status'] == 'offline'):
      <add logic here>

If I want the same metric to summarize the table when no filter used ("self.props.filter.results.data" is "None"), notice that the filtered results list does populate:
image

So now the code has to detect when a filter is used and when it doesn't. That's no issue, but the task seems too much to read in "props.data" and include logic to either (1) scrub out the style keys (if they exist) and collapse the "value" keys, or (2) parse the tree and dynamically detect if there is row formatting or cell formatting, or both. Especially when Ignition already has this capability but is choosing to do it only a filter exists.

Note, I'm using 8.1.5.

Ah, apologies. Formatting (changing column/cell displayed value appearance) and styling (changing the Table appearance) are entirely different aspects of the table.

Now that I can see what you’re encountering, this does indeed seem wrong. I can provide a workaround for now:

	rows = []
	if self.getSibling("Table").props.filter.text == '':
		data = self.getSibling("Table").props.data
	else:
		data  = self.getSibling("Table").props.filter.results.data
	system.perspective.print(data)
	for row in data:
		my_dict = {}
		try:
			my_dict['population'] = row['population']['value']
		except TypeError:
			my_dict['population'] = row['population']
		rows.append(my_dict)

This will gracefully handle the differing structures of the default data. you will probably need to update this logic a bit to work with your unique structure, but the try/except usage should allow for swapping between the two structures at this time.

I’m going to open an internal ticket to change this behavior so that both structures are identical, BUT there is a very good probability that this will not be fixed until 8.2.0 because if we change this behavior at this time then it could break existing customer projects who are expecting the current incorrect behavior.

What is your opinion on changing “props.filter.results.data” to display the full data set when no filter is entered?

I don’t think that’s something we should do. In general, the more properties a component has the less performant it becomes. For Tables which have 1000 rows, you would now be adding a second 1000 rows into the component (assuming filtering and results are enabled). Sorting would have to be done on twice as many entries. Also, if the data is the same when no filter is applied, then what is really gained by duplicating the values? The only gain I can think of (aside from the ability to target one property rather than two) is to have access to the sorted results of the table, which is not a big gain. Additionally, that would ALSO have to wit until 8.2.0 because I expect that users currently have scripts which rely on the presence of no filter test to return no results.

Thanks. To confirm, the intended improvement potentially targeted for 8.2 is to match the structure including style and value names from the “props.data” node into the “props.filter.results.data” when a filter is active? Basically each item matching the filter from the “props.data” array will be copied into the “props.filter.results.data” array?

Correct. That’s not to say that we might do something else, but that makes the most sense at this time.

I’m hitting a wall with the Try/Except:

	array = []
	array.append({'Status':{'value':'zero','style':''},'CPU':'one'})
	array.append( {'value':{'Status':'zero','CPU':'one'},'style':''} )

	for row in array:
		try:
			return row['value']
		except TypeError:
			return row

When trying the above code, the script won’t get past the try with the row not having a value key.

Removing the word “TypeError” seems to let it pass.

Thoughts?

You’re now going to hit a KeyError, because the dictionary does not have the key you are requesting.

My example worked because I knew the dictionary I was working with had the population and 'value keys present. In your code, the first dictionary encountered has no value key (it as only Status and CPU). Either modify the first entry in your problematic code to include a value key or modify your code to handle the KeyError.

	array = []
	array.append({'Status':{'value':'zero','style':''},'CPU':'one'})
	array.append( {'value':{'Status':'zero','CPU':'one'},'style':''} )

	for row in array:
		try:
			return row['value']
		except TypeError:
			return row
        except KeyError:
            return None  # or something else

Thanks for that pointer. For those interested, below is the script I landed on. It accommodates both options: row-level formatting and cell-level formatting.

The following script is bound to “this.props.filter.text” of a table:

	if value == '':
		dataSource = self.props.data
	else:
		dataSource = self.props.filter.results.data
	
	dataSetNoStyle = []
	#Check if the row dictionary is nested within a key="value" to accomodate row formatting.  If so, remove.
	for row_dict_rowStyles in dataSource:
		row_dict_cellStyles = {}
		row_dict_noCellStyles = {}
		try:
			row_dict_cellStyles = row_dict_rowStyles['value']
		except KeyError:
			row_dict_cellStyles = row_dict_rowStyles
		#Check if the value of a cell is nested within a key="value" to accomodate cell formatting. If so, remove.
		for key in row_dict_cellStyles:
			try:
				row_dict_noCellStyles[key] = row_dict_cellStyles[key]['value']
			except TypeError:
				row_dict_noCellStyles[key] = row_dict_cellStyles[key]
		dataSetNoStyle.append(row_dict_noCellStyles)	

Would be curious to hear of any optimizations :slight_smile: