Speed problem in the execution of an action in perspective

[Cristian_Gomez]

Hello, good afternoon
I have a speed problem in the execution of an action in perspective. The version of the company is 8.1.20 so it does not include the filtering property in the tables, therefore I used dop-down boxes for this purpose, but the execution of the command is taking more than 30 seconds, any idea to optimize please.

[PGriffith]

It'd be nice if you posted your code as text, instead of images, if you want folks to take a look at it and optimize it.

From experience, it's probably going to be significantly faster to construct a list of row indices to delete/keep and 'trim' the input dataset using system.dataset.deleteRows as one operation rather than continually resizing a PyDataset/list of rows. You could also be doing some extra work to 'short circuit' your conditions - as it's written now, you might be checking a row for a foreman/pumper match even if the district match failed, for instance.

[Cristian_Gomez]

Thank you very much, I will work on the basis of what you say.
I will put the initial code here as well.

def transform(self, value, quality, timestamp):
	selectedDistrict=self.getSibling("Dropdown").custom.SelectedValue
	selectedForemen=self.getSibling("Dropdown_1").custom.SelectedValue
	selectedPumper=self.getSibling("Dropdown_0").custom.SelectedValue
	HeaderList=["Meter","Graph","Type","Time","Meter ID","Status","County","Well ID","Distric","Foreman","Pumper","Valve Control","PID Control","Gas Rate","Gas Today", "Gas Yest","Net Gas","Avg 7","Best 7","Vol Ex Diff","Vol Ex Pct","DP","CPress","CAvg","CDraw","TPress","TAvg","TDraw","SP","Temp","FTT","FTY","RT Order","Demand Poll","Demand Poll Status","Battery","Poll Status","Type_Full","Sales Gas"]
	data = system.dataset.toPyDataSet(value)
	ListFilt = []
	if selectedDistrict != "<All District>" or selectedForemen != "<All Foremen>" or selectedPumper != "<All Pumpers>":
		for row in data:
			DistrictMatch = selectedDistrict == "<All District>" or row ["District"] == selectedDistrict
			ForemenMatch = selectedForemen == "<All Foremen>" or row ["Foreman"] == selectedForemen
			PumperMatch = selectedPumper == "<All Pumpers>" or row ["Pumper"] == selectedPumper
			if DistrictMatch and ForemenMatch and PumperMatch:
				ListFilt.append(row)
		filtered_dataset = system.dataset.toDataSet(HeaderList, ListFilt)
		return filtered_dataset	
	return value

[lrose]

I would expect this to execute quicker.

def transform(self, value, quality, timestamp):
	selectedDistrict=self.getSibling("Dropdown").custom.SelectedValue
	selectedForemen=self.getSibling("Dropdown_1").custom.SelectedValue
	selectedPumper=self.getSibling("Dropdown_0").custom.SelectedValue
	data = system.dataset.toPyDataSet(value)
	if any(selectedDistrict != "<All District>",selectedForemen != "<All Foremen>",selectedPumper != "<All Pumpers>"):
		ListFilt = []
		for index,row in enumerate(data):
			DistrictMatch = selectedDistrict in {"<All District>",row ["District"]}
			ForemenMatch = selectedForemen in {"<All Foremen>",row ["Foreman"]}
			PumperMatch = selectedPumper in {"<All Pumpers>",row ["Pumper"]}
			if all(DistrictMatch,ForemenMatch,PumperMatch):
				ListFilt.append(index)
		return system.dataset.deleteRows(data, ListFilt)
	return value

[Cristian_Gomez]

Thank you very much, the code as such performs the reverse process, since it eliminates the rows where the statements are true, but it gives me the idea to re-structure it.

[pturmel]

You might find my new where() expression function helpful:

Automation Professionals' Integration Toolkit Module

I have spent my holiday poking at this. Wanted to provide an efficient replacement for a common use of python transforms: Make a (Perspective) table that dynamically populates both .data and .columns from source dataset that isn't constrained to any particular structure or data types. In Vision, one can achieve this (mostly) by pre-populating the column attributes dataset with all of the possible column names your application might deliver. No such thing in Perspective. But a transform can …

In this case, your transform would look something like this:

where({value},
    ({path.to.District.Dropdown.custom.SelectedValue} != '<All District>' && {path.to.District.Dropdown.custom.SelectedValue} != it()['District']) ||
    ({path.to.Foreman.Dropdown.custom.SelectedValue} != '<All Foremen>' && {path.to.Foreman.Dropdown.custom.SelectedValue} != it()['Foreman']) ||
    ({path.to.Pumper.Dropdown.custom.SelectedValue} != '<All Pumpers>' && {path.to.Pumper.Dropdown.custom.SelectedValue} != it()['Pumper'])
)

I suspect this will run much faster than any script transform.