Perspective Ad-Hoc Trends with Remote Tag Providers

I was working with the Perspective AdHoc trends view created by Matt Raybourn on the exchange, and noticed that when working with remote tag providers it was excruciatingly slow to load the tag tree. The reason for this is that the browsing is recursive, and a separate system.tag.browse call is executed for every single tag. In a case where you have hundreds or thousands of tags, this can start to take much longer than acceptable to load in the client.

The system.tag.browse and system.tag.getConfiguration functions are very similar, except that the getConfiguration function has a built in “recursive” parameter, that allows you to send one remote call and collect every tag and all of its children, instead of needing to execute multiple remote calls to build the page.

I rewrote the binding transform on the items property of the tag browse tree to the following, and this allows remote providers to be scanned and rendered much more quickly than using system.tag.browse, it is a little more complicated, and system.tag.getConfiguration doesnt always return things in the same order they exist in the tag folder so I had to sort them, but this can make the page much more usable in cases where previously it would take up to 30 minutes to load a remote tag provider with 500 tags. This may not be the absolute fastest way to query large remote tag providers, but its the best one I have come up with.

Hope this helps someone, and if Matt Raybourn sees it than it may be valuable to be on the actual exchange item itself!

	#This could come from a property
	tagProvider = "[Texas North]"
	
	#[{label, expanded, data, items[]}]
	#The replacement function for browseConfiguration (Using the full dictionary of tags)
	def browseTagConfiguration(tagConfiguration, startPath):
		tags = []
		
		for result in tagConfiguration:
			tagPath = startPath + "/" + str(result["path"])
			
			#If there is a number as the last item in the path, use that as the sorting index, comment this out for alphabetical sorting
	#		try:
	#			nameArray = tagPath.split(' ')
	#			index = int(nameArray[len(nameArray) - 1])
	#		except:
	#			index = 9999
			#Leave this for alphabetical sorting
			index = 0
			
			
			#Check the tagType to determine the item structure
			if str(result["tagType"]) in ["Folder", "UdtInstance", "Provider"]:
				tag = {'label': result["path"], 'expanded': False, 'data': {'folder':tagPath, 'index':index}}
				
				#An empty folder will not contain the "tags" key in its configuration
				if result.has_key("tags"):
	
					#Numerical sorting
					#tag['items'] = sorted(browseTagConfiguration(result["tags"],tagPath ), key = lambda i: i["data"]['index'])
					
					#Alphabetical sorting
					tag['items'] = sorted(browseTagConfiguration(result["tags"],tagPath ), key = lambda i: i['label'])
					
					if len(tag['items']) > 0:
						tags.append(tag)
			else:
				#A tag without history enabled will not contain the "historyEnabled" key in its configuration
				if result.has_key("historyEnabled"):
					#I only wanted to return tags that I was collecting history on
					if result["historyEnabled"]:
						tags.append({'label': result["path"], 'data': {'tag': tagPath, 'index':index}, 'items': [], 'expanded': False})
	
		return tags
	
	
	def browseHistoricalTags(path):
			tags = []
			#results = system.tag.browse(path).getResults()
			results = system.tag.browseHistoricalTags(path).getResults()
			
			if results != None:
				for result in results:
					if result.hasChildren() == True:
						tag = {'label': result.path.lastPathComponent, 'expanded': False, 'data': {'folder': result.path}}
						tag['items'] = browseHistoricalTags(result.path)
						if len(tag['items']) > 0:
							tags.append(tag)
					else:
						tags.append({'label': result.path.lastPathComponent, 'data': {'tag': result.path}})
			return tags
	
	if value:
		tags = browseHistoricalTags(tagProvider)
	else:
		tagConfiguration = system.tag.getConfiguration(tagProvider, 1)[0]

		#Numerical sorting
		#tags = sorted(browseTagConfiguration(tagConfiguration["tags"],''), key = lambda i: i["data"]['index'])
		#Alphabetical Sorting
		tags = sorted(browseTagConfiguration(tagConfiguration["tags"],tagProvider ), key = lambda i: i['label'])
		
	
	return tags

EDIT: Originally I accidentally left the incorrect function in for browseHistoricalTags! I have replaced it with the correct one.

8 Likes

Well done!

1 Like

Very cool, and kudos for sharing your solution! @mraybourn :slight_smile:

1 Like

Yeah, that is awesome, thanks for posting it!

1 Like

I was a bit sad that from 7 to 8, the tag browse function lost its recursive option. Do you know why that was?

As a follow up to this,

If you have a system with a huge number of tags, this can start to become a bit slow as it tries to recursively iterate through a huge number of tags.

So I built another version that browses one folder at a time progressively

6 Likes

This view was awesome. I think something like this to browse a PLC will be very cool. I was able to develop something similar with the system.opc.browseServer function but I was not able to get the data type of the tags so could not efficiently import tags to the ignition gateway.

LOVE IT! Thanks, you saved me hours I think.

1 Like

I hope this is related enough to not be a hijack…

Has anyone seen or used an example of system.tag.browseHistorical() on a remote gateway when the script is running on a gateway that doesn’t have the tag history module licensed? I have a scale-out architecture and my front-end server only has Perspective licensed, so I can’t create a historical tag provider. I can browse real time tags on the remote gateway, but not its historical tags using the path string like ‘histprov:myDB:/drv:myIgnitionGateway:myTagProvider:/tag:myFolderOfTags’ because there is no histprov on the gateway running that script. I have access to the same database connection that the historical data is stored to, so it feels like I should be able to address it via a path using the driver and the database connection.

I’m I missing something obvious? Or do I need to design a custom query to fetch the history tags directly from the sqlth tables in this case?

Thanks!

Have you tried using sendRequest() ? (From remote gateway to ask origin gateway to query history.)

That I have not tried! I’ll explore that and report back…

Thanks Pete!

I think you mean Phil :smile:

1 Like

LOL!! Yep. Thanks Phil!

1 Like