Best/Fastest way of browsing all tags?

Ignition
1

Could someone tell me the best/fastest way to browse every tag in my tag tree and print out the attributes of each tag (https://docs.inductiveautomation.com/display/DOC/Tag+Attributes)?

So far I have been using the system.tags.BrowseTags() function. It works, but it seems to be very time consuming.

I’ve been testing on a single folder which contains about 7300 tags everything included (OPC, Memory, folders, UDT, …). It takes the script upwards of 15 minutes to run. We have about 200 of these folders I want to browse.

2

Is it the actual browseTags that takes so long, or is it processing the tags afterwards.

Could you perform some time measurements inside your script, and share them (together with your script), so we can give suggestions?

Most of these things are IO bound, so it should be easy to parallelize the slow parts.

3

@Sanders17
It’s a good question, and I don’t know.
However I just need to know the best/fastest way to browse all tags and print their attributes regardless of how I do it now.

4

Optimizing == measuring!

If something is too slow, you need to measure what part is slow, and try to find alternatives for it.

I tried reading my tags with the following script, and could browse 22672 tags in under 2 seconds. So BrowseTags probably isn’t the slow part of your script.

import time

start = time.time()
tags = system.tag.browseTags("", recursive=True)
print "browsed %s tags in %s seconds" % (len(tags), time.time() - start)

output: browsed 22672 tags in 1.93400001526 seconds

5

@Sanderd17
Next step is to print their attributes. How would you do that?
It’s probably that part that is slow then.

6

The real question is "How do YOU do that?".

To quote stack overflow (How do I ask a good question? - Help Center - Stack Overflow)

3 Likes
7

For now the best way is browseTags() using the sort of manual recursion shown here, very rarely would you use the built in recursive parameter for browseTags IMO. I have a method that finds all tags inside UDTs in a tag provider that I explain a bit here. No code, sorry, but the general idea for speeding it up is there. Depending on the amount of tags/UDTs it could still take a while. Sounds like you have more than just UDTs you’re looking for which may not work as well with this method.

Besides making your own recursive function (from the first link) I’d suggest you do this on the gateway. Personally I have a boolean memory tag with a tag change script so that when its true, it runs the script, dumps to a dataset memory tag, then writes the boolean memory tag false again so its all done in the gateway. It’s much faster on the gateway usually and you could set a timer script to do this too so it runs in the middle of the night or something so it doesn’t tank your servers performance.

If you need to do this in the client, make sure you’re using a invokeAsynchronous/invokeLater structure so it doesn’t freeze everything while you do it.

This is all to just get the paths though. Once you do that you can use system.tag.readAll() and append the attributes to the path to find what you need (tag attributes = path.attribute so a tag at path Test/Tag would have the OPC Server attribute at Test/Tag.OPCServer). If you’re currently using just system.tag.read() for each and every new path you find, that will slow things down a ton. readAll() is much faster for the same amount of tags.

2 Likes
8

@mrogers
Wow, thanks for a great reply.
Interesting you suggest to run it on the gateway - it makes a lot of sense. Currently have just run the script in a client in the script console.
Will look into this and test when I have some spare time :+1:

9

Right, so now I have a script that reads out each and every tag property for each tag in a folder containing 7506 tags. Even the .PropertyOverrides map from the UDT’s are distributed correctly to each corresponding tag.

The system.tag.readAll() function speeds things up dramatically compared to reading each tag individually.

I’m down to ~4 minutes for the script to run which for now is acceptable.
Thanks to @Sanderd17 for the timer script example.

The next thing I tried was to browse our entire tag tree containing a massive 788181 tags!
The system.tag.browseTags() only took 49 seconds to run.
From the browseTag data I then built a list with every tagpath addad ‘.ToolTip’ just for a test.

When runing a system.tag.readAll() with a list of 788181 tags I get an ‘Error executing gateway funcion.’.

Is there a recomended max size of the list provided to the system.tag.readAll() function?

10

There shouldn’t be a max size, but it would help if theres any more information included in the error besides that one sentence. Are you reading them in a gateway timer script or something run on the gateway? I’d say make sure theres a provider in the path (from the browsetag object, use .fullPath instead of .path), but other than that I’m not sure.

11

You might want to take a look at system.tag.browseConfiguration("[default]",True). It should be the fastest, iirc.

12

For those looking at this now, in my personal testing, using system.tag.browse with recursive set to true is significantly faster than doing a manual recursive search.

13

True enough, except that running such from a designer script console or a Vision client dramatically raises the odds of hitting the 60-second fixed RPC timeout.