Tag Reference Tracker

[ggross]

Intro

At ICC, we announced that, in the 8.1.34 release, a new tool to identify what resource in Ignition is utilizing different tags would be released; a feature we call the Tag Reference Tracker. The code for this long awaited feature has been merged and will be available in the nightly build starting the morning of 10/26/2023. The original request for this feature was created a little over 9 years ago and there were multiple variations of what people were looking for this feature to be. We had many starts and stops along the way trying to find a solution that resolves the underlying problems mentioned in that feature request, we hope that what we delivered meets your expectations.

How it works

The Tag Reference Tracker functionality consists of the following:

1. A SQLite database per tag provider that keeps track of the following:

   • A Tag Path
   • A Unique Location that is accessing a specific tag path
   • The timestamp it was last accessed by the resource
   • How it was accessed: We track Subscriptions, Reads, Writes and Configuration changes
   • The number of times the resource has accessed the tag in that manner

   There can be many rows per tag path depending on the usage throughout your Ignition projects, but there should only be 1 row per type of interaction with the tag where the count is updated for each use. These databases are created in the ${IGNITION_HOME}/data/diagnostics/tags/ directory.

2. New Tag Diagnostics dialog in the Designer

   • Tag diagnostics has been moved out of the Tag Editor, and can be accessed via the View Tag Diagnostics option in the right-click context menu on a tag:
     
   • New Active Subscriptions tab: A new tab that will show you what resources have an active subscription to the tag you are viewing Diagnostics for. Double clicking on an entry will take you to the resource subscribed to the tag.:
     
   • New Reference Log tab: This will show historical interactions with the tag including non-subscription interactions. For example, should something like a Perspective View or a Transaction group read a value or write to a tag as part of its workflow, the reference will be reported here if the Tag Reference Tracker store is enabled for the provider. Double clicking on an entry will take you to the resource noted as performing an action on the tag.
     
     

A couple things to note:

• New Gateways and newly created realtime providers will have the SQLite storage options enabled by default.
• On upgrade the option to store to the SQLite database is disabled. Depending on the type of provider in use, it can be enabled in the following areas:
  • Realtime providers (any provider listed on the Config > Tags > Realtime screen on the Gateway): Config > Tags > Realtime > {Provider} edit > Advanced > Enable Tag Reference Tracker Store
  • Non Realtime providers (System provider, 3rd Parties): Config > Gateway Settings > Enable Tag Reference Tracker Store. This is a global setting that enables storage for all 3rd Party Managed Tag Providers.
• SQLite entries are created on the Gateway the resources are accessing a tag from:
  • If a provider is being used remotely, you will see that the remote provider is using the tag but you would need to look at the Reference Log in a Designer on the remote provider to get any details about the resources using the tag.
  • In a Gateway setup where redundancy is in play, each gateway will have a unique SQLite database and the entries are not consolidated. As the backup cannot be accessed via a Designer, the only way to get information from a backup is directly interacting with the db.

Performance Considerations

1. CPU Usage: On our Performance test runs of 10 Gateways setup with 100,000 tags in a remote provider configuration, we saw ~3% increase in CPU usage.

2. Disk Space: The SQLite DBs that are created are not tiny. For a 1 million tag provider, the size the SQLite database was ~750MB. We encourage you to turn on Providers one at a time and monitor how it affects your disk space. The tag diagnostics information is not included in any Ignition backups.

3. In our testing we have found that we can handle ~125k writes/sec to the SQLite DB. This is a non-blocking action and Ignition should be able to recover if there are a flurry of changes above the threshold, but sustained levels of activity above the threshold should be avoided.

Limitations

1. Scripting: When you are running a Jython script within Ignition, it is a compiled version of the script. Because of this, we will only be able to provide a reference to what called the script, not the script itself.

2. Vision and Designer references: Because our swing clients offload a lot of work from the gateway, they make remote procedure calls that request data but don't give any context for how it is used. While we will note if a Designer or Vision Client made a reference to a tag, we cannot provide insight into what Vision/Designer resource made the interaction.

3. Perspective Embedded views: We will be able to provide information on the embedded view that directly made the change, but we don't have a way to show information on the calling view that might have passed parameters in.

As always, we appreciate any feedback you may have.

Thanks,
Garth

[nminchin]

Super excited to check this out. I can see it being extremely useful for both diagnosing issues, and cleaning up systems that have adapted over time. These may have potentially a large number of tags that may not ever be used and are taking up resources unnecessarily. Also for working out how stuff is all tied together in a system you're working on developed by others. Cool stuff

[Arnaud_Declerck]

Well i would say that i'm starting my day with a great news. Can wait to check that up.

[michael.flagler]

Are there any plans to rework Designer and Vision to pass some of this metadata back to the gateway with the RPCs they make so that the reference tracker will be more useful for projects using Vision?

[ggross]

At this point there are not any plans. It would take substantial rework of how everything holds together and it would likely increase network traffic substantially. We will always look for ways to make this happen in a way that isn't detrimental to the performance of Ignition, but it is not something that is on the roadmap.

Garth

[pturmel]

@ggross

likely increase network traffic substantially

When/if this is pursued, consider making it a local function of the designer/vision client tag manager that adds a tab to the runtime diagnostics dialog. No need for extra network traffic.

Fundamentally, the local tag manager would have access to the same information about Vision bindings that the gateway tag manager has about Perspective bindings.

And you could skip the per-provider enable/disable, as there wouldn't be any gateway load to worry about. You could just have a project setting and/or environment variable trigger to enable it for the whole client.

[ggross]

@pturmel

When/if this is pursued, consider making it a local function of the designer/vision client tag manager that adds a tab to the runtime diagnostics dialog. No need for extra network traffic.

While this could be a first step, I think we need to understand what is the intended goal for this information? Is it:

1. Troubleshooting a single client and how the project it is running uses a tag?
2. Giving a global picture of how a project(s) use a tag?

I could see arguments for both, and I think most would say they would take something over nothing. There also would also be a good amount of work adding the necessary hooks to answer either question, and ultimately it will likely be a either/or not a both.

There is a lot of refactoring of the backend occurring for 8.3 and there are opportunities to be able to make changes that might impact a direction we could take for reporting tag reference information from Vision. At this point, nothing will change for what is reported in Vision (for this feature) until after 8.3 is released (at the earliest) and we have a better idea of what is possible.

[pturmel]

3. Following one or more clues from the gateway-based tool to investigate further on a client-by-client basis.

Although, if you save to a SQLite DB on the client side, then exercising each project's Vision client through some automated testing sequence should generate a decent snapshot for general use. Or could document what happens while in the designer on a per-window basis.

[SBL_MC]

I was hoping renaming a tag will be any more smooth with the tag reference tool, asking me if I'd like to update all the bindings related.

Is find and replace still the way?

[PGriffith]

TRT is purely a side channel collecting information at the moment. In the future we might drive "smarter" bits of the platform off the information collected by TRT (we're collecting this information is such way that it's useful for more than just the current presented UI), but you're correct, this doesn't offer any replacement or augmentation for find/replace at this time.

[ataya]

Hi @ggross ,

Can the default be false on a future version? Considering this encouragement, it will be one less concern to turn this off.

We encourage you to turn on Providers one at a time and monitor how it affects your disk space.

Thank you.

[ggross]

@ataya

Can the default be false on a future version? Considering this encouragement, it will be one less concern to turn this off.

The concern on existing systems was taking systems running very close to the edge of available resources and enabling something that could easily cause some problems depending on the number of tags a system is running.

As this feature requires use of the system over time to be useful; having it enabled by default on newly created providers, where there is no existing expectation of performance metrics/disk space metrics, is the ideal time to enable this feature. This allows users to track things over time and there isn't an immediate impact that has a possibility of being negative.

Are you running into issues since the feature has been released? Is so, could you expand on what the issues are and how they are impacting you directly?

Thanks,
Garth

[ataya]

Hi @ggross,

Thanks for your reply.

In a scenario with more than 100 gateways most running 1 million+ tags some nearing or exceeding 10 million tags...

And as soon as a new tag provider is created, it gets populated by 100k+ tags in a matter of minutes, remote tag providers added to many front end gateways, windows/pages built within days, 100s of client launchers/browsers deployed automatically to load the windows/pages and subscribe to tags.

My concern is this has the potential to get out of control too quickly.

Regardless of initial low consumption, if a new setting is using extra resources, enabling it should be a deliberate decision is my 2 cents.

Thanks.