Best Approach for Calculating a Running 5-Minute Average for 100K Tags

Ignition
,
1

Hello Everyone,

I'm trying to calculate the running 5-minute average of a raw tag without storing it in a SQL database. Currently, I store the values in a buffer dataset and use a trigger every 5 minutes to run a function that calculates the weighted average, writing the result to a separate 5-minute average tag.

My main concern is scalability—would this approach work efficiently for a 100K tag system? Or are there more optimized methods to handle this kind of rolling average calculation at scale?

Would appreciate any insights or best practices! :rocket:

Event Change script on the Raw Tag:

image
image1708×845 69.5 KB

Event Change Script on the 5-Min Trigger

image
image1727×1019 74 KB

2

Using tag scripts would probably not scale well for 100K tags. Read through this topic: Best practices and pitfalls with Tag Event Scrips. Here are some options:

  1. You could try using pturmel's Integration Toolkit See this post: Best practice for tag calculating derivative (difference between values at 2 timestamps) - #2 by pturmel

  2. You could also try calculating them in a gateway timer script, reading the tags together, storing the buffer for all the tags together in some dataset, and writing to the "average" tags together. (I'd also have the list of tagpaths in the dataset itself rather than browsing for tags). Though you'd definitely have to test for performance if you're going to do this for 100K tags at once; (Can the system.tag.read*() funcitons even take in that many paths?)

  3. 100K tags is a lot, but you could perhaps also experiment with system.tag.queryTagCalculations()?

I've done something similar using option 2 and have found it to be very performant, but it wasn't anything close to 100K tags.

1 Like
3

I'll add option #4.
Use a java concurrent queue: Java concurrent queue for tag change scripts rather than doing the calculation on the tag change script.

5 Likes
4

Certainly can. And it's lightning fast, since all tag.read does is reads the cached tag values, it doesn't actually trigger the tags to read current values from their sources (eg plcs). If you need that, then you can use opc.read*, which yes, would likely take some time, relatively speaking

2 Likes
5

@djs Thank you for your valuable insights. Regarding option 3, I'd like to confirm my understanding. Does system.tag.queryTagCalculations() rely on data being previously historized in a SQL or historian database?

6

{ My strikethrough. }

Yes, historized to an Ignition historian.

1 Like
7

Yes, as pturmel said, it would send queries to an Ignition historian. So you wouldn't want to use this on each tag individually, but for all the tags at once if you choose to go this route. Otherwise you might overwhelm your DB.

This could be an option if you use the averages for a report and don't actually care about storing them. If you need the averages historized, I'd explore other options.

If you care about the averages being calculated on change, then I'd look into option 4 given by bschroeder.