Tag History - Can you ignore a certain value? - Do not log if equal to or greater than X

Wondering if there is a way to not log certain values to the Tag History? The existing system changes the tags values to a max value during a communication failure.

1 Like

I do not know of any way to prevent the Historian from logging a change like this (outside of setting deadband, which is not valid in this case).

Although I doubt that this is your real problem, but is just what is causing your real problem.

I would expect that if you're having a communication failure that the quality code should be bad. You can ask the historian system to ignore bad quality when querying it.

If that is still not good enough, then if you can describe the issue this is causing and what you're trying to accomplish then we may be able to point you towards a solution.

The way they described it, the system that's currently logging historical data doesn't know if the data is bad or not. Communication to the device it's getting their data from is good but that device may not be talking to all the other devices it's gathering data from. They then have all of their reports, trends, graphs, remove the high values from those requested from the current database. I was just wondering if there was a way to stop the Ignition tag historian from logging these high values in the first place.

I wonder if you set an engineering High limit and set the engineering clamp to off, then the tag will be bad quality if it exceeds those limits

The limits are applied when writing to a tag. No affect on reading from the source.

You probably should make an expression tag that passes the original value when within range, or a static value with bad quality (using forceQuality()) when out of range. Historize the latter tag instead of the original.

3 Likes

Was thinking of something like that but was worried (irrationally?) about system overhead, the timestamp of the data, how to mark it bad and any added upkeep when adding new tags and so on.

Do you think using an expression tag with another tag to historize as you suggested, would be easier then something like the existing system where they remove the high values from anything requesting historical data.

I already have a script that is POSTing the historical data to another database and I guess I could check the value and not add it to the list of values to POST but I think I need to average the value first and I don't know how to remove the high values from the average.

Are you requesting the Average with one of the tag history functions?

The expression tag seems much easier IMO, but that doesn't mean there arent other ways to accomplish this if needed.

I'm not currently but I think that is what they will want. At least I think this is just giving me a value each minute not an average of the value over the minute.

history = system.tag.queryTagHistory(paths=folderPathArray, intervalMinutes=1, startDate=startTime, endDate=endTime, returnSize=POSTDelay)

Yeah, it's an odd one and that makes it hard to Google up an answer. I've never really thought about how and when a tag gets a bad data code. If the I/O server can't talk to a PLC, that seems right and also if the 4-20 is zero and so on.

They will also be using KEPServerEX and I'm trying to see if it can mark the data bad before the Ignition Tag Historian gets it. It has Advanced Tags for logic and math functions but I've never used them.

Would it be possible to delete these high values from the Tag Historian after they are logged? I think they only need to be logged once a minute so maybe a script could run every few minutes and look for any samples at a certain value and delete them or change their quality to bad?

Oooh! Dodgy! The query would have to handle history partitioning, etc.

Especially for tags that use analog deadband, as it will scramble the reconstruction of the "compressed" samples.

Caution, this way lies madness!

Seriously though, you can write a Custom Tag History Aggregate.

If say you want to produce a simple average and only use values within a certain range then you can do that. It isn't the cleanest solution, Phil's expression tag approach is far nicer and cleaner.

However, if you really want to do it, it can be done.

There are some caveat's but it could be done with something like this:

def simpleAverageWithLimit(qv,interpolated,finished,blockContext,queryContext):
    cnt = blockContext.get('cnt',0)
    tot = blockContext.get('tot',0)

    mn = 0
    mx = 30000
    if qv.quality.good and mn < qv.value < mx:
        cnt += 1
        tot += qv.value

    if finished:
        return tot / cnt
    
    blockContext['cnt'] = cnt
    blockContext['tot'] = tot

If deleting is not doable then I'm guessing there is no way to just change the quality to bad then?

emphasis added

Yes, before recording it. Afterwards, no. For the same reason deletion is a problem.

Yes, but I was trying to avoid the extra tags and such. Oh well. :frowning:

Was just thinking that changing the dataintegrity value in the table would maybe be less troublesome than trying to delete the sample/record. The sample no longer existing is just as bad as its quality randomly being changed to bad?

Changing the quality in the table will break the assumptions that Ignition's historian makes when deciding to record a given point in analog deadband mode. Bad breakage. Not sure what would happen in discrete deadband mode. Probably not good, either.

The expression tag is the the answer™.

Thanks for the help, just trying to trying to find if there is an answer and then the best one without spending a few weeks trying each one out. Support just said no. Then I asked about using two tags and they said yeah but you'd be add more tags and the timestamps could be an issue and then I asked about deleting the sample in the historian. They said that could be done with a number of different quires to get the tags value and ID and then delete it but recommend not doing it and did not say anything about it breaking anything specific but in general manually managing the tag historian table can break your trends/historian.

Not knowing enough about it there are a lot of things that sound like good maybes. A second tag for all your tags, Transaction Groups, deleting the tag from the Historian, marking it bad and so on. Maybe there is a way to have KEPServerEX mark the value bad before Ignition gets it?

Why all of them?

This is the kind of technique you use for your hard cases, not for everything. For the general case, fix your hardware datapath to deliver appropriate values. That is, fix the problem at the source.