I have a problem with certain tags failing to historize. These tags appear to have been sent to sqth_te (so they have a tagpath and tag_id), but do not appear in any of the partitioned historical tables. History is turned on in the UDT, and it looks like it propagated down to the instances, but i’m not convinced that the UDT instances’ tags are actually historizing even though their values are changing. This doesn’t happen for all tags/udts, but I am unsure as to why this is.
The error I am getting is this:
No historical tag information could be found for any of the specified paths. Check that paths are correct. Paths: [devices/rod pumps/32h/interface/sts/todayruntime, devices/rod pumps/32h/interface/sts/inferredprod, devices/rod pumps/32h/interface/sts/todaycycles]
Most of the easy chart bindings that we’re using have cell update binding, but again, I think history is simply not being recorded for several tags.
The tag history options we’re using are (mostly) default and identical in each UDT:
History Enabled = true
Storage Provider = “History”
Deadband Style = “Auto”
Deadband Mode = “Absolute”
Historical Deadband = “0.01”
Sample Mode = “On Change”
Min Time Between Samples = 0
Min Time Units = “Seconds”
Max Time Between Samples = 0
Max Time Units = “Hours”
It looks like only ~3515 / 4520 tags are being historized, the rest are being ignored. Not sure if re-starting the tag does the trick (in fact, some of these UDTs were completely abolished and re-created). All of the tag groups are the same.
I am definitely seeing some issues with history of Members of UDTs when creating them via a script. In my steps to replicate this, all items are missing sqlth_te records though. Are you certain that these records are getting created? If there is more to this, I want to get to the bottom of it.
If the records do exist, can you answer the following questions:
What type of DB is the history being stored in (MySQL, SQL Server, Oracle), and what is the version?
How are your UDTs getting created? Were they migrated from 7, or newly created? If newly created, were they scripted, imported or entered in some other manner?
Is there a specific type of tag that looks to be having problems (OPC, Memory, Expression, ect.)?
Is there a specific datatype that looks to be having a problem?
Would it be possible to export the definition of one of the tags you are having problems with and post it?
In my testing with the 8.0.0 release I saw the issue of tags not creating the entry in the sqlth_te table, however in my case it seemed to be related to a current tag status of Uncertain_LastKnownValue that seems to happen when the gateway on my laptop starts. When the tag has this status I am unable to edit values or restart the tag and it seems that any normal tag processing does not work. I have to go through and disable then re-enable all the tags at which point the gateway seems to pick up normal processing of the tags. In my case most of the tags that this happens with are OPC tags that are linked to a device that is not connected at the moment. After I disable and re-enable the tag status changes to Bad_Stale. When the tag is in a UDT I can only disable and re-enable the tag through the UDT definition and not through the tag tree as well.
I haven’t tested recently but at one point I wasn’t getting history logging on instances of my UDT until after I restarted the tags. That issue appeared to be fixed in one of the last dailies.
Try restarting your tags and see if anything changes.
Yeah, we are seeing that some (maybe not all, but definitely some) sqlth_te records are being generated, just not being deposited into any of the history tables.
The DB is MS SQL Server 2016.
Our UDTs were mostly migrated from 7.9. I'd say around 90% of them. Most of their history does work, however. The UDTs that are experiencing this problem were indeed created via a generating script (the data type was made in 7.9.10, however, but the instances were recently re-generated)
They are all Derived tags. However, in particular, the problematic ones are pointing at SQL tags. Our UDT tag structure has two layers: a mapping (OPC, SQL, and memory tags) and an interface (all derived tags, some memory tags) layer.
We're trying to historize integers and floats, not sure if strings are problematic as well.
It is interesting that you mention derived tags pointing to SQL tags as being an issue. We recently fixed an issue where SQL tag members in UDTs weren’t always starting properly on a restart of the Gateway. This did get fixed in the final release of 8. I will try to duplicate it as well, but if you have the ability to check to see if this is still an issue with the final 8.0 release it would be appreciated.
Yeah it’s still an issue with the full release. I just realized that the mapping layer are not actually SQL tags, they are all memory tags getting their values from 2-3 SQL tag datasets that write to them on property change to minimize how many times we hit the database. So we never really had a problem getting the data out of the database. The tags are just refusing to historize.
I tried re-starting individual tags as well as the entire folder (not sure if that actually does anything), and nothing seems to have changed. The tags do not change very often, I must add, maybe once every 10 minutes?
I attached two images of the queries I used to check the tables:
We’ve done a lot of work on this recently, which should make it into the 8.0.1-rc by Monday. From what we’ve found so far, the main issue doesn’t appear to be problems with setting propagation, but instead some issues with the way values are generated, bundled, sent, etc, stemming from the fact that the default mode in 8 is “on change”, and not tag group (previously “scan class”) based execution like in 7.
One thing you could look at to confirm this would be the quarantined area of the store & forward system. Basically, what we’ve found is that on startup/config, many values can be generated, some with duplicate time stamps (especially in the case of expression/derived tags). Those can cause storage to fail, and can affect other tags as well, as data gets combined in the S&F system. Also, a flood of value changes can cause the memory buffer to overflow, losing some values potentially. Again, these conditions are mostly on startup or initial config, but with values that don’t change often, they can have a big impact.
We’ve made some significant improvements to all of this, and I’ll post again once a build is up. Just wanted to give an update before the weekend.
Could you try with 8.0.1-rc when you get a chance and see if things have been improved? We’ve made lots of small adjustments to how history storage is managed on tags, and have resolved all the reported issues that we were able to replicate.
I did install 8.0.1-rc, but for some reason, it caused severe errors with system.util.getLogger() (getLogger was no longer found), which causes some parts of the program to become unworkable so I was forced to downgrade back to a version of 8.0.0.
I’m going to give it another shot later today or early tomorrow and see if it was fixed.
[Ignition - Windows Installer 64-bit] 8.0.1-Nightly
The early access version didn’t seem to be available yesterday around noon when I downloaded it so I went with the nightly. I’ll try the early access now that it’s up.
I think I’m having a similar problem on 8.0.3 RC2, I had some UDT historical tags running on MySQL but decided to move them to MSSQL. All of them worked except 2, I made another DB connection to see if that was the problem but the same 2 tags didn’t get created in the sqlth_te table. For now it seems I managed to fix this by manually adding those 2 tag paths to the sqlth_te table, but this would be problematic in the future as we plan to expand the usage of that UDT considerably.
I am seeing this issue on version 8.1.21.
Have tried Tag Enable/Disable.
Have tried Tag restarts.
The tags in question are reference tags and are all in bad quality.
They are part of a UDT, any tags that are of good quality are not showing the error in the logs.
The magnifying glass on the log viewer "log properties/views" shows random views that this is associated with it.
Plenty other tags in this UDT which have same history setup and are also in bad quality and are not showing this error in the logs.
