We have been running Ignition for a couple of years now and just started to think about how much data we acctually need to save. Data pruning seems simple enough, but I have a question about what happens with data that has incorrect timestamp. I the best of worlds this should ofcourse be fixed on machine-level. But some machines just have strange behavior. They can have the correct time but startup at 1970, or 2010 or any other date, before they switch to the correct one. Most of them is connected with OPC-UA.
So, if I set it up today and want to keep 12 months of data, what happens when Ignition tries to save data with a timestamp older then 12 months? The data itself isn't of much value (as we don't know the correct timestamp anyway), but will the log and/or store and forward fill up with this data? Or will it simply create a table again that gets deleted during the next data pruning? Or is it something else?
Data pruning will only prune partitions (tables) where all records timestamps are outside of the configured pruning range.
Are you manually logging data in to the historian tables? The timestamps recorded in these tables would have very little to do with any times configured on the machines.
With OPC UA the timestamp of the tag is always set from the time of the machine. So if a machine has a datetime of 2012-03-22 11:34:00 it will end up in a table named 2012_03 even though the current year is 2024. To view that data you need to browse back to that date, nothing will show up in PowerChart if you set it to latest hour or so.
So I'm guessing this would count as data outside of a pruning range of 12 months, even though the entry was acctually just made.
Except we aren't talking about OPC UA, we're talking about a DB insertion operation.
I have a machine which has a date of 8/22/2024 4:38:04 AM yet a historized tag has a date of 8/22/2024 09:29:47AM which is the time the value was recorded (based on the time set on the machine where the DB is installed). The only way I know to get a date other than the time of insertion is to force it using system.tag.storeTagHistory() with a given timestamp, or manually inserting values into the data table with queries.
You should double check this. For devices using a native driver in Ignition's OPC UA Server, the inserted time is the gateway's time. For devices using a OPC UA client connection, the inserted time is the remote OPC server's time.
Tag history timestamps are long integer milliseconds computed by Ignition, not DB native timestamps.
The tables get deleted along with the entry in sqlt_partitions. So anything with an older, incorrect, timestamp will recreate the table for that month (or whatever your partitions are). That table will then be deleted again when the next pruning occurs. So works as I expected it to do.
While we did this we also made expression-tags for the few strange behaving machines that have timestamps all over (can be set correctly but occasionally reports wrong timestamp). If you make an expression-tag that references to an OPC-tag, the gateway time will be used for the timestamp instead of the OPC-timestamp (needs to be an expression-tag, doesn't work with a reference-tag).
Good idea with the expression tag. Another way around if you want to collect with gateway time is use Periodic instead of OnChange for sample mode, if it will work for application.