# Data pruning and values with incorrect timestamp **URL:** \ **Category:** Ignition\ **Created:** [August 14, 2024, 6:13am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629 "2024-08-14T06:13:45Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![martin.johansson](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@martin.johansson](https://forum.inductiveautomation.com/u/martin.johansson)\ **Post date:** [August 14, 2024, 6:13am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/1 "2024-08-14T06:13:45Z")
Hi, We have been running Ignition for a couple of years now and just started to think about how much data we acctually need to save. Data pruning seems simple enough, but I have a question about what happens with data that has incorrect timestamp. I the best of worlds this should ofcourse be fixed on machine-level. But some machines just have strange behavior. They can have the correct time but startup at 1970, or 2010 or any other date, before they switch to the correct one. Most of them is connected with OPC-UA. So, if I set it up today and want to keep 12 months of data, what happens when Ignition tries to save data with a timestamp older then 12 months? The data itself isn't of much value (as we don't know the correct timestamp anyway), but will the log and/or store and forward fill up with this data? Or will it simply create a table again that gets deleted during the next data pruning? Or is it something else? ---
**Author:** ![martin.johansson](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@martin.johansson](https://forum.inductiveautomation.com/u/martin.johansson)\ **Post date:** [August 21, 2024, 8:12am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/2 "2024-08-21T08:12:20Z")
Bump. ---
**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\ **Post date:** [August 21, 2024, 5:09pm UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/3 "2024-08-21T17:09:22Z")
Data pruning will only prune partitions (tables) where all records timestamps are outside of the configured pruning range. Are you manually logging data in to the historian tables? The timestamps recorded in these tables would have very little to do with any times configured on the machines. ---
**Author:** ![martin.johansson](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@martin.johansson](https://forum.inductiveautomation.com/u/martin.johansson)\ **Post date:** [August 22, 2024, 4:53am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/4 "2024-08-22T04:53:57Z")
With OPC UA the timestamp of the tag is always set from the time of the machine. So if a machine has a datetime of 2012-03-22 11:34:00 it will end up in a table named 2012\_03 even though the current year is 2024. To view that data you need to browse back to that date, nothing will show up in PowerChart if you set it to latest hour or so. So I'm guessing this would count as data outside of a pruning range of 12 months, even though the entry was acctually just made. ---
**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\ **Post date:** [August 22, 2024, 1:33pm UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/5 "2024-08-22T13:33:34Z")
> [@martin.johansson](#): > > With OPC UA the timestamp of the tag is always set from the time of the machine. Except we aren't talking about OPC UA, we're talking about a DB insertion operation. I have a machine which has a date of `8/22/2024 4:38:04 AM` yet a historized tag has a date of `8/22/2024 09:29:47AM` which is the time the value was recorded (based on the time set on the machine where the DB is installed). The only way I know to get a date other than the time of insertion is to force it using `system.tag.storeTagHistory()` with a given timestamp, or manually inserting values into the data table with queries. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 22, 2024, 1:56pm UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/6 "2024-08-22T13:56:01Z")
> [@lrose](#): > > the time the value was recorded (based on the time set on the machine where the DB is installed) You should double check this. For devices using a native driver in Ignition's OPC UA Server, the inserted time is the gateway's time. For devices using a OPC UA client connection, the inserted time is the remote OPC server's time. Tag history timestamps are long integer milliseconds computed by Ignition, not DB native timestamps. ---
**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\ **Post date:** [August 22, 2024, 1:59pm UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/7 "2024-08-22T13:59:37Z")
> [@pturmel](#): > > For devices using a native driver in Ignition's OPC UA Server, the inserted time is the gateway's time Well, I am using a native driver in Ignition's OPC UA Server, so that lines up, perhaps that is where the miscommunication is happening. ---
**Author:** ![martin.johansson](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@martin.johansson](https://forum.inductiveautomation.com/u/martin.johansson)\ **Post date:** [September 2, 2024, 5:18am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/8 "2024-09-02T05:18:45Z")
Still looking for an answer on this question. Don't really want to activate pruning without knowing the result of this. ---
**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\ **Post date:** [September 2, 2024, 6:46am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/9 "2024-09-02T06:46:51Z")
You could test it on a sandbox. ---
**Author:** ![martin.johansson](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@martin.johansson](https://forum.inductiveautomation.com/u/martin.johansson)\ **Post date:** [September 12, 2024, 5:53am UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/10 "2024-09-12T05:53:31Z")
I ended up doing that. The tables get deleted along with the entry in sqlt\_partitions. So anything with an older, incorrect, timestamp will recreate the table for that month (or whatever your partitions are). That table will then be deleted again when the next pruning occurs. So works as I expected it to do. While we did this we also made expression-tags for the few strange behaving machines that have timestamps all over (can be set correctly but occasionally reports wrong timestamp). If you make an expression-tag that references to an OPC-tag, the gateway time will be used for the timestamp instead of the OPC-timestamp (needs to be an expression-tag, doesn't work with a reference-tag). ---
**Author:** ![JohnIrving](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/johnirving/32/37638_2.png) [@JohnIrving](https://forum.inductiveautomation.com/u/JohnIrving)\ **Post date:** [September 12, 2024, 2:31pm UTC](https://forum.inductiveautomation.com/t/data-pruning-and-values-with-incorrect-timestamp/94629/11 "2024-09-12T14:31:36Z")
Good idea with the expression tag. Another way around if you want to collect with gateway time is use _Periodic_ instead of _OnChange_ for sample mode, if it will work for application.