Delete or Set dataIntegrity=0 on history table records sql_data_1_YYYY_mm

Ignition
1

Hi all

I use PostGreSQL
In history tables, sql_data_1_YYYY_mm for some of the tags I have values that are wrong and messing up all graphs and additionally calculated data and tables

I built a pSQL script that determines what records are bad and saves into a table the begin and end t_stamps of the adiacent valid records of the specific tag.
After that I built a pSQL code that finds all history records from all tags related to the same device (same UDT) and will get rid of the records between the good t_stamp's adiacent to the spiked one. This because only on one tag I can biuld a rule to find the spikes, but all tags read wrong data from that device for a few seconds.
AL OK until this point.

MY QUESTION: what is the best practice?
1.to delete the records from the sql_data_1_YYYY_mm tables
or
2. to set the dataIntegrity to 0 for the records that I want to be ignored

In case of option 2 will those records be ignored in all places such as graphs, history, and other built in interogations such as TagHistoryBindings for tables? except ofchourse manually written select scripts that read directly the raw tables.

Also, with the option 2 the bad data will be still there for future investigations if required.

1 Like
2

In lack of any suggestions, I went to disable bad history records by updating dataintegrity=0 for the wrong records.
It looks like it's working fine, and finally I can see valid graphs that scale to the real data and not to the overly exagerated spikes.
Thansk anyway.