Having an issue on a client's site where they have started using pre-processed partitions. The data is being imported into the SQL server quite quickly, but we have noticed delays in processing of the data for the pre-processed partitions. Worst case we had trends on SCADA that had no data outside of Sunday 3AM, and it wasn't until around Monday 1PM (46 hours later) that all of a sudden that data appeared in the pre-processed partition and therefore the SCADA trend. What is the mechanism that controls the generation of the pre-processed partition and is there any way to diagnose it or adjust the update frequency? For reference for this month we are sitting at 550 million raw data points, 168 million pre processed data points.
Update, we waited 1.5hrs and then we failed over to the backup server. At that point, data came through and continued to process live data (it seemed to pre-process live data in real time), however it did not back fill the pre-processed data for the missing time interval. We then restarted the main server and it reassumed control, and it now is also pre-processing the live data but as yet has not backfilled.
The data is confirmed to exist in some capacity as raw data, as zooming in on the trend so that the raw data table is used rather than the pre-processed data table, shows some data in the data range that is flatlined when presumably using the pre-processed data.
Update, apparently it is "normal" behaviour that the OptimizedTableEngine shuts itself down until gateway restart:
We have to ultimately resolve why it shuts in the first place, and in the interim find a solution to automatically restart the gateway when it fails.
1 Like
Doesn't sound great that this is by design... 