PreProcessing Historical Data

Can someone let me know what the vtype column numbers relate to. I know it will be Min,Max,Avg and so on but didn’t see any documentation as to the cross referencing of the values logged to meaning.

Also from looking at the partitions table no matter how the partitioning is set up it looks like this preprocessed data table will partition at the monthly level.

Next question is does this table get pruned? I would like the option to have pruning of this table set separately from everything else.

Colby,
After setting everything up and letting it run all night the Local machine is not preprosessing the data for some reason. See the attached for settings for both. Also I am using the Historian splitter you wrote for use on this test machine setup. Also note the preprocessing table was created in the local DB but there have been no records inserted into it. the Office appears to be working as expected though.




Hi,

Are there any errors in the console log? You can export them and attach them if you want me to take a look.

If you go to Status>Store and Forward, how do things look? Are they going through? Are there any quarantined records? If everything seems fine, I would guess that some sort of error must have occurred when starting the processing engine, and so it’s not running. By disabling and re-enabling pre-processing, you can restart the engine, which should relog the error, if my guess is correct.

If there’s still nothing, you can try going to Console>Levels and set the logger “SQLTags.History.SimpleDBHistoryWriter” to debug (you might also want to search for “OptimizedTableEngine” and set that to debug as well). Among all of that, something should be able to give us some clue as to what’s going on.

Also, as you’ve noted, the pre-processing is fixed to 1 month partitions. I’ll make a feature request ticket for a setting for that, as well as different pruning options, for 7.7.

Regards,

Colby
Looks like everything is working at this time. Sorry for late reply.
Do you have any documentation as far as vtype field values and what the mean.
I need to write queries against this table and need this info.

Hey,

Sorry, I missed this post. The vtype is used to indicate the role of the value, as well as the order for pre-processed windows. It is a bit flag value, so values can have multiple roles.

First off, 0 means that the value is the only value for the window. 1=min, 2=max, 4=last value in window, 32=“first” value (to order min and max), and 64=first value of window.

All summed up, though, I think what is important for you is that querying with “order by t_stamp asc, vtype desc” will get you the values in the correct order.

Regards,