Historian is not returning all raw data

General Discussion
1

I am trying to return the raw data from the historian and plot it on a chart. I am not getting all of the data points that exist in the table. I also tried System.Tag.QueryTagHistory and got the same results. What would cause the historian to omit data?

Ignition 8.1.28

2

Show your code. But some requirements to cleanly get historian data "As Stored" via system.tag.queryTagHistory():

  • Use returnSize = -1

  • Omit aggregationMode

  • Use returnFormat = 'Tall'

1 Like
3

Phil - Thanks for the response. I didn't save my earlier code attempts, but here is a retry:

Tags = ['melting/furnace5/fulltiltcycletime']
EndDate = system.date.now()
StartDate = system.date.addDays(EndDate,-14)
print StartDate

ds = system.tag.queryTagHistory(paths=Tags, startDate=StartDate, endDate=EndDate, returnSize=-1,returnFormat='Tall')
print system.dataset.toCSV(ds)

Results:

Mon Feb 05 14:01:55 EST 2024
"path","value","quality","timestamp"

4

Hi, d. Please see Wiki - how to post code on this forum. It's especially important for Python / Jython.

All your results are showing up as quotations of quotations of quotations (with nothing actually quoted) so you've got something wrong in your post. (Quotation markdown syntax is >.) Can you edit your previous post (pencil icon) and fix it? Thanks.

Use the Preview pane to check your posts before posting.

5

How are you determining there is missing data, solely based on what you see in a chart or table? If so, data can interpolated, so what you see in a table or chart might not be what is actually stored in the historian database.

6

T-
Thanks for the info.

7

I am directly querying the SQL tables. I should be getting 80+ records for that tagpath.

8

May I ask how you are determining you should have 80+ records?

9

I run this query to get the TagIDs:

select * from sqlth_te
where tagpath = 'melting/furnace5/fulltiltcycletime'

It returns two entries: 29448, 29474

Then I run this query to get the data.

select *, dateadd(second,t_stamp/1000,'1970-01-01')
from sqlt_data_3_2024_02
where tagid in (29448, 29474)
order by t_stamp
10

Pretty sure you need the tag provider there.

11

Good Catch. I guess I am used to SQL queries. I modified my code to include the provider and now I get 2 out of 80+ expected records


Tags = ['[OMA]melting/furnace5/fulltiltcycletime']
EndDate = system.date.now()
StartDate = system.date.addDays(EndDate,-14)
print StartDate

ds = system.tag.queryTagHistory(paths=Tags, startDate=StartDate, endDate=EndDate, returnSize=-1,returnFormat='Tall')
print system.dataset.toCSV(ds)
12

I thought historian only used a single ID? For both IDs, is the retired column column null?

13

Retired is null for both TagIDs

14

I didn't think historian could use multiple IDs like that, perhaps someone else with more knowledge can verify.

15

We did a reboot at patch time and this seemed to fix the issue. Still don't know the root cause.