Query Tag History Across Partitions Results in 0's Possible Bug

Hi All,

I believe I have come across a bug in Ignition 8.1.19 (an upgrade is possible but only if this issue was resolved) when using system.tag.queryTagHistory() and using a date range that starts on one side of a partition and ends on the other a 0 is added to the dataset. From there I am adding the values to a list and calculating the range from minimum and maximum. I haven't been able to find a work around for the above using any of the additional parameters in queryTagHistory() or using the range, or minimum and maximum with queryTagCalculations().
I have also verified that in either partition for the last hour there is no 0 value to be returned from the database tables so my assumption is somewhere in the query its detecting there is no data for that timestamp in the current partition and rather than moving to the next returns a 0.
The script I am using is below:

#Gather parameters
graphType = "Electrical"
meterName = "Abar Vacuum"
tagArea = "Duablock"
startDate = system.date.getDate(2023, 8, 30)
startDate = system.date.setTime(startDate, 20, 0, 0)
endDate = system.date.getDate(2023, 9, 1)
endDate = system.date.setTime(endDate, 4, 0, 0)
aggregation = "Hour"
returnSize = 1
if aggregation == "Hour":
	#Calculate required dates for Hour intervals
	tempHourSD = system.date.getHour24(startDate)
	startDate = system.date.setTime(startDate, tempHourSD,0,0)
	tempHourED = system.date.getHour24(endDate)
	endDate = system.date.setTime(endDate, tempHourED,0,0)
	returnSize = system.date.hoursBetween(startDate,endDate)
elif aggregation == "Day":
	#Calculate required dates for Day intervals
	startDate = system.date.setTime(startDate, 0,0,0)
	endDate = system.date.addDays(system.date.setTime(endDate, 0,0,0),1)
	returnSize = system.date.daysBetween(startDate,endDate)
elif aggregation == "Month":
	#Calculate required dates for month intervals
	tempYearSD = system.date.getYear(startDate)
	tempMonthSD = system.date.getMonth(startDate) #Jan is 0
	startDate = system.date.getDate(tempYearSD,tempMonthSD,1)
	tempYearED = system.date.getYear(endDate)
	tempMonthED = system.date.getMonth(endDate) #Jan is 0
	endDate = system.date.getDate(tempYearED,tempMonthED+1,1)
	returnSize = system.date.monthsBetween(startDate,endDate)
tag = []
#Create tag path
if graphType == "Electrical":
	tag.append("[Sub_Metering]" + tagArea + "/" + meterName + "/Active Energy Delivered")
elif graphType == "Water":
	tag.append("[Sub_Metering]" + tagArea + "/" + meterName + "/Water Usage")
else:
	tag.append("[Sub_Metering]" + tagArea + "/" + meterName + "/Gas Usage")
idx = 0
headers = ["X","Y"]
data = []
while idx < returnSize:
	if aggregation == "Hour":
		startDateTemp = system.date.addHours(startDate,idx)
		endDateTemp = (system.date.addHours(startDate,idx+1))
		if returnSize > 24:
			formattedDate = system.date.format(startDateTemp,"dd HH:mm")
		else:	
			formattedDate = system.date.format(startDateTemp,"HH:mm")
	elif aggregation == "Day":
		startDateTemp = system.date.addDays(startDate,idx)
		endDateTemp = system.date.addDays(startDate,idx+1)
		formattedDate = system.date.format(startDateTemp,"E dd")
	elif aggregation == "Month":
		startDateTemp = system.date.addMonths(startDate,idx)
		endDateTemp = system.date.addMonths(startDate,idx+1)
		formattedDate = system.date.format(startDateTemp,"dd-MMM-yy")
	DS = system.tag.queryTagHistory(tag, startDateTemp, endDateTemp)
	#create individual intervals for each hour/day/month and grab the data.
	dataRange = 0
	if DS.getRowCount() != 0:
		tempList = []
		for row in range(DS.getRowCount()):
			tempList.append(DS.getValueAt(row,1))
		minVal = min(tempList)
		maxVal = max(tempList)
		dataRange = maxVal - minVal
	data.append([formattedDate,dataRange])
	idx = idx + 1
DS2 = system.dataset.toDataSet(headers,data)

I have tried using all combinations of includeBoundingValues, noInterpolation, and ignoreBadQuality on both queryTagHistory and queryTagCalculations, and setting the date so the date range doesn't cross the partition but approaches the bounds, and setting the returnSize to -1 and 0 to remove seed values.

Hopefully someone has come across this before.

Tim

So while I was typing up this post I also found the solution retesting the various combinations. Instead of reducing the end date to be within the partition bounds I forced it into the next. It appears the history partitions end date is the same millis as the start date of the next partition. I believe internally the system is then detecting that the date range is within the single partition and uses a seed value of 0 for the boundary value whereas when I force it into the next partition it grabs the first value as the boundary value.

So the above solved the issue for the final time period however if there is no change in the time period it will still use the 0 and thus calculate a range of current value - 0. I can supress this by removing 0's from the dataset coming in but if they're recorded I'd like to be able to reference them.

After a few back-and-forths with support it was confirmed to be a bug in how interpolation works for analog deadband historical tags. It was getting the current tag value if there was no data in the date range selected within the partition rather than getting the next data point in the database in the following partition.
eg for a date range in September with data in October's partition the value returned was the tags current value (February) rather than the first value in Octobers partition.
No ETA for a fix

2 Likes

We are seemingly experiencing this issue still in 8.1.41 so I assume the fix still isn't in :frowning:

I should clarify that if I query a time range in a history partition where no value for the tag has been written yet then I get 0s. We have tags that rarely change but we are sometimes using their historical values for display purposes. We have the max time between writes set to 24 hours but even at 1 hour we still have a period after the new partition table has been created but before the hourly value has been written where we only get 0s. This worked for us in 7.9 but has stopped working in 8.1

Have you confirmed the tags are using Analog for the Tag History Deadband Style (or Auto if they are a Float or Double data type)? If so, yes it likely still exists (BUG-2060). Depending on the tag/value trend line preferences forcing the Deadband Style to Discrete may help, but for Analog it sounds like the issue is present.

The tags I have been having issue with are integer tags using Discrete for deadband.