Create pruned dataset in Python scripting

Ignition
1

In a Python script transform, I have a historian dataset that looks like this. It might have any number of columns from 1 to 10 in addition to the t_stamp column. I want to builld it into a new dataset using only rows that come after a given timestamp -- in this case, it is 7:00am. I want to also insert a row at the exact 7:00:00 AM mark with 0 for each value. Basically I am pruning times before a cutoff time and initializing each column.

image
image1194×496 65.3 KB

I started along this path:

dataset = # (see above)
cutoffDate = "2024-03-15 7:00:00 AM"
for i in range(dataset.getRowCount()):
		t_stamp = dataset.getValueAt(i,"t_stamp")
		if t_stamp >= cutoffDate:
			# (insert the entire row into a new dataset...)

...But kind of got stuck when I realized there's not an easy way to grab a row of data out of a dataset and throw it into a new dataset (or maybe there is?). Would a pyDataSet work better?

Alternatively, is there a way for system.tag.queryTagHistory to stop grabbing "early" values? My dataset comes from that function, and even when supplied with the proper cutoffDate of 2024-03-15 7:00:00 AM, it is grabbing a handful of values before this timestamp (in some cases but not all). Any guidance is appreciated.

2

I would use my where() expression function from my Integration Toolkit. It would look something like this:

where(
	{path.to.historian.binding.prop},
	dateIsAfter(
		it()['t_stamp'],
		{path.to.cutoff.timestamp.prop}
	)
)
3

Thanks Phil; I ended up using built-in functions to do this but at some point I do want to grab that Integration Toolkit and apply it. I keep running into problems that it would solve easily. But what ended up working for me was this:

historyDataset = # (time series / historian dataset here)
startDate = "2024-03-18 7:00:00 AM" # this is a dynamic value based on when this transform is taking place.

deletionRowIndices = []
for i in range(historyDataset.getRowCount()):
	t_stamp = historyDataset.getValueAt(i,"t_stamp")
	if t_stamp <= startDate:
		deletionRowIndices.append(i)
newDataset0 = system.dataset.deleteRows(historyDataset, deletionRowIndices)
	
# Append a row of all 0's at the startDate to initialize the chart.
	
columnCount = historyDataset.getColumnCount()
newRow = [startDate]
for j in range(columnCount - 1):
	newRow.append(0)
	
newDataset1 = system.dataset.addRow(newDataset0,0,newRow)

return newDataset1