Issues Combining Datasets/Using Dataset Tags

Ignition
1

Hello All,

Thank you very much to everyone who has helped me so far today. I am now trying to combine the datasets that I had previously stored as dataset tags. However, when I try to append their together, I get a really strange result. I have attached what my first two datasets looked like when I stored them in the tags. When I store them, it also shows the weird structure within the tag itself. Does anyone know why it is doing this? Additionally, does anyone know if there is a way when you create tags from the script editor, if it is possible to specific the data type as a dataset, instead of having to go and manually change it?

Attempted Combined Dataset
Attempted Combined Dataset788×379 13.6 KB

Dataset 1
Dataset 1782×382 10.5 KB

Dataset 2
Dataset 2789×380 11 KB

Example code showing how I sent the stored the data in the dataset tag:

idata = system.file.openFile('json')
bytes = system.file.readFileAsBytes(idata)

from org.apache.commons.io.input import BOMInputStream
from java.io import ByteArrayInputStream, ByteArrayOutputStream
from java.lang import String

bomInput = BOMInputStream(ByteArrayInputStream(bytes))
outputStream = ByteArrayOutputStream(len(bytes))
bomInput.transferTo(outputStream)
bytesOut = outputStream.toByteArray()
strOut = String(bytesOut, "utf-8")

obj = system.util.jsonDecode(strOut)

newlod = [{d['Term']: d['Estimate']} for d in obj]
headers, data = zip(*[d.items()[0] for d in newlod])
ds = system.dataset.toDataSet(headers, [data])

print(ds)
event.source.parent.getComponent('Power Table').data = ds

# The provider and folder the Tag will be placed at. 
baseTagPath = "[default]Coefficients"
 
# Properties that will be configured on that Tag.
tagName = "MLR"
valueSource = "memory"
Value = ds
sampleMode = "TagGroup"
tagGroup = "Default"
 
# Configure the tag.
tag = {
            "name": tagName,           
            "value" : Value,
            "valueSource": valueSource,
            "sampleMode" : sampleMode,
            "tagGroup" : tagGroup
        }
 
# Set the collision policy to Abort. Thus, if a Tag already exists at the base path,
# we will not override the Tag. If you are overwriting an existing Tag, then set this to "o".
collisionPolicy = "o"
 
# Create the Tag.
system.tag.configure(baseTagPath, [tag], collisionPolicy)
2

Yea... what strange result ?

You're showing code that's barely relevant to the issue at hand.
You have 2 datasets and you want to combine them. We don't need to see the code that produces the datasets, you have them already.
What did you try to combine them, and what didn't work ?

The issue you might have with your two datasets is that the columns might not be in the same order. You'll need to fix that.

Apart from that... the only thing we can do without details about the actual issue, is give you a generic solution. A better one might exist (actually I believe there's a better solution to the whole thing, I still don't see why you're building 2 dataset tags to then combine them).

data = [second_ds.getValueAt(0, second_ds.getColumnIndex(col)) for col in first_ds.columnNames]
combined_ds = system.dataset.addRow(first_ds, data)
3

Fyi, the easiest way to reorder dataset columns is to use the system.dataset.filter function. It's super handy

1 Like
4

the columnRearrange expression function is also really handy.

2 Likes
5

Right, I forgot it could do that.
I knew there was something like this so I went looking for a rearrange function but couldn't find it and rolled my own.

6

Thank you for the help! It works now!

7

Thank you!

8

simplified version:

ds = system.dataset.addRow(ds1, [ds2.getValueAt(0, c) for c in ds1.columnNames])

getValueAt can take the name of the column, which makes finding the index unnecessary.
So we can just use the column names of the first dataset to extract the values in the right order.

1 Like