# Issues Combining Datasets/Using Dataset Tags

**URL:** <https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539>\
**Category:** Ignition\
**Created:** [August 7, 2023, 9:01pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539 "2023-08-07T21:01:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joseph\_Alcide](https://avatars.discourse-cdn.com/v4/letter/j/ebca7d/32.png) [@Joseph\_Alcide](https://forum.inductiveautomation.com/u/Joseph_Alcide)\
**Post date:** [August 7, 2023, 9:01pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/1 "2023-08-07T21:01:42Z")

</div>

Hello All,

Thank you very much to everyone who has helped me so far today. I am now trying to combine the datasets that I had previously stored as dataset tags. However, when I try to append their together, I get a really strange result. I have attached what my first two datasets looked like when I stored them in the tags. When I store them, it also shows the weird structure within the tag itself. Does anyone know why it is doing this? Additionally, does anyone know if there is a way when you create tags from the script editor, if it is possible to specific the data type as a dataset, instead of having to go and manually change it?

 ![Attempted Combined Dataset](https://us1.discourse-cdn.com/inductiveautomation/original/3X/4/e/4ecc5cfeb0a004a82cc9bc5ab5daccef71789b92.png)  
 ![Dataset 1](https://us1.discourse-cdn.com/inductiveautomation/original/3X/3/3/33af08095273c547c996d6c98e7b764d1f19e9e1.png)  
 ![Dataset 2](https://us1.discourse-cdn.com/inductiveautomation/original/3X/4/a/4aeaeaafd49fe1adc3b888a82f8ce459f142b438.png)

Example code showing how I sent the stored the data in the dataset tag:

```auto
idata = system.file.openFile('json')
bytes = system.file.readFileAsBytes(idata)

from org.apache.commons.io.input import BOMInputStream
from java.io import ByteArrayInputStream, ByteArrayOutputStream
from java.lang import String

bomInput = BOMInputStream(ByteArrayInputStream(bytes))
outputStream = ByteArrayOutputStream(len(bytes))
bomInput.transferTo(outputStream)
bytesOut = outputStream.toByteArray()
strOut = String(bytesOut, "utf-8")

obj = system.util.jsonDecode(strOut)

newlod = [{d['Term']: d['Estimate']} for d in obj]
headers, data = zip(*[d.items()[0] for d in newlod])
ds = system.dataset.toDataSet(headers, [data])

print(ds)
event.source.parent.getComponent('Power Table').data = ds

# The provider and folder the Tag will be placed at. 
baseTagPath = "[default]Coefficients"
 
# Properties that will be configured on that Tag.
tagName = "MLR"
valueSource = "memory"
Value = ds
sampleMode = "TagGroup"
tagGroup = "Default"
 
# Configure the tag.
tag = {
            "name": tagName,           
            "value" : Value,
            "valueSource": valueSource,
            "sampleMode" : sampleMode,
            "tagGroup" : tagGroup
        }
 
# Set the collision policy to Abort. Thus, if a Tag already exists at the base path,
# we will not override the Tag. If you are overwriting an existing Tag, then set this to "o".
collisionPolicy = "o"
 
# Create the Tag.
system.tag.configure(baseTagPath, [tag], collisionPolicy)

```

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [August 7, 2023, 9:42pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/2 "2023-08-07T21:42:45Z")

</div>

> [@Joseph\_Alcide](#):
>
> However, when I try to append their together, I get a really strange result

Yea... what strange result ?

You're showing code that's barely relevant to the issue at hand.  
You have 2 datasets and you want to combine them. We don't need to see the code that produces the datasets, you have them already.  
What did you try to combine them, and what didn't work ?

The issue you might have with your two datasets is that the columns might not be in the same order. You'll need to fix that.

Apart from that... the only thing we can do without details about the actual issue, is give you a generic solution. A better one might exist (actually I believe there's a better solution to the whole thing, I still don't see why you're building 2 dataset tags to then combine them).

```python
data = [second_ds.getValueAt(0, second_ds.getColumnIndex(col)) for col in first_ds.columnNames]
combined_ds = system.dataset.addRow(first_ds, data)

```

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [August 8, 2023, 2:18am UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/3 "2023-08-08T02:18:59Z")

</div>

> [@pascal.fragnoud](#):
>
> The issue you might have with your two datasets is that the columns might not be in the same order. You'll need to fix that.

Fyi, the easiest way to reorder dataset columns is to use the `system.dataset.filter` function. It's super handy

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [August 8, 2023, 4:22am UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/4 "2023-08-08T04:22:50Z")

</div>

the `columnRearrange` expression function is also really handy.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [August 8, 2023, 6:22am UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/5 "2023-08-08T06:22:37Z")

</div>

Right, I forgot it could do that.  
I knew there was something like this so I went looking for a `rearrange` function but couldn't find it and rolled my own.

---

<div class="post-metadata">

**Author:** ![Joseph\_Alcide](https://avatars.discourse-cdn.com/v4/letter/j/ebca7d/32.png) [@Joseph\_Alcide](https://forum.inductiveautomation.com/u/Joseph_Alcide)\
**Post date:** [August 8, 2023, 1:13pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/6 "2023-08-08T13:13:17Z")

</div>

Thank you for the help! It works now!

---

<div class="post-metadata">

**Author:** ![Joseph\_Alcide](https://avatars.discourse-cdn.com/v4/letter/j/ebca7d/32.png) [@Joseph\_Alcide](https://forum.inductiveautomation.com/u/Joseph_Alcide)\
**Post date:** [August 8, 2023, 1:13pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/7 "2023-08-08T13:13:32Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [August 8, 2023, 1:58pm UTC](https://forum.inductiveautomation.com/t/issues-combining-datasets-using-dataset-tags/77539/8 "2023-08-08T13:58:34Z")

</div>

> [@pascal.fragnoud](#):
>
> ```auto
> data = [second_ds.getValueAt(0, second_ds.getColumnIndex(col)) for col in first_ds.columnNames]
> 
> ```

simplified version:

```python
ds = system.dataset.addRow(ds1, [ds2.getValueAt(0, c) for c in ds1.columnNames])

```

`getValueAt` can take the name of the column, which makes finding the index unnecessary.  
So we can just use the column names of the first dataset to extract the values in the right order.