# system.tag.queryTagHistory - merge row by timestamp

**URL:** <https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373>\
**Category:** Module Development\
**Created:** [December 12, 2018, 3:43pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373 "2018-12-12T15:43:23Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 12, 2018, 3:43pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/1 "2018-12-12T15:43:23Z")

</div>

Hi @Colby.Clegg,

I query history db with :`
this.gatewayContext.getTagManager().queryHistory(tagHistoryQueryParams, this.streamingDatasetWriter);
`

which provide the same function as : **system.tag.queryTagHistory**

With mode **WIDE** and ReturnSize(-1) and no interpolation, the result dataset contain one line for each each tag datachange.

I would like to obtain a result dataset with all the raw tags datachanges (no interpolation) merged when the t\_stamp is the same.

Is it possible with **system.tag.queryTagHistory**???

History DB :

![](https://us1.discourse-cdn.com/inductiveautomation/original/2X/a/a8a42c980f40e6302d10f6fdd025f436480a36dd.png)

Result :

 ![](https://us1.discourse-cdn.com/inductiveautomation/original/2X/4/46b8e70406d8bca5250af72d652aae9793040668.png)

In this case =\> only the last Row, but if some datachanges with different timestamp =\> different row.

**Other question** : what is the kind of streamingDataset we need to use with

```auto
this.gatewayContext.getTagManager().queryHistory
```

to block until all the data are available ?

---

<div class="post-metadata">

**Author:** ![chasondeshotel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/chasondeshotel/32/20032_2.png) [@chasondeshotel](https://forum.inductiveautomation.com/u/chasondeshotel)\
**Post date:** [December 12, 2018, 5:06pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/2 "2018-12-12T17:06:55Z")

</div>

I’m curious, if that provides the same function as system.tag.queryTagHistory, why not just use system.tag.queryTagHistory?

Also what is the streamingDatasetWriter?

I’m fairly new to Ignition and have done quite a bit of scripting with tags and tag history, I’m wondering what there is to gain with your method. Sorry I know this doesn’t answer your question

---

<div class="post-metadata">

**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\
**Post date:** [December 12, 2018, 5:21pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/3 "2018-12-12T17:21:24Z")

</div>

> [@mazeyrat](#):
>
> Other question : what is the kind of streamingDataset we need to use with

To do this, you would just use a `BasicStreamingDataset` - although in your case, I think a custom `StreamingDatasetWriter` would be better - though you could try just overriding `write()` on `BasicStreamingDataset`. You'll need to update the `dataList` and `qualityList` fields inside the dataset. By overriding write, you could match multiple streamed rows by the timestamp.

> [@chasondeshotel](#):
>
> I'm fairly new to Ignition and have done quite a bit of scripting with tags and tag history, I'm wondering what there is to gain with your method. Sorry I know this doesn't answer your question

@mazeyrat is doing module development - so this is actual Java code directly against the API.

---

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 12, 2018, 6:32pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/4 "2018-12-12T18:32:09Z")

</div>

@paul-griffith thanks a lot, overrinding seem to be a good solution to merge data.  
But what do you mean by  
**You’ll need to update the dataList and qualityList fields inside the dataset.**  
I need to create new fields ? I can’t modify the raw data inside the datset ?  
write method only append a new row ?

Another question,  
if I use a BasicStreamingDataset with TagManager::queryHistory,  
when the function return, all the data are available in the dataset or I need to override finish() method to be notified ?

Can I use a StreamingDataset class with TagManager::queryHistory and use readFully() to wait until all the data are available ?

---

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 12, 2018, 9:27pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/5 "2018-12-12T21:27:24Z")

</div>

> [@paul-griffith](#):
>
> BasicStreamingDataset

Ok I think I've understood what you mean with the overriding of write method of the BasicStreamingDataset.  
I will keep row in a buffer as long as the date is the same and flush the row when the date change.

---

<div class="post-metadata">

**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\
**Post date:** [December 12, 2018, 9:31pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/6 "2018-12-12T21:31:59Z")

</div>

> [@mazeyrat](#):
>
> You'll need to update the dataList and qualityList fields inside the dataset.I need to create new fields ? I can't modify the raw data inside the datset ?write method only append a new row ?

The default implementation of `write` in `BasicStreamingDataset` just adds each data and quality array to fields on the class - `dataList` and `qualityList`. If you override `write`, you can keep the rest of the class working fine as long as you take responsibility for updating those lists yourself. The definitions are pretty simple:

```auto
    List<Object[]> dataList;
    List<DataQuality[]> qualityList;

```

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [December 12, 2018, 9:48pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/7 "2018-12-12T21:48:35Z")

</div>

> [@mazeyrat](#):
>
> flush the row when the date change

Don't forget to flush when you get `finish()`.

---

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 12, 2018, 10:09pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/8 "2018-12-12T22:09:37Z")

</div>

Yes Indeed. Thanks for the reminder.

---

<div class="post-metadata">

**Author:** ![Colby.Clegg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/colby.clegg/32/3282_2.png) [@Colby.Clegg](https://forum.inductiveautomation.com/u/Colby.Clegg)\
**Post date:** [December 13, 2018, 4:22pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/9 "2018-12-13T16:22:07Z")

</div>

If you use the TagHistoryQueryFlags.NO\_INTERPOLATION flag, it won’t include any rows that _only_ have interpolated values. Then, you can set your return size to be based on the smallest amount of time you would ever see changing values (typically 1 second).

For example, if you query for 1 day and request 86400 results, but have NO\_INTERPOLATION set, what you should get is a row at a 1 second resolution each time any of the values in that row have changed. If you didn’t have the flag, you would of course get a row for each second of the day.

---

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 14, 2018, 4:29pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/10 "2018-12-14T16:29:25Z")

</div>

Thanks a lot @Colby.Clegg, I 've tested this solution to gather only the raw history datachange with the following settings :

```auto
flags = Flags.createFromValue(TagHistoryQueryFlags.BOUNDING_VALUES_YES);
flags.and(TagHistoryQueryFlags.NO_INTERPOLATION);
flags.and(TagHistoryQueryFlags.NO_SCANCLASS_VALIDATION);
```

with a return size set to :

```auto
Math.abs(endDate.getTime() - startDate.getTime())/1000
```

and I have one row for ecah second.  
on this period my history contain only 14 tags value datachange.  
is it normal ?

---

<div class="post-metadata">

**Author:** ![Colby.Clegg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/colby.clegg/32/3282_2.png) [@Colby.Clegg](https://forum.inductiveautomation.com/u/Colby.Clegg)\
**Post date:** [December 14, 2018, 4:57pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/11 "2018-12-14T16:57:52Z")

</div>

Flag.and() doesn’t mutate the object, it returns a new one. So try:  
`flags = Flags.of(TagHistoryQueryFlags.BOUNDING_VALUES_YES, TagHistoryQueryFlags.NO_INTERPOLATION, TagHistoryQueryFlags.NO_SCANCLASS_VALIDATION);`

---

<div class="post-metadata">

**Author:** ![mazeyrat](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/mazeyrat/32/10955_2.png) [@mazeyrat](https://forum.inductiveautomation.com/u/mazeyrat)\
**Post date:** [December 14, 2018, 6:40pm UTC](https://forum.inductiveautomation.com/t/system-tag-querytaghistory-merge-row-by-timestamp/21373/12 "2018-12-14T18:40:16Z")

</div>

> [@Colby.Clegg](#):
>
> Flag.and() doesn't mutate the object, it returns a new one

Perfect. It return now exactly what's I needed ! 👏 👏 👏