# Update either dataset or array

**URL:** <https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226>\
**Category:** Ignition\
**Created:** [May 31, 2024, 9:27pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226 "2024-05-31T21:27:31Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [May 31, 2024, 9:27pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/1 "2024-05-31T21:27:31Z")

</div>

Anyone have a set script or an example of this. I want to create either a float array or a dataset with 60 values. I want to read in a tag and then put that live value into element 0, move element 0 into element 1, element 1 into element 2, etc. The last element in position 59 can be replaced by 58. Essentially I want to keep track of the last 60 values in a dataset or array.

---

<div class="post-metadata">

**Author:** ![Transistor](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/transistor/32/3380_2.png) [@Transistor](https://forum.inductiveautomation.com/u/Transistor)\
**Post date:** [May 31, 2024, 9:32pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/2 "2024-05-31T21:32:57Z")

</div>

1. Are you trying to create a first-in - first-out shift register?
2. What is the clock source (that will trigger a shift)?
3. You need to move 58 → 59 first, 57 → 58 next, etc. I you do it your way then you will fill the array with element 0.
4. Often a more efficient technique is to _not_ shift the array elements but have an array pointer which increments on each shift and wraps when the end is reached. The latest value is written into the element that the pointer is pointing to.

Can you clarify?

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [May 31, 2024, 9:37pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/3 "2024-05-31T21:37:10Z")

</div>

Yes first in first out. I want to do it every second if possible. True about moving 58 into 59, 57 into 58, etc. Your idea about the array pointer would work. Any quick example code you could share?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [May 31, 2024, 10:04pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/4 "2024-05-31T22:04:20Z")

</div>

If shifting is entirely based on time interval, consider just using the [recorder()](https://www.automation-pros.com/toolkit/doc/datasets.html#recorder) expression function from my Integration Toolkit. Configure it to keep 60 rows at 1000ms intervals.

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 1, 2024, 1:39pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/5 "2024-06-01T13:39:44Z")

</div>

Thanks. I checked it out and looks like it would work pretty well. Do I have to import anything in order to use the recorder function?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 1, 2024, 4:28pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/6 "2024-06-01T16:28:25Z")

</div>

You have to install the module (free) in your gateway. It adds a bunch of expression and scripting functions to your system.

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 3, 2024, 12:23pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/7 "2024-06-03T12:23:37Z")

</div>

I looked up the integration toolkit and couldn't find a direct link. Can you share the link or direct me on where to download it from? Thanks

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 12:27pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/8 "2024-06-03T12:27:07Z")

</div>

Here, officially:

[https://www.automation-pros.com/modulesales.html](https://www.automation-pros.com/modulesales.html)

and here (but links back to the above for download):

> **[Ignition Third-Party Module Showcase](https://inductiveautomation.com/moduleshowcase/module/automation-professionals-llc-integration-toolkit)**
>
> Easily extend Ignition's core functionality with Third-Party Modules. The Module Showcase is a great place to find a solution that best meets your needs.

and here (for announcements):

> [@Automation Professionals' Integration Toolkit Module](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934):
>
> Automation Professionals is pleased to announce the immediately availability of a major upgrade to its free Simulation Aids Integration Toolkit module. The new feature justifying the "major upgrade" language is the implementation of eleven thirty-one new expression functions which use iteration to accomplish complex transforms of datasets and lists, to or from other datasets and lists, applying filtering, ordering, and computed new columns along the way. While there are certain operations that…

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 3, 2024, 2:27pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/9 "2024-06-03T14:27:43Z")

</div>

Thanks again. I was able to install the module and the recorder function came right up. With that in mind would I just be creating an expression tag of type dataset and the expression would be the recorder function?

Here is what I have but it is not populating...

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/6/4/6443a93f3feabd926ff9242fc008f4485d775213.png)

I have tried four different ways to do the columns and row values including brackets and no brackets and the tag as a string or as a tag with '{}' around them.

Without an example in the documentation I am unsure of how to get it to work.

Do I need to have a separate dataset memory tag and then another expression tag with the recorder function where I can use the dataset in the recorder function versus the colName and colValue?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 2:34pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/10 "2024-06-03T14:34:08Z")

</div>

> [@michael.black](#):
>
> Without an example in the documentation

Hmm. Will fix.

The column name needs to be a string. The tag needs to be a curly-brace reference. Something like this:

```python
recorder(
	1000, // Pace
	60, // # of rows
	'Weight',
	{[~]Process Trending/Delaq/PV_Silo1TotalWeight}
)

```

If you are going to be subscribing to this tag in Vision (or monitoring in the designer), you should do this:

```python
nonTransient(
	recorder(
		1000, // Pace
		60, // # of rows
		'Weight',
		{[~]Process Trending/Delaq/PV_Silo1TotalWeight}
	)
)

```

If you organize your tags such that the recorder is in the same folder/UDT as the source values, you can use this syntax instead of a full path:

```python
....
	{[.]PV_Silo1TotalWeight}
....

```

Be sure to set the recorder's expression tag to **Event Driven** execution.

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 3, 2024, 2:47pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/11 "2024-06-03T14:47:45Z")

</div>

Here are my results...

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/3/a/3ae823a8e83e4f5a30d584507c01e127db11dd50.png)

I am getting a configuration error for the expression dataset tag.

Suggestions?

I dont have to have an empty dataset already created correct?

How can I troubleshoot the configuration error for the tag?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 2:53pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/12 "2024-06-03T14:53:27Z")

</div>

> [@michael.black](#):
>
> empty dataset already create

No.

> [@michael.black](#):
>
> How can I troubleshoot

First, check the tag diagnostics. (Upper right in the tag editor.)

If not indicated there, wrap the whole expression in debugMe() like so:

```python
debugMe('someKey',
nonTransient(
	recorder(
		1000, // Pace
		60, // # of rows
		'Weight',
		{[~]Process Trending/Delaq/PV_Silo1TotalWeight}
	)
)
)

```

And look at your logs.

(I generally _don't_ indent inside `debugMe()` since I expect to remove that after I find the problem.)

Also note this possible issue with recent versions of Ignition (not just for my toolkit):

> [@Custom Expression Function not recognized "Unknown Function"](https://forum.inductiveautomation.com/t/custom-expression-function-not-recognized-unknown-function/91206/4):
>
> Interesting. I had a report here in the forum about my Integration Toolkit functions not working for expression tags but everywhere else. That reporter fixed it by restarting their gateway and I never reproduced it. Bug in the tag provider hierarchy, perhaps?

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 3, 2024, 2:56pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/13 "2024-06-03T14:56:09Z")

</div>

Looks like its not accepting the function 'recorder'

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/d/d/ddbd4c4f0e42b1f813f770866ea733351dcdcbc7.png)

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 2:58pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/14 "2024-06-03T14:58:57Z")

</div>

Looks like that tag provider bug I linked. You probably need to restart your gateway. Or maybe just the tag provider.

---

<div class="post-metadata">

**Author:** ![michael.black](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@michael.black](https://forum.inductiveautomation.com/u/michael.black)\
**Post date:** [June 3, 2024, 5:18pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/15 "2024-06-03T17:18:06Z")

</div>

I reset the tag provider and it worked! I did notice that whenever the tag was restarted all the values reset. I wanted to write the whole dataset to a memory tag so the values don't get erased if that happens. I tried setting a value change script on the expression tag to read in the expression tag dataset and write it to a memory tag dataset and thats not working....

Here is the expression on the expression tag value changed script...

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/d/c/dc8cd70e609c962a60f7ca245eb6582d521dca76.png)

Here is the memory dataset tag. I did not set any columns.

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/c/6/c6103fe3a89f0654c116660de0e736c749db85ed.png)

Do I have to set the memory dataset with the correct columns (empty) first? Do I have the value change script incorrect?

The memory tag is NULL and its not getting updated at all.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 5:38pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/16 "2024-06-03T17:38:30Z")

</div>

Do not edit/restart the tag. The function instance holding the data is thrown away by Ignition if you do. The recorder must start over when you do that. (Just like it has to start from scratch on gateway startup.)

You can manipulate what the expression does at runtime by moving the pace and rowLimit values to peer memory tags, and you can dynamically control the output column list by providing a dataset of sample values instead of explicitly naming and binding your tag(s).

If you need history in spite of tag resets, use the historian and/or transaction groups. Or, perhaps, use my toolkit's `globalVarMap()` infrastructure to make a long-lived jython variable with your content (scripted, non-trivial).

Side note: When you have a valueChange script on a tag, don't use `readBlocking()` to obtain the new value--it is handed to your script with zero overhead in `currentValue.value`.

---

<div class="post-metadata">

**Author:** ![Ryan\_Deardorff](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan_deardorff/32/63815_2.png) [@Ryan\_Deardorff](https://forum.inductiveautomation.com/u/Ryan_Deardorff)\
**Post date:** [June 3, 2024, 5:50pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/17 "2024-06-03T17:50:32Z")

</div>

I created a script I named FIFO to do just that, using Java's LinkedBlockingQueue:

```python
from java.util.concurrent import LinkedBlockingQueue

def getQueue(name, capacity=4000):
	if not system.util.getGlobals().has_key(name):
		return LinkedBlockingQueue(capacity)
	return system.util.getGlobals()[name]
	
def setQueue(name, q):
	system.util.getGlobals()[name] = q
	
def addToQueue(name, value):
	q = getQueue(name)
	if q.remainingCapacity() == 0:
		q.poll()
	q.offer(value)
	setQueue(name, q)
	
def countInstancesOf(name, value):
	q = getQueue(name)
	ct = 0
	for val in q.iterator():
		if val == value:
			ct += 1
	return ct
	
def clearQueue(name, capacity=4000):
	setQueue(name, LinkedBlockingQueue(capacity)) 
	

```

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [June 3, 2024, 5:56pm UTC](https://forum.inductiveautomation.com/t/update-either-dataset-or-array/91226/18 "2024-06-03T17:56:01Z")

</div>

> [@Ryan\_Deardorff](#):
>
> ```python
> def getQueue(name, capacity=4000):
> if not system.util.getGlobals().has_key(name):
> return LinkedBlockingQueue(capacity)
> return system.util.getGlobals()[name]
> 	
> def setQueue(name, q):
> system.util.getGlobals()[name] = q
> 
> ```

Those are racy and prone to memory leaks. Always use `.setdefault` to reliably initialize a value in a dictionary when a check for the key's presence fails. When the value is an object that will be used all over the library, use the `.setdefault` technique to establish it as a top-level variable in the library.

Study this topic for the techniques needed to avoid crashing your gateway:

> [@Java concurrent queue for tag change scripts](https://forum.inductiveautomation.com/t/topic/82044/1):
>
> Continuing the discussion from [[SOLVED] Are tag Value Changed scripts running asynchronously?](https://forum.inductiveautomation.com/t/solved-are-tag-value-changed-scripts-running-asynchronously/68704/5): We've got code in our standard library that I need to log a value change on a tag and then associate various data points with that value change. The value is located in a UDT, and because this is part of our "standard" library we use for our customers, the number of tags is variable and quite possibly numerous so having a gateway tag event script isn't feasible for this solution. I was reading up on …

If you cannot use my `globalVarMap()`, you have to approximate it with an extra layer in `system.util.getGlobals()`.