# Java concurrent queue for tag change scripts

**URL:** <https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044>\
**Category:** Ignition\
**Created:** [November 17, 2023, 1:53pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044 "2023-11-17T13:53:19Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [January 31, 2024, 10:58pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/41 "2024-01-31T22:58:34Z")

</div>

> [@pturmel](#):
>
> Okay. Draft for your edification: [queueUtil.py](https://www.automation-pros.com/ignition/queueUtil.py)

This is great, Phil, much appreciated!! I'm plugging it in now.

> [@paul-griffith](#):
>
> I'm all over this thread, full of ideas about a first party implementation of message queueing/looping/whatever.

This would be awesome!

Related to the premise of this thread, and maybe it's an insurmountable task, but it'd also be great to have a reference of things _not to do_ in Ignition. With so many ways that you can _technically_ do things, if you don't have a solid background in Java/Python programming (i.e. a software developer), there are _so_ many ways you can shoot yourself in the foot (and step on a bear trap with the other). Often these don't become apparent until you start seeing performance issues on a system that's grown much larger over time. And more often than not, you don't even know where to start, or what part of your code is the culprit. At least if we have a growing list of things to avoid, then we might have better chance of avoiding at least some of the mistakes. I imagine that your support team, as well as everyone here on the forums, would have a list, at least in your heads (although support hopefully have this documented somewhere in a KB), of the common things that integrators do that they shouldn't. Pinging @Paul.Scott for your thoughts for an addition to the user manual perhaps?

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [February 1, 2024, 10:14am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/42 "2024-02-01T10:14:58Z")

</div>

I don't mind a small performance hit if it's in the benefit of simplicity and clarity. And I'm probably not alone in this boat.

> [@nminchin](#):
>
> (and step on a bear trap with the other)

Are you calling Phil a bear ?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [February 1, 2024, 1:12pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/43 "2024-02-01T13:12:33Z")

</div>

> [@pascal.fragnoud](#):
>
> performance hit if it's in the benefit of simplicity

Huh. Did you look at what I published? Pretty simple.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [February 1, 2024, 1:53pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/44 "2024-02-01T13:53:44Z")

</div>

I did, and it is !  
But you can't beat built-in in terms of simplicity.

And by that, I mean a built-in solution would be documented in IA's doc, referenced in the functions index, it would also be a standard that other people use, which means threads with tips, use cases and examples on the forum, and you would not need to add it to your script library.  
It would just be there for you to call.

Now, I agree that having a custom implementation of something can sometimes be easier, as you can just go and look at the code to figure out how it works, or change it a little bit to suit your needs... but that's out of the scope of my initial comment.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 1, 2024, 9:59pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/45 "2024-02-01T21:59:22Z")

</div>

It's now, after getting an error running the decorator part, that I realise I never knew avout decorator functions... Into the rabbit hole I go!

As an aside, I'm sure there are a million other things I'm simply unaware of... I think I need to do some formal learning on Python

For context, when calling this part from your docstring, @pturmel :

```auto
processABCqueue = queueUtil.WrappedBlockingQueue('processABC', 100)

@queueUtil.drainEvents(processABCqueue, 100, 'Process ABC')
def processABC(entry):
	# Unpack the entry to its components.
	tagPath = entry
	# Do whatever is needed.

```

I get the error:  
TypeError: drainEvents() takes exactly 4 arguments (3 given)

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [February 1, 2024, 10:07pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/46 "2024-02-01T22:07:10Z")

</div>

Ah, shoot. I didn't define that decorator properly. When parameterizing a decorator, another level of indirection is needed.

Give me a minute.

Ok. Updated in place: [queueUtil.py](https://www.automation-pros.com/ignition/queueUtil.py)

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 1, 2024, 11:45pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/47 "2024-02-01T23:45:42Z")

</div>

Still getting an error, should it be this?

```auto
def drainEvents(queue, maxPer, logPrefix):
	"""Decorator that provides queue iteration and simple error handling.

	Args:
		f (callable): The bare function object to decorate.
		queue (WrappedBlockingQueue): The queue to iterate through.
		maxPer (int): Limit to iteration. Typically similar to queue capacity.
		logPrefix (str): Message to include with logged throwables and exceptions.

	`f` must take one argument: the entry taken from the head of the queue.

	The resulting function should be called from a gateway timer event at a pace
	that keeps the queue from filling up and meets the typical latency requirements
	of the application.

	Returns:
		callable: Decorated function. Note that the resulting callable TAKES NO ARGS.
	"""
	def innerDrain(f):
		processed = 0
		entry = queue.poll()
		while entry:
			try:
				f(entry)
			except Throwable, t:
				logger.warn(logPrefix, t)
			except Exception, e:
				logger.warn(logPrefix, later.PythonAsJavaException(e))
			processed += 1
			if processed > maxPer:
				break
			entry = queue.poll()
	return innerDrain

```

I'm still getting my head around decorators... I understand the basics, but the cogs are still turning understanding them in a more complex context!

Edit: oh wait, looks like you might have edited it already to that?

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [February 1, 2024, 11:49pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/48 "2024-02-01T23:49:04Z")

</div>

For reference, it says it in the documentation, and it's easy enough to find, but just to make it a bit quicker, a link to later.py can be found here:

> [@Ignition 8 invokeLater function doesn't seem to exist](https://forum.inductiveautomation.com/t/ignition-8-invokelater-function-doesnt-seem-to-exist/25174/4):
>
> I’ve updated [later.py](https://www.automation-pros.com/ignition/later.py) for Ignition v8. It includes an implementation of invokeLater usable in gateway scope.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 1, 2024, 11:50pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/49 "2024-02-01T23:50:03Z")

</div>

I just substituted the name in the other py url with "later" 😆

e.g.  
[https://www.automation-pros.com/ignition/queueUtil.py](https://www.automation-pros.com/ignition/queueUtil.py)

--\>  
[https://www.automation-pros.com/ignition/later.py](https://www.automation-pros.com/ignition/later.py)

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [February 1, 2024, 11:50pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/50 "2024-02-01T23:50:06Z")

</div>

You're missing a nested function def in the `innerDrain()`.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 1, 2024, 11:54pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/51 "2024-02-01T23:54:01Z")

</div>

![Gears GIFs | Tenor](https://us1.discourse-cdn.com/inductiveautomation/original/3X/c/f/cf3068b55e9efc7ffc28b4c04ba347c484fc5fcf.gif)

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 1, 2024, 11:57pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/52 "2024-02-01T23:57:33Z")

</div>

@pturmel I think this part of the doco is missing the actual `processABC()` call:

```auto
Typical usage in a library script in a leaf project, called from a timer event:

processABCqueue = queueUtil.WrappedBlockingQueue('processABC', 100)

@queueUtil.drainEvents(processABCqueue, 100, 'Process ABC')
def processABC(entry):
	# Unpack the entry to its components.
	tagPath, currentValue, initialChange, missedEvents = entry
	# Do whatever is needed.

```

E.g. add this to the bottom

```auto
processABC()

```

I think? 😬

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [February 2, 2024, 12:58am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/53 "2024-02-02T00:58:59Z")

</div>

No, that goes in the timer event script. Which I describe, but don't show. Like the tag event that I describe but don't show.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [February 2, 2024, 1:01am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/54 "2024-02-02T01:01:30Z")

</div>

> [@lrose](#):
>
> nested function def in the `innerDrain()`.

No, `innerDrain` gets nested in `innerDecorator`.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 2, 2024, 1:18am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/55 "2024-02-02T01:18:52Z")

</div>

Right, I get it now. That stuff is still inside a library, `processABC()` gets called from within the timer event

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [February 2, 2024, 5:18am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/56 "2024-02-02T05:18:42Z")

</div>

Missing import line: `from java.lang import Throwable`

Other than that, I have it working 🙂 very nice! Thanks for the support, as always

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [February 2, 2024, 8:51am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/57 "2024-02-02T08:51:11Z")

</div>

You can search for "decorator factory", that's what they call the pattern used to produce decorators that take parameters.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [February 2, 2024, 3:05pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/58 "2024-02-02T15:05:20Z")

</div>

You're welcome! Thanks for testing on everyone's behalf.

Updated the public link to include this fix, and a more helpful module docstring:

[https://www.automation-pros.com/ignition/queueUtil.py](https://www.automation-pros.com/ignition/queueUtil.py)

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [April 12, 2024, 5:34pm UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/59 "2024-04-12T17:34:24Z")

</div>

There is now a half-example (not using the drain decorator) over here:

> [@Modbus Slave/Server to Generate Read Only Tags](https://forum.inductiveautomation.com/t/modbus-slave-server-to-generate-read-only-tags/87583/21):
>
> I've create a generic "Modbus Publish" script that eliminates the need for a second set of tags. I've publicly posted it here: [https://www.automation-pros.com/ignition/modbusPublish.py](https://www.automation-pros.com/ignition/modbusPublish.py) It requires the "Queue Utility" script posted here: [https://www.automation-pros.com/ignition/queueUtil.py](https://www.automation-pros.com/ignition/queueUtil.py) and described over here: Both scripts rely on my Integration Toolkit's globalVarMap() script function, but you can rework that (not quite 1:1) to use system.util.getGlobals() if you must. (Not recomme…

Edit: The work-around for a long-standing script library "auto-import" bug is being added to my example scripts. Consider including it in yours:

> [@Project Library Script Initialization Races](https://forum.inductiveautomation.com/t/project-library-script-initialization-races/98636):
>
> Are you annoyed by pseudo-random breakage in Ignition project library scripting? Odd top-level constants being garbled, or operations in the top level that have side effects running twice? Or more? Not happening much, but always around project edits, and sometimes even on gateway restart? Yeah, me too. frowning_face Poking around under the hood back in the spring, for a module development task, I ran across some problematic IA bytecode and pinged some people. But it went nowhere, as I co…

---

<div class="post-metadata">

**Author:** ![louagejasper](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/louagejasper/32/86311_2.png) [@louagejasper](https://forum.inductiveautomation.com/u/louagejasper)\
**Post date:** [December 10, 2024, 11:22am UTC](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044/60 "2024-12-10T11:22:43Z")

</div>

I've read through this whole topic, interesting and learned some stuff on coding as well. However, I don't fully understand the use case.

**What I made from it:**

1. You have potentially too many tag event scripts that might take too long
2. You want to queue your script executions
3. So you build a queuing mechanism for scripts  
Is that correct @nminchin? I think I'm missing something here...

If correct, I have some questions:

1. What happens if you would just execute those gateway event scripts directly? I thought this is already done async in Ignition itself?
2. Why would system.util.invokeAsync() not be good for this use case?

[Previous page](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044.md?page=2)

[Next page](https://forum.inductiveautomation.com/t/java-concurrent-queue-for-tag-change-scripts/82044.md?page=4)