Java concurrent queue for tag change scripts

This is great, Phil, much appreciated!! I'm plugging it in now.

This would be awesome!

Related to the premise of this thread, and maybe it's an insurmountable task, but it'd also be great to have a reference of things not to do in Ignition. With so many ways that you can technically do things, if you don't have a solid background in Java/Python programming (i.e. a software developer), there are so many ways you can shoot yourself in the foot (and step on a bear trap with the other). Often these don't become apparent until you start seeing performance issues on a system that's grown much larger over time. And more often than not, you don't even know where to start, or what part of your code is the culprit. At least if we have a growing list of things to avoid, then we might have better chance of avoiding at least some of the mistakes. I imagine that your support team, as well as everyone here on the forums, would have a list, at least in your heads (although support hopefully have this documented somewhere in a KB), of the common things that integrators do that they shouldn't. Pinging @Paul.Scott for your thoughts for an addition to the user manual perhaps?

1 Like

I don't mind a small performance hit if it's in the benefit of simplicity and clarity. And I'm probably not alone in this boat.

Are you calling Phil a bear ?

5 Likes

Huh. Did you look at what I published? Pretty simple.

I did, and it is !
But you can't beat built-in in terms of simplicity.

And by that, I mean a built-in solution would be documented in IA's doc, referenced in the functions index, it would also be a standard that other people use, which means threads with tips, use cases and examples on the forum, and you would not need to add it to your script library.
It would just be there for you to call.

Now, I agree that having a custom implementation of something can sometimes be easier, as you can just go and look at the code to figure out how it works, or change it a little bit to suit your needs... but that's out of the scope of my initial comment.

7 Likes

It's now, after getting an error running the decorator part, that I realise I never knew avout decorator functions... Into the rabbit hole I go!

As an aside, I'm sure there are a million other things I'm simply unaware of... I think I need to do some formal learning on Python

For context, when calling this part from your docstring, @pturmel :

processABCqueue = queueUtil.WrappedBlockingQueue('processABC', 100)

@queueUtil.drainEvents(processABCqueue, 100, 'Process ABC')
def processABC(entry):
	# Unpack the entry to its components.
	tagPath = entry
	# Do whatever is needed.

I get the error:
TypeError: drainEvents() takes exactly 4 arguments (3 given)

Ah, shoot. I didn't define that decorator properly. When parameterizing a decorator, another level of indirection is needed.

Give me a minute.

Ok. Updated in place: queueUtil.py

3 Likes

Still getting an error, should it be this?

def drainEvents(queue, maxPer, logPrefix):
	"""Decorator that provides queue iteration and simple error handling.

	Args:
		f (callable): The bare function object to decorate.
		queue (WrappedBlockingQueue): The queue to iterate through.
		maxPer (int): Limit to iteration.  Typically similar to queue capacity.
		logPrefix (str): Message to include with logged throwables and exceptions.

	`f` must take one argument: the entry taken from the head of the queue.

	The resulting function should be called from a gateway timer event at a pace
	that keeps the queue from filling up and meets the typical latency requirements
	of the application.

	Returns:
		callable: Decorated function.  Note that the resulting callable TAKES NO ARGS.
	"""
	def innerDrain(f):
		processed = 0
		entry = queue.poll()
		while entry:
			try:
				f(entry)
			except Throwable, t:
				logger.warn(logPrefix, t)
			except Exception, e:
				logger.warn(logPrefix, later.PythonAsJavaException(e))
			processed += 1
			if processed > maxPer:
				break
			entry = queue.poll()
	return innerDrain

I'm still getting my head around decorators... I understand the basics, but the cogs are still turning understanding them in a more complex context!

Edit: oh wait, looks like you might have edited it already to that?

For reference, it says it in the documentation, and it's easy enough to find, but just to make it a bit quicker, a link to later.py can be found here:

I just substituted the name in the other py url with "later" :laughing:

e.g.
https://www.automation-pros.com/ignition/queueUtil.py

-->
https://www.automation-pros.com/ignition/later.py

1 Like

You're missing a nested function def in the innerDrain().

1 Like

Gears GIFs | Tenor

1 Like

@pturmel I think this part of the doco is missing the actual processABC() call:

Typical usage in a library script in a leaf project, called from a timer event:

processABCqueue = queueUtil.WrappedBlockingQueue('processABC', 100)

@queueUtil.drainEvents(processABCqueue, 100, 'Process ABC')
def processABC(entry):
	# Unpack the entry to its components.
	tagPath, currentValue, initialChange, missedEvents = entry
	# Do whatever is needed.

E.g. add this to the bottom

processABC()

I think? :grimacing:

No, that goes in the timer event script. Which I describe, but don't show. Like the tag event that I describe but don't show.

No, innerDrain gets nested in innerDecorator.

Right, I get it now. That stuff is still inside a library, processABC() gets called from within the timer event

Missing import line: from java.lang import Throwable

Other than that, I have it working :slight_smile: very nice! Thanks for the support, as always

1 Like

You can search for "decorator factory", that's what they call the pattern used to produce decorators that take parameters.

2 Likes

You're welcome! Thanks for testing on everyone's behalf.

Updated the public link to include this fix, and a more helpful module docstring:

https://www.automation-pros.com/ignition/queueUtil.py

4 Likes

There is now a half-example (not using the drain decorator) over here:

Edit: The work-around for a long-standing script library "auto-import" bug is being added to my example scripts. Consider including it in yours:

1 Like