Automation Professionals' Integration Toolkit Module

pturmel · July 19, 2023, 4:46pm

Ok. Looks like I have the Vision compatibility issue conquered. I had to use reflection to extract the real function instance from the dynamic dispatch instance. Ugly.

Anyways, for Ignition v8.1+: ~~v2.0.5.232001639~~

Side note: to help debug this, since Vision won't show live expression results in the dialog box, I bound the following to the .text property of a Text Area component, to show what I was getting for various subexpressions:

objectScript(
	"args[0].class.canonicalName + '\\n' + repr(args[0])",
	traceMe('foreach',
		forEach(
			columnsOf({Root Container.Text Area.RawData}),
			it()
			
		)
	)
)

The traces show in the designer or Vision client console as usual.

pturmel · July 20, 2023, 1:33am

I had some dead time today. Party!

Anyways, several new functions:

alias() wraps a dataset with a string prefix for each column name
nanoTime() just calls java's System.nanoTime() and returns the timestamp.
crossJoin() does what you'd think with two datasets.
innerJoin() takes two datasets, plus pairs of iterator expressions to match up.
leftJoin() as for innerJoin(), but includes a null right row if a row on the left has no match on the right.

For Ignition v8.1+: ~~v2.0.6.232010111~~

lrose · July 20, 2023, 2:17pm

Maybe there is an issue still with nested forEach() expressions? As I was attempting to simplify one of my expressions that was previously using objectScript to do some iterations, and I ran into this. I worked up a simple example that shows what I am seeing.

Working Expression:

unionAll(
    asMap('Col1','i','Col2','i','Col3','i'),
    asList(
        asList(0,1,2),
        forEach(
            asList(3,4,5),
            it()
        )
    )
)

Non-Working Expression:

unionAll(
    asMap('Col1','i','Col2','i','Col3','i'),
    asList(
        asList(0,1,2),
        forEach(
            asList(3,4,5),
            forEach(
                asList(1),
                it(1)
                )[0]
        )
    )
)

The two expressions should produce the same dataset, unless my understanding is flawed, which is quite possible. The second one however results in the following error.

Error

10:03:15.144 [AWT-EventQueue-0] ERROR com.inductiveautomation.ignition.client.util.gui.ErrorUtil - null
java.lang.Exception: Error executing expression binding on
Stop Occurances.Root Container.Table.data
	at com.inductiveautomation.factorypmi.application.binding.ExpressionPropertyAdapter.runExpression(ExpressionPropertyAdapter.java:92)
	at com.inductiveautomation.factorypmi.application.binding.ExpressionPropertyAdapter.startup(ExpressionPropertyAdapter.java:113)
	at com.inductiveautomation.factorypmi.application.binding.DefaultInteractionController.setPropertyAdapter(DefaultInteractionController.java:219)
	at com.inductiveautomation.factorypmi.designer.property.configurators.ExpressionConfigurator.bind(ExpressionConfigurator.java:246)
	at com.inductiveautomation.factorypmi.designer.property.configurators.ExpressionConfigurator.tryCommit(ExpressionConfigurator.java:192)
	at com.inductiveautomation.factorypmi.designer.property.configurators.ConfiguratorMultiplexor$EditorParent.tryCommit(ConfiguratorMultiplexor.java:398)
	at com.inductiveautomation.factorypmi.designer.property.configurators.ConfiguratorMultiplexor.tryCommit(ConfiguratorMultiplexor.java:546)
	at com.inductiveautomation.factorypmi.designer.property.configurators.DynamicOptsDialog.doOK(DynamicOptsDialog.java:95)
	at com.inductiveautomation.factorypmi.designer.property.configurators.DynamicOptsDialog$1.actionPerformed(DynamicOptsDialog.java:64)
	at java.desktop/javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.desktop/java.awt.Component.processMouseEvent(Unknown Source)
	at java.desktop/javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.Component.processEvent(Unknown Source)
	at java.desktop/java.awt.Container.processEvent(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.desktop/java.awt.WaitDispatchSupport$2.run(Unknown Source)
	at java.desktop/java.awt.WaitDispatchSupport$4.run(Unknown Source)
	at java.desktop/java.awt.WaitDispatchSupport$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.desktop/java.awt.WaitDispatchSupport.enter(Unknown Source)
	at java.desktop/java.awt.Dialog.show(Unknown Source)
	at java.desktop/java.awt.Component.show(Unknown Source)
	at java.desktop/java.awt.Component.setVisible(Unknown Source)
	at java.desktop/java.awt.Window.setVisible(Unknown Source)
	at java.desktop/java.awt.Dialog.setVisible(Unknown Source)
	at com.inductiveautomation.factorypmi.designer.property.configurators.DynamicOptsDialog.showDialog(DynamicOptsDialog.java:163)
	at com.inductiveautomation.factorypmi.designer.model.VisionDesignerImpl.openBindingDialog(VisionDesignerImpl.java:1169)
	at com.inductiveautomation.factorypmi.designer.property.editors.bb.DynamicOptionsButton.actionPerformed(DynamicOptionsButton.java:37)
	at java.desktop/javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at com.jidesoft.plaf.basic.BasicJideButtonListener.mouseReleased(Unknown Source)
	at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
	at java.desktop/java.awt.Component.processMouseEvent(Unknown Source)
	at java.desktop/javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.Component.processEvent(Unknown Source)
	at java.desktop/java.awt.Container.processEvent(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicTableUI$Handler.repostEvent(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicTableUI$Handler.mouseReleased(Unknown Source)
	at com.jidesoft.swing.DelegateMouseInputListener.mouseReleased(Unknown Source)
	at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
	at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
	at java.desktop/java.awt.Component.processMouseEvent(Unknown Source)
	at java.desktop/javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.Component.processEvent(Unknown Source)
	at java.desktop/java.awt.Container.processEvent(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.lang.ClassCastException: class com.inductiveautomation.ignition.client.expressions.ClientDynamicDispatchFunction cannot be cast to class com.automation_pros.simaids.expressions.SimpleIterator (com.inductiveautomation.ignition.client.expressions.ClientDynamicDispatchFunction and com.automation_pros.simaids.expressions.SimpleIterator are in unnamed module of loader java.net.URLClassLoader @2c7ad468)
	at com.automation_pros.simaids.expressions.IterationHelper.anyIterationFunction(IterationHelper.java:308)
	at com.automation_pros.simaids.expressions.IterationHelper.condSubstExpression(IterationHelper.java:224)
	at com.automation_pros.simaids.expressions.IterationHelper.preExecuteAll(IterationHelper.java:154)
	at com.automation_pros.simaids.expressions.SimpleIterator.execute(SimpleIterator.java:202)
	at com.inductiveautomation.ignition.client.expressions.ClientDynamicDispatchFunction.execute(ClientDynamicDispatchFunction.java:43)
	at com.inductiveautomation.ignition.common.expressions.FunctionExpression.execute(FunctionExpression.java:69)
	at com.inductiveautomation.ignition.common.expressions.functions.BaseFunction.executeAll(BaseFunction.java:64)
	at com.automation_pros.simaids.expressions.AsList.execute(AsList.java:39)
	at com.inductiveautomation.ignition.client.expressions.ClientDynamicDispatchFunction.execute(ClientDynamicDispatchFunction.java:43)
	at com.inductiveautomation.ignition.common.expressions.FunctionExpression.execute(FunctionExpression.java:69)
	at com.inductiveautomation.ignition.common.expressions.functions.BaseFunction.executeAll(BaseFunction.java:64)
	at com.automation_pros.simaids.expressions.UnionAll.execute(UnionAll.java:52)
	at com.inductiveautomation.ignition.client.expressions.ClientDynamicDispatchFunction.execute(ClientDynamicDispatchFunction.java:43)
	at com.inductiveautomation.ignition.common.expressions.FunctionExpression.execute(FunctionExpression.java:69)
	at com.inductiveautomation.factorypmi.application.binding.ExpressionPropertyAdapter.runExpression(ExpressionPropertyAdapter.java:83)
	... 102 common frames omitted

I am trying to replicate the following comprehension which sums the counts by shift on data shaped similar to what I provided above (same columnCount).

[sum([args[0].getValueAt(r,c + 8 * s) for r in xrange(args[0].rowCount) for c in xrange(1,9)]) for s in xrange(3) for _ in (xrange(8))]

As ugly as that may or may not be, it works.

pturmel · July 20, 2023, 5:32pm

Eww. Lingering bug in the Vision compatibility code. Triggered by the depth argument to it(). This should fix it .

For Ignition v8.1+: ~~v2.0.6.232011742~~

Edit: Replaced with slightly faster version.

lrose · July 20, 2023, 6:30pm

Thanks! I don't know if this is any prettier, but it is definitely faster.

Also, not sure if you had intended forEach to be used this way (as basically a traditional for loop), but it works. Would be great to have the equivalent of range(), but this works as well, it would really only save a few keystrokes.

asList(
	flatten(
		asList(
			asList("Shift Totals"),
			flatten(
				forEach(
					asList(0,1,2),
					forEach(
						asList(0,1,2,3,4,5,6,7),
						sum(
							forEach(
								asList(1,2,3,4,5,6,7,8),
								sum(
									forEach(
										{Root Container.Group.Power Table.pivotedData},
										it()[it(1) + 8 * it(3)]
									)
								)
							)
						)
					)
				)
			),
			asList(0,0,0)
		)
	)
)

So what was originally calling two different scripts to do iterations and process totals can now be done completely as an expression.

NOTE: I'm aware that this returns the same sum 8 times, once for each shift. This is going into a power table with cell spanning that corrects that awkwardness.

pturmel · July 20, 2023, 6:34pm

Yes. Since your source material is is similar to python's range() output, consider using the integer mode for your loops. No need to make constant lists of integers.

lrose · July 24, 2023, 3:59pm

Okay, so I got around to attempting to put this into production, but I noticed that things just didn't seem correct.

What I saw was that the expression will evaluate when you click ok on the binding editor dialog. But if you then close the display, then when you reopen the display, the binding does not evaluate.

This is also true in the client. It appears that the expression does not evaluate in the client, and nothing I have tried will successfully trigger it.

I put the loggers on trace, but there are no log entries, and there are no errors generated to indicate the binding failed. It's just like it doesn't evaluate.

New display with only a table component on it.

Expression

unionAll(
    asMap(
        forEach(3,
            "col" + toStr(it()), "i"
       )
    ),
    forEach(4,
        forEach(3,
            getMillis(now(1000))
        )
    )
)

Pretty sure the issue is with the it()

Result after accepting the binding entry:

Result after closing and reopening display in designer, or upon viewing in client.

pturmel · July 24, 2023, 4:35pm

The problem is almost certainly the now(). It requires a context, and doesn't get one when it is nested. Any function that polls or has any asynchronous behavior is not compatible with looping.

(I would have expected an error message, but the expression system might just swallow this case. Consider wrapping that expression with my debugMe function--that will log errors that the expression system might discard.)

pturmel · July 24, 2023, 4:40pm

Also be careful using "i" instead of "I" as a column type (along with other primitives). Expressions never actually hand off any primitive type, so you are at the dataset builder's mercy when it gets an object instead of a primitive.

lrose · July 24, 2023, 4:58pm

This was just a simple example to illustrate the issue. I will set up a case with debugMe() and see if it gives any new information.

Good to know about now() though, I’ll keep that in mind going forward.

Really I was just trying to force the expression to re-evaluate.

I did try system.db.refresh() but that also didn’t force a refresh. More information coming.

pturmel · July 24, 2023, 5:02pm

Try dropping the asMap() around the column definitions. I'll bet that case gets three identically-named columns.

pturmel · July 24, 2023, 5:44pm

Found a null pointer error....

lrose · July 24, 2023, 6:05pm

I tried the following expression on the text property of a text area component.

debugMe("forEach_test_expression",
    forEach(3,
        'col' + toStr(it())
    )
)

No errors reported in the gateway logs, or the wrapper logs.

After accepting initial expression:

After closing and reopening display:

pturmel · July 24, 2023, 8:14pm

Bleh. I hate ClientDynamicDispatchFunction. Apparently, it is not only intended to save window opens when modules are missing, but its implementation can carry another instance of itself before you get to the real function. And, it doesn't instantiate the real function until connect() is called. Which doubly breaks in my constification utility, as I cannot allow that to be called.

Oy! (Why, @PGriffith, WHY?)

Try this, for Ignition v8.1+ as usual: ~~v2.0.7.232051926~~

lrose · July 24, 2023, 8:33pm

Initial testing looks promising. I will try in the morning with a more complex expression and real data and see what it looks back. Thanks for the quick response and turn around!

PGriffith · July 24, 2023, 9:12pm

Take it up with Colby at ICC, I didn't write it

pturmel · July 24, 2023, 9:14pm

I'll add it to my topic list...

lrose · July 25, 2023, 5:41pm

This seems to have corrected the issue I was seeing. Thanks!

pturmel · August 11, 2023, 6:36pm

Null pointer exceptions are always a bug. This latest update prevents a couple of them, and provides more helpful information where available:

For Ignition v8.1+: ~~v2.0.9.232231702~~

pturmel · August 13, 2023, 1:40pm

Another NPE cleanup and a fix in groupBy() which was the root cause:

For Ignition v8.1+: ~~v2.0.10.232251331~~

{ Cleaned out old links and removed buggy files from my server. }