Automation Professionals' Integration Toolkit Module

pturmel · August 13, 2023, 7:03pm

Sigh. That groupBy() fix broke innerJoin() and leftJoin(). (Subclasses.)

This is better, for Ignition v8.1+: v2.0.10.232251855

pturmel · August 14, 2023, 4:02pm

Another new feature to report. After I realized I could coerce my iterator functions into executing a common sub-expression once, as shown here:

I decided it was too lame. And would greatly complicate any other use of it() in the expression. And doesn't really track well with the origin of the idea, which was the use of Perspective transforms to ensure a common expression executed just once.

With this update, the expression:

forEach(
	asList(
		split({path.to.some.string}, '/')
	),
	it()[len(it())-1, 0]
)[0]

can now be written:

transform(
	split({path.to.some.string}, '/'),
	value()[len(value())-1, 0]
)

Which is really, really similar to a Perspective expression binding of:

split({path.to.some.string}, '/')

followed by the expression transform:

{value}[len({value})-1, 0]

So, if you have a value expression and an expression transform in Perspective, you can now use that combination elsewhere. Just replace {value} with value(). Specifically, it brings the equivalent of Perspective transforms to:

Expression tags (both gateway and Vision client tags),
Vision component bindings,
Identity Provider attribute mapping,
Alarm pipeline expression blocks, and
SFC property expressions.

If necessary, you can use objectScript() (or runScript()) to emulate a script transform. If you are following the best practice of have all script transforms be a one-line callout to a project script, the conversion is trivial.

Also, this function can be nested to emulate multiple transforms. If necessary, the depth argument to value() can be used to reach back to a prior stage's result. The nesting of transform() is independent of any use of the iterator functions and does not impact the it() hierarchy depth at all.

For Ignition v8.1+, as usual: ~~v2.0.11.232261523~~

{Unlike the iteration functions, where expressions involving it() cannot also include connectable functions like now() or tag(), transform() doesn't impose any such conditions on expressions involving value().}

pturmel · August 16, 2023, 9:27pm

Annnnnnd... another new feature: the tags() expression function. Note the plural. It takes a list of tag paths, or a dataset with tag paths in the first column.

It defaults to operating in subscription mode, without an initial read, so the first execution will always yield nulls. Quickly replaced with proper values as the subscriptions report in. If the list changes between executions, only the changed tag paths will report a null at that point.

It can be set to not use subscriptions, in which case it is basically performing a readBlocking().

For Ignition v8.1+: ~~v2.0.12.232282039~~

Documentation of the new tags() function is here:

https://www.automation-pros.com/simaids/doc/expression.tags.html

bmusson · August 16, 2023, 9:44pm

Ooh this one will be extremely useful.
What's the reason behind the range restrictions for the timing parameter?

pturmel · August 16, 2023, 9:56pm

A subscription needs a little bit of time to get started, even in gateway scope. That 5ms minimum is to avoid of multiple re-executions hitting while a large list of tags are being subscribed. Vision client scope should use a larger value. (Round trip time to the gateway plus a bit extra--only really meaningful during subscription.)

The upper bound of 1000ms is somewhat arbitrary, but matches the native tag() function's internal timeout. I don't want updates to be ignored accidentally long enough for users to complain. If you don't want asynchronous updates to impact the expression for longer than that, you can use zero and trigger refreshing by other means.

lrose · August 16, 2023, 10:21pm

Will this function suffer from the same pitfalls as the tag() expression or have you mitigated that in some way?

PGriffith · August 16, 2023, 10:32pm

The problem with tag() is that dynamic building of a string path means you can't set up subscriptions and always have to calculate the full internal expression and do a direct read. Phil's function looks to just take strings directly, so no such pitfall.

pturmel · August 16, 2023, 10:33pm

I don't know all the details of tag()'s problems, but I suspect there are two key contributors:

tag() startup tries to supply a proper initial value, with a 1-second timeout, on startup everywhere it is used. Use a hundred tag() calls in the bindings on a Vision template or window and you can block your UI for up to a hundred seconds, looks like. Not sure how Perspective's parallelism fits here, but you'll either tie up a thread for a hundred seconds, or tie up a hundred threads from the Perspective pool, or somewhere in between. My tags() function deliberately does not attempt to do this. (Indirect tag bindings don't do this, either.)
Every tag() expression function is in its own little world, and will trigger expression re-execution where it is used, whenever a new value shows up. If you have twenty tag changes for twenty tag() functions, you will get about twenty re-executions. With my tags() function, if twenty of the tags get new values within the configured timing window, you get one re-execution.

On top of that, I am offering a "one-shot" non-subscription mode that functions like a large list supplied to system.tag.readBlocking(). Without the jython overhead.

pturmel · August 16, 2023, 10:40pm

Meh. Don't think that matters here at all. Some users will be nesting one of my iterators with complex operations to provide the tag path list. (i'm doing that myself in the application that prompted this.)

This new function uses the tag manager's list forms of .subscribeAsync() and unsubscribeAsync() on only the changes from one execution to another. Should be very efficient even with large lists that have some or all elements changing. As an optimization, it also defers unsubscribe for 1000ms in case a tag path is dropped and quickly re-added.

In any case, while lightly tested, I suspect this one is a winner.

pturmel · August 16, 2023, 10:53pm

tag() does subscribe to the assembled tag path, and drops the sub when the string changes before creating the new sub. No optimizations though.

My tags() function doesn't eliminate the cost of computing the tag paths, if you are doing that in the same expression. So that's a wash versus tag().

I would guess that an expression with just tags(), drawing from a tag path list computed separately when a reference changes, would compare favorably to multiple indirect bindings, especially if the timing value was carefully chosen for the data source(s) involved.

pturmel · August 16, 2023, 11:02pm

FWIW, a days work:

$ git diff --stat HEAD~1..HEAD */src
 Client/src/com/automation_pros/simaids/client/ClientHook.java           |   8 ++
 Common/src/com/automation_pros/simaids/expressions/IterationHelper.java |  49 ++++++-
 Common/src/com/automation_pros/simaids/expressions/Tags.java            | 383 ++++++++++++++++++++++++++++++++++++++++++++++++++
 Designer/src/com/automation_pros/simaids/designer/DesignerHook.java     |   8 ++
 Gateway/src/com/automation_pros/simaids/gateway/GatewayHook.java        |   8 ++
 5 files changed, 455 insertions(+), 1 deletion(-)

jlandwerlen · August 16, 2023, 11:10pm

Show off, although, I can delete stuff like nobody else.

ryan.white · August 17, 2023, 1:33am

I'm running into an issue where the forEach expression function is returning nulls in its output list
I'm currently using the latest version of Sim Aids(v2.0.12.232282039), Ignition version 8.1.25. Per the documentation on forEach:

If the expression yields a null, it will be omitted from the output list.

However when using the following binding:

Binding

forEach(
	{session.custom.allMachines},
	if(
		it()['CellID'] = {this.custom.selectedCell},
		asMap(
			"value", it()['MachineID'],
			"label", if(
						!isNull(it()['DisplayName']),
						it()['DisplayName'],
						it()['MachineDesc']
					)
			),
			None
	)
)

with the following data on session.custom.allMachines:

Dataset

"#NAMES"
"MachineID", "CellID", "MachineDesc", "Active", "DisplayName"
"#TYPES"
"str", "str", "str", "B", "str"
"#ROWS", "24"
"Machine_A","Cell_2","Machine_A_Desc","1",""
"Machine_B","Cell_0","Machine_B_Desc","0",""
"Machine_C","Cell_Machining","Machine_C_Desc","1","Machine C"
"Machine_D","Cell_Machining","Machine_D_Desc","1","Machine D"
"Machine_E","Cell_3","Machine_E_Desc","0","Machine E"
"Machine_F","Cell_1","Machine_F_Desc","0","MachineF"
"Machine_G","Cell_Molded","Machine_G_Desc","1","Machine G"
"Machine_H","Cell_Molded","Machine_H_Desc","1","Machine H"
"Machine_I","Cell_0","Machine_I_Desc","1","Machine I"
"Machine_J","Cell_1","Machine_J_Desc","1","Machine J"
"Machine_K","Cell_1","Machine_K_Desc","1","Machine K"
"Machine_L","Cell_1","Machine_L_Desc","1","Machine L"
"Machine_M","Cell_Machining","Machine_M_Desc","1","Machine M"
"Machine_N","Cell_Molded","Machine_N_Desc","1","Machine N"
"Machine_O","Cell_Molded","Machine_O_Desc","1","Machine O"
"Machine_P","Cell_1","Machine_P_Desc","0","Machine P"
"Machine_Q","Cell_2","Machine_Q_Desc","0","Machine Q"
"Machine_R","Cell_Molded","Machine_R_Desc","0","Machine R"
"Machine_S","Cell_Refurb","Machine_S_Desc","0",""
"Machine_T","Cell_0","Machine_T_Desc","1",""
"Machine_U","Cell_2","Machine_U_Desc","1",""
"Machine_V","Cell_2","Machine_V_Desc","1",""
"Machine_W","Cell_0","Machine_W_Desc","1",""
"Machine_X","Cell_0","Machine_X_Desc","1",""

yields the following list of results containing nulls when this.custom.selectedCell = 'Cell_0':

Result with nulls

[
	null,
	{
		"value": "Machine_B",
		"label": "Machine B"
	},
	null,
	null,
	null,
	null,
	null,
	null,
	{
		"value": "Machine_I",
		"label": "Machine I"
	},
	null,
	null,
	null,
	null,
	null,
	null,
	null,
	null,
	null,
	null,
	{
		"value": "Machine_T",
		"label": "Machine_T_Desc"
	},
	null,
	null,
	{
		"value": "Machine_W",
		"label": "Machine_W_Desc"
	},
	{
		"value": "Machine_X",
		"label": "Machine_X_Desc"
	}
]

when I am expecting this:

Expected Result

[
	{
		"value": "Machine_B",
		"label": "Machine B"
	},
	{
		"value": "Machine_I",
		"label": "Machine I"
	},
	{
		"value": "Machine_T",
		"label": "Machine_T_Desc"
	},
	{
		"value": "Machine_W",
		"label": "Machine_W_Desc"
	},
	{
		"value": "Machine_X",
		"label": "Machine_X_Desc"
	}
]

There are always nulls present regardless of what value this.custom.selectedCell is set to. If I set the value to null or "", I'll just get a list of nulls the same length as the base dataset I'm using.

pturmel · August 17, 2023, 12:02pm

Yeah, that was changed. I fixed the docs in one place but not another.

Use the where() function to filter.

(I changed it because it made it impossible to supply nulls programmatically when building arbitrary datasets. Will fix the online doc and include in the next release.)

msteele · August 17, 2023, 1:37pm

@pturmel you must have some interesting use cases leading to all these enhancements

pturmel · August 21, 2023, 7:47pm

Bugfix. transform() did not behave correctly when nested inside an iterator. /:

For Ignition v8.1+ as usual: ~~v2.0.13.232331938~~

pturmel · September 20, 2023, 9:36pm

I've been working on totally revised documentation while I putter away at the next batch of features. It's now good enough to post (though this set is ahead of the publicly available module features).

~~https://www.automation-pros.com/simaids/site/index.html~~

For those who wish their minds blown, the globalVarMap() (and then scroll down...) and Reflection functions will probably do it. v2.0.14 is not far off.

pturmel · September 23, 2023, 9:16pm

Ok. Beta release with a bunch of new features.

For Ignition v8.1+: s>v2.0.14.232662056

New Expression Functions since v2.0.13:

globalVarMap(key)
sessionVarMap()
pageVarMap()
viewVarMap()
mungeColumnName(name)
unQualify(object)

New Scripting Functions since v2.0.13:

system.util.globalVarMap(key)
~~system.util.simAidsCtx()~~ now system.util.toolkitCtx()
system.util.indirectAction(functionName, proxyFor, projectName={current})
system.perspective.sessionVarMap(…)
system.perspective.pageVarMap(…)
system.perspective.viewVarMap(…)
system.reflect.object(javaObject, verbose=False)
system.reflect.coerced(pyObject, class, verbose=False)
system.reflect.pyObject(pyObject, verbose=False)

Breaking change:

selectStar() has a new function signature that breaks prior use, but makes new column construction with dynamic content possible.

Only lightly tested. The new docs are not yet embedded in the module. Use the link in the prior comment for now.

pturmel · October 1, 2023, 9:20pm

Based on popular demand, Automation Professionals is pleased to release the newly renamed Integration Toolkit module, formerly known as Simulation Aids, continuing prior version numbering (and retaining the Module ID).

For Ignition v8.1+: ~~v2.0.14.232741936~~

The documentation has been completely rewritten, and is now properly embedded in the module file for local review. Links to older docs have been removed.

pturmel · October 2, 2023, 9:37pm

Sigh. Another bugfix (regression). I broke nested transforms (in a different way from last time).

For Ignition v8.1+: ~~v2.0.14.232751505~~