# Automation Professionals' Integration Toolkit Module **URL:** \ **Category:** 3rd Party Modules\ **Tags:** ignition83, scripting, ignition81, perspective, vision\ **Created:** [June 13, 2023, 12:42am UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934 "2023-06-13T00:42:35Z")\ **Posts on this page:** 20\ **Page:** 4
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 13, 2023, 7:03pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/61 "2023-08-13T19:03:56Z")
Sigh. That `groupBy()` fix broke `innerJoin()` and `leftJoin()`. (Subclasses.) ☹ This is better, for Ignition v8.1+: [v2.0.10.232251855](https://www.automation-pros.com/ignition/Simulation%20Aids-2.0.10.232251855-v81.modl) ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 14, 2023, 4:02pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/62 "2023-08-14T16:02:21Z")
Another new feature to report. 😄 After I realized I could coerce my iterator functions into executing a common sub-expression once, as shown here: > [@Expression: get 3rd to last element of split(..) result](https://forum.inductiveautomation.com/t/expression-get-3rd-to-last-element-of-split-result/15159/12): > > I've never been real happy with how expressions deal with common sub-expressions. That is, not happy at all. The impetus to find a solution isn't there in Perspective, because you can put such expressions ahead of a transform so they will only be executed once. (And if you have more than one, my new asList() expression can bundle multiple such into a single value.) But expression tags and Vision UI bindings do not have transforms. frowning_face I realized while reading this topic that I'… I decided it was too lame. And would greatly complicate any other use of `it()` in the expression. And doesn't really track well with the origin of the idea, which was the use of Perspective transforms to ensure a common expression executed just once. With this update, the expression: ```auto forEach( asList( split({path.to.some.string}, '/') ), it()[len(it())-1, 0] )[0] ``` can now be written: ```auto transform( split({path.to.some.string}, '/'), value()[len(value())-1, 0] ) ``` Which is really, really similar to a Perspective expression binding of: ```auto split({path.to.some.string}, '/') ``` followed by the expression transform: ```auto {value}[len({value})-1, 0] ``` So, if you have a value expression and an expression transform in Perspective, you can now use that combination elsewhere. Just replace `{value}` with `value()`. Specifically, it brings the equivalent of Perspective transforms to: - Expression tags (both gateway and Vision client tags), - Vision component bindings, - Identity Provider attribute mapping, - Alarm pipeline expression blocks, and - SFC property expressions. If necessary, you can use `objectScript()` (or `runScript()`) to emulate a script transform. If you are following the best practice of have all script transforms be a one-line callout to a project script, the conversion is trivial. Also, this function can be nested to emulate multiple transforms. If necessary, the `depth` argument to `value()` can be used to reach back to a prior stage's result. The nesting of `transform()` is independent of any use of the iterator functions and does not impact the `it()` hierarchy depth at all. For Ignition v8.1+, as usual: ~~v2.0.11.232261523~~ {Unlike the iteration functions, where expressions involving `it()` cannot also include connectable functions like `now()` or `tag()`, `transform()` doesn't impose any such conditions on expressions involving `value()`.} ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 9:27pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/63 "2023-08-16T21:27:35Z")
_Annnnnnd..._ another new feature: the `tags()` expression function. Note the plural. It takes a list of tag paths, or a dataset with tag paths in the first column. It defaults to operating in subscription mode, without an initial read, so the first execution will always yield nulls. Quickly replaced with proper values as the subscriptions report in. If the list changes between executions, only the changed tag paths will report a null at that point. It can be set to **not** use subscriptions, in which case it is basically performing a `readBlocking()`. For Ignition v8.1+: ~~v2.0.12.232282039~~ Documentation of the new `tags()` function is here: [https://www.automation-pros.com/toolkit/doc/utility.html#tags](https://www.automation-pros.com/toolkit/doc/utility.html#tags) ---
**Author:** ![bmusson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/bmusson/32/87530_2.png) [@bmusson](https://forum.inductiveautomation.com/u/bmusson)\ **Post date:** [August 16, 2023, 9:44pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/64 "2023-08-16T21:44:59Z")
Ooh this one will be extremely useful. What's the reason behind the range restrictions for the `timing` parameter? ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 9:56pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/65 "2023-08-16T21:56:12Z")
A subscription needs a little bit of time to get started, even in gateway scope. That 5ms minimum is to avoid of multiple re-executions hitting while a large list of tags are being subscribed. Vision client scope should use a larger value. (Round trip time to the gateway plus a bit extra--only really meaningful during subscription.) The upper bound of 1000ms is somewhat arbitrary, but matches the native `tag()` function's internal timeout. I don't want updates to be ignored accidentally long enough for users to complain. If you don't want asynchronous updates to impact the expression for longer than that, you can use zero and trigger refreshing by other means. ---
**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\ **Post date:** [August 16, 2023, 10:21pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/66 "2023-08-16T22:21:36Z")
Will this function suffer from the same pitfalls as the `tag()` expression or have you mitigated that in some way? ---
**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\ **Post date:** [August 16, 2023, 10:32pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/67 "2023-08-16T22:32:28Z")
The problem with tag() is that dynamic building of a string path means you _can't_ set up subscriptions and always have to calculate the full internal expression and do a direct read. Phil's function looks to just take strings directly, so no such pitfall. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 10:33pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/68 "2023-08-16T22:33:35Z")
I don't know all the details of `tag()`'s problems, but I suspect there are two key contributors: - `tag()` startup tries to supply a proper initial value, with a 1-second timeout, on startup everywhere it is used. Use a hundred `tag()` calls in the bindings on a Vision template or window and you can block your UI for up to a hundred seconds, looks like. Not sure how Perspective's parallelism fits here, but you'll either tie up a thread for a hundred seconds, or tie up a hundred threads from the Perspective pool, or somewhere in between. My `tags()` function deliberately does not attempt to do this. (Indirect tag bindings don't do this, either.) - Every `tag()` expression function is in its own little world, and will trigger expression re-execution where it is used, whenever a new value shows up. If you have twenty tag changes for twenty `tag()` functions, you will get about twenty re-executions. With my `tags()` function, if twenty of the tags get new values within the configured timing window, you get one re-execution. On top of that, I am offering a "one-shot" non-subscription mode that functions like a large list supplied to `system.tag.readBlocking()`. Without the jython overhead. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 10:40pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/69 "2023-08-16T22:40:21Z")
> [@paul-griffith](#): > > dynamic building of a string path Meh. Don't think that matters here at all. Some users will be nesting one of my iterators with complex operations to provide the tag path list. (i'm doing that myself in the application that prompted this.) This new function uses the tag manager's list forms of `.subscribeAsync()` and `unsubscribeAsync()` on only the changes from one execution to another. Should be very efficient even with large lists that have some or all elements changing. As an optimization, it also defers unsubscribe for 1000ms in case a tag path is dropped and quickly re-added. In any case, while lightly tested, I suspect this one is a winner. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 10:53pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/70 "2023-08-16T22:53:31Z")
> [@paul-griffith](#): > > you _can't_ set up subscriptions and always have to calculate the full internal expression and do a direct read `tag()` does subscribe to the assembled tag path, and drops the sub when the string changes before creating the new sub. No optimizations though. My `tags()` function doesn't eliminate the cost of computing the tag paths, if you are doing that in the same expression. So that's a wash versus `tag()`. I would guess that an expression with just `tags()`, drawing from a tag path list computed separately when a reference changes, would compare favorably to multiple indirect bindings, especially if the `timing` value was carefully chosen for the data source(s) involved. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 16, 2023, 11:02pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/71 "2023-08-16T23:02:38Z")
FWIW, a days work: ```auto $ git diff --stat HEAD~1..HEAD */src Client/src/com/automation_pros/simaids/client/ClientHook.java | 8 ++ Common/src/com/automation_pros/simaids/expressions/IterationHelper.java | 49 ++++++- Common/src/com/automation_pros/simaids/expressions/Tags.java | 383 ++++++++++++++++++++++++++++++++++++++++++++++++++ Designer/src/com/automation_pros/simaids/designer/DesignerHook.java | 8 ++ Gateway/src/com/automation_pros/simaids/gateway/GatewayHook.java | 8 ++ 5 files changed, 455 insertions(+), 1 deletion(-) ``` 😁 ---
**Author:** ![jlandwerlen](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@jlandwerlen](https://forum.inductiveautomation.com/u/jlandwerlen)\ **Post date:** [August 16, 2023, 11:10pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/72 "2023-08-16T23:10:37Z")
Show off, although, I can delete stuff like nobody else. 🙂 ---
**Author:** ![ryan.white](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/ryan.white/32/73193_2.png) [@ryan.white](https://forum.inductiveautomation.com/u/ryan.white)\ **Post date:** [August 17, 2023, 1:33am UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/73 "2023-08-17T01:33:28Z")
I'm running into an issue where the `forEach` expression function is returning nulls in its output list I'm currently using the latest version of Sim Aids(v2.0.12.232282039), Ignition version 8.1.25. Per the documentation on `forEach`: > If the expression yields a null, it will be omitted from the output list. However when using the following binding: > **Binding** > > ```auto > forEach( > {session.custom.allMachines}, > if( > it()['CellID'] = {this.custom.selectedCell}, > asMap( > "value", it()['MachineID'], > "label", if( > !isNull(it()['DisplayName']), > it()['DisplayName'], > it()['MachineDesc'] > ) > ), > None > ) > ) > > ``` with the following data on `session.custom.allMachines`: > **Dataset** > > ```auto > "#NAMES" > "MachineID", "CellID", "MachineDesc", "Active", "DisplayName" > "#TYPES" > "str", "str", "str", "B", "str" > "#ROWS", "24" > "Machine_A","Cell_2","Machine_A_Desc","1","" > "Machine_B","Cell_0","Machine_B_Desc","0","" > "Machine_C","Cell_Machining","Machine_C_Desc","1","Machine C" > "Machine_D","Cell_Machining","Machine_D_Desc","1","Machine D" > "Machine_E","Cell_3","Machine_E_Desc","0","Machine E" > "Machine_F","Cell_1","Machine_F_Desc","0","MachineF" > "Machine_G","Cell_Molded","Machine_G_Desc","1","Machine G" > "Machine_H","Cell_Molded","Machine_H_Desc","1","Machine H" > "Machine_I","Cell_0","Machine_I_Desc","1","Machine I" > "Machine_J","Cell_1","Machine_J_Desc","1","Machine J" > "Machine_K","Cell_1","Machine_K_Desc","1","Machine K" > "Machine_L","Cell_1","Machine_L_Desc","1","Machine L" > "Machine_M","Cell_Machining","Machine_M_Desc","1","Machine M" > "Machine_N","Cell_Molded","Machine_N_Desc","1","Machine N" > "Machine_O","Cell_Molded","Machine_O_Desc","1","Machine O" > "Machine_P","Cell_1","Machine_P_Desc","0","Machine P" > "Machine_Q","Cell_2","Machine_Q_Desc","0","Machine Q" > "Machine_R","Cell_Molded","Machine_R_Desc","0","Machine R" > "Machine_S","Cell_Refurb","Machine_S_Desc","0","" > "Machine_T","Cell_0","Machine_T_Desc","1","" > "Machine_U","Cell_2","Machine_U_Desc","1","" > "Machine_V","Cell_2","Machine_V_Desc","1","" > "Machine_W","Cell_0","Machine_W_Desc","1","" > "Machine_X","Cell_0","Machine_X_Desc","1","" > > ``` yields the following list of results containing nulls when `this.custom.selectedCell = 'Cell_0'`: > **Result with nulls** > > ```auto > [ > null, > { > "value": "Machine_B", > "label": "Machine B" > }, > null, > null, > null, > null, > null, > null, > { > "value": "Machine_I", > "label": "Machine I" > }, > null, > null, > null, > null, > null, > null, > null, > null, > null, > null, > { > "value": "Machine_T", > "label": "Machine_T_Desc" > }, > null, > null, > { > "value": "Machine_W", > "label": "Machine_W_Desc" > }, > { > "value": "Machine_X", > "label": "Machine_X_Desc" > } > ] > > ``` when I am expecting this: > **Expected Result** > > ```auto > [ > { > "value": "Machine_B", > "label": "Machine B" > }, > { > "value": "Machine_I", > "label": "Machine I" > }, > { > "value": "Machine_T", > "label": "Machine_T_Desc" > }, > { > "value": "Machine_W", > "label": "Machine_W_Desc" > }, > { > "value": "Machine_X", > "label": "Machine_X_Desc" > } > ] > > ``` There are always nulls present regardless of what value `this.custom.selectedCell` is set to. If I set the value to `null` or `""`, I'll just get a list of nulls the same length as the base dataset I'm using. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 17, 2023, 12:02pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/74 "2023-08-17T12:02:02Z")
> [@ryan.white](#): > > If the expression yields a null, it will be omitted from the output list. Yeah, that was changed. I fixed the docs in one place but not another. Use the `where()` function to filter. (I changed it because it made it impossible to supply nulls programmatically when building arbitrary datasets. Will fix the online doc and include in the next release.) ---
**Author:** ![msteele](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/msteele/32/77889_2.png) [@msteele](https://forum.inductiveautomation.com/u/msteele)\ **Post date:** [August 17, 2023, 1:37pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/75 "2023-08-17T13:37:25Z")
@pturmel you must have some interesting use cases leading to all these enhancements ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [August 21, 2023, 7:47pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/76 "2023-08-21T19:47:42Z")
Bugfix. `transform()` did not behave correctly when nested inside an iterator. /: For Ignition v8.1+ as usual: ~~v2.0.13.232331938~~ ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 20, 2023, 9:36pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/77 "2023-09-20T21:36:32Z")
I've been working on totally revised documentation while I putter away at the next batch of features. It's now good enough to post (though this set is ahead of the publicly available module features). ~~[https://www.automation-pros.com/simaids/site/index.html](https://www.automation-pros.com/simaids/site/index.html)~~ For those who wish their minds blown, the [globalVarMap()](https://www.automation-pros.com/simaids/site/scripting.html#globalvarmap) (and then scroll down...) and [Reflection](https://www.automation-pros.com/simaids/site/reflection.html) functions will probably do it. v2.0.14 is not far off. 😁 ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 23, 2023, 9:16pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/78 "2023-09-23T21:16:52Z")
Ok. **Beta** release with a bunch of new features. For Ignition v8.1+: s\>v2.0.14.232662056 New Expression Functions since v2.0.13: - globalVarMap(key) - sessionVarMap() - pageVarMap() - viewVarMap() - mungeColumnName(name) - unQualify(object) New Scripting Functions since v2.0.13: - system.util.globalVarMap(key) - ~~system.util.simAidsCtx()~~ _now_ system.util.toolkitCtx() - system.util.indirectAction(functionName, proxyFor, projectName={current}) - system.perspective.sessionVarMap(…) - system.perspective.pageVarMap(…) - system.perspective.viewVarMap(…) - system.reflect.object(javaObject, verbose=False) - system.reflect.coerced(pyObject, class, verbose=False) - system.reflect.pyObject(pyObject, verbose=False) Breaking change: `selectStar()` has a new function signature that breaks prior use, but makes new column construction with dynamic content possible. Only lightly tested. The new docs are not yet embedded in the module. Use the link in the prior comment for now. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [October 1, 2023, 9:20pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/79 "2023-10-01T21:20:07Z")
Based on popular demand, Automation Professionals is pleased to release the newly renamed **Integration Toolkit** module, formerly known as Simulation Aids, continuing prior version numbering (and retaining the Module ID). For Ignition v8.1+: ~~v2.0.14.232741936~~ The [documentation has been completely rewritten](https://www.automation-pros.com/toolkit/doc/), and is now properly embedded in the module file for local review. Links to older docs have been removed. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [October 2, 2023, 9:37pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/80 "2023-10-02T21:37:53Z")
Sigh. Another bugfix (regression). I broke nested transforms (in a different way from last time). For Ignition v8.1+: ~~v2.0.14.232751505~~ [Previous page](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934.md?page=3) [Next page](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934.md?page=5)