# Automation Professionals' Integration Toolkit Module **URL:** \ **Category:** 3rd Party Modules\ **Tags:** ignition81, vision, scripting, ignition83, perspective\ **Created:** [June 13, 2023, 12:42am UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934 "2023-06-13T00:42:35Z")\ **Posts on this page:** 1\ **Showing post:** 5
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [June 19, 2023, 9:17pm UTC](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934/5 "2023-06-19T21:17:29Z")
I have spent my holiday poking at this. Wanted to provide an efficient replacement for a common use of python transforms: Make a (Perspective) table that dynamically populates both `.data` and `.columns` from source dataset that isn't constrained to any particular structure or data types. In Vision, one can achieve this (mostly) by pre-populating the column attributes dataset with all of the possible column names your application might deliver. No such thing in Perspective. But a transform can take the column names and types from a dataset, perform lookups on one or the other in a map of presets for `.columns`, and construct the corresponding entry. I realized pretty quickly that I forgot to provide a way to iterate over the column information of a dataset, so there's now a `columnsOf()` expression to obtain that, and I realized that iterating over maps (as key-value pairs) is pretty handy, too. So that has been added to the iterators, and the `asList()` and `asMap()` functions have been updated to convert back and forth. And I realized that I had no way to compare the performance of expression functions versus similar scripting solutions. I can't instrument the machinery of script transforms themselves, but I was able to instrument calls to `runScript()` as if calling a script transform. Should have close to the same jython overhead. Anyways, I created a test case view for a "chameleon" table--takes any dataset tag via a tagpath parameter, and displays it with the simplest possible jsonifcation into `.data` and proof-of-concept for map-lookup generation of `.columns`. This is the library script that supports the python transforms: ```auto from java.lang import Throwable from com.inductiveautomation.ignition.common.xmlserialization import ClassNameResolver from com.inductiveautomation.ignition.common.model.values import QualifiedValue logger = system.util.getLogger('transforms') resolver = ClassNameResolver.createBasic() # Convert dataset to list of dictionaries for Perspective table data property. def dsToJsonData(ds): headings = list(ds.columnNames) pyds = system.dataset.toPyDataSet(ds) return [dict(zip(headings, row)) for row in pyds] def columnsOf(ds): return zip(ds.columnNames, map(resolver.getName, ds.columnTypes)) def deQualify(value): if isinstance(value, QualifiedValue): return value.value return value def unMap(value, *args): value = deQualify(value) if isinstance(value, dict): for key in args: try: return deQualify(value[key]) except: pass return value # Convert dataset columns to list of dictionaries for Perspective table columns property. def dsToJsonColumns(ds, handlingMap): headings = list(ds.columnNames) try: return [dict(field=colName, header=dict(title=unMap(deQualify(handlingMap.get(colName, colName)), 'title'))) for colName in headings] except Throwable, t: logger.info("dsToJsonColumns() unexpected error", t) except Exception, e: logger.info("dsToJsonColumns() unexpected exception", shared.later.PythonAsJavaException(e)) ``` The `resolver` is there for later decision-making based on column types, without having to deal with actual class instances. Anyways, the bindings that call into that script module look like this: ```auto timeMe('chameleon-pydata', runScript('transforms.dsToJsonData', 0, {view.custom.dataset})) ``` and: ```auto timeMe('chameleon-pycolumns', runScript('transforms.dsToJsonColumns', 0, {view.custom.dataset}, {this.custom.columnHandling})) ``` The pure expression bindings that perform the same transformation using my new iterators look like this: ```auto timeMe('chameleon-data', forEach( {view.custom.dataset}, asMap(it()) ) ) ``` and: ```auto timeMe('chameleon-columns', forEach( columnsOf({view.custom.dataset}), asMap('field', it()[0], 'header', asMap('title', try(unMap({this.custom.columnHandling}[it()[0]], 'title'), it()[0])) ) ) ) ``` The python columns transform has a slight advantage in that it is not converting the column type class instances to their string names (or abbreviations), which the `columnsOf()` expression function does unconditionally. So the speed advantage is a bit understated in these results from a little while ago: ```auto [20:20:27]: Duration 279070ns timeme=chameleon-data, target=props.data [20:20:27]: Duration 1439812ns timeme=chameleon-pydata, target=custom.pyData [20:20:27]: Duration 372097ns timeme=chameleon-columns, target=props.columns [20:20:27]: Duration 755441ns timeme=chameleon-pycolumns, target=custom.pyColumns [20:20:27]: Duration 242858ns timeme=chameleon-data, target=props.data [20:20:27]: Duration 1213344ns timeme=chameleon-pydata, target=custom.pyData [20:20:27]: Duration 231521ns timeme=chameleon-columns, target=props.columns [20:20:27]: Duration 773343ns timeme=chameleon-pycolumns, target=custom.pyColumns [20:20:47]: Duration 298823ns timeme=chameleon-data, target=props.data [20:20:47]: Duration 1316618ns timeme=chameleon-pydata, target=custom.pyData [20:20:47]: Duration 295162ns timeme=chameleon-columns, target=props.columns [20:20:47]: Duration 707748ns timeme=chameleon-pycolumns, target=custom.pyColumns [20:20:47]: Duration 247223ns timeme=chameleon-data, target=props.data [20:20:47]: Duration 1050020ns timeme=chameleon-pydata, target=custom.pyData [20:20:47]: Duration 265910ns timeme=chameleon-columns, target=props.columns [20:20:47]: Duration 619060ns timeme=chameleon-pycolumns, target=custom.pyColumns ``` The new Simulation Aids expressions are 2.5x to 5x faster than the python transforms, working with a 50Rx3C source dataset. Latest for Ignition v8.1.x: ~~v2.0.2.231701910~~ I think there are a few more tweaks to come. 😁 --- _[View the full topic](https://forum.inductiveautomation.com/t/automation-professionals-integration-toolkit-module/74934)._