Vision Project Resource Editing Via XML

[Porter_Ellis]

I am attempting to use the XMLDeserializer class to grab existing windows in a project, then add/delete/edit Vision components to the window. This is my personal step toward agentic Ignition development, and seems like the lesser of all evils regarding programmatic component creation and persistence. I can already collect the window object, deserialize it, and inject components into the XML before reserializing the window. However, I do not know if my methods are supported, as they have been fairly unstable thus far.

If someone (@paul-griffith) could shed some light on the XML structure and perhaps provide a simple example of adding a PMILabel object to a window with an expression-bound text value, that would be incredibly helpful.

I am aware that the binding system relies on the InteractionController object on the window itself, but have had trouble registering an acceptable binding (besides tag bindings).

[paul-griffith]

It's unclear to me in which scope you're trying to do your agentic stuff. If you're calling scripts within the designer process externally somehow, that's probably going to be more fruitful than directly authoring/editing XML changes. Though, if you are going down the raw XML road, make sure you're changing the project property to save your Vision resources as XML instead of binary - then you can directly read them via the filesystem, potentially.

Bindings are all instances of the Adapter class:

Adapter

Each binding instance is created by a bespoke XBindingConfigurator class that handles the GUI and the creation logic, and exposes a public static bind method that's ultimately invoked reflectively by other machinery.

XML serialization of adapters within an interaction controller relies on reflection over their beans plus some extra bits, e.g WindowDelegate extends from a DefaultObjectSerializationDelegate.

[bkarabinchak.psi]

These may be relevant to your interests

Create Windows Vision from a script

Problem : We wanted to convert a project from another supervisor to Ignition. We opted for parsing the pages of the old supervisor into an XML file respecting the standards of a Windows Vision. Once this is done we will browse the generated XML files and then, by scripting in a Designer scope, we will inject the pages into the Ignition project. A fairly simple method to do this is to use the CopyPasteHandler.startPaste() class to simulate a Shift + Right Click -> Set XML From Clipboard. T…

GUI creation with AI

Hey, I just wanted to share some experiments I did after reading this post Create Windows Vision from a script - Ignition - Inductive Automation Forum . Based on that idea I tried feeding the xml code of a few example windows into different chatbots and prompting them to create new windows based on that. While it did work better than expected with GPT4 and not only made a correct layout it even was able to assign small scripts to components. I still don't think that it is viable since the xml…

ActionAdapter Method

Hey I came across this amazig post Create Windows Vision from a script - Ignition - Inductive Automation Forum and through that had a look at the underlying xml files for the vision windows and I would like to learn more about it especially the ActionAdapter method (com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter) that seems to be used to assign the java.awt.events to the components. Has anyone played around with this or knows how to use it from inside ignition? My j…

Note the last forum post about actually doing ActionAdapter stuff programatically seems to be uniformly discouraged and a headache.

[pturmel]

The key here is manipulating the XML, versus manipulating window/template programmatically in the designer. The latter offers numerous "under the hood" techniques. Editing the XML directly (outside Ignition), is murderously difficult, and Ignition itself doesn't even try.

You are asking about techniques that even IA doesn't do. (And the count of people within IA that worked with the XMLEncoder is small.)

[Porter_Ellis]

@paul-griffith

calling scripts within the designer process externally somehow

This is also an option, but seemed less flexible than direct XML editing, as it seems IA discourages messing about with the binding system. However, if there is a methodology for managing components this way (via supported libraries/scripting packages), then I would much prefer that. In fact, it would most likely perform better as a tool under the agent that XML manipulation would.

@paul-griffith

It's unclear to me in which scope you're trying to do your agentic stuff.

From the MCP module scope (which seems to be gateway scope) of Ignition 8.3.05. Which reminds me that being gateway-scoped was the original reason I started down the XML path...

[bkarabinchak.psi]

I think the HMI designer espeically with vision is very nice, intuitive and the least bang for your buck for using AI for assisted design. Espeically best case scenario it seems like at most you can have AI put the componets in XML but you will have to do all your bindings manually anyway. I just wouldn't go down this rabbit hole.

I personally only use to debug scripts with syntax errors that I can't see, help me write/debug complicated SQL, and to parse / give me the structure of text that is otherwise just a slog of a task. I think these are big time saves imo.

[Porter_Ellis]

@bkarabinchak.psi

just wouldn't go down this rabbit hole.

I am going down this rabbit hole. It interests me and I believe it is valuable to the future of my development in Ignition. See my response to @paul-griffith and notice my effort to integrate with an IA built MCP module.

@bkarabinchak.psi

I think these are big time saves imo

I also think these are time savers, as I use these methods regularly. But I want to push it FURTHER.

[paul-griffith]

It isn't possible to use any of the scripting methods (or any of the means to create or edit components we use first-party at all) from gateway scope, so you're going to be swimming upstream.

Direct XML editing is your only option within the bounds of the MCP module; doing direct pushes into the project manager is going to be the "most supported" way - if you look around on the forums you can find advice where I've directed people around use of the ChangeOperation class and how to do that.

All that said - what you're really asking for is an artifact that doesn't actually exist. There is no single XML formatting specification I can share with you - if there was, I'd be happy to. The XML format Vision uses is entirely implicit based on the conflicting responsibilities of the XMLSerializer/Deserializer, whatever registered delegates they have, and the specific behavior of the Vision module that's only written out in code. The only way to 'learn' this system is to treat it as a black box.

[channing]

@pturmel

versus manipulating window/template programmatically in the designer.

What are the "under the hood techniques" you speak of? I'm hoping to grab the InteractionController of "win" below and scripturally mutate some bindings. Possible?

Snip from my ResourceEditor class, designer scope:

DesignableProject project = context.getProject();
ResourcePath path_win = new ResourcePath(new ResourceType("com.inductiveautomation.vision", "windows"), "test_3");
Resource win = project.getResource(path_win).orElse(null);

[pturmel]

In the designer, use the system.* functions that return window objects to open the windows of interest. The window will have an .interactionController property corresponding to .getInteractionController().

(You will need to explore that object, and the nested objects, and all of the various kinds of adapters. Down the rabbit hole you go....)

Whatever you manage to change this way (in the designer) will be reserialized into gateway resources when the designer saves the project.

[channing]

The rabbit hole of unlimited power...

I appreciate the help!

[bobby.miller]

This is exactly the use case for frontier LLM models. Working on this as we speak to automate component translation from FTV to Ignition Vision. I'll demo next week.

To answer OP, I'm updating the vision setting to save as XML, then working with the project gateway component files. Too hard to work inside of ignition for this use case, in my view.