I am trying to integrate an Edge deployment with third party app that can only be manipulated with mouse clicks. This is being done to relieve workload on the operator and ensure accuracy. My intent is to use a screen for Ignition and a separate screen for third party apps with an Edge license.
Although being triggered the actual mouse click is not occuring.
The following code is in Project Library/DXF/FilePrep/
def doubleClick(X,Y):
from java.awt import Robot
from java.awt.event import InputEvent
rbt = Robot()
rbt.setAutoDelay(100)
rbt.mouseMove(X, Y)
rbt.mousePress(InputEvent.BUTTON1_MASK) # click
rbt.mouseRelease(InputEvent.BUTTON1_MASK) # drop
rbt.mousePress(InputEvent.BUTTON1_MASK) # click
rbt.mouseRelease(InputEvent.BUTTON1_MASK) # drop
logger = system.util.getLogger("My Log")
logger.info("Double click has fired.")
My event trigger is button with an onClick event and script...
I can see in the logs that the script is running while not appearing to generate any faults and there is other code saving files that are verified to be working. My gut feeling is this is a permissions issue, or something related to Perspective html security but I am fairly new to this so I could be way off.
You are trying to use Java Swing technology (Vision) in a script that will only ever run from the gateway (because it is Perspective).
You will not be able to send any mouse events to any application outside the browser or outside Perspective workstation. That would be malware-ish behavior that browsers go to great lengths to prevent.
This is kind of what I was afraid of, so for the sake of the argument is there a way to bypass that limitation or through another means of scripting?
We are forced to use Perspective and this deployment would be isolated on an automation network so security is not as critical as other applications. As this system will live on an island and will only every be access at the gateway PC I am hoping there is a work around.
No, not within Perspective. That would be malware. If you manage it, and browser makers notice, that hole will get plugged soon.
You could write a real application that communicates with the gateway (via Webdev, but not sure thats available for Edge) to relay requests for such operations.
If you can guarantee where you're running these instances, you could theoretically "escape" the browser in a safe way using a deeplink.
That is, inside Perspective trigger a navigation event to customApplication://someArguments (instead of http://someUrl).
Then, if the local system has a protocol handler registered (this must be done out of band, you cannot do this on the page for the reasons Phil called out already), the web browser will prompt the user asking if they want to use customApplication to open the link:
Then your custom application can be launched as-close-to-seamlessly-as-is-possible, from Perspective.
I'm handwaving entirely over the "custom application" piece of this. You'll need to code something that both registers with the OS as a URL handler (or can be registered manually as a handler) and is capable of doing the automation of clicks in your other application(s). This is a non-trivial engineering task requiring some actual software development.
But if you have to use Perspective, that's (as far as I know) the best you can do.
For anyone that was interested in something similar to this I was able to get the desired behavior by using flask to handle the external scripting not permitted by Perspective.
The Ignition side is triggered by a button press that uses system.perspective.navigate to open a URL with a variable. Depending on the variable inserted I can trigger different scripts, the only downside is I have to reload the Perspective screen.
Pretty sure I can find a more elegant way to do that however.....
Where is this third party app running in, a iframe or is it a different application entirely?
If it's an iframe, you probably could inject some js in your perspective to find and "click" your buttons. Well only if this thrid party app allows cross origin (probably)
