# Emulating Mouse Clicks With Perspective **URL:** \ **Category:** Ignition\ **Tags:** perspective, scripting\ **Created:** [September 11, 2024, 4:36pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864 "2024-09-11T16:36:05Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Ryan\_Norton](https://avatars.discourse-cdn.com/v4/letter/r/ebca7d/32.png) [@Ryan\_Norton](https://forum.inductiveautomation.com/u/Ryan_Norton)\ **Post date:** [September 11, 2024, 4:36pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/1 "2024-09-11T16:36:05Z")
I am trying to integrate an Edge deployment with third party app that can only be manipulated with mouse clicks. This is being done to relieve workload on the operator and ensure accuracy. My intent is to use a screen for Ignition and a separate screen for third party apps with an Edge license. Although being triggered the actual mouse click is not occuring. The following code is in Project Library/DXF/FilePrep/ ```python def doubleClick(X,Y): from java.awt import Robot from java.awt.event import InputEvent rbt = Robot() rbt.setAutoDelay(100) rbt.mouseMove(X, Y) rbt.mousePress(InputEvent.BUTTON1_MASK) # click rbt.mouseRelease(InputEvent.BUTTON1_MASK) # drop rbt.mousePress(InputEvent.BUTTON1_MASK) # click rbt.mouseRelease(InputEvent.BUTTON1_MASK) # drop logger = system.util.getLogger("My Log") logger.info("Double click has fired.") ``` My event trigger is button with an onClick event and script... ```python def runAction(self, event): DXF.FilePrep.doubleClick(50, 500) ``` I can see in the logs that the script is running while not appearing to generate any faults and there is other code saving files that are verified to be working. My gut feeling is this is a permissions issue, or something related to Perspective html security but I am fairly new to this so I could be way off. Any thoughts? ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 11, 2024, 4:48pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/2 "2024-09-11T16:48:24Z")
> [@Ryan\_Norton](#): > > Perspective You are trying to use Java Swing technology (Vision) in a script that will only ever run from the gateway (because it is Perspective). You will not be able to send any mouse events to any application outside the browser or outside Perspective workstation. That would be malware-ish behavior that browsers go to great lengths to prevent. ---
**Author:** ![Ryan\_Norton](https://avatars.discourse-cdn.com/v4/letter/r/ebca7d/32.png) [@Ryan\_Norton](https://forum.inductiveautomation.com/u/Ryan_Norton)\ **Post date:** [September 11, 2024, 4:51pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/3 "2024-09-11T16:51:50Z")
This is kind of what I was afraid of, so for the sake of the argument is there a way to bypass that limitation or through another means of scripting? We are forced to use Perspective and this deployment would be isolated on an automation network so security is not as critical as other applications. As this system will live on an island and will only every be access at the gateway PC I am hoping there is a work around. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 11, 2024, 4:53pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/4 "2024-09-11T16:53:14Z")
> [@Ryan\_Norton](#): > > bypass that limitation No, not within Perspective. That would be malware. If you manage it, and browser makers notice, that hole will get plugged soon. You could write a real application that communicates with the gateway (via Webdev, but not sure thats available for Edge) to relay requests for such operations. > [@Ryan\_Norton](#): > > We are forced to use Perspective Sounds like someone screwed up. ---
**Author:** ![paul-griffith](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/paul-griffith/32/37249_2.png) [@paul-griffith](https://forum.inductiveautomation.com/u/paul-griffith)\ **Post date:** [September 11, 2024, 5:11pm UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/5 "2024-09-11T17:11:40Z")
_If_ you can guarantee where you're running these instances, you could **theoretically**"escape" the browser in a safe way using a deeplink. That is, inside Perspective trigger a navigation event to `customApplication://someArguments` (instead of `http://someUrl`). Then, **if** the local system has a protocol handler registered (this must be done out of band, you **cannot** do this on the page for the reasons Phil called out already), the web browser will **prompt** the user asking if they want to use `customApplication` to open the link: ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/1/9/19a37a1a97dc4fa1859aaa38179051cf2f81adb0.png) Then your custom application can be launched as-close-to-seamlessly-as-is-possible, from Perspective. I'm handwaving **entirely** over the "custom application" piece of this. You'll need to code something that both registers with the OS as a URL handler (or can be registered manually as a handler) _and_ is capable of doing the automation of clicks in your other application(s). This is a non-trivial engineering task requiring some actual software development. But if you _have_ to use Perspective, that's (as far as I know) the best you can do. ---
**Author:** ![Ryan\_Norton](https://avatars.discourse-cdn.com/v4/letter/r/ebca7d/32.png) [@Ryan\_Norton](https://forum.inductiveautomation.com/u/Ryan_Norton)\ **Post date:** [September 14, 2024, 12:03am UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/6 "2024-09-14T00:03:43Z")
For anyone that was interested in something similar to this I was able to get the desired behavior by using flask to handle the external scripting not permitted by Perspective. The Ignition side is triggered by a button press that uses system.perspective.navigate to open a URL with a variable. Depending on the variable inserted I can trigger different scripts, the only downside is I have to reload the Perspective screen. Pretty sure I can find a more elegant way to do that however..... ---
**Author:** ![victordcq](https://avatars.discourse-cdn.com/v4/letter/v/f6c823/32.png) [@victordcq](https://forum.inductiveautomation.com/u/victordcq)\ **Post date:** [September 16, 2024, 6:44am UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/7 "2024-09-16T06:44:12Z")
I've done something like this a few times, example here > [@WebDev to open File Explorer](https://forum.inductiveautomation.com/t/webdev-to-open-file-explorer/47969/5): > > ((hm i might have replied on the wrong topic xd)) You can install an uri (on windows) that works for browsers (not the app) You gotta unzip and run it (as admin) on every client pc tho [registerExplorerURIScheme.zip](https://forum.inductiveautomation.com/uploads/short-url/pPlxsSccetxa62JGDTDVaHosx42.zip) (286 Bytes) this one goes like so, i use the link component to open in a new tab: explorer:C:\yourfile.txt ---
**Author:** ![victordcq](https://avatars.discourse-cdn.com/v4/letter/v/f6c823/32.png) [@victordcq](https://forum.inductiveautomation.com/u/victordcq)\ **Post date:** [September 16, 2024, 6:46am UTC](https://forum.inductiveautomation.com/t/emulating-mouse-clicks-with-perspective/95864/8 "2024-09-16T06:46:03Z")
Where is this third party app running in, a iframe or is it a different application entirely? If it's an iframe, you probably could inject some js in your perspective to find and "click" your buttons. Well only if this thrid party app allows cross origin (probably)