IndexException whenever using PyArgParser

kgamble · August 4, 2022, 6:05pm

I am trying to create a scripting function using PyArgParser.getParser and following a few other postings with it, however no matter what I do I keep running into the same exception whenever I pass any args or kwargs into the function.

My Implementation

@Override
@ScriptFunction(docBundlePrefix = "ConfigScripts")
@KeywordArgs(names={"configPath"}, types={String.class})
public PyObject getConfig(PyObject[] pyArgs, String[] keywords) throws ProjectInvalidException, JSONException {
    PyArgParser args = PyArgParser.parseArgs(pyArgs, keywords, new String[] {"configPath"}, new Class[] {String.class}, "getConfig");

    String configPath = args.getString("configPath").orElse(null);

    if (configPath == null) {
        throw new IllegalArgumentException("configPath is a required keyword or positional argument");
    }

    return getConfigImpl(configPath);
}

I have also tried using the PyArgParser.parseArgs(pyArgs, keywords, ConfigScriptModule.class, "getConfig") variant to get the parser, and no matter what I do I run into the same ArrayIndexOutOfBoundsException. Note, line 42 is the line with the PyArgParser.parseArgs call.

Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -2 out of bounds for length 0
	at com.inductiveautomation.ignition.common.script.PyArgParser.<init>(PyArgParser.java:153)
	at com.inductiveautomation.ignition.common.script.PyArgParser.parseArgs(PyArgParser.java:55)
	at com.bwdesigngroup.ignition.configmanager.common.scripting.ConfigScriptModule.getConfig(ConfigScriptModule.java:42)

Full Traceback

Jython 2.7.2 (uncontrolled:000000000000, Jan 25 2022, 14:39:15)
[OpenJDK 64-Bit Server VM (Azul Systems, Inc.)] on java11.0.15

>>> 
Java Traceback:
Traceback (most recent call last):
  File "<input>", line 1, in <module>
	at com.sun.proxy.$Proxy57.getConfig(Unknown Source)
	at com.bwdesigngroup.ignition.configmanager.client.scripting.ClientScriptModule.getConfigImpl(ClientScriptModule.java:37)
	at com.bwdesigngroup.ignition.configmanager.common.scripting.ConfigScriptModule.getConfig(ConfigScriptModule.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
java.lang.reflect.UndeclaredThrowableException: java.lang.reflect.UndeclaredThrowableException

	at org.python.core.Py.JavaError(Py.java:547)
	at org.python.core.Py.JavaError(Py.java:538)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:192)
	at com.inductiveautomation.ignition.common.script.ScriptManager$ReflectedInstanceFunction.__call__(ScriptManager.java:549)
	at org.python.core.PyObject.__call__(PyObject.java:400)
	at org.python.pycode._pyx11.f$0(<input>:1)
	at org.python.pycode._pyx11.call_function(<input>)
	at org.python.core.PyTableCode.call(PyTableCode.java:173)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1687)
	at org.python.core.Py.exec(Py.java:1731)
	at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:277)
	at org.python.util.InteractiveInterpreter.runcode(InteractiveInterpreter.java:130)
	at com.inductiveautomation.ignition.designer.gui.tools.jythonconsole.JythonConsole$ConsoleWorker.doInBackground(JythonConsole.java:611)
	at com.inductiveautomation.ignition.designer.gui.tools.jythonconsole.JythonConsole$ConsoleWorker.doInBackground(JythonConsole.java:599)
	at java.desktop/javax.swing.SwingWorker$1.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.desktop/javax.swing.SwingWorker.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.reflect.UndeclaredThrowableException
	at com.sun.proxy.$Proxy57.getConfig(Unknown Source)
	at com.bwdesigngroup.ignition.configmanager.client.scripting.ClientScriptModule.getConfigImpl(ClientScriptModule.java:37)
	at com.bwdesigngroup.ignition.configmanager.common.scripting.ConfigScriptModule.getConfig(ConfigScriptModule.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
	... 18 more
Caused by: com.inductiveautomation.ignition.client.gateway_interface.GatewayException: Index -2 out of bounds for length 0
	at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.newGatewayException(GatewayInterface.java:351)
	at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:325)
	at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:278)
	at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.moduleInvokeSafe(GatewayInterface.java:917)
	at com.inductiveautomation.ignition.client.gateway_interface.ModuleRPCFactory$DynamicRPCHandler.invoke(ModuleRPCFactory.java:53)
	... 26 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -2 out of bounds for length 0
	at com.inductiveautomation.ignition.common.script.PyArgParser.<init>(PyArgParser.java:153)
	at com.inductiveautomation.ignition.common.script.PyArgParser.parseArgs(PyArgParser.java:55)
	at com.bwdesigngroup.ignition.configmanager.common.scripting.ConfigScriptModule.getConfig(ConfigScriptModule.java:42)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(null)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(null)
	at java.lang.reflect.Method.invoke(null)
	at com.inductiveautomation.ignition.gateway.servlets.gateway.functions.ModuleInvoke.invoke(ModuleInvoke.java:167)
	at com.inductiveautomation.ignition.gateway.servlets.Gateway.doPost(Gateway.java:431)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at com.inductiveautomation.ignition.gateway.bootstrap.MapServlet.service(MapServlet.java:86)
	at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at com.inductiveautomation.catapult.handlers.RemoteHostNameLookupHandler.handle(RemoteHostNameLookupHandler.java:121)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.lang.Thread.run(null)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
	at com.sun.proxy.$Proxy57.getConfig(Unknown Source)
	at com.bwdesigngroup.ignition.configmanager.client.scripting.ClientScriptModule.getConfigImpl(ClientScriptModule.java:37)
	at com.bwdesigngroup.ignition.configmanager.common.scripting.ConfigScriptModule.getConfig(ConfigScriptModule.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
java.lang.reflect.UndeclaredThrowableException: java.lang.reflect.UndeclaredThrowableException
>>>

I appreciate any ideas, I really am just aiming for this to be a one positional arg (or keyword arg) function.

PGriffith · August 4, 2022, 8:05pm

I think things are getting confused over the RPC bridge. Are you sure your signatures line up throughout? Are you trying to directly call a method that accepts PyObject[]/String[] from Java code?

As an aside, you can simplify your first snippet by using orElseThrow() on the Optional<String>:
String configPath = args.getString("configPath").orElseThrow(new IllegalArgumentException("configPath is a required keyword or positional argument"));

bmusson · August 4, 2022, 8:12pm

Ever simpler:

String configPath = args.requireString("configPath")

kgamble · August 4, 2022, 8:16pm

I stepped through all of them to make sure I had the right types, I only take the PyObject[]/String[] from the script interface and the abstract script module.

My actual implemented function takes the String configPath parameter that comes out of the python params, so my client side implementation also only takes that, then when I make the RPC I am creating new objects. Maybe there is something wrong here?

@Override
    protected PyObject getConfigImpl(String configPath) throws ProjectInvalidException, JSONException {
        return rpc.getConfig(new PyObject[]{}, new String[]{"configPath", configPath});
        // return rpc.getConfig(configPath);
    }

This looks much cleaner, however I am getting an issue with the parameter for orElseThrow(), do I need to be casting that argument?

The method orElseThrow(Supplier<? extends X>) in the type Optional<String> is not applicable for the arguments (IllegalArgumentException)Java(67108979)

EDIT: I like the requireString() even better, unless that doesn't work when the value gets passed as a positional, will have to try that out when I get the PyArgParser working...

PGriffith · August 4, 2022, 8:21pm

I don't think you want to be passing PyObject[] or String[]s over RPC. Your RPC methods should just take the (direct) Java arguments they need, so in this case I would expect just a plain String parameter. You may want to use a more specific return type than PyObject for your return value, as well, depending on what it's supposed to act as.

Oh, right. orElseThrow() takes a supplier (a functional interface), so you would construct a lambda:
String configPath = args.getString("configPath").orElseThrow(() -> new IllegalArgumentException("configPath is a required keyword or positional argument"));

But Ben is correct that requireX is cleaner; it will throw a Jython TypeError instead of an IAE, making catching from Jython code a little more idiomatic.

kgamble · August 4, 2022, 8:47pm

Essentially this method is calling SystemUtilities.jsonDecode on a Json string and returning that to the client, is there a more clear PyObject type to use in that case?

I think I understood this correctly, I got it working anyways. So to clarify:

My Client Implementation calls a new method that takes java types

@Override
protected PyObject getConfigImpl(String configPath) throws ProjectInvalidException, JSONException {
    return rpc.getConfigOverRPC(configPath);
}

In my script module I added an @NoHint method that is accessed directly over RPC, that calls the same function as the hinted one, but without ever using python types

@NoHint
public PyObject getConfigOverRPC(String configPath) throws ProjectInvalidException, JSONException {
    return getConfigImpl(configPath);
}

My hinted function remains, and doesn't ever get called in the client scope, it just looks like it does visually to the user

@Override
@ScriptFunction(docBundlePrefix = "ConfigScripts")
@KeywordArgs(names={"configPath"}, types={String.class})
public PyObject getConfig(PyObject[] pyArgs, String[] keywords) throws ProjectInvalidException, JSONException {
    PyArgParser args = PyArgParser.parseArgs(pyArgs, keywords, new String[] {"configPath"}, new Class[] {String.class}, "getConfig");

    String configPath = args.requireString("configPath");

    return getConfigImpl(configPath);
}

Did I get this correct? Or overcomplicate it somewhere?

PGriffith · August 4, 2022, 8:56pm

If you know it's going to be a JsonObject, you could return a PyDictionary or PyStringMap. Or return the JSON string over RPC, and then deserialize it into Python object(s) on the client side.

I think this approach is fine. Are you planning to make this script callable from gateway scope, as well? If so, you may want to (for simplicity) have something like this:

// common

abstract class ScriptModule {
	public PyObject getConfig(PyObject[], String[]) {
		// do arg handling
		String jsonBody = getConfigImpl(arg);
		return jsonDecode(jsonBody);
	}

	abstract String getConfigImpl(String arg) {}
}


// gateway

class GatewayScriptModule {
	GatewayScriptModule(RPC) {
		this.rpc = RPC;
	}

	String getConfigImpl(String arg) {
		return rpc.getConfigImpl(arg);
	}
}


// client

class ClientScriptModule {
	ClientScriptModule() {
		this.rpc = //obtain RPC reference
	}

	String getConfigImpl(String arg) {
		return rpc.getConfigImpl(arg);
	}
}

The idea being that you minimize how often things have to be defined. Your argument parsing can live in one place (in common); your RPC code can live in one place (in gateway), and thus you have to do the minimum amount of work to extend things down the road.

But if you've got it working, I definitely wouldn't sweat it too much.

bmusson · August 4, 2022, 9:24pm

I’d flip this so that the RPC method delegates to the GatewayScriptModule, instead of the other way around.

class ClientScriptModule {
	ClientScriptModule() {
		this.rpc = //obtain RPC reference
	}

	String getConfigImpl(String arg) {
		return rpc.getConfigImpl(arg);
	}
}

class GatewayScriptModule {
	String getConfigImpl(String arg) {
        String value = // Gateway stuff
		return value
	}
}

class ModuleRPC {

    ModuleRPC () {
        this.gatewayScriptModule = new GatewayScriptModule()
    }

    String getConfigImpl(String arg) {
        return this.gatewayScriptModule.getConfigImpl(arg)
    }
}

If you were in Kotlin-land, you could get fancy with Kotlin delegates and do:

class ClientScriptModule : ScriptModule by ModuleRPCFactory.create(MODULE_ID, ModuleRPC ::class.java) 

class ModuleRPC : ScriptModule by GatewayScriptModule(context)

kgamble · August 4, 2022, 9:38pm

So, per another thread on here I am actually providing a project name in the constructor of my GatewayScriptModule, because ultimately this script is return project level resources.

In both of these examples I lose that parameter in my constructor, so is there another way to get it other than pulling it out of python locals when the script managers are initialized?

And in regards to Pauls example, when in the concrete getConfig method, you call getConfigImpl, it will end up calling the implementation of it, which calls back to rpc.getConfigImpl, which ends up in an endless loop of implementation calls. At what point do I actually end up reading my project resource through the implemented function?

bmusson · August 5, 2022, 7:29pm

This kind of use case is why having the ability to treat your RPC handler and your gateway script module separately is important.

To modify my previous example to take the projectName:

class GatewayScriptModule {

	GatewayScriptModule(String projectName) {
		this.projectName = projectName
	}

	String getConfigImpl(String arg) {
        String value = // Gateway stuff using this.projectName
		return value
	}
}

class ModuleRPC {

    ModuleRPC(ClientReqSession session, String projectName) {
		this.session = session
		this.projectName = projectName
        this.gatewayScriptModule = new GatewayScriptModule(projectName)
    }

    String getConfigImpl(String arg) {
        return this.gatewayScriptModule.getConfigImpl(arg)
    }
}