Designer - out-of-memory error

Module Development
1

I have a designer project that contains unit-tests (Pyunit) for one of our modules. I can repeatably generate an out-of-memory situation. Takes about 5000 executions of a designer/client scope script executed by our module code - at a 100ms rate. Here's the console error:

And sure enough the gatway console memory display shows that the PermGen space is full. Only about 25% of my heap space is in use. So my question is: What is the PermGen space, is there something I should be doing to mitigate this, can I size it? Is this also a gateway issue ?

Here's my designer startup script:

  java -cp $LAUNCH_DIR/launchclient.jar -Djavaws.sr.gateway.addr.0="localhost:8088:8043/main" -Djavaws.sr.scope="D" -Djavaws.sr.main=com.inductiveautomation.ignition.designer.DesignerStartupHook -Xmx1024M -Xdebug -XX:HeapDumpPath="/Users/me/Datafile" -XX:+HeapDumpOnOutOfMemoryError -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9000 com.inductiveautomation.ignition.client.launch.BootstrapSwing

I've tried attaching the heap dump file a few times, but it's kinda big.
--- chuck

2

Permgen can be thought of as the memory area where class definitions live, among some other things… and if you load too many classes you’ll eventually run out of permgen space.

We already see this issue when the Symbol Factory module is present, which is why the designer jnlp specifies an increased permgen size in its vm-args parameter.

It might be the case that running tests in the designer using Pyunit like you’re doing is pushing things over the limit… I don’t know. That’s not really something we’ve ever done.

Does running your scripts outside of Pyunit cause an permgen error as well?

3

How exactly are these scripts being run? When you compile and run a python script, Jython compiles your python into a java class on the fly. So I’m wondering if you’re doing this in a way that compiles the script each execution?

4

When I did the analysis of the heap, the MAT tool said that java.lang.Class accounted for much of the accumulated storage ---- perhaps you’ve got the explanation. Looks like I’m compiling on the fly - I just don’t know how to do it any other way,

The code that calls the script (and gets executed thousands of times) looks like:

scriptManager = gatewayContext.getProjectManager().getProjectScriptManager(projectId);

. . .
 scriptManager.addGlobalVariable(resultsName, list);  
try {
	scriptManager.runCode(scriptName,scriptManager.createLocalsMap(),null);
}
catch( JythonExecException jee) {
	log.error(TAG+"JythonException executing python "+scriptName+ " ("+jee.getLocalizedMessage()+")",jee);
}
catch(Exception ex) {
	log.error(TAG+"Error executing python "+scriptName+ " ("+ex.getLocalizedMessage()+")",ex);
}

. . .

Suggestions?
– chuck

5

Ok, so a better pattern for this would be to have a field of type PyCode and store the compiled version. Here is some pseudocode minus error handling:

[code]class MyScriptRunner {
String scriptCode;
PyCode compiledScript;

public void runIt() {
if (compiledScript == null){
compiledScript = Py.compile_flags(scriptCode,
“”,
CompileMode.exec,
CompilerFlags.getCompilerFlags());
}
ScriptManager manager = …
PyStringMap locals = manager.createLocalsMap();
manager.runCode(compiledScript, locals);
}
}[/code]

6

Thanks Carl,
Only trouble is that all I am given in the Java module is the name of a python module and method. I have neither a file path nor the code as a string. Is there still a way to do this?

— chuck

7

that “filename” doesn’t have to be an actual name of a file. It’s just something python wants to describe where this code came from. It can by any string.

8

Carl,
This seems to clear up the memory issue. I created a script that simply calls the gateway script thst I want to execute. I compiled the code like:

/**
	 * Compile a simple script that does nothing but call the specified script. 
	 * We assume that the script name includes a module, plus script name, plus method.
	 * We want the module for our import statement.
	 * @param scriptToCall - the script we wish to execute.
	 * @return
	 */
	private PyCode compileScript(String scriptToCall) {
		PyCode code = null;
		String module = "app";
		int index = scriptToCall.lastIndexOf(".");
		if( index>0 ) module = scriptToCall.substring(0,index);
		index = module.lastIndexOf(".");
		if( index>0 ) module = module.substring(0,index);
		String script = String.format("import %s;%s",module,scriptToCall);
		try {
	         code = Py.compile_flags(script,module,CompileMode.exec,CompilerFlags.getCompilerFlags());
	     }
		catch(Exception ex) {
			log.error(TAG+"Failed to compile script");
		}
		return code;
	}