Machine Learning in Ignition using MATLAB

Hello,
We have a number of AI/ML applications we are looking to put into Ignition. I have been working on one method of doing so. The process I came up with is as follows:

  1. Develop the ML algorithm in a dedicated Math/Statistics Software (In our case, MATLAB)
  2. Use the available code generation tools to export the algorithm to JAVA (from MATLAB)
  3. Write a simple JAVA class wrapper for the MATLAB algorithm JAVA generated code
  4. Place the .jar and .jar.pack.gz files in server locations (there was another post on this)
  5. Install the MATLAB run-time library on the ignition server (This is where I am having a problem)
  6. Import the java Class and interact via scripting

As a first use-case, I decided to experiment with googlenet.

My first test was to verify that I could pass a python list of lists from ignition to the java class and perform linear algebra. This was successful. See the first Code set below.

My second test was to verify I could export the googlenet network and run the algorithm in eclipse. This was succcessful. See the second Code set below.

Lastly, I tried to incorporate it in Ignition. I ran into the error below. In Eclipse, I had to point to the MATLAB runtime javabuilder.jar file, which is here (C:\Program Files\MATLAB\MATLAB Runtime\v96\toolbox\javabuilder\jar). I installed the runtime library in the same location on the server. How can I make ignition see this?

MATLAB creates a NNpredict.jar file. This is also referenced by my class. I also put this on the server. I ran into a problem packing this. It is 75 MB and Eclipse gave me a heap memory error. I got around this by using: pack200 -Evalue --effort=0 NNpredict.jar.pack.gz NNpredict.jar
Could this be causing the problem? While I mention it, how can I increase the heap memory for pack200? The internet did not yield any solutions.

In the example below, I am classifying an image of a cat, in case you are wondering about cat.png. Also, I basically gave myself a crash course in Java this week. There is a good chance there is something wrong with my code… I read the first half of Schildt, Java, the complete reference twelfth edition. There is probably some key thing in the second half that would have made this work…

Anyway, thanks in advance. If I can get this to work, we will have ML and AI available to our entire SCADA platform!!!

16:23:57.240 [AWT-EventQueue-0] ERROR com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter - <HTML>Error executing script for event:&nbsp;<code><b>actionPerformed</b></code><BR>on component:&nbsp;<code><b>Button 1</b></code>.
com.inductiveautomation.ignition.common.script.JythonExecException: Traceback (most recent call last):
  File "<event:actionPerformed>", line 1, in <module>
	at java.base/java.lang.Class.forName0(Native Method)

	at java.base/java.lang.Class.forName(Unknown Source)

	at org.python.core.Py.loadAndInitClass(Py.java:909)

	at org.python.core.Py.findClassInternal(Py.java:844)

	at org.python.core.Py.findClassEx(Py.java:895)

	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:133)

	at org.python.core.packagecache.PackageManager.findClass(PackageManager.java:28)

	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:122)

	at org.python.core.PyJavaPackage.__findattr_ex__(PyJavaPackage.java:137)

	at org.python.core.PyObject.__findattr__(PyObject.java:863)

	at org.python.core.packagecache.PackageManager.lookupName(PackageManager.java:136)

	at org.python.core.JavaImporter.find_module(JavaImporter.java:39)

	at org.python.core.JavaImporter.find_module(JavaImporter.java:25)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base/java.lang.reflect.Method.invoke(Unknown Source)


java.lang.NoClassDefFoundError: java.lang.NoClassDefFoundError: com/mathworks/toolbox/javabuilder/MWArray

	at org.python.core.Py.JavaError(Py.java:495)
	at org.python.core.Py.findClassEx(Py.java:901)
	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:133)
	at org.python.core.packagecache.PackageManager.findClass(PackageManager.java:28)
	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:122)
	at org.python.core.PyJavaPackage.__findattr_ex__(PyJavaPackage.java:137)
	at org.python.core.PyObject.__findattr__(PyObject.java:863)
	at org.python.core.packagecache.PackageManager.lookupName(PackageManager.java:136)
	at org.python.core.JavaImporter.find_module(JavaImporter.java:39)
	at org.python.core.JavaImporter.find_module(JavaImporter.java:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:186)
	at org.python.core.PyObject.__call__(PyObject.java:345)
	at org.python.core.PyMethod.instancemethod___call__(PyMethod.java:220)
	at org.python.core.PyMethod.__call__(PyMethod.java:211)
	at org.python.core.PyMethod.__call__(PyMethod.java:201)
	at org.python.core.PyMethod.__call__(PyMethod.java:196)
	at org.python.core.imp.find_module(imp.java:469)
	at org.python.core.imp.import_next(imp.java:718)
	at org.python.core.imp.import_module_level(imp.java:827)
	at org.python.core.imp.importName(imp.java:917)
	at org.python.core.ImportFunction.__call__(__builtin__.java:1220)
	at org.python.core.PyObject.__call__(PyObject.java:357)
	at org.python.core.__builtin__.__import__(__builtin__.java:1173)
	at org.python.core.imp.importOne(imp.java:936)
	at org.python.pycode._pyx3.f$0(<event:actionPerformed>:1)
	at org.python.pycode._pyx3.call_function(<event:actionPerformed>)
	at org.python.core.PyTableCode.call(PyTableCode.java:165)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1275)
	at com.inductiveautomation.ignition.common.script.ScriptManager.runCode(ScriptManager.java:636)
	at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.runActions(ActionAdapter.java:193)
	at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.invoke(ActionAdapter.java:284)
	at com.inductiveautomation.factorypmi.application.binding.action.RelayInvocationHandler.invoke(RelayInvocationHandler.java:57)
	at com.sun.proxy.$Proxy36.actionPerformed(Unknown Source)
	at java.desktop/javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at java.desktop/javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.desktop/java.awt.Component.processMouseEvent(Unknown Source)
	at java.desktop/javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.Component.processEvent(Unknown Source)
	at java.desktop/java.awt.Container.processEvent(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.EventQueue.access$500(Unknown Source)
	at java.desktop/java.awt.EventQueue$3.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$3.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.run(Unknown Source)
Caused by: org.python.core.PyException: Traceback (most recent call last):
  File "<event:actionPerformed>", line 1, in <module>
	at java.base/java.lang.Class.forName0(Native Method)

	at java.base/java.lang.Class.forName(Unknown Source)

	at org.python.core.Py.loadAndInitClass(Py.java:909)

	at org.python.core.Py.findClassInternal(Py.java:844)

	at org.python.core.Py.findClassEx(Py.java:895)

	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:133)

	at org.python.core.packagecache.PackageManager.findClass(PackageManager.java:28)

	at org.python.core.packagecache.SysPackageManager.findClass(SysPackageManager.java:122)

	at org.python.core.PyJavaPackage.__findattr_ex__(PyJavaPackage.java:137)

	at org.python.core.PyObject.__findattr__(PyObject.java:863)

	at org.python.core.packagecache.PackageManager.lookupName(PackageManager.java:136)

	at org.python.core.JavaImporter.find_module(JavaImporter.java:39)

	at org.python.core.JavaImporter.find_module(JavaImporter.java:25)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base/java.lang.reflect.Method.invoke(Unknown Source)


java.lang.NoClassDefFoundError: java.lang.NoClassDefFoundError: com/mathworks/toolbox/javabuilder/MWArray

	... 74 common frames omitted
Caused by: java.lang.NoClassDefFoundError: com/mathworks/toolbox/javabuilder/MWArray
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Unknown Source)
	at org.python.core.Py.loadAndInitClass(Py.java:909)
	at org.python.core.Py.findClassInternal(Py.java:844)
	at org.python.core.Py.findClassEx(Py.java:895)
	... 72 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.mathworks.toolbox.javabuilder.MWArray
	at java.base/java.net.URLClassLoader.findClass(Unknown Source)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	... 77 common frames omitted

The initial linear algebra test code in java is here:

public class MatrixOperations {
	
		private double[][] mat1;
		private double[][] mat2;
		// Constructor to initialize matrix Class
		public MatrixOperations(double[][] mat1, double[][] mat2) {
			//Assume the array has consistent row lengths
			this.mat1=new double[mat1.length][mat1[0].length];
			for (int i=0;i<mat1.length;i++) {
				for (int j=0;j<mat1[0].length;j++) {
					this.mat1[i][j]=mat1[i][j];
				}
			}
			this.mat2=new double[mat2.length][mat2[0].length];
			for (int i=0;i<mat2.length;i++) {
				for (int j=0;j<mat2[0].length;j++) {
					this.mat2[i][j]=mat2[i][j];
				}
			}
			
		}
		public double[][] add(){
			//if they are adding, assume the matrices are the same size
			double[][] computation=new double[this.mat1.length][this.mat1[0].length];
			for (int i=0;i<this.mat1.length;i++) {
				for (int j=0;j<this.mat1[0].length;j++) {
					computation[i][j]=this.mat1[i][j]+mat2[i][j];
				}
			}
			return computation;
		}
		public double[][] multiply() {
			//if they are multiplying, assume the matrix sizes are compatible
			double[][] computation=new double[this.mat1.length][this.mat2[0].length];
			for (int i=0;i<this.mat1.length;i++) {
				for (int j=0;j<this.mat2[0].length;j++) {
					for (int k=0;k<this.mat1[0].length;k++) {
						computation[i][j]+=this.mat1[i][k]*mat2[k][j];
					}
				}
			}
			return computation;
		}

	public static void main(String[] args) {
		byte x= -2;
		System.out.println(x);
		System.out.println("This is a simple program that computes matrix algrebra");
		double[][] xIn = {{1.0, 1.0}, {2.0, 2.0}, {3.0, 3.0}, {4.0, 4.0}, {5.0, 5.0}};
		double[][] yIn = {{1.0, 1.0, 1.0, 1.0, 1.0}, {2.0, 2.0, 2.0, 2.0, 2.0}};	
		int size1 = xIn.length;
		int size2 = xIn[0].length;	
		System.out.println("Here are two example matrices of size "+ size1 + " by " + size2 +" and " + size2 + " by " + size1);
		int i, j;
		for(i=0; i<size1; i++){
			for (j=0;j<size2; j++){
				{System.out.print(xIn[i][j]+" ");}
			}
			System.out.println();
		}
		System.out.println();
		for(i=0; i<size2; i++){
			for (j=0;j<size1; j++){
				{System.out.print(yIn[i][j]+" ");}
			}
			System.out.println();
		}		
		double[][] zMult = new double[size1][size1];
		MatrixOperations MAT = new MatrixOperations(xIn,yIn);
		zMult=MAT.multiply();	
		System.out.println();
		System.out.println("There Product is: ");
		for(i=0; i<size1; i++){
			for (j=0;j<size1; j++){
				{System.out.print(zMult[i][j]+" ");}
			}
			System.out.println();
		}
	}
}

In ignition, I run the following:

import MatrixOperations 
n=5
Matrix1=[]
Matrix2=[]
for i in range(n):
	Matrix1.append([1]*n)
	Matrix2.append([1]*n)
	
Compute=MatrixOperations(Matrix1,Matrix2)
#print Compute.main([''])
ComputeMultiply=Compute.multiply()
ComputeAdd=Compute.add()
print ComputeMultiply[1][1]
print ComputeAdd[1][1]

The googlenet code is here:



import java.util.Arrays;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.awt.image.DataBufferByte;
import com.mathworks.toolbox.javabuilder.*;
import NNpredict.Class1;




public class googlenetwrapper {

	private static Class1 class1Instance;

	private static void setup() throws MWException {
		class1Instance = new Class1();
	}
	
	public static double[][] runNNpredict(int[][][] inData) {
		MWArray In = null;
		MWNumericArray Out = null;
		Object[] results = null;
		double[][] Pvector = null;
		try {
			In = new MWNumericArray(inData, MWClassID.UINT8);
			results = class1Instance.NNpredict(1, In);
			if (results[0] instanceof MWNumericArray) {
				Out = (MWNumericArray) results[0];
			}
			Pvector = (double[][]) Out.toDoubleArray();
			return Pvector;
		} catch (Exception e) {
			e.printStackTrace();
			return Pvector;
		} finally {
			// Dispose of native resources
			MWArray.disposeArray(In);
			MWArray.disposeArray(results);
		}
	}
	
	public static void main(String[] args) {
		// Demonstrate functionality of class.  Read in example image, run googlenet on it, and display top 10 probabilities
		//Read in image and convert to 3D array
		try {
			BufferedImage image = ImageIO.read(new File("D:\\MATLAB\\ExportTesting\\Java\\cat.png"));
			int width=image.getWidth();
			int height=image.getHeight();
			int[][][] pixels = new int[height][width][3];
			byte[] imagepixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
			int n=0;
			for (int i = 0; i < height; i++) {
		         for (int j = 0; j < width; j++) {
		        	 for (int k=0;k<3;k++) {
		        		 pixels[i][j][k]=imagepixels[n];
		        		 n++;
		        	 }
		         }
		      }
			//Initialize MATLABs objects
			try {
				setup();
			} catch (Exception e) {
				e.printStackTrace();
				System.exit(1);
			}
			//Pass image array to network
			try {
				//Run ML
				double[][] Pvector=runNNpredict(pixels);
				//Include Index and sort on Magnitude
				double[][] Svector=new double[Pvector[0].length][2];
				for (int i=0; i<Pvector[0].length;i++) {
					Svector[i][0]=Pvector[0][i];
					Svector[i][1]=i;
				}
				Arrays.sort(Svector,(a, b) -> Double.compare(-a[0], -b[0])); 
				//Display results
				double[][] TopTen=new double[10][2];
				System.out.println("The top ten probabilities and class indexes are:");
				for (int i=0; i<10;i++) {
					TopTen[i][0]=Svector[i][0];
					TopTen[i][1]=Svector[i][1];
					System.out.print(TopTen[i][0]+" "+TopTen[i][1]);
					System.out.println();
				}
			} catch (Exception e) {
				e.printStackTrace();
				System.exit(1);
			} finally {
				// Dispose of native resources
				class1Instance.dispose();
			}
		} catch(IOException e) {}
	}
}

I can speak to some of this.

  • In gateway scope, you don’t need the pack200 files. Those are only for delivery of the code to the designer and/or Vision clients. When you advance to v8+, they aren’t necessary at all.

  • Ignition is not going to look outside its own folders for jar files. You might be able to symlink the necessary files.

  • MatLab’s jar is probably pulling in native code. When the jar is pulled in from an unusual folder location, java may need to use the system path to link any DLL embedded in the jar to any dependencies outside the jar. If there’s no embedded DLL, it is more difficult.

  • It is possible in a 3rd-party module to have your module classloader dynamically load a jar from anywhere in the filesystem. That might suffice to get dependencies loaded (haven’t had that problem). However, module classloaders are isolated from each other and from the system as a whole. Exposing module resources to scripting must be deliberate (initializing script managers from the gateway hook). Doing that with dynamically loaded jars requires dynamic generation of a stub class using java’s ASM. Brutal (I do this for OpenCV in my image streamer module).

  • More…

Hello pturmel,
Thanks for the feedback. This week I have been looking into what you proposed. It is some advanced stuff and I am struggling to figure out where to start. Would you be able to share how you did this with your openCV module? or offer any other guidance?

So, showing the whole thing is not on the table. Sorry. But one utility class I wrote will help you through the trickiest part of it:

FieldClassBuilder.java (Note the MIT License.)

This builder will assemble any desired classes from anywhere in your module, including any dynamically loaded classes you might access, into a generated class with a public static final field for each supplied class.

The output from build() is suitable for use with ScriptManager.addScriptModule() in your various hook classes.

I recommend Google’s ClassPath utilities (which are distributed with Ignition) to enumerate classes in dynamically loaded jars.

(Moving this topic to Module Development…)

2 Likes

I ended up giving up on this approach. I found another solution using industrial protocols, though the example uses python instead of matlab. See that post here.

So, you moved to python and its opencv lib, dropping Matlab. Since a substantial fraction of OpenCV’s API is exposed in its java bindings, you might consider leveraging my image streamer module (which imports everything of OpenCV it can into Ignition’s jython namespace) to run opencv operations in-process.

{ Note that OpenCV’s java API is closer to the C/C++ bindings, not python, and uses the Mat object instead of numpy. }

Can you share a link so I can learn more? I would need to be able to train a NN offline and then upload it to ignition to use with this module. I presume the same functionality applies? I store the neural net files somewhere on the server and point ignition to it?

The module is here:

You would add it to your Ignition server. When it starts, it reports the OpenCV classes it manages to import from the java library. Something like this:

Direct Load org.opencv.aruco.Aruco to system.cv2.aruco
Indirect Load org.opencv.aruco.Board to system.cv2.aruco
Indirect Load org.opencv.aruco.CharucoBoard to system.cv2.aruco
Indirect Load org.opencv.aruco.DetectorParameters to system.cv2.aruco
Indirect Load org.opencv.aruco.Dictionary to system.cv2.aruco
Indirect Load org.opencv.aruco.GridBoard to system.cv2.aruco
Indirect Load org.opencv.bgsegm.BackgroundSubtractorCNT to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.BackgroundSubtractorGMG to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.BackgroundSubtractorGSOC to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.BackgroundSubtractorLSBP to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.BackgroundSubtractorLSBPDesc to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.BackgroundSubtractorMOG to system.cv2.bgsegm
Direct Load org.opencv.bgsegm.Bgsegm to system.cv2.bgsegm
Indirect Load org.opencv.bgsegm.SyntheticSequenceGenerator to system.cv2.bgsegm
Direct Load org.opencv.bioinspired.Bioinspired to system.cv2.bioinspired
Indirect Load org.opencv.bioinspired.Retina to system.cv2.bioinspired
Indirect Load org.opencv.bioinspired.RetinaFastToneMapping to system.cv2.bioinspired
Indirect Load org.opencv.bioinspired.TransientAreasSegmentationModule to system.cv2.bioinspired
Direct Load org.opencv.calib3d.Calib3d to system.cv2.calib3d
Indirect Load org.opencv.calib3d.StereoBM to system.cv2.calib3d
Indirect Load org.opencv.calib3d.StereoMatcher to system.cv2.calib3d
Indirect Load org.opencv.calib3d.StereoSGBM to system.cv2.calib3d
Indirect Load org.opencv.core.Algorithm to system.cv2
Direct Load org.opencv.core.Core to system.cv2
Indirect Load org.opencv.core.CvException to system.cv2
Direct Load org.opencv.core.CvType to system.cv2
Indirect Load org.opencv.core.DMatch to system.cv2
Indirect Load org.opencv.core.KeyPoint to system.cv2
Indirect Load org.opencv.core.Mat to system.cv2
Indirect Load org.opencv.core.MatOfByte to system.cv2
Indirect Load org.opencv.core.MatOfDMatch to system.cv2
Indirect Load org.opencv.core.MatOfDouble to system.cv2
Indirect Load org.opencv.core.MatOfFloat to system.cv2
Indirect Load org.opencv.core.MatOfFloat4 to system.cv2
Indirect Load org.opencv.core.MatOfFloat6 to system.cv2
Indirect Load org.opencv.core.MatOfInt to system.cv2
Indirect Load org.opencv.core.MatOfInt4 to system.cv2
Indirect Load org.opencv.core.MatOfKeyPoint to system.cv2
Indirect Load org.opencv.core.MatOfPoint to system.cv2
Indirect Load org.opencv.core.MatOfPoint2f to system.cv2
Indirect Load org.opencv.core.MatOfPoint3 to system.cv2
Indirect Load org.opencv.core.MatOfPoint3f to system.cv2
Indirect Load org.opencv.core.MatOfRect to system.cv2
Indirect Load org.opencv.core.MatOfRect2d to system.cv2
Indirect Load org.opencv.core.MatOfRotatedRect to system.cv2
Indirect Load org.opencv.core.Point to system.cv2
Indirect Load org.opencv.core.Point3 to system.cv2
Indirect Load org.opencv.core.Range to system.cv2
Indirect Load org.opencv.core.Rect to system.cv2
Indirect Load org.opencv.core.Rect2d to system.cv2
Indirect Load org.opencv.core.RotatedRect to system.cv2
Indirect Load org.opencv.core.Scalar to system.cv2
Indirect Load org.opencv.core.Size to system.cv2
Indirect Load org.opencv.core.TermCriteria to system.cv2
Indirect Load org.opencv.core.TickMeter to system.cv2
Indirect Load org.opencv.dnn.ClassificationModel to system.cv2.dnn
Indirect Load org.opencv.dnn.DetectionModel to system.cv2.dnn
Indirect Load org.opencv.dnn.DictValue to system.cv2.dnn
Direct Load org.opencv.dnn.Dnn to system.cv2.dnn
Indirect Load org.opencv.dnn.KeypointsModel to system.cv2.dnn
Indirect Load org.opencv.dnn.Layer to system.cv2.dnn
Indirect Load org.opencv.dnn.Model to system.cv2.dnn
Indirect Load org.opencv.dnn.Net to system.cv2.dnn
Indirect Load org.opencv.dnn.SegmentationModel to system.cv2.dnn
Indirect Load org.opencv.face.BIF to system.cv2.face
Indirect Load org.opencv.face.BasicFaceRecognizer to system.cv2.face
Indirect Load org.opencv.face.EigenFaceRecognizer to system.cv2.face
Direct Load org.opencv.face.Face to system.cv2.face
Indirect Load org.opencv.face.FaceRecognizer to system.cv2.face
Indirect Load org.opencv.face.Facemark to system.cv2.face
Indirect Load org.opencv.face.FacemarkAAM to system.cv2.face
Indirect Load org.opencv.face.FacemarkKazemi to system.cv2.face
Indirect Load org.opencv.face.FacemarkLBF to system.cv2.face
Indirect Load org.opencv.face.FacemarkTrain to system.cv2.face
Indirect Load org.opencv.face.FisherFaceRecognizer to system.cv2.face
Indirect Load org.opencv.face.LBPHFaceRecognizer to system.cv2.face
Indirect Load org.opencv.face.MACE to system.cv2.face
Indirect Load org.opencv.face.PredictCollector to system.cv2.face
Indirect Load org.opencv.face.StandardCollector to system.cv2.face
Indirect Load org.opencv.features2d.AKAZE to system.cv2.features2d
Indirect Load org.opencv.features2d.AgastFeatureDetector to system.cv2.features2d
Indirect Load org.opencv.features2d.BFMatcher to system.cv2.features2d
Indirect Load org.opencv.features2d.BOWImgDescriptorExtractor to system.cv2.features2d
Indirect Load org.opencv.features2d.BOWKMeansTrainer to system.cv2.features2d
Indirect Load org.opencv.features2d.BOWTrainer to system.cv2.features2d
Indirect Load org.opencv.features2d.BRISK to system.cv2.features2d
Indirect Load org.opencv.features2d.DescriptorMatcher to system.cv2.features2d
Indirect Load org.opencv.features2d.FastFeatureDetector to system.cv2.features2d
Indirect Load org.opencv.features2d.Feature2D to system.cv2.features2d
Direct Load org.opencv.features2d.Features2d to system.cv2.features2d
Indirect Load org.opencv.features2d.FlannBasedMatcher to system.cv2.features2d
Indirect Load org.opencv.features2d.GFTTDetector to system.cv2.features2d
Indirect Load org.opencv.features2d.KAZE to system.cv2.features2d
Indirect Load org.opencv.features2d.MSER to system.cv2.features2d
Indirect Load org.opencv.features2d.ORB to system.cv2.features2d
Indirect Load org.opencv.features2d.Params to system.cv2.features2d
Indirect Load org.opencv.features2d.SimpleBlobDetector to system.cv2.features2d
Direct Load org.opencv.highgui.HighGui to system.cv2
Indirect Load org.opencv.highgui.ImageWindow to system.cv2
Indirect Load org.opencv.img_hash.AverageHash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.BlockMeanHash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.ColorMomentHash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.ImgHashBase to system.cv2.img_hash
Direct Load org.opencv.img_hash.Img_hash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.MarrHildrethHash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.PHash to system.cv2.img_hash
Indirect Load org.opencv.img_hash.RadialVarianceHash to system.cv2.img_hash
Direct Load org.opencv.imgcodecs.Imgcodecs to system.cv2.imgcodecs
Indirect Load org.opencv.imgproc.CLAHE to system.cv2.imgproc
Indirect Load org.opencv.imgproc.GeneralizedHough to system.cv2.imgproc
Indirect Load org.opencv.imgproc.GeneralizedHoughBallard to system.cv2.imgproc
Indirect Load org.opencv.imgproc.GeneralizedHoughGuil to system.cv2.imgproc
Direct Load org.opencv.imgproc.Imgproc to system.cv2.imgproc
Override Load org.opencv.imgproc.LineSegmentDetector to system.cv2.line_descriptor as LSDetector
Indirect Load org.opencv.imgproc.Moments to system.cv2.imgproc
Indirect Load org.opencv.imgproc.Subdiv2D to system.cv2.imgproc
Indirect Load org.opencv.ml.ANN_MLP to system.cv2.ml
Indirect Load org.opencv.ml.Boost to system.cv2.ml
Indirect Load org.opencv.ml.DTrees to system.cv2.ml
Indirect Load org.opencv.ml.EM to system.cv2.ml
Indirect Load org.opencv.ml.KNearest to system.cv2.ml
Indirect Load org.opencv.ml.LogisticRegression to system.cv2.ml
Direct Load org.opencv.ml.Ml to system.cv2.ml
Indirect Load org.opencv.ml.NormalBayesClassifier to system.cv2.ml
Indirect Load org.opencv.ml.ParamGrid to system.cv2.ml
Indirect Load org.opencv.ml.RTrees to system.cv2.ml
Indirect Load org.opencv.ml.SVM to system.cv2.ml
Indirect Load org.opencv.ml.SVMSGD to system.cv2.ml
Indirect Load org.opencv.ml.StatModel to system.cv2.ml
Indirect Load org.opencv.ml.TrainData to system.cv2.ml
Indirect Load org.opencv.objdetect.BaseCascadeClassifier to system.cv2.objdetect
Indirect Load org.opencv.objdetect.CascadeClassifier to system.cv2.objdetect
Indirect Load org.opencv.objdetect.HOGDescriptor to system.cv2.objdetect
Direct Load org.opencv.objdetect.Objdetect to system.cv2.objdetect
Indirect Load org.opencv.objdetect.QRCodeDetector to system.cv2.objdetect
Discarding unknown org.opencv.osgi.OpenCVInterface
Discarding unknown org.opencv.osgi.OpenCVNativeLoader
Indirect Load org.opencv.phase_unwrapping.HistogramPhaseUnwrapping to system.cv2.phase_unwrapping
Indirect Load org.opencv.phase_unwrapping.PhaseUnwrapping to system.cv2.phase_unwrapping
Direct Load org.opencv.phase_unwrapping.Phase_unwrapping to system.cv2.phase_unwrapping
Indirect Load org.opencv.photo.AlignExposures to system.cv2.photo
Indirect Load org.opencv.photo.AlignMTB to system.cv2.photo
Indirect Load org.opencv.photo.CalibrateCRF to system.cv2.photo
Indirect Load org.opencv.photo.CalibrateDebevec to system.cv2.photo
Indirect Load org.opencv.photo.CalibrateRobertson to system.cv2.photo
Indirect Load org.opencv.photo.MergeDebevec to system.cv2.photo
Indirect Load org.opencv.photo.MergeExposures to system.cv2.photo
Indirect Load org.opencv.photo.MergeMertens to system.cv2.photo
Indirect Load org.opencv.photo.MergeRobertson to system.cv2.photo
Direct Load org.opencv.photo.Photo to system.cv2.photo
Indirect Load org.opencv.photo.Tonemap to system.cv2.photo
Indirect Load org.opencv.photo.TonemapDrago to system.cv2.photo
Indirect Load org.opencv.photo.TonemapMantiuk to system.cv2.photo
Indirect Load org.opencv.photo.TonemapReinhard to system.cv2.photo
Direct Load org.opencv.plot.Plot to system.cv2.plot
Indirect Load org.opencv.plot.Plot2d to system.cv2.plot
Indirect Load org.opencv.structured_light.GrayCodePattern to system.cv2.structured_light
Indirect Load org.opencv.structured_light.Params to system.cv2.structured_light
Indirect Load org.opencv.structured_light.SinusoidalPattern to system.cv2.structured_light
Indirect Load org.opencv.structured_light.StructuredLightPattern to system.cv2.structured_light
Direct Load org.opencv.structured_light.Structured_light to system.cv2.structured_light
Indirect Load org.opencv.text.BaseOCR to system.cv2.text
Indirect Load org.opencv.text.Callback to system.cv2.text
Indirect Load org.opencv.text.ClassifierCallback to system.cv2.text
Indirect Load org.opencv.text.ERFilter to system.cv2.text
Indirect Load org.opencv.text.OCRBeamSearchDecoder to system.cv2.text
Indirect Load org.opencv.text.OCRHMMDecoder to system.cv2.text
Indirect Load org.opencv.text.OCRTesseract to system.cv2.text
Direct Load org.opencv.text.Text to system.cv2.text
Indirect Load org.opencv.text.TextDetector to system.cv2.text
Indirect Load org.opencv.text.TextDetectorCNN to system.cv2.text
Indirect Load org.opencv.tracking.MultiTracker to system.cv2.tracking
Indirect Load org.opencv.tracking.Tracker to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerBoosting to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerCSRT to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerGOTURN to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerKCF to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerMIL to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerMOSSE to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerMedianFlow to system.cv2.tracking
Indirect Load org.opencv.tracking.TrackerTLD to system.cv2.tracking
Direct Load org.opencv.tracking.Tracking to system.cv2.tracking
Direct Load org.opencv.utils.Converters to system.cv2
Indirect Load org.opencv.video.BackgroundSubtractor to system.cv2
Indirect Load org.opencv.video.BackgroundSubtractorKNN to system.cv2
Indirect Load org.opencv.video.BackgroundSubtractorMOG2 to system.cv2
Indirect Load org.opencv.video.DISOpticalFlow to system.cv2
Indirect Load org.opencv.video.DenseOpticalFlow to system.cv2
Indirect Load org.opencv.video.FarnebackOpticalFlow to system.cv2
Indirect Load org.opencv.video.KalmanFilter to system.cv2
Indirect Load org.opencv.video.SparseOpticalFlow to system.cv2
Indirect Load org.opencv.video.SparsePyrLKOpticalFlow to system.cv2
Indirect Load org.opencv.video.VariationalRefinement to system.cv2
Direct Load org.opencv.video.Video to system.cv2
VideoCapture is org.opencv.videoio.VideoCapture
Indirect Load org.opencv.videoio.VideoCapture to system.cv2
Indirect Load org.opencv.videoio.VideoWriter to system.cv2
Direct Load org.opencv.videoio.Videoio to system.cv2
Indirect Load org.opencv.ximgproc.AdaptiveManifoldFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.ContourFitting to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.DTFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.DisparityFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.DisparityWLSFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.EdgeAwareInterpolator to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.EdgeBoxes to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.FastBilateralSolverFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.FastGlobalSmootherFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.FastLineDetector to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.GraphSegmentation to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.GuidedFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.RFFeatureGetter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.RICInterpolator to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.RidgeDetectionFilter to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentation to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategy to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategyColor to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategyFill to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategyMultiple to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategySize to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SelectiveSearchSegmentationStrategyTexture to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SparseMatchInterpolator to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.StructuredEdgeDetection to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SuperpixelLSC to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SuperpixelSEEDS to system.cv2.ximgproc
Indirect Load org.opencv.ximgproc.SuperpixelSLIC to system.cv2.ximgproc
Direct Load org.opencv.ximgproc.Ximgproc to system.cv2.ximgproc
Indirect Load org.opencv.xphoto.GrayworldWB to system.cv2.xphoto
Indirect Load org.opencv.xphoto.LearningBasedWB to system.cv2.xphoto
Indirect Load org.opencv.xphoto.SimpleWB to system.cv2.xphoto
Indirect Load org.opencv.xphoto.TonemapDurand to system.cv2.xphoto
Indirect Load org.opencv.xphoto.WhiteBalancer to system.cv2.xphoto
Direct Load org.opencv.xphoto.Xphoto to system.cv2.xphoto
Generated field class for system.cv2
Generated field class for system.cv2.line_descriptor
Generated field class for system.cv2.plot
Generated field class for system.cv2.xphoto
Generated field class for system.cv2.face
Generated field class for system.cv2.features2d
Generated field class for system.cv2.ximgproc
Generated field class for system.cv2.imgproc
Generated field class for system.cv2.calib3d
Generated field class for system.cv2.aruco
Generated field class for system.cv2.bioinspired
Generated field class for system.cv2.bgsegm
Generated field class for system.cv2.tracking
Generated field class for system.cv2.phase_unwrapping
Generated field class for system.cv2.structured_light
Generated field class for system.cv2.objdetect
Generated field class for system.cv2.text
Generated field class for system.cv2.ml
Generated field class for system.cv2.dnn
Generated field class for system.cv2.img_hash
Generated field class for system.cv2.photo

Actual content will vary with the version of OpenCV you are using.

You would need to verify that the classes you are using through the python lib have corresponding elements in this java lib. (Exposed as system.cv2 in Ignition’s jython, but really the java interface.)

Use dir(system.cv2) to see all the field constants, too.