Running mypy or other static type-checking on Ignition Jython scripts

Ignition
1

On Python projects unrelated to Ignition, I’ve been starting to use type annotations and static type checking tools like mypy. I find the type annotations very helpful as reminders of what function parameters are supposed to be, and mypy helps find corner cases that would’ve lead to exceptions.

If you’ve never seen type annotations in Python, the mypy cheat sheets are the simplest demonstration and practical explanation I’ve found. Annotations used to be stored as comments in Python2, but Python3 made it part of the language. Reading the official Python PEP on typing is a steep learning curve, but does cover more difficult concepts.

Coincidentally, I was trying to setup @thecesrom.git’s excellent ignition-api library in my editing environment, and noticed it had annotations for everything it stubbed.

For better or worse, I decided to try annotating and type-checking some of my Ignition scripts. Yeah, I knew that the Jython environment, Java libraries, and wonky namespace setup were going to make it all likely to fail, but that’s half the fun.

I setup a Python2.7 virtualenv, loaded it with ignition-api and the modules needed for mypy to run, and then pointed mypy at a code.py file I had annotated. I had to add a lot of imports at the beginning for everything in system.*, all the other script libraries from my Ignition project, and typing to make mypy happy. It still complained a lot about unknown libraries, but mostly worked.

The biggest sticking points to using this on a regular basis is that I can’t leave all those import lines that mypy wants to see in the script file without messing up Ignition, and I don’t have any sane way for mypy to understand script libraries importing each other. I’ve been pondering making some sort of utility that scans a project directory and copy/renames all of the files out into normal Python module hierarchy filenames and pre-pends the hidden import lines to the front of the scripts, but that’s a heck of a lot of work…

I also noticed that both ignition-api and mypy try to bring in slightly different libraries named typing. I can’t remember how I resolved that in my virtualenv, but I got around it somehow…

Has anyone else ever played with this and gotten anywhere? Do you have any tricks for running mypy easily without pre-processing the Ignition script files every time?

This whole experiment has convinced me to start annotating all of my Ignition scripts, even if I can’t run mypy, because I found myself instinctively reading the hints to understand functions. Plus, if I keep annotating things, it’ll be less work if I ever find a way to easily run mypy.

2 Likes
2

No thoughts but this is the biggest thing I miss when I am coding with python 3 and can set up typing versus moving to Ignition. I would love to be able to do what you are saying.

Personally right now I still rely completely on documentation to keep track of it

def doSomething(someParam, someOtherParam):
    """
    Does something
    Args:
        someParam: int, representings something of importance
        someOtherParam: str, something else
    Returns:
        DataSet, data relating to something
    """
    pass

Please add this to the wishlist IA!

3

Parsing type hints would definitely be cool for autocompletion. Running mypy directly is probably not in the cards. I wonder if you could do something with __init__.py to make external checkers happy? I assume mypy complains about the system functions and any Java calls you make?

4

Yes, mypy initially complained that I never imported anything from system or my script libraries.

I already had import java.xxx lines so mypy knew that they existed even though it couldn’t type check them. It’s possible some other Jython user out there has made stubs for all the Java libraries; I haven’t gone looking.

But let’s suppose I had these files:

my_project/ignition/script-python/foo/code.py
my_project/ignition/script-python/bar/code.py

If bar uses things from both system and foo, I would need to add to the top of bar/code.py something like this before running mypy on it to get rid of the worst errors:

import system
import system.util
import system.dataset
import system.tag
import foo

At this point, mypy can find and type-check system functions thanks to thecesrom’s stub library, and it agrees that foo at least exists, but it has no idea where to find foo to type-check things used from that scope.

Being able to type-check between my own script libraries would be one of the things that would make mypy worth the effort to me, but is also looking like one of the harder bits to get working.

5

Hello there, @justin.brzozoski.

Probably this may sound odd, but after some packages like black dropped support for Python 2 I decided to use Python 3 packages for code quality, including mypy.

For mypy in install the [python2] extras, and starting with version 8.1.18 I include types-enum34, as I am using the enum34 package for declaring Enums under system.bacnet.

As seen on my CI file:

As for pre-commit and all of the libraries I use for code quality, I install them using Python 3, via pipx. (See: .pre-commit-config.yaml).

Finally, if you want to include typing in Ignition, you must install that under $IGNITION_DIR/user-lib/pylib as it is not a standard package for Python 2. Which makes it a bit problematic if one were to forget that.

I can get away with adding type hints to all system functions for ignition-api, but I have not done it on any of my Ignition projects. For that, I rely on PyCharm's code analysis which warns about passing the wrong type.

Thanks for installing ignition-api!

6

Furthermore, I can run sort-all, black, isort, and everything that pre-commit executes against code under $IGNITION_DIR/data/projects/<project>.

But I would expect mypy to fail caused by the way Ignition stores Python code by creating a code.py. Which will break any import statement as mypy won’t be able to find the referenced module.

I typically create a pure scripting project in PyCharm or vscode and copy/paste my code manually.

7

I have an update which may be helpful.

After I had to install types-enum34 for mypy, it gave me the idea that I could create actual stub files (*.pyi) for ignition-api. So after a quick search, I learned that mypy comes with a tool called stubgen which will generate the necessary files for running mypy against your project.

I was able to generate the stub files for ignition-api/8.1, and after I fixed all from _typeshed import Incomplete imports and references, I was able to check one of my personal projects (incendium).

Here are the steps to get you going:

  1. Clone ignition-api/8.1
  2. Switch to the 8.1 directory
  3. Run stubgen src
    • This will create a directory called named out; this is where all the stub files are located
  4. Fix all Incomplete imports and references; I replaced that import with from typing import Any
  5. On your Terminal run:
    export MYPYPATH=path/to/ignition-api/8.1/out
  6. Now you’ll be able to run mypy against your own scripting project

Happy coding!

2 Likes
8

Replying to my doppelgänger so it can reply with one very important announcement.

9

One more update.

After I followed my own instructions mypy still reported some errors when checking incendium, so I decided to fix them and after a couple of days I was able to create a stubs package. So I present to the world the ignition-api-stubs package exclusively for checking your Ignition scripting projects with mypy[python2].

Installation

Requirements:

  • Python 3.6+
  • You have added type hints to your scripting project

NOTE: I strongly recommend creating a virtual environment.

Command:

python3 -m pip install ignition-api-stubs

Usage

$ cd <to-your-scripting-project>
$ mypy --py2 <source-directory>

I’ve added type hints to my incendium project, and I was able to run mypy successfully.

$ cd incendium
$ python3 -m pip install ignition-api-stubs
$ mypy --py2 src
Success: no issues found in 14 source files

Conclusion

This package is currently in Alpha, so expect errors. But most importantly, please report them here following our guidelines.

Hopefully some of you may find it useful.

Happy coding!

2 Likes