[IGN-6503]Project Library Script Initialization Races

[pturmel]

@mazeyrat

functions in the try/finally blocks are not detected in designer

That is why I did not recommend using an outer try, but to do error checking in the functions themselves.

This v8.1 workaround is not perfect, cannot be perfect, and will never be perfect. If you need this to be perfect, upgrade to v8.3.

[cmaynes]

I just installed v8.3.1 and turned off the workaround I had in place (i.e. a decorator on every custom class definition using a thread lock and cache stored in project globals to only allow one class definition to exist for each class).

Unfortunately, at least in the designer, I am seeing the same problems I was before. I have a class defined in one module which I import into multiple others, but for some reason the module with the class definition gets executed repeatedly and some callers end up with different class definitions than others. Maybe this is a separate issue from what @pturmel was talking about, but it remains a significant challenge to development in Ignition.

I wonder now that the race condition problem should be fixed with the import mechanism, if this is happening because imports are not exactly the same everywhere and whatever is being used as a primary key is different for the same code module depending on how the module is being imported/accessed.

• One caller may import the class directly from my module (e.g. “from shared.custom import MyClass”) while another caller may just import the module and access the class as an attribute (e.g. “import shared.custom; shared.custom.MyClass”).
• In an attempt to create a flat interface for callers, I sometimes define a private package and a public module which imports from the private package. External callers import from the public module. This is meant to be similar to how __init__.py can work in python, where it imports from the modules inside the package, avoiding the need of the caller to import from each individual module. There are then imports inside shared._custom between modules as well as imports by external callers made using the custom public module.

    
    shared/
      _custom/
        module_a
        module_b
  
    custom: import from shared.\_custom.module_a/b/c
  

[Clancy_Cavanaugh]

Importing from project library scripts is known to cause problems. Instead of importing you should be using the full path/name of the resource you would normally import.

# Don't use import
from shared.x import y
# instead call it as
shared.x.y()

Import from global and project script library failing randomly

Try not importing it at all. Just call shared.util.MWESLogging.getLog(). Due to object lifetimes in the gateway, it is unwise to ever import anything from shared.* or project.*. Just use the full names wherever you need them. It shouldn't be a problem in the client either, but the designer also has weirdness. Import all you want from java. That is a pure jython operation and is effectively bulletproof. Other than the fact you can't use wildcards. See this topic:

[cmaynes]

Thanks, @Clancy_Cavanaugh , that might work, but at this point I think I have a few hundred thousand lines of code. Updating every import statement in every script module/event script/tag script/etc. is not feasible for me at this point. If I were able to start fresh, that is definitely what I would try.

[Clancy_Cavanaugh]

Probably still worth it to fix. Even if you put your decorator back in, you can run into issues with some running scripts using the old versions until they are reinitialized.

I believe I remember there being a memory leak possibility as well, but that may have only been if you are trying to persist something with system.util.getGlobals().

[bschroeder]

I would get cranking on refactoring your code. Importing project library scripts is known to be full of issues.

[cmaynes]

While I appreciate the flexibility of Ignition, from my personal perspective it has been challenging sometimes to discern what the "official" solution is from IA vs something which may work now but may also break in a future release. It would have helped me a great deal, coming from a Python background, to maybe have the video tutorials and documentation make mention of things which may not work as one might expect given that background. Specific mention of issues around custom classes, imports between scripting modules, and which libraries to use for handling different problems, might have helped in avoiding long-term growing pains.

[bkarabinchak.psi]

Regarding libraries the consensus is

1. First party libraries are preferred aka the system.* namespace
2. Java libraries are second if you cannot work a solution with the first group of functions
3. Jython libraries are last espeically anything regarding networking you should 100% not use the jython libraries as there are known issues

Great thread here too with a lot of good info Do's and Don'ts when developing First Project with Ignition?

[robertm]

There are definitely some areas in the docs and tutorials that actively promote bad habits. Binding full UDTs comes to mind.

[nminchin]

@robertm

that actively promote bad habits. Binding full UDTs comes to mind.

There's another I saw from memory where tag values were being sent to a popup instead of the tag paths… that would definitely confuse newbies when they try to go and write to them in the popup..

The content should need to go through a dev first, or an experienced set of eyes who has scada design experience, as that's the most relevant. Software application devs don't always have the experience to know what the end user wants or how they will use it. (Even scada devs don't always know exactly how the operators will break use the scada!)

Square Hole!

[louagejasper]

So it's fixed but not really fixed? Or is it fixed when you import consistently around all your projects?

[pturmel]

The race condition described in this topic (parallel initialization threads corrupting the script module) is fixed in v8.3. In v8.1, project library auto-import is unlocked. As noted above, the attempt to fix this with a script manager lock crashed and burned due to AB-BA deadlock with jython's own import lock.

The fix in v8.3 uses the jython import lock to also guard the library auto-importer. Adding a lock check to a previously unlocked fast path is a likely performance impact.

Note that the jython import statement always takes and then releases the import lock, every time it executes. This is the primary reason you should never place any import statement inside a function or method--you will have lock contention when busy. Ignition's auto-importer only takes that lock when necessary, IIUC. The auto-importer's fast path is also the reason you should never import from project library scripts at all--avoid lock contention on startup, too.

[louagejasper]

Ah ok, I'm not familiar with import locks or guards, but what I understand:
importing at the top of my script is always safe in 8.3?

[pturmel]

@louagejasper

my script

If you mean the top of a project library script, yes. You should not use import anywhere in an event script.

[louagejasper]

Perfect, that is wonderful news to me!