[IGN-4437]Project Library Randomly Not Found

We've seen some instances in the past few months of our Perspective applications having trouble calling library functions when no changes have been made to the project, the logs will just begin having entries stating the scripting path isn't recognized. When this happens we have to open the project and re-save it and this fixes the issue, which means it shouldn't be an actual problem with anything in the library (e.g. syntax errors or actual changes applied to views or the libraries themselves).

I'm wondering if anyone else has seen this? I managed to find this post on the forums from last year but unfortunately there were no responses. We have seen it roughly 3-4 times for Ignition versions 8.1.14 and 8.1.22. To the best of my knowledge there is no other information in the gateway logs and it doesn't seem to line up with high CPU or memory usage on the host server, but I wouldn't say that with 100% confidence.

Edit: Also worth pointing out that the library that stops being recognized is not consistent, it has been different libraries for different occurrences.

I've had this issue crop up before.

For me it happens so infrequently that I just restart the gateway and move on with my day. I haven't actually seen it occur in a few months so i thought they might have fixed the issue but apparently not :confused:

1 Like

Thanks for chiming in Matthew. It's not terribly frequent for us either, but doesn't stop it from being an annoyance :grimacing:

haha I wholeheartedly agree.

Do you use project inheritance? IIRC our libraries that were breaking were the ones in the inherited project but i don't 100% remember

Not at the moment, though we will be making a inheritable project soon to house our libraries in. Curious to see if that will make things worse.

Overnight we upgraded a system from 8.1.18 to 8.1.23. After the upgrade, we have started periodically getting issues where any calls to shared scripts fail with the message "global name 'Shared' is not defined". Restarting the perspective module resolves the problem, and then it reappears a few hours later.

How can I go about troubleshooting this further?

Editing to add: at least some of the time, only certain Perspective sessions display the behaviour. I have two sessions running of the same project, and only one of them is displaying the error. Closing and re-opening that session does not resolve the issue. Eventually all sessions will behave this way and require a restart of the perspective module, but it does not happen on all sessions at once.

Any insight into the script which is causing the problem?

Any script that calls a shared script from the global project fails.

A few of us have had this occur.

I personally haven't seen it in a few months, so I actually thought it was fixed. You're on a newer version than I'm on though, so apparently it's still an issue

1 Like

Yeah, I suspect there's a (relatively) rare synchronization error or race condition or whatever here that's hard to hit on small systems, but happens relatively often on busier systems, (especially with many developers?). We'll have to try to reproduce in-house, unless someone is able to come up with reliable steps.

It is quite a busy system. There were at least 3-4 developers active most of yesterday, each with 1-3 designers open, while this was occurring. And perhaps a dozen sessions open?

Are you aware of anything we can do to resolve it in the meantime other than restarting the perspective module?

Pushing changes to the project may be sufficient to get the scripting system to "restart". Particularly if you make a change to the script library.
Or you could try changing a file in the pylib directory on the gateway, though that will affect all projects, not just one.

I've combined these two threads together and added the internal issue we're tracking this as. If anyone is able to provide anything to aid replication, it would be greatly appreciated. You can also contact support if this continues to be a significant issue, which will help raise its prioritization on our backlog.

4 Likes

As mentioned before, it definitely seems to happen more often when I (and/or other developers) have multiple designers open. If it's of relevance, most of the developers are using MacOS, the gateway is a linux Docker instance, and the clients are a mixture of Windows PC's running Perspective Workstation, and Windows/Macs running various web browsers (Edge, Chrome, Firefox, Safari)

1 Like

Does it manifest as an error on all Perspective sessions (including in the designer?), or just a random one out of the list of open sessions? Or does it only affect the designer?

It affects both the designer and sessions running in either web browsers or the Perspective Workstation app, but not consistently. One session will start playing up, a few minutes later another, maybe 20-30 minutes later another will start doing it. If I close the affected session and re-open, it doesn't fix it, but other sessions on other devices remain fine during this time.

1 Like

Try clearing cookies after closing a session before re-opening. (This might be a session tracking bug, perhaps in Jetty.)

1 Like

I'll give that a try. How do I clear cookies from a Perspective Workstation session?

I don't know. I was thinking browser sessions.

I can't speak to the designer question, but anytime it crops up for us, it's because the Perspective sessions that the end users are using are failing to behave correctly. So I believe (in my experience) it's been all sessions.

1 Like