I just updated our Ignition gateways and I keep getting error with the IEC driver module.
Im running on Windows 2016 and this is what my backup gateway is showing:
I have tried to remove license / manually uninstall the module on the backup gateway and resync with reactivation but everytime I tried, it failed with the same error message.
I downloadded the file manually and tried to install it on both gateway (master and backup) and I am getting the same error without sync the servers.
I ended up uninstalling the module, because I was getting incompatibility warnings on the gateway status page.
I'll talk with QA on Monday and see if there's something fundamentally broken when using Windows Server 2016 or if it works and it's not working on just on your just system for some reason.
I started getting this error after the update (8.1.17 to 8.1.25)
Unexpected error initializing redundancy session. Will try again in 1 minute.
com.inductiveautomation.metro.api.ex.TimeoutExpiredException: TimeoutExpiredException[queueId=service_GatewayRedundancyService, targetIntent=_services.invoke_|2, msgData=com.inductiveautomation.ignition.gateway.redundancy.services.GatewayRedundancyService#registerPeerNode([class com.inductiveautomation.metro.api.ServerId, class com.inductiveautomation.ignition.gateway.cluster.NodeInfo])]
at com.inductiveautomation.metro.impl.queues.StandardQueue.processTimeouts(StandardQueue.java:229)
at com.inductiveautomation.metro.impl.queues.StandardQueue.lambda$maybeScheduleTimeoutChecker$1(StandardQueue.java:211)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
spam every minute only on redundant gateway. We are on version 1607 of Windows with most recent updates.
So in trying to duplicate this issue I have been unsuccessful and Windows Server 2016 seems to be working as expected with my installation of the software. A few questions I have:
Which version of Server 2016 is everyone using? I am currently using Server 2016 Standard Edition with all current patches applied.
It sounds like this is impacting backup gateways. Is is affecting the master Gateway at all?
Are all Gateways running the same OS?
How was Ignition installed? (Installer or ZIP install)
Are you running Ignition or Edge?
Are there any additional programs installed on the Gateway that could be monitoring/possibly blocking a .dll loading? Because we are loading a native library, this could be a possibilty.
Anything else unique about these environments that you think could play into this.
Ignition is running as a service and, in this case, we are requesting %TEMP% as the location to move the files. Because your Ignition service is running as a User, that is the location it is being stored at. If it were run as a System service, it would likely be stored in c:\Windows\temp. While not wrong, we do have a temp directory within the Ignition install that we should be targeting (and will be making a change for).
I am guessing that AppLocker is preventing DLLs from being loaded from that directory and us making that change will potentially resolve this issue. If we can get this change into a nightly release would it be possible to see if it resolves the issue by installing just the new module?
Us changing where the temp directory is located should hopefully resolve the issue you are seeing with the module loading. It would make sense based on everything I am reading about .dll placement and settings that can be applied.
For the other error, it looks like it is occurring when it is attempting to process the initial connection. We don't think that it should have anything to do with the 61850 module, as this would cause the peer incompatible status that you also mentioned seeing, but seeing that a connection was successful seems like there might be disconnects occurring. When you removed the module, did the com.inductiveautomation.metro.api.ex.TimeoutExpiredException error stop occurring?
I've tried on 2 different Windows gateways (both Windows 10 LTSC / 1809) with same result. I don't see a similar error on the one Linux gateway I tested with.
@malone46844, I moved your post into this existing thread, because it's almost certainly the same root cause. If you're able to help Garth work identify what's different between your system and our test systems, it would be greatly appreciated - even if not, you can be assured we're aware of the issue and working on a fix.
If you are seeing the dependent library error when Ignition attempts to load the 61850 driver, the fix is going to be to download and install the latest Microsoft Visual C++ Redistributable package:
The Triangle Microworks library that we leverage has dependancies on items included in that package. Uninstalling the module is also an option if you don't have a need for it.
We are looking into what can be done to prevent this requirement, but that is the current solution. We appreciate your help in letting us know and providing information to help assist in troubleshooting the issue.
Edit: The following KB article has been posted about this issue:
Is the IEC 61850 driver compatible with Linux Ignition installations? Mention of Visual C++ libraries would seem to make this MS Windows dependent.
We are receiving this error when attempting to install on OEL8 Linux:
/tmp/iec61850_native2868284322401336071/libcLibrary.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /tmp/iec61850_native2868284322401336071/libTMWBaseCPP.so)
but it was working fine in 8.1.17 without this module.
- even if not, you can be assured we're aware of the issue and working on a fix.