Allen-Bradley Driver Locking Up

OK, I am having a new issue relates to this thread. After the cable company made some repairs on their end, the system had been running fairly well. There are occasional communications dropouts (still due to cable issues) but our customer is happy with performance.

Normally, when the site drops out, as long as the Internet connection at the site comes back up, the VPN reconnects, and everything goes back to normal. Occasionally, though, when the VPN reconnects (verified by the VPN routers’ status pages, and by pinging across the VPN) the Allen-Bradley driver does not reconnect to the controller. After they notify me, I can go into the Module configuration, restart the module, and everything will reconnect and work properly.

Looking at the wrapper log for the last major site outages, it looks like the connection problem involves this error message at some point:

INFO | jvm 1 | 2011/05/04 06:28:13 | java.io.IOException: An established connection was aborted by the software in your host machine INFO | jvm 1 | 2011/05/04 06:28:13 | at sun.nio.ch.SocketDispatcher.read0(Native Method) INFO | jvm 1 | 2011/05/04 06:28:13 | at sun.nio.ch.SocketDispatcher.read(Unknown Source) INFO | jvm 1 | 2011/05/04 06:28:13 | at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source) INFO | jvm 1 | 2011/05/04 06:28:13 | at sun.nio.ch.IOUtil.read(Unknown Source) INFO | jvm 1 | 2011/05/04 06:28:13 | at sun.nio.ch.SocketChannelImpl.read(Unknown Source) INFO | jvm 1 | 2011/05/04 06:28:13 | at com.inductiveautomation.ignition.common.nio.AbstractNioManager.read(AbstractNioManager.java:240) INFO | jvm 1 | 2011/05/04 06:28:13 | at com.inductiveautomation.ignition.common.nio.AbstractNioManager.run(AbstractNioManager.java:210) INFO | jvm 1 | 2011/05/04 06:28:13 | at java.lang.Thread.run(Unknown Source)

I’ll admit I don’t understand quite a bit of what’s in the wrapper log file, so I am attaching a couple here for you to check out. The errors for this site appear start in wrapper.log.2 around 18:28:01 on 5/3/2011, and appear to happen off and on until around 00:38:01, when the errors start appearing almost continuously. I had previously set up the system to e-mail me on any communications failure alerts. I didn’t get an alert until 4:36AM on 5/4/2011.

So, to avoid manually resetting the driver every few days for the rest of the time they own the system, do you have any suggestions? Thanks!
wrapper.log.zip (97.5 KB)

I don’t seem to see anything in the log but evidence that they are trying to reconnect, and successfully in a lot of cases.

The next time a device is stuck in disconnected mode but you know the connection is up can you go into the logging levels (Configure> Console> Levels) and search for MicroLogixDriver (assuming this is a MicroLogix you are talking about). You should see an entry that says MicroLogixDriver.<your_device_name>.

Turn the level to DEBUG and let it run for a few minutes. Then reset the driver by doing an edit/save so it successfully reconnects and send us all of the logs.

I also assume you are running 7.2.5.

Kevin,

Will do. It doesn’t happen very often (maybe once a week or so) but I’ll make myself a note to do this the next time it locks.

And, yes, we’re running 7.2.5

Also, just to make sure, to reset it now, I am going into Module Configuration, and clicking Restart on the Allen-Bradley Drivers module. You’re saying go into the Devices configuration, and do an Edit/Save on just the device that is stuck, correct?

Thanks!

[quote=“jgreenewv”]Kevin,
Also, just to make sure, to reset it now, I am going into Module Configuration, and clicking Restart on the Allen-Bradley Drivers module. You’re saying go into the Devices configuration, and do an Edit/Save on just the device that is stuck, correct?

Thanks![/quote]

Right, this will force only the device you edit/save on to reconnect. If this doesn’t cause the device to successfully reconnect but restarting the entire module does then that is something worth noting.

As requested, attached is the wrapper.log for this evening. The offending site is GroundStorageTank. Thanks!
wrapper.zip (10.6 KB)

Unfortunately the logs didn’t provide the information needed to unlock this mystery…

Is there any chance you can call in and do a GoToMeeting at a time when a device should be connected but isn’t?

Kevin,

If it happens again during normal hours, I’ll call in to tech support. Thanks.