Gateway: Constant Timeout Problems

[nifemi]

Good day,

I’m having serious issues with one of my gateway installations, here is a brief breakdown of what I’ve seen so far:

• “Error Browsing OPC” Timeout exceptions whenever I open the designer. This makes it impossible to create new tags or modify the existing tags, since the OPC server becomes inaccessible.

• “Operation exceeded the allowed amount of time to finish” whenever I modify any device settings, or OPC server settings.

Any suggestions are appreciated… Thank you.

A few pertinent details about the gateway:

Version: 7.8.4
Devices: 22 modbus tcp, 4 siemens s7
Tags: 510 across all devices
Memory Usage: ~ 400mb / 3gb
CPU usage: ~ 30%
OPC max per operation: 10 (any higher leads to all tags having “uncertain” quality)
Scan Class: @5000ms
Custom Modules: 1 module that subscribes to all tags, and pushes value changes to a RabbitMQ exchange. 1 module that retrieves several file streams from an FTP server every 5 minutes, and creates/updates driver tags with the file contents.

[pturmel]

The only time I’ve gotten similar errors, it turned out to be a lock inversion problem in my own module (an early beta of the Ethernet/IP suite), where editing a device setting would deadlock between the gateway web interface session and the driver’s setting change listener’s apply change method.
In your trace, you can see the gateway web session side of such a deadlock. You need to investigate what might be blocking in a record change listener. If you can’t find a related bug in your custom module, you’ll probably have to try to reproduce this without it installed, as there are many ways for a custom module to hang a gateway.

…says the voice of experience… /-:

[nifemi]

Hmm, thanks for the tip… there’s a rather involved tag change listener in one of my modules, I’ll start there.