I am working on a site that requires all tags to belong to a 3 second polled tag group. Unfortunately, we have been experiencing intermittent comms failures for all tags under this tag group configuration. As a temporary fix, we have transferred all tags over to a subscribed tag group, which has fixed the issue. We are currently troubleshooting ways to prevent these intermittent comms issues and are hoping this forum might have some additional ideas on how to successfully use the required polled tag group configuration.
A few details about the project:
All tags come from ModbusTCP devices
There are currently ~300 connected devices with ~25,000 tags
All devices use default port, local address, communication timeout, and advanced settings
Our troubleshooting steps so far include:
Only transferring tags with good quality back to the 3s polled tag group
Disabling span gaps for most currently enabled devices
After implementing these changes, we had zero spikes in polled tag failures for around 5 hours. Unfortunately, we are back to consistent spikes in tag failure. I’ll attach a screenshot of a page that monitors tag group status which shows the frequency and severity of these polled tag group failure spikes.
I am also confused as to why there would be any difference between a polled/subscribed tag group configuration for ModbusTCP devices, as these devices should always inherently be polled.
Because you are preventing optimization by polling instead of subscribing. Modbus is always a polling protocol, so the tag group mode only changes behavior within Ignition (and does), between Ignition's own OPC server and the loopback OPC client.
So if we were to packet inspect Polled vs Subscribed shouldn't actually change anything in the frequency of Modbus polls between the Ignition server and Modbus devices no? It shouldn't affect the actual response latency either between the Modbus device and the Modbus driver either no?
Why in that case are they getting intermittent TCP socket connections dropped to the devices when their CPU usage is barely registering anything?
@Tyler_Ricks
I thought when we looked at this a while back we were seeing relatively higher Mean Response Times as well with Polled vs Subscribed right? And if we only had a couple known good devices online it was fine?
If you are mixing polled and subscribed, you can cause everything to slow down. Just subscribe to everything. There is no point to using polled mode on Ignition's loopback connection.
You will likely need to see a wireshark capture to see which end is breaking those connections. I suspect it isn't Ignition.
Using a single tag group in Polled/Read mode instead of Subscribe mode is not a good idea.
This means that every 3 seconds a single Read request is issued to Ignition's OPC UA Server for all 25,000 tags across all 300 devices. Now you have to wait for every response from every downstream device before you get your Read response back.
There's zero reason to do this instead of using a Subscribe mode Tag Group at the same rate, which allows each device to poll at 3 seconds, reporting values as it obtains them, independent of all the others.
