We recently upgraded our system to use Ignition 8.3 and are having trouble with the gateway network and remote tag history providers. The remote tag history providers will stop recording data for an unknown reason. We also do not know a specific, repeatable set of steps but we have been able to get them to start recording data again only to sometime later find they have stopped recording data. When the data is not recording, we see that the remote history providers show a status of "ERROR / Gateway not connected" but that the gateway network connections on all machines have a running status. When we attempt to edit the remote historian configurations, we can see the various gateway options in the gateway selection drop-down, but when we select the gateway with the history providers that we would like to use, we do not see the list of history providers updated with the list from that gateway. The list of history providers either is not populated at all, or it may have the list of history providers from a previously selected gateway. Does anyone have any suggestions for how to resolve this issue?
You'll probably get more help if you list what version of 8.3 you migrated to as well as what version you upgraded from. If you did any intermediary upgrades also list those.
Thanks for the suggestion; We upgraded to Ignition version 8.3.4 (b2026030309) from Ignition 8.1.32 (b2023091211). We did not have any intermediary upgrades.
As I am investigating further, it appears that there may be overall health concerns for the gateway redundant pair containing the history providers which are used by the other gateways remotely. The backup gateway shows performance warnings that Thread deadlocks have been detected. The primary gateway shows that it's redundancy peer is not connected. The primary gateway shows an incoming gateway network connection from the backup gateway that is running. The backup gateway shows no corresponding outgoing connection. The backup gateway is also showing a redundancy gateway role of "N/A" even though it has been configured as "Backup".
After rebuilding all the gateway network connections, including the connections between the redundant gateway pair, and restarting the redundant gateway pair machines, the system initially appeared to be working correctly. The redundant pair showed peers connected with the expected roles and remote history providers were able to start transferring the large amount of stored data they were unable to send previously along with many quarantined values that I was able to retry successfully. After some time however, the backup gateway reverted to the "N/A" redundancy gateway role. The performance warnings banner also appears showing deadlocked threads detected.