Hello,
I am troubleshooting slow Tag synchronization between redundant Ignition Gateways in one of our environments.
The environment contains several independent Ignition Gateway systems. The issue occurs regularly in one affected environment, while the reference environments are working normally.
Environment:
• Ignition version: 8.1.54
• The installed modules are identical on the Master and Backup Gateways.
• Operating system: Windows Server 2022
• Redundancy mode: [Master/Backup]
• Database: MySQL
• PLC model: Siemens S7-1500 series
Architecture:
PLC | PLC communication | Active Gateway <---- Redundancy communication ----> Standby Gateway
The active Gateway communicates with the PLC and synchronizes Tag updates to the standby Gateway.
Symptoms:
1. Tag values on the active Gateway update normally in some cases.
2. The standby Gateway receives Tag changes with a noticeable delay.
3. During the problem, Tag values or quality states on the standby Gateway may lag behind.
4. The issue is intermittent and does not occur continuously.
5. Restarting or switching the redundant Gateway may temporarily improve the situation.
6. CPU and memory usage do not appear to be close to their limits.
7. The problem becomes more noticeable when many Tags change at the same time.
Approximate Tag configuration:
| Tag Group | Mode | Rate |
|---|---|---|
| HIGH_FREQ | Direct | 500 ms |
| MID_FREQ | Direct | 1000 ms |
| LOW_FREQ | Direct | 10000 ms |
| STATIC | Leased | 10000 m |
Approximate Tag count:
| Tag Provider | Approximate Tag count |
|---|---|
| Tag Provider 1 | 6685 |
| Tag Provider 2 | 2924 |
| Total | approximately 9600 |
During the problem, the redundancy statistics showed approximately:
Queued Updates/sec: 684.4 Dispatched Updates/sec : 677.3 Pending Updates: 225
The queue does not always remain at the same level. It may decrease during periods of low Tag activity and increase again when more Tags change.
Troubleshooting already performed:
• Checked CPU and memory usage on both Gateways.
• Checked the Tag Group rates.
• Checked the redundancy update queue.
• Checked the number of Tags in each Tag Provider.
Questions:
8. Is a pending update count of approximately 225 and an update rate of approximately 680 updates/sec considered high for Ignition redundancy on version 8.1.54?
9. Does Ignition redundancy synchronize every Tag value change, or are multiple changes combined into update batches?
10. Is there a recommended maximum number of high-frequency Tags for a redundant Gateway pair?
11. Could a 500 ms Direct Tag Group cause excessive redundancy traffic when several thousand Tags are changing?
12. Is there a recommended way to identify which Tag Providers or Tag Groups generate the largest number of redundancy updates?
13. Would changing non-critical Tags from 500 ms to 1000 ms, 2000 ms, or 5000 ms reduce the redundancy queue?
14. Should Tags that do not need continuous subscription be configured as Leased Tags to reduce Gateway-side scanning load?
15. Are there any known issues in Ignition 8.1.54 related to redundant Tag synchronization performance?
16. Which Gateway logs or diagnostic pages should be collected to determine whether the bottleneck is related to PLC communication, Tag execution, redundancy network communication, or standby Gateway processing?
17. Are there recommended settings for redundant Gateways with approximately 9600 Tags and several hundred Tag updates per second?
Additional information:
• The issue is intermittent rather than permanent.
• The affected environment has more frequently changing Tags than the reference environments.
• The main requirement is to keep control and interlock Tags responsive while reducing unnecessary synchronization traffic from display-only Tags.