So I've set up my ignition to have a master and a backup. when my master goes down the backup starts and when backup goes down master starts. Now these are all automatic processes which are good.
The problem is when master goes down and backup comes up and then master comes back up the backup immediately switches over to master. I do not want this scenario, I want it to remain in backup for a while to collect all the tag data and any information to update to master then switch to master. When it just immediately switches over some data were not yet updated from backup to master.
In ignition redundancy config there is a setting called
Startup Connection Allowance
I've set this to the default 30000 milliseconds but I do not think this is the correct setting to achieve the above.
Note, proper operation of redundancy expects both servers to run all the time under normal conditions. That is needed to maintain synchronization prior to failover.
when it switches from master to backup, does the backup tag browser still work/exists? My logics that run every few seconds which I see it running in backup's gateway running scripts and it logs to a log file does do its logging however none of the tag write blocking in these scripts ever write. This seems to only happen in the backup. So without the scripts writing to the tags none of my tag change scripts fire off.
same scripts
master
script runs every second -> update tags -> logs/execute tag change scripts
backup
script runs every seconds -> does not update tags -> logs/does not execute tag change scripts
is switching from master to backup consider a initialChange so the first time the script changes it will not fire off if i have
if not initialChange:
however the same does not apply from backup to master?
the below logic was in place to prevent scripts from firing off again such as when its a entire gateway restart (not a failover) this will ensure that tags do not refire, when we have a failover to ensure tags that are true do not fire off again, or saving the project.
if not initialChange and previousValue and currentValue.value:
im assuming this is affecting the cause for backup scripts not to fire off? am i doing something incorrectly?
Don't use tag change events of any kind to "hand off" from one script to another. Just have the source script run the functions the events are now running. Staying in a single interpreter is more efficient, anyways.
Edit: Further note--the initialChange flag doesn't have a solid meaning for system startup or failover or tag restart or project restart. It can mean all of those things, sometimes combined, but really the tag system doesn't know what the previous value is at that point in time. When that flag is true, the current value could be a change, but the tag system doesn't know. If it matters to your code, your code needs to figure it out from other mechanisms, like history or database.
there are hundred of scripts that does different tasks, some that requires it to run once, others constantly and some that requires certain tags to finish first, some tags changes so fast we need capture it do its logic, some base on user inputs and so on... which is why many of these constantly running scripts will signal other tags to do its logic as a one time thing whenever a condition is met. however, if the system was to failover or restart I need a way to make sure that these scripts that do not finish running continue to run.
to do this im using inprogress tags, when tag starts set to 1, finish set to 0, if failover on startup i can run the logic and then set to 0.
so my main concern is that my current condition in all my tag change scripts and gateway tag change scripts
if not initialChange and previousValue and currentValue.value:
do stuff....
and
if not initialChange and event.previousValue and event.currentValue.value:
do stuff...
when we do a failover i do not want tags that are true to refire from either gateway or tag browser, i also do not want tags to refire if they are true if the entire gateway crashes or restarts without a failover.
Your current approach cannot be made robust in a redundant system. Tag value synchronization is simply not that fast.
You need to check in your event code whether you are running on master or backup, use a JVM-specific global to keep track to detect failover. Then run code to reconstruct state over the past 10, 20, or perhaps 30 seconds.
You absolutely will lose some high speed events during failover. Sorry. Use PLC ring buffers with handshaking all the way through to a high-availability database (not in the same servers as your Ignition gateways) to ensure you do not lose events.
are the globals not lost during a restart or a failover or if the backup and master are not in sync?
so something like every few seconds update a db with our current tags and if we notice a failover repopulate the tags with the last known values but now we can have something to prevent scripts from refiring?
The global dictionary from system.util.getGlobals(), or my toolkit's better system.util.globalVarMap(), are not synchronized at all between master and backup. Which is why scripts that use them can put timestamps in them that will expose the gap on an inactive gateway (where event scripts don't run).
This stuff gets complicated quickly, and high-speed events that you must catch must be caught and held by the PLC, and only transferred by Ignition.