How to evict and replace master node in redundant setup

Ignition
1

version 7.9.8
We have two nodes (master and backup)

We’re looking to replace the master node with a newer computer. We already replaced the backup node with a new computer. What is the best way to accomplish this? Some options and thoughts running through my head:

  1. Set Master mode to Independent - hoping I can then set backup node to Master but not sure if it will let me.
    OR
  2. Set backup node to Master - Can you even have 2 Masters? Seems like that wouldn’t allow me to do that.
    OR
  3. Set the master node to Backup - I doubt you can have two backups. No one will be in charge.
    OR
    4.Simultaneously set the master to backup and the backup to master–I wish there was a switch option that could be initiated from one of the gateways. Maybe Failing over kind of sets the backup as a “Master”.

Surely someone has had to do this before. I really need to be careful I don’t sync a “fresh” gateway with no config to a running backup node that all of my 200 clients are connected to.

Any ideas?

2

I think you’re overcomplicating things.

  1. Take a .gwbk from the master.
  2. Shut down the (old) master. Clients will switch over to the backup.
  3. Spin up Ignition on the (new) master. Isolate it from the network.
  4. Restore the .gwbk onto the master.
  5. Configure redundancy on the backup to point to the new master address.
  6. Connect the master to the network.

Clients should seamlessly transfer to the new master once it assumes responsibility.

3

Thanks for the quick reply, Paul. All of our clients are using the Native Client Launcher with a launch.xml that points to master and backup IP addresses so I’m not sure how they’d find the new master if I changed IPs. I was planning on assigning the oldl master a bogus IP, spin up a new master and assign it the original IP. But I was scared that the backup node is pointing to that IP and would sync the fresh install (before I had a chance to restore the gwbk) and wipe out the backup node.

I guess I could try to isolate it like you said but that might be difficult.

Are we able to put a bogus IP in the backup node Redundancy setting to specify a master address that doesn’t exist and eliminate the risk of wiping out the config on the backup? And then I could change it back to the original (old) master IP once the new master is built and gwbk restored?

4

Yes, that would be fine. The backup also won’t overwrite a node that claims to be the master - it’ll overwrite itself, rather than push its changes to a node that’s reporting it’s a master.

5

Then maybe I am overcomplicating something. Here’s the scenario I’m trying to prevent:

  1. Master IP 1.1.1.1 and Backup 1.1.1.2
  2. Master is taken offline or blows up, clients switch to Backup
  3. New Master is rebuilt with IP 1.1.1.1
  4. Backup is trying constantly to sync with 1.1.1.1 and as soon as it sees the new master is online with IP 1.1.1.1, the Backup will pull the config from the new Master. But I haven’t had a chance to make any changes on the new Master so the Backup pulls a “fresh” config from the Master and overwrites itself.

Are you saying there are mechanisms in place to prevent that situation?

6

Well, that’s why you want to restore a .gwbk onto the master node. If the backup sees an independent node (the default state a fresh gateway will be in) at 1.1.1.1, it won’t do anything - it just won’t validate it as a peer. Once you restore the .gwbk onto 1.1.1.1, it’ll be in a valid state, and the backup should figure things out.

Isolating from the network is the ‘safe’ way to do things, to make sure nothing unexpected happens, but really shouldn’t even be necessary.

7

Nice. That makes sense. I totally forgot it would be in Independent mode on first launch–and not Master mode. Thanks again

1 Like