Ideal way to do redundant pair of Ignition Edge gateway- dedicated NICs and direct connection?

[RayJohn]

Hello all:

We plan on building out a of pair of redundant Ignition Edge gateway.

Based on some digging, there are some gotchas:

Architecture for Distributed Control Center

Ignition redundancy is both latency sensitive and subject to split brain syndrome. If the comms goes down between sites, both servers will end up active and therefore numerous properties start to diverge. So let me amend my answer to: Ignition redundant servers must be installed on a common LAN with a robust network layout (preferably multi-path, like RTSP), such that any loss of comms also results in loss of comms to all clients and all devices and all databases. (Trunk port for multiple VL…

Client connections to server with multiple NICs

That is a design error. The redundant gateway needs to be on the exact same networks as the primary gateway. And both primary and redundant gateways need to be set to use their names as their public addresses. If you are connecting remotely, put entries in your own /etc/hosts file with the appropriate names if you cannot make your local DNS provide them. (In most cases, there should not be a redundancy network. It can lead to "split brain" situations.)

Based on @pturmel 's comments - it's looking like I should dedicate a NIC on each machine, and directly patch them to each other, and just give them static IPs - in order to avoid this split brain situation. Then have the Network Bind Interface setting as the static IP

The 2 other NICs of each machine I will team/bond them and send that off to redundant pair of firewalls.

Am I on the right track here?

Thanks!

[pturmel]

Opposite. The layout most resistant to split-brain is to have only one network and subnet on the gateways, and let your infrastructure route everything else.

[RayJohn]

Thanks for the response, Phil.

This 3rd NIC proposal is due to concerns with redundant firewall pair also acting as a switch - if the firewall fail over, we might get the split brains on the Ignition gateway pair.

[michael.flagler]

If using redundant firewalls/routers, they should be set up to use VRRP/HSRP or some other FHRP to provide the redundancy. This means they'll have a virtual shared IP and MAC address and the redundancy should be transparent to the machines.

Bonding the NICs on the Edge gateways should be fine, but your firewalls should be directly connected with a cable also so that they can determine their own redundancy status. I'm assuming since they would have their networks also connected directly between the firewalls that any packets arriving on the backup firewall would get forwarded to the primary firewall to be sent to the WAN.

Another option may be to only have a single connection from each Edge device (not bonded, or if bonded, they both go to the same firewall) where your Master Edge gets connected to the master firewall, and the backup Edge connects to the Edge firewall. (Firewalls still need connected to each other to determine redundancy status).

But....my recommendation would be to get someone very knowledgeable in network routing/redundancy more than I am to look this over to determine any gotchas and to fully test this before putting it into production.

[pturmel]

Consider using OpenVSwitch or similar on the gateways, and trunking w/ RSTP between them and also to the firewalls. So the gateways only have the one subnet and IP address each, and all the redundancy is handled at layer 2 with rapid spanning tree. Put an extra trunk between the firewalls, and you can lose any 2 without getting to split brain.

You do need to then treat any trunk failure as a prime emergency.

Edit: Or what Michael said.

[RayJohn]

One can "just" slap another Ignition Edge gateway in front of the redundant pair, as a gateway/aggregator? - assuming that Ignition can handle other Ignition OPC UA redundancy better than other products...

(I'll blink four time if I'm about to be taken by Inductive blackops)