Upgrading Ignition gateway troubles

I have a gateway with redundancy and we recently upgraded this server from 8.1.17 to 8.1.20.

I initiated the failover in order to upgrade the master. When the master gateway started all the gateway network connections and redundancy connection were highly unstable. The cpu was going through the roof and was cycling between 60-99 %.

This caused quite the problem with our integrations as the backup took over for for increments of 1-1.5 seconds which in turn caused multiple clients to be launched from both gateways.

Is the connection instabilty a product of the high load or is it something else? Is this a known issue to have?

You should probably talk this over with our support department for a more in-depth answer, but based on the information provided, yes, high CPU could indirectly cause connectivity issues. The most likely path would be:

  1. Gateway CPU is struggling for whatever root cause
  2. Garbage collection causes increasingly long pauses because the CPU is struggling
  3. External services (such as remote communications, potentially the master/backup failover) see that the gateway is not responding (because it’s paused doing garbage collection) and fail or otherwise trigger negative effects.

There are some settings you can tweak for the gateway’s redundancy to make it more tolerant of short duration faults like this, but the primary thing to identify is why the high CPU usage was happening in the first place. Support is your best bet for that.