Redundant server upgrade to 7.4

Hey guys,

I upgraded our servers this weekend to 7.4 and the main server went great. It upgraded, restarted and had no problems. When I upgraded the backup server it did the upgrade and when it went to restart it would keep popping up in the status screen and then it would go away.

I checked it out on the server and the gateway keeps restarting itself, it’s getting some kind of error but I can’t make out what it is. I’m including the wrapper file and a thread dump from the backup gateway. Thanks!

wrapper.log (2.16 MB)

It wouldn’t let me upload the thread dump, I had to change the extension to .txt

threadDump.txt (7.83 KB)

Edit:

I completely uninstalled Ignition on the backup server and then installed a clean copy, it started fine. I then set up the redundancy settings to connect to the master gateway and it downloaded the project from the master and then started doing the same thing. It would keep restarting until it finally gives up.

Here is the wrapper.log file for the clean install of the backup server:

wrapper.log (331 KB)

This does not help you with your problem, but what I have learned from 20+ years in this business,
NEVER install a new product over a weekend or on an out of town job.

[quote=“Curlyandshemp”]This does not help you with your problem, but what I have learned from 20+ years in this business,
NEVER install a new product over a weekend or on an out of town job.[/quote]

and NEVER on Friday at 4pm

You’re right, that doesn’t help me with my problem… :wink:

It’s not a big deal since the primary server is working fine, I wouldn’t do a server upgrade without several backup plans in place. During the week or otherwise. If it was that big of a deal then I’d just go back the working version and do a gateway restore, and you can be sure that if it was that big of a deal I wouldn’t be waiting for an answer on the forums. :stuck_out_tongue:

You’ve got me beat on experience, but in the 11 years I’ve been doing this I’ve learned one undeniable truth: NOTHING goes as planned. I’d rather spend the time getting stuff working correctly on the weekends where I have plenty of time than telling production that the servers will be down for an hour during a production day to have a disaster happen and loose 3 hours of production data. Been there, done that. I’ll stick to the weekends… :thumb_right:

That reminds me of a quote from General Eisenhower, "Plans are worthless, but planning is everything"

This is a problem that has been cropping up lately due to a change somewhere along the way of how backups work. The problem is that the modules installed on the backup don’t match those of the master. The system is supposed to do a “full” restore and get them, but they’re not being included.

To fix, you basically just need to copy the “modules” folder under “{Install Dir}/home/user-lib” from the master over to the backup, and restart the backup. After doing this, the backup should do a “data-only” restore, instead of a “full” restore, and then everything should be fine (You’ll see this in the wrapper.log file, if you care- it previously said “INFO [BackupStateManager ] [07:55:10,798]: Initiating a full restore.”). Or you’ll just see that the backup starts up correctly.

Regards,

That took care of the problem. Thanks, Colby!