Store and Forward hasn't caught up after near a week of a db move

We moved from a local mysql 5.7 db to an external mysql 5.7 db. The realtime tags are fine and the db is getting old data from when the db was moved, but it seems like it’s never catching up. Like it’s 3 days behind after near a week to catch up to realtime. Any thoughts on this? Thx, jake

Insufficient hardware? Perhaps. Rotating media is especially troublesome with modern workloads.

Ancient software? Definitely. (Should be on MySQL 8+ or MariaDB 10+)

Configuration flaws? Perhaps. (Insufficient indices will crush INSERT performance on big tables. Unfortunately, too many indices can also crush INSERT performance on big tables.)

Share more details (lots of details) to get more precise advice.

1 Like

I stayed with 5.7 cause it was the standard for ignition. The db is on 16vpc 104Gb with a 2Tb ssd. Ignition has a heap of 216Gb on a 32vpc 256Gb system. We never had an issue before, but the db was local, so perhaps just having the db external we need to clear out the store and forward and consider that data lost if it can’t catch up.Thx, jake

You mean it's the mysql connector version that comes with Ignition? This is for backwards compatibility only for upgrading of old systems. You should definitely not be using old database versions just because ignition's connectors are old. Use the latest db and update the associated connector in ignition.

Also, using a non-local db is most likely going to cause you issues unless you have a small number of tags being trended, aren't storing trend data very quickly, or have really good bandwidth. At least from my experience. I would always use a local db

You don’t mean local as in on the same machine right? Am I missing something?

There are several, posts on the forum about the drawbacks of running a DB on the same machine as the gateway. I would recommend never running a DB on the same system as the gateway if you can avoid it. I understand their are reasons you might have to, but if you can avoid it, do so.

2 Likes

ah, yes I probably should have clarified “local” to mean part of the local network, not local to the Ignition install. DB and Ignition should be installed on separate “hardware” (physical or virtual) but should be on the same local gigabit+ network

3 Likes

There are drawbacks, as in scaling out, but we can’t get the store and forward to catch up. So we keep getting behind on historian data.

It’s in the same data center in google cloud. I actually had a call with ignition today and they can’t get it to sync, we are just writing too much data for the db to not be local. We are looking at ways, possible several backend servers with local db’s.