We have a redundant ignition with local MySQL DB running on VMWare VMs. about 20k tags.
Everyday the store & forward starts to fill up at some time, and never completely empties.
We use tag historian for almost all tags.
We tried several configuration of write times and sizes as recommended., but we are are able to empty the local cache.
The hardware has plenty of RAM and CPU, so that doesn’t seems to be the issue.
I have noticed some clockdrift messages. Can this be related?
What has been done to tune your Ignition install and tune your MySQL install? I would not expect the settings out of the box to be suited to 20k tags, especially if the DB and GW are on the same machine.
Highly recommended. And peruse the MySQL docs for tuning. As for Ignition, you need to edit ignition.conf to use an appropriate amount of memory, and I also suggest using G1GC. See this thread.