Store and Forward - Disk Cache slow

Using 8.0.8. Our Store and Forward disk cache is extremely slow. If we disable the disk cache, we can enter records from our tags into our MSSQL 2016 enterprise database at a minimum rate of 5000 transactions/sec. Transactions / sec are usually in the 1000-2000 range. When we enable our ignition local disk cache, the store and forward disk cache transactions go quickly as well, but only as long as the disk cache is virtually empty. Turning on the ‘force use of disk cache’ option will eventually cause too many records to build up in the cache, and our forwarding rate will slow to 0-400 transactions / sec. This causes more records to build up, and eventually the rate of insertion slows to zero.

The work-around is to disable the store and forward disk cache, and we have a new database connection set up with one of these old caches, slowly draining into the database at a rate of about 30 transactions / sec. This causes deadlock quarantines, but those can be retried, so at least we can get our data eventually.

I have a ticket in with support, but it is not moving and we’re stuck with a problem where if our sql server is rebooted, we’re going to end up losing transactions. The memory buffer is only big enough to store about 20 seconds of data, and support recommended that we decrease the size of this buffer (not sure why). This isn’t long enough for our SQL server to reboot fully, especially if the server requires updates (Windows).

Anybody run into this issue, where your server and database can handle the load but the local store & forward disk cache isn’t able to forward the transactions fast enough to keep up?

I had something similar happen and was able to resolve it by increasing the number of connections I had with the history database.

When the store and forward starts filling up Ignition will use all of the DB connections to push to the DB. If those connections are also getting used by Ignition to render charts and such there might just not be enough connections to push all of the data through.

Another thread that might help: Store and Forward Thruput

I had this problem as well. Turns out when upgrading the jdbc driver is not updated to a Java 11 compatible version, that has to be updated manually. This is my post, may solve your problem.

I am having the same issue with MS SQL Server 2017 and Ignition 8.0.13.
If I enable the disk cache, it sometimes fills really quickly and it’s very hard to drain it because it is so slow.
I ended up deactivating the disk cache as well.

I updated the JDBC driver to the latest one (mssql-jdbc-8.2.2.jre11.jar) but it did not seem to make a difference.

Did you solve this issue?

I have a customer that is experiencing this same issue as well. They are using MS SQL Server 2016 and just recently upgraded to Ignition 8.1.0. The disk cache fills up over a period of a couple hours and cannot forward records fast enough to keep up. While using the cache, it receives data at around 75 transactions/sec and forwarding to the database around 65 transactions/sec. I even deleted the old data cache and created a new one, with no observable improvement to performance. I’m not sure why the system is now doing this as it had no issues while using Ignition 7.9.12.

The temporary fix was to disable the cache to allow the tag historian to save to the database. Updating to the latest driver (mssql-jdbc-8.4.1.jre11) had no effect.

Has anyone been able to resolve this issue?