Ignition 8.3 Gateway stuck in "Starting" phase – Comparison with 8.1 behavior

wrapper.log (997.8 KB)

Hello,

I am experiencing a critical issue where my Gateway remains stuck in the "Starting" state for several hours in Ignition 8.3.4. his behavior started recently and has progressively worsened; it now takes approximately 2 hours to start and about 30 minutes to complete a gateway backup.

Interestingly, we have a very similar project running on Ignition 8.1 that does not exhibit this behavior, which makes me suspect there might be a change in how resources or initialization tasks are handled in version 8.3.

I attached the full wrapper.log for reference. The logs show repetitive script errors followed by a SingleConnectionDatasource lock (blocked for over 30,000 ms). Initialization of tag providers seems to be interrupted by these locks, which likely causes the massive delay.

I am looking for insights into any changes in the 8.3 startup sequence that could cause these datasource locks or if there are known issues with scripts/resource syncing blocking the internal DB during boot.

Thank you.

What size is the data/config/ignition/tags/valueStore.idb file? There was an issue that was resolved in 8.3.6 (IGN-15506) where the database size was growing because defined indexes were getting ignored and it was causing duplicate rows for tag values to be stored. For those that hit this issue, it was causing the database size to grow substantially and would cause an issue like this.

I recommend calling support as they would be able to understand if this issue is impacting you and should be able to assist in fixing the database to get it back into a usable state if this is indeed the issue.

Hi, thank you so much for the reply!

The issue was exactly what you described regarding the valueStore.idb file size. Upgrading to Ignition 8.3.6 successfully stopped the file from growing further, confirming bug IGN-15506.

To shrink the existing 5+ GB file, support provided a two-step workaround using the sqlite3 CLI tool to filter out the duplicate rows:

  1. Open the bloated valueStore.idb file, set the output to a text file, and run the .dump command to export everything into a SQL script (backup.sql).
  2. Open a new database file (fixed.idb) and execute .read backup.sql. This forces SQLite to strictly enforce the unique indexes, automatically dropping all the duplicate rows (you will see a lot of UNIQUE constraint failed errors, which is expected).

Finally, you just need to replace the old valueStore.idb with the newly generated and cleaned fixed.idb.

Since this was a Development environment, the fact that some memory tags reverted to older values wasn't an issue for us. The database size dropped dramatically, and the Gateway is now starting up instantly.

Thanks for sharing the solution. I was on version 8.3.1 RC1 and the SQLite database had grown to 7.6 GB.

After removing the duplicates, it shrank to around 700 KB and I noticed a drastic improvement in RAM and CPU usage (also upgraded to version 8.3.6).

I found that setting the following PRAGMAs before running .read backup.sql significantly improved the import speed:

PRAGMA journal_mode=OFF;
PRAGMA synchronous=OFF;