I'm having performance issues with our Ignition Gateway. It is very slow to boot (~15 min) on very performant hardware (64 vCPU 256gb RAM). I'm guessing the overinflated config.idb may be the culprit as it has grown to 3.8GB. This gateway is used fairly experimentally so there could be a number of factors leading to this bloat and it should largely be able to be cleaned up / removed without a problem but it seems to have gotten so out of hand that anytime you make a change it seems to bog down the gateway enough that the changes never actually get logged.
For example, I suspect a lot of the bloat comes from our MQTT Engine module. It currently has 14M tags, most of which are unused, bad quality, etc. When I delete them using system.tag.deleteTag, I see the tag count drop, and I see an idb-journal file begin to grow (very slowly), but it doesn't seem to push those changes to the config.idb. So if the gateway resets (which it needs to, as once the changes are made, the gateway largely locks up and is unusable) the tags reappear.
This seems to make it not possible to self clean the config.idb. Is my best bet to modify the config.idb with sqllite directly or are there other options?
Thanks!
Before you do anything else, even keep reading this post, make sure you've got multiple backups of your system and your config.idb if there's anything you care about at all. 3.8gB is nuts.
Your first, best option would be to contact our support department.
Your second best option would be to use the SQLite command line tools (available as a direct download from the SQLite website) to see what actual tables are taking up all that space in your IDB.
Then, if and only if you know exactly what the table is used for (or not used for), can you consider making direct manipulations to an offline copy of your config DB. Only if manipulations succeeded (and are guaranteed to do what you want, see earlier) should you consider replacing the live config IDB in use by your gateway with a modified one (while the service is stopped).
Since I had suspected the bloated tags from the MQTT engine were the problem, I went ahead and executed a DELETE query in the TAGCONFIG table for all tags with the PROVIDERID associated with the engine on an offline backup of the config.idb. Started up the service with this new config and its working great. Have a few back ups just in case and will check over time but I think this should be resolved. The .idb is now ~100k
1 Like
Definitely sounds like something went or is going catastrophically wrong with the MQTT engine. Might be worth dropping the Cirruslink team a line and/or a gateway backup, to see if they can get to the root cause.