Version: 8.1.36 (b2024010211)
I am investigating some issues on a customer’s gateway, and noticed that I cannot access the audit logs.
I get the error “org.sqlite.SQLiteException: [SQLITE_BUSY] The database file is locked (database is locked)”
The memory and cpu usages look good on the Gateway page and within the VM’s task manager.
Are there any suggestions on what I should check or methods for fixing this issue?
This may also relate to the other image I am attaching for gateway log errors for “DiskDataStore”.
Is something catastrophically wrong with the customer's hard disk?
The two errors are in disparate systems, with basically the only thing in common being that they're both filesystem based operations.
The audit exception is because the SQLite DB file we use to store audit information can't be reached. SQLite is a special single-file-based "DB" with some deliberate limitations, and one of those is that generally speaking only one process can be operating on a DB at a time. The "database is locked" error means either:
- Some other process has the auditing DB open at the same time you're trying to view it (unlikely, you'd presumably know about it)
- Ignition is holding a connection open to that auditing DB to insert data into it for way too long (inserts into SQLite take nano-to-microseconds except in catastrophic cases)
- SQLite is confused about the state of the DB because of some significant issue with the filesystem and is reporting back that the DB is "locked" because it can't figure out what's going on.
The DiskDataStore errors meanwhile indicate that a totally different subsystem, the gateway web page rendering's attempt to cache, is failing to run.
The only common thread between those two things is the filesystem, which implies that's the actual root cause here.
1 Like
Ok, thanks for the insights. I will tell my contacts to look into how the server is configured.
In the meantime, do you have any suggestions for diagnostic tools or fixes?
I'd start with OS level troubleshooting; I'm not well versed enough but I imagine Windows has some kind of disk integrity reporting tools built in or available, and/or there might be associated problem logs in Event Viewer or some other background mechanism. If it's a VM, maybe check the host system's logs as well. If there's other software/services running on the same hardware, check if it's having any similar symptoms; might help at least prove if it's an Ignition issue or not (I really don't think it is).
