Scheduled Gateway Backups Failing

OS ver RHEL 8.5
Java ver 11.0.14.1+1-LTS
Ignition ver 8.1.16

Scheduled Gateway backups are failing with the following error:

java.io.FileNotFoundException: /usr/local/bin/ignition/{shareName}/{serverName-timeStamp}.gwbk (Resource temporarily unavailable)
at java.base/java.io.FileOutputStream.open0(Native Method)
at java.base/java.io.FileOutputStream.open(Unknown Source)
at java.base/java.io.FileOutputStream.(Unknown Source)
at java.base/java.io.FileOutputStream.(Unknown Source)
at com.inductiveautomation.ignition.gateway.ScheduledBackupManager.createBackup(ScheduledBackupManager.java:145)
at com.inductiveautomation.ignition.gateway.ScheduledBackupManager.run(ScheduledBackupManager.java:119)
at com.inductiveautomation.ignition.common.execution.impl.BasicExecutionEngine$TrackedTask.run(BasicExecutionEngine.java:582)
at it.sauronsoftware.cron4j.RunnableTask.execute(Unknown Source)
at it.sauronsoftware.cron4j.TaskExecutor$Runner.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

I’ve verified that permissions are correct on the /usr/local/bin/ignition/{shareName} folder and have even created files there from the server outside of the Ignition Gateway using the Ignition user account.

/usr/local/bin/ignition/{shareName} is a mount that is set up in fstab:

//{server}/{shareName} /usr/local/bin/ignition/{shareName} cifs credentials=/root/.smbcredentials,file_mode=0777,dir_mode=0777 0 0

I’ve tried changing the scheduled time to different times thinking maybe there was a conflict with something else to no avail. However, if I log into the Gateway and set the scheduled backup for say 5 minutes into the future, the Scheduled Job runs successfully. I even thought maybe it was running successfully because I was logged into the Gateway so after changing the schedule, logged out and it still ran successfully. Obviously I don’t want to be logging in daily and setting the backups for a near future time. Any suggestions? TIA

I've figured out the issue:

Our share is located on a NetApp SAN. With ONTAP v9.7P8 and onwards, there is a bug that closes CIFS sessions due to idle timeout for Linux clients. The default timeout is 900 seconds. I changed the idle timeout to 2 days and our Gateway backups work after that.

NetApp command: vserver cifs options modify -vserver XXXX -client-session-timeout

Hope this helps anyone else that may encounter this issue.

3 Likes