Service wrapper is not restarting JVM in case of fault

Ignition
1

Hello,
this night our Ignition server stopped working. The cause for the fault was most problably the logfile maintenance with a logger setting to debug. The wrapper detected a problem with the JVM but failed to restart it. From the log i guess there was a timeout expiring.
Are the any settings in the wrapper configuration to optimize this behaviour?

Ignition version is 8.1.7 running on Debian Linux 10.

INFO   | jvm 2    | 2021/07/30 00:12:50 | D [o.e.m.o.s.c.s.OpcUaSubscriptionManager] [22:12:50]: Sending PublishRequest, requestHandle=6459928, acknowledgements=[id=65552/seq=404370] 
INFO   | jvm 2    | 2021/07/30 00:12:50 | Scheduling logfile maintenance. Last maintenance: Fri Jul 30 00:11:31 CEST 2021, events since then: 5036
INFO   | jvm 2    | 2021/07/30 00:12:50 | Starting logfile maintenance. Max entries: 50000
INFO   | jvm 2    | 2021/07/30 00:12:50 | D [o.e.m.o.s.c.s.OpcUaSubscriptionManager] [22:12:50]: Received PublishResponse, sequenceNumber=249177 
INFO   | jvm 2    | 2021/07/30 00:12:50 | D [o.e.m.o.s.c.s.OpcUaSubscriptionManager] [22:12:50]: onPublishComplete() response for subscriptionId=6 
...
INFO   | jvm 2    | 2021/07/30 00:12:52 | D [o.e.m.o.s.c.s.OpcUaSubscriptionManager] [22:12:51]: getTimeoutHint() maxKeepAlive=60000.0 maxPendingPublishes=2 timeoutHint=180000.0 
INFO   | jvm 2    | 2021/07/30 00:12:52 | D [o.e.m.o.s.c.s.OpcUaSubscriptionManager] [22:12:51]: Sending PublishRequest, requestHandle=32838933, acknowledgements=[] 
STATUS | wrapper  | 2021/07/30 00:13:25 | JVM appears hung: Timed out waiting for signal from JVM.  Restarting JVM.
ERROR  | wrapper  | 2021/07/30 00:13:40 | Timed out waiting for the OS to forcibly terminate the JVM process, unable to continue.
ERROR  | wrapper  | 2021/07/30 00:13:40 |   The scheduled restart of the JVM has been cancelled.
INFO   | wrapper  | 2021/07/30 00:15:10 | Wrapper Process has not received any CPU time for 89 seconds.  Extending timeouts.
STATUS | wrapper  | 2021/07/30 00:15:10 | JVM received a signal SIGKILL (9).
STATUS | wrapper  | 2021/07/30 00:15:10 | <-- Wrapper Stopped
2

This is suspicious. I've seen this when other stuff running on the gateway machine hog all the CPU time. Don't just look at Ignition's activity at that point.

3

The cause of the failure was most probably missing physical ram. It looks like the logfile database system used process memory that is not displayed in Ignitions own memory statistic. I attached more physical memory to the VM, so this special failure should not happen again.

But my question was more general regarding the wrapper settings. I think i need to adjust the timeout the wrapper waits for the JVM after it sends a SIGKILL.

4

Maybe this one: Java Service Wrapper - wrapper.ping.timeout Property

5

Thank you for the link. I think it is this one: wrapper.jvm_exit.timeout