Stuck at "gateway starting"

Hi,

The ignition gateway was working fine but one day suddenly stopped working, and it is stuck at "gateway starting" screen when I try to access through browser. Here is the wrapper file...

Could someone give me a hand please?

wrapper.log (20.9 KB)

Try deleting (or renaming or moving to desktop) the $IGNITION/lib/runtime/jre-$OS/lib/security/cacerts file and then restart the Ignition Gateway again.

Hi that solved the issue. Why might this happen? I'd like to know so I can prevent it from happening in the future... I was suspecting the disk space running out ...

edit, also how did you know that, that would solve the problem?

Could have been an abrupt shutdown that left the cacerts file corrupt, not entirely sure.

I knew because 1) I've seen this before and 2) I'm able to figure out what's happening in the code from the stack trace in your logs, which is not something you would be able to do.

2 Likes

Thank you so much for your answers.

Hi I know this is solved but I have a follow up question about deleting this file.

we have multiple systems running on customer sites and have seen this issue a few times. it has always been fixed by manually deleting the file and thinking nothing more of it.

However, we are now thinking of making a script to run on start up to check for the file and delete it if found.

  1. Are there any long-term side effects from always deleting this file? are there any specific cases where the file is needed?
  2. When is this file created? If we delete the file does it ever reappear or do we not need to worry after it has been deleted once?

Hmm, looking at this again, I'm not sure deleting it is the right move, because I'm not sure if it ever gets created again except when we update the Java runtime that gets shipped or something.

I would think that yes you need this file unless you never need to make SSL/TLS connections to any other hosts...

Thanks for the fast reply, Kevin.

Just to clarify this would only affect us if were using a web server?

So if we are not using the web server our approach of just deleting the file is ok?

Any kind of outbound connections with system.net.httpClient() or system.net.http* would also be affected I would think, as long as the host was HTTPS.

I'm trying to figure out if recommending this was supposed to be an intermediate troubleshooting step or a solution, because I'm not seeing anything that restores this file. We make a backup of it before importing supplemental certs, but nothing seems to use it.

I was under the impression java's cacert file was optional, with java falling back to OS cert stores. IIRC.

No, not as far as I'm aware. Just verified in jshell after moving cacerts into cacerts.bak:

kevin@DV-kevin ~/L/J/J/c/C/Home> bin/jshell
|  Welcome to JShell -- Version 24.0.1
|  For an introduction type: /help intro

jshell> import java.net.http.*;
   ...>
   ...> var client = HttpClient.newHttpClient();
   ...> var request = HttpRequest.newBuilder().uri(URI.create("https://www.google.com")).build();
   ...>
   ...> client.send(request, HttpResponse.BodyHandlers.ofString());
client ==> jdk.internal.net.http.HttpClientImpl@6fd02e5(1)
request ==> https://www.google.com GET
|  Exception javax.net.ssl.SSLException: (internal_error) Unhandled exception
|        at HttpClientImpl.send (HttpClientImpl.java:932)
|        at HttpClientFacade.send (HttpClientFacade.java:133)
|        at (#4:1)
|  Caused by: javax.net.ssl.SSLException: (internal_error) Unhandled exception
|        at Alert.createSSLException (Alert.java:132)
|        at TransportContext.fatal (TransportContext.java:376)
|        at TransportContext.fatal (TransportContext.java:319)
|        at SSLEngineImpl$DelegatedTask.run (SSLEngineImpl.java:1209)
|        at ArrayList.forEach (ArrayList.java:1604)
|        at SSLFlowDelegate.lambda$executeTasks$0 (SSLFlowDelegate.java:1148)
|        at HttpClientImpl$DelegatingExecutor.execute (HttpClientImpl.java:168)
|        at SSLFlowDelegate.executeTasks (SSLFlowDelegate.java:1143)
|        at SSLFlowDelegate.doHandshake (SSLFlowDelegate.java:1109)
|        at SSLFlowDelegate$Reader.processData (SSLFlowDelegate.java:508)
|        at SSLFlowDelegate$Reader$ReaderDownstreamPusher.run (SSLFlowDelegate.java:283)
|        at SequentialScheduler$LockingRestartableTask.run (SequentialScheduler.java:182)
|        at SequentialScheduler$CompleteRestartableTask.run (SequentialScheduler.java:149)
|        at SequentialScheduler$SchedulableTask.run (SequentialScheduler.java:207)
|        at ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1095)
|        at ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:619)
|        at Thread.run (Thread.java:1447)
|  Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
|        at PKIXValidator.<init> (PKIXValidator.java:96)
|        at Validator.getInstance (Validator.java:173)
|        at X509TrustManagerImpl.getValidator (X509TrustManagerImpl.java:308)
|        at X509TrustManagerImpl.checkTrustedInit (X509TrustManagerImpl.java:183)
|        at X509TrustManagerImpl.checkTrusted (X509TrustManagerImpl.java:254)
|        at X509TrustManagerImpl.checkServerTrusted (X509TrustManagerImpl.java:144)
|        at CertificateMessage$T13CertificateConsumer.checkServerCerts (CertificateMessage.java:1304)
|        at CertificateMessage$T13CertificateConsumer.onConsumeCertificate (CertificateMessage.java:1203)
|        at CertificateMessage$T13CertificateConsumer.consume (CertificateMessage.java:1146)
|        at SSLHandshake.consume (SSLHandshake.java:393)
|        at HandshakeContext.dispatch (HandshakeContext.java:476)
|        at SSLEngineImpl$DelegatedTask.run (SSLEngineImpl.java:1207)
|        ...
|  Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
|        at PKIXParameters.setTrustAnchors (PKIXParameters.java:202)
|        at PKIXParameters.<init> (PKIXParameters.java:121)
|        at PKIXBuilderParameters.<init> (PKIXBuilderParameters.java:105)
|        at PKIXValidator.<init> (PKIXValidator.java:93)
|        ...
1 Like

Would Gateway Networks or OPC UA be affected? i see SLL mentioned on the Gateway network page

Perhaps stash a clean copy of that file somewhere and restore from it on every startup?

1 Like