Gateaway faulted on projet startup (Error during context startup)

Stephane_Boneff · December 12, 2022, 9:27am

Hello,

During the start on my PC with a gateaway installed, I have this morning a problem with the gateaway starting : java.lang.NullPointerException: null on project startup :
What can I do ?
Rebooting don't change anything. The PC was shutdown normaly before the weekend.
I didn't find any topic on that problem.

windows 10, gateaway V 8.1.11, JV : 11.0.11+9-LTS

Thanks for any help !

INFO   | jvm 1    | 2022/12/12 08:28:52 | I [Project                       ] [07:28:52]: Starting project: IU_Perspective_Tests project=IU_Perspective_Tests
INFO   | jvm 1    | 2022/12/12 08:28:52 | I [Project                       ] [07:28:52]: Starting project: global project=global
INFO   | jvm 1    | 2022/12/12 08:28:53 | E [IgnitionGateway               ] [07:28:53]: Error during context startup. 
INFO   | jvm 1    | 2022/12/12 08:28:53 | java.lang.NullPointerException: null
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.io.ByteArrayInputStream.<init>(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.common.xmlserialization.deserialization.XMLDeserializer.deserialize(XMLDeserializer.java:248)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.NamedQueryExecutor.lambda$initializeQueries$0(NamedQueryExecutor.java:96)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.ArrayList.forEach(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.NamedQueryExecutor.initializeQueries(NamedQueryExecutor.java:92)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.NamedQueryExecutor.initializeQueries(NamedQueryExecutor.java:82)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.NamedQueryExecutor.<init>(NamedQueryExecutor.java:76)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.GatewayNamedQueryManager$NamedQueryLifeCycle.onStartup(GatewayNamedQueryManager.java:326)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.project.ProjectLifecycle.onStartup(ProjectLifecycle.java:38)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.common.lifecycle.AbstractLifecycle.startup(AbstractLifecycle.java:18)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.project.ProjectLifecycleFactory$1.maybeStartLifecycle(ProjectLifecycleFactory.java:234)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.project.ProjectLifecycleFactory$1.lambda$projectAdded$0(ProjectLifecycleFactory.java:124)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.Optional.ifPresent(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.project.ProjectLifecycleFactory$1.projectAdded(ProjectLifecycleFactory.java:124)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.ArrayList.forEach(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.project.ProjectLifecycleFactory.onStartup(ProjectLifecycleFactory.java:47)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.common.lifecycle.AbstractLifecycle.startup(AbstractLifecycle.java:18)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.db.namedquery.GatewayNamedQueryManager.onStartup(GatewayNamedQueryManager.java:57)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.common.lifecycle.AbstractLifecycle.startup(AbstractLifecycle.java:18)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.IgnitionGateway.startupInternal(IgnitionGateway.java:1113)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.redundancy.RedundancyManagerImpl.startup(RedundancyManagerImpl.java:293)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.IgnitionGateway.initRedundancy(IgnitionGateway.java:724)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.gateway.IgnitionGateway.lambda$initInternal$0(IgnitionGateway.java:658)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at com.inductiveautomation.ignition.common.execution.impl.BasicExecutionEngine$ThrowableCatchingRunnable.run(BasicExecutionEngine.java:518)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | 	at java.base/java.lang.Thread.run(Unknown Source)
INFO   | jvm 1    | 2022/12/12 08:28:53 | I [IgnitionGateway               ] [07:28:53]: Ignition[state=STARTING] ContextState = FAULTED

avaughn · December 12, 2022, 2:09pm

Did you recently import named queries from a newer version? Are you certain you're running version 8.1.11? I've only ever seen this error come up when moving named queries from 8.1.6+ to a gateway running 8.1.5 or older. If you can't point to any specific named queries that were recently imported, it may be best to reach out to support to take a look at the system.

Stephane_Boneff · December 12, 2022, 2:33pm

Version 8.1.1 in fact.
A project was imported from 8.1.17. It worked fine last friday. The project that seems is the problem : "global"

avaughn · December 12, 2022, 2:36pm

In that case, the easiest solution is to upgrade to 8.1.7+ or preferably 8.1.17 or newer since that's where the resources came from. The alternative is to find the named queries that were imported from the install directory and delete or move those files out of the install directory and then restart the Ignition service. You should be able to navigate to the projects folder and search for all .sql files (new format as of 8.1.6) as those will be what needs to be removed.

pturmel · December 12, 2022, 2:39pm

Huh. How did that happen? (Projects have version numbers in them, and historically Ignition rejected attempts to import from newer files.)

avaughn · December 12, 2022, 2:49pm

Unless it's in one of the serialized resources I'm not seeing anything in a project export that indicates a version like you see in a gateway backup. I'll take a look and see if I can find if that changed at some point.

pturmel · December 12, 2022, 2:59pm

Hmm. Apparently it changed from v7.9 to v8.0 when they became zip files. I just never noticed, and never had cause to try to force a project to go backwards.

Dropping that version check seems spectacularly unwise. How much of support's reports are traced back to loading newer resources into older gateways?

avaughn · December 12, 2022, 3:07pm

That's good to know. I'll bring it up as something to potentially add back.

The only issue I've seen get traced back is this issue specifically with moving named queries from newer gateways post 8.1.6. Even this has only come up a handful of times.

Stephane_Boneff · December 12, 2022, 3:58pm

thanks to all for the help !

I try fist by deleting the named query of the lastest imported project but windows don't let me do that. Despite GW is offline, the files seem to be locked.
java.exe still alive a files are locked.

avaughn · December 12, 2022, 4:02pm

The service is probably still running even though the gateway is faulted. You can try stopping the service by running the stop-ignition.bat file or by going to services and locating the Ignition service and selecting to stop it. Then once it's stopped try deleting the files again.

Stephane_Boneff · December 12, 2022, 4:17pm

Yes I've thought to stop the service but despite it's stopped, the files are still locked The files are not visibles in perfmon.
No more java process in memory

avaughn · December 12, 2022, 4:19pm

At this point, it may be best to reach out to support for further assistance.

Stephane_Boneff · December 12, 2022, 4:22pm

I will try upgrade in 8.1.17. More news on Wednesday, not before.

avaughn · December 12, 2022, 4:25pm

If the files are locked, you may have issues running the installer as well. A full server reboot may be a good idea.

Stephane_Boneff · December 12, 2022, 4:34pm

great news : Upgrade to 8.1.17 solves the problem !
The gateaway works fine now. The named query from 8.1.17 GW imported in 8.1.1 GW was surely the root cause.
Thanks to all for the help !