HELP! - 7.9.2 upgrade - Gateway not starting

General Discussion
1

Hi, I’m in need of help urgently. Tonight I upgraded from 7.9.1 to 7.9.2 on Ubuntu 16.04 (headless). Also upgraded to latest Java 1.8-131 from 1.8-121. Following the upgrades my Gateway fails to start. Wrapper log shows a permissions error at some point following attempt to execute main. I’ve tried rolling back the Ignition version, with no luck. Haven’t tried Java yet, working on that next…

Sent in an email to support, but hoping someone here might have something to try. Clearly permissions are broken somewhere, but there is no indication of what/where.

Thanks.
Clint.

Here is the section showing the error:

INFO | jvm 1 | 2017/04/30 21:22:33 | I [o.e.j.s.ServerConnector ] [01:22:33]: Started ServerConnector@33801e47{SSL,[ssl, http/1.1]}{0.0.0.0:8060}
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp:
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: Encountered an error running main:
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: MultiException[java.net.SocketException: Permission denied, java.net.SocketException: Permission denied]
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.Server.doStart(Server.java:347)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at com.inductiveautomation.catapult.Jetpad.start(Jetpad.java:323)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at com.inductiveautomation.catapult.Catapult.start(Catapult.java:135)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at com.inductiveautomation.catapult.Catapult.main(Catapult.java:62)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at java.lang.reflect.Method.invoke(Method.java:498)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:325)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at java.lang.Thread.run(Thread.java:748)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: Suppressed: java.net.SocketException: Permission denied
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind0(Native Method)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind(Net.java:433)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind(Net.java:425)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:326)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.Server.doStart(Server.java:384)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: … 10 more
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: Caused by: java.net.SocketException: Permission denied
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind0(Native Method)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind(Net.java:433)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.Net.bind(Net.java:425)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:326)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: at org.eclipse.jetty.server.Server.doStart(Server.java:384)
INFO | jvm 1 | 2017/04/30 21:22:33 | WrapperSimpleApp Error: … 10 more

2

Did you try manually starting the gateway with a user that has root access to verify the Ignition configuration is ok?
Is the Java path in your ./data/ignition.conf correct?

3

Hi Ryan, thanks for jumping in. Yes I tried manually starting the service, and with ./ignition.sh as root, and as my admin user. Same failure occurs. After some scotch and a couple of hours thinking about it last night, I’m wondering if this has anything to do with my SSL certificate keystore…this is the first upgrade since installing a signed cert. But I’m not sure what would have changed that causes that…

I’ve restored my snapshot, so I’m back to 7.9.1 today, but that was a huge PAI, due to the fact that my Amazon EC2 instance gets a new IP address when you detach and re-attached volumes from your old snapshot…hope not to have to do that again.

I’ll check the java path to be sure.

Happy Monday…

4

This is the setup in my ignition.conf…so unless the system path was broken with the Java update it should be fine, and the ignition.conf file was not changed or replaced on the ignition update…

image.png854×258 39 KB

5

What port were you trying to use? That permissions error looks like what you'd get when you try to bind to a port < 1024.

6

Hey Kevin, ya we now are looking in that direction, IA support called and they indicated that as well. So something has changed and Ignition can’t bind to port 443. Once I spin up a test server I’ll hopefully be able to figure this out in short order.