Gateway Status: Faulted

I’m running Ignition on Windows Web Server R2, I upgraded to the latest version 7.1.2 and things seemed ok for a few days. Now when I try to access the web page I get “Gateway status: UNKNOWN” followed shortly by “Gateway status:FAULTED”. The gateway service is still running, but I can’t access anything.
Attached is the Wrapper.log file.
If there are any other logs I can get to provide more info, let me know.
wrapper.log (786 KB)

It looks like something has happened to the internal database and it is causing Ignition to be unable to start properly. You should be able to fix this easily with a backup of your gateway. If you dont have a recent backup, you can get one from this folder: C:\Program Files\Inductive Automation\Ignition\contexts\main\db\autobackup

This is a problem that has been fixed for 7.1.4 which should be out on Tuesday.

I’m seeing this again. I’m using 7.14, and in the past I would try various combos of restarting and deleting files in the Main folder, but I need a specific procedure for recovering from this if possible.

This is what’s in the wrapper log:

STATUS | wrapper  | 2010/08/13 09:44:43 | --> Wrapper Started as Service
STATUS | wrapper  | 2010/08/13 09:44:43 | Java Service Wrapper Standard Edition 32-bit 3.3.9
STATUS | wrapper  | 2010/08/13 09:44:43 |   Copyright (C) 1999-2009 Tanuki Software, Ltd.  All Rights Reserved.
STATUS | wrapper  | 2010/08/13 09:44:43 |     http://wrapper.tanukisoftware.org
STATUS | wrapper  | 2010/08/13 09:44:43 |   Licensed to Inductive Automation for Ignition Gateway
STATUS | wrapper  | 2010/08/13 09:44:43 | 
STATUS | wrapper  | 2010/08/13 09:44:43 | Launching a JVM...
INFO   | jvm 1    | 2010/08/13 09:44:44 | WrapperManager: Initializing...
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.Embedded start
INFO   | jvm 1    | 2010/08/13 09:44:49 | INFO: Starting tomcat server
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.core.StandardEngine start
INFO   | jvm 1    | 2010/08/13 09:44:49 | INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/jsp_2_0.xsd
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/jsp_2_1.xsd
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/web-jsptaglibrary_1_1.dtd
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/web-jsptaglibrary_1_2.dtd
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/web-jsptaglibrary_2_0.xsd
INFO   | jvm 1    | 2010/08/13 09:44:49 | Aug 13, 2010 9:44:49 AM org.apache.catalina.startup.DigesterFactory register
INFO   | jvm 1    | 2010/08/13 09:44:49 | WARNING: Could not get url for /javax/servlet/jsp/resources/web-jsptaglibrary_2_1.xsd
INFO   | jvm 1    | 2010/08/13 09:44:50 | INFO  [SRClassLoader                 ] [09:44:50,107]: SRClassLoader init
INFO   | jvm 1    | 2010/08/13 09:44:51 | INFO  [SRContext                     ] [09:44:51,771]: Ignition["/main", state=STOPPED] ContextState = STARTING
INFO   | jvm 1    | 2010/08/13 09:44:51 | INFO  [BundleUtil                    ] [09:44:51,776]: Requested load of resource bundle 'common' for locale 'en_US', received bundle for ''
INFO   | jvm 1    | 2010/08/13 09:44:51 | INFO  [SRContext                     ] [09:44:51,781]: Starting Ignition 7.1.4 (b5476)
INFO   | jvm 1    | 2010/08/13 09:44:51 | INFO  [SRContext                     ] [09:44:51,782]: Reloading JDBC classloader.
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [SecureRandomProvider          ] [09:44:56,567]: Creating SecureRandom object...
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [SecureRandomProvider          ] [09:44:56,567]: Generating seed...
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [SecureRandomProvider          ] [09:44:56,570]: ... Seed Created.
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [SecureRandomProvider          ] [09:44:56,570]: ... SecureRandom Created.
INFO   | jvm 1    | 2010/08/13 09:44:56 | ERROR [SRContext                     ] [09:44:56,607]: Error during context startup while initializing context properties.
INFO   | jvm 1    | 2010/08/13 09:44:56 | java.util.InvalidPropertiesFormatException: org.xml.sax.SAXParseException: Content is not allowed in prolog.
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at java.util.XMLUtils.load(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at java.util.Properties.loadFromXML(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.common.PersistentProperties.tryLoad(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.common.PersistentProperties.<init>(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.common.PersistentProperties.<init>(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.gateway.SRContext.initContextProperties(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.gateway.SRContext.init(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.wicket.protocol.http.WicketFilter.init(WicketFilter.java:719)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.ignition.gateway.bootstrap.SRFilter.init(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:108)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3709)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4363)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.apache.catalina.startup.Embedded.start(Embedded.java:825)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.catapult.CatapultBase.startTomcat(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.inductiveautomation.catapult.Catapult.main(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at java.lang.reflect.Method.invoke(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:244)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at java.lang.Thread.run(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	at java.util.XMLUtils.getLoadingDoc(Unknown Source)
INFO   | jvm 1    | 2010/08/13 09:44:56 | 	... 27 more
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [SRContext                     ] [09:44:56,608]: Ignition["/main", state=STARTING] ContextState = FAULTED
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [Application                   ] [09:44:56,674]: [SRContext] init: Wicket core library initializer
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,676]: registered listener interface [RequestListenerInterface name=IBehaviorListener, method=public abstract void org.apache.wicket.behavior.IBehaviorListener.onRequest()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,676]: registered listener interface [RequestListenerInterface name=IBehaviorListener, method=public abstract void org.apache.wicket.behavior.IBehaviorListener.onRequest()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,701]: registered listener interface [RequestListenerInterface name=IFormSubmitListener, method=public abstract void org.apache.wicket.markup.html.form.IFormSubmitListener.onFormSubmitted()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,701]: registered listener interface [RequestListenerInterface name=IFormSubmitListener, method=public abstract void org.apache.wicket.markup.html.form.IFormSubmitListener.onFormSubmitted()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,701]: registered listener interface [RequestListenerInterface name=ILinkListener, method=public abstract void org.apache.wicket.markup.html.link.ILinkListener.onLinkClicked()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,701]: registered listener interface [RequestListenerInterface name=ILinkListener, method=public abstract void org.apache.wicket.markup.html.link.ILinkListener.onLinkClicked()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IOnChangeListener, method=public abstract void org.apache.wicket.markup.html.form.IOnChangeListener.onSelectionChanged()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IOnChangeListener, method=public abstract void org.apache.wicket.markup.html.form.IOnChangeListener.onSelectionChanged()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IRedirectListener, method=public abstract void org.apache.wicket.IRedirectListener.onRedirect()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IRedirectListener, method=public abstract void org.apache.wicket.IRedirectListener.onRedirect()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IResourceListener, method=public abstract void org.apache.wicket.IResourceListener.onResourceRequested()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,707]: registered listener interface [RequestListenerInterface name=IResourceListener, method=public abstract void org.apache.wicket.IResourceListener.onResourceRequested()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,708]: registered listener interface [RequestListenerInterface name=IActivePageBehaviorListener, method=public abstract void org.apache.wicket.behavior.IBehaviorListener.onRequest()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [RequestListenerInterface      ] [09:44:56,708]: registered listener interface [RequestListenerInterface name=IActivePageBehaviorListener, method=public abstract void org.apache.wicket.behavior.IBehaviorListener.onRequest()]
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [Application                   ] [09:44:56,708]: [SRContext] init: Wicket extensions initializer
INFO   | jvm 1    | 2010/08/13 09:44:56 | INFO  [WebApplication                ] [09:44:56,902]: [SRContext] Started Wicket version 1.4.5 in deployment mode
INFO   | jvm 1    | 2010/08/13 09:44:57 | Aug 13, 2010 9:44:57 AM org.apache.coyote.http11.Http11Protocol init
INFO   | jvm 1    | 2010/08/13 09:44:57 | INFO: Initializing Coyote HTTP/1.1 on http-8088
INFO   | jvm 1    | 2010/08/13 09:44:57 | Aug 13, 2010 9:44:57 AM org.apache.coyote.http11.Http11Protocol start
INFO   | jvm 1    | 2010/08/13 09:44:57 | INFO: Starting Coyote HTTP/1.1 on http-8088
INFO   | jvm 1    | 2010/08/13 09:44:57 | Aug 13, 2010 9:44:57 AM org.apache.coyote.http11.Http11Protocol init
INFO   | jvm 1    | 2010/08/13 09:44:57 | INFO: Initializing Coyote HTTP/1.1 on http-8043
INFO   | jvm 1    | 2010/08/13 09:44:57 | Aug 13, 2010 9:44:57 AM org.apache.coyote.http11.Http11Protocol start
INFO   | jvm 1    | 2010/08/13 09:44:57 | INFO: Starting Coyote HTTP/1.1 on http-8043

This is either your cluster.xml or gateway.xml file getting corrupted. This usually happens during an unclean shutdown but it may be possible other ways as well.

What you need to do is delete the cluster.xml and cluster.xml.bak files from the $IgnitionInstall/contexts/main/home folder. If it still won’t start up then also delete gateway.xml and gateway.xml.bak. Make sure to leave the .xml_clean files alone.

In 7.1.5 we’ve tried to make this properties system more robust against bad shutdowns and be able to automatically recover in cases like this.

1 Like