Hi,
I use Ignition 8.0.8. when I run system.tag.queryTagHistory, I get ‘Read Time Out’ error after 60 seconds. This only happen when I fetch data for a number of days and take more than 60sec.
Project’s Read Timeout property is set to 900,000.
Thank,
Raj
system.tag.queryTagHistory has a dedicated timeout parameter, because history queries can take a long time and shouldn’t require setting the overall client read timeout egregiously high.
Thanks for the info Griffith. How do I change this parameter and where can I find it?
I understand that it shouldn’t be too high for a SCADA system but for our application it is critical.
Just add timeout=900000 to your system.tag.queryTagHistory() scripting function call; eg:
system.tag.queryTagHistory(paths, <other arguments>, timeout=900000)
I didn’t notice that. Thank you very much for the help.
Now I get this error:
caused by GatewayException: An I/O error occurred while sending to the backend.
caused by PSQLException: An I/O error occurred while sending to the backend.
Server is pretty solid with good configuration.
Can you post the full stack trace? PSQLException is coming from the Postgres JDBC driver, not anything in Ignition’s code.
16:33:29.982 [AWT-EventQueue-0] ERROR com.inductiveautomation.ignition.client.util.gui.ErrorUtil - <HTML>Error executing script for event: <code><b>mouseClicked</b></code><BR>on component: <code><b>Download</b></code>.
com.inductiveautomation.ignition.common.script.JythonExecException: Traceback (most recent call last):
File "<event:mouseClicked>", line 66, in <module>
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.newGatewayException(GatewayInterface.java:341)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:315)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.runTagHistoryQuery(GatewayInterface.java:752)
at com.inductiveautomation.ignition.client.script.ClientTagUtilities.queryTagHistoryImpl(ClientTagUtilities.java:320)
at com.inductiveautomation.ignition.common.script.builtin.AbstractTagUtilities.queryTagHistory(AbstractTagUtilities.java:565)
at jdk.internal.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
com.inductiveautomation.ignition.client.gateway_interface.GatewayException: com.inductiveautomation.ignition.client.gateway_interface.GatewayException: An I/O error occurred while sending to the backend.
at org.python.core.Py.JavaError(Py.java:552)
at org.python.core.Py.JavaError(Py.java:543)
at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
at com.inductiveautomation.ignition.common.script.ScriptManager$ReflectedInstanceFunction.__call__(ScriptManager.java:519)
at org.python.core.PyObject.__call__(PyObject.java:413)
at org.python.pycode._pyx432.f$0(<event:mouseClicked>:108)
at org.python.pycode._pyx432.call_function(<event:mouseClicked>)
at org.python.core.PyTableCode.call(PyTableCode.java:171)
at org.python.core.PyCode.call(PyCode.java:18)
at org.python.core.Py.runCode(Py.java:1614)
at com.inductiveautomation.ignition.common.script.ScriptManager.runCode(ScriptManager.java:770)
at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.runActions(ActionAdapter.java:200)
at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.invoke(ActionAdapter.java:291)
at com.inductiveautomation.factorypmi.application.binding.action.RelayInvocationHandler.invoke(RelayInvocationHandler.java:57)
at com.sun.proxy.$Proxy44.mouseClicked(Unknown Source)
at java.desktop/java.awt.AWTEventMulticaster.mouseClicked(Unknown Source)
at java.desktop/java.awt.Component.processMouseEvent(Unknown Source)
at java.desktop/javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.desktop/java.awt.Component.processEvent(Unknown Source)
at java.desktop/java.awt.Container.processEvent(Unknown Source)
at java.desktop/java.awt.Component.dispatchEventImpl(Unknown Source)
at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.desktop/java.awt.Container.dispatchEventImpl(Unknown Source)
at java.desktop/java.awt.Window.dispatchEventImpl(Unknown Source)
at java.desktop/java.awt.Component.dispatchEvent(Unknown Source)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
at java.desktop/java.awt.EventQueue$5.run(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.desktop/java.awt.EventDispatchThread.run(Unknown Source)
Caused by: org.python.core.PyException: Traceback (most recent call last):
File "<event:mouseClicked>", line 66, in <module>
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.newGatewayException(GatewayInterface.java:341)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:315)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.runTagHistoryQuery(GatewayInterface.java:752)
at com.inductiveautomation.ignition.client.script.ClientTagUtilities.queryTagHistoryImpl(ClientTagUtilities.java:320)
at com.inductiveautomation.ignition.common.script.builtin.AbstractTagUtilities.queryTagHistory(AbstractTagUtilities.java:565)
at jdk.internal.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
com.inductiveautomation.ignition.client.gateway_interface.GatewayException: com.inductiveautomation.ignition.client.gateway_interface.GatewayException: An I/O error occurred while sending to the backend.
... 46 common frames omitted
Caused by: com.inductiveautomation.ignition.client.gateway_interface.GatewayException: An I/O error occurred while sending to the backend.
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.newGatewayException(GatewayInterface.java:341)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:315)
at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.runTagHistoryQuery(GatewayInterface.java:752)
at com.inductiveautomation.ignition.client.script.ClientTagUtilities.queryTagHistoryImpl(ClientTagUtilities.java:320)
at com.inductiveautomation.ignition.common.script.builtin.AbstractTagUtilities.queryTagHistory(AbstractTagUtilities.java:565)
at jdk.internal.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:188)
... 43 common frames omitted
Caused by: com.inductiveautomation.ignition.client.gateway_interface.GatewayResponse$GatewayThrowable: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:335)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)
at com.inductiveautomation.ignition.gateway.datasource.SRConnectionWrapper$SRPreparedStatement.executeQuery(SRConnectionWrapper.java:1007)
at com.inductiveautomation.gateway.tags.history.query.DatasourceQueryExecutor.advancePartition(DatasourceQueryExecutor.java:1037)
at com.inductiveautomation.gateway.tags.history.query.DatasourceQueryExecutor.primeRead(DatasourceQueryExecutor.java:958)
at com.inductiveautomation.ignition.gateway.sqltags.history.query.AbstractHistoryLoader.startReading(AbstractHistoryLoader.java:104)
at com.inductiveautomation.ignition.gateway.sqltags.history.query.AggregateHistoryQueryExecutor.startReading(AggregateHistoryQueryExecutor.java:85)
at com.inductiveautomation.ignition.gateway.sqltags.history.query.HistoryWriter.readData(HistoryWriter.java:312)
at com.inductiveautomation.ignition.gateway.sqltags.history.query.HistoryWriter.execute(HistoryWriter.java:230)
at com.inductiveautomation.ignition.gateway.sqltags.history.TagHistoryManagerImpl.queryHistory(TagHistoryManagerImpl.java:491)
at com.inductiveautomation.ignition.gateway.tags.model.ProjectDefaultTagManagerFacade.queryHistory(ProjectDefaultTagManagerFacade.java:416)
at com.inductiveautomation.ignition.gateway.servlets.gateway.functions.QueryTagHistory.invoke(QueryTagHistory.java:90)
at com.inductiveautomation.ignition.gateway.servlets.Gateway.doPost(Gateway.java:411)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.inductiveautomation.ignition.gateway.bootstrap.MapServlet.service(MapServlet.java:86)
at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1391)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:760)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(null)
Well, it’s coming from Postgres. Some searching seems to suggest that the connection may be getting dropped because it’s taking too long to return results - do you have a firewall or any other devices between the gateway and the database?
Hi PGriffith,
No, both reside on the same server. Anything else that might contribute to this error?
Nothing I’m aware of. @pturmel is the resident Postgres expert; he may have run into this before.
If it’s an urgent problem, I’d get in contact with support - someone there may have also encountered this before.
PostgreSQL has its own settings for query timeouts. You might want to check those. You might also want to figure out why these queries take so long. With proper indices, appropriate RAM allocations, and a decent LAN, you should be able to return hundreds of thousands of history records in a few seconds.
FWIW, the settings in postgresql.conf “out-of-the-box” are only appropriate for the wimpiest machines–think embedded applications. Any real DB hardware will need substantial increases to the memory usage settings.
Thanks @pturmel and @PGriffith for the tips. I will go over postgres settings.
Thank you very much for the quick help.
From what I've noticed playing around with the timeout parameter, it seems like depending on the scope, it doesn't actually work - In the designer (aka Vision scope) it will timeout after about 60 seconds no matter what. Has anyone else noticed this?
The designer, and Vision clients, have a hard-coded 60-second timeout for gateway callouts.