BUG: Vision 7.9 fails to load 8.0 remote tags until 2nd Vision client opened

witman · October 10, 2019, 11:48pm

We have a Vision project on a v7.9.9 server accessing tags from a 8.0.4 server via a remote tag provider. When we open the page accessing these remote tags in Designer, they all display as expected. When we open it in a Vision Client on the same computer as the Designer (or another computer), all of these tags have an unknown quality overlay like this:

If we leave the first client showing quality overlays as above open and then open a second client on the same computer and open this page, the remote tags display as expected:

Once the second client is running, we can flip to a different page and then back to this page in the first client and the tags will display as expected in the first client too (if we don’t do this, the quality overlays in the first client never clear). At this point, we can close the second client and the first one keeps working. All of this behaviour seems to be consistent and repeatable.

Client diagnostics yield some insight. The client with the quality overlays has two of these errors for each remote weight tag (but no errors for the other remote tags–note that all tags in the screen clippings above are remote tags from the same remote tag provider, and other remote OPC tags like the percent and ingredient name don’t show their true values in the first client despite the fact only the tags showing weights (‘kg’ to the right of them) are giving these errors:

Tap to view error details

Message: Error fetching tag "batch/silos/7/levelWeightEstimated"
Time: Thu Oct 10 6:05:15PM
Severity: ERROR
Logger: ignition.TagManager.Gateway
Stack Trace: com.inductiveautomation.ignition.client.gateway_interface.GatewayException: Error loading remote tag.
     at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.newGatewayException(GatewayInterface.java:341)
     at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:315)
     at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.sendMessage(GatewayInterface.java:268)
     at com.inductiveautomation.ignition.client.gateway_interface.GatewayInterface.invoke(GatewayInterface.java:894)
     at com.inductiveautomation.ignition.client.sqltags.impl.db.DatasourceTagManager.getTag(DatasourceTagManager.java:474)
     at com.inductiveautomation.ignition.client.sqltags.impl.ClientTagManagerImpl.getTag(ClientTagManagerImpl.java:341)
     at com.inductiveautomation.ignition.common.script.builtin.AbstractTagUtilities.read(AbstractTagUtilities.java:184)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
     at java.lang.reflect.Method.invoke(Unknown Source)
     at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:186)
     at com.inductiveautomation.ignition.common.script.ScriptManager$ReflectedInstanceFunction.__call__(ScriptManager.java:431)
     at org.python.core.PyObject.__call__(PyObject.java:387)
     at org.python.core.PyObject.__call__(PyObject.java:391)
     at org.python.pycode._pyx27.f$0(<event:propertyChange>:30)
     at org.python.pycode._pyx27.call_function(<event:propertyChange>)
     at org.python.core.PyTableCode.call(PyTableCode.java:165)
     at org.python.core.PyCode.call(PyCode.java:18)
     at org.python.core.Py.runCode(Py.java:1275)
     at com.inductiveautomation.ignition.common.script.ScriptManager.runCode(ScriptManager.java:636)
     at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.runActions(ActionAdapter.java:180)
     at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter.access$000(ActionAdapter.java:40)
     at com.inductiveautomation.factorypmi.application.binding.action.ActionAdapter$ActionRunner.run(ActionAdapter.java:286)
     at java.awt.event.InvocationEvent.dispatch(Unknown Source)
     at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
     at java.awt.EventQueue.access$500(Unknown Source)
     at java.awt.EventQueue$3.run(Unknown Source)
     at java.awt.EventQueue$3.run(Unknown Source)
     at java.security.AccessController.doPrivileged(Native Method)
     at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
     at java.awt.EventQueue.dispatchEvent(Unknown Source)
     at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
     at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
     at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
     at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
     at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
     at java.awt.EventDispatchThread.run(Unknown Source)
 Caused by: com.inductiveautomation.ignition.client.gateway_interface.GatewayResponse$GatewayThrowable: Error loading remote tag.
     at com.inductiveautomation.ignition.gateway.sqltags.distributed.client.RemoteTagProvider.getTag(RemoteTagProvider.java:402)
     at com.inductiveautomation.ignition.gateway.sqltags.SQLTagsManagerImpl.getTag(SQLTagsManagerImpl.java:482)
     at com.inductiveautomation.ignition.gateway.project.ProjectSettingsCache$ProjectDefaultTagManagerFacade.getTag(ProjectSettingsCache.java:621)
     at com.inductiveautomation.ignition.gateway.servlets.gateway.functions.SQLTags.getTag(SQLTags.java:192)
     at sun.reflect.GeneratedMethodAccessor40.invoke(null)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(null)
     at java.lang.reflect.Method.invoke(null)
     at com.inductiveautomation.ignition.gateway.servlets.gateway.AbstractGatewayFunction.invoke(AbstractGatewayFunction.java:208)
     at com.inductiveautomation.ignition.gateway.servlets.Gateway.doPost(Gateway.java:404)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
     at com.inductiveautomation.ignition.gateway.bootstrap.MapServlet.service(MapServlet.java:85)
     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837)
     at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
     at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
     at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
     at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
     at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
     at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
     at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
     at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
     at org.eclipse.jetty.server.Server.handle(Server.java:518)
     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
     at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
     at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
     at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
     at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
     at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
     at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
     at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
     at java.lang.Thread.run(null)

The second client opened will typically show only one of these errors for only one of the weight tags–and none of the tags get quality overlays in this client:

Here are the tags for one silo:

As noted above, only the weight tags (capacityWeight, levelWeight, levelWeightEstimated, & weightUsed) are generating the errors in client diagnostics, but while those quality overlays are stuck on the only tags that show their values from here are the memory tags–and even they show the quality overlay (note the overlay on “Dryer Silo 1” text, which is based on tag dryerSiloNumber).

It seems like something is causing the first client opened on a computer to give up on these tags while still doing something in the background that results in the second client running on the same computer successfully connecting to these tags. Any idea what might be happening here, and if there’s something we can do to workaround it?

So far we’ve tried upgrading the server supplying the tags to 8.0.4 (was 8.0.2) and removing the weight unit appended to the number format specified on these tags (the only obvious difference between these tags and tags from the same remote provider that are not causing these errors). Neither of these changes made any difference.

witman · October 15, 2019, 3:50pm

TLDR: The 7.9.12 Designer Tag Browser consistently connects to tags on 8.0.4 remote tag provider, but the Designer page display no longer does and the client continues to have the same issues connecting to 8.0.4 remote tags as in 7.9.9.

Upgrading the project server from version 7.9.9 to 7.9.12 did not resolve the issue, but changed the behaviour a bit. In 7.9.9, the tags always displayed as expected on the page in Designer. Now in 7.9.12, the tag overlays and zero values occur on the page in Designer just like in the client (which still works as in 7.9.9 where tags will display as expected in second client opened, etc.–no change here going to 7.9.12).

One other change in 7.9.12 is that the tags always display consistently in the 7.9.12 tag browser [EDIT: the first half of this sentence is not true, the second half is.], and once displayed here the overlays immediately clear on the page open in Designer.

In 7.9.9 some of these tags would display with yellow tag symbols at times, and two of them would sometimes show -error- as their value.

We also tried using scripting to individually read these tags once on componentRunning in the client as well as displaying their values with direct bindings. The former worked around issues reading array tags in 7.9.9 bindings, but neither of these helped with the issues documented in this post,

witman · October 15, 2019, 4:12pm

Another detail: if the tags are showing in a Vision window in the designer when the first client is opened on the same computer ast the designer, the client will connect to the tags (no need to open a second client).

And if a client is opened with tag overlays for these before we browse the tags in designer on the same computer, we get the same issues as in 7.9.9 in the 7.9.12 Tag Browser. Some tags are showing yellow tag graphics rather than magenta and a couple of them show their value as -error-.

witman · January 30, 2020, 7:49am

Upgrading both gateways to 8.0.7 completely resolves this issue. It’s not surprising there’s an issue here with the changes to tags in version 8, but the workaround was surprising.