[IGN-2343, IGN-8727] "Unable to send updates to clients" error in logs

Here's a stack trace from v8.1.19

java.io.IOException: Channel is not open.

at com.inductiveautomation.perspective.gateway.model.PageModel.send(PageModel.java:357)

at com.inductiveautomation.perspective.gateway.model.PropertySyncManager$SendUpdates.run(PropertySyncManager.java:224)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at com.inductiveautomation.perspective.gateway.threading.BlockingWork$BlockingWorkRunnable.run(BlockingWork.java:58)

at java.base/java.lang.Thread.run(Unknown Source)

Looking back, my request for the stacktrace was not going to be helpful, because what we really need is the stacktrace from when the Websocket became disconnected. These "unable to send updates" errors are the Gateway trying to send data to a session which is no longer connected. The Gateway has no idea why - only that the connection is no longer open. The extra logging which was put in place was done in the handling of the actual closure event. Look for something akin to "Websocket disconnected from session" when the "Unable to send updates to clients" logging first began. The stacktrace from the Websocket disconnect is the (hopefully) useful piece.

3 Likes

The same on v8.1.31
I had one Perspective session and one Designer session open and three VPN connection errors. I also noticed it took about 10 seconds to save a change in Designer. Sorry, no other details to share.

I have been getting this error all day I just found out as well.

Posts which inform us that you have encountered the issue will not help us resolve the issue for you, other than we can note on our internal ticket that additional customers are encountering the issue. For us to move forward in diagnosing, resolving, or working around the issue we require the requested stacktraces either here on the forums or through the Support team.

1 Like

Mine is identical to nminchin's trace.
Same version too, 8.1.19.

I will see if the one designer that is open can be closed tomorrow and see if it fixes it.
Then do a reset if not, and see if that does it.
Probably that will fix it.
If not, probably I'll do a ticket.
Sorry to bug.

Cody has clearly said that Nick's trace is not the one needed.

Yah, that is why I didn't post it.

There is one quark that I think is notable.

When I hover the magnifying glass on the log page, the project name is for a different project than the session-project.
The view is from that session-project.

Neither the project nor the session-project are inheritable.


We turned off our designers. Restarted the gateway. The issue went away.

Similar situation for me as rbachman stated.
I'm running v8.1.33

rbachman: " I think that the issue is related to Perspective views open in the Designer when the connection in the Designer is lost to the server. For example, if I have am editing a Perspective session in the Designer and get disconnected from our VPN, it appears that it may be the cause of the issue."

Still getting the warnings in the log even after terminating all Perspective sessions and closing the Designer. Restarting the gateway resolved the issue.

Same in 8.1.35

One of the thing I am noticing right now is that almost all logs produced have the same session-project

We forward all our logs to Splunk
We got some random occurence, but nothing major before January

The project that appear as session-project was modified. Modifications includes, for unknown reason, "\n" in text field (really not sure this is linked to the error)
image

here also the logs exported from Splunk at the moment the error begins last time
(removed logs as no longer required here)

1 Like

Great, thank you. I've attached those logs to the active ticket and will be reviewing them with a Dev tomorrow to see if we can glean anything. Was there any filtering done to these logs you provided?

Can you tell us anything about what events occurred between 23:33:37 and 23:38:37? It's clear from the logs that at least one session encountered an idle timeout (inactivity timeout), but it would be good to know if that session was a browser, designer, or mobile session, and what the user was doing. It's slightly curious because the only views reporting as idle are in a dashboard - when we should expect the View housing the dashboard to also report the idle timeout.

Could we also get more information about the projects called out in the logs and how they're used? a direct/private message is fine if you're hesitant to share that information on the forum.

At 23:30 it is the end of a break in the shop, so the operator went back to his workstation.

The session was in a Browser, Microsoft Edge.

Don't know what the operator did exactly. Right now, he uses 2 project in Ignition, so he might have switch browser Tab, loaded the other project in the same tab, or interacted with the active interface

The interface is a HMI that toggle bit, or update recipe values

I can send you, later today, a full backup of the server in a support ticket if you need

I don't think that's necessary, as this doesn't seem to be a result of the structure or content of the project in use, and is rather far more likely to be the result of some as-of-yet-unidentified communication issue between the Gateway and the Session(s). Is there any chance a Designer session was in use at this time, or are you confident that only browser sessions were in use during this time? Do these projects require authentication?

I know that there was no active user in the Designer, but I can't confirm that there was no Designer session openned (If there is a specific log, from Ignition gateway, so I can find designer session start/end, I can extract the information from Splunk)

No authentication required at this moment

1 Like