Read Timeout Tag on Gateway Network

Ignition
1

Hi all,

from time to time i received (in my opinion) randomly errors from production.
At the end i figured out that a Tag read failed with timeout.
In the wrapper log. i found this entry. 2 Servers are connected via Gateway Network SERVER 1 has Vision installed and reading tags from SERVER2.

Ignition Version both Servers running 8.1.44.
Both Servers running on the same VMWARE Environment and on the same host (at this time).

I will be happy if anyone can point me in the correct direction where to search :wink:

According IT "must be a application problem"

br,
Roland

INFO   | jvm 1    | 2025/09/23 09:39:27 | E [m.T.W.WebSocketConnection     ] [09:39:27.457]: -> outgoing local='SERVER01' remote='SERVER02' method=Metro-RunDownloadThread: Exception occurred in the local system, message id='24501' gan-remote-connection-id=https://1.1.1.1:8060/system
INFO   | jvm 1    | 2025/09/23 09:39:27 | java.util.concurrent.TimeoutException: null
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at java.base/java.util.concurrent.CompletableFuture.timedGet(Unknown Source)
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at java.base/java.util.concurrent.CompletableFuture.get(Unknown Source)
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at com.inductiveautomation.metro.impl.protocol.websocket.WebSocketConnection$RunDownload.run(WebSocketConnection.java:2018)
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
INFO   | jvm 1    | 2025/09/23 09:39:27 | 	at java.base/java.lang.Thread.run(Unknown Source)
INFO   | jvm 1    | 2025/09/23 09:39:27 | W [G.R.T.SubscriptionManager     ] [09:39:27.504]: Subscription update received from server '_0:0:SERVER02' for invalid subscription id '46'. Current id is '47' gan-remote-gateway-name=SERVER02
INFO   | jvm 1    | 2025/09/23 09:39:27 | W [G.L.S.T.SubscriptionManager   ] [09:39:27.514]: [_0:0:SERVER02_tagsubupd_SERVER01:27] The remote subscription id does not match, the tag subscriptions will be re-synchronized. gan-remote-gateway-name=SERVER02
2

An attempt to fetch something related to gateway network (the snippet of logs isn't enough to say what) via HTTP GET (out of band from the websocket) was attempted for ten minutes and timed out. Something's up in your environment.

2 Likes
3

Hi Paul,

thank you for youre reply.

INFO   | jvm 1    | 2025/09/23 09:29:51 | E [m.T.W.MetroWebSocket          ] [09:29:51.178]: <- incoming local='server01' remote='server03' method=onConnect: Could not create new web socket for 'server03|f05d84f2-8bd1-4a48-ab27-e8872041f292' at https://1.1.1.3:8060/system: A connection with system name or id 'server3' already exists on the GatewayNetwork! The new connection from https://1.1.1.3:8060/system has been rejected 
INFO   | jvm 1    | 2025/09/23 09:29:51 | java.lang.Exception: A connection with system name or id 'server03' already exists on the GatewayNetwork! The new connection from https://1.1.1.3:8060/system has been rejected
INFO   | jvm 1    | 2025/09/23 09:29:51 | 	at com.inductiveautomation.metro.impl.protocol.websocket.MetroWebSocket.onConnect(MetroWebSocket.java:170)
INFO   | jvm 1    | 2025/09/23 09:29:51 | 	at org.eclipse.jetty.websocket.common.JettyWebSocketFrameHandler.onOpen(JettyWebSocketFrameHandler.java:176)

As you said, nearly 10 minutes earlier there is this entry, i checked the naming and also the uuid of all servers they are differnet.

The Server01 is in Germany has 2 Networkcards and the server03 is located in Mexico.
Is it possible that maybe for some reason, lets say a "internet timeout" let them connect again on different nics?

Do you have any hint in which direction i can search?

For now i deactivate the GW connection to server03 in MX to see if problem still persists, but could take i while, it come up maybe once every 2 weeks.

br,
Roland

4

I'd recommend getting in contact with our support department officially so they can help you troubleshoot in greater depth:
https://support.inductiveautomation.com/hc/en-us

1 Like