F5 Load Balancer

tbackus · July 22, 2025, 1:10pm

This morning we tried to stand up two Perspective servers behind an F5 load balancer. One server is already configured and running in preparation for this. When we brought up the second server, we started getting errors in existing clients. As they would navigate they would get the initial perspective loading page, but it would stop at "Connecting....."

Log file shows these errors on both servers during the time both servers were active in the load balancer:

java.lang.UnsupportedOperationException: Feature unsupported after Upgraded to WebSocket

at org.eclipse.jetty.websocket.core.server.internal.UpgradeHttpServletRequest.getSession(UpgradeHttpServletRequest.java:293)

at com.inductiveautomation.perspective.gateway.comm.Routes.getOrCreateSession(Routes.java:1765)

at com.inductiveautomation.perspective.gateway.comm.PerspectiveWebSocketServlet$PerspectiveWebSocketCreator.createWebSocket(PerspectiveWebSocketServlet.java:185)

at com.inductiveautomation.perspective.gateway.comm.PerspectiveWebSocketServlet$PerspectiveWebSocketCreator.createWebSocket(PerspectiveWebSocketServlet.java:123)

at org.eclipse.jetty.websocket.server.JettyWebSocketServlet$WrappedJettyCreator.createWebSocket(JettyWebSocketServlet.java:276)

at org.eclipse.jetty.websocket.core.server.internal.CreatorNegotiator.lambda$negotiate$0(CreatorNegotiator.java:64)

at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1451)

at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1488)

at org.eclipse.jetty.websocket.core.server.internal.CreatorNegotiator.negotiate(CreatorNegotiator.java:63)

at org.eclipse.jetty.websocket.core.server.internal.AbstractHandshaker.upgradeRequest(AbstractHandshaker.java:76)

at org.eclipse.jetty.websocket.core.server.internal.HandshakerSelector.upgradeRequest(HandshakerSelector.java:39)

at org.eclipse.jetty.websocket.core.server.WebSocketMappings.upgrade(WebSocketMappings.java:241)

at org.eclipse.jetty.websocket.server.JettyWebSocketServlet.service(JettyWebSocketServlet.java:181)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)

at com.inductiveautomation.ignition.gateway.bootstrap.MapServlet.service(MapServlet.java:86)

at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1410)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)

at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)

at com.inductiveautomation.catapult.handlers.RemoteHostNameLookupHandler.handle(RemoteHostNameLookupHandler.java:121)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)

at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)

at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:51)

at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)

at org.eclipse.jetty.server.Server.handle(Server.java:563)

at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)

at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558)

at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379)

at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)

at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)

at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)

at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)

at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)

at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)

at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)

After trying different things it seemed like we never could get good stable connections.
We did confirm that both servers are setup identical and had all the projects copied via EAM. We are able to use both servers independently without issue.

Wondering if anyone has any words of wisdom or gotcha's we should be looking for at this point?

Thanks

Kevin.Herron · July 22, 2025, 1:14pm

A couple support tickets where this error was encountered were solved by clearing browser cache/cookies... unclear if a load balancer was involved in these.

The only piece of advice I have for load balancer setup is that sessions must be sticky. Others have more experience I'm sure.

Denny_Sheng · August 11, 2025, 10:31pm

We are experiencing the same issue. Have you identified a resolution? is there any specific setting need to be done at F5 side ?

Thanks

tbackus · August 12, 2025, 12:03pm

There is. I'm actually at my corporate offices today, I'll try and talk to the F5 guru and get the exact setting name, but I basically gave him the same info as above to make it sticky, and they use the port number 8043 as the test for the F5 to know if a server is online and should participate in the load balance activity.

Denny_Sheng · August 12, 2025, 1:41pm

Great to hear you have a resolution. Kindly update us on the testing progress—we’d like to replicate the setup into our F5 environment

Thanks

Denny

tbackus · August 13, 2025, 8:18pm

Default Persistence Profile is what was needed:
This is the screenshot they gave me, hopefully this helps.

Denny_Sheng · August 14, 2025, 7:00pm

Thanks you so much , the default persistence profile : source_addr works well for us !