[BUG-14137] Ignition IdP failover

Dears,
I am currently using the 8.0.1 release.
I set two Ignition IdPs, one based on an AD user source, the other to a database user source. The database user source works as a failover source when the AD is unavailable.

It seems the authentication through the backup IdP does not work properly: the username is verified, instead the user roles are not retrieved.

Are you able to replicate this issue?
Thanks a lot!

Hi @andrea.morando -

Are there any errors in the logs when you perform the login? Try setting the following gateway loggers to debug:

gateway.IdpAdapter
gateway.UserAttributeMapper
gateway.ExpressionSecurityLevelPolicy
gateway.WebAuthSessionImpl

Also make sure that the roles you want for the given user are set in both user sources. What could be happening is: the authentication is successful for the primary user source, but when the security levels are granted, a fetch for the user from the primary user source occurs and could be failing over at that point to the secondary user source (or vice-versa).

Dear,
as suggested, I traced those loggers and I got the following message:

INFO | jvm 1 | 2019/06/06 10:37:20 | T [g.WebAuthSessionImpl ] [08:37:20]: Shutting down web auth session view=Faceplates/Common/DateTime@T[0]$0:0$0:2, idp-adapter-id=4
INFO | jvm 1 | 2019/06/06 10:37:27 | T [g.WebAuthSessionImpl ] [08:37:26]: Starting up web auth session idp-adapter-id=4, route-group=federate, route-path=/callback/:type
INFO | jvm 1 | 2019/06/06 10:37:27 | D [g.WebAuthSessionImpl ] [08:37:26]: Unable to handle login response route-group=federate, route-path=/callback/:type
INFO | jvm 1 | 2019/06/06 10:37:27 | com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapterException: Unable to parse the WebAuthResponse from the HTTP request
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapter.parseAttributes(IdpAdapter.java:130)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.onLoginResponseInternal(WebAuthSessionImpl.java:176)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.lambda$onLoginResponse$1(WebAuthSessionImpl.java:191)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.mdc(WebAuthSessionImpl.java:93)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.onLoginResponse(WebAuthSessionImpl.java:191)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapterConfigRoutes$TestLoginWebAuthResponseHandler.handle(IdpAdapterConfigRoutes.java:297)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.federation.FederationRoutes.callback(FederationRoutes.java:135)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.dataroutes.Route.service(Route.java:247)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.dataroutes.RouteGroupImpl.service(RouteGroupImpl.java:49)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.dataroutes.DataServlet.service(DataServlet.java:87)
INFO | jvm 1 | 2019/06/06 10:37:27 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.Server.handle(Server.java:530)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
INFO | jvm 1 | 2019/06/06 10:37:27 | at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
INFO | jvm 1 | 2019/06/06 10:37:27 | at java.base/java.lang.Thread.run(Unknown Source)
INFO | jvm 1 | 2019/06/06 10:37:27 | Caused by: com.inductiveautomation.ignition.gateway.auth.web.strategy.WebAuthStrategyException: Unable to exchange the auth code for the token
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.web.strategy.oidc.OIDCWebAuthStrategy.parseWebAuthResponse(OIDCWebAuthStrategy.java:122)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.web.strategy.oidc.OIDCWebAuthStrategy.parseWebAuthResponse(OIDCWebAuthStrategy.java:35)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapter.parseAttributes(IdpAdapter.java:128)
INFO | jvm 1 | 2019/06/06 10:37:27 | ... 43 common frames omitted
INFO | jvm 1 | 2019/06/06 10:37:27 | Caused by: com.inductiveautomation.ignition.gateway.auth.oidc.client.service.OIDCClientServiceException: Problem getting the token
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.oidc.client.service.IgnitionOIDCClientService.getToken(IgnitionOIDCClientService.java:86)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.web.strategy.oidc.OIDCWebAuthStrategy.parseWebAuthResponse(OIDCWebAuthStrategy.java:120)
INFO | jvm 1 | 2019/06/06 10:37:27 | ... 45 common frames omitted
INFO | jvm 1 | 2019/06/06 10:37:27 | Caused by: com.inductiveautomation.ignition.gateway.auth.oidc.provider.service.OIDCProviderServiceException: Unable to generate token
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.oidc.provider.service.OIDCProviderService.getToken(OIDCProviderService.java:199)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.oidc.client.service.IgnitionOIDCClientService.getToken(IgnitionOIDCClientService.java:84)
INFO | jvm 1 | 2019/06/06 10:37:27 | ... 46 common frames omitted
INFO | jvm 1 | 2019/06/06 10:37:27 | Caused by: java.lang.IllegalStateException: User source profile '6' missing user with username 'moranaXXXX
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.oidc.provider.service.OIDCProviderService.getUser(OIDCProviderService.java:213)
INFO | jvm 1 | 2019/06/06 10:37:27 | at com.inductiveautomation.ignition.gateway.auth.oidc.provider.service.OIDCProviderService.getToken(OIDCProviderService.java:188)
INFO | jvm 1 | 2019/06/06 10:37:27 | ... 47 common frames omitted

I set an "Igniton IdP" whose User Source is A. A has a Failover Source B. Then I set a second "Ignition IdP" to user source B.
I would expect that if I fail the login towards the IdP A, the system tries to login towards IdP B.
Am I wrong?

Thanks,
Andrea

Hi @andrea.morando -

You do not need more than one IdP. The failover will happen at the user source layer. So something like this should work:

Perspective Project -> Ignition IdP
Ignition IdP -> User Source A
User Source A fails over to User Source B

Did you configure a hard failover or a soft failover? Double check that user moranaXXXX exists in both user sources with the expected roles.

Hi,
The project is linked to an Ignition Idp (-> User Source A), but it is unable to handle login response and the message is the same as the one posted.
This the result of the Test Login:

Best,
Andrea

Hi @andrea.morando -

I am assuming this means that the same exception is in the gateway logs? Something like User source profile ‘6’ missing user with username 'moranaXXXX?

If so, have you double checked that both user source A and the failover user source B have a user with username moranaXXXX?

Hi @andrea.morando -

I have filed an internal ticket for this issue. It seems that soft failover does not work if the user exists in the secondary source and not in the primary. We’ll let you know when the issue is resolved.

Thank you, that’s really my current situation.
The user moranaXXXX exists in the secondary source and not in the primary.

This issue was fixed in the 8.0.3 Nightly build that was uploaded today (7/5).