Unable to get IdP with name 'AD_internal_hybrid' from cache

Transistor · March 13, 2023, 11:03am

Version: 8.1.22.

We're picking through the status log errors and have made good progress in weeding out many of them. I've had five occurrances of Unable to get IdP with name 'AD_internal_hybrid' from cache over the past three days but by various projects. I think the IdP is working OK because login is working correctly on our Perspective applications.

IdP error log properties

and the log details - for what they're worth:

Log details

java.util.concurrent.ExecutionException: com.inductiveautomation.ignition.gateway.auth.idp.NotFoundException
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:531)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:492)
at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:83)
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:196)
at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2312)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapterManagerImpl.getIdpAdapter(IdpAdapterManagerImpl.java:704)
at java.base/java.util.Optional.flatMap(Unknown Source)
at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.startupInternal(WebAuthSessionImpl.java:121)
at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.mdc(WebAuthSessionImpl.java:102)
at com.inductiveautomation.ignition.gateway.auth.idp.WebAuthSessionImpl.startup(WebAuthSessionImpl.java:128)
at com.inductiveautomation.perspective.gateway.session.SecuredPerspectiveProjectSession.onStartupInternal(SecuredPerspectiveProjectSession.java:604)
at com.inductiveautomation.perspective.gateway.session.SessionKernelWrapper.onStartup(SessionKernelWrapper.java:145)
at com.inductiveautomation.ignition.common.lifecycle.AbstractLifecycle.startup(AbstractLifecycle.java:18)
at com.inductiveautomation.ignition.common.util.ExecutionQueue$FragileRunnableFuture.getValue(ExecutionQueue.java:286)
at com.inductiveautomation.ignition.common.util.ExecutionQueue$FragileRunnableFuture.getValue(ExecutionQueue.java:275)
at com.inductiveautomation.ignition.common.util.ExecutionQueue$FragileCompletableFuture.run(ExecutionQueue.java:266)
at com.inductiveautomation.ignition.common.util.TimedRunnable.run(TimedRunnable.java:21)
at com.inductiveautomation.ignition.common.util.ExecutionQueue$PollAndExecute.run(ExecutionQueue.java:239)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.inductiveautomation.ignition.gateway.auth.idp.NotFoundException: null
at java.base/java.util.Optional.orElseThrow(Unknown Source)
at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapterManagerImpl$IdpAdapterCacheLoader.load(IdpAdapterManagerImpl.java:153)
at com.inductiveautomation.ignition.gateway.auth.idp.IdpAdapterManagerImpl$IdpAdapterCacheLoader.load(IdpAdapterManagerImpl.java:147)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
... 21 common frames omitted

Any suggestions about what is causing this? Is it an Active Directory timeout external to Ignition?

PGriffith · March 13, 2023, 4:12pm

If you look in the internal database, specifically the IDP_ADAPTERS table, do you see a record with the called out name? There's not a lot of pieces that could fail to cause this error; it's basically: retrieve all the records, put them into a Map<String, IdpAdapter>, then retrieve by name. That retrieval by name is what's failing.

Transistor · March 13, 2023, 7:33pm

Thanks, Paul. Can you remind me how to access the internal database? (I think I looked into it a couple of years ago but have no idea how.)

PGriffith · March 13, 2023, 7:44pm

From any page in the status section, change the trailing url the "sys.internaldb". Or use the Konami code. Or use Kindling, or any other SQLite browser, on a GWBK.

Transistor · March 13, 2023, 8:00pm

Making progress ...

What next? (Thanks for the hand-holding.)

PGriffith · March 13, 2023, 8:28pm

Hmm, the binary data should just be a JSON strings, I'm not sure how best to coerce it out. Probably have to Google a SQLite conversion function so that the IDB page doesn't think it's binary data anymore.

Transistor · March 13, 2023, 9:31pm

Ha!
SELECT idp_adapters_id, json_extract(config, '$') FROM idp_adapters

returns four items. The one I think is relevant is #3:

IdP_Adapters_Id JSON

{
	"name": "IdP_AD_internal_hybrid",
	"description": "Automatically generated Ignition Identity Provider which uses the User Source Profile named \"AD internal hybrid\".",
	"authStrategy": {
		"type": "ignition",
		"config": {
			"version": 1,
			"userSource": "AD internal hybrid",
			"authMethods": ["username-and-password"],
			"defaultAuthMethod": "username-and-password",
			"badgeSecret": false,
			"sessionInactivityTimeout": 30,
			"sessionExp": 0,
			"rememberMeExp": 0
		}
	},
	"userAttributeMapper": {
		"id": {
			"type": "direct",
			"config": {
				"attributePath": "sub"
			}
		},
		"userName": {
			"type": "direct",
			"config": {
				"attributePath": "preferred_username"
			}
		},
		"firstName": {
			"type": "direct",
			"config": {
				"attributePath": "given_name"
			}
		},
		"lastName": {
			"type": "direct",
			"config": {
				"attributePath": "family_name"
			}
		},
		"email": {
			"type": "direct",
			"config": {
				"attributePath": "email"
			}
		},
		"roles": {
			"type": "direct",
			"config": {
				"attributePath": "roles"
			}
		}
	},
	"directSecurityLevelPolicies": {
		"id": {},
		"username": {}
	},
	"derivedSecurityLevelPolicies": []
}

I don't see any problem there? Anyone?

PGriffith · March 13, 2023, 9:39pm

Well, that's not the same as AD_internal_hybrid, which is what's apparently being requested...

I don't see anything suggesting there's any automatic addition of a prefix or anything.

Transistor · March 13, 2023, 9:58pm

Well spotted. The list is:

temp
IdP_AD_internal_hybrid
opcua_module
default

Meanwhile Config | SECURITY | Identity Providers shows,

PGriffith · March 13, 2023, 10:11pm

Check the project's properties? Does it have a stale reference? Is the project IdP configured in Project Properties -> General, or Project Properties -> Perspective -> General? The latter path is deprecated for some legacy reasons, and once you make a change in the former, the latter should be automatically hidden.

Transistor · March 13, 2023, 10:31pm

The project IdP is configured in both Project Properties | General and in Project Properties | Perspective | General. Changing the former to or default and applying the change didn't hide the latter.

I'll investigate more tomorrow. Thank you again for the follow through.

Transistor · March 14, 2023, 6:03pm

Duh! And save the project to update the gateway.
I think we may have solved the problem.