Gateway not Running after Upgrade to 8.0.16

Hello,

I have recently upgraded the gateway from 8.0.14 to 8.0.16. however, the gateway is not running (frozen for more than an hour). I restarted the ignition service but no luck. I also tried the 8.0.16 installer again and got the same error. Kindly advise.

You’ll probably need to look for relevant errors in your wrapper.log file.

Hi pturmel,

Here is a screenshot of the wrapper.log.

Can you upload the entire wrapper log somewhere we can take a look at it?

the log file is to big to upload, could you advise an alternative?
Thanks

zip it first and then try again?

If that doesn’t work let me know and I’ll DM you a Dropbox link to upload to.

1 Like

zip files cannot be uploaded

Thank you, that would be better.

Regards,

Ok, there appears to be a stack overflow error originating from one of the Cirrus Link modules:

| E [C.BasicExecutionEngine        ] [18:29:21]: One-shot task com.inductiveautomation.ignition.gateway.IgnitionGateway$$Lambda$351/0x00000001003efc40@65bac5f9 threw uncaught exception. module-name=MQTT Engine
| java.lang.StackOverflowError: null
| 	at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1965)
| 	at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3642)
| 	at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3618)
| 	at com.google.common.cache.LocalCache$Segment.storeLoadedValue(LocalCache.java:3306)
| 	at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2458)
| 	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2417)
| 	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)
| 	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)
| 	at com.google.common.cache.LocalCache.get(LocalCache.java:4147)
| 	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)
| 	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)
| 	at com.inductiveautomation.ignition.common.tags.paths.parser.TagPathParser.parse(TagPathParser.java:149)
| 	at com.inductiveautomation.ignition.common.tags.paths.parser.TagPathParser.parseSafe(TagPathParser.java:137)
| 	at com.cirruslink.mqtt.common.util.TagUtils.getTagConfigsRecursive(TagUtils.java:617)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$1(TagUtils.java:633)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$2(TagUtils.java:626)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.getTagConfigsRecursive(TagUtils.java:621)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$1(TagUtils.java:633)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$2(TagUtils.java:626)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.getTagConfigsRecursive(TagUtils.java:621)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$1(TagUtils.java:633)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$2(TagUtils.java:626)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.getTagConfigsRecursive(TagUtils.java:621)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$1(TagUtils.java:633)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$2(TagUtils.java:626)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.getTagConfigsRecursive(TagUtils.java:621)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$1(TagUtils.java:633)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)
| 	at com.cirruslink.mqtt.common.util.TagUtils.lambda$getTagConfigsRecursive$2(TagUtils.java:626)
| 	at java.base/java.util.ArrayList.forEach(Unknown Source)

As a workaround you can try removing the MQTT modules from the $IGNITION/user-lib/modules folder and then start the gateway again. If that works you can reinstall them and see if things are working. You might also check to see if there are newer MQTT modules available to install.

@wes0johnson may be able to comment on whether they’ve seen this before.

1 Like

This isn’t something I’ve seen before. I’d be interested to know the MQTT Engine version. How many tags are present in MQTT Engine? Also, are there UDTs being used in MQTT Engine?

1 Like

Thanks a lot, that worked.

no UDTs were used. the total number of tags in the engine did not exceed 500.

Now the same problem happened but the Groov EPIC device. how can I remove the MQTT modules? the Gateway is not starting!

If you have a shell on the EPIC then you can navigate to $IGNITION_INSTALL/user-lib/modules and delete the Cirrus Link modules.

Not sure exactly where the user-lib folder ends up on those devices.

Thanks for your reply, how can I find out if there is a shell on the EPIC device?

I don’t know anything more about these. Do you have any way to SSH into it and get a command line or run commands somehow?

SSH requires additional license which we do not have!

Sorry I’m not sure how to further help you. Honestly I don’t even know which company’s support department you should contact at this point. I imagine ours would just ask you the same thing about SSH access…

now it is working. I went to groov Manage, Ignition and then disabled it. reenabled it and it fine now.

Thanks Kevin

Hi melwadeya (and anyone else viewing this reply)-

SSH (shell access) licenses for groov EPIC and groov RIO are available for no charge directly from Opto 22. You may place your no-charge order online from this page:
https://www.opto22.com/products/product-container/groov-lic-shell

Or simply start a chat or call Opto at 800-321-6786 to place the order.

Upon completion of the order for the free license, you’ll receive an activation key that you’ll activate to your EPIC or RIO via the manage.groov.com Portal. Download the license file, apply it to your controller or I/O, and you’ll be all set.

For anyone experiencing difficulty with Ignition Edge running on our groov EPIC or (future version of) groov RIO, our Product Support is free via online chat, email, or phone:
https://www.opto22.com (chat window bottom right)
support@opto22.com
800-321-6786

We’re are happy and eager to help. -Benson

1 Like