Ignition OPC UA Server Faulted and wont reset

Ignition
1

My Ignition was working fine yesterday.
After shutting down and then starting up today to further create screens etc, I now get the following message on the Ignition OPC UA Server.

UaException: status=Bad_Timeout, message=timed out waiting for acknowledge
at org.eclipse.milo.opcua.stack.client.transport.uasc.UascClientAcknowledgeHandler.lambda$startHelloTimeout$0(UascClientAcknowledgeHandler.java:134)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
at java.base/java.lang.Thread.run(Unknown Source)

8.1.22 (b2022110109)
Azul Systems, Inc. 11.0.16.1

I have tried restarting the OPC UA server and still comes up the same and won't go out of faulted state.

2

If you check the logs around the time of a module or gateway restart do you see a message about a failed bind to the default OPC-UA port, 62541? It sounds like the OPC-UA Server may not be listening

3

Thank you for the response, no I cant find one. The first message I can find is about this:

UascServerHelloHandler 26Nov2022 08:32:35 [remote=/127.0.0.1:59470] Exception caught; sent ErrorMessage{error=StatusCode{name=Bad_UnexpectedError, value=0x80010000, quality=bad}, reason=Direct buffer memory}

java.lang.OutOfMemoryError: Direct buffer memory

at java.base/java.nio.Bits.reserveMemory(Unknown Source)

at java.base/java.nio.DirectByteBuffer.(Unknown Source)

at java.base/java.nio.ByteBuffer.allocateDirect(Unknown Source)

at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:649)

at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:624)

at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:203)

at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:173)

at io.netty.buffer.PoolArena.allocate(PoolArena.java:134)

at io.netty.buffer.PoolArena.allocate(PoolArena.java:126)

at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:396)

at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)

at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)

at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:140)

at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:120)

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:150)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)

at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)

at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

at java.base/java.lang.Thread.run(Unknown Source)

I managed to restart the gateway and it came good.
Why would it throw an error after working ok without having made any changes?

4

Has somebody been restarting the OPC UA module or is this an Ignition Edge gateway that has had the trial reset a number of times?

5

It is an Edge gateway trial that has been reset a few times. Is this common when doing this?

6

Some "known" issues where the OPC UA module can leak memory when restarted, and resetting the Edge trial also restarts the OPC UA module right now.

7

Ok no worries, thank you for the information. At least I know the cause now (and what to do to rectify in the mean time).

8

I am having this same issue. What is the resolution?

9

If it's actually the same issue then the immediate fix would be a gateway restart.

If not then a new topic with a lot more information, or a call to support, is in order.

10

Thanks for the quick reply. I have a faulted OPC UA Server with the same fault information as the OP with different software versions. It's a Windows 10 PC with Ignition Edge. Restarting the OPC UA module did not help, and restarting the gateway did not help either.

*UaException: status=Bad_Timeout, message=timed out waiting for acknowledge
at org.eclipse.milo.opcua.stack.client.transport.uasc.UascClientAcknowledgeHandler.lambda$startHelloTimeout$0(UascClientAcknowledgeHandler.java:134)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
at java.base/java.lang.Thread.run(Unknown Source)

8.1.25 (b2023021409)
Azul Systems, Inc. 11.0.17*

11

So, not the same issue. I suggest you follow Kevin's instructions.

12

I found when I had my issue - using the "Stop-ignition" Batch file and "Start-ignition" Batch files in the Ignition folder under program files to help. Otherwise stopping and starting the service "Ignition" in services.msc under "Run" of windows to help.

13

Based on your comment I must be missing something. It's the same error so I don't understand how it's not the same issue.

I did restart both the module and gateways per Kevin and mentioned above those things did not work.

14

I think we're both more fixated on the follow up error he posted from the logs.

Can you upload your logs here? As well as screenshots of your OPC UA server settings and the loopback OPC UA connection settings?

15

It is an error that is a very generic "something's wrong". The OP was instructed to look for (and found) the real error. Your error isn't going away with restart, so it definitely isn't the same real error. Kevin had specific instructions for that case.

16

UascServerHelloHandler

[remote=/127.0.0.1:53899] Exception caught; sent ErrorMessage{error=StatusCode{name=Bad_UnexpectedError, value=0x80010000, quality=bad}, reason=Direct buffer memory}

java.lang.OutOfMemoryError: Direct buffer memory

at java.base/java.nio.Bits.reserveMemory(Unknown Source)

at java.base/java.nio.DirectByteBuffer.(Unknown Source)

at java.base/java.nio.ByteBuffer.allocateDirect(Unknown Source)

at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:649)

at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:624)

at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:203)

at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:173)

at io.netty.buffer.PoolArena.allocate(PoolArena.java:134)

at io.netty.buffer.PoolArena.allocate(PoolArena.java:126)

at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:396)

at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)

at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)

at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:140)

at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:120)

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:150)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)

at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)

at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

at java.base/java.lang.Thread.run(Unknown Source)

The OPC UA server and loopback connection settings are all default.

17

Try giving Ignition a larger memory allotment (in ignition.conf).

18

I doubled the initial and max heap memory allotments, restarted ignition, and it doesn't seem to have changed anything.

19

Sounds like time to call support and have them look over your shoulder.

20

Seems like. Thanks for your time and help.