Hey all,
I am having an issue with a client site. They recently updated their ISP connection and the OPCUA connection they had to the Ignition server no longer connects. I am getting a strange error from the UascServerAsymmetricHandler. Just an UnexpectedError response from the OPCUA Client trying to connect. I did a search but haven’t come up with anything. The OPCUA server is accessible. I was able to connect on one of the other servers on this VLAN and pull data. The perspective HMI is functioning correctly. I checked for rejected certs with no luck there either.
Any help is appreciated. Please see the screenshot.
Thanks,
Kyle
"OutOfMemoryError"
High speed networking uses a particular memory type within the JVM for speed, and you've hit its limit.
I'll bet @Kevin.Herron knows what to tweak....
Hmm, shouldn’t need to tweak anything… I wouldn’t expect to see this unless you’ve actually just run out of direct memory (Netty isn’t the only thing using it, but probably the largest), or you have many many clients connected or something.
Logs and a Wireshark capture might help just to see if anything funny is going on.
If you need to recover then you’ll probably just have to restart the gateway now.
If this continues, adding -Dio.netty.noPreferDirect=true to ignition.conf might help shift the memory allocation to the heap where a heap dump or other standard tools can help more. Support can help you with this.
Typically, this site has a single OPC connection from an off-taker. It was functioning until they made changes to the ISP connection. Once the connection was re-established they started complaining about not being able to view the site via OPCUA. Would hitting this limit once cause the client to run into this issue continuously or is it that they are just trying to read data from too many points at once?
The heap size is still at 2GB. Would increasing my max heap size in the ignition.conf help with this if I am actually running out of memory. The gateway hardly shows a load and the memory never exceeds 1GB. I can provide a PCAP of the communication between the client trying to connect and the server and any logs. I haven’t tried a restart of the gateway since this is live, but we can schedule something with our client. I figured that may solve any issues, but want to figure out what I can do in the configuration to keep this from happening in the future.
The heap size isn’t relevant here except that in the absence of explicit configuration of the direct memory size, the amount of direct memory the JVM will use is equal to the max heap size. So it’s possible that whatever server this is running on doesn’t have an additional 2GB free memory and the JVM has been trying to allocate it.
Also possible there’s a memory leak, but that’s always surprising when it’s not being reported enough that a pattern emerges.
What version of Ignition is this?
1 Like
Ignition version is 8.1.23
The server itself has plenty of memory to allocate. When this gateway was configured the max heap size wasn’t set and so it stayed default at 2GB. We have 20GB free most of the time on the server and I plan to increase the heap size when we get permission to restart the gateway. I just wasn’t sure if the two were codependent and we were limiting the max direct memory of the JVM by having the heap size limited to 2GB.
You might as well get an upgrade scheduled as well - that will be the basis any troubleshooting, bug fixes, etc… needs to start from if this behavior continues and needs to be further diagnosed.
