Ignition partial Crash - OPC UA

Ignition
1

Good morning to everyone,
We are experiencing some issues with Ignition.
The behavior is this:
Ignition work propelly for about 3-4 hour, after that the tag browser and all the tag became unavailable.
The OPC (Tani OPC Server) still work and can read/write the tag.
The only code we had in the log is this for the moment:
UascClientMessageHandler (org.eclipse.milo.opcua.stack.client.transport.uasc.UascClientMessageHandler)
No pending request with requestId=XXXXXX for ServiceFault

To solve the problem we have to disable and enable the Opc connection.

We are using Ignition 8.0.16 connected with Tani OPC-UA 1.25.15 (the latest version), PLC Siemens 1500.

We have another plant with the same configuration (Tani OPC) but with Rockwell and we don't experience any issues.

For sure is not the Tani OPC, we optimize the comunication of the tag but no matter how we speed up the performance, it seem that after sometime the problem come back.

we try this to solve the problem:
Give more memory for the heap
Optimize the scan class
One time we found in quarantine the trusted certificate and we allow it.

It seems that the tag service became unavailable, the issues is really similar to this:

Thank you for the help

2

If the tag browser is not working then this is unlikely related to the OPC message in your logs.

On the other hand, if the OPC browser stop showing tags for that server then it might be related.

Which is it? This may require you to call into support while it’s happening to show them.

3

Thank you for the fast reply! :slight_smile:
I asked to my co-worker:
In the tag browser we see the tag and if we try to change something ( a bool for example) sometimes it work but really slow, but we can try to take some screen shot when it happen again.
How I can call the support? By opening a ticket or by other mean?
Thank you for the help

4

Ok it just happened now.
We can see the tag, we can change them, in the designer work but on the HMI pannel no.
We have this new message now:

UascClientMessageHandler 16Sep2020 17:45:47 No pending request with requestId=23511 for ServiceFault
ClockDriftDetector 16Sep2020 17:43:58 Clock drift, degraded performance, or pause-the-world detected. Max allowed deviation=1000ms, actual deviation=1146ms
UascClientMessageHandler 16Sep2020 17:43:51 No pending request with requestId=22975 for ServiceFault
5

This message means some kind of failed response is arriving from the server after Ignition's client has already given up and deemed it timed out.

Maybe you can get a Wireshark capture. The connection will need to have security disabled to make sense of it, though.

6

Wireshark is now installed on the server.
What should I look for?
I don’t have a lot of experience with this software.

7

Take a capture while this issue is occurring and then upload it somewhere and we will look at it.

8

Hello,
My co-worker took the data for me.
Here the Loopback data.

Pacchetti_Loopback.rar (2.7 MB)

Do You need the HMI network too?

9

There are 2 ServiceFault responses, the first is in response to a request that was made before the capture started. The second is a WriteRequest that the server is failing to respond to within 60 seconds.

I’m not sure either of these would explain the stranger issues you mentioned seeing. I think you’ll have to call into support and let them take a look and get some logs.

10

I open a ticket, but untill now no reply.
Is it normal or I made a mistake?

11

Unfortunately an email ticket can take as long as 2 weeks to get a response right now. If it’s urgent you can call support instead and you’ll get helped immediately. The phone queue takes priority over the email queue.

12

@mfontana, we have the same configuration and it seems we have the same issue. Did you solved the issue? How? Thx

13

@mfontana, @Kevin.Herron
I have the same setup (Tani OPC and Siemens 1500) and seeing the same issue. Also looking for a solution.
Any ideas?
Thanks.

14

Get in touch with TANI support. The write responses sometimes take a long time to come back from the server, there’s nothing you can do on the Ignition side about it.

15

Hello to everyone,
we are still waiting a reply from the support.
We sent a way to replicare the issue in Ignition.

16

Prevent read failures in Siemens driver when consecutive reads of same tags happen too quickly.

There is a fix in the 8.0.17, I just notice while I was reading the new release note.
We will try it but no comunication from Inductive automation...

17

I am trying 8.1.0 and issue is still there. I will try with Tani V2 soon, I will let you know if something magic happen

18

I already try Tani 2.0, Kepware, Softnet…
The issue is still there.

When you ask a tag with the same address it will create the issue.
For example in the system.opc.read(), you ask 2 time the same address, it will give you error, Bad Quality.

We already try 8.1.0, same thing unfortunatly.
We are in concact with EFA automation (italian providar) but everything is really slow.

I will try the 8.0.17 and see if it change something with external OPC, but the bug is in Ignition, no matter what we do (7.9.16 did not have this issues).

19

We have problem with read function, not write… We sent the picture to the support.

Error 1
Error 11539Ă—593 131 KB
Error 2
Error 21539Ă—593 131 KB
Working
Working1539Ă—593 131 KB

20

To be clear, this is the issue we found.

If we ask to Ingition on the Server to ask the Tag from the function System.opc.read, we read without any issue, if we do the same on another PC, we have the issue.
I don’t think that the TANI is the issue.
We disable all the firewall.
Here the screenshot from today.

Client Not Working
Client Not Working1600Ă—165 57.5 KB
Server Working
Server Working1920Ă—1080 154 KB