Ignition partial Crash - OPC UA

I've tried with 8.1.0 and Tani V2.0.2 and still have a problem. I've sent project files and WireShark captures to Tani who are looking into it, but no word back yet.

Thank you, I also call Tani support, letā€™s hope that someone find the solution.

Tani Support just reply now.
Agree that is an issue in Ignition.
Here the reply:

you can add an item multiple times to the Tani Server.
It seems to be an issue from ignition.
You can try it out with an other OPC UA Client.

Can you get a Wireshark capture of this issue occurring? Perhaps start a new thread as well, because this seems to be a different issue than the original.

We already sent the packege data of wireshark, open a ticket with EFA, open a ticket with Inductive automation, create a local test setup with 3 PLC, give my information to the support to connect here (Iā€™m still waiting by the way).

We gave maximum support, lose 4 days to setup and recreate different situation (Kepware, TANI, OPC ignition, Softnet) and different version (8.0.16, 8.1, 7.9.16) and find a way to replicate the issue and sent picture to help, and gave my Skype account to speed up the processā€¦
We keep having crash of the Tag Provider and we are losing production time (and money) because of Ignition.
The issue is still the same and we are gathering more information. When there is the UascClientMessageHandler , we have the issue.

I donā€™t want to be mean or uncostructive in anyway, just to let you know what we are going though, itā€™s getting really hard to workā€¦

I will try to get again a some package with wireshark, but now itā€™s at least 3 month we have this issue, if you want we have a setup here ready for Inductive Automation.

Did you send additional / different captures other than the one posted earlier in this thread when you were still asking about writes timing out?

Iā€™m not sure how much more clear it could be that the write timeout and subsequent log message from UascClientMessageHandler is due to the server not responding in time.

Hereā€™s what happens:

  1. Ignition sends a WriteRequest with a timeoutHint of 60,000ms
  2. 60,000ms has elapsed, Ignition decides this request has timed out, it stops bookkeeping this request.
  3. 60,000ms has elapsed, the TANI server realizes it has not serviced this request in time, it sends a ServiceFault response indicating Bad_Timeout
  4. The response loses the race by a few milliseconds; Ignition has already removed that request from its bookkeeping.
  5. You see a message about ā€œno pending request forā€¦ā€.

Are you telling me TANI support cannot see that this is an issue with the server taking too long to handle a WriteRequest?

Itā€™s very clear from your earlier capture:

How this relates to your issue with the reads I am unsure. Maybe it does, maybe it doesnā€™t.

We gave to the support 4 PC, 3 PLC, 3 different version of Ingition, 4 OPC.
We gave you all the Hints we found, and we are still waiting for help.
Here the last comunication with the support.

Posted on: 11/06/2020 09:07 AM

Goodmorning,
We did a lot of test in office with 3 CPU Siemens, 6 HMI, e 1 Server.
We think that we found what is causing the issue.

A way to cause the Service Fault is that ignition when there are more request for the same variable, let's assume that an operator keep clicking on a movement for adjusting precisly the position, Ignition will keep asking or writing the same value.

If the request is so fast that it enter the "read write" request at the same time, it will create the issue.

I will leave my skype contact
...

Another test we did is to create a driver by ourself and we don't have any issue.
This is what we did:

  • We create a script that undestand what tag need to be read and write,
  • Force to run with the opc at 100 ms and 250 ms with 40 000 Item
  • Use 3 different version of Ignition (7.9.16, 8.0.16, 8.1.0rc3)
  • use 4 different version of OPC (Ignition, Siemens, Kepware, TANI)

We gave to Induction Automation all our findings and support, and try to help as much as possible.

Kevin, we are not expert in OPC comunication, we are only trying to find the solution.
We call every support, EFA, Inductive Automation and TANI.
Untill now the only one that at least try to check the issue is TANI support.
As a customer, what else can I do?

Letā€™s assume that you are right.
The Timeout is 60 s
WriteRequest is at 2020-9-23 at 10:34
Error at 2020-9-23 at 10:35
Why the tag group crash?

Letā€™s say that itā€™s the Fault of the TANI or even our fault, no problem with that.
If Iā€™m wrong Iā€™m happy with that so I will learn something in the process.

But why if I have a single tag failure all the other tag in the same TagGroup are not available anymore? Why if I create a second TagGroup and switch it work and if I came back it doesnā€™t?

@mfontana, @Kevin.Herron

Iā€™ve been working with Werner at TANI for the past week and heā€™s been very helpful in working through these bugs. He discovered on Friday that the main problem was that the TANI OPC server was receiving a WriteReqeust from Ignition that contained the same tag multiple times. He hadnā€™t see that before and after a quick fix the TANI driver with Ignition now works great. Iā€™ve tested it over the past few days and itā€™s been working perfectly. Version 2.0.2 on the TANI download page has been updated with this fix if you want to try it out.

2 Likes

Thank for the update!
Unfortunatly itā€™s not working.
Mr. Werner is trying to contact @Kevin.Herron because now Ignition see the folder as a string in the OPC Browse, so now we can not import easely the Item of the OPC.
For the issue about the multiple of write request we canā€™t test it untill all the issue are solved.

Thank You for your support

Yes, Iā€™ve seen this too. I already had most of my tags created so was testing an existing project. Importing tags is a new problem. Iā€™ve found that once the tags are created, they work well. Hopefully TANI can fix this tag import issue quickly.

This is not an Ignition issue as I have seen the same problem in other OPC Clients (UA Expert).

Hi everybody,
Are the issues between Ignition and TANI solved, is everything stable and running?
We develop a project with Ignition and Siemens PLCā€™s and we are wondering if we should use TANI OPC Server or the Ignition driver for Siemens?

Thanks for answering,
Adrian

What models are they? The S7-1200 and S7-1500 now both have embedded OPC UA servers on them.

Hi,
We use both types 1200 and 1500.
But never used the embedded OPC. Weā€™ll test this way if is the best and stable (I think weā€™ll need license for OPC server for each PLC).
Should I understand that TANI server is not an option?

No no, TANI has been great in my experience. Everything is working now as far as I know, but somebody who is actually running it in production might be better to comment on it.

Hi David,

I have to choose the way to link Ignition with Siemens PLCā€™s.
Should I go with TANI OPC Server?
How are the things in your factory, is now stable and running?

Thank you for the response,

Adrian

Hi Adrian,

We are in the process of commissioning a site using an Siemens S7-1516-3 with Tani.
There were a few teething problems that are now sorted out.

Tani is setup to read over 15,000 tags with a poll interval of 800ms and it performs well. We tested the siemens on-board OPC server but could not get it to work efficiently. The Siemens documentation shows itā€™s limited to 5,000 tags so we couldnā€™t use it for our situation anyway. It might be ok with a higher model CPU.

We also briefly tested a few other third party OPC servers and none performed as well as Tani.

Caveat: We do have one remaining communications problem that I havenā€™t investigated yet. Every few hours Ignition flashes red for a second. I donā€™t know the cause of this. It could be Ignition, Tani, or the CPU itself. Iā€™ll post back when I work it out.

In summary, Iā€™m happy with the Tani software, it works well and Iā€™d recommend it. The issues we had initially were sorted out by Tani support very quickly.

Hi Adrian,
We were using Tani on 3 separate Plant.
But we run on too many issues.

Tani try really hard to solve it but unfortunatly our customers were too annoyed with the issues.
In the end we gave back the Tani and install on 2 of them the Siemens OPC and on one of them the Ignition OPC.

With the Siemens OPC no issue, on the Ignition we had some minor issue that we already know.

Usually our plant have 5 plc with on each of them have the between 2000 and 4000 tags.

We used the TANI both version 1 and version 2 on Ignition 8.0.17.
For our experience is better to not use the TANI and use the Siemens OPC