Slow response system.tag.write

Ignition
1

[color=#FF0000]This is the thing:
-I have created almost 20000 tags that I use in 3 different Siemens PLC.
-The network here is new made with optical fiber with a velocity of 1Gbps. The network is tested and I’m sure this is not the problem. The ignition project should be super-fast with this kind of network.
-I’ve created different Scan classes for the different PLC tags, so every PLC has their own scan class. Also there is some other scan classes for the tags that are used frequently, such as command tags, states tags.
-We put a max memory in the gateway of 4 GB and in each client, and a max memory of 2 GB (each client has a RAM memory of 4GB).
-We are working with Ignition v7.8.4 and a BD MySQL
-In the console Logs there is no error. There are Info Logs but NO warning or error logs.

The problem is that every time we try to write a value to the PLC, the time that take to that change to take effect is approximately 11 seconds and we think that is not acceptable, it should be so much faster.

Any hint on what may be causing this delay ? Thanks!!!
[/color]

2

[quote=“giulianacaula”]This is the thing:
-I have created almost 20000 tags that I use in 3 different Siemens PLC.
-The network here is new made with optical fiber with a velocity of 1Gbps. The network is tested and I’m sure this is not the problem. The ignition project should be super-fast with this kind of network[/quote]You haven’t provided any details of the processor models or scan class timing settings, so this is an educated guess: Your Siemens PLCs don’t have enough processing capacity for the number of tags at the pace you’ve set. Making the network itself faster won’t change the message processing capacity of the Siemens PLCs.
Look at the diagnostic pages for the drivers for your PLCs. That will show you how fast messages are being processed. Consider adding extra driver connections to each PLC, and spreading the tags evenly amongst them (as measured by the diagnostics). For some PLC brands, this will improve matters. (I don’t know if it will for Siemens.)

3

Thank you for the rapid response.

I’m sorry I didn’t give much details.
The PLCs are Siemens S7300.
I believe this is not the issue because the SCADA is running now and its been running for 7 years ago in a different Software (InTouch) and the speed is good. We are migrating the 3 SCADAs that the client has to Ignition and we don’t find anything strange that may be causing this problem.

The scan classes timing goeas from 500ms to 3000ms. There are 10 in total. Is there any poblem if I create a lot of scan classes ?

4

[quote=“giulianacaula”]The PLCs are Siemens S7300.
I believe this is not the issue because the SCADA is running now and its been running for 7 years ago in a different Software (InTouch) and the speed is good.[/quote]How many tags is InTouch reading and writing? And how fast? What does the network traffic look like for that pace? How many simultaneous connections is InTouch opening to each PLC? { Wireshark captures. }[quote=“giulianacaula”]We are migrating the 3 SCADAs that the client has to Ignition and we don’t find anything strange that may be causing this problem.[/quote]Well, the diagnostics pages are where you need to look. Please do. Use Wireshark to compare InTouch traffic to Ignition traffic, to see why Ignition is so much slower.[quote=“giulianacaula”]The scan classes timing goeas from 500ms to 3000ms. There are 10 in total. Is there any poblem if I create a lot of scan classes ?[/quote]No, the number of scan classes isn’t a problem. Some users have hundreds of scan classes. But remember that a scan class @ 500ms is the same load on the PLC as six of them at 3000ms. That’s why the diagnostic page is so important – it shows the requests handled per second.
Also: Did you create SQLtags for everything in the PLCs, including tags you aren’t actually using in your GUI? That will badly load down a PLC connection. Only make SQLtags for things you need to display or otherwise work with in your project.

5

I will do the Wireshark captures next week when I come back to the client’s to finish the work.

Now all I can do is look on the Diagnostics page. I’m not sure if this is the diagnostic you are refering to, but this is the diagnostic for 1 of the PLCs. I don’t know if the values are high, low, ok or what, so if you can give me a hand on this I will be very much appreciated!! Here it is:

Request Cycle
Request Throughput 17,5/sec
Min Request Duration 23,0ms
Max Request Duration 212,0ms
Avg Request Duration 56,5ms
Last Request Duration 61,0ms

Request Schedule
SubscriptionItem Count 2892
ScheduledRequest Count 91

@900ms
Request Count 1
Request Throughput 0,2/sec
Min Request Duration 4908,0ms
Max Request Duration 6033,0ms
Avg Request Duration 5132,7ms
Last Request Duration 5120,0ms
Avg Wire Time 51,6ms
Avg Queue Time 5081,1ms

@800ms
Request Count 1
Request Throughput 0,2/sec
Min Request Duration 4844,0ms
Max Request Duration 6038,0ms
Avg Request Duration 5141,5ms
Last Request Duration 5211,0ms
Avg Wire Time 55,5ms
Avg Queue Time 5086,0ms

@3000ms
Request Count 4
Request Throughput 0,8/sec
Min Request Duration 104,0ms
Max Request Duration 6863,0ms
Avg Request Duration 5148,3ms
Last Request Duration 5158,0ms
Avg Wire Time 56,5ms
Avg Queue Time 5091,8ms

@2100ms
Request Count 5
Request Throughput 0,9/sec
Min Request Duration 120,0ms
Max Request Duration 6694,0ms
Avg Request Duration 5178,3ms
Last Request Duration 5176,0ms
Avg Wire Time 54,4ms
Avg Queue Time 5123,9ms

@1300ms
Request Count 55
Request Throughput 10,6/sec
Min Request Duration 89,0ms
Max Request Duration 6243,0ms
Avg Request Duration 5281,2ms
Last Request Duration 5281,0ms
Avg Wire Time 60,7ms
Avg Queue Time 5226,4ms

@1200ms
Request Count 8
Request Throughput 1,5/sec
Min Request Duration 4204,0ms
Max Request Duration 6117,0ms
Avg Request Duration 5188,9ms
Last Request Duration 5286,0ms
Avg Wire Time 56,3ms
Avg Queue Time 5132,6ms

@1100ms
Request Count 7
Request Throughput 1,4/sec
Min Request Duration 4381,0ms
Max Request Duration 6108,0ms
Avg Request Duration 5156,4ms
Last Request Duration 5247,0ms
Avg Wire Time 51,7ms
Avg Queue Time 5104,7ms

@1000ms
Request Count 10
Request Throughput 1,9/sec
Min Request Duration 131,0ms
Max Request Duration 6436,0ms
Avg Request Duration 5238,5ms
Last Request Duration 5312,0ms
Avg Wire Time 59,5ms
Avg Queue Time 5179,0ms

The amount of tags that I told you earlier is the total of the 3 PLC together. Of all those tags, the only SQLtags I’m using are just 1 per machine (i.e. one per motor, one per valv, etc) to show its name and that’s all.

Thanks,

6

[quote=“giulianacaula”]I will do the Wireshark captures next week when I come back to the client’s to finish the work.[/quote]Ok. You’ll probably need this to figure out what you’ve done differently from InTouch.[quote=“giulianacaula”]Now all I can do is look on the Diagnostics page. I’m not sure if this is the diagnostic you are refering to, but this is the diagnostic for 1 of the PLCs. I don’t know if the values are high, low, ok or what, so if you can give me a hand on this I will be very much appreciated!![/quote]Yes, this is what I was referring to. These numbers are terrible. Your PLC is processing each request in 50 to 60 milliseconds. Which is why your overall request throughput is only 17.5 per second. Based on the numbers in your report, your scan classes need the PLC to handle 71.4 requests per second to work correctly.
I don’t have any experience with this many tags in a Siemens system, but I see a few possibilities:

  1. The S7-300 really needs 50-60 ms to process each request. You would have to reduce your tag count in Ignition. Don’t create SQLtags for PLC data you don’t need in Ignition!
  2. The S7-300 may take 50-60 ms per request, but could handle multiple requests at once. You would have to create more device connections in Ignition’s OPC server to make the PLC work on more than one at a time. Then spread the tags among them so each connection stays below 17/second. Based on the above, you’d need five connections to this PLC.
  3. Investigate the limits of the PDU size for your PLC. The default is 240 bytes. Supposedly, some processors can handle PDUs up to 960 bytes, with a loss of # of connections allowed. But this would allow more tag reads in a single request in your connection(s).
  4. Consider trying Kepware’s drivers for this PLC to see if they have better optimizations that Ignition (I doubt it).
    Or you could do all of the above.[quote=“giulianacaula”]The amount of tags that I told you earlier is the total of the 3 PLC together. Of all those tags, the only SQLtags I’m using are just 1 per machine (i.e. one per motor, one per valv, etc) to show its name and that’s all.[/quote]Then why do you have thousands of tags? I can’t emphasize enough how important it is to create SQLtags just for the PLC data you need.
7

Thank you so much, I’ll try to make the 5 connections you say and see if it make it better.
I’ll let you know the results!!!

Thanks again

8

I'm sorry for my ignorance, but how do you know the request per second that the PLC needs to handle to work correctly? I mean that "71.4 request per seconds", how do you know that specific number?

I apologize again but I just want to know to keep irt in mind for future projects.
If you could tell me, I'll be very much appreciated!!!
Thank you so much!!!

9

Let’s take the example of a single scan class to keep the math easy.

If you have a scan class running at 200ms, and the request count on that scan class is 10, then you need to handle 50 requests per second. (10 / .2 = 50) If you increased that scan class to 400ms and still have 10 requests, you only need to handle 25 requests per second (10 / .4 = 25)

You can do this calculation for each scan class and add them all together to see how many requests per second you are expecting your device to handle. If you’ve over loaded the device, your only three options are to find a way to get the device to handle more requests, reduce the total number of requests (i.e. reduce the number of tags) or slow down the rate of requests (increase the time for the scan class).

1 Like
10

Kathy, thanks for the answer. It help me a lot!

I’ve already tried to slow down the rate of requests and didn’t help much, and reduce the number of tags it’s not an option here.
I’m going to try to make multiple devices connections to see if it work this way.

Do you have some other tips? Thanks!

11

I just create multiple device connections to make the test.

I have 3 PLC
For PLC_01 I’ve created 9 connections and all of them connected just fine.
For PLC_02 I’ve created 5 connections and 4 of them are connected and 1 disconnected.
For PLC_03 I’ve created 12 connections and only 3 of them are connected.

There are too many device connections? How many device connection does Ignition can run at the same time?
I checked and all of the connection are created the same way, with the same parameters, hostname, etc.

This is the diagnostic for one of the device connections that is connected:

Request Cycle
Request Throughput 5,9/sec
Min Request Duration 26,0ms
Max Request Duration 115,0ms
Avg Request Duration 59,5ms
Last Request Duration 65,0ms

Request Schedule
SubscriptionItem Count 294
ScheduledRequest Count 9

@1500ms
Request Count 9
Request Throughput 5,9/sec
Min Request Duration 26,0ms
Max Request Duration 1091,0ms
Avg Request Duration 60,3ms
Last Request Duration 65,0ms
Avg Wire Time 59,5ms
Avg Queue Time 0,8ms

Is that better than before? or is still bad?

Any suggestions?
Thanks!!!

12

[quote=“giulianacaula”]I just create multiple device connections to make the test.

I have 3 PLC
For PLC_01 I’ve created 9 connections and all of them connected just fine.
For PLC_02 I’ve created 5 connections and 4 of them are connected and 1 disconnected.
For PLC_03 I’ve created 12 connections and only 3 of them are connected.

There are too many device connections? How many device connection does Ignition can run at the same time?
I checked and all of the connection are created the same way, with the same parameters, hostname, etc.[/quote]Ignition doesn’t impose a limit on the number of connections. The PLC does impose a limit, depending on the processor model. { I don’t use Siemens much, so you’ll have to do your own research on this part. } Other software connecting at the same time will have a negative impact on Ignition’s connections.[quote=“giulianacaula”]This is the diagnostic for one of the device connections that is connected:

Request Cycle
Request Throughput 5,9/sec
Min Request Duration 26,0ms
Max Request Duration 115,0ms
Avg Request Duration 59,5ms
Last Request Duration 65,0ms

Request Schedule
SubscriptionItem Count 294
ScheduledRequest Count 9

@1500ms
Request Count 9
Request Throughput 5,9/sec
Min Request Duration 26,0ms
Max Request Duration 1091,0ms
Avg Request Duration 60,3ms
Last Request Duration 65,0ms
Avg Wire Time 59,5ms
Avg Queue Time 0,8ms

Is that better than before? or is still bad?[/quote]This is much better. Notice that the average Queue Time is a fraction of a millisecond. That is very good. It needs to be less than the average request duration for the connection to keep up with the requests.[quote=“giulianacaula”]Any suggestions?
Thanks!!![/quote]Upgrade to faster PLC processors?

13

Yes, it was that the probem with the devices connections, the PLC.

Now all the connections are working and connected.
We’ve made a test and the time of response when I write a value to the PLC is no more than 4 seconds, and the test was made with the InTouch SCADA running, I think that when only the Ignition SCADA is running that time will decrese! I hope!!!

Thank you so much, you help me a LOT!!!

14

[quote=“giulianacaula”]Thank you so much, you help me a LOT!!![/quote]You’re welcome!