Ignition Bouncing between 40%-70% and sometimes unresponsive

Ignition
1

We currently have it setup connected to a PLC and running about 30 000 tags. I noticed that the CPU on the ignition gateway is running between 40% - 67%. Also sometimes when a button is pressed it does not go through and must be pressed multiple times until it finally goes through. The error on the logs says “write time out”. Does anyone have any ideas what it could be?

Here is the logs:

Logger Time Message
ProjectRunner 09Jun2017 14:55:13 Tried to perform RPC operation on an unrecognized session. May indicate that the previous session was lost. Creating RPC listener for new session id.
ProjectRunner 09Jun2017 14:46:31 Tried to perform RPC operation on an unrecognized session. May indicate that the previous session was lost. Creating RPC listener for new session id.
Route 09Jun2017 14:24:45 Error handling route.
…com.inductiveautomation.ignition.gateway 09Jun2017 14:17:20 Starting up client project monitor. project=Paramalat_Winnipeg, uuid=8f605af6-7bde-6f75-428b-506455d8f7dd, editCount=3790, scope=4, version=Published
SAP_HTST_Que1_GI1 09Jun2017 14:07:26 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 13:07:23 CDT 2017]].
SAP_HTST_Que2_GI1 09Jun2017 14:07:26 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 13:07:23 CDT 2017]].
SAP_HTST_Que3_GI1 09Jun2017 14:07:26 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 13:07:23 CDT 2017]].
Provider 09Jun2017 14:06:28 [[default]Parmalat/HTST1/Production_Control/HTST1/Water_To_Prod_PB] Error writing to OPC address [[Ignition OPC-UA Server]ns=1;s=[Parmalat_Main]HTST1Prod.WP_Button]. Result code: [Bad_Timeout] The operation timed out.
WriteContext 09Jun2017 14:06:28 Write timed out.
SynchronousWrite 09Jun2017 14:06:28 WriteItem did not receive value before timeout: HTST1Prod.WP_Button
Provider 09Jun2017 12:07:11 [[default]Parmalat/Engineering_Values/Past_AA_CIP_EV/EV 21] Error writing to OPC address [[Ignition OPC-UA Server]ns=1;s=[Parmalat_Main]PastAEV[21]]. Result code: [Bad_Timeout] The operation timed out.
WriteContext 09Jun2017 12:07:11 Write timed out.
SynchronousWrite 09Jun2017 12:07:11 WriteItem did not receive value before timeout: PastAEV[21]
Report_ReportHTST1 09Jun2017 11:59:11 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:59:08 CDT 2017]].
Report_ReportFracCartonFiller 09Jun2017 11:59:11 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:59:08 CDT 2017]].
Production_HTST2_Que1_HTSTParameters_PLC 09Jun2017 11:57:01 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Production_HTST2_Que3_HTSTParameters_PLC 09Jun2017 11:57:01 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Production_HTST2_Que2_HTSTParameters_PLC 09Jun2017 11:57:01 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportRecoveryDarkSupplyLineBlend 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportCrtnAFiller 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportRecoverPastTankLine 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportJugFiller 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportT806 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportT103 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportT204 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportRecoveryDarkToBlendingLine 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportRecoveryLFHFSupplyLineBlend 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportSkimBlendLine 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportHTST2 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportT204Line 09Jun2017 11:57:00 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
Report_ReportHTST2Line 09Jun2017 11:56:59 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
Report_ReportBlendHTSTLines 09Jun2017 11:56:59 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
Report_ReportT805 09Jun2017 11:56:59 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
SAP_HTST_Que1_GI1 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_HTST_Que3_GI1 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_HTST_Que2_GI1 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Milk_Receipt 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI14 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI13 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI10 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI8 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI12 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI9 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI1 09Jun2017 11:56:58 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:54 CDT 2017]].
SAP_Milk_Receipt 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_OrderConfirmationHeaderFull 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI15 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI5 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI6 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI2 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GI3 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_Mixing_GR1 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[0, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_HTST_Que1_GI1 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_HTST_Que3_GI1 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
SAP_HTST_Que2_GI1 09Jun2017 11:56:57 Trigger did not evaluate - source item currently has bad quality [[false, Unknown, Fri Jun 09 10:56:53 CDT 2017]].
…com.inductiveautomation.ignition.gateway 09Jun2017 11:18:33 Starting up client project monitor. project=Paramalat_Winnipeg, uuid=8f605af6-7bde-6f75-428b-506455d8f7dd, editCount=3790, scope=4, version=Published
…com.inductiveautomation.ignition.gateway 09Jun2017 11:17:28 Starting up client project monitor. project=Paramalat_Winnipeg, uuid=8f605af6-7bde-6f75-428b-506455d8f7dd, editCount=3790, scope=4, version=Published

2

I started seeing “Tried to perform RPC operation on an unrecognized session” warnings too but I haven’t seen any performance issues …yet…
I did notice that CPU usage spikes are much higher when I am viewing the Logs with a filter, and it spikes really high when the filter starts ‘jumping’ out of control (for example if you filter with two or more space delimited entries). I don’t see a correlation yet. I’ll be watching this topic with interest.

1 Like
3

I have the same issue but no memory, cpu or thread spike. How did you fix this? nothing looks wrong but designer takes an incredibly long time to open views.

4

Signing everyone out of the host machine and restarting the host machine fixed this problem for me.

Not sure what caused it but happy that it is back to normal.

1 Like
5

Don't ever run a designer on your host machine if it is easily overloaded (or ever at all, IMNSHO). If you need to do maintenance via RDP, log in, get it done, and log right back out. Don't leave a user session running.

3 Likes
6

Hey turns out my issue was PLC and DB issues. Some how a null was sent in from the DB to the PLC through ignition at some point which caused everything to crash and the only way to fix this is to reboot the server. Sadly I never found a solution more then never allowing a null. If for some reason I need to send anything related to a null. I would just send a 0. Issue hasnt happened since. Hope that helps.