# Slowed down server **URL:** \ **Category:** Ignition\ **Created:** [September 10, 2018, 8:34am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602 "2018-09-10T08:34:42Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 10, 2018, 8:34am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/1 "2018-09-10T08:34:42Z")
HI. One of my previous customer has ignition (dont remember whih version), where they have to restart the server with ignition on because the server it self becomes slow (vm warer:ed server. They have alotted 8GB of ram and with weeks of logging the ignition never goes above 900MB of ram usage. When the server slows down, they start to get production stop. I know that ignition has the “recipy” part of the plc work data, the flow is: PLC sends a type to ignition, this tag has an on trigger with script, that looks thru entered recipes in ingition. Ignition takes the data for this part type, from a local MSSQL Datavase and put it into designated datablock in the plc, then sets a flag recipe is loaded and done. PLC does work rince repeat. ATM i dont have access to the project but wonder what i should start to look at, it does not sound like memory leak since the memory usage is stable over weeks, except the server slows down. I am not sure, if a scheduler to reboot the gateway would solve this (bandaid for a broken leg), or what best approach would be. Any ideas? ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 10, 2018, 1:31pm UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/2 "2018-09-10T13:31:07Z")
The most common problem I’ve seen for Ignition lagginess over time is due to the garbage collector. Switch to [G1GC](http://forum.inductiveautomation.com/search?q=G1GC). ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 11, 2018, 5:38am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/3 "2018-09-11T05:38:22Z")
Hi @pturmel. When i get access to the server at the customers place, i will start by changing that flag for the gateway as a first fix. When using gateway scripts, am i correct to assume that i dont really have to dispose objects and variables on my own, since java should GC is when needed? Also, i was thinking to change init memory to 512mb ram and max to 2048, but do not think this will change anything since they allow the server to have 8GB ram and ignition uses just below 900 MB. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 11, 2018, 12:31pm UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/4 "2018-09-11T12:31:00Z")
> [@StefanGronberg](#): > > When using gateway scripts, am i correct to assume that i dont really have to dispose objects and variables on my own, since java should GC is when needed? No, you don't have to dispose of objects. However, you must take care that stale versions of your jython objects don't get stuck in memory by references elsewhere. See this comment for some general advice: > [@Some Project Resources not publishing](http://forum.inductiveautomation.com/t/some-project-resources-not-publishing/16888/3): > > This may be a side-effect of the fact that Ignition doesn’t/can’t track the usage and retention of python code objects throughout the gateway. This means that when your script modules have interlocking dependencies, updating one script won’t impact the code objects that other scripts have imported. You may need to make a trivial change in all of your scripts and save them all together to get it to “take”. You may also need to disable then re-enable the entire project. Consider renaming the o… > [@StefanGronberg](#): > > Also, i was thinking to change init memory to 512mb ram and max to 2048, but do not think this will change anything since they allow the server to have 8GB ram and ignition uses just below 900 MB. In my experience, Ignition settles to lower RAM usage with G1GC than with the CMS default. But wherever it settles (peak of the sawtooth in a usage trend) should be encompassed by your initmemory. I'm not sure there's ever value in having init memory != max memory -- just let Ignition grab the memory you want it have up front and keep it out of the grubby hands of competing processes. ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 18, 2018, 6:08am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/5 "2018-09-18T06:08:25Z")
Hi, i will soon get access to the server, i was woundering, if there is some debug settings i perhaps should enable to trace possible slow downs happening, afaik the logging is at default setting for ignition. But i am not sure if more logging would catch what ever causing the slow down because i do not know if its ignition, java or the server is self that causes the slow down. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [September 18, 2018, 2:39pm UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/6 "2018-09-18T14:39:00Z")
Some of the settings I recommend for G1GC include turning on logging of the garbage collector itself. This is very valueable for identifying pause-the-world events. ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 19, 2018, 8:21am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/7 "2018-09-19T08:21:02Z")
I got access to the server today, and added wrapper.java.additional.5=-XX:+UseG1GC The service then failed to start. > STATUS | wrapper | 2018/09/19 10:07:13 | Launching a JVM... > INFO | jvm 1 | 2018/09/19 10:07:13 | Error: Could not create the Java Virtual Machine. > INFO | jvm 1 | 2018/09/19 10:07:13 | Error: A fatal exception has occurred. Program will exit. > INFO | jvm 1 | 2018/09/19 10:07:13 | Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed > ERROR | wrapper | 2018/09/19 10:07:13 | JVM exited while loading the application. > STATUS | wrapper | 2018/09/19 10:07:18 | Reloading Wrapper configuration... > STATUS | wrapper | 2018/09/19 10:07:18 | Launching a JVM... > INFO | jvm 2 | 2018/09/19 10:07:18 | Error: Could not create the Java Virtual Machine. > INFO | jvm 2 | 2018/09/19 10:07:18 | Error: A fatal exception has occurred. Program will exit. > INFO | jvm 2 | 2018/09/19 10:07:18 | Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed > ERROR | wrapper | 2018/09/19 10:07:18 | JVM exited while loading the application. > STATUS | wrapper | 2018/09/19 10:07:23 | Reloading Wrapper configuration... > STATUS | wrapper | 2018/09/19 10:07:23 | Launching a JVM... > INFO | jvm 3 | 2018/09/19 10:07:24 | Error: Could not create the Java Virtual Machine. > INFO | jvm 3 | 2018/09/19 10:07:24 | Error: A fatal exception has occurred. Program will exit. > INFO | jvm 3 | 2018/09/19 10:07:24 | Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed > ERROR | wrapper | 2018/09/19 10:07:24 | JVM exited while loading the application. > STATUS | wrapper | 2018/09/19 10:07:28 | Reloading Wrapper configuration... > STATUS | wrapper | 2018/09/19 10:07:29 | Launching a JVM... > INFO | jvm 4 | 2018/09/19 10:07:29 | Error: Could not create the Java Virtual Machine. > INFO | jvm 4 | 2018/09/19 10:07:29 | Error: A fatal exception has occurred. Program will exit. > INFO | jvm 4 | 2018/09/19 10:07:29 | Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed > ERROR | wrapper | 2018/09/19 10:07:29 | JVM exited while loading the application. > STATUS | wrapper | 2018/09/19 10:07:34 | Reloading Wrapper configuration... > STATUS | wrapper | 2018/09/19 10:07:34 | Launching a JVM... > INFO | jvm 5 | 2018/09/19 10:07:34 | Error: Could not create the Java Virtual Machine. > INFO | jvm 5 | 2018/09/19 10:07:34 | Error: A fatal exception has occurred. Program will exit. > INFO | jvm 5 | 2018/09/19 10:07:34 | Conflicting collector combinations in option list; please refer to the release notes for the combinations allowed > ERROR | wrapper | 2018/09/19 10:07:34 | JVM exited while loading the application. > FATAL | wrapper | 2018/09/19 10:07:34 | There were 5 failed launches in a row, each lasting less than 300 seconds. Giving up. > FATAL | wrapper | 2018/09/19 10:07:34 | There may be a configuration problem: please check the logs. > STATUS | wrapper | 2018/09/19 10:07:34 | \<-- Wrapper Stopped When i removed the G1GC line (commented out) the service started once again. They are using Ignition 7.8.2. Ignition.conf attached. [ignition.zip](https://us1.discourse-cdn.com/inductiveautomation/original/2X/9/975d88fb7ea56fb52a68e5d23db548407d57d1ed.zip) (2.8 KB) ---
**Author:** ![mt1](https://avatars.discourse-cdn.com/v4/letter/m/53a042/32.png) [@mt1](https://forum.inductiveautomation.com/u/mt1)\ **Post date:** [September 19, 2018, 8:39am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/8 "2018-09-19T08:39:10Z")
Have you commented/removed the line: `wrapper.java.additional.1=-XX:+UseConcMarkSweepGC` after adding `...+UseG1GC` more over I see that there are two lines with `.5` This is my part of .conf. The commented line are what i had. ```auto # Java Additional Parameters # wrapper.java.additional.1=-XX:+UseConcMarkSweepGC # wrapper.java.additional.2=-XX:+CMSClassUnloadingEnabled wrapper.java.additional.1=-XX:+UseG1GC wrapper.java.additional.2=-XX:MaxGCPauseMillis=100 wrapper.java.additional.3=-Ddata.dir=data wrapper.java.additional.4=-Dorg.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES=false #wrapper.java.additional.5=-Xdebug #wrapper.java.additional.6=-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000 # Initial Java Heap Size (in MB) wrapper.java.initmemory=1024 # Maximum Java Heap Size (in MB) wrapper.java.maxmemory=4096 ``` ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 19, 2018, 8:44am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/9 "2018-09-19T08:44:44Z")
Hi. No the .1 was left uncommented, and the second .5 is commented out. I will test removing/replacing the .1 with G1GC locally and see if it works for me. ---
**Author:** ![mt1](https://avatars.discourse-cdn.com/v4/letter/m/53a042/32.png) [@mt1](https://forum.inductiveautomation.com/u/mt1)\ **Post date:** [September 21, 2018, 9:34am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/10 "2018-09-21T09:34:29Z")
Hi. Now Is it working? ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [September 21, 2018, 9:56am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/11 "2018-09-21T09:56:07Z")
Not sure yet, i asked them to check the server for signs of slowdown when they have their weekly reboot of it, but before the slow down took up to a week before it was shown. ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [October 8, 2018, 6:43am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/12 "2018-10-08T06:43:47Z")
The server has now been running since 2018-09-29, and i just checked in on it, seems to run fine, altho since i changed the GC setting, the ram usage as slowly creeped up to around 2GB of ram usage over the week. Not sure if it will stop around there or it will keep go up. ---
**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\ **Post date:** [October 8, 2018, 1:21pm UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/13 "2018-10-08T13:21:07Z")
Sounds like a memory leak. Switching to G1GC has probably just lengthened the time from boot to breakage. Unless it is some caching activity that will stabilize. If it keeps going up, you’ll want to get support involved. ---
**Author:** ![StefanGronberg](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/stefangronberg/32/6288_2.png) [@StefanGronberg](https://forum.inductiveautomation.com/u/StefanGronberg)\ **Post date:** [October 17, 2018, 5:15am UTC](https://forum.inductiveautomation.com/t/slowed-down-server/19602/14 "2018-10-17T05:15:58Z")
The server has been running nonstop since 2018-09-29, the memory has not risen anymore since last check. So the new G1GC seems to have fixed what the old one did not do, or does it much better.