We seem to have a sizable memory leak in our production version. It is admittedly an old version 7.1.8 and I will try to upgrade when I can and if you know of these being explicitly fixed that will only help my case.
Anyway I have a Win2008 X64 server with 4 GB RAM hosting Ignition exclusively, the server is using the 64 bit wrapper with 3584 MB dedicated to max memory and wrapper timeout is the default 2 minutes. Today we started having database connection problems. The status page was indicating usage of 3400-something MB so I figured it was restart time. There were 17 clients connected at the time which is pretty standard load (we sometimes get to 30 or so).
After the restart, Ignition immediately spiked up to 3000 MB and crashed within 5 minutes and then did so a second time. I then started closing clients.
The only think of note I could find in the wrapper log was about 100000 lines of “Client subscription proxy got a tag change event after it was shut down.” This is pretty normal I think based on the fact that we use an external database driver and probably really are sending data after “shut down” since we frequently have subscriptions that we fork to our internal historian and some of those make it to the database for a couple of minutes after screens are closed.
We got this message about 50 times for several minutes before the actual failure.
INFO | jvm 1 | 2011/08/04 21:40:08 | WARN [BasicExecutionEngine ] [21:40:08,672]: Tried to unregister non existant unit [sqltags scanclasses [besql_db] default].
The final messages in the log were related to the wrapper terminating the instance:
ERROR | wrapper | 2011/08/04 21:40:42 | JVM appears hung: Timed out waiting for signal from JVM.
ERROR | wrapper | 2011/08/04 21:40:42 | JVM did not exit on request, terminatedSo one question is this a known behavior that has been fixed in 7.2? and should we really be hitting the 3.5 GB limit within minutes of startup? Should I be doing something differently to mitigate this problem? (other than upgrade which I will go to 7.2.6 probably on Monday).