Project Debugging

I have an app that just closes in the middle of the night, and am trying to figure out why. Are there any logs that would give a clue as to what is going on?

Based on events in the database, I can see that it crashed at 2:59:45 AM today. I looked in the Windows event viewer, and didn’t see anything. The SQLServer logs look ok. I don’t see anything in the system console, but I don’t know if that only stores the current state or if there is a history somewhere.

Where do I start?

What exactly is crashing? A client? The gateway?

The client (which happened to be running on the gateway PC).

I actually have two client apps running, and the other one does not crash.

You can turn on logging through the Java control panel - this will create a log on the local hard drive. Mine is kept under [tt]%USER_HOME%\AppData\LocalLow\Sun\Java\Deployment\log[/tt]

Maybe that will give you a hint as to what is going on.

Ok, my app shut down again last night, and I had logging enabled. The last two lines in the log were “Out of Memory” and “You might want to try the -mx flag to increase heap size”.

Looking at task manager, I have 500 mb available. The Ignition gateway service is using 450mb. Does that seem right? After restarting the gateway, the usage was about 350mb.

Any ideas?

The Gateway’s memory usage isn’t what is coming into question here - its the Client’s memory usage. Java will only use as much memory as you allow it to. You can increase the amount allotted for each client - it is part of the Project’s properties. What is it currently set to?

Ok, it turned out that I had a rogue query out there that was returning over 18k rows under a certain condition. I’ve fixed that for good. :slight_smile: The project memory was set to 256mb, and I’ll leave it there since I haven’t seen any number greater than 90mb all day.

Another question- the gateway is using 1.2 gb of memory. Does this seem right? I don’t remember it ever using more than SQL Server.

The Gateway is a bit memory hungry, but this does seem high. How many tags do you have, and what version are you running?

I’m using the latest version, and I don’t have any tags. There are no transaction groups running either, and I had only two clients running. What exactly causes larger memory usage at the gateway?

Thats sort of an impossible question to answer with the information at hand. Various things use memory. It doesn't sound like your gateway is doing much of anything, however, which makes it odd that it is using so much memory. When you restart your Gateway, how quickly does it ramp up to 1.2gb? Gradually? After some specific operation?

Ok, I may have found something. I restarted the service this morning and it started with about 350mb. A couple of hours later, I started working in the designer, and made some typical errors (bad queries, wrong case for local variables). At this point, the usage shot up to 660mb and is staying right there. If I wasn’t running live production I would intentionally try to test this scenario again, but it really seemed to coincide.

Does this make sense? Are their certain errors that could be causing memory leaks or something? I don’t remember this happening with FPMI.

The Gateway could certainly use some memory use tuning, to be sure. However, just because it jumped from 350 to 660 does not mean there is a memory leak. Also, just because it “stayed there” doesn’t indicate that there is a memory leak either. In fact, the only sure indicator of a memory leak is if it keeps increasing until it hits the max and you get OutOfMemory errors.

One question for you - what is the size of the [tt]settings.lobs[/tt] file inside your Ignition/contexts/main/db folder? It could be that by opening up the Designer and saving you’re triggering the auto-backup mechanism which is using a lot of memory (although this isn’t really something to worry about, just looking for an explanation). If your .lobs file is really big you should use the db compactor tool that we mentioned in the 7.1.0 release notes.

We will do some memory-use optimization on the Gateway on our side and see if there isn’t some grossly wasteful operation we’re doing that could be improved.

I don’t think it’s related to errors after all. I restarted the service at 9 this morning, and by five the memory usage had climbed up to 1.24 gb. I had two designers and two clients open, and I wasn’t doing anything unusual (made quite a few changes to the clients though, but without a bunch of errors like I had last week).

I spoke to Bobby earlier, and I’ll send a gateway backup tomorrow. I’ll also take a look at the settings.lobs file. My main concern is that the gateway shuts down every couple of days, and that seems to be tied to this escalating memory usage. Like you say, it probably isn’t due to a memory leak, but I can’t seem to find a pattern of what’s causing it.

The lobs file is 170 mb. Is that an issue? If so, can I compress it during production without restarting the Ignition service?

The 170mb lobs file could probably benefit from the compression tool. You cannot do this live, you’d have to find some downtime in order to do it.