Large blob insert into database reboots Gateway

Ignition
1

Hello,

In our case, roughly like 10-15MB file (which probably translates to 20-30MB actual encoded data),
when put to database from a client using SQL INSERT, will cause Gateway to reboot.

system.db.startTransaction()'s timeout is set to 10006010 and no more causes the following:

      INFO   | jvm 26   | 2011/12/28 10:45:11 | WARN  [TxTimoutDaemon                ] [10:45:11,824]: Transaction "c18595b2-69c1-4916-9aa9-0238c4813902" being closed due to inactivity timeout. Make sure you close your

But this one persists:

      ERROR  | wrapper  | 2011/12/28 10:46:58 | JVM appears hung: Timed out waiting for signal from JVM.
      ERROR  | wrapper  | 2011/12/28 10:46:59 | JVM did not exit on request, terminated
      STATUS | wrapper  | 2011/12/28 10:47:05 | Launching a JVM...
      INFO   | jvm 27   | 2011/12/28 10:47:09 | WrapperManager: Initializing...
      INFO   | jvm 27   | 2011/12/28 10:47:16 | 28.12.2011 10:47:16 org.apache.catalina.startup.Embedded start
      INFO   | jvm 27   | 2011/12/28 10:47:16 | INFO: Starting tomcat server

I’ve tried setting both of these in the “ignition.conf” file:

# default is 10
wrapper.cpu.timeout=40

# default is 30
wrapper.ping.timeout=120

The client will eventually fail with message: “Read timed out”.

It seems like the file gets to the gateway ok, but the gateway keeps quiet too long
when running the INSERT into the database, which triggers the restart.

Obvious workaround is to split the file internally, so that the SQL INSERTs will never
get too big and split/join the parts client-side to make a whole file.

Anyway, I’d like to hear your position on this.

Thanks!

2

Hi,

It’s hard to imagine how a slow db operation could cause that kind of lockup. Is this a consistent, repeatable event? That is, does the system run fine, and then as soon as you try to insert the object it crashes like this?

A few pieces of info might help:

  1. What version of Java is the gateway running on? (System Status panel under “Config” in the gateway).
  2. What database is it writing to?
  3. What kind of system is this- is it a single core VM? 32-bit or 64-bit?
  4. If repeatable, open Windows task manager and watch the memory usage of the gateway process (one of the Java.exe processes, not IgnitionGateway, which is just the service manager). Does it increase right before the crash? How high does it get?

A single core machine would be more susceptible to this, and running out of memory might manifest itself in this way, so I’d like to check that first.

Regards,

3

I stumbled across this while looking for something else, but it jogged by brain a bit: It talks about the max packet size and large blobs. Something you probably already knew, but if not…

http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_max_allowed_packet