Ignition freeze in Virtual Server

Ignition
1

Hey all,

Today I had our Ignition server “crash” when a function of VMware, called V-Motion, transferred our virtual machine to another physical machine.

vmware.com/products/vmotion/overview.html

Apparently, from what our IT department tells me, VMware saw that one physical machine was too heavily loaded and shuffled the Ignition server over to another physical machine. During this move the Ignition service pegged the CPU at 100% and basically locked up the server. To fix the issue we had to “power off” the Ignition virtual server and bring it back up. They told me that some software does not like to be transferred between physical PC’s as they have had this issue before with other software.

My question is, has anyone had any problems with this virtual functionality before? This is the first time I’ve had a problem with a virtual environment, but from what I can tell, this is the first time this V-motion piece has ever kicked in.

2

Jeez. Consider that feature officially unsupported.

Also if you grab the log files from Ignition we might be able to find something useful in them.

3

Here is the wrapper.log file. Let me know if you would like anything else.

It happened somewhere around 12:30 PM or so as far as I can tell.

wrapper20120716.log (1.04 MB)

4

Oddly, I vmotion my server all the time.

5

I don’t know if it had something to do with the fact that we have redundant servers as well. Whatever happened locked up the backup server as well even though it’s a separate virtual machine and was not transferred via VMotion.

6

Ahh, I do not have redundant servers at the moment.

I could create 2 new VMs and run in the 2hour mode long enough to try to duplicate the issue though. Maybe i’ll try that tomorrow.

7

I wouldn’t think that VMotion would freeze Ignition, but that sounds plausible given all that goes into that. Out of curiosity, a few questions:

  1. (IT) How long does the VMotion operation normally take?
  2. (IT) Describe the shared storage setup and network connections between physical ESX/ESXi servers.
  3. (You/IT) Do you have a good time to test it? You could even create a smaller separate VM, run Ignition with your project, with and without clients connected and manually initiate the VMotion between ESX/ESXi hosts. This will depend on the capacity of physical servers AND/OR breaks in operation.
8

Here are the answers to your questions from our server administrator:

  1. 2-5 minutes

  2. Each esx server has 1 gig uplink for the service console, 1 gig uplink for VMkernel, and 1 gig uplink for virtual machines. The esx servers are connected to a EMC CX3-20 san thru fiber channel.

  3. I can vmotion your ignition server at any time.

I probably wouldn’t fool with switching stuff around until the weekend though.