Java crashing on clients after a few days. hs_err_pid inside

Ignition
1

We have a couple Ignition clients whose Java crashes randomly after 10-14 days, and I need some help debugging it. It never starts back up and the computer sits there with the Desktop showing.

They are all the same hardware, same OS, same Project.
Hardware: WYSE T50 thin client
OS: Ubuntu 10.04 LTS
CPU: 1GHz ARMv7
RAM: 1GB

The Project is very simple–8 tag subscriptions, no SQL queries.
Client Memory setting in Project:
Min: 32MB
Max: 256MB

Gateway version: 7.6.4 (b2013112117) | 64-bit
Gateway Java version: Oracle Corporation 1.7.0_51
Windows 2008R2 VM

Client Java version:
Java™ SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot™ Client VM (build 24.55-b03, mixed mode)

I’ve tried different versions of Oracle Java and they all exhibit the same behavior. I haven’t tried 1.8 yet, though.

I’m thinking about writing a watchdog service and restarting Java if it crashes, but I’d really like to get to the root of this problem. The hs_err_pidXXXX.logs are attached. I can’t understand them and I hope you guys can. I’m hoping it’s something simple with a memory setting…

Client1
[attachment=1]hs_err_pid1570.log[/attachment]

Client2
[attachment=0]hs_err_pid1650.log[/attachment]

2

Are these thin clients connecting to a Terminal server or are you running the ignition application native on the Thin client itself? In regards to the lock-ups , if you are running the application native, the ARM 7 processor is going to give you problems. If you are wanting to troubleshoot this problem I would look at the wrapper.log files and see what the gateway is say about these lock-ups.

3

The thin clients are running Java natively–ie, NOT connecting to any Terminal Servers, etc.

Just to be clear, the clients aren’t locking up. Java is crashing while running the project.

I’m having difficulties correlating the times that the clients crash. I ended up writing a watchdog service to verify the Java process is running. If it’s not, it starts it back up and writes to an error log with the date/time. Presumably, this will allow me to look at the wrapper.log file to correlate the issues but I’m not sure what level logging will show me anything useful…especially because I don’t think there is an issue with the Gateway. Which logger/level settings should I use?

Does IA not support Oracle Java on ARMv7 ?

4

It’s as “supported” as it gets. It sounds like you’re just running into a bug in the JRE, maybe one that’s only present in the ARM version, or even on these ARM chips - who knows!

There’s pretty much nothing we can do. We will submit the crash logs to Oracle and then just hope and pray.

It’s certainly worth your time to see if Java 8 has already fixed this.