Is there any way to deal with deadlocked threads, short of restarting the Ignition service? I had a situation yesterday where a gateway timer script was caught in a deadlock, and it would have been handy to be able to just kill its thread and restart it later. I did try disabling the script and saving the project, but this didn’t clear up the deadlock.
So is it possible to just kill one of the blocking threads in such a scenario? And if so, how?
There’s no way to kill those threads. How did a timer script get caught in a deadlock? Do you have a thread dump?
No, unfortunately I didn’t think to grab the thread dump until after I’d already restarted the service.
The script posts http requests to a lab device every 30 seconds to see if there are any new test results, and then posts follow-up requests to retrieve those results and stores them in the database. The device has some issues, and would occasionally stop responding to these requests. I used to have to stop and start that script every time the device failed to respond to a specific post, but since a timeout was added to system.net.httpPost() that hasn’t been a problem.
Then yesterday the users reported that they’d had to reboot the device and it hadn’t given them results since. I did the usually stop and start of the script, and when that didn’t work I went over to the thread console and saw the deadlock. I believe the blocked threads were the DedicatedScriptTimer, one of the http-80 threads, and Timer-0.
If it ever happens again, I’ll be sure to download the thread dump before restarting. And if the users can wait a bit, I’ll call support before resorting to the restart.