"SSL_write() failed (10053: An established connection was aborted by the software in your host machine) while sending request to upstream" error in nginx logs when used as a reverse proxy for Ignition

In our application, we have an Ignition gateway setup behind an nginx reverse proxy that, for all Ignition purposes, seems to work just fine. However, we have an intermittent issue related to the Web Dev module it seems.

I have a Web Dev route set up to handle POST requests. It is a script and simply takes JSON data that gets sent with the POST request and inserts it into a database. There are multiple devices that send to this endpoint on a regular basis, and occasionally (and seemingly randomly) the request will fail. The device sees an HTTP 502 error from its end, and I notice that in the nginx error logs I see:

SSL_write() failed (10053: An established connection was aborted by the software in your host machine) while sending request to upstream

I have a logging statement in the script for the endpoint that logs whenever the script starts running, and when this error occurs, the logging never happens, indicating that the script is never being entered at all and the failure happens before the script handles the request.

I am able to find very little information about this error online, and restarting the gateway/the entire gateway server don’t seem to make the issue go away or become less frequent.

Here’s a pastebin of the nginx.conf I’m using (with the real domain of our application replaced with example.com, but everything else is the same) https://pastebin.ubuntu.com/p/5MTc88RbFx/

After further investigation, I found a difference between this device and the other devices that cause significantly fewer errors. The data being uploaded by the device that causes lots of errors is on the order of ~100-500kB. The devices that don’t fail nearly as often are uploading on the order of ~10-50kB. I’m wondering if there’s a bug causing Ignition’s internal web server to fail on these moderately large requests? I can’t say for sure.

The wording of that error makes it sound to me like a problem in nginx, not Ignition. If it were Ignition failing to handle those requests intermittently you’d probably see an error more like “unexpected end of stream reached” or similar.

Maybe you can run Wireshark / tshark to capture some of the traffic between nginx and Ignition.

1 Like