Failure to send binary data greather than 64KB using Requests

Ignition
,
1

I'm trying to send an image to an Azure vision endpoint. The data stored in the DB looks something like this.

DB Value:
array('b', [-119, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73 .. etc

Its converted to a byteArray to be packaged as the data
Byte Array Value:
�PNG

IHDR����r8gAMA���a .....etc

The basic endpoint definition and instructions are the following.
https://eastus2.api.cognitive.microsoft.com/customvision/v3.0/Prediction/[project id]/classify/iterations/[iteration Name]/image
Set Prediction-Key Header to : key value
Set Content-Type Header to : application/octet-stream
Set Body to :

We had initially tried using the built in Ignition function, system.net.httpPost as our method to send the images to the endpoint, but we kept getting a “Bad Image Format” error returned from Azure.
The post hints about it being an issue with the byteArray, but from our testing it seems to be something happening in httpPost.

Next we pivoted to using the python libraries: Request (backed by URLLIB3)

• Using the requests library within the Ignition platform, sending images that are less than 64K works successfully as expected
• Using the requests library within the Ignition platform, sending images that are greater than 64K do not work successfully. We receive a timeout with a ConnectionAborted: BadLineStatus(‘’’’)
o Setting the Content-Length value in the header didn’t help either unfortunately.

As a validation step, I simply did an install of python 3.7, loaded up requests and urllib3 library and performed the exact same calls (fetched data from our database, called out to the Azure endpoint).
• Using the requests library from the command line against python sending images that are less than 64K works successfully as expected
• Using the requests library from the command line against python, sending images that are greater than 64K works successfully as expected

This may be relevant - jython 2.7.1 limits size of http 'post' · Issue #2535 · ControlSystemStudio/cs-studio · GitHub

I understand that there are a lot of moving parts here…
• Ignition is jython - 2.7
• Requests and urllib3 are additional libraries
• The external test we performed where it works isn’t a good 1 to 1 representation, but it took the Ignition system / JVM out of the equation..

Since we can’t get the call to fail in the same way outside of the ignition environment, we’re struggling to get a good understanding as to why or where in the pipeline its failing. Any ideas about where this might be failing or a different method that is known to work for this kind of problem would be great.
I'd like to avoid creating a custom module just to get around this.

Thanks

2

https://bugs.jython.org/issue2618
Looks like the root cause.
I’d suggest giving system.net.httpClient a try - it’s using the purely Java HttpClient class under the hood, so there’s nowhere that uses Jython internals like the socket library.

2 Likes
3

Aces. The httpClient does indeed help us get around this problem. Much appreciated.

4

Can you give some example with the request post method to sendPhoto
using system.net.httpClient

Please

5

Since this post it seems that someone has found a fix for this in the link:
https://bugs.jython.org/issue2618
If I wanted to try to implement this fix by updating _socket.py would I be able to do so? What would I need to watch out for?

6

Sure, you should be able to freely modify the file. I don’t think there are any ‘gotchas’.

1 Like
7

Worked perfectly, I am now able to send multipart/form-data from python requests with files greater than 64KB!