Jython socket.send large sends

Ignition
1

After hours of trying to find a work around to socket timeouts, I figured I should open the _socket.py and see what is going on under the hood, low an behold, there is my problem!

I’m calling socket.sendall(str) and yes, my data is > 64KB and hanging the socket upon send and most definitely needs to be chunked!

This was not happening in 7.9. Can I safely copy the _socket.py from 7.9 into an 8 install as a temp fix? Or does anyone have a quick work around? The data I am sending is UTF-16 encoded XML.

image
image959×618 48.6 KB

2

Use Java sockets instead: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/net/Socket.html

3

Will try… my java fu is not so strong and this code was running in 7.9 for years, so changing from python to java is kinda a big deal as it runs something like 50000 txns/day.

4

What’s actually wrong here though? Hanging on socket send indefinitely or just taking longer than you wished?

5

It hangs on the socket and just stops sending data at the 64KB mark.

socket.sendall(xml)

If I watch the TCP traffic (I setup a man in the middle proxy) the send just terminates abruptly and doesn’t send the rest of the data.

6

Can you just “chunk” it yourself then? I’m not sure anything is actually wrong with that Jython std lib code other than maybe it’s inefficient memory-wise to not chunk it up.

It’s b0rked.

7

I’m trying to work on that now, the receiving system is very sensitive on encoding so the chunking in addition to the UTF-16LE requirements keep messing up the encoding, so handling it at the library/OS level is more manageable for me.

8

Does seem like a Jython bug: https://bugs.jython.org/issue2618

I can reproduce it fairly easily outside Ignition. I think chunking it yourself is the easiest thing for now.

9

You should be able to derive a workaround from this:

import socket

data = "".join(["x" for i in range(0, 128000)])

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost", 12345))

sent = 0
while (sent < len(data)):
	sent += s.send(data[sent:])

s.close()

The important part is repeatedly using send instead of sendall.

10

Let me emphasize this. Jython's socket module is known to have terrible performance and is not likely fixable within the JVM. At 50,000 transactions per day, I'd bet you notice the improvement.

{ Pssst! You can use these java classes in jython! }

11

Thanks Kevin! That worked, nice simple fix too. I neglected to notice the fact that I could just read back the position and the keep sending the part that didn't get sent out yet.

Yes, Yes, Yes. :rofl: I really need to work on my Java skills. Its def on my list, maybe this is a good time to start converting this code to Java. The system I interface to (our MES) actually has a java library to use, however its unmaintained and they refused to give us the source code so we did not want to implement it. Might be able to de-compile and pull out all the good parts and build an module.