SUDS Web Service Calls Gobbling up TCP Connections on Gateway

I have an application that connects to an SAP WSDL to create, cancel, and lookup inventory information.
Ignition v7.9.4, using built-in SUDS libraries.
I’m able to connect to the web service, and call all methods on the wsdl without issue, but I’m running into a problem with opening and hanging up TCP connections.
I have a gateway timer script that runs once a minute. It opens a client once each time the script is called, then does a query for inventory units that have recently been created, to see if the units have been moved to a new storage location. For each row returned from the query, a lookup method on the webservice is called. For each call, a new TCPv4 connection is opened and remains opened, until eventually, all ports are exhausted and the server must be rebooted.

I’ve tried replacing the deprecated suds lib on the server with the jurko suds lib . I’ve also tried different methods of caching or not caching on the client constructors:

#		client = Client(url=url, username=username, password=password, timeout=timeout, faults=faults, cache=None)
#		client = Client(url=url, username=username, password=password, timeout=timeout, faults=faults, cache=NoCache())				# 2018-05-22 NoCache()
		client = Client(url=url, username=username, password=password, timeout=timeout, faults=faults, cache=oc, cachingpolicy=1)	# 2018-05-22 ObjectCache()

None of the above has any effect on how many connections get utilized. I wouldn’t mind caching the client for efficiency’s sake, but nor would it hurt my application to open it fresh every time, as long as it frees up server resources.

I’m aware that SUDS has been deprecated for awhile, but I’ve got a fairly good size codebase built around this library, if I can just debug this issue. Any help is greatly appreciated.

Thanks!

I’ve been working on this off and on, adding print statements throughout suds and urllib scripts to understand this maddening issue.

Today, I stepped back and installed Python 2.5.3 on my dev PC, so I could see if the suds library exhibited the same behavior when used in a native Python environment. It does not. The following code I can run from a terminal, and it makes no change to the open TCPv4 connections on my machine. If I run the same script in the Ignition scripting console, or from any other scope within Ignition, then my machine opens and leaves exactly 2 connections per service call:

from suds.client import Client
from suds.cache import NoCache  

################################################################################
#
#   This script is used in a terminal with python25 exe
#   It tests the suds soap library native against a bug that manifests
#   In inductive automation that opens 2x numCalls TCPv4 connections
#   In Ignition designer or any scripting scope, open a windows powershell
#   and type in 'Get-Counter -Counter \TCPv4\*' then run this command before
#   and after running this script.
#   In Python 2.5.3, this script leaves no connections hanging
#   In Ignition v7.9.4 (Jython 2.5.3), it leaves numCalls*2 TCPv4 connections open
#   
################################################################################

url = '<myUrl>'
username = '<myUsername>'
password = '<myPassword>'
timeout = 30
faults = None
cache = NoCache()
holdingUnit = '00300237001234567890'
numCalls = 100

client = Client(url=url, username=username, password=password, timeout=timeout, faults=faults, cache=cache)          
print client

for i in range(numCalls):
    resp = client.service.Z_PALLET_GET_INFO(holdingUnit)
    print resp

#raw_input()     # keep command line open to examine output

Has anyone ever seen a similar behavior with SUDS on any version of Ignition?

For anyone reading this in the future, if you’re trying to get SUDS to work, back away slowly! :laughing:

The sepasoft web services module is java, so it appears to bypass the problems with Jython 2.5. In any case, it doesn’t present the same CLOSE_WAIT socket issue. Wish I’d abandoned SUDS sooner and switched to this module.

1 Like

This is odd, we are using SUDS for some connections to our MES but I have not found any issues yet. I’ll have to try some of your above tests. I tried the Sepasoft module before, but it could not handle some of the complicated endpoints our MES provides and the cost is significant.

Yeah, I’d be interested to see whether the same bug manifests for you.
For what it is worth, I should say that the issue wasn’t really a SUDS issue at all, as I had no issues if running the scripts in a native python interpreter, only when running in Ignition under the JVM wrapper.
Also, as far as functionality, SUDS got the job done as far as getting and posting the data I needed, it was only as I began put a significant number of calls to the web service on the gateway that the hanging sockets caused major performance issues.

I do get some odd performance issues from time to time so I now wonder if its related… Define “significant”? I think we are pushing 80 req / minute average right now.

By significant, I mean things like Active Directory calls failing, clients losing connection to the gateway, tag write timeouts, etc. Also, occasional URLOpen errors returned from SUDS, but normally the other issues manifested first. Once the gateway server was rebooted, these issues would go away temporarily.

On my application the calls/minute probably varies between 50-100/minute.

Hi Ryan, I kind of have my ear to the ground for Web Services problems and questions, as I have a product ownership role for the Web Services module here at Sepasoft. I wonder how recently you’ve tried a trial of our module – we’ve issued many updates regarding the parsing of WSDLs over the past couple of years, and it could be that one of them resolves the issue you had with your web services.