FTP Client Speed via Scripting

I am connecting to HMI’s and robots using python ftplib. I am able to upload/download files. The files are typically less than 10KB, the biggest is about 150KB. The problem I have is it takes about 4.5 seconds per ftp request whether that is connecting, getting a list of files, or downloading a file. If I run it as a python script outside Ignition, one of these operations takes less than 300ms. Is there something I can do to improve the performance?

Here’s an example of my functions for connecting, getting the list of files, and downloading:

# Create the ftp connection to the robot
# return that ftp handle
def createConnection(robotIp, robotUsername, robotPassword):
	try:
		ftp = ftplib.FTP(robotIp, robotUsername, robotPassword)
		ftp.login()
		return ftp
	except Exception, ex:
		print("Error connecting to %s : %s" % (robotIp, str(ex)))
		return None
		
# Get the list of files of each file type then
# return the list of files
def getBackupFileList(ftp):
	files = []
	fileTypes = ["*.JBI", "*.DAT", "*.CND", "*.LST"]
	for fileType in fileTypes:
		print(fileType)
		command = 'LIST %s' % (fileType)
		ftp.retrlines(command, files.append)
	for i in range(len(files)):
		files[i] = files[i].replace(files[i][0:54], '')
	return files
	
# Save the file specified by fileName in the specified folder
def downloadFile(ftp, fileName, folder):
	saveFilePath = os.path.join(folder, fileName)
	try:
		file = open(saveFilePath, 'w')
		def writeLine(data):
			file.write(data + '\n')
		ftp.retrlines('RETR ' + fileName, writeLine)
		file.close()
	except Exception, ex:
		print("Error downloading %s : %s" % (fileName, str(ex)))

There is a knowledgebase article which goes into PC performance parameters as a solution for problems in the timing. Much of it has to do with the type of storage that you are using ( physical storage, as in things that you point to- RAM and or Hard Disk and or Solid State Disk Drive ) and various cache settings. It is oriented toward that error message folks receive if they are not using the correct Time Server parameters. That said, a script file is using what experts call a ‘high-level’ or ‘more complex’ language to access the inner workings ( hardware ). FTP is also far away from Assembly Language or Machine Language. If you need a ‘faster answer’ to your quest for better response time, other users have suggested that you visit the support page ( which also includes the knowledgebase as a menu choice at entry ). I recommend you read the support policies first, because voice support is generally best if you have an account. Non-priority e-mail is free ( for now ). Try the following to get to support at IA : https://support.inductiveautomation.com Thanks in advance for telling the rest of us what the result was and therefore we may learn something also.

Here is more on that KnowledgeBase:
there is a Knowledgebase article by Anna Christian entitled “ClockDriftDetector warnings in the Gateway Console” posted 07/21/2016 08:11 A.M.
There are also some diagnostic tools to be found regarding speed / performance in a recent post regarding Azure/VM migration. I am always a bit nervous when it comes to the 3-finger salute ( Control-Alt-Delete ) which is a shortcut to Task Manager ( or mangler, as you prefer ).

I think we’ve seen others report this as well. The Jython stdlib’s FTP and other networking-related functions are really slow for some reason. I don’t know if there was ever a solution though.

I think we’ve seen others report this as well. The Jython stdlib’s FTP and other networking-related functions are really slow for some reason. I don’t know if there was ever a solution though.

If that’s the case, then any opinions on what would be the best way to trigger some external method of running the FTP functions? For this particular operation, I would like to be able to pass in the settings for an HMI/robot, let it download the files to the HDD, then notify if there were any problems.

One option (that a few people on the forum have reported as working much better) is switching to Java’s built in classes, rather than using FTPLib; there are lots of examples online of Java code to make an FTP connection, and it’s usually not too complicated to port Java code into Jython.

2 Likes

Thank you for the suggestions. Using PGriffith’s suggestion I was able to use the java libraries and reduce the time to download a batch of files from 4.5 minutes to 11.5 seconds. For anyone use who is curious, here’s a chunk of what I came up with:

import os
from java.net import URL
from java.net import URLConnection
from java.io import BufferedReader
from java.io import InputStream
from java.io import InputStreamReader

def getInputStream(robotController, path):
	ftpUrl = "ftp://%s:%s@%s/%s" % (robotController[2], robotController[3], robotController[1], path)
	url = URL(ftpUrl)
	urlConnection = url.openConnection()
	inputStream = urlConnection.getInputStream()
	return inputStream

def downloadFile(robotController, fileName, folder):
	inputStream = getInputStream(robotController, fileName)
	
	streamReader = InputStreamReader(inputStream)
	reader = BufferedReader(streamReader)
	
	fileData = []
	line = reader.readLine()
	while line != None:
		fileData.append(line)
		line = reader.readLine()
	inputStream.close()
	
	saveFilePath = os.path.join(folder, fileName)
	try:
		file = open(saveFilePath, 'w')
		for line in fileData:
			file.write("%s\n" % (line))
		file.close()
	except Exception, ex:
		writeMessage("Error writing %s : %s" % (fileName, str(ex)))
3 Likes

Dang! We really should try and figure out why Jython’s FTPLib is so miserably slow - but kudos for figuring it out, and posting it back here!

2 Likes

I spoke too soon. I had problems with the java.net.URLConnection and connecting to older generations of robots. Now I’m attempting to use the apache commons ftp library. I put the commons-net-3.6.jar in lib/core/gateway and /lib/core/common and restarted the gateway. Then in the script console, I’m trying the following:

from org.apache.commons.net.ftp import FTPClient

robotIp = "192.168.10.20"

ftp = FTPClient()
ftp.connect(robotIp)
reply = ftp.getReplyCode()
print(reply)

I get the error : ImportError:
Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named net

I’m not a java developer so forgive my unfamiliarity with jar’s and jar.pack.gz. Any guesses at what I’m doing wrong?