FTP Client Speed via Scripting

I am connecting to HMI’s and robots using python ftplib. I am able to upload/download files. The files are typically less than 10KB, the biggest is about 150KB. The problem I have is it takes about 4.5 seconds per ftp request whether that is connecting, getting a list of files, or downloading a file. If I run it as a python script outside Ignition, one of these operations takes less than 300ms. Is there something I can do to improve the performance?

Here’s an example of my functions for connecting, getting the list of files, and downloading:

# Create the ftp connection to the robot
# return that ftp handle
def createConnection(robotIp, robotUsername, robotPassword):
	try:
		ftp = ftplib.FTP(robotIp, robotUsername, robotPassword)
		ftp.login()
		return ftp
	except Exception, ex:
		print("Error connecting to %s : %s" % (robotIp, str(ex)))
		return None
		
# Get the list of files of each file type then
# return the list of files
def getBackupFileList(ftp):
	files = []
	fileTypes = ["*.JBI", "*.DAT", "*.CND", "*.LST"]
	for fileType in fileTypes:
		print(fileType)
		command = 'LIST %s' % (fileType)
		ftp.retrlines(command, files.append)
	for i in range(len(files)):
		files[i] = files[i].replace(files[i][0:54], '')
	return files
	
# Save the file specified by fileName in the specified folder
def downloadFile(ftp, fileName, folder):
	saveFilePath = os.path.join(folder, fileName)
	try:
		file = open(saveFilePath, 'w')
		def writeLine(data):
			file.write(data + '\n')
		ftp.retrlines('RETR ' + fileName, writeLine)
		file.close()
	except Exception, ex:
		print("Error downloading %s : %s" % (fileName, str(ex)))

There is a knowledgebase article which goes into PC performance parameters as a solution for problems in the timing. Much of it has to do with the type of storage that you are using ( physical storage, as in things that you point to- RAM and or Hard Disk and or Solid State Disk Drive ) and various cache settings. It is oriented toward that error message folks receive if they are not using the correct Time Server parameters. That said, a script file is using what experts call a ‘high-level’ or ‘more complex’ language to access the inner workings ( hardware ). FTP is also far away from Assembly Language or Machine Language. If you need a ‘faster answer’ to your quest for better response time, other users have suggested that you visit the support page ( which also includes the knowledgebase as a menu choice at entry ). I recommend you read the support policies first, because voice support is generally best if you have an account. Non-priority e-mail is free ( for now ). Try the following to get to support at IA : https://support.inductiveautomation.com Thanks in advance for telling the rest of us what the result was and therefore we may learn something also.

Here is more on that KnowledgeBase:
there is a Knowledgebase article by Anna Christian entitled “ClockDriftDetector warnings in the Gateway Console” posted 07/21/2016 08:11 A.M.
There are also some diagnostic tools to be found regarding speed / performance in a recent post regarding Azure/VM migration. I am always a bit nervous when it comes to the 3-finger salute ( Control-Alt-Delete ) which is a shortcut to Task Manager ( or mangler, as you prefer ).

I think we’ve seen others report this as well. The Jython stdlib’s FTP and other networking-related functions are really slow for some reason. I don’t know if there was ever a solution though.

I think we’ve seen others report this as well. The Jython stdlib’s FTP and other networking-related functions are really slow for some reason. I don’t know if there was ever a solution though.

If that's the case, then any opinions on what would be the best way to trigger some external method of running the FTP functions? For this particular operation, I would like to be able to pass in the settings for an HMI/robot, let it download the files to the HDD, then notify if there were any problems.

One option (that a few people on the forum have reported as working much better) is switching to Java’s built in classes, rather than using FTPLib; there are lots of examples online of Java code to make an FTP connection, and it’s usually not too complicated to port Java code into Jython.

2 Likes

Thank you for the suggestions. Using PGriffith’s suggestion I was able to use the java libraries and reduce the time to download a batch of files from 4.5 minutes to 11.5 seconds. For anyone use who is curious, here’s a chunk of what I came up with:

import os
from java.net import URL
from java.net import URLConnection
from java.io import BufferedReader
from java.io import InputStream
from java.io import InputStreamReader

def getInputStream(robotController, path):
	ftpUrl = "ftp://%s:%s@%s/%s" % (robotController[2], robotController[3], robotController[1], path)
	url = URL(ftpUrl)
	urlConnection = url.openConnection()
	inputStream = urlConnection.getInputStream()
	return inputStream

def downloadFile(robotController, fileName, folder):
	inputStream = getInputStream(robotController, fileName)
	
	streamReader = InputStreamReader(inputStream)
	reader = BufferedReader(streamReader)
	
	fileData = []
	line = reader.readLine()
	while line != None:
		fileData.append(line)
		line = reader.readLine()
	inputStream.close()
	
	saveFilePath = os.path.join(folder, fileName)
	try:
		file = open(saveFilePath, 'w')
		for line in fileData:
			file.write("%s\n" % (line))
		file.close()
	except Exception, ex:
		writeMessage("Error writing %s : %s" % (fileName, str(ex)))
4 Likes

Dang! We really should try and figure out why Jython's FTPLib is so miserably slow - but kudos for figuring it out, and posting it back here!

2 Likes

I spoke too soon. I had problems with the java.net.URLConnection and connecting to older generations of robots. Now I’m attempting to use the apache commons ftp library. I put the commons-net-3.6.jar in lib/core/gateway and /lib/core/common and restarted the gateway. Then in the script console, I’m trying the following:

from org.apache.commons.net.ftp import FTPClient

robotIp = "192.168.10.20"

ftp = FTPClient()
ftp.connect(robotIp)
reply = ftp.getReplyCode()
print(reply)

I get the error : ImportError:
Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named net

I’m not a java developer so forgive my unfamiliarity with jar’s and jar.pack.gz. Any guesses at what I’m doing wrong?

Hello people,
I know this is an old thread, but has somebody found some resolution for the FTP-based issue?
Are there any new built-in functions from Ignition to use the Apache library?
what will be the alternative or somebody found some other methods?

No, there's nothing new built in to Ignition because we don't have any first party functionality that needs FTP and, generally, there hasn't been enough demand to warrant it.

The 'drop a jar into the lib/ directory' approach is still going to be the easiest/cheapest way to get this working. Building a module that presents a system.ftp namespace would also be very doable.

Reporting? It brings in ftp4j, though not accessible to scripting.

1 Like

Thanks. Let me try to have some modifications internally and will try to make it as a module.