Hello all,
I am attempting to use the Perspective FileUpload component to upload multiple files, store on the component then send via email upon click of a button. I am using a table to display the file metadata so the user knows all files are ready for sending.
The core functionality is working, however, the bytearray handling is painfully slow: in the order of 10-15s for ~500kB files to append to the array.
Are there any suggestions on a cleaner way to do this or speed things up?
Note - there is some test lines in the send email button script to ensure valid email is entered in a text box and output a string into label for feedback etc.
Instead of storing the attachment data in custom props try writing the data to a temp file on disk and reading it back when you send the email. Use the props to store metadata about where you wrote the file.
Thanks Kevin. I opted for storing in a DB instead of to disk, but it has improved the speed ten fold.
Why does storing the file to custom props slow everything down?
Don't use bare except clauses. While it shouldn't happen in this particular case (note the use of "shouldn't"), this suppresses any error that you didn't think of. At the very least, check what kind of exception sendEmail raises and catch this one precisely.
A couple of remarks on code style (I know you didn't ask for this, but I can't help it ):
Avoid calling things string. Give your variables meaningful name as much as possible. Maybe this could be debug_message or something like that.
using string formatting can help reduce some clutter and make things more readable. Also, you won't have to cast things to strings manually: debug_message = "Email succesfully sent at {}".format(system.date.now())
There are several formatting syntaxes to chose from. I personally prefer .format as it makes it very clear what it does, and you don't need to specify types
Do you really need to write debug messages to a tag ? Wouldn't logging make things easier ?
It also make you do repeated calls to tag writing functions, which isn't ideal.
@Christopher_Cameron
I’m trying to do the same thing. I’m uploading as varbinary(MAX) to the database, but then struggling to make the query database result into a list of byte-formats for sending the email.
Could you share how you’re doing to to-database → from database - > to email attachment data process, please?
Thanks
@Cameronin Please take this with a grain of salt, no idea if I follow any form of best practice.
Storing to DB
The DB table is setup to have an auto incrementing primary key. The system.dv.runPrepUpdate can retrieve this key and I store this into a variable called flag and store as a custom prop:
flag = system.db.runPrepUpdate(
"INSERT INTO files (file_name, file_bytes,upload_date) VALUES (?,?,?)",
[fileName,fileBytes,system.date.now()],
"DEV_001",
getKey=1
)
self.custom.MetaData_Array = system.dataset.addRow(self.custom.MetaData_Array,[flag,fileName,event.file.size])
Send Email: Button that runs a script to retrieve the data and send
FileArray = []
NameArray = []
num_rows = self.getSibling('FileUpload').custom.MetaData_Array.getRowCount()
pyData = system.dataset.toPyDataSet(self.getSibling('FileUpload').custom.MetaData_Array)
for i in range(num_rows):
identifier = pyData[i][0]
FileTemp = system.db.runPrepQuery("SELECT file_bytes FROM files WHERE id = ?",[identifier],database="DEV_001")
NameArray.append(pyData[i][1])
FileArray.append(FileTemp[0][0])
The proceed to send the attachment array as per my original post.
If you’re converting your dataset to a pyDataSet, you might as well take advantage of it being iterable:
for row in pyData:
identifier = row[0]
FileTemp = system.db.runPrepQuery("SELECT file_bytes FROM files WHERE id = ?",[identifier],database="DEV_001")
nameArray.append(row[1])
FileArray.append(FileTemp[0][0])
Suggestions:
gather identifiers to get all the requested files in one query, instead of having a query for each file
put together the name of the files and the file’s data in a dictionary (or an object if you’re an oo guy)
maybe something like this:
identifiers = [row[0] for row in py_ds]
files = system.db.runPrepQuery("SELECT id, file_bytes FROM files WHERE id IN ?", identifiers, database="DEV_001")
files = [
{
'file_name': r[1],
'file_data': f[1]
} for f, r in zip(files, py_ds) if f[0] == r[0]
]
I will add onto what @pascal.fragnoud said with a couple of other things.
Avoid making multiple calls to system.tag.write* or read*. In this case you’re using the deprecated system.tag.write() which I believe just calls system.tag.writeBlocking(). This means that each call is blocking this thread until the write has completed. In this instance probably not a huge adder to the execution time, but it isn’t helping.
There are optional clauses that can be used with the try statement, depending on what you’re wanting to accomplish. The first is the else clause which will execute if there are no exceptions, this is useful for insuring that the except clause is only catching errors from the code you’re actually trying to protect. The second is the finally clause which will execute after all other clauses have executed, regardless of an exception occurring.
I agree with @pascal.fragnoud here in that using a logger or custom property if I’m trying to display to a view is the best practice, but if for some reason I had to write the result to a tag I would probably do it like this:
resultMessage = ''
try:
system.net.sendEmail(smtp, sender, subject, body, 0, [recipient], self.getSibling('FileUpload').custom.fileName, self.getSibling('FileUpload').custom.fileData)
#don't know what exception type should actually be handled here, perhaps
except SMTPResponseException as e:
resultMessage = "Email failed to send. {}".format(e.smtp_error)
except Exception as e:
resultMessage = "Email failed to send. {}".format(e)
else:
resultMessage = "Email successfully sent: {}".format(system.date.now())
finally:
system.tag.writeBlocking(['DEV_MemoryTags/Result_String'],[resultMessage])
or perhaps use system.tag.writeAsync() if you don’t want to block the thread.
):