Hello,
I have issues with special characters in CSV.
I have got a dataset with strings containing some characters like “é” or “à”, encoded like this :
string = u"This is the string with é and à" (as an example)
I export the dataset as a .csv file.
When I want to upload back this file, the encoding of the characters is not recognized and I have question marks standing for them.
Is there a way to pass the utf8 encoding through the .csv to keep the special characters ?
Thank you for your help,
Can you post your full script? Using system.dataset.toCSV/fromCSV, I get the same characters out and back in:
from com.inductiveautomation.ignition.common.util import DatasetBuilder
from java.lang import String, Integer, Double, Boolean
from java.util import Date
ds = (DatasetBuilder
.newBuilder()
.colTypes(Integer, String, Double, Boolean, Date)
.colNames("Int", "String", "Double", "Boolean", "Date")
.addRow(123, u"This is the string with é and à", 123, True, system.date.now())
.addRow(None, None, None, None, None)
.addRow(123, "Abc", 123, True, system.date.now())
.build())
t = system.dataset.toCSV(ds, True, True, False)
p = system.file.saveFile("test.csv")
print p
system.file.writeFile(p, t)
read = system.file.readFileAsString(p)
readDs = system.dataset.fromCSV(read)
pyds = system.dataset.toPyDataSet(readDs)
for col in pyds:
for row in col:
print row
What might be the issue is the encoding when reading the file back in - you can force an encoding argument (I’d recommend UTF-8, for sanity) to system.file.writeFile / readFileAsString to ensure things are decoded correctly.
Thank you for your answer.
Here is my dataset in a table :
I an downloading it with this script :
And uploading the data with this :
And i get this back :
Try adding the encoding argument to system.file.writeFile. If you open the file in a text editor separately, does it have encoding errors or have the properly encoded values?
Ok I have added the encoding in writeFile() and it is working ! Thank you !
