Issue using system.dataset.fromCSV?

bkarabinchak.psi · August 17, 2022, 2:01pm

Using Ignition 8.1.17

I am trying to import a CSV file. My code is as follows -

path = "C:\\Users\\bkarabinchak\\Desktop\\items.csv"
data = system.file.readFileAsString(path)
invDS = system.dataset.fromCSV(data)
print invDS

Here’s the first 100 rows of the file that trigger the error -
items2.csv (44.4 KB)

And the error I am getting I cannot parse out what is going wrong exactly -

Java Traceback:
Traceback (most recent call last):
 File "<input>", line 3, in <module>
   at java.base/java.lang.Class.forName0(Native Method)

   at java.base/java.lang.Class.forName(Unknown Source)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForNameImpl(ClassNameResolver.java:196)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForName(ClassNameResolver.java:169)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSVJava(DatasetUtilities.java:730)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSV(DatasetUtilities.java:680)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

   at java.base/java.lang.reflect.Method.invoke(Unknown Source)

java.lang.ClassNotFoundException: java.lang.ClassNotFoundException: 


   at org.python.core.Py.JavaError(Py.java:547)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSV(DatasetUtilities.java:686)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

   at java.base/java.lang.reflect.Method.invoke(Unknown Source)

   at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)

   at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:208)

   at org.python.core.PyObject.__call__(PyObject.java:461)

   at org.python.core.PyObject.__call__(PyObject.java:465)

   at org.python.pycode._pyx230.f$0(<input>:4)

   at org.python.pycode._pyx230.call_function(<input>)

   at org.python.core.PyTableCode.call(PyTableCode.java:173)

   at org.python.core.PyCode.call(PyCode.java:18)

   at org.python.core.Py.runCode(Py.java:1687)

   at org.python.core.Py.exec(Py.java:1731)

   at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:277)

   at org.python.util.InteractiveInterpreter.runcode(InteractiveInterpreter.java:130)

   at com.inductiveautomation.ignition.designer.gui.tools.jythonconsole.JythonConsole$ConsoleWorker.doInBackground(JythonConsole.java:611)

   at com.inductiveautomation.ignition.designer.gui.tools.jythonconsole.JythonConsole$ConsoleWorker.doInBackground(JythonConsole.java:599)

   at java.desktop/javax.swing.SwingWorker$1.call(Unknown Source)

   at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

   at java.desktop/javax.swing.SwingWorker.run(Unknown Source)

   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

   at java.base/java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: 

   at java.base/java.lang.Class.forName0(Native Method)

   at java.base/java.lang.Class.forName(Unknown Source)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForNameImpl(ClassNameResolver.java:196)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForName(ClassNameResolver.java:169)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSVJava(DatasetUtilities.java:730)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSV(DatasetUtilities.java:680)

   ... 24 more

Traceback (most recent call last):
 File "<input>", line 3, in <module>
   at java.base/java.lang.Class.forName0(Native Method)

   at java.base/java.lang.Class.forName(Unknown Source)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForNameImpl(ClassNameResolver.java:196)

   at com.inductiveautomation.ignition.common.xmlserialization.ClassNameResolver.classForName(ClassNameResolver.java:169)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSVJava(DatasetUtilities.java:730)

   at com.inductiveautomation.ignition.common.script.builtin.DatasetUtilities.fromCSV(DatasetUtilities.java:680)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

   at java.base/java.lang.reflect.Method.invoke(Unknown Source)

java.lang.ClassNotFoundException: java.lang.ClassNotFoundException:

andrews · August 17, 2022, 2:20pm

This might not be super useful as I did not write it and its from version 7.8.5

But the system uploads a csv to a table then uses that table for whatever needs there are. below is the code.

import csv

#Valve Import
file = open("\\\\wh890v\\datasource$\\Valves.csv")
csvData = csv.reader(file)

header = csvData.next()
pyData = []
for row in csvData:
	pyData.append(row)
	finalData = system.dataset.toDataSet(header,pyData)
	event.source.parent.getComponent('Valve Table').data = finalData

Daniel.Snyder · August 17, 2022, 2:25pm

If you open your csv file in excel, row 60 seems messed up:

Looking at it in a text editor:

There’s something weird going on with that row… Perhaps that is what is messing it up?

bkarabinchak.psi · August 17, 2022, 2:28pm

Completely removed that row, still getting the same error though.

PGriffith · August 17, 2022, 3:14pm

Your dataset has a whole bunch of nameless, dataless columns?

#NAMES
Item ID,Item Description,Item Class,Inactive,Subject to Commission,Description for Sales,Blank Field,Description for Purchases,Sales Price 1,Sales Price 1 Calculation,Sales Price 1 Rounding,Sales Price 1 Rounding #,Sales Price 2,Sales Price 2 Calculation,Sales Price 2 Rounding,Sales Price 2 Rounding #,Sales Price 3,Sales Price 3 Calculation,Sales Price 3 Rounding,Sales Price 3 Rounding #,Sales Price 4,Sales Price 4 Calculation,Sales Price 4 Rounding,Sales Price 4 Rounding #,Sales Price 5,Sales Price 5 Calculation,Sales Price 5 Rounding,Sales Price 5 Rounding #,Sales Price 6,Sales Price 6 Calculation,Sales Price 6 Rounding,Sales Price 6 Rounding #,Sales Price 7,Sales Price 7 Calculation,Sales Price 7 Rounding,Sales Price 7 Rounding #,Sales Price 8,Sales Price 8 Calculation,Sales Price 8 Rounding,Sales Price 8 Rounding #,Sales Price 9,Sales Price 9 Calculation,Sales Price 9 Rounding,Sales Price 9 Rounding #,Sales Price 10,Sales Price 10 Calculation,Sales Price 10 Rounding,Sales Price 10 Rounding #,Qty Discount ID,Item Tax Type,Last Unit Cost,Costing Method,G/L Sales Account,G/L Inventory Account,G/L COGS/Salary Acct,UPC / SKU,Item Type,Location,Stocking U/M,Use Multi-Packs,Purchasing U/M,Purchasing U/M Description,Purch U/M No. of Stocking Units,Purchasing Weight,Purchasing UPC / SCC,Use Purchasing U/M as Default,Sales U/M,Sales U/M Description,Sales U/M No. of Stocking Units,Sales Weight,Sales UPC / SCC,Use Sales U/M as Default,Weight,Minimum Stock,Reorder Quantity,Vendor ID,Buyer ID,Alternate Vendor,Substitution,Weight,Special Note,Master Stock Item ID,Primary Attrib. ID 1,Primary Attrib. Desc. 1,Primary Attrib. ID 2,Primary Attrib. Desc. 2,Primary Attrib. ID 3,Primary Attrib. Desc. 3,Primary Attrib. ID 4,Primary Attrib. Desc. 4,Primary Attrib. ID 5,Primary Attrib. Desc. 5,Primary Attrib. ID 6,Primary Attrib. Desc. 6,Primary Attrib. ID 7,Primary Attrib. Desc. 7,Primary Attrib. ID 8,Primary Attrib. Desc. 8,Primary Attrib. ID 9,Primary Attrib. Desc. 9,Primary Attrib. ID 10,Primary Attrib. Desc. 10,Primary Attrib. ID 11,Primary Attrib. Desc. 11,Primary Attrib. ID 12,Primary Attrib. Desc. 12,Primary Attrib. ID 13,Primary Attrib. Desc. 13,Primary Attrib. ID 14,Primary Attrib. Desc. 14,Primary Attrib. ID 15,Primary Attrib. Desc. 15,Primary Attrib. ID 16,Primary Attrib. Desc. 16,Primary Attrib. ID 17,Primary Attrib. Desc. 17,Primary Attrib. ID 18,Primary Attrib. Desc. 18,Primary Attrib. ID 19,Primary Attrib. Desc. 19,Primary Attrib. ID 20,Primary Attrib. Desc. 20,Secondary Attrib. ID 1,Secondary Attrib. Desc. 1,Secondary Attrib. ID 2,Secondary Attrib. Desc. 2,Secondary Attrib. ID 3,Secondary Attrib. Desc. 3,Secondary Attrib. ID 4,Secondary Attrib. Desc. 4,Secondary Attrib. ID 5,Secondary Attrib. Desc. 5,Secondary Attrib. ID 6,Secondary Attrib. Desc. 6,Secondary Attrib. ID 7,Secondary Attrib. Desc. 7,Secondary Attrib. ID 8,Secondary Attrib. Desc. 8,Secondary Attrib. ID 9,Secondary Attrib. Desc. 9,Secondary Attrib. ID 10,Secondary Attrib. Desc. 10,Secondary Attrib. ID 11,Secondary Attrib. Desc. 11,Secondary Attrib. ID 12,Secondary Attrib. Desc. 12,Secondary Attrib. ID 13,Secondary Attrib. Desc. 13,Secondary Attrib. ID 14,Secondary Attrib. Desc. 14,Secondary Attrib. ID 15,Secondary Attrib. Desc. 15,Secondary Attrib. ID 16,Secondary Attrib. Desc. 16,Secondary Attrib. ID 17,Secondary Attrib. Desc. 17,Secondary Attrib. ID 18,Secondary Attrib. Desc. 18,Secondary Attrib. ID 19,Secondary Attrib. Desc. 19,Secondary Attrib. ID 20,Secondary Attrib. Desc. 20,Item Note,Print Components,Number of Components,Primary Attrib. Name,Substock Primary Attrib. ID,Substock Primary Attrib. Desc.,Secondary Attrib. Name,Substock Second Attrib. ID,Substock Second Attrib. Desc.,Quantity on Sales Orders,Quantity on Purchase Orders,Quantity On Hand,Is Taxable,Warranty Period,Revision Number,Effective Date,Component Number,Component ID,Quantity Needed,Part Number,,,,,,,,,,,,,,,,,
#TYPES
str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,str,,,,,,,,,,,,,,,,,

Note the trailing commas. That’s what’s causing the error; the #TYPES line is supposed to define either a well known alias or the actual java classname for the column’s type, but there’s nothing to use for the last few columns.

You might be able to simply run a line-by-line rstrip(",") safely, but better would be to fix the data at the source, if possible.

bkarabinchak.psi · August 17, 2022, 3:16pm

Ugh can’t believe I missed. I had cleaned up a bunch of extra commas that were after #NAMES/#TYPES but didn’t even see those. I will fix the file and it should all be good.

bkarabinchak.psi · August 17, 2022, 3:40pm

Turned out the csv export I got from another program was really poorly formatted gave me a bunch of blank column names etc that I did not notice. Adding a name and a datatype there fixed it. Thanks for the second pair of eyes