Expression Tag RunScript Invalid Datatype for Dataset (PyDictionary)

amarks · June 28, 2023, 2:10pm

I have an expression tag that runs a script (getTrendList()) via runScript().

This is the script:

def getAllTrendPens():
	data = system.db.runNamedQuery("getHistoryTags")
		
	if data is not None:
		historyTagPaths = data.getColumnAsList(0)
		
		trendLabelPaths = [historyTag+".Trend_Label" for historyTag in historyTagPaths]
	
		
		trendLabels_qv = system.tag.readBlocking(trendLabelPaths)
	
		trendLabels = [label.value for label in trendLabels_qv]
		
		colors = ["#66FFFF","#B266FF","#FF3333", "#66FF66", "#000099", "#FF66B2", "#CCCC00", "#000000", "#606060" ]
		
		pens = []
		
		for i, pen in enumerate(trendLabels):
			if pen is not None:
				color = colors[i % len(colors)]
				#print pen
				#print historyTagPaths[i]
				jsonObject = {
						
								  "name": pen,
								  "visible": False,
								  "enabled": True,
								  "selectable": True,
								  "axis": "",
								  "plot": 0,
								  "display": {
								    "type": "line",
								    "interpolation": "curveLinear",
								    "breakLine": True,
								    "radius": 3,
								    "styles": {
								      "normal": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 0.8,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 0.8
								        }
								      },
								      "highlighted": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 1,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 1
								        }
								      },
								      "selected": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 1,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 1
								        }
								      },
								      "muted": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 0.4,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 0.4
								        }
								      }
								    }
								  },
								  "data": {
								    "source": "histprov:MSSQLSERVER1:/drv:ignition-scada-pc:default:/tag:"+historyTagPaths[i],
								    "aggregateMode": "default"
								  }
						
							}
				pens.append(jsonObject)
				
	return pens
	
def getTrendList():

	headers = ["Name", "Pen"]

	pens = trendPens.getAllTrendPens()
	
	names = [pen["name"] for pen in pens]
	
	data = [[name,pen] for name, pen in zip(names, pens)]
		
	dataset = system.dataset.toDataSet(headers, data)
	
	return dataset

When I call trendPens.getTrendList() from the script console it returns a dataset, but when called from the expression I get a type conversion error.

Error_TypeConversion("class org.python.core.PyDictionary: Invalid DataType for Dataset.")

pturmel · June 28, 2023, 2:26pm

You can't nest complex data types in datasets that will be assigned to tags. Because it blows up their serialization for retention. Your jsonObject isn't JSON, it is a python dictionary. Convert it to JSON (string) if you want to store it in a dataset in a tag.

amarks · June 28, 2023, 2:51pm

and what's the best way to do that?

edit: I found

system.util.jsonEncode()

amarks · June 28, 2023, 4:44pm

How can I get the "name" field out of the JSON object after encoding? Script looks like this now:

def getAllTrendPens():
	data = system.db.runNamedQuery("getHistoryTags")
		
	if data is not None:
		historyTagPaths = data.getColumnAsList(0)
		
		trendLabelPaths = [historyTag+".Trend_Label" for historyTag in historyTagPaths]
	
		
		trendLabels_qv = system.tag.readBlocking(trendLabelPaths)
	
		trendLabels = [label.value for label in trendLabels_qv]
		
		colors = ["#66FFFF","#B266FF","#FF3333", "#66FF66", "#000099", "#FF66B2", "#CCCC00", "#000000", "#606060" ]
		
		pens = []
		
		for i, pen in enumerate(trendLabels):
			if pen is not None:
				color = colors[i % len(colors)]
				#print pen
				#print historyTagPaths[i]
				jsonObject = {
						
								  "name": pen,
								  "visible": False,
								  "enabled": True,
								  "selectable": True,
								  "axis": "",
								  "plot": 0,
								  "display": {
								    "type": "line",
								    "interpolation": "curveLinear",
								    "breakLine": True,
								    "radius": 3,
								    "styles": {
								      "normal": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 0.8,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 0.8
								        }
								      },
								      "highlighted": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 1,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 1
								        }
								      },
								      "selected": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 1,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 1
								        }
								      },
								      "muted": {
								        "stroke": {
								          "color": color,
								          "width": 1,
								          "opacity": 0.4,
								          "dashArray": 0
								        },
								        "fill": {
								          "color": color,
								          "opacity": 0.4
								        }
								      }
								    }
								  },
								  "data": {
								    "source": "histprov:MSSQLSERVER1:/drv:ignition-scada-pc:default:/tag:"+historyTagPaths[i],
								    "aggregateMode": "default"
								  }
						
							}
				pens.append(system.util.jsonEncode(jsonObject))
				
	return pens
	
def getTrendList():

	headers = ["Name", "Pen"]

	pens = trendPens.getAllTrendPens()
	
	names = [pen[0] for pen in pens]
	
	data = [[name,pen] for name, pen in zip(names, pens)]
		
	dataset = system.dataset.toDataSet(headers, data)
	
	return dataset

But my name column is just "{"

Edit: I used system.util.jsonDecode() like so:

def getTrendList():

	headers = ["Name", "Pen"]

	pens = trendPens.getAllTrendPens()
	
	names = [system.util.jsonDecode(pen)["name"] for pen in pens]
	
	data = [[name,pen] for name, pen in zip(names, pens)]
		
	dataset = system.dataset.toDataSet(headers, data)
	
	return dataset

pturmel · June 28, 2023, 5:12pm

A Document tag type instead of a Dataset tag type might be more friendly in your application.

amarks · June 28, 2023, 6:21pm

How so? I'll be binding the dataset to a dropdown's option prop. I have not used the Document datatype.

pturmel · June 28, 2023, 6:23pm

It should let you avoid the conversion to/from strings. I haven't tried it with a drop-down options prop.

chaoliang · April 8, 2024, 3:31am

I have an issue here with a API call.

After a API call, I get the the following response with the script below:

postData_pyDict = request['postData']
data = system.util.jsonEncode(postData_pyDict)

the value of data tag:

{
    "template_name": "new template 23",
    "template_detail": {
      "pens": [
        { "name": "XXX", "visible": true },
        { "name": "YYY", "visible": false }
      ]
    },
    "user_id": 4,
    "user_name": "chao2"
}

and the data type of "data" is

class org.python.core.PyUnicode

the data type of "postData_pyDict" is

class org.python.core.PyDictionary

How do I get the value of the "template_detail"?

chaoliang · April 8, 2024, 3:34am

the value of "postData_pyDict" is:

{u'user_name': 'chao', u'template_detail': {u'pens': [{u'visible': True, u'name': 'XXX'}, {u'visible': False, u'name': 'YYY'}]}, u'template_name': 'new template 23', u'user_id': 4}

it looks like a letter "u" is added to each string.

pturmel · April 8, 2024, 11:50am

~~data['template_detail']~~ See below

postData_pyDict['template_detail]'

No, the u'' syntax indicates a unicode string--the u is not part of the string. Just a visual marker frrom jython's repr() mechanism.

pascal.fragnoud · April 8, 2024, 12:25pm

Why are you converting to a string with jsonEncode ?
Why are you calling it a tag ? You haven't shown anything that's actually a tag.
Are you running that script in an expression tag ?

I'd say postData_pyDict['template_detail'].

pturmel · April 8, 2024, 12:32pm

Whoops, yes.

chaoliang · April 8, 2024, 11:21pm

Sorry I didn't describe the issue completed,

the u' caused issue to me, when writing the value to the database.
the other values are all good, just this tag which has "u'" in the string, it will be written to the database as well. see picture below.

chaoliang · April 8, 2024, 11:26pm

when getting the value, it's a json string. There's a " u' " added in front of each key, but not in front of the value of the key.

chaoliang · April 8, 2024, 11:33pm

The next step is to retrieve the template_detail value from the database for displaying on the screen.

because u' is added to this record, it caused issue when trying to display on the screen.

pturmel · April 8, 2024, 11:34pm

That u comes from jython stringification of a dictionary. Jython stringification is not the same thing as jsonification. Jython "repr()" style stringification is intended to be re-readable (for simple types) by a jython interpreter. JSON is intended to be read by a (surprise!) JSON parser.

Whatever you are doing to put this in the database is causing jython stringification. If you want something else, you will have to invoke it yourself.

chaoliang · April 9, 2024, 2:54am

I used system.util.jsonDecode to convert the string when retrieving the value from the database, it shows as json object, but still with u.

{
        "template_name": "abc",
        "update_time": null,
        "create_time": "2024-04-05 02:41:19.076",
        "template_detail": {
            "u'pens'": [
                {
                    "u'axis'": "Primary",
                    "u'penName'": "XX",
                    "u'key'": 0,
                    "u'tagName'": "XXXX",
                    "u'visible'": true,
                    "u'chartType'": "line"
                },
                {
                    "u'axis'": "Primary",
                    "u'penName'": "XXXX",
                    "u'key'": 1,
                    "u'tagName'": "XXXX",
                    "u'visible'": true,
                    "u'chartType'": "line"
                }
            ]
        },
        "template_id": 35
    }

Previously I used the following script to remove the u, but it seems not a genuine solution, as there might be a actual string containing the same u' characters in it.

data = data.replace("u'","'").replace("'", '"')

chaoliang · April 9, 2024, 3:07am

Found the following script working before saving the data to database. it will remove the u prefix.

ast.literal_eval(json.dumps(postData_pyDict['template_detail']))

and when retrieving from the database, using the following script will convert it back to json object.

system.util.jsonDecode()

the result is:

{
        "template_name": "new template 231112",
        "update_time": null,
        "create_time": "2024-04-09 03:02:17.504",
        "template_detail": {
            "pens": [
                {
                    "visible": true,
                    "name": "XXX"
                },
                {
                    "visible": false,
                    "name": "YYY"
                }
            ]
        },
        "template_id": 46
    }

chaoliang · April 9, 2024, 3:09am

sorry for hijacking this post.

pturmel · April 9, 2024, 3:09am

You are too hung up on removing the u. It is a artifact of printing. Ignore it.