Sending Datasets through Messages

Ignition
1

I’m using system.util.sendRequest to call functionality in one project (call it A) from another (call it B) and running into some strange behavior. Would appreciate some advice.

In project B, I have some code like this:

def GetSomething(id):
	payload = {
		'id': id,
	}
	
	result = system.util.sendRequest(
		project='A', 
		messageHandler='GetSomething',
		payload=payload, 
		remoteServer=ENTERPRISE_SERVER
	)
									
	return result

In project B, I have code like this:

def GetSomething(id):
	
	try:
		namedQuery = 'GetSomething'
		data = system.db.runNamedQuery(namedQuery, {'id': id})
		return data
	except:
		return system.dataset.toDataSet([],[])

This works, which tells me that the message handler is configured correctly.

HOWEVER, when I try to send return a dict that contains the dataset, I get a bunch of weird behavior (connection timeouts). This is basically what causes the errors:

def GetSomething(id):
	
	try:
		namedQuery = 'GetSomething'
		data = system.db.runNamedQuery(namedQuery, {'id': id})
		return {
			'error': False,
			'data': data
		}
	except:
		return {
			'error': True,
			'message': 'Descriptive error message'
		}

What’s going on here? I know I can send dicts through messages. Why can’t I send a dict with a dataset as a value through a message? Am I missing something?

2

It’s a bug.

If you wrap your dataset in a BasicDataset by importing
com.inductiveautomation.ignition.common.BasicDataset, then construct it from your runNamedQuery return object:
'data': BasicDataset(data)
It should send over the wire okay.

I believe this bug is on track to be fixed in 8.1.13 or so.

5

Python dictionaries explicitly disclaim any ordering of keys. Makes it very hard to truly “restore” a dataset from a dictionary. Just wrap in the BasicDataset. It serializes very efficiently.

6

Thanks guys.

@pturmel not sure what you mean by “explicitly disclaim order of keys”, but I think I see what you’re getting at with in that an decoded-and-then-encoded dataset wouldn’t always have the same ordering of rows or columns. It’s a good point.

I agree that the BasicDataset wrapper solves the problem. But I’m also glad to have discovered some helper methods in the process. Personally, I’m from the JSON data school. And I really like the ability to be able to work with datasets like this:

results = ConvertDatasetToListOfDicts(dataset)
for row in results:
    row['NEWCOLUMN'] = row['COLUMNNAME'] + 5
dataset = ConvertListOfDictsToDataset(results)

And for anyone who wants to turn python datasets into a list of dict's (like JSON), here some utility methods for that:

def ConvertDatasetToListOfDicts(dataset):
	data = system.dataset.toPyDataSet(dataset)
	return [
		{
			colName: value
			for colName, value in zip(data.columnNames, list(row))
		} for i, row in enumerate(data) 
	]	    

def ConvertListOfDictsToDataset(data):
	# assumes the dicts are only one level deep (i.e. no nested dicts)
	# assumes that data is similarly shaped in each object
	
	headers = []
	rows = []
	
	if len(data) != 0:
		prototype = data[0]
		for key in prototype:
			headers.append(key)
		
		for row in data:
			values = []
			for column in headers:
				values.append(row[column])			
			rows.append(values)
	
	return system.dataset.toDataSet(headers,rows)

Hope someone is able to get some use out of that code. Thanks again for the workaround guys!

7

Python's (and Jython's) docs point out that dictionaries are implemented with hash tables that do not retain the order that key/value pairs were inserted. You get back whatever key order the hash table implementation pleases. The only guarantee is that the order of keys and values will correspond if there hasn't been any changes between accesses.