StackOverflowError on jsonEncode of dict containing Dataset

Ignition
1

I need to persist some objects into my database and the quickest way seemed to be to convert them to JSON, then save that. That works ok with a Dataset, but fails on an object that contains a Dataset.

d = system.dataset.toDataset(["foo"], [[1]])
print system.util.jsonEncode(d)
# {"columns":[{"name":"foo","type":"java.lang.Integer"}],"rows":[[1]]}

d2 = {"myDataset": d}
print system.util.jsonEncode(d2)
# RuntimeError: maximum recursion depth exceeded (Java StackOverflowError)

d3 = {"myDataset": system.dataset.toPyDataSet(d)}
print system.util.jsonEncode(d3)
# RuntimeError: maximum recursion depth exceeded (Java StackOverflowError)

Python's json.dumps also fails, even on the basic Dataset. Java's jackson returns a similar error to jsonEncode.

I'm surprised I couldn't find anything else on the forum about this. Does anyone have a workaround, or do I have to convert all the Datasets to Python dicts/lists myself first?

2

Relevant thread. Looks like encoding the dataset first may work.

3

I can’t remember when it changed but a Dataset is now also interchangeable with a pyDataset.
Encode the dataset then add it to the dict as json, then it should work.
I believe the conversion may not work correctly unless it can detect the Dataset type directly.

4

That happened when the version 8.1.x jumped to 8.3.x.

If on 8.3 and below, this does not apply.

Link to change

5

Yeah, so far that seems to be the only solution. My problem with that approach is now you don't have an object containing a dataset, you have an object containing a string representation of a dataset, which you can then encode to JSON. That means you can't cleanly go backwards. I mean, if you want to decode that JSON, you can't just run jsonDecode on it, you have to know that additionally, certain strings have to be left as strings and others need to be decoded further. Basically, you need a schema at that point.

6

There is essentially no penalty for converting to a pyDataSet. I would just do that.

I would probably write a wrapper functions which traverse the object and convert all the datasets prior to calling jsonEncode, then then do the oposite calling jsonDecode and then traversing the object and converting back to datasets.