jsonDecode bug?

Ignition
, ,
1

We are seeing an issue with system.util.jsonDecode that is causing us issues at multilingual plants.
Using system.util.jsonDecode(jstring) when the client is in Engish (en) it returns a pyDataSet and works fine.
When in Spanish, or Spanish_Mexico it returns a string?

The json string is just a dataset that we convert to json using system.util.jsonEncode, store it to a record in the database, then retrieve it later for use in a chart.
I have worked around it by checking for locale, then parsing the string manually into a dataset, but I do think this is a bug?

Pasted the string below, to reproduce just do

jstring = 'That Long String Below'
print system.util.jsonDecode(jstring)

Then switch the preview language to Spanish or Spanish Mexico Locale.
And it returns a string vs a dataset when in English.


{u'columns': [{u'type': 'java.util.Date', u'name': 't_stamp'}, {u'type': 'java.lang.Double', u'name': 'mntr/line 1/wm1/trs/trs_curtrstemp'}], u'rows': [['Tue Feb 18 22:47:34 CST 2025', 32.175926208496094], ['Tue Feb 18 22:47:35 CST 2025', 32.175926208496094], ['Tue Feb 18 22:47:36 CST 2025', 33.70949172973633], ['Tue Feb 18 22:47:37 CST 2025', 33.70949172973633], ['Tue Feb 18 22:47:38 CST 2025', 33.854164123535156], ['Tue Feb 18 22:47:39 CST 2025', 34.2881965637207], ['Tue Feb 18 22:47:40 CST 2025', 34.2881965637207], ['Tue Feb 18 22:47:41 CST 2025', 34.2881965637207], ['Tue Feb 18 22:47:42 CST 2025', 34.751155853271484], ['Tue Feb 18 22:47:43 CST 2025', 34.751155853271484], ['Tue Feb 18 22:47:44 CST 2025', 35.127315521240234], ['Tue Feb 18 22:47:45 CST 2025', 35.32986068725586], ['Tue Feb 18 22:47:46 CST 2025', 35.47453689575195], ['Tue Feb 18 22:47:47 CST 2025', 35.61921310424805], ['Tue Feb 18 22:47:48 CST 2025', 35.61921310424805], ['Tue Feb 18 22:47:49 CST 2025', 36.05324172973633], ['Tue Feb 18 22:47:50 CST 2025', 36.05324172973633], ['Tue Feb 18 22:47:51 CST 2025', 36.05324172973633], ['Tue Feb 18 22:47:52 CST 2025', 36.28472137451172], ['Tue Feb 18 22:47:53 CST 2025', 36.458335876464844], ['Tue Feb 18 22:47:54 CST 2025', 36.458335876464844], ['Tue Feb 18 22:47:55 CST 2025', 36.574073791503906], ['Tue Feb 18 22:47:56 CST 2025', 36.574073791503906], ['Tue Feb 18 22:47:57 CST 2025', 36.77661895751953], ['Tue Feb 18 22:47:58 CST 2025', 36.77661895751953], ['Tue Feb 18 22:47:59 CST 2025', 37.03703689575195], ['Tue Feb 18 22:48:00 CST 2025', 37.094905853271484], ['Tue Feb 18 22:48:01 CST 2025', 37.18171310424805], ['Tue Feb 18 22:48:02 CST 2025', 37.18171310424805], ['Tue Feb 18 22:48:03 CST 2025', 37.18171310424805], ['Tue Feb 18 22:48:04 CST 2025', 37.38425827026367], ['Tue Feb 18 22:48:05 CST 2025', 37.38425827026367], ['Tue Feb 18 22:48:06 CST 2025', 37.38425827026367], ['Tue Feb 18 22:48:07 CST 2025', 37.5], ['Tue Feb 18 22:48:08 CST 2025', 37.61574172973633], ['Tue Feb 18 22:48:09 CST 2025', 37.61574172973633], ['Tue Feb 18 22:48:10 CST 2025', 37.61574172973633], ['Tue Feb 18 22:48:11 CST 2025', 37.61574172973633], ['Tue Feb 18 22:48:12 CST 2025', 37.876155853271484], ['Tue Feb 18 22:48:13 CST 2025', 37.876155853271484], ['Tue Feb 18 22:48:14 CST 2025', 38.020835876464844], ['Tue Feb 18 22:48:15 CST 2025', 38.020835876464844], ['Tue Feb 18 22:48:16 CST 2025', 38.020835876464844], ['Tue Feb 18 22:48:17 CST 2025', 38.10763931274414], ['Tue Feb 18 22:48:18 CST 2025', 38.16550827026367], ['Tue Feb 18 22:48:19 CST 2025', 38.16550827026367], ['Tue Feb 18 22:48:20 CST 2025', 38.16550827026367], ['Tue Feb 18 22:48:21 CST 2025', 38.3680534362793], ['Tue Feb 18 22:48:22 CST 2025', 38.3680534362793], ['Tue Feb 18 22:48:23 CST 2025', 38.45486068725586], ['Tue Feb 18 22:48:24 CST 2025', 38.45486068725586], ['Tue Feb 18 22:48:25 CST 2025', 38.51272964477539], ['Tue Feb 18 22:48:26 CST 2025', 38.59953689575195], ['Tue Feb 18 22:48:27 CST 2025', 38.59953689575195], ['Tue Feb 18 22:48:28 CST 2025', 38.686344146728516], ['Tue Feb 18 22:48:29 CST 2025', 38.686344146728516], ['Tue Feb 18 22:48:30 CST 2025', 38.802085876464844], ['Tue Feb 18 22:48:31 CST 2025', 38.802085876464844], ['Tue Feb 18 22:48:32 CST 2025', 38.83102035522461], ['Tue Feb 18 22:48:33 CST 2025', 38.88888931274414], ['Tue Feb 18 22:48:34 CST 2025', 38.88888931274414], ['Tue Feb 18 22:48:35 CST 2025', 39.033565521240234], ['Tue Feb 18 22:48:36 CST 2025', 39.033565521240234], ['Tue Feb 18 22:48:37 CST 2025', 39.033565521240234], ['Tue Feb 18 22:48:38 CST 2025', 39.091434478759766], ['Tue Feb 18 22:48:39 CST 2025', 39.23611068725586], ['Tue Feb 18 22:48:40 CST 2025', 39.207176208496094], ['Tue Feb 18 22:48:41 CST 2025', 39.35185241699219], ['Tue Feb 18 22:48:42 CST 2025', 39.35185241699219], ['Tue Feb 18 22:48:43 CST 2025', 39.35185241699219], ['Tue Feb 18 22:48:44 CST 2025', 39.52546310424805], ['Tue Feb 18 22:48:45 CST 2025', 39.52546310424805], ['Tue Feb 18 22:48:46 CST 2025', 39.52546310424805], ['Tue Feb 18 22:48:47 CST 2025', 39.61227035522461], ['Tue Feb 18 22:48:48 CST 2025', 39.67013931274414], ['Tue Feb 18 22:48:49 CST 2025', 39.67013931274414], ['Tue Feb 18 22:48:50 CST 2025', 39.67013931274414], ['Tue Feb 18 22:48:51 CST 2025', 39.814815521240234], ['Tue Feb 18 22:48:52 CST 2025', 39.814815521240234], ['Tue Feb 18 22:48:53 CST 2025', 39.95949172973633], ['Tue Feb 18 22:48:54 CST 2025', 39.95949172973633], ['Tue Feb 18 22:48:55 CST 2025', 39.95949172973633], ['Tue Feb 18 22:48:56 CST 2025', 39.988426208496094], ['Tue Feb 18 22:48:57 CST 2025', 39.988426208496094], ['Tue Feb 18 22:48:58 CST 2025', 40.19097137451172], ['Tue Feb 18 22:48:59 CST 2025', 40.19097137451172], ['Tue Feb 18 22:49:00 CST 2025', 40.27777862548828], ['Tue Feb 18 22:49:01 CST 2025', 40.27777862548828], ['Tue Feb 18 22:49:02 CST 2025', 40.422454833984375], ['Tue Feb 18 22:49:03 CST 2025', 40.39352035522461], ['Tue Feb 18 22:49:04 CST 2025', 40.39352035522461], ['Tue Feb 18 22:49:05 CST 2025', 40.39352035522461], ['Tue Feb 18 22:49:06 CST 2025', 40.50925827026367], ['Tue Feb 18 22:49:07 CST 2025', 40.50925827026367], ['Tue Feb 18 22:49:08 CST 2025', 40.68286895751953], ['Tue Feb 18 22:49:09 CST 2025', 40.68286895751953], ['Tue Feb 18 22:49:10 CST 2025', 40.79861068725586], ['Tue Feb 18 22:49:11 CST 2025', 40.85647964477539], ['Tue Feb 18 22:49:12 CST 2025', 40.85647964477539], ['Tue Feb 18 22:49:13 CST 2025', 40.85647964477539], ['Tue Feb 18 22:49:14 CST 2025', 41.05902862548828], ['Tue Feb 18 22:49:15 CST 2025', 41.05902862548828], ['Tue Feb 18 22:49:16 CST 2025', 41.145835876464844], ['Tue Feb 18 22:49:17 CST 2025', 41.23263931274414], ['Tue Feb 18 22:49:18 CST 2025', 41.23263931274414], ['Tue Feb 18 22:49:19 CST 2025', 41.261573791503906], ['Tue Feb 18 22:49:20 CST 2025', 41.261573791503906], ['Tue Feb 18 22:49:21 CST 2025', 41.435184478759766], ['Tue Feb 18 22:49:22 CST 2025', 41.435184478759766], ['Tue Feb 18 22:49:23 CST 2025', 41.435184478759766], ['Tue Feb 18 22:49:24 CST 2025', 41.57986068725586], ['Tue Feb 18 22:49:25 CST 2025', 41.666664123535156], ['Tue Feb 18 22:49:26 CST 2025', 41.69560241699219], ['Tue Feb 18 22:49:27 CST 2025', 41.69560241699219], ['Tue Feb 18 22:49:28 CST 2025', 41.927085876464844], ['Tue Feb 18 22:49:29 CST 2025', 41.927085876464844], ['Tue Feb 18 22:49:30 CST 2025', 41.927085876464844], ['Tue Feb 18 22:49:31 CST 2025', 42.01388931274414], ['Tue Feb 18 22:49:32 CST 2025', 42.158565521240234], ['Tue Feb 18 22:49:33 CST 2025', 42.2743034362793], ['Tue Feb 18 22:49:34 CST 2025', 42.2743034362793], ['Tue Feb 18 22:49:35 CST 2025', 42.2743034362793], ['Tue Feb 18 22:49:36 CST 2025', 42.30324172973633], ['Tue Feb 18 22:49:37 CST 2025', 42.30324172973633], ['Tue Feb 18 22:49:38 CST 2025', 42.53472137451172], ['Tue Feb 18 22:49:39 CST 2025', 42.67939758300781], ['Tue Feb 18 22:49:40 CST 2025', 42.65046310424805], ['Tue Feb 18 22:49:41 CST 2025', 42.65046310424805], ['Tue Feb 18 22:49:42 CST 2025', 42.708335876464844], ['Tue Feb 18 22:49:43 CST 2025', 42.708335876464844], ['Tue Feb 18 22:49:44 CST 2025', 42.8819465637207], ['Tue Feb 18 22:49:45 CST 2025', 42.8819465637207], ['Tue Feb 18 22:49:46 CST 2025', 43.08449172973633], ['Tue Feb 18 22:49:47 CST 2025', 43.14236068725586], ['Tue Feb 18 22:49:48 CST 2025', 43.25810241699219], ['Tue Feb 18 22:49:49 CST 2025', 43.25810241699219], ['Tue Feb 18 22:49:50 CST 2025', 43.25810241699219], ['Tue Feb 18 22:49:51 CST 2025', 43.43171310424805], ['Tue Feb 18 22:49:52 CST 2025', 43.43171310424805], ['Tue Feb 18 22:49:53 CST 2025', 43.43171310424805], ['Tue Feb 18 22:49:54 CST 2025', 43.605323791503906], ['Tue Feb 18 22:49:55 CST 2025', 43.6631965637207], ['Tue Feb 18 22:49:56 CST 2025', 43.80786895751953], ['Tue Feb 18 22:49:57 CST 2025', 43.80786895751953], ['Tue Feb 18 22:49:58 CST 2025', 43.80786895751953], ['Tue Feb 18 22:49:59 CST 2025', 43.952545166015625], ['Tue Feb 18 22:50:00 CST 2025', 43.952545166015625], ['Tue Feb 18 22:50:01 CST 2025', 44.155094146728516], ['Tue Feb 18 22:50:02 CST 2025', 44.24189758300781], ['Tue Feb 18 22:50:03 CST 2025', 44.24189758300781], ['Tue Feb 18 22:50:04 CST 2025', 44.328704833984375], ['Tue Feb 18 22:50:05 CST 2025', 44.328704833984375], ['Tue Feb 18 22:50:06 CST 2025', 44.4444465637207], ['Tue Feb 18 22:50:07 CST 2025', 44.4444465637207], ['Tue Feb 18 22:50:08 CST 2025', 44.70486068725586], ['Tue Feb 18 22:50:09 CST 2025', 44.733795166015625], ['Tue Feb 18 22:50:10 CST 2025', 44.733795166015625], ['Tue Feb 18 22:50:11 CST 2025', 44.87847137451172], ['Tue Feb 18 22:50:12 CST 2025', 44.87847137451172], ['Tue Feb 18 22:50:13 CST 2025', 44.87847137451172], ['Tue Feb 18 22:50:14 CST 2025', 45.052085876464844], ['Tue Feb 18 22:50:15 CST 2025', 45.052085876464844], ['Tue Feb 18 22:50:16 CST 2025', 45.3125], ['Tue Feb 18 22:50:17 CST 2025', 45.3125], ['Tue Feb 18 22:50:18 CST 2025', 45.42824172973633], ['Tue Feb 18 22:50:19 CST 2025', 45.572914123535156], ['Tue Feb 18 22:50:20 CST 2025', 45.572914123535156], ['Tue Feb 18 22:50:21 CST 2025', 45.717594146728516], ['Tue Feb 18 22:50:22 CST 2025', 45.717594146728516], ['Tue Feb 18 22:50:23 CST 2025', 45.717594146728516], ['Tue Feb 18 22:50:24 CST 2025', 45.891204833984375], ['Tue Feb 18 22:50:25 CST 2025', 45.97800827026367], ['Tue Feb 18 22:50:26 CST 2025', 46.15161895751953], ['Tue Feb 18 22:50:27 CST 2025', 46.15161895751953], ['Tue Feb 18 22:50:28 CST 2025', 46.296295166015625], ['Tue Feb 18 22:50:29 CST 2025', 46.296295166015625], ['Tue Feb 18 22:50:30 CST 2025', 46.296295166015625], ['Tue Feb 18 22:50:31 CST 2025', 46.469905853271484], ['Tue Feb 18 22:50:32 CST 2025', 46.614585876464844], ['Tue Feb 18 22:50:33 CST 2025', 46.75925827026367], ['Tue Feb 18 22:50:34 CST 2025', 46.75925827026367], ['Tue Feb 18 22:50:35 CST 2025', 46.7881965637207], ['Tue Feb 18 22:50:36 CST 2025', 46.7881965637207], ['Tue Feb 18 22:50:37 CST 2025', 47.077545166015625], ['Tue Feb 18 22:50:38 CST 2025', 47.077545166015625], ['Tue Feb 18 22:50:39 CST 2025', 47.22222137451172], ['Tue Feb 18 22:50:40 CST 2025', 47.33796310424805], ['Tue Feb 18 22:50:41 CST 2025', 47.33796310424805], ['Tue Feb 18 22:50:42 CST 2025', 47.42477035522461], ['Tue Feb 18 22:50:43 CST 2025', 47.42477035522461], ['Tue Feb 18 22:50:44 CST 2025', 47.7430534362793], ['Tue Feb 18 22:50:45 CST 2025', 47.7430534362793], ['Tue Feb 18 22:50:46 CST 2025', 47.7430534362793], ['Tue Feb 18 22:50:47 CST 2025', 47.88772964477539], ['Tue Feb 18 22:50:48 CST 2025', 47.916664123535156], ['Tue Feb 18 22:50:49 CST 2025', 48.032405853271484], ['Tue Feb 18 22:50:50 CST 2025', 48.032405853271484], ['Tue Feb 18 22:50:51 CST 2025', 48.032405853271484], ['Tue Feb 18 22:50:52 CST 2025', 48.32175827026367], ['Tue Feb 18 22:50:53 CST 2025', 48.32175827026367], ['Tue Feb 18 22:50:54 CST 2025', 48.61111068725586], ['Tue Feb 18 22:50:55 CST 2025', 48.66897964477539], ['Tue Feb 18 22:50:56 CST 2025', 48.75578689575195], ['Tue Feb 18 22:50:57 CST 2025', 48.75578689575195], ['Tue Feb 18 22:50:58 CST 2025', 48.75578689575195], ['Tue Feb 18 22:50:59 CST 2025', 48.90046310424805], ['Tue Feb 18 22:51:00 CST 2025', 48.90046310424805], ['Tue Feb 18 22:51:01 CST 2025', 49.189815521240234], ['Tue Feb 18 22:51:02 CST 2025', 49.189815521240234], ['Tue Feb 18 22:51:03 CST 2025', 49.33449172973633], ['Tue Feb 18 22:51:04 CST 2025', 49.45022964477539], ['Tue Feb 18 22:51:05 CST 2025', 49.479164123535156], ['Tue Feb 18 22:51:06 CST 2025', 49.479164123535156], ['Tue Feb 18 22:51:07 CST 2025', 49.594905853271484], ['Tue Feb 18 22:51:08 CST 2025', 49.594905853271484], ['Tue Feb 18 22:51:09 CST 2025', 49.94213104248047], ['Tue Feb 18 22:51:10 CST 2025', 49.94213104248047], ['Tue Feb 18 22:51:11 CST 2025', 50.02893829345703], ['Tue Feb 18 22:51:12 CST 2025', 49.971065521240234], ['Tue Feb 18 22:51:13 CST 2025', 49.971065521240234], ['Tue Feb 18 22:51:14 CST 2025', 49.971065521240234], ['Tue Feb 18 22:51:15 CST 2025', 50.31829071044922], ['Tue Feb 18 22:51:16 CST 2025', 50.31829071044922], ['Tue Feb 18 22:51:17 CST 2025', 50.46295928955078], ['Tue Feb 18 22:51:18 CST 2025', 50.46295928955078], ['Tue Feb 18 22:51:19 CST 2025', 50.54977035522461], ['Tue Feb 18 22:51:20 CST 2025', 50.66550827026367], ['Tue Feb 18 22:51:21 CST 2025', 50.66550827026367], ['Tue Feb 18 22:51:22 CST 2025', 50.89699172973633], ['Tue Feb 18 22:51:23 CST 2025', 50.89699172973633], ['Tue Feb 18 22:51:24 CST 2025', 50.89699172973633], ['Tue Feb 18 22:51:25 CST 2025', 51.12847137451172], ['Tue Feb 18 22:51:26 CST 2025', 51.12847137451172], ['Tue Feb 18 22:51:27 CST 2025', 51.27314758300781], ['Tue Feb 18 22:51:28 CST 2025', 51.33102035522461], ['Tue Feb 18 22:51:29 CST 2025', 51.33102035522461], ['Tue Feb 18 22:51:30 CST 2025', 51.53356170654297], ['Tue Feb 18 22:51:31 CST 2025', 51.53356170654297], ['Tue Feb 18 22:51:32 CST 2025', 51.53356170654297], ['Tue Feb 18 22:51:33 CST 2025', 51.67824172973633], ['Tue Feb 18 22:51:34 CST 2025', 51.82291793823242], ['Tue Feb 18 22:51:35 CST 2025', 51.93865966796875], ['Tue Feb 18 22:51:36 CST 2025', 51.93865966796875], ['Tue Feb 18 22:51:37 CST 2025', 52.05439758300781], ['Tue Feb 18 22:51:38 CST 2025', 52.05439758300781], ['Tue Feb 18 22:51:39 CST 2025', 52.05439758300781], ['Tue Feb 18 22:51:40 CST 2025', 52.14120101928711], ['Tue Feb 18 22:51:41 CST 2025', 52.37268829345703], ['Tue Feb 18 22:51:42 CST 2025', 52.37268829345703], ['Tue Feb 18 22:51:43 CST 2025', 52.34375], ['Tue Feb 18 22:51:44 CST 2025', 52.37268829345703], ['Tue Feb 18 22:51:45 CST 2025', 52.37268829345703], ['Tue Feb 18 22:51:46 CST 2025', 52.54629898071289], ['Tue Feb 18 22:51:47 CST 2025', 52.54629898071289], ['Tue Feb 18 22:51:48 CST 2025', 52.71990966796875], ['Tue Feb 18 22:51:49 CST 2025', 52.71990966796875], ['Tue Feb 18 22:51:50 CST 2025', 52.77777862548828], ['Tue Feb 18 22:51:51 CST 2025', 52.83564758300781], ['Tue Feb 18 22:51:52 CST 2025', 52.83564758300781], ['Tue Feb 18 22:51:53 CST 2025', 52.98031997680664], ['Tue Feb 18 22:51:54 CST 2025', 52.98031997680664], ['Tue Feb 18 22:51:55 CST 2025', 52.98031997680664], ['Tue Feb 18 22:51:56 CST 2025', 53.09606170654297], ['Tue Feb 18 22:51:57 CST 2025', 53.21180725097656], ['Tue Feb 18 22:51:58 CST 2025', 53.29861068725586], ['Tue Feb 18 22:51:59 CST 2025', 53.29861068725586], ['Tue Feb 18 22:52:00 CST 2025', 53.29861068725586], ['Tue Feb 18 22:52:01 CST 2025', 53.38541793823242], ['Tue Feb 18 22:52:02 CST 2025', 53.38541793823242], ['Tue Feb 18 22:52:03 CST 2025', 53.38541793823242], ['Tue Feb 18 22:52:04 CST 2025', 53.58795928955078], ['Tue Feb 18 22:52:05 CST 2025', 53.64583206176758], ['Tue Feb 18 22:52:06 CST 2025', 53.67477035522461], ['Tue Feb 18 22:52:07 CST 2025', 53.67477035522461], ['Tue Feb 18 22:52:08 CST 2025', 53.70370101928711], ['Tue Feb 18 22:52:09 CST 2025', 53.70370101928711], ['Tue Feb 18 22:52:10 CST 2025', 53.81944274902344], ['Tue Feb 18 22:52:11 CST 2025', 53.81944274902344], ['Tue Feb 18 22:52:12 CST 2025', 53.96411895751953], ['Tue Feb 18 22:52:13 CST 2025', 53.96411895751953], ['Tue Feb 18 22:52:14 CST 2025',]]}
2

Eewww!

Don't do that. Convert to some form of lists of dictionaries or dictionaries of lists, or something along those lines.

3

That still doesn't explain the difference in returns though?
If it returns a dataset when the language is english, it should return it when it is spanish no?

4

I would think so, but you are likely running into a nuance of auto-translating datasets to JSON. There's no standard for that.

5

Was this post edited to change references from jsonEncode to jsonDecode?

6

No, but the data is making a round trip through both. English makes it. Spanish does not. Next step would be to show some samples of each.

7

I'm pretty sure the routine that tries to decode the JSON values into a Dataset eventually attempts a Locale-aware String to Number conversion, and that might be failing to convert numbers like 35.61921310424805 in a locale that would use a comma instead.

1 Like
8

We use Encode to create the json string which is then saved to a record with the rest of the test data.

Then on retrieval of the test, we use Decode to unpack it.

It never gets to the dataset after decode when the language/locale is es or es_MX

When it is en, it returns a perfect dataset, otherwise it comes in as above.

9

Hmmm?

10

it is just a 12 minute historical query of a temperature tag.
Code simplified but you will get the jist of it. just the t_stamp and temperature.

The insert.

ds = system.tag.queryTagHistory()
jData = system.util.jsonEncode(ds,0)
system.db.runPrepUpdate("Insert into BLAH (TestDate,Val1,Val2,rawData) VALUES (?,?,?,?)",[testStart,TestVal1,TestVal2,jData],database='DatabaseName',getKey=1)

The retrieval.

ds = system.db.runPrepQuery("SELECT TestDate,Val1,Val2,rawData FROM BLAH WHERE ID=?",[TestIDFromHMI],database='DataBaseName')
jData = system.util.jsonDecode(ds.getValueAt(0,'rawData'))

When it is in English it returns an actual dataset, in spanish it returns the entire json string.

11

Right. Show us what the encoded jData looks like in each language. Or compare them yourself.

12

It is in the first post, the string saved to the database is exactly that.
It is done on the gateway side, the client doesn't store the data to the database.

13

Ah. Do you get that same string when encoding in the Spanish locale? You should--JSON is not supposed to be structurally affected by locale.

Beware: your dates are being stringified because JSON doesn't have standard for that.

14

I'm confused.

The content you posted in the OP:
{u'columns': [{u'type':

That's not a JSON string. That's Python repr output.
Did you add an extra decode step somewhere?

1 Like
15

No, it is a PyDataSet run through jsonEncode and then saved to the db.

I copied that string right out of SQL Studio, had to cut the records down to paste it here though.

The only difference is setting the locale on the client when retrieving it.
Doing a print on the retrieved and deCoded string

English:
<PyDataset rows:600 cols:2>

Spanish:
The entire string as above.(Trimmed for paste limit)
{u'columns': [{u'type': 'java.util.Date', u'name': 't_stamp'}, {u'type': 'java.lang.Double', u'name': 'mntr/line 1/wm1/trs/trs_curtrstemp'}], u'rows': [['Mon Feb 17 05:16:19 CST 2025', 33.88310241699219], ['Mon Feb 17 05:16:20 CST 2025', 33.998844146728516], ['Mon Feb 17 05:16:21 CST 2025', 34.346065521240234], ['Mon Feb 17 05:16:22 CST 2025', 34.43286895751953], ['Mon Feb 17 05:16:23 CST 2025', 34.43286895751953], ['Mon Feb 17 05:16:24 CST 2025', 34.577545166015625],

16

So that means that system.util.jsonEncode() is not actually encoding the (Spanish) dataset. It is falling back to stringifying it and encoding the string. So, don't do that. (In other words, it is working by accident in English.)

(Seems to me this should be another table in the DB with a foreign key pointing back at this record.)

17

The reason we are doing it this way is the record retention policy and how long that is.
If we do it as a foreign key sub table, it will get into the millions of rows in short order. Doing it this way it is a 15kb json string in a column.

it works, just not the one to one way between english and spanish, so a few extra lines of code on the decode routine and all is good for now.
Just wanted to see if it was a bug :man_shrugging:

18

Yes, it is a bug in your code. Eventually someone will put something in the dataset in English that can't be encoded either. Convert to pure lists/dicts if you want to keep using JSON.

2 Likes
19

Yeah, I'll echo Phil here; decide on the JSON format you want (e.g.

{
"columnName": [values...]
}

or

{
    "columns": [
        {
            "name": "columnName",
            "data": [values...]
        }...
    ]
}

Or whatever else you want, and explicitly translate your dataset(s) into that structure before running them through system.util.jsonEncode. What you're currently doing is just too fragile.

2 Likes
20

Personally I'd lean towards the second style, as you could also define a java type for each column to avoid getting bit by the auto type assignment used by system.dataset.toDataSet

3 Likes