Unable to get json response: MalformedJsonException

Maker
1

Hi Everyone,

Here I am 3 hours later for something I thought would be easy but hitting a brick wall. :expressionless:
Any help would be awesome :smile: :+1:

Prespective (MakerEdition) 8.1.14

I would like to get a json file from a server in my LAN. When I try to get the file using the httpClient() it succeeds (200) but I get an error when reading the json.

The response is encoded using gzip. I know this as using the curl command from command line requires the ā€œā€“compressedā€ flag otherwise I get gibberish. Further if I look at the response headers I can see:

{u'cache-control': [max-age=0, no-cache, no-store, must-revalidate], u'connection': [close],
uā€™content-encodingā€™: [gzip]
, u'content-length': [188], u'content-type': [application/json], u'date': [Mon, 18 Apr 2022 13:29:21 GMT], u'expires': [Fri, 31 Dec 1999 01:00:00 GMT], u'pragma': [no-cache], u'server': [Mono-HTTPAPI/1.0], u'vary': [*]}

I been through the answer MalformedJsonException: Use JsonReader.setLenient(true) however my case appears to actually be using gzip encoding.

Using that code as an example:

client = system.net.httpClient()
url = "http://<INTERNAL_IP>:8082/VirtualRadar/AircraftList.json"
headers = {
	"Accept": "application/json",
	"Accept-Encoding": "gzip, deflate"
}
res = client.get(url, headers=headers)
print res
print res.getHeaders()
print res.getJson('gzip')

I get the output:

<Response@1189296431 'http://<INTERNAL_IP>:8082/VirtualRadar/AircraftList.json' [200]>
{u'cache-control': [max-age=0, no-cache, no-store, must-revalidate], u'connection': [close], u'content-encoding': [gzip], u'content-length': [188], u'content-type': [application/json], u'date': [Mon, 18 Apr 2022 13:45:48 GMT], u'expires': [Fri, 31 Dec 1999 01:00:00 GMT], u'pragma': [no-cache], u'server': [Mono-HTTPAPI/1.0], u'vary': [*]}
Traceback (most recent call last):
  File "<input>", line 15, in <module>
ValueError: Unable to parse response body as JSON: com.inductiveautomation.ignition.common.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 33 path $

As per the documentation Response Methods and Attributes I have tried setting the ā€œ.getJson([encoding])ā€ to the following but I get the same error:

print res.getJson([gzip])

I am not sure if its to do with my headers or if the response needs decoding or probably both. Any ideas?

Thanks in advance!

EDIT

Sorry, tired and late at night; Last edit.

I am doing all this from the script console if that makes any difference.

The response when using curl:

The command:

curl http://<INTERNAL_IP>:8082/VirtualRadar/AircraftList.json --compressed -s | jq

The output:

{
  "src": 1,
  "feeds": [
    {
      "id": 1,
      "name": "Receiver localhost",
      "polarPlot": true
    }
  ],
  "srcFeed": 1,
  "showSil": false,
  "showFlg": false,
  "showPic": false,
  "flgH": 20,
  "flgW": 85,
  "acList": [],
  "totalAc": 0,
  "lastDv": "-1",
  "shtTrlSec": 30,
  "stm": 1650291163748
}
2

Any chance you could supply the file in question? A MalfrormedJsonException is pretty clear: the JSON in the file is not in a format that can be parsed as valid json. So your request is succeeding in retrieving the file, but failing to parse the contents when accessed as json.

1 Like
3

Iā€™m going to open a ticket for this. I think the problem is simply that system.net.httpClient doesnā€™t automatically handle zip/gzip compressed content.

The arguments youā€™re trying to pass are for content type, not content encoding.

1 Like
4

As a workaround, you can do something like this to decompress the GZIP body:

from java.lang import String
from java.io import ByteArrayInputStream, ByteArrayOutputStream
from java.util.zip import GZIPInputStream

client = system.net.httpClient()

response = client.get("https://httpbin.org/gzip")

encoding = "UTF-8"
gzip = GZIPInputStream(ByteArrayInputStream(response.body))
bytesOut = ByteArrayOutputStream()
gzip.transferTo(bytesOut)
decoded = bytesOut.toByteArray()

print String(decoded, encoding)
2 Likes
5

Wow this is an epic responses, thanks everyone! :+1:

I have worked in and with Rockwell over the last 10 years and still have to use the FTView SCADA software. Any technical issues at this level would have required much hoop jumping, probably a few months and would have ended with a disappointing answer.

This response is to a forum question for ā€œMaker editionā€ (a FREE product) and I received help from QA, a Lead Engineer and a Dev within 7 hours of posting. I am blown away, thank you.

@PGriffith the workaround works and should get me on my way.

Above and beyond guys, thank you.

2 Likes
6

@cmallonee, thanks I have attached the file in the EDIT section. This was retrieved from the server using curl from the bash command line. I have put this trough a json validator and it comes back as valid.
Thanks for looking at this.