httpGet returning unicode html

Morning humans

I have manged to put together an api call that appears to work in that it returns a thing.
But the returned object is in unicode/ html, very dense, and I can’t begin to work out how to slice and dice it to extract useful things.
I have a project script:

import java.lang
from java.net import URLEncoder
import logging

class JythonClient:
	
	def Coruson_api(self, userName, passWord):
		baseUrl = 'https://platform.gaelenlighten.com:443/api'
		self.client = system.net.httpClient(username = userName, password = passWord)
		headers = {"tenant":"slg.gaelenlighten.com"}
		
		
	def get_reports(self, coruson_request, userName, passWord):
		baseUrl = 'https://platform.gaelenlighten.com:443/api'
		base = 'https://platform.gaelenlighten.com:443/api/reports?'
		encode_request = java.net.URLEncoder.encode(str(coruson_request), 'UTF-8')
		url = base+encode_request
		headers = {"tenant":"slg.gaelenlighten.com"}

		response = system.net.httpGet(url = url, username = userName, password = passWord, headerValues = headers)
		
		return response

and a component script on a button, that gets the required username and password, and the request dictionary info:

def runAction(self, event):
	'''
	This event is fired when the 'action' of the component occurs.

	Arguments:
		self: A reference to the component that is invoking this function.
		event: An empty event object.
	'''
	userName = (textField).props.text
	passWord = (password field).props.text
	
	reports = apiReports.JythonClient().system_api(userName, passWord)
	formName = "FormName"
	owner = (dropdown).props.options[(dropdown).props.value]['label']
	
	startDate = (component).props.value
	endDate = (component).props.value
	coruson_request = {"FormName":formName, "StartDate":startDate, "EndDate":endDate, "Owner":owner}
	response = apiReports.JythonClient().get_reports(coruson_request, userName, passWord)
	
	self.custom.response = response

Is there something I need to add/remove in order to get this as json and/or convert it, and/or make it readable as a table or other component?

I’m trying to bind the response to a table at the moment, but it just gets upset about having a string instead of an array.
Help?

There are a few things I find weird about the code you’ve shown:

  • You’re calling apiReports.JythonClient().system_api(userName, passWord) But there’s no method named system_api in the class definition. I suspect you swapped Coruson_api for system_api in the second code snippet.
  • encode_request = java.net.URLEncoder.encode(str(coruson_request), 'UTF-8') looks weird. I’m not sure it does what you think it does.
    After trying what that function does, I think it converts a string to an url formatted string. But it doesn’t convert a dict into url parameters… So you’re formatting a dict into a url compatible string. Which is probably not what you want to do
    EDIT: I don’t think this is very clear… Let me rephrase: You’re converting a dict into a string, and then replacing characters that aren’t allowed in a url by codes that correspond to those characters, without converting it into url parameters before. Basically, it converts the dict’s brackets and dots into character codes.
    dicts look like {'foo': "FOO", 'bar': 42}
    url parameters look like foo=FOO&bar=42
    urllib.urlencode() can convert a dict to url parameters for you.
  • the doc for the api is a mess and I couldn’t find how to specify a return format, but maybe there’s a parameter you can add to get a json response
  • do you NEED classes for this ? What good is the Coruson_api (or system_api) method here ? What is it supposed to do ? You’re passing everything to get_reports anyway.
  • you’re importing from java.net import URLEncoder, then calling java.net.URLEncoder.encode()

Lastly, what does the returned string look like ? It might help determine if it’s a proper response.

  • I did! I was trying to do the ‘make code generic’ thing for posting it here, then I got distracted. They match irl.
  • I think urllib has started having compatibility issues? I’ve been replacing it with the java.net.URLEncoder.encode() instead, which has worked elsewhere (when urllib.urlencode was throwing errors).
    I did think putting the string part first might do something weird, but it didn’t seem to be happy with the dictionary. It might want different syntax?
    *I probably don’t need the class, no. But it was in the code I nicked off the forums, so I kept it…
    *Yeah. The info I found about urllib getting wonky and using encoder instead had it set out that way, so I haven’t faffed with it. It’s working happily in another project, so I didn’t fix a not-apparently-broken thing…

Thank you for the comprehensive response!
Do you know what the syntax ought to be for feeding a dictionary to java.net.URLEncoder.encode()?

According to its javadoc, that's not what it's for. What it does is

Translates a string into application/x-www-form-urlencoded format using a specific encoding scheme. This method uses the supplied encoding scheme to obtain the bytes for unsafe characters.

You need to format your dict into a query string first. If you don't want to use urllib.urlencode (it should be safe to use), maybe java has a function for this, but I don't know much about java.
Otherwise, you can do it yourself, with something like this:

def format_params(params):
	if params:
		return "?" + "&".join("{}={}".format(key, value) for key, value in params.items())
	else:
		return ""

Then you can feed that string to java's URLEncoder.urlencode()

1 Like

I got it working :smiley:
running the base url and dictionary through httpClient did the encoding, then using the output of that as the url in httpGet got me a good response.
putting “format”:“json” as part of the dictionary forced that.
Now I just have to get rid of a bit of superfluous info the table won’t parse, and we might be set…
smhd

Thank you again

I would really, really strongly encourage you to migrate to system.net.httpClient, which takes care of basically all of the encoding and decoding for you when dealing with a typical REST API:
https://docs.inductiveautomation.com/display/DOC81/system.net.httpClient

3 Likes