Coming in 8.0.6, and available in nightly builds as of today, a not-as-pretty but hopefully pretty exciting new feature: system.net.httpClient. This is a new function that will deprecate the existing system.net.http* functions. I’m excited for users to try it out. It wraps the new-to-Java-11 HttpClient with some convenience stuff for usage from Python. Full details are below, but basic usage is pretty similar to the existing functions, just much more powerful:
client = system.net.httpClient()
promise = client.getAsync("https://httpbin.org/get", params={"a": 1, "b": 2})
print "doing something while waiting"
response = promise.get()
if response.good:
print response.json['args']['a']
A key feature are asynchronous requests - each common HTTP verb (get, post, put, patch, etc) is covered with a matching asynchronous variant, plus there are generic fallback request()/requestAsync() methods to enable custom HTTP verbs to be used.
Usage Examples
Synchronous (Blocking)
client = system.net.httpClient()
print client
>>> <JythonHttpClient@1072310501 timeout=10000, bypass_cert_validation=False, has_credentials=False, proxy=None, cookie_policy='ACCEPT_ORIGINAL_SERVER', redirect_policy='NEVER'>
Create a new JythonHttpClient instance with default parameters. This client accepts most common HTTP methods (GET, POST, PATCH, etc) directly, as well as arbitrary HTTP methods (.request())
response = client.get("https://httpbin.org/get")
print response
>>> <Response@1848533253 'https://httpbin.org/get' [200]>
Run a simple GET request against a URL. Returns a Response object, which has various convenience methods, such as .text, .json, etc.
print response.json
>>> {'headers': {'User-Agent': u'Ignition', 'Host': u'httpbin.org'}, 'origin': u'70.96.196.62, 70.96.196.62', 'url': u'https://httpbin.org/get', 'args': {}}
Return the JSON body of a response directly as a Python object (most likely, a dictionary containing other dictionaries). If the body can’t be decoded as JSON, throws an error.
Complete Example
client = system.net.httpClient()
response = client.get("https://httpbin.org/get", params={"a": 1, "b": 2})
if response.good:
print response.json['args']['a']
Asynchronous (Non-Blocking)
def callback(arg):
## do something out of band
print 'asyncHandler', arg
First, define a callback function, which will be executed with the result of the HTTP request (or the error, if an error was thrown during evaluation). All httpClient functions return a Promise that contains a Response object.
c = system.net.httpClient()
p = c.getAsync("https://httpbin.org/status/404")
Then, create your client and call the async variant of an HTTP method. This will directly return a Promise object.
p.whenComplete(callback)
Add your callback to the Promise object. The whenComplete(callback) handler will fire whenever the Promise resolves, but is not expected to return anything, in contrast with .then(callback), which is expected to return a new value which will itself be wrapped in a Promise, allowing for chaining.
print p
print p.get()
>>> <Promise@1427747612 isDone=false>
<Response@1299364187 'https://httpbin.org/status/404' [404]>
Finally, you can also directly return the result by calling .get([timeout]). If not specified, the timeout defaults to 60 seconds. This is optional - you can fire off asynchronous requests and they will be automatically executed, even if you don’t ever ask them to block.
Complete Example
client = system.net.httpClient()
promise = client.getAsync("https://httpbin.org/get", params={"a": 1, "b": 2})
print "doing something while waiting"
response = promise.get()
if response.good:
print response.json['args']['a']
system.net.httpClient()
Creates a new JythonHttpClient instance.
Parameters
| Parameter | Type | Default Value | Description |
|---|---|---|---|
| timeout | int | 60000 | A value, in milliseconds, to set the client’s connect timeout setting to. |
| bypass_cert_validation | bool | False | A boolean indicating whether to bypass SSL verification on the target host. Pass True to skip verification. |
| username | str | None | A string indicating the username to use for authentication if the remote server requests authentication; specifically, by responding with a WWW-Authenticate or Proxy-Authenticate header. Only supports Basic authorization. If username is present but not password, an empty string will be used for password. |
| password | str | None | A string indicating the password to use for authentication. |
| proxy | str | None | The address of a proxy, which will be used for HTTP and HTTPS traffic. If the port is not specified, it will be assumed from the protocol in the URL, i.e. 80/443. |
| cookie_policy | str | “ACCEPT_ORIGINAL_SERVER” | A string representing this client’s cookie policy. Accepts values ACCEPT_ALL, ACCEPT_NONE, and ACCEPT_ORIGINAL_SERVER. |
| redirect_policy | str | “NORMAL” | A string representing this client’s redirect policy. Accepts values NEVER, ALWAYS, and NORMAL (allows redirects, except HTTPS -> HTTP downgrades. |
| customizer | Callable | None | A reference to a callable function. This function will be called with one argument (an instance of HttpClient.Builder). The function should operate on the builder class and allows for arbitrary customization of the created HTTP client. |
JythonHttpClient
A wrapper around Java’s HttpClient.
Methods
.get(), .post(), .put(), .delete(), .head(), .patch(), .options(), .trace() -> Response
Dispatch an HTTP request. Returns a Response object.
Parameters
| Parameter | Type | Default Value | Description |
|---|---|---|---|
| url | str | Required | The URL to connect to. |
| parameters | str, dict | None | Parameters to append to the URL. If supplied as a string, will be directly appended to the URL. If supplied as a dictionary, key/value pairs will be automatically URL encoded. |
| data* | str | None | Data to send in the request. String data will be sent with a Content-Type of text/plain; charset=UTF-8, unless the Content-Type header was already specified. |
| data* | dict | None | Data to send in the request. Will be automatically encoded into JSON, and set the Content-Type header to application/json;charset=UTF-8 unless the Content-Type header was already specified. |
| data* | byte[] | None | Data to send in the request. A byte array will be sent directly, and set the Content-Type header to application/octet-stream unless the Content-Type header was already specified. |
| file | str | None | The path to a file relative to the HTTP client. If specified, and the path is valid, will directly send the data in the file. The file attribute overrides any value set in data; only the file’s data will be sent. |
| headers | dict | None | A dictionary of headers to add to the request. |
| username | str | None | Username to add to a Basic Authorization header in the outgoing request. If username is specified, but not password, password is assumed to be an empty string. |
| password | str | None | Password to add to a Basic Authorization header in the outgoing request. |
| timeout | int | 60000 | Timeout for this request, in milliseconds. |
.request() -> Response
Same as the above functions, but with an additional required argument, method:
Parameters
| Parameter | Type | Default Value | Description |
|---|---|---|---|
| url | str | Required | The URL to connect to. |
| method | str | Required | The method to use in the request. |
| parameters | str, dict | None | Parameters to append to the URL. If supplied as a string, will be directly appended to the URL. If supplied as a dictionary, key/value pairs will be automatically URL encoded. |
| data* | str | None | Data to send in the request. String data will be sent with a Content-Type of text/plain; charset=UTF-8, unless the Content-Type header was already specified. |
| data* | dict | None | Data to send in the request. Will be automatically encoded into JSON, and set the Content-Type header to application/json;charset=UTF-8 unless the Content-Type header was already specified. |
| data* | byte[] | None | Data to send in the request. A byte array will be sent directly, and set the Content-Type header to application/octet-stream unless the Content-Type header was already specified. |
| file | str | None | The path to a file relative to the HTTP client. If specified, and the path is valid, will directly send the data in the file. The file attribute overrides any value set in data; only the file’s data will be sent. |
| headers | dict | None | A dictionary of headers to add to the request. |
| username | str | None | Username to add to a Basic Authorization header in the outgoing request. If username is specified, but not password, password is assumed to be an empty string. |
| password | str | None | Password to add to a Basic Authorization header in the outgoing request. |
| timeout | int | 60000 | Timeout for this request, in milliseconds. |
.getAsync(), .postAsync(), .putAsync(), .deleteAsync(), .headAsync(), .patchAsync(), .optionsAsync(), .traceAsync() -> Promise
Returns a new Promise instance.
Parameters
| Parameter | Type | Default Value | Description |
|---|---|---|---|
| url | str | Required | The URL to connect to. |
| parameters | str, dict | None | Parameters to append to the URL. If supplied as a string, will be directly appended to the URL. If supplied as a dictionary, key/value pairs will be automatically URL encoded. |
| data* | str | None | Data to send in the request. String data will be sent with a Content-Type of text/plain; charset=UTF-8, unless the Content-Type header was already specified. |
| data* | dict | None | Data to send in the request. Will be automatically encoded into JSON, and set the Content-Type header to application/json;charset=UTF-8 unless the Content-Type header was already specified. |
| data* | byte[] | None | Data to send in the request. A byte array will be sent directly, and set the Content-Type header to application/octet-stream unless the Content-Type header was already specified. |
| file | str | None | The path to a file relative to the HTTP client. If specified, and the path is valid, will directly send the data in the file. The file attribute overrides any value set in data; only the file’s data will be sent. |
| headers | dict | None | A dictionary of headers to add to the request. |
| username | str | None | Username to add to a Basic Authorization header in the outgoing request. If username is specified, but not password, password is assumed to be an empty string. |
| password | str | None | Password to add to a Basic Authorization header in the outgoing request. |
| timeout | int | 60000 | Timeout for this request, in milliseconds. |
.requestAsync() -> Promise
Same as the above functions, but with an additional required argument, method:
Parameters
| Parameter | Type | Default Value | Description |
|---|---|---|---|
| url | str | Required | The URL to connect to. |
| method | str | Required | The method to use in the request. |
| parameters | str, dict | None | Parameters to append to the URL. If supplied as a string, will be directly appended to the URL. If supplied as a dictionary, key/value pairs will be automatically URL encoded. |
| data* | str | None | Data to send in the request. String data will be sent with a Content-Type of text/plain; charset=UTF-8, unless the Content-Type header was already specified. |
| data* | dict | None | Data to send in the request. Will be automatically encoded into JSON, and set the Content-Type header to application/json;charset=UTF-8 unless the Content-Type header was already specified. |
| data* | byte[] | None | Data to send in the request. A byte array will be sent directly, and set the Content-Type header to application/octet-stream unless the Content-Type header was already specified. |
| file | str | None | The path to a file relative to the HTTP client. If specified, and the path is valid, will directly send the data in the file. The file attribute overrides any value set in data; only the file’s data will be sent. |
| headers | dict | None | A dictionary of headers to add to the request. |
| username | str | None | Username to add to a Basic Authorization header in the outgoing request. If username is specified, but not password, password is assumed to be an empty string. |
| password | str | None | Password to add to a Basic Authorization header in the outgoing request. |
| timeout | int | 60000 | Timeout for this request, in milliseconds. |
Attributes
.javaClient/.getJavaClient() -> HttpClient
Returns the underlying Java HTTPClient.
.cookieManager/.getCookieManager()
Returns a CookieManager, which can be used to override the cookie storage policy (setCookiePolicy(policy), where policy is a java.net.CookiePolicy), or to retrieve the CookieStore directly via (getCookieStore()). The CookieStore will itself have various methods to add or retrieve cookies, as detailed on the CookieStore interface.
Response
A wrapper around a Java HttpResponse.
Methods
.body/.getBody() -> byte[]
Returns the response content directly.
.json/.getJson([encoding]) -> dict
Return the response content, with the encoding specified by the response (or the encoding specified by the encoding argument, decoded into Python (dictionary, list, etc). If response can’t be decoded to JSON, throws an error.
.text/.getText([encoding]) -> str
Return the response content decoded into a string - either with the charset specified in the response (or UTF-8 if not specified), or using the encoding specified in the function call.
Attributes
.statusCode/.getStatusCode() -> int
Return the status code of the response object, i.e. 200 or 404.
.good/.isGood() -> bool
Returns True if the response was good (i.e. 200) or False if it was a client or server error (status code between 400 and 599).
.clientError/.isClientError() -> bool
Returns True if the response was a client error, as in an HTTP 4XX response.
.serverError/.isServerError() -> bool
Returns True if the response was a server error, as in an HTTP 5XX response.
.url/.getUrl() -> str
Returns the URL this Response connected to.
.headers/.getHeaders() -> dict
Returns a case-insensitive “dictionary” of headers present on the response. Values will be in a list, even if only a single value is present.
.javaResponse/.getJavaResponse() -> HttpResponse
Returns the underlying Java HttpResponse behind this Response.
.cookieManager/.getCookieManager() -> CookieManager
Returns a CookieManager, which can be used to override the cookie storage policy (setCookiePolicy(policy), where policy is a java.net.CookiePolicy), or to retrieve the CookieStore directly via (getCookieStore()). The CookieStore will itself have various methods to add or retrieve cookies, as detailed on the CookieStore interface.
.request/.getRequest() -> RequestWrapper
Returns a RequestWrapper object, that thinly wraps a java.net.http.HttpRequest.
Promise
A wrapper around a Java CompletableFuture that returns some type
Methods
.get([timeout]) -> <T>
Block for timeout until a result is available. The result object can technically be any type, if chaining, but will be a Response object if calling one of the HTTPClient methods directly. If the timeout is met without a result, throws an exception. The default timeout is 60s, just like existing system.net.http* functions.
.then(callback) -> Promise
Allows chaining; returns a new Promise, wrapping callback. Callback should either accept two arguments (result, error) or have a single argument but be able to accept exceptions as well as valid values.
.handleException(callback) -> Promise
In the event of an exception in a potential chain of promises, handleException will be called with one argument (the thrown error) and is expected to return a new fallback value for the next step in the promise chain.
.whenComplete(callback) -> void
Call the provided callback when this promise finishes evaluating. Callback will be called with return value as the first argument, and any thrown error as the second argument. Any return value will be ignored.
.cancel() -> bool
Attempt to cancel the wrapped Java future. Returns True if the cancellation succeeded.
Attributes
.future/.getFuture()
Returns the underlying Java CompletableFuture object that this Promise contains.
.done/.isDone()
Returns True if the underlying future has completed - with a good result, or an exception.
RequestWrapper
A wrapper around a base Java HttpRequest.
Attributes
.url/.getUrl() -> str
Returns the actual URL that was contacted in the request.
.method/.getMethod() -> str
Return the HTTP method used in this request; GET, POST, PATCH, etc.
.headers/.getHeaders() -> dict
Returns a case-insensitive “dictionary” of headers present on the request. Values will always be in a list, even if only a single value is present.
.timeout/.getTimeout() -> int
Returns the timeout this query was set to, or -1 if timeout was invalid.
.version/.getVersion() -> str
Returns the HTTP version used for this request; either HTTP_1_1 or HTTP_2
.javaRequest/.getJavaRequest() -> HttpRequest
Returns the underlying HttpRequest object directly.