Connecting to Smart Home Devices

Forgive my ignorance here, but I don’t believe this particular topic has been covered much yet. Maybe the rest of you just innately understand what to do? :slight_smile: Unfortunately, I do not. I’m looking to mess around with a home automation application to monitor and control the various smart devices in my home. Currently, everything is connected to my Google Home account. I’m having trouble figuring out how to retrieve data from Google Home for Ignition to display. Conceptually, I assume I’ll probably have some property bindings on templates that use script transforms to make API calls to retrieve data? Or perhaps a gateway timer event that retrieves data and writes it all to tags? In any case, I assume the majority of my devices will be interacted with via web APIs. All of the APIs I have looked into so far require getting authentication in the form of a token and that is where I am struggling. Looking through Google’s documentation and their cloud platform site seem to suggest that they only expect commercial users to using their APIs, hence there are a lot of hoops to jump through just to get an OAuth2 token?

Am I missing something here? How are all of you retrieving/sending data from/to devices in your home automation projects?

Hi @swolnisty, it sounds like you are on the right track. One of the jobs on my list is to integrate my nest thermostat into Ignition. From the (minimal) research I’ve done so far, you’re right, there are a lot hoops and it seems Google have been locking down their APIs over the past few years (much to the disdain of quite a few people using them).

You might have given me the kick to attempt this :smiley: so I will let you know if I find anything further, if you don’t mind doing the same?

For reference, my setup is mainly on the LAN using MQTT to talk to devices. I am no expert when it comes to API integration, but I thought I would post to see if we could fumble our way through it together :slight_smile:

Is WebDev necessary? I was under the impression WebDev was to implement your own API against resources under control of an Ignition gateway and httpClient scripting calls would be used to interact with an external API. I am, admittedly, not an expert, so I very well could be wrong.

1 Like

Apologies, you are correct @zacht, it is probably overkill and incorrect. Edited the post to reflect this, I’ll make sure to do a bit more reading :slight_smile:

It sounds like we are in similar places with maker edition. I’ve been casually working on this problem off and on for the last five months or so, but have not had much luck yet. One of the devices I’ve been focusing on getting data from is a Nest thermostat as well. I’ll be sure to keep you posted if I have any success!

@swolnisty, I now remember why I was put off the idea of integrating to Google products :weary:

Have you come across the Google Device Access program? If not, this is required. You need to to go through their 4 step guide, which is full of fun! :woozy_face:

First step is to pay Google $5 so they can harvest more data :laughing:. This is the point that I decided it wasn’t worth the effort for me. However, if you don’t mind forking out for it (not much I know, in the grand scheme of things) the rest of the process is pretty well documented. I also found a decent guide here with some screenshots of the sign-up/configuration process and example code for the Nest Thermostat.

As I have suggested, I can’t justify the effort required as I only have a single thermostat to integrate with. But hopefully this will give you and others a leg up. Apologies if you have already come across this, but I thought I’d summarise it for others and for future reference.

I am going to go back to more fun projects, like shutting-up my doorbell when the child is napping :slight_smile:. I will continue to avoid Google if possible, and only allow them to make money off of my unique humidity conditions [puts tinfoil hat on and rocks self to sleep] :crazy_face: :laughing:

In all seriousness though, please post your results because I would still be interested in how you got on :slight_smile:

Thanks for the info, @matthew.ayre!

I believe I was attempting to follow those steps last September when it was announced that non-commercial users would be able to access their Nest devices (or something to that effect, my memory is a little hazy on what triggered me to start looking into it more seriously at that point). I do recall the steps listed in their guide as having been completely broken for my use case at the time though. The good news for your entertainment is that I have already paid Google’s data harvesting toll (though I completely understand why anyone would choose to not pay a fee to access devices they already have paid for :roll_eyes:), so I have no reason to not try the rest of their steps.

A brief look through their updated guide and I see they added a button to easily generate the OAuth2 info I need and it appears to be MUCH simpler than the other avenue I was pursuing with Google’s API library:
image

I’ll monkey around with it this evening and let you know where I end up!

No worries @swolnisty. I assumed you had probably been through the pain of the documentation, but I thought I would link it all anyway for the other n00bs like me :smile:

I’d happily pay $5 tbh if I had more devices to tinker with. But in my case, it’s not worth it for a device that largely works autonomously anyway. I only wanted to use the integration to make my home dashboard more pretty. Oh well, looks like I have more real estate for the next shiny project :laughing:

Thank you in advance for keeping us up to date :slight_smile:

I was able to spend a little bit of time digging through this tonight. That blog post you found, @matthew.ayre, was super helpful with getting through some of the stuff that I am less familiar with during set up!

I am running into some difficulty getting the access/refresh tokens though. The blog suggests using this script to get them from the API:

# Get tokens

import requests

params = (
    ('client_id', client_id),
    ('client_secret', client_secret),
    ('code', code),
    ('grant_type', 'authorization_code'),
    ('redirect_uri', redirect_uri),
)

response = requests.post('https://www.googleapis.com/oauth2/v4/token', params=params)

response_json = response.json()
access_token = response_json['token_type'] + ' ' + str(response_json['access_token'])
print('Access token: ' + access_token)
refresh_token = response_json['refresh_token']
print('Refresh token: ' + refresh_token)

I am defining a method in a custom script library to do this (variables not defined in this script are hard coded outside of the method) :

def getToken():
	url = 'https://www.googleapis.com/oauth2/v4/token'
	postParams = {
				'client_id'		: client_id,
				'client_secret'	: client_secret,
				'code'			: code,
				'grant_type'	: 'authorization_code',
				'redirect_uri'	: redirect_uri
			}
	
	response = system.net.httpPost(url, postParams)

	json = system.util.jsonEncode(response)
	access_token = json['access_token']
	refresh_token = json['refresh_token']
	system.tag.writeAsync([accessTokenPath, refreshTokenPath], [access_token, refresh_token])
	comments = """Access token: %d
				Refresh token:  %d""" % (access_token, refresh_token)
	return comments

Unfortunately, I am only getting http response 400 from this method. I have a feeling that this is likely due to my extremely limited exposure to using web APIs (which is part of the reason I want to figure out these device connections).

I’ll keep you posted as I make progress!

Hi @swolnisty, glad you’re making progress. Assuming your client_id, client_secret, and code are all correct, then I can’t see anything wrong with this (through my inexperienced eyes). Is the redirect_uri set to ‘https://www.google.com’?

If you do it in a curl command via a terminal (like in the Get Access Token guide) does it produce the same results? If the curl command does give you the tokens, could you hardcode them in (I know not very secure, but for testing purposes…) and then setup a timer script to refresh the token before it times out?

Sadly, not much of a different result using curl in a terminal. I’m getting this as my result:

{
  "error": "invalid_client",
  "error_description": "Unauthorized"
}

I went back through all the steps again, just make sure I had the right client_id, client_secret, and code. Still the same results. I’ll keep you posted with any updates I come across though!

I have no idea what changed, but I finally made some progress on this. I basically went to the URL to get the code again like mentioned in the blog post referenced before:

project_id = 'your-project-id'
client_id = 'your-client-id.apps.googleusercontent.com'
client_secret = 'your-client-secret'
redirect_uri = 'https://www.google.com'

url = 'https://nestservices.google.com/partnerconnections/'+project_id+'/auth?redirect_uri='+redirect_uri+'&access_type=offline&prompt=consent&client_id='+client_id+'&response_type=code&scope=https://www.googleapis.com/auth/sdm.service'
print("Go to this URL to log in:")
print(url)

And this time the code that I got back seemed to work? I guess the third time is the charm!

1 Like

Hey @swolnisty, did you manage to get over the final hurdles and get comms to the thermostat?

I did! I have been meaning to post a follow-up reply that goes over the exact way I’m getting data from the API for a while, but have been a bit too tired after work to do so yet. It’s still on the docket though!

I’m actually curious to see how others are updating data in their maker applications though. I have defined a custom script library for the nest communications and I call a value update method from it once a minute from a gateway event script that writes API values to tags. If that method call fails, I have it call another custom method that updates the access token so that the next time the method is called it returns good data.

1 Like

I’m looking at getting the ecobee 3 lite smart thermostat. It looks like the api should be fairly usable.
ecobee API

2 Likes

For those curious, below is the script library I have created so far for getting data back from my Nest thermostat. It requires walking through the first few steps of the guide @matthew.ayre posted in the sixth reply to this thread before you’re able to use it though. I have this saved as a script named ‘devices.nest’.

project_id = 'your-project-id-here'
client_id = 'your-client-id-here'
client_secret = 'your-client-secret-here'
redirect_uri = 'https://www.google.com'
device_id = 'your-device-id-here'
code = 'your-code-here'
scope = 'https://www.googleapis.com/auth/sdm.service'
basePath = '[default]Devices/Nest/'
accessTokenPath = basePath + 'Access Token'
refreshTokenPath = basePath + 'Refresh Token'
tokenTimePath = basePath + 'Token Timestamp'
namePath = basePath + 'Device Name'
humidityFbkPath = basePath + 'Current Humidity'
tempFbkPath = basePath + 'Current Temperature'
modeFbkPath = basePath + 'Current Mode'
stateFbkPath = basePath + 'Current State'
heatSpPath = basePath + 'Heat Setpoint'
coolSpPath = basePath + 'Cool Setpoint'
commsFbkPath = basePath + 'Comms'

# This can be called from the script console after configuring Google's security permissions to grant access to your Nest device
def getToken():
	url = 'https://www.googleapis.com/oauth2/v4/token?client_id=' + client_id + '&client_secret=' + client_secret + '&code=' + code + '&grant_type=authorization_code&redirect_uri=' + redirect_uri

	response = system.net.httpPost(url=url, postData={})
	json = system.util.jsonEncode(response)
	access_token = json['access_token']
	refresh_token = json['refresh_token']
	system.tag.writeAsync([accessTokenPath, refreshTokenPath], [access_token, refresh_token])
	comments = """Access token: %d
				Refresh token:  %d""" % (access_token, refresh_token)
	return comments

# Refresh the access token and write it to the Access Token memory tag
def refreshToken():
	url = 'https://www.googleapis.com/oauth2/v4/token'
	refresh_token = system.tag.readBlocking([refreshTokenPath])[0].value
	postParams = {
				'client_id'		: client_id,
				'client_secret'	: client_secret,
				'refresh_token'	: refresh_token,
				'grant_type'	: 'refresh_token'
				}
			
	response = system.net.httpPost(url, postParams)
	
	json = system.util.jsonDecode(response)
	access_token = json['access_token']
	timestamp = system.date.now()
	system.tag.writeAsync([accessTokenPath, tokenTimePath], [access_token, timestamp])
	comments = 'Access token: ' + access_token
	return comments

# Build out the headerValues argument used in most of the system.net.httpGet() API calls
def getHeaders():
	access_token = system.tag.readBlocking([accessTokenPath])[0].value
	return {
			'Content-Type'	: 'application/json',
			'Authorization'	: 'Bearer ' + access_token,
			}
	
# Convert celsius temperatures to fahrenheit
def cToF(temp):
	return (temp * 9/5) + 32

# Convert fahrenheit temperatures to celsius
def fToC(temp):
	return (temp - 32) * 5/9

# Write the current values return from the Nest API to my pre-defined tags.
def getCurrentValues():
	deviceName = system.tag.readBlocking([namePath])[0].value
	url = 'https://smartdevicemanagement.googleapis.com/v1/' + deviceName
	headers = getHeaders()
	response = system.util.jsonDecode(system.net.httpGet(url, headerValues=headers))
	
	paths =		[
				humidityFbkPath,
				tempFbkPath,
				coolSpPath,
				heatSpPath,
				modeFbkPath,
				stateFbkPath,
				commsFbkPath
				]
	
	traits = response['traits']
	sp = traits['sdm.devices.traits.ThermostatTemperatureSetpoint']
	comms = False
	if str(traits['sdm.devices.traits.Connectivity']['status']) == 'ONLINE':
		comms = True
	tempC = traits['sdm.devices.traits.Temperature']['ambientTemperatureCelsius']
	coolSpC = sp['coolCelsius']
	heatSpC = sp['heatCelsius']
	
	values =	[
				traits['sdm.devices.traits.Humidity']['ambientHumidityPercent'],
				cToF(tempC),
				cToF(coolSpC),
				cToF(heatSpC),
				traits['sdm.devices.traits.ThermostatMode']['mode'],
				traits['sdm.devices.traits.ThermostatHvac']['status'],
				comms
				]
	
	system.tag.writeAsync(paths, values)

I am hardcoding my constant variables at the top of the script and anything else that could change in the future I’m writing to memory tags. I have all my Nest related memory tags kept in a folder at [default]Devices/Nest/, personally.

I update the values by simply using a gateway event timer script that runs once a minute that looks something like this:

try:
	devices.nest.getCurrentValues()
except:
	devices.nest.refreshToken()

So far, my script only handles reading values back from the Nest. Eventually I’ll bother with writing data back out to the API, but this is all I care to do at the moment while I fuss with connecting the rest of my smart things to this project.

3 Likes

Thanks for the detailed post @swolnisty. I'm starting to consider having a go at doing this (now that you've done all the hard work for me :laughing:). If I do, I'll post what I have (if it's not a complete rip-off of your script :wink:).

I know the feeling, I got another device hooked up and working this weekend only three more to go :sweat_smile: then a nice big garden automation/planning project to continue :smiley:

1 Like

Steal away! I posted my code here for just that. :slight_smile:

The more I look into connecting to my other smart devices, the more I wonder if there is a totally free way to connect to your Nest, maybe using an open source smart device management environment like Home Assistant or HomeBridge? Just hypothesizing, I have done zero research into either solution at this point.

1 Like

It looks like you still have to pay with Home Assistant.

I think I based this viewpoint on reading the Home Assistant forums when Google locked down their API's and starting charging for access. Although I have consumed far too much alcohol since then, so my memory is a little fuzzy :sweat_smile:

1 Like

I installed two ecobee3 lite thermostats in my house, and it looks like their API access is free. I setup my account as a developer and the access token stuff. I can use cURL to make use of their APIs now :slight_smile:

2 Likes