Trying to prove that Skynet should be running on PowerShell!

Category Archives: Home Automation

Some background…
We’ve been buying our groceries online for a few years now. We find it super convenient and it saves us a lot of time. I even created a PowerShell module for it some time ago!

There is one (very minor) annoyance with it though, remembering to use the discount coupons you get after you’ve bought groceries for a certain amount. These coupons or codes get’s sent out before your current order has been delivered which means that you cant add them for your next order (can’t reach checkout while you have an active order waiting for delivery).

This means I have to wait for my order to be delivered and then add to it at the checkout step for my next order, at which point I’ve forgot all about it and maybe even deleted/archived the e-mail containing the pdf-file with the coupon.

I thought of this as the perfect scenario to check out a (relatively) new service from Microsoft called Flow, the idea behind Flow is to make it simple to automate things without the need of writing any code, but that doesn’t mean you can’t do that as well 🙂

How to achieve this?When building automation I usually try to write down the steps needed to achieve the “end-to-end automation”. In this case that would be:

Make sure the e-mails containing the coupons can be found automatically

Get the coupon from the e-mail moved somewhere where it can be accessed by a PowerShell runbook in Azure Automation

Create a PowerShell function that can parse pdf-files so the code inside can be retrieved

Create another PowerShell function that can post the code to the online grocery store

Profit! 🙂

These steps have now been achieved, and here’s how I did it:

Fetching the E-mail and the attachments (Step 1 and 2)
This is amazingly simple using Microsoft Flow. After you’ve signed up and logged in, just go to “My Flows” and click “Create from template”. There are quite a few to pick from so the easiest way to achieve this is to use the search function at the top of page, since I’m using Outlook.com as my personal e-mail provider, and thought the simplest way to store the attachments was using blob storage, I simply searched for “outlook blob” and found these templates I could use:

In my case, the first one fits perfectly so that’s the one I chose as a starting point. Click on it, pick “choose this template” and first connect your Azure storage account (needs to be created in advance):

Then connect your e-mail account by logging in:

If everything worked, you can go on and press “Continue”

You’ll then arrive at the page where you can configure the different steps in your flow, and if you want to, add some conditions. After you’ve clicked “edit” on both steps and updated them they should look something like this:

As you can see, I changed the folder this flow should look in from “Inbox” to “Flow” to prevent it from harvesting all the attachments I receive. I can then simply add a mail rule to put the e-mails I want in that folder.

Same thing for the “Create file”-step, “mailattachments” should correspond to a container on your storage account.

That’s it for parsing the e-mails. If you would like to, you could also add a http request after these steps to trigger the runbook automatically (webhook) as soon as a new attachment has been saved to the blob storage, but in this case, I’ll just schedule that to run at a regular intervall.

Parsing the pdf-file and posting the discount code (step 3, 4 and 5!)
To be able to get text out of the pdf-file I used the iTextSharp library. Then wrap that up in a PowerShell function, which in it’s simplest form might look something like this:

Time to schedule that in Azure Automation, and make sure all the modules needed are available for it when it runs! (I run this on a hybrid worker)

Conclusion
While I have had a few issues with Microsoft Flow along the way (it is still in preview after all), it seems like a really cool service. And since you can make a http request to a webhook in Azure Automation, and/or just integrate them through some other service like the blob storage example in this post, the possibilities are pretty much endless.

Can you automate anything with Azure Automation?
While there are some limitations on what it can and cannot do, I thought I could have a bit of fun using some of the fairly new features in Azure Automation to show that even though the main purpose of this service is to automate management tasks in the cloud (and in your local datacenter using hybrid workers), since it’s built on PowerShell, there really isn’t that much you cannot do.

I’ve been trying to come up with a scenario that is a bit of fun, and at the same time shows how you can use features like hybrid workers and webhooks to overcome almost any obstacles you have when automating something that spans over different services and locations.

This is what I came up with:Let’s say I’m on my way home from work, it’s autumn and the rain is pouring down. I feel tired, cold, and just want to come home and grab a nice cup of coffee. To detect if something is running out of resources (in this case me), and to trigger something that can fix it (in this case, coffee) is a pretty common scenario in IT.

And to simulate that a process like this might span over multiple services where some are in the cloud (in this case iCloud, twitter and a weather service), while others are in your local datacenter (or in this case the coffee brewer in my kitchen), we are going to run parts of the code in Azure Automation and parts of it on a hybrid worker, everything orchestrated with Azure Automation.

The steps involved, at a high level, will be:

Fetch the location of my phone through iCloud

Use that information to fetch the weather data at that location, and check if it’s going to rain

If rain was detected, start with sending out a tweet asking if I would like some coffee, if the reply is positive, brew some coffee.

If no reply to the tweet is detected, it will send out an e-mail with a link to a page where the coffee brewer can be started with a button (that calls a runbook through a webhook)

So let’s get started!

So, where am I? And how’s the weather?
I don’t work at the same place everyday, so I don’t want to hard code the location where the weather is checked. Wouldn’t it be great if I could just fetch this information dynamically somehow? Well, with Azure Automation and some PowerShell, I can.

Since I carry around a smartphone with a GPS all day long, I thought that would be a good source for location details, and since you can fetch your location information through iCloud when you have an iPhone, this was the method I chose to do it.

Disclaimer: This code is for educational purposes ONLY, I do not take any responsibility if you use this outside of the ToS for the different services utilized here.

So I started with creating a PowerShell-function that could fetch my phones location through iCloud, if you want to take a look at it, it’s available here.

We then need to fetch some weatherdata at that location, luckily, I’ve already built a function like that before, blog post available here.

So, I have the tools to fetch my current location and the weather at that location. But how do to use this in Azure Automation?

Importing custom modules in Azure Automation
This is actually really simple! You can import almost any PowerShell module into Azure Automation, as long as you zip it up in folder with the same name as your module file. So I took my two functions above and put them into a WebUtilities.psm1-file. I then put that file into a WebUtilities-folder, and finally zipped it all up as “WebUtilities.zip”. If you want to learn more about how to create integration modules for Azure Automation, including creating an optional file containing information about a Azure Automation connection-variable, more information about that is available here.

We then need to import this into Azure Automation. The screenshots that follows are from the “classic portal”, but you can do this in the preview portal as well:

First find the automation account you want to use, go to assets, and then click “Import Module” at the bottom:

Browse to your zip-file and click open to select it and press “Complete” down in the right corner:

Azure Automation will then begin to import the module and extract the activities it contains, you can follow the process at the bottom of the page:

These functions are now available in our PowerShell Workflows and PowerShell runbooks. Neat huh?

(The custom modules you import will not, at the time I’m writing this, be pushed to your hybrid runbook workers automatically. The Azure Automation team is working on that though, so it will happen eventually. In the meantime, you need to do this yourself.)

Writing the code…
It is now time to use the functions and actually write the code needed to tie everything together. There are many cool new features regarding Azure Automation but one of my favorites are the PowerShell ISE AddOn the Azure Automation team is working on, if you work with Azure Automation I can’t recommend you to check out the GitHub repository for it enough, and ever since I did the build straight from the source it has been working pretty well considering it’s still a very early release.

This is how my setup looks (ISESteroids, another great product, is also used here):

In addition to enabling you to use all of the features of the PowerShell ISE (and ISESteroids if you use that), this AddOn enables you to for example; fetch your runbooks straight from Azure, upload changes, run the code locally with emulated activities, test the code in Azure, and manage your assets so they are available when you test the code locally.

The productivity boost you get from this in comparison to the text authoring and testing experience in the portal, at least in my experience, is huge. So go ahead and try it out!

So, back to the code itself. As stated above, the steps involved here will be:

Fetch the location of my phone through iCloud

Use that information to fetch the weather data at that location, and check if it’s going to rain

If rain was detected, start with sending out a tweet asking if I would like some coffee, if the reply is positive, brew some coffee.

If no reply to the tweet is detected, it will send out an e-mail with a link to a page where the coffee brewer can be started with a button (that calls a runbook through a webhook)

And since the PowerShell community is so awesome, this is a pretty common scenario aswell, you build a few functions of your own, and you find some from others. Just zip it up and import it in the same way as the above functions. To use the MyTwitter-module, you also need to add API keys, just follow Adam’s instructions and you’ll be fine!

If you have read some posts at this blog before, you probably know that I enjoy creating home automation scripts quite a lot, and I’ve named this little project Jarvis after the famous AI, the ‘JarvisGroup’ specified above (Start-AzureAutomationRunbook cmdlet) is the hybrid worker group that runs some of these scripts. If you want to learn more about hybrid runbook workers and how to deploy them, check out this link.

Currently, you can’t use webhooks to trigger runbooks on a hybrid worker, as a workaround, I have another runbook that uses the Start-AzureAutomationRunbook cmdlet to trigger it on the hybrid worker instead, the code of that looks like this:

The module containing the Connect-TelldusLive and Set-TDDevice cmdlets are installed on the target hybrid worker since that’s where it will execute (and as stated above, the module won’t be pushed out to hybrid workers automatically from Azure Automation even if you have imported them there, but that will be fixed in the future).

So, we’re all set now…

But, does it all work?
Well, you’d obviously have to come by for coffee some time to see this for yourself, but yes, it actually does! 🙂

The code for that form with the token masked (be aware that posting a form like this on a public website without authentication is a MAJOR security risk depending on the runbook type, it’s only for demo purposes in this case):

Summary
I hope this post have helped you to see how flexible Azure Automation actually is. PowerShell is truly versatile and a great “glue-language” to tie different services together. Even though using Azure for turning on a coffee brewer might be a bit overkill, if it’s possible to integrate a weather service, an iPhone, e-mail, twitter and a coffee brewer using it, it can probably manage your IT environment aswell, don’t you think? 🙂

Remember the post about buying groceries with PowerShell? One of the usage examples suggested that you could order some snacks when a tv show you like is having a season finale (or premiere for that matter!).

But how to check for this in an automated way? With PowerShell of course 🙂

I’ve written two advanced functions for doing this:

Get-TVShowNextAirDate retrieves a list of most tv shows that are currently airing and is listing their next airdate.

Get-TVShowAirDate retrieves all known airdates of the tv show specified (supports pipelining! 😉 )

The first thing that comes to mind when doing home automation is probably controlling lights and sockets depending on if someone is home or not. Most apps I’ve seen for this can’t handle multiple family members though, which is quite useless. You don’t want to turn on all the lights when you come home if someone else is sleeping or watching a movie with dimmed lights for example.

So how to figure out who is currently at home?

The method I chose was to check if our phones are connected to WiFi. There are a few obstacles to overcome before this works though, for example:

Many smartphones turn off WiFi to save battery (for example the iPhone)

They use dynamic IP-addresses and name resolution is not always available, so how to find them?

The script therefor checks for how long a phone has been offline before it does any changes, and it starts off by doing a pingsweep to populate the arp-table to find the different phones MAC-addresses.

This scripts updates the status of each phone in a file on disk and also changes a device in Telldus Live! to On/Off. That makes it possible to filter events in Telldus Live! based on who’s home, or when the last person leaves.

I’ve scheduled it to run quite often to always check if someone has arrived. Hopefully the comments in code will give you the help you need, if not, post a comment below and I’ll be glad to help you!

The csv file ($FamilyMemberFile, C:\TelldusScripts\FamilyMembers.csv) used in the script above should have the columns Name, DeviceName, DeviceMAC, WiFiSubnet, StatusFile, DeviceID.

Name = The name of the person who owns the device
DeviceName = The DNS name of the device. iPhones use “DeviceName.local”
DeviceMAC = MAC-address of the devices, use “-” as a separator.
WiFiSubnet = The subnet where the phone gets its IP-address.
StatusFile = The path to a file unique for this phone where the status can be written (if it’s home or not)
DeviceID = The device id in Telldus Live! that is used to filter if this person is home.

How? Well, you need to find someone who sells groceries through a website, and can deliver them to your house.

The rest is just webrequests!

This is not only (but maybee mostly… 😉 ) made as a part of my weird quest for doing strange things with PowerShell, but also a small part of my home automation project. I’ll give you some usage examples:

Let’s say the weather-cmdlet posted earlier gives you reports about rain, wind and cold weather for the coming week, how about adding some popcorn to your basket and other things you want for a nice movie-night at home?

I just started on a new part of my home automation project, controlling my receiver (Onkyo TX-NR616).

I thought that this might be a good opportunity to do a “walk-through” on how to create a script for something that clearly was not meant to be controlled with PowerShell, in this case a surround receiver.

The basic methods and principles used here should be applicable to more or less anything you want to automate and I hope someone might find it useful!

Small disclaimer:
I take no responsibility in how you use the information in this post, make sure you know what you are doing! Even though any damage to a receiver or other equipment should be unlikely, I wont give you any guarantees…

So… The steps involved are basically:

Figure out how the item (website, device, software…) is controlled. Does it have a documented API? Is it using REST? Website? Mobile app?

Figure out how to utilize this with PowerShell. If it is a REST-based API, you are lucky, websites or webservices are also highly automatable (always check ToS before doing it though…), if it has a mobile app or similar, it most definitely is using some sort of API which may or may not be easy to figure out.

I usually build a simple script to start doing some testing. Once I figured out how the API works, I turn that into an advanced function.

Enjoy doing yet another thing with the blue console of wonder!

I will now try to describe how I used this basic approach to control my receiver!

So, how is it controlled?
In this case, I knew there was a mobile app available, and therefore, there must be some kind of API. After some research I found out that they are using something called “Integra Serial Communication Protocol”, or eISCP as it is called when used over an ethernet connection.

But what if we didn’t have any documentation? Or if we just want to use a few commands (like power on/off), do we really need to learn the complete documentation?

That depends, sometimes you can just “copy -> paste” the commands from a trace. That will be described under the next part of this post; “Using a network trace”.

Using a network traceDocumentation obviously helps a lot, but if we pretend there was none available, should you just give up? Of course not! 🙂

One approach if that is the case is to basically just sniff the network traffic. You can do this with whatever tool you prefer, two free examples are Wireshark and Microsoft Network Monitor.

If you can find a windows application for the device, that helps since you can just sniff your local network traffic. Another alternative if you want to stick to a windows computer but all you can find to control your devices is a mobile app, you could try to run routing and remote access (or internet connection sharing) to just make the traffic go through your PC, which then acts as a router. After that, your network analyzer/sniffer should pick up the traffic, which hopefully will not be encrypted (there are ways of solving that too, but we’ll stick to the simpler scenarios here…).

When you got this ready, just start your sniffer and your application used to control your device, and record whatever command you want to be able to automate.

This is a screenshot of a Wireshark trace recording a “power on” command sent to the Onkyo receiver:

Right click the first package sent to receiver and choose “follow TCP stream”. That makes it a bit easier to see what is happening, it looks like this:

The colouring makes it easier to see what gets sent to the receiver and what gets sent back, the first command (light red colouring above) shows what the app sent to turn on the receiver, the dots just represent “non printable characters”, so the command looks somewhat like “ISCP !1PWR01” which does seem like a “power on” command!

Now select “C Arrays” instead of Hex Dump, select the package you want and copy the text, like this:

You now have the whole command needed to turn the device on! Since what you will be sending is byte arrays and not HEX, you need to convert the HEX to Bytes instead. To do this, open a PowerShell prompt, paste the text you copied, add a pipe and write “% { [BYTE]$_ }”, press enter two times and they are converted! Should look like this:

Those bytes are what you need to send using the Send-method of System.Net.Sockets.Socket. More on that follows under the next part of this post!

Sending TCP streams with PowerShellWe now know what we should send, but how?
There are many examples around on how to do this, I will post one that at least I thought was fairly straight forward.

The code looks like this: (Credit to this thread for getting me started!)

And the receiver turns on! This code lacks quite a lot, like a method of reading the reply, closing the socket, error handling etc., but it will hopefully give you an idea on how to tackle something like this! If you want something slightly more polished, check the last part of this post…

But I want to do a lot of different things, not just turn it on/off!
I will fix a module for controlling Onkyo receivers as soon as I get more time, in the meantime I have made a “quick and dirty” advanced function that allows you to add the command to a parameter. To give you an idea:

If you look in the documentation I linked to above, you will probably figure out the commands for more or less anything. I have tested this on my receiver to for example use Spotify, change volume, change inputs and so on…

The module will probably include something like this function to enable a wider range of commands, but will also have cmdlets prepared for powering on, selecting different inputs (including subcategories under “Network”).

This function should just be considered an example on how you can do something like this with PowerShell, and was made since most of you might not be using Onkyo-receivers but still would like some sample code.

As a part of my home automation project I wanted to be able to check the weather, and since I’m trying to do as many parts of the automation as possible with PowerShell, I needed to write a cmdlet for this. But where to get the data?

I was very happy to find out that the Swedish Meteorological and Hydrological Institute (SMHI) actually has a published API, open and free to use!

The API is very simple, after you send a request with longitude and latitude it sends back a json-object with weather data for that location (tagged with the location of the closest weather station used for the report) for up to 10 days.

Note: As a visitor pointed out in the comments, SMHI does not have weather data for all countries, I’ve tested a couple of locations in Europe which seems to work fine though. You will get a “error 400 bad request” error if the location is out of scope.

The cmdlet does some small changes to the returned object (expands a few properties and changes some shortnames to something that is easier to understand, converting some values to percentages and so on…) and returns each forecast as a separate object. The forecasts can cover anything from 1 hour to a day or so, so make sure you check the start/end dates.

The main difference is that some executing cmdlets dont generate output if you dont use “-Verbose” and that authentication is set up by the “Connect-TelldusLive”-cmdlet (Which uses a PSCredential). That means that the Username and Password parameters are removed from all other cmdlets. This makes the usage a lot simpler and also a lot quicker.

A screenshot to get you an idea on how to use it: (Device names are in Swedish, sorry…)

This module is now published at the PowerShell Gallery. You can get the latest version at this link.

There are certainly other solutions out there which are great, even excellent. For me personally, it’s mainly because it’s fun to be able to build parts of it by yourself, you learn a lot by doing it, but you can also base your tasks on almost any piece of information out there.

I’ll give you an example!

I’ve started to go to work after the traffic calms down in the morning, I have a pretty good idea of when this usually happens, but sometimes there is no traffic jams at all, and sometimes it’s completely hopeless.

Wouldn’t it be nice to be able to utilize live traffic information on the internet, and based on that trigger your wake up call? At least I thought so 🙂

First of all, try to find a provider for traffic information near you, and make sure you don’t break their ToS by fetching that information in a automatic way (ie. web scraping).

I won’t go into detail on how to build a webscrape-cmdlet right now, but I’ll show you how to use it when it’s done. (A guide to web scraping available here, here and here.)

This is the script I run every morning, I think the code comments will be enough to explain how it works:

# Import the Telldus module
Import-Module '.\Telldus.psm1'
# Import the Module containing your traffic parser
Import-Module '.\WebDataModule.psm1'
# Set your home and work address
$HomeAddress="Homeroad 1, MyTown"
$WorkAddress="Workaround 1, WorkTown"
# Set a max traveltime limit (in this case, in minutes)
[int] $TravelTimeLimit = 30
# I want it to be under this value for $NumberOfTimes consecutive times
[int] $NumberOfTimes = 3
# Make sure it does'nt get $True on first run
[int] $CurrentTravelTime = $TravelTimeLimit+1
# Reset variable to zero
[int] $NumberOfTimesVerifiedOK = 0
# Run until the traveltime limit has been passed enough times
while ($NumberOfTimesVerifiedOK -lt $NumberOfTimes) {
# Reset variable
$CurrentTravelTime = $null
# Load new data, the "Get-Traffic"-cmdlet is my traffic parser
[int] $CurrentTravelTime = Get-Traffic -FromAddress $HomeAddress -ToAddress $WorkAddress | select -ExpandProperty TravelTime
# Check if it is below your traveltime limit, and that it is not $null (cmdlet failed)
# Increase $NumberOfTimesVerifiedOK if it was ok, or reset to zero if it wasn't
if ($CurrentTravelTime -ne $null -AND $CurrentTravelTime -lt $TravelTimeLimit) {
$NumberOfTimesVerifiedOK++
}
else {
$NumberOfTimesVerifiedOK = 0
}
# Write current status
Write-Output "Traffic has been verified as OK $NumberOfTimesVerifiedOK consecutive times"
# Pause for a while before checking again, 10 minutes or so...
Start-Sleep -Seconds 600
}
# The while loop will exit when traveltime has been verified enough times.
# Write status
Write-Output "Initiating sunrise, current travel time to $WorkAddress is $CurrentTravelTime minutes, and has been below $TravelTimeLimit for $NumberOfTimes consecutive times."
# Time to initiate the "sunrise effect"
# Set the device id for the lamp you want to light up
$BedroomLampDeviceID="123456"
# Set start dimlevel
$SunriseDimlevel = 1
# Set how much it should increase everytime we "raise" it
$SunriseSteps = 5
# Set your Telldus credentials
$Username="[email protected]"
$Password="MySecretPassword"
# Kick off the "sunrise-loop"
while ($SunriseDimlevel -lt 255) {
# Write some status
Write-Output "Setting dimlevel to $SunriseDimlevel"
# Set the new dimlevel
Set-TDDimmer -Username $Username -Password $Password -DeviceID $BedroomLampDeviceID -Level $SunriseDimlevel
# Sleep for a while (30 seconds makes the "sunrise" ~30 minutes long depending on your $SunriseSteps value)
Start-Sleep -Seconds 30
# Set the next dimlevel
$SunriseDimlevel=$SunriseDimlevel+$SunriseSteps
}
# Set the lamp to full power (loop has exited) and exit
Set-TDDimmer -Username $Username -Password $Password -DeviceID $BedroomLampDeviceID -Level 255
Write-Output "Maximum level is set."

This script is scheduled to run in the morning (on week days) around the earliest time I want to go up, the first loop will run until traffic calms down, and then start the “sunrise”-loop which will run until the light reaches its maximum level (255).

You could of course turn on other stuff as well, like a coffee brewer (make sure you don’t do this while you are away…), a radio, play some music or something else.

Since we moved to an apartment with a balcony we wanted to have some plants there. The problem is that we’re a lot better at killing plants than actually taking care of them. And even the few days when we actually did remember to water them, the plants could dry out on hot days.

So, how to solve this? With PowerShell of course! 🙂

You could start of with the Telldus PowerShell Module from this blogpost (though, this one is preferred since it handles credentials a lot better). The module is under development but works for checking temperatures, dimming or turning on/off devices etc.

You also need something to water with, I went with a pump from Gardena which runs for 1 minute when turning it on, or 1 min/24h if turned on constantly. This means that switching it off/on will start a watering session. The waterflow is controlled by using different outlets from the pump.

After adding the devices to Telldus Live! you can just open up your favorite powershell editor, import the module and start coding. (The module requires at least PowerShell v3)

An example of how it can be done follows. (The below module has been updated, make sure you check out the new version that uses a PSCredential among other things!)

First import the module, set the credentials and some variables needed: (passthrough for credentials (using “secure string”) is on the ToDo-list)

Import-Module C:\Scripts\Telldus\Module\TelldusModule.psm1
$Username="[email protected]"
$Password="MySecretPassword"
# At what temperature (and above) should I start the pump?
$WhenToWaterTemp=25
# If the humidity get's below this value, I will also start the pump
$WhenToWaterHumidity=40
# How old could the sensor data be before I'll consider it invalid? (in hours)
$DataAgeLimitInHours=2
# Time zone difference compared to telldus servers.
$TimeZoneDifference=2

You don’t need to check for how old the sensor data is, but these sensors can sometimes stop updating, and if they do when it’s hot outside, you might end up drowning your plants if you schedule the script to run often.

That’s basically it! I added some functionality for doing push-notifications to my phone (Growl API and Boxcar works great for this). Or you could just send an e-mail (you probably want to know if the sensor data isn’t getting updated, for example).

Hope this gives you an idea on what you can do with PowerShell and home automation.

I will try to add other things that I’ve automated with this module when I get the time!