Automated Ops

Put down the GUI and step away from the Remote Desktop Connection

Well, maybe you don’t and you want to know why the hell I would. I’ve been building PowerShell tools for about 10 years and I’m pretty good at it. PowerShell is even available on Linux now. So, why Go? It’s freaking everywhere. Here are some of the FOSS tools I’ve been working with that are built on Go.
GitLab CI Runner Telegraf Packer Terraform Kubernetes A lot of these tools have a lot of open issues around the Windows build and I can’t contribute if I don’t understand Go and the community is huge.

An unhealthy obsession with numbers I like to watch numbers move which might have some relationship to being a successful operator of complex systems. Unfortunately it extends outside of looking for pattern in streams of monitoring data and I can get a bit obsessed watching Google Analytics for my blog or the download count for PoshSpec.
I have been keeping an eye on PoshSpec numbers in the Statistics page of the Gallery just hoping to crack the top 50.

In the beginning, there was PS C:\>_ I have been a regular PowerShell users for years. Probably about a decade now and I avoided customizing the shell on my workstation for the vast majority of that time. Despite PSRemoting being a thing for almost the entirety of PowerShell’s existence now. For one reason or another, I would usually end up RDPing into a server just to open PowerShell. I did not want to become dependent on modules like PSReadLine or console emulators like ConEmu for fear I would be handicapped when I had to use PowerShell on another system.

Introduction This is half of a tandem blog posting about NameIt. You can catch the other half over at Doug Finke’s Blog. If you’ve landed here first, take a few minutes to go read Doug’s post and get the full background on the history of this project. Doug started NameIt as a simple module to generate random strings based on a template, like ???##, which would give you three random characters and two digits.

Review In Part 2, we took a peek at the Zip method.
posh> [System.Linq.Enumerable]::Zip OverloadDefinitions ------------------- static System.Collections.Generic.IEnumerable[TResult] Zip[TFirst, TSecond, TResult](System.Collections.Generic.IEnumerable[TFirst] first, System.Collections.Generic.IEnumerable[TSecond] second, System.Func[TFirst,TSecond,TResult] resultSelector) Play That Func-y Music You may have heard a Func in C# called an anonymous function or a lambda.
In the MSDN example, it looks like this.
var totalCommission = quarterlySales.Zip(quarterlyRate, (first, second) => first * second).Sum(); In C#, the Zip method is a member of a Collection and in the above example, they are calling the Zip method of quarterlySales collection and passing in the quarterlyRate collection as well as the lambda expression.

Predicates Okay. So, we have this collection.
posh> $list Mike Bob Wendy Michele Which has a method called FindAll that takes a Predicate.
posh> $list.FindAll OverloadDefinitions ------------------- System.Collections.Generic.List[string] FindAll(System.Predicate[string] match) What is a predicate? WikiPedia simply puts it like this.
A predicate is a statement that may be true or false depending on the values of its variables
If you’ve written some SQL in your life, a Where clause is a predicate.

Sometimes, when you’re working with some data, you want a little bit more functionality or performance than you get from the run of the mill arrays. This is the first in a series of posts about [System.Collections.Generic.List].
Introductions First things first, Generic Lists are statically typed.
PS C:\Temp> $list = New-Object System.Collections.Generic.List[int] PS C:\Temp> $list.add(1) PS C:\Temp> $list.add("two") Cannot convert argument "item", with value: "two", for "Add" to type "System.

A couple of months ago I published a module to help reduce the amount of code you had to write to build infrastructure validation tests with Pester. Since then, I’ve tried to understand more about how people use Pester. Specifically, how they work with the framework to solve their problems. When you start to look around, the community effort to “Pester all the things” is much larger that I would have guessed.

Almost two years ago, I learned about a cool project that does some very clever string manipulation. I started messing around with Humanizer and extending existing Types to add some new properties that show off the functionality of Humanizer. Doug Finke started a project at the same time to do the same thing, but using functions. So, we combined our efforts to create one module, PowerShellHumanizer. I, personally, haven’t done much with it since then.

It’s been a busy few months. I recently changed jobs and one of my first projects is building some new custom DSC Resources. I chose to use Class-based resources, because Classes are cool. There were some bumps along the way, one of them being how do you run Unit Tests on such a thing.
Part of the reason I went with a Class-based resource was because it would be easy to test.

This is a list of books I have read over the years and found interesting and influential in understanding Operations, Business, and most importantly, People. Most of these books are not about Technical Operations or Computer Science, but about understanding complex systems, including how we think.
System Thinking Thinking in Systems: A Primer by Donella H. Meadows The Checklist Manifesto: How to Get Things Right by Atul Gawande (my review) Human Thought and Bias On Intelligence by Jeff Hawkins Thinking, Fast and Slow by Daniel Kahneman (my review) Quiet: The Power of Introverts in a World That Can’t Stop Talking by Susan Cain Business Organization The Goal: A Process of Ongoing Improvement by Eliyahu M.

Getting Started I recently read this article, Consul for Cluster Health Monitoring, and was interested if I could use this to execute PowerShell based health checks.
Consul has a Win32 build so it seemed reasonable despite there being no Windows examples in the documentation.
Here’s the command I ran to start up a Consul server. Ideally, you’d want a few of these in each physical site and then a Consul agent on each node.

I’ve seen a couple questions about cross-domain PowerShell Remoting online this week. I do this often and actually just recently finally started building some tooling around making it easier to work with on a daily basis.
http://www.reddit.com/r/PowerShell/comments/37nw55/crossdomain_remoting/
@LogicalDiagram @jsnover @almostjulian @abhik501 Can you point me to solid directions, and what WMF versions are required?
— Rob Nelson (@rnelson0) May 28, 2015 I referenced this Scripting Guys article which has some good information on remoting to an untrusted domain.

AKA: When a new blog post turns into yak shaving.
I’ve seen some questions online about PowerShell Remoting across different domains. It’s not a topic that’s well covered but there is some content available. I wanted to put up a quick post to help people find that content. Well, I’ve replaced my laptop since my last blog post and this one isn’t set up with the Ruby libraries necessary to work with Octopress.

I made a couple of changes to the function to support recursively stepping into directories. I started messing around with this snippet in order to reproduce the functionality of using the Windows OpenFile dialog with a one-liner. Now it’s a bit more than a one-liner, but the GridView UI is actually pretty useful if you don’t know exactly what file you’re looking for.

I found two great tools yesterday. Actually, that’s an grossly inaccurate statement. One of the tools I learned about at the PowerShell Summit earlier this year, but was just finally published to GitHub. The second tool isn’t really a tool, but a site indexing a bunch of tools, or frameworks, for generating static websites.

I have been sitting here staring at my computer for over two hours now. Essentially locked out of my system for a full disk virus scanned enforced by The Man (aka Corporate IT). The scan thrashes my slow laptop harddrive so much that opening even a simple web page requires a Buddhist monk’s level of patience. The job has a runtime limit of two and a half hours.
While watching the progress bar—it’s the only thing I can really do—it became clear that the scan would never inspect every file in the time limit. I opened up the log and confirmed there were a lot of scans that were terminated around 2.5 hours. Out of curiosity, I used PowerShell to do some basic analysis of this log.

I’ve been working with New Relic a lot in recent weeks and have deployed their agent several times. Obviously, this required some automation and the best way to automate software installs on Windows is Chocolatey. Unfortunately, New Relic didn’t publish their agents to Chocolatey and the existing packages were out of date.

I brought up the request on the user forum and New Relic expressed interested, but also said, “feel free to give it a try yourself.” So, I did.
Here is the .Net agent.

Humanizer helps you generate some nice output, but it’s kind of a pain to use in PowerShell. You can use Update-TypeData to make the syntax easier to work with. Then, your code starts to look more like the examples on the project’s site.

I finished this book a few weeks ago and have struggled with how to write a review. Thinking, Fast and Slow by Daniel Kahneman is the most epic self-help book I have ever read. It’s also different in the fact that there is no “do this or that to live a better life” type advise, but simply over 400 pages of examples of hour your brain can fail you in so many situations.
That lead probably doesn’t inspire you to pick up this thick tome and commit to absorbing the knowledge within, but this is seriously an extremely important piece of work. If you are a knowledge worker and consider solving problems a primary job function, there is no excuse to ignore this book.

This is a follow-up to my last post on using PSate for test automation. I demonstrated running Invoke-WebRequest and validating the StatusCode as a simple test for if a site is available. Now let’s take it to the next level and run an artificial transaction. Again, PSate is a handy test runner to manage this and the open-source project, WatiN(pronounced as What-in) is a great tool for automating a real browser.

Recently I have been putting effort into learning the art and science of unit testing using Pester and PSate. I was going to write up something on my experiences, but Jakub Jareš beat me to it with this”) good article on PowerShell Magazine. If you are new to the idea of unit testing, go read that article and then come back here.

So, what is the point of this article? I was working on an unusual maintenance recently and it was around 4 in the morning and it occurred to me that it would be nice if I had a big dashboard that showed everything was “green”. It can take 15 minutes or more for monitoring to spool up and do all it’s checks and what happens if you forget to re-enable monitoring after the maintenance. Your mind is completely unreliable after a 20 hour day.

I happened to look through all of the previous posts today and found that 2/25/2013 was my first. So, that makes a year and by no conscious effort, 52 posts—one a week on average. The site receives about 500 visits a month from over 40 countries. The most popular post, by far, has been get-storedcredentials-module.

If you are still not convinced, here is a bit more about the book and why I feel it’s a top read for anyone who thinks this blog is even slightly interesting. This is a paragraph from the Introduction.

I just recently found out about a fun project, PoSH Server that has actually been around quite a while. Executing PowerShell from a web page is a powerful idea for a Windows Administrator. Simply access a URL and get back some HTML formatted data—easy to run and consume the results. Just like everything involving PowerShell, there are a number of way to tackle this and I’ve tried quite a few including: PowerShellASP (now PowerShellServer) by /n Software; and Edge for NodeJS. The beauty of PoSH Server is that it’s written entirely in PowerShell which makes it easy to run and tweak. You don’t even need IIS installed. If you have PowerShell installed, install the module and run Start-PoshServer. That is all. Well, that’s enough to get you started. Where you go from there is the challenging and fun part.

The good folks over a Royal Pingdom posted an article today entitled “Powerful command line tools for network administrators.” Of course, by network administrator, they mean Linux admin. Here’s my version for you Windows admins. I guess you could call it “PowerShell command line tools for network administrators.”

If you’re doing security right, the credential you use to log into your workstation is not the same used to managed the Production environment. Because of that practice, I have to run Get-Credential almost every time I open up a new Powershell session and type in details manually. There are plenty of options for storing credentials securely and one has been standard in Windows for over a decade—Windows Credential Manager. Why has no one built a script for getting and setting credentials from this store? Well, it’s exposed through a Win32 API. For all my searching I could not find a .Net class for working with it or even an open source .Net wrapper. There’s just this example code. Fortunately, a few people have pieced together the interesting bits to get credentials out of the Credential Manager. I cleaned up the code a bit and made it a Script Module so it will auto-load when I type the alias gsc.

Yesterday, Microsoft revealed a new feature of Powershell v4 that is included in Server 2012 R2—Powershell Desired State Configuration. At first glance this looks to compete directly with Chef and Puppet, but during the presentation, OpsCode actually stepped in to present how they’ve already adopted the new technology into Chef. If you already use Chef and are familiar with the Ruby based DSL, you can continue to use it. System Center 2012 R2 is also built on top of this framework. It is the new way of thinking for Windows system configuration automation.

Here is an example configuration script. It only scratches the surface of the capabilities demoed by Kenneth Hansen and Jeffrey Snover at TechEd.

Basic command-line audio device control from Powershell including Nuget Package Manager Console.
On GitHub
Features: Set Volume and toggle Mute on the Default Playback Device. Get a list of devices and set the Default Audio Device.
Install. Download New-Item “$profile\Modules\AudioDeviceCmdlets” -Type directory -Force Copy CoreAudioApi.dll, AudioDeviceCmdlets.dll and AudioDeviceCmdlets.dll-Help.xml Import the binary module. This can go into your profile.
Import-Module AudioDeviceCmdlets You may need to set the execution policy.

I’m working on the finishing touches of a Deployment Automation script useing psake (pronounced like sakethe Japanese rice wine) as the framework for creating discreet jobs and enforcing dependencies.

PowerShell Pipeworks is a pretty interesting module for PowerShell. Most modules for Posh are focused on doing some discrete task(s). The design principle of Cmdlets is to make single function executables just like Unix shell commands.

I think Pipeworks is the first 3rd party framework I’ve seen developed on Powershell. It’s scripting glue you can use to build automation with Service Oriented Architecture principles. With PowerShell v3 and the whole Windows Management Framework it was packaged in, there is a new Powershell feature called Workflows. Workflows was designed and released also to solve the problem of orchestrating more complex sequences of logic.

Everyone wants to do SaaS these days and that includes Infrastructure Monitoring software. That’s a problem when half of your servers do not have internet connectivity and you’d like it to stay that way.

I’ve wanted to share some of my my experiences working in a Microsoft based SaaS environment and thought about blogging before, but I’m not really a writer. I finally started this site as an experiment to try technologies that I don’t normally get to try in my day to day professional experience.
The site has been up for a few days, no visitors, but maybe someone will read this someday.