Over the years I’ve learned about many of the advantages of using a modern terminal and shell. I’m talking about using bash with GNOME terminal on a modern GNU/Linux distribution.
I particularly like switching between the terminal and GUI applications. It’s now even better.
Automatic Hyperlinks:
Occasionally, some program running in your shell might output what looked like a link:
gnome terminal and vte will automatically add links to text that looks like a web hyperlink or an email addressThey can be seen when you hover over them, and a right-click will display a useful context menu:

I’m old school cool. Real hackers chat on open, distributed platforms. Most technical discussion can be found on the Freenode IRC network. It’s not perfect, but the advantages clearly outweigh the drawbacks.
Recently, I needed to join an existing large “community” on the centralized, proprietary walled garden that is the Slack network.
The Problem:
Connecting to the the Slack server requires that you use either the proprietary client or their proprietary web app.

I’ve been working on lots of new stuff in mgmt and I had a synchronization problem that needed solving… Long story short, I built it into a piece of re-usable functionality, exactly like you might find in the sync package. For details and examples, please continue reading…
The Problem:
I want to multicast a signal to an arbitrary number of goroutines. As you might already know, this can already be done with a chan struct{}.

I’ve spent about four years at Red Hat, and now it’s time to move on…
TL;DR: had to leave Red Hat and start Patreon to fund mgmt.
What follows is a bit of historical rambling, and some forward looking statements.
Retrospective:
Long-time readers of my blog will know that I was very active in the puppet ecosystem for many years. I learned a lot while writing puppet code, and while building some of my outrageous puppet hacks.

Sometimes while I’m deep inside mgmt’s project directory, I want to run an operation from the Makefile which lives in the root! Unfortunately, if you do so while nested, you’ll just get:
[email protected]:~/code/mgmt/resources$ make build make: *** No rule to make target 'build'. Stop. The Ten Minute Solution:
I figured I’d hack out a quick solution. What I came up with looks like this:
#!/bin/bash # James Shubin, 2018 # run `make` in the first directory (or its parent recursively) that it works in MF='Makefile' # looks for this file, could look for others, but that's silly CWD=$(pwd) # starting here while true; do if [ -e "$MF" ]; then make [email protected] # run make!

Mgmt has given us a lot of great features, demos, and insight into how infrastructure automation should be managed. Unfortunately it wasn’t a complete tool yet, because it was missing a crucial piece for its completeness. That all changes today with the addition of: the language.
An ideal language allows us to express easily what is useful for the programming task, and at the same time makes it difficult to write what leads to incomprehensible or incorrect programs.

As you may have noticed, we recently re-licensed mgmt from the AGPL (Affero General Public License) to the regular GPL. This is a post explaining the decision and which hopefully includes some insights at the intersection of technology and legal issues.
Disclaimer:
I am not a lawyer, and these are not necessarily the opinions of my employer. I think I’m knowledgeable in this area, but I’m happy to be corrected in the comments.

Recently, while operating two different remote-controlled appliances, I realized that it was high time for a discussion about declarative and imperative paradigms. Let’s start by looking at the two remotes:
Two different “remotes”. The one on the left operates a television, and the one on the right controls a central heating and cooling system.At first glance you will notice that one of these remotes is dark, and the other is light.

LaForge and the fine folks at Osmocom (Sysmocom) recently had a femtocell giveaway. I didn’t expect to have much time to hack on things, but they were still quite generous in sending me one. It arrived, and I took some un-boxing photos for anyone who is curious.
A box arrived in the mail…Which recurses into an inner box…Inner box is box like.Finally… The unit is displayed.Here it is in all its glory.

In mgmt we have meta parameters. They are similar in concept to what you might be familiar with from other tools, except that they are more clearly defined (in a single struct) and vastly more powerful.
In mgmt, a meta parameter is a parameter which is codified entirely in the engine, and which can be used by any resource. In contrast with Puppet, require/before are considered meta parameters, whereas in mgmt, the equivalent is a graph edge, which is not a meta parameter.

I’ve been hacking in golang since before version 1.4, and the speed at which my builds finished has been mostly trending downwards. Let’s look into the reasons and some fixes. TL;DR click-bait title: “Get 4x faster golang builds with this one trick!”.
Here are the three reasons my builds got slower:
The compiler
Before version 1.5, the compiler was written in C but with that release, it moved to being pure golang.

I’m old school and still rocking an old X220 laptop because I didn’t like the new ones. My battery life isn’t as great as I’d like it to be, but it gets worse when some “webapp” (which I’d much rather have as a native GTK+ app) causes Firefox to rev my CPU with their websocket (hi gmail!) poller.
This seems to happen most often on planes or when I’m disconnected from the internet.

I previously published “A revisionist history of configuration management”. I meant for that to be the intro to this article, but it ended up being long enough that it deserved a separate post. I will explain Send/Recv in this article, but first a few clarifications to the aforementioned article.
Clarifications
I mentioned that my “revisionist history” was inaccurate, but I failed to mention that it was also not exhaustive! Many things were left out either because they were proprietary, niche, not well-known, of obscure design or simply for brevity.

I’ve got a brand new core feature in mgmt called send/recv which I plan to show you shortly, but first I’d like to start with some background.
History
This is my historical perspective and interpretation about the last twenty years in configuration management. It’s likely inaccurate and slightly revisionist, but it should be correct enough to tell the design story that I want to share.
Sometime after people started to realize that writing bash scripts wasn’t a safe, scalable, or reusable way to automate systems, CFEngine burst onto the scene with the first real solution to this problem.

Bootstrapping a cluster from your laptop, or managing machines without needing to first setup a separate config management infrastructure are both very reasonable and fundamental asks. I was particularly inspired by Ansible’s agent-less remote execution model, but never wanted to build a centralized orchestrator. I soon realized that I could have my ice cream and eat it too.
Prior knowledge
If you haven’t read the earlier articles about mgmt, then I recommend you start with those, and then come back here.

The mgmt config project got a logo! The full commit is here. Thanks to Sarah Jane Cox for creating it.
Happy Hacking,
James
PS: I might have a few stickers to give out too! Ask me next time you see me if you’d like one! Alternatively, use the artwork to make your own and share with your friends!

All good sysadmins eventually learn about using tail -F to tail files. Yes upper-case F is superior.
Around the time I wrote that article, I remember wanting to stream dmesg output too! The functionality wasn’t available without some sort of polling hack, but it turns out that kernel support for this actually landed around the same time in version 3.5.0!
Most GNU/Linux distros are probably running a new enough version by now, and you can now dmesg –follow (or dmesg -w):

The Technical Blog of James was seen on an outdoor electronic display in downtown Montreal! Thanks to one of my readers for sending this in.
I guess the smart phone revolution is over, and people are taking to reading my articles on bigger screens! The “poutine” is decent proof that this is probably Montreal.If you’ve got access to a large electronic display, put up the blog, snap a photo, and send it my way!

I was stuck on a 14 hour flight last week, and to my disappointment, only one of the two headphone speakers were working. The plane’s media centre has an audio connector that looks like this:
Someone should consider probing this USB port.The hole to the left is smaller than a 3.5mm headphone jack, and designed for a proprietary headphone connector that I didn’t have, and the two holes to the right are part of a different proprietary connector which match with the cheap airline headphones to provide the left and right audio channels.

In mgmt, deploying and managing your clustered config management infrastructure needs to be as automatic as the infrastructure you’re using mgmt to manage. With mgmt, instead of a centralized data store, we function as a distributed system, built on top of etcd and the raft protocol.
In this article, I’ll cover how this feature works.
Foreword:
Mgmt is a next generation configuration management project. If you haven’t heard of it yet, or you don’t remember why we use a distributed database, start by reading the previous articles:

I’m thrilled to tell you that I’ll be speaking about mgmt in Hong Kong and South Africa. It will be my first time to both countries and my first time to Asia and Africa!
In Hong Kong I’ll be speaking at HKOSCon2016.
In South Africa I’ll be speaking at DebConf16.
I’m looking forward to meeting with many of the hard-working Debian hackers, and collaborating with them to build and promote excellent Free Software.

I have a GNU/Linux server which I mount a few LUKS encrypted drives on. I only ever interact with the server over SSH, and I never want to keep the LUKS credentials on the remote server. I don’t have anything especially sensitive on the drives, but I think it’s a good security practice to encrypt it all, if only to add noise into the system and for solidarity with those who harbour much more sensitive data.

In this post, I’ll tell you about the recently released “automatic grouping” or “AutoGroup” feature in mgmt, a next generation configuration management prototype. If you aren’t already familiar with mgmt, I’d recommend you start by reading the introductory post, and the second post. There’s also an introductory video.
Resources in a graph
Most configuration management systems use something called a directed acyclic graph, or DAG. This is a fancy way of saying that it is a bunch of circles (vertices) which are connected with arrows (edges).

It’s been two months since I announced mgmt, and now it’s time to continue the story by telling you more about the design of what’s now in git master. Before I get into those details, let me quickly recap what’s happened since then.
Mgmt community recap:
I gave the first public presentation about mgmt at CfgMgmtCamp. I repeated the talk at DevConf.cz. The video recording is available. Felix wrote about his work cross compiling puppet code to mgmt.

On today’s issue of “one hour hacks”, I’ll show you how you can stop your git drive-by’s to git master from breaking your CI tests… Let’s continue!
The problem:
Sometimes I’ve got a shitty one-line patch that I want to push to git master. I’m usually right, and everything tests out fine, but usually isn’t always, and then I look silly while I frantically try to fix git master on a project that I maintain.

I’ve been writing a lot of golang lately. I’ve hit painful problems in the past. Here are some debugging tips. Hopefully they help you out. I bet you don’t know #2.
#0 Use log.Printf:
This should go without saying, but I’m ashamed to say it’s what I use the most. We’ve only been C programming for 44+ years, and it’s still what is most useful!
#1 Use go run -race:

It’s no secret to the readers of this blog that I’ve been active in the configuration management space for some time. I owe most of my knowledge to what I’ve learned while working with Puppet and from other hackers working in and around various other communities.
I’ve published, a number, of articles, in an, attempt, to push, the field, forwards, and to, share the, knowledge, that I’ve, learned, with others. I’ve spent many nights thinking about these problems, but it is not without some chagrin that I realized that the current state-of-the-art in configuration management cannot easily (or elegantly) solve all the problems for which I wish to write solutions.

I’ve got a few upcoming speaking engagements. If you’ll be attending one of these events, come see me or any of the other excellent speakers!
Please remember to check the official schedules in case there are any changes!
I’ll be speaking at the Brussels CentOS Dojo: Automated Infrastructure Testing with Oh-My-Vagrant …and the CentOS CI Time/date unconfirmed: I’ll be showing some CI tricks, and showing you how the CentOS CI is the perfect CI for multi-machine test environments.

Daniel P. Berrangé wrote about trying out a single node ceph cluster. I decided to take his article and turn it into an Oh-My-Vagrant omv.yaml file. It took me about two minutes to do so, and two hours to debug a problem caused by something I had broken on my laptop.
If you’d like to replicate his article in less than 5 minutes, pull down the omv.yaml file that I’ve just published and run omv up.

We’re constantly clicking on all sorts of different URL’s throughout the day. These clickable links appear in webpages (including in “web apps” like gmail) in mail clients like Evolution, in terminals such as GNOME-terminal, and any other GTK+ app on your GNU/Linux desktop. I wanted to perform custom actions when arbitrary URL’s are clicked, including running certain links in separate Firefox profiles. There are a bunch of different steps you have to do to get this working, but it should be easy to follow along.

The Oh-My-Vagrant project became public about one year ago and at the time it was more of a fancy template than a robust project, but 188 commits (and counting) later, it has gotten surprisingly useful and mature.
[email protected]:~/code/oh-my-vagrant$ git rev-list HEAD --count 188 [email protected]:~/code/oh-my-vagrant$ git log $(git log --pretty=format:%H|tail -1) commit 4faa6c89cce01c62130ef5a6d5fa0fff833da371 Author: James Shubin <[email protected]> Date: Thu Aug 28 01:08:03 2014 -0400 Initial commit of vagrant-puppet-docker-template... This is an attempt to prototype a default environment for vagrant+puppet+docker hacking.

My employer keeps paying me, which I appreciate, so it’s good to spend some time to make sure RHEL7 customers get a great developer experience! So here’s how to make vagrant, vagrant-libvirt and Oh-My-Vagrant work on RHEL 7+. The same steps should work for CentOS 7+.
I’ll first paste the commands you need to run, and then I’ll explain what’s happening for those that are interested:
# run these commands, and then get hacking!

I am definitely not an RPM expert, in fact, I’m afraid of it, but with recent tools such as COPR, and my glorious Makefile, some aspects of it have become palatable. This post will be about a recent journey I had building the most useless RPM ever.
A video of my journey building this RPM.Because of reasons, I wanted to satisfy an RPM dependency for a package that I wanted to install without rebuilding that RPM.

I’ve been hacking in golang for a while, but I’ll admit that I didn’t get too deep into some of the language nuances until more recently. Since some of them have started to bite me, here’s a little post-mortem of one of the problems I was having.
After hacking and testing code all day, I made a seemingly innocuous change, and when running my program, I saw the following error:

Git submodules are actually a very beautiful thing. You might prefer the word powerful or elegant, but that’s not the point. The downside is that they are sometimes misused, so as always, use with care. I’ve used them in projects like puppet-gluster, oh-my-vagrant, and others. If you’re not familiar with them, do a bit of reading and come back later, I’ll wait.
I recently did some work packaging Oh-My-Vagrant as RPM’s.

Making Oh-My-Vagrant (OMV) more developer accessible and easy to install (from a distribution package like RPM) has always been a goal, but was previously never a priority. This is all sorted out now. In this article, I’ll explain how “mainstream” mode works, and how the RPM work was done. (I promise this will be somewhat interesting!)
Prerequisites:
If you haven’t read any of the previous articles about Oh-My-Vagrant, I’d recommend you start there.

In this new crazy world of containers and immutable hosts, one might still want to run previous generation software such as Puppet on a current generation Atomic host. This article will explain how you can do that, and offer some proof of concept code.
The atomic host doesn’t provide a yum or dnf command, because the software is pre-baked into a read-only /usr/ partition. To “install” (to use) additional software, it usually needs to be distributed and run as a container.

I’ve added the ability to deploy a Kubernetes cluster with Oh-My-Vagrant (omv). I’ve also built an automated developer experience so that you can test your Kubernetes powered app in minutes. If you want to redeploy a new version, or see how your app behaves during a rolling update, you can use omv to test this out in minutes! I’ve recorded a screencast (~15 min), if you’d like to see some of this in action.

The Oh-My-Vagrant (omv) project is an easy way to bootstrap a development environment. It is particularly useful for spinning up an arbitrary number of virtual machines in Vagrant without writing ruby code. For multi-machine container development, omv can be used to help this happen more naturally.
Oh-My-Vagrant can be very useful as a docker application development environment. I’ve made a quick (<9min) screencast demoing this topic. Please have a look:

With Oh-My-Vagrant (omv) you can set up a dev environment in seconds. (Read the omv introduction if you’ve never used it before!) Since everything is defined in a single omv.yaml file, it is easy to share your cluster prototype with a friend! The one missing feature was associating code with this config file. This is now possible! Let me show you how it works…
In the omv.yaml file there is an extern variable.

Here are two quick git tricks that I’ve added to my toolbox lately…
I wanted to create a git alias that takes in argv from the command, but in the middle of the command. Here’s the hack that I came up with for the [alias] section of my ~/.gitconfig:
[alias] # cherryfetch fetches a repo ($1) / branch ($2) and applies it rebased! # the && true at the end eats up the appended args cherryfetch = !

Vagrant is a great tool for development, but Red Hat Enterprise Linux (RHEL) customers have typically been left out, because it has been impossible to get RHEL boxes! It would be extremely elegant if hackers could quickly test and prototype their code on the same OS as they’re running in production.
Secondly, when hacking on projects that have a long initial setup phase (eg: a long rpm install) it would be excellent if hackers could roll their own modified base boxes, so that certain common operations could be re-factored out into the base image.

You might want to write code that can tell how many iterations have passed since some action occurred. Alternatively, you might want to know if it’s the first time a machine has run Puppet. To do these types of things, you might wish to have a monotonically increasing counter in your Puppet manifest. Since one did not exist, I set out to build one!
The code:
If you just want to try the code, and skip the ramble, you can include common::counter into your manifest.

I use the Evolution mail client. It does have a few annoying bugs, but it has a plethora of great features too! Hopefully this post will inspire you to help hack on this piece of software and fix the bugs!
Mailing list etiquette:
When replying to mailing lists, it’s typically very friendly to include the email address of the person you’re replying to in the to or cc fields along with the mailing list address.

Recently, when I tried to access http://slashdot.org/ in Firefox, I would see my browser title bar flash briefly to “AT&T GUI”, and then I would get redirected to: http://slashdot.org/cgi-bin/redirect.ha which returns slashdot’s custom error 404 page! What is going on? (Read on for answer…) Did slashdot mess up their mod_rewrite config? (Nope, works fine in a different browser…) Did my HTTPS everywhere extension go crazy? (Nope, still broken when disabled…) Are my HTTP requests being MITM-ed?

The fact that we manage the switches in our data centres differently than any other server is patently absurd, but we do so because we want to harness the power of a tiny bit of silicon which happens to be able to dramatically speed up the switching bandwidth.
beware of proprietary silicon, it’s absurd!That tiny bit of silicon is known as an ASIC, or an application specific integrated circuit, and one particularly well performing ASIC (which is present in many commercially available switches) is called the Trident.

I’ve wanted a feature in Evolution for a while. It was formally requested in 2002, and it just recently got fixed in git master. I only started publicly groaning about this missing feature in 2013, and mcrha finally patched it. I tested the feature and found a small bug, mcrha patched that too, and I finally re-tested it. Now I’m blogging about this process so that you can get involved too!

I had an itch to scratch, and I wanted to get a bit more familiar with Openshift. I had used it in the past, but it was time to have another go. The app and the code are now available. Feel free to check out: https://pdfdoc-purpleidea.rhcloud.com/ This is a simple app that takes the URL of a markdown file on GitHub, and outputs a pandoc converted PDF. I wanted to use pandoc specifically, because it produces PDF’s that were beautifully created with LaTeX.

I just patched puppet-gluster and puppet-ipa to bring their infrastructure up to date with the current state of affairs…
What’s new? Better README’s Rake syntax checking (fewer oopsies) CI (testing) with travis on git push (automatic testing for everyone) Use of .pmtignore to ignore files from puppet module packages (finally) Pushing modules to the forge with blacksmith (sweet!) This last point deserves another mention. Puppetlabs created the “forge” to try to provide some sort of added value to their stewardship.

I usually avoid proprietary cloud services because of freedom, privacy and vendor lock-in concerns. In addition, there are some excellent libre (and hosted) services such as WordPress, Wikipedia and OpenShift which don’t have the above problems. Thirdly, there are every day Free Software tools such as Fedora GNU/Linux, Libreoffice, and git-annex-assistant which make my computing much more powerful. Finally, there are some hosted services that I use that don’t lock me in because I use them as push-only mirrors, and I only interact with them using Free Software tools.

If you’re a reader of my code or of this blog, it’s no secret that I hack on a lot of puppet and vagrant. Recently I’ve fooled around with a bit of docker, too. I realized that the vagrant, environments I built for puppet-gluster and puppet-ipa needed to be generalized, and they needed new features too. Therefore…
Introducing: Oh My Vagrant!
Oh My Vagrant is an attempt to provide an easy to use development environment so that you can be up and hacking quickly, and focusing on the real devops problems.

Seagate has just publicly announced 8TB HDD’s in a 3.5” form factor. I decided to do some rough calculations to understand the density a bit better…
Note: I have decided to ignore the distinction between Terabytes (TB) and Tebibytes (TiB), since I always work in base 2, but I hate the -bi naming conventions. Seagate is most likely announcing an 8TB HDD, which is actually smaller than a true 8TiB drive.

(Note: this hybrid management technique is being demonstrated in the puppet-ipa module for FreeIPA, but the idea could be used for other modules and scenarios too. See below for some use cases…)
The error message that puppet hackers are probably most familiar is:
Error: Duplicate declaration: Thing[/foo/bar] is already declared in file /tmp/baz.pp:2; cannot redeclare at /tmp/baz.pp:4 on node computer.example.com Typically this means that there is either a bug in your code, or someone has defined something more than once.

Master SDN hacker Flavio sent me some tunes. They were sitting on my desktop in a folder:
$ ls ~/Desktop/ uncopyrighted_tunes_from_flavio/ I wanted to listen them while hacking, but what was the easiest way…? I wanted to use the nautilus file browser to select which folder to play, and the totem music/video player to do the playing.
Drop a file named totem into:
~/.local/share/nautilus/scripts/ with the contents:
#!/bin/bash # o hai from purpleidea exec totem -- "[email protected]" and make it executable with:

Configuration management is an essential part of securing your infrastructure because it can make sure that it is set up correctly. It is essential that configuration management only enhance security, and not weaken it. Unfortunately, the status-quo of secret management in puppet is pretty poor.
In the worst (and most common) case, plain text passwords are found in manifests. If the module author tried harder, sometimes these password strings are pre-hashed (and sometimes salted) and fed directly into the consumer.

Earlier this year, R.I.Pienaar released his brilliant data in modules hack, a few months ago, I got the chance to start implementing it in Puppet-Gluster, and today I have found the time to blog about it.
What is it?
R.I.’s hack lets you store hiera data inside a puppet module. This can have many uses including letting you throw out the nested mess that is commonly params.pp, and replace it with something file based that is elegant and hierarchical.

When GNOME shell breaks, you get to keep both pieces. The nice thing about shell failures in GNOME 3, is that if you’re able to do a restart, the active windows are mostly not disturbed. The common way to do this is to type ALT-F2, r, <ENTER>.
Unfortunately, you can’t always type that in if your shell is very borked. If you are lucky enough to have SSH access, and another machine, you can log in remotely and run this script:

Vagrant has become the de facto tool for devops. Faster iterations, clean environments, and less overhead. This isn’t an article about why you should use Vagrant. This is an article about how to get up and running with Vagrant on Fedora with libvirt easily!
Background:
This article is an update of my original Vagrant on Fedora with libvirt article. There is still lots of good information in that article, but this one should be easier to follow and uses updated versions of Vagrant and vagrant-libvirt.

This is a quick trick for making working with git submodules more magic.
One day you might find that using git submodules is needed for your project. It’s probably not necessary for everyday hacking, but if you’re glue-ing things together, it can be quite useful. Puppet-Gluster uses this technique to easily include all the dependencies needed for a Puppet-Gluster+Vagrant automatic deployment.
If you’re a good hacker, you develop things in separate feature branches.

So this happened:
James just James at RedHat headquarters in North Carolina wearing his new red hat.RedHat made me an offer, and I am happy to say that I have just started this week!
I am proud to have joined a company that employs many of the worlds foremost, highly professional and clever hackers. It is indubitably the best Free Software [1] / Open Source company out there, and they ship some of the greatest and most elegant software available.

I’ve been afraid of RPM and package maintaining [1] for years, but thanks to Kaleb Keithley, I have finally made some RPM’s that weren’t generated from a high level tool. Now that I have the boilerplate done, it’s a relatively painless process!
In case you don’t know kkeithley, he is a wizard [2] who happens to also be especially cool and hardworking. If you meet him, be sure to buy him a $BEVERAGE.

Puppet is missing a number of much-needed features. That’s the bad news. The good news is that I’ve been able to write some of these as modules that don’t need to change the Puppet core! This is an article about one of these features.
Posit: It’s not possible to apply all of your Puppet manifests in a single run.
I believe that this holds true for the current implementation of Puppet.

I use gnome-terminal for most of my hacking. In fact, I use it so much, that I’ll often have multiple tabs open for a particular project. Here’s my workflow: Control+Alt+t (My shortcut to open a new gnome-terminal window.) cd ~/code/some_cool_hack/ # directory of some cool hack Control-Shift-t (Shortcut to open a new gnome-terminal tab.) Hack, hack, hack… The problem is that the new tab that I’ve created will have a $PWD of ~, instead of keeping the $PWD of ~/code/some_cool_hack/, which is the project I’m working on!

I’ll be giving a talk at SCALE today about automatically deploying GlusterFS with Puppet-Gluster and Vagrant. I’ll be giving some live demos, and this will cover some of the material from:
Automatically deploying GlusterFS with Puppet-Gluster + Vagrant!
and it will contain excerpts from:
Screencasts of Puppet-Gluster + Vagrant
I’ll also be talking about some new upcoming features, and am happy to answer all of your questions!
The talk will be part of Infrastructure.

To give you a break from the usual GNU/Linux/DevOps/Puppet/GlusterFS drab, I’ve decided to have a go at writing a different kind of technical article. This article will show you how to build the traditional Canadian snow dwelling known as a quinzee. If you will be travelling to Canada, I recommended that you read through this article ahead of time, so that you don’t offend your host by being unfamiliar with their traditional living accommodations.

I’m using a Lenovo X201 with 8GiB of RAM. Apart from some minor issues, I’ve been very satisfied with this laptop. It’s over four years old, and so I decided to see what’s available on the horizon. I did not buy an X240 because of the following reasons:
The X240 has only one slot for RAM and thus supports a maximum of 8GiB.
I think it’s pretty ridiculous for any successor to the X230 to support less RAM.

I decided to record some screencasts to show how easy it is to deploy GlusterFS using Puppet-Gluster+Vagrant. You can follow along even if you don’t know anything about Puppet or Vagrant. The hardest part of this process was producing the actual videos!
If recommend first reading my earlier articles if you’re planning on following along: Vagrant on Fedora with libvirt Vagrant vsftp and other tricks Vagrant clustered SSH and ‘screen’ Automatically deploying GlusterFS with Puppet-Gluster + Vagrant!

I needed a base image “box” for my Puppet-Gluster+Vagrant work. It would have been great if good boxes already existed, and even better if it were easy to build my own. As it turns out, I wasn’t able to satisfy either of these conditions, so I’ve had to build one myself! I’ve published all of my code, so that you can use these techniques and tools too!
Status quo:
Having an NIH problem is bad for your vision, and it’s best to benefit from existing tools before creating your own.

The GlusterFS community is having a “test day”. Puppet-Gluster+Vagrant is a great tool to help with this, and it has now been patched to support alpha, beta, qa, and rc releases! Because it was built so well (*cough*, shameless plug), it only took one patch.
Okay, first make sure that your Puppet-Gluster+Vagrant setup is working properly. I have only tested this on Fedora 20. Please read: Automatically deploying GlusterFS with Puppet-Gluster+Vagrant!

Puppet-Gluster was always about automating the deployment of GlusterFS. Getting your own Puppet server and the associated infrastructure running was never included “out of the box”. Today, it is! (This is big news!)
I’ve used Vagrant to automatically build these GlusterFS clusters. I’ve tested this with Fedora 20, and vagrant-libvirt. This won’t work with Fedora 19 because of bz#876541. I recommend first reading my earlier articles for Vagrant and Fedora: Vagrant on Fedora with libvirt Vagrant vsftp and other tricks Vagrant clustered SSH and ‘screen’ Once you’re comfortable with the material in the above articles, we can continue…

Some fun updates for vagrant hackers… I wanted to use the venerable clustered SSH (cssh) and screen with vagrant. I decided to expand on my vsftp script. First read: Vagrant on Fedora with libvirt
and Vagrant vsftp and other tricks
to get up to speed on the background information.
Vagrant screen:
First, a simple screen hack… I often use my vssh alias to quickly ssh into a machine, but I don’t want to have to waste time with sudo-ing to root and then running screen each time.

As I previously wrote, I’ve been busy with Vagrant on Fedora with libvirt, and have even been submitting, patches and issues! (This “closed” issue needs solving!) Here are some of the tricks that I’ve used while hacking away.
Default provider:
I should have mentioned this in my earlier article but I forgot: If you’re always using the same provider, you might want to set it as the default. In my case I’m using vagrant-libvirt.

Apparently lots of people are using Vagrant these days, so I figured I’d try it out. I wanted to get it working on Fedora, and without Virtualbox. This is an intro article on Vagrant, and what I’ve done. I did this on Fedora 19. Feel free to suggest improvements.
Intro:
Vagrant is a tool that easily provisions virtual machines, and then kicks off a configuration management deployment like Puppet. It’s often used for development.

I’m now syndicated on the Fedora Project planet. If you haven’t read through my blog yet, let me introduce myself, I’m James, and I write The Technical Blog of James.
I’m a sysadmin, DevOps/Puppet hacker, I.T./network architect and physiologist. Hi! I run Fedora as my primary desktop, but I also use it for servers, particularly for development before future versions of RHEL and CentOS release.
I’m most well-known for Puppet-Gluster, but I’ve also written a decent Puppet-IPA (FreeIPA) module.

As a follow-up to my original article on recursion in Puppet, and in my attempt to Push Puppet (to its limit), I’ll now attempt some more advanced recursion techniques in Puppet.
In my original recursion example, the type does recurse, but the callee cannot return any value to the caller because it is a type, and not strictly a function. This limitation immediately limits the usefulness of this technique, but I’ll try to press on!

Ironically, one of the reasons that I started writing Puppet code, was so that I could spend more time designing and building, and less time writing documentation. I suppose I’m a victim of my success, because Puppet-Gluster has grown large enough to warrant its own documentation.
So I gave in, and put together some documentation.
It’s available as markdown, and, also as a pdf. As before, there is an examples/ directory which you might want to use as additional reference.

People often ask how to do iteration in Puppet. Most Puppet users have a background in imperative programming, and are already very familiar with for loops. Puppet is sometimes confusing at first, because it is actually (or technically, contains) a declarative, domain-specific language. In general, DSL’s aren’t always Turing complete, nor do they need to support loops, but this doesn’t mean you can’t iterate.
Until recently, Puppet didn’t have an explicit looping construct, and it is quite possible to build complex modules without using this new construct.

Hi there,
I hope you enjoyed my “Pushing Puppet (to its limit)” talk and demos from Puppet Camp D.C., LISA 2013. As requested, I’ve posted the code and slides.
Here is the code: https://github.com/purpleidea/puppet-pushing
This module will require three modules as dependencies. The dependencies are: My Puppet-Common module https://github.com/purpleidea/puppet-common My Puppet-Runonce module https://github.com/purpleidea/puppet-runonce My Puppet-FSM module https://github.com/purpleidea/puppet-fsm Each example doesn’t require all the dependencies, so if you’re only interested in the FSM, you only need that module.

I’m here at LISA 2013 at the Gluster Community Day. I’ve been asked by Joe Brockmeier to give a little recap about what’s been going on. So here it is!
Wesley Duffee-Braun started off with a nice overview talk about GlusterFS. The great thing about his talk was that he gave a live demo, running on virtual machines, on his laptop. If you’re a new GlusterFS user, this is good exposure to help you get started.

I’m speaking at LISA 2013, the “Large Installation System Administration” conference. This conference runs all week in Washington. I’ll be giving two talks during the week, and attending at least one BOF.
My first talk is on Monday during the Gluster Community Day. I’ll be speaking about puppet-gluster, and giving a live demo. I’ll be showing some new features too. If you’d like to talk more about puppet-gluster, or want to attend the talk, give me a shout, or sign up at the above Gluster Community Day link.

Here’s a one minute read, about a trick which I discovered today:
When running an strace, it’s common to do something like:
strace -p<pid> Smarter hackers know that they can use some bash magic and do:
strace -p`pidof <process name>` However, if you’re tracing a script named foo.py, this won’t work because the real process is the script’s interpreter, and pidof python, might return other unrelated python scripts.

Oh, hi there.
In case you’re interested, I’ve just made a first release of my puppet-shorewall module. This isn’t meant as an exhaustive shorewall module, but it does provide most of the usual functionality that most users need.
In particular, it’s the module dependency that I use for many of my other puppet modules that provide firewalling. This is probably where you’re most likely to consume it.
In general most modules just implement shorewall::rule, so if you really don’t want to use this code, you can implement that signature yourself, or not use automatic firewalling.

I’m usually on IRC, but I don’t often notice incoming pings until after the fact. I had to both write, and modify various scripts to get what I wanted, but now it’s all done, and you can benefit from my hacking by following along…
The Setup
Laptop -> Gnome-Terminal -> SSH -> Screen -> Irssi This way, I’m connected to IRC, even when my laptop isn’t. I run irssi in a screen session on an SSH server that I manage, and I use gnome-terminal on my laptop.

This October 12th to 14th Montreal hosted the GNOME boston summit. Many thanks to Canonical for sponsoring breakfast, Savoir Faire Linux for hosting a great 6 à 10 with fancy snacks, and RedHat for sponsoring a pool night. What follows is some technical commentary about stuff that went on.
JHBuild
JHBuild is a tool to make it easy to download/clone (from git) and compile all the GNOME modules and applications. It was easy to get going.

In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, tobuild in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine (FSM) in puppet.
Is this a real finite state machine, and why would you do this?
Computer science professionals might not approve of the purity level, but they will hopefully appreciate the hack value.

Theron Conrey writes about using: BitTorrent Sync as Geo-Replication for Storage We got a chance to talk about this idea at Linuxcon. I’m not entirely convinced there aren’t some problem edge cases with this solution, but I think it will be hard to tell as long as the BitTorrent sync library is proprietary. I did come up with a special case of Theron’s idea that I believe could work well.

I’m here in New Orleans hacking up a storm and getting to meet fellow gluster users IRL. John Mark Walker started off with a great “State of the GlusterFS union” style talk.
Today Louis (semiosis) gave a great talk about running glusterfs on amazon. It was highly pragmatic and he explained how he chose the number of bricks per host. The talk will be posted online shortly.
Marco Ceppi from Canonical gave a talk about juju and gluster.

Continuing on from yesterday, I’ve met even more interesting people. I chatted with Dianne Mueller about some interesting ideas for gluster+openshift. More to come on that front soon. Hung out with Jono Bacon and talked a bit about puppet-gluster on Ubuntu. If there is interest in the community for this, please let me know. Thanks to John Mark Walker and RedHat for sponsoring me and introducing me to many of these folks.

I’m here in New Orleans at Linux Con, hacking on puppet-gluster and talking to lots of interesting folks. I’ve met gluster hacker Theron Conrey, and my host John Mark Walker, Fedora and Raspberry Pi experts Spot and Ruth Suehle, and many others too.
The hotel is very nice. The bathroom sink has two taps of course, but both of them are hot. The New Orleans heat is probably the cause of this.

Hey there,
I’ve done a bit of puppet-gluster hacking lately to try to squeeze some extra features and testing in before Linuxcon. Here’s a short list: SELinux fixes to keep Dan Walsh happy :) Ping and status checks before volume creation. Now puppet-gluster will be less noisy about failures or missing executions that are due to the necessary incremental nature of puppet-gluster runs. You’ll need multiple puppet runs to get a complete setup, so don’t let puppet complain part way through.

John Mark Walker, (from Redhat) has been kind enough to invite me to speak at the Linuxcon Gluster Workshop in New Orleans. I’ll be speaking about puppet-gluster, giving demos, and hopefully showing off some new features. I’m also looking forward to meeting up with gluster expert Joe Julian.
If there are features that puppet-gluster is missing, or you have a use case that I haven’t covered, please let me know, and I’ll try to work on it for you ahead of the conference.

I love tabs, they’re so much easier to work with, but YAML doesn’t like them. I’m constantly adding them in accidentally, and puppet’s error message is a bit cryptic:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: malformed format string - %S at /etc/puppet/manifests/foo.pp:18 on node bar.example.com This happens during a puppet run, which in my case loads up YAML files. The tricky part was that the error wasn’t at all related to the foo.

It was time to take the plunge and upgrade from Fedora 18 to Fedora 19. Fedora 18 was one of the worst releases ever, so I figured it could only get better. I ran my backups as usual, however this time I didn’t seem to need them, the upgrade process went off without a hitch! I used the fedup-cli process over the network. I always run these things inside of screen.