It's been a long time coming but I'm happy to announce the release of
puppet-lint 1.0.0!

Along with a bunch of bugfixes and a rewrite of most of the code, there's some
(hopefully) exciting new features in this release.

Automatic fixing of errors

Previewed in the 0.4.0 pre-release, simple problems can now be automatically
fixed by puppet-lint (some problems require complex refactoring and so
puppet-lint won't attempt anything on your behalf).

In 1.0.0, problems detected by the following checks can be automatically fixed
by running puppet-lint with --fix:

slash_comments

star_comments

unquoted_node_name

unquoted_resource_title

unquoted_file_mode

file_mode

ensure_not_symlink_target

double_quoted_strings

only_variable_string

variables_not_enclosed

quoted_booleans

hard_tabs

trailing_whitespace

arrow_alignment

Complimentary to --fix, puppet-lint now has a --only-checks parameter that
you can pass a comma seperated list of checks that you want to run. This works
great when doing an initial pass over your code to fix problems. Let's say
that you just want to realign all your =>s, run puppet-lint --fix
--only-checks arrow_alignment modules/.

Plugin system

From it's inception, puppet-lint has enforced the official style
guide and I've had to
turn down a lot of good ideas and contributions because they're not a part of
that guide. To that end, I've added a plugin system in 1.0.0 so that we can
add and distribute custom checks outside of the main code base.

My hope is that together as a community we can come up with some new checks and
influence new versions of the style guide. I've written
a tutorial that steps through
how to write a basic check. If you've got a good idea for a check but aren't
up to writing it yourself, create an issue in the puppet-lint repo with the
"new check" label
and maybe someone else will be able to.

You can find the currently tiny list of community
plugins here. They're distributed as Ruby
Gems so you can easily install them however you're currently managing
puppet-lint (gem, bundler, etc).

Control comments

A much requested feature, you can now disable tests via special comments in
your code. Read more about it
here.

New checks

A couple of new checks have been added to core puppet-lint in 1.0.0.

unquoted_node_name - Checks for unquoted node names

puppet_url_without_modules - Checks for fileserver URLs without the
modules mountpoint.

What's gone

The class_parameter_defaults check has been removed. This check was based on
a misreading of the style guide.

For a more complete list of changes, I'd recommend reading the
changelog.

It's been a while, but finally there's a new release of rspec-puppet.
Originally slated to be 0.2.0 but promoted to 1.0.0 due to a couple of
backwards incompatible changes.

What might cause you problems

create_resource matcher

This deprecated matcher has been removed entirely now. If you still have code
using it, you should switch to using the generic contain_<resource> matcher
instead.

include_class matcher

This matcher has now been deprecated and will be removed in the next major
release. The reason this is on it's way out is that it doesn't support
parameterised classes and this has caused a lot of confusion for many users.

If you use include_class anywhere, you'll see the following depreciation
notice.

DEPRECATION WARNING: you are using deprecated behaviour that will
be removed from a future version of RSpec.
* include_class is deprecated.
* please use contain_class instead.

So, change:

1it{shouldinclude_class('foo')}

To:

1it{shouldcontain_class('foo')}

Changes to how parameters are matched

In previous versions of rspec-puppet, parameter values were matched extremely
naively, where all values were flattened down to a string before comparison.
This means that ['b', 'ba'] would have been equal to ['bb', 'a'] for
example. As of 1.0.0, this behaviour has changed and we now compare data
structures like arrays and hashes correctly, so you may have to adjust your
tests accordingly.

What's new

hiera support

Set the path to your hiera.yaml file in your RSpec.configure block and
you're good to go.

compile matcher

This new matcher should the be first thing in any rspec-puppet test suite. It
checks that the catalogue will compile correctly and that there are no
dependency cycles in the generated graph.

1it{shouldcompile}

This matcher also has a chain method to enable checking that all dependencies
in the catalogue have been met - with_all_deps.

1it{shouldcompile.with_all_deps}

While at first glance, it might seem that this shouldn't be optional however
there are cases where you might not want to test this (if, for example, you are
testing a module that notifies a resource in a different module).

relationship tests

Some new additions to the contain_<resource> matcher are the resource
relationship tests.

only_with tests

Also new to the contain_<resource> matcher are the only_with tests. Unlike
the with tests which only test that the specified parameters have been
defined, only_with tests that these are the only parameters passed to
a resource.

Like the with tests, you can specify a single parameter with the
only_with_<parameter> method:

Since the first release of puppet-lint, one of the top two feature requests has
been the ability to automatically fix trivial style issues (detecting problems
is all well and good but no one wants to manually fix the eleventy thousand
problems that have accumulated over the years).

To that end, I have just shipped a beta release of puppet-lint (0.4.0.pre1)
which includes the experimental fixing support! I highly encourage everyone to
give it a try and report any issues they find.

Currently, puppet-lint supports fixing a limited subset of detectable problems.
There will probably be more added to this list over time however some problems
will always require a human to decide how to proceed.

At this time, puppet-lint do the following for you:

Converting // comments into # comments.

Quoting unquoted resource titles.

Quoting unquoted file mode strings.

Converting 3 digit octal file modes into 4 digit modes.

Converting double quoted strings without variables into single quoted
strings.

Converting double quoted strings that only contain a variable into an
unquoted variable.

Enclosing variables in double quoted strings that haven't been enclosed in
braces.

Caveat Emptor

I'm going to assume that your manifests are stored in some sort of version
control and that you're comfortable discarding the changes puppet-lint makes if
you don't like them.

Trying it out

First of all install the new version, either with RubyGems:

1gem install --pre puppet-lint -v 0.4.0.pre1

or Bundler:

1gem'puppet-lint','0.4.0.pre1'

Then just call puppet-lint with the -f or --fix option

1$ puppet-lint -f test.pp
2FIXED: double quoted string containing no variables on line 13FIXED: string containing only a variable on line 34FIXED: indentation of => is not properly aligned on line 55FIXED: unquoted file mode on line 56FIXED: mode should be represented as a 4 digit octal value or symbolic mode on line 5

You might have noticed that there hasn't been a puppet-lint release in quite
some time and that I've become a bit lax when it comes to keeping up with bugs
and pull requests.

Puppet-lint originally started as something to fill in some time while
10,000 metres above the Tasman Sea and like all good hack projects, I kept
adding more and more code onto it to get each release out the door as quickly
as possible. This lead to a number of questionable design decisions, the
biggest of which was using Puppet's lexer to tokenise the manifests that are
being analysed.

Don't get me wrong, the Puppet lexer is fantastic and I doubt I can
write a better one, but as it's a part of a separate active project, developing
against such a moving target results in constantly adding new corner cases with
every Puppet release. This has resulted in a messy code base that I haven't
really desired to touch, to the detriment of the project.

So what now?

If you've been following
the project on GitHub, you might have
noticed a flurry of recent activity in the
dust_bunny branch.
Over the last week, I made the call to rip out the Puppet lexer and replace it
with a simple one of my own. Along with removing the constantly moving target,
it has had a couple of other major benefits:

Faster start up time, as we're no longer loading Puppet.

A custom lexer tailored to our needs means we can now access additional data
that was previously discarded (like comments).

This has also given me an excuse to start a much needed cleanup of the code to
ensure that it is consistent and (hopefully) well documented. Anyone who's
tried to add a new check will know what I mean.

Great, why are you telling me?

While everything I've described doesn't sound very interesting from a features
point of view, it does represent a massive change to the internals of
puppet-lint and there's a very real possibility of introducing regressions.

So, what I need you to do is install the pre-release version of 0.2.0

gem install puppet-lint -v 0.2.0.pre1 --pre

And then run it over your manifests. If you run into any bugs, false positives
or negatives etc, please report them on the project issue
tracker and tag them with the
dust_bunny label.

Once any issues that have arisen as a result of these changes have been
resolved, I'll go back through the old issues and get them fixed up before
pushing out a proper 0.2.0 release with all the features you've been waiting
for.

TL;DR

Lots of internal changes happened in puppet-lint recently and I need you to
help test it out and make sure no regressions have snuck in.

I'm going to assume you've got a Puppet module already on GitHub. To save
messing around with bundler on your local machine, I recommend installing
puppet-lint and rspec-puppet as system gems while you're getting this all
set up.

puppet-lint

Next up, we'll also get some automatic lint testing of your manifests going to
ensure you're writing manifests that comply with the Puppet Labs style guide.
This is simply a matter adding the following line near the top of your
Rakefile

1require'puppet-lint/tasks/puppet-lint'

You can now run rake lint to run puppet-lint over your manifests.

Travis CI

Travis CI is a wonderful free continuous integration
service that integrates with GitHub, running whatever
tests you want against your code every time you push.

To get Travis CI automatically testing your module you need to add a couple of
files to the root directory of your module.

First, create a Gemfile which tells bundler which ruby gems your tests need in
order to run. If your module needs any additional gems, just add them to the
bottom of this file.

TL;DR

Whenever I need to setup a new service on one of my hosts, the first thing I do
is head to the forge and
GitHub to try and find a decent Puppet module that
already exists for it.

I almost always leave in disappointment.

Puppet modules are libraries

Much like string.h provides everything you need to manipulate strings in C,
your Puppet modules should provide everything needed to manage a service out of
the box. By that I mean, I want to pull down your module to enable the
functionality I need in Puppet without modifying your module at all.

Package, File, Service

Regrettably, most of the modules out there don't deviate from the basic
package, file, service model.

While this is slightly more useful, it doesn't cover the possibility of wanting
to override a value on a per-host basis.

Package, File, Service, Facter and... Global Variables? Seriously?

If you ask most people for the "best practice" method of passing parameters to
their modules, they'll probably recommend using global variables - where you
set a variable in your node definition or ENC and it magically gets picked up
inside the various classes that you've included.

Do we really need to go into why this is a bad idea? Puppet's variable scoping
is confusing enough on it's own.

So what should you be doing

To put it simply, write your Puppet modules like you would write a library for
your favorite programming language. Don't know a programming language? You're
working towards the almighty "Infrastructure as Code" ideal. Stop making
excuses like "I don't know how to program so I didn't know any better" and go
and learn a language.

Don't make your users edit your code to do the work they want.

Don't rely on global variables to pass information to your modules.

Make it easy for other people to add features to your module.

While it looks like the 3rd point contradicts the 1st point, it doesn't.
There's a big difference between someone sending you a patch to your module to
support a distro that it doesn't currently support and someone having to go and
edit a template inside your module in order to change a config value that you
didn't think anyone would need to change.

An example of a good module

Take a look at the Puppet Labs NTP
module.
It's not perfect, but it's pretty close.

First off, it's using a parameterised class, so there's no global variables
polluting the namespace while still allowing us to change the parameters used
to configure NTP without editing the module itself. If you're using an older
version of Puppet that doesn't support parameterised classes, a defined type
will work just as well.

Secondly, it has support for a good number of different distributions but it
has a default case that causes Puppet to abort and let the user know that this
module isn't supported on their machine. This is much better than just blindly
making changes to a system using values that may or may not work for it.

For the folks using ENCs that don't support parameterised classes or defined
types, you're still OK because you can just put this into your role definition
like so

Also, replacing the autoupdate parameter with a version parameter so that
it was possible to specify a particular package version, 'latest', 'installed'
etc. would be a great addition.

That's all folks

How would you like to see modules written? What currently annoys you about the
modules out there? Do you disagree with everything I've written and want the
last 10 minutes of your life back? Let me know.

So, I ran into a bit of a problem yesterday. I was working on a Puppet
module for the Sensu monitoring framework,
part of which involved writing a custom Puppet type to manage the configuration
of service checks. Normally this is a trivial matter, however in this case,
the checks must be configured on both the server and the client and must
therefore be able to notify either the server process or the client process or
both to restart after a configuration change.

Not the most elegant solution in the world. Ideally what I needed was
something like the existing autorequire functionality that creates require
relationships between your custom type and the named resources automatically
but only if the resources exist in the node catalogue.

Unfortunately, such functionality doesn't exist (yet), so I had to go some
digging around in Puppet's codebase to implement it. The first thing to do was
to establish a way to create arbitrary notify relationships. This turned out
to be easily done:

If you've worked with configuration management systems for a while, a situation
like this has probably cropped up and ruined your day. If you're lucky
enough to have a homogeneous environment, then you might have a staging
environment that you can test changes out on, but what if you don't? A slight
mistake in that harmless change you're working on could stop a service on
hundreds of machines or worse (purge the mysql-server package and all it's data
*cough*).

Why you should be writing unit tests for your Puppet modules

Prevent situations like the one above.

Catch any problems moving between Puppet releases before it hits production.

Now we can do this too

Getting started

In this article, we'll cover setting up your testing environment and cover how
to write unit tests for your Puppet functions. I'm going to make a few
assumptions now:

You store your Puppet manifests in git.

You run a *nix machine as your workstation.

You don't mind getting your hands dirty with a bit of simple Ruby.

You have Ruby installed (1.8.7).

First of all, we're going to install Bundler to manage
the dependencies our Puppet testing rig will have.

OK, time to configure RSpec. Create
a spec directory in the root of your Puppet repository and create a file in
the spec folder called spec_helper.rb. Adjust c.module_path and
c.manifest_dir to point to your modules and manifests directories in your
repository.