Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

sagecreek writes "If you are in charge of a small network with just a few servers, you may still be doing configuration management primarily by hand. And you may take particular pride in maintaining that 'artisan' role. After all, it's mostly up to you to set up new users and their machines, fix current problems, manage the servers and their software, create databases and their user accounts, and try to keep the network and user configurations as uniform as possible despite running several different brands--and vintages--of hardware and software. However, warns infrastructure consultant John Arundel, '[b]eyond ten or so servers, there simply isn't a choice. You can't manage an infrastructure like this by hand. If you're using a cloud computing architecture, where servers are created and destroyed minute-by-minute in response to changing demand, the artisan approach to server crafting just won't work.' In his new book, Puppet 3 Beginner's Guide, Arundel emphasizes: 'Manual configuration management is tedious and repetitive, it's error-prone, and it doesn't scale well. Puppet is a tool for automating this process.'" Read below for the rest of sagecreek's review.

Puppet 3 Beginner's Guide

author

John Arundel

pages

184

publisher

Packt Publishing

rating

8 out of 10

reviewer

sagecreek

ISBN

978-1-78216-124-0

summary

Learn how to fully utilize Puppet through simple, practical examples

tester dataActually, among "UNIX-like systems," there are at least three major configuration management (CM) packages — Puppet, Chef, and CFEngine — plus some other competitors, Arundel notes. He calls them "all great solutions to the CM problem...it's not very important which one you choose as long as you choose one." But he hopes, of course, you will check out Puppet and his new, well-written how-to book.

Puppet 3 Beginner's Guide is structured to help system administrators "start from scratch...and learn how to fully utilize Puppet through simple, practical examples."

Arundel's book places important emphasis on the rapidly closing "divide between 'devs,' who wrangle code, and 'ops,' who wrangle configurations. Traditionally, the skills sets of the two groups haven't overlapped much," he notes. "It was common until recently for system administrators not to write complex programs, and for developers to have little or no experience of building and managing servers."

Today, he points out, system admins are "facing the challenge of scaling systems to enormous size for the web, [and] have had to get smart about programming and automation." Meanwhile, "[d]evelopers, who now often build applications, services, and businesses by themselves, couldn't do what they do without knowing how to set up and fix servers."

Therefore, "[t]he term 'devops' has begun to be used to describe the growing overlap between these skill sets," Arundel emphasizes. "Devops write code, herd servers, build apps, scale systems, analyze outages, and fix bugs. With the advent of CM systems, devs and ops are now all just people who work with code."

Arundel's 184-page Puppet 3 Beginner's Guide has 10 chapters that are smoothly structured with numerous headings, subheadings, short paragraphs, code examples, and other illustrations. He has generated his code examples using the Ubuntu 12.04 LTS "Precise" distribution of Linux. But he explains how to load the software using "Red Hat Linux, CentOS, or another Linux distribution that uses the Yum package system," as well.

In Chapter 2, "First Steps with Puppet," the author details how to install Puppet, create a simple manifest, and apply it to a machine. He also offers some basic Puppet language examples.

Chapter 3, "Packages, Files, and Services," focuses on "how to use these key resource types...and how they work together" and presents "a complete and useful example based on the Nginx web server."

In Chapter 4, "Managing Puppet with Git," Arundel shows "a simple and powerful way to connect machines together using Puppet, and to distribute your manifests and work on them together collaboratively using the version control system Git."

The emphasis in Chapter 5, "Managing Users," is on "good practices for user administration" and implementing them with Puppet. The chapter also covers "how to control access using SSH and manage user privileges using sudo."

The topics covered in Chapter 6, "Tasks and Templates," include using "Puppet's resource types to run commands, schedule regular tasks, and distribute large trees of files." Also covered: "how to insert values dynamically into files using templates."

In Chapter 7, "Definitions and Classes," Arundel explains "how to organize Puppet code into reusable modules and objects. We'll see how to create definitions and classes, and how to pass parameters to them."

Chapter 8, "Expressions and Logic," dives deeper into Puppet code. It "shows how to control flow using conditional statements and logical expressions, and how to build arithmetic and string expressions. It also covers operators, arrays, and hashes."

Chapter 9, "Reporting and Troubleshooting," deals with what the author terms "the practical side of working with Puppet," including diagnosing and solving common problems, debugging the software's operations, and understanding Puppet's error messages.

The final section, Chapter 10, "Moving on Up," wraps up with a range of topics, including how to make Puppet code "more elegant, more readable, and more maintainable." Arundel also offers "links and suggestions for further reading." And he describes nine projects to help you "improve your skills and your infrastructure at the same time." The projects, he says, "provide a series of stepping-stones from your first use of Puppet to a completely automated environment."

Puppet's maker, Puppet Labs, offers some virtual-machine options for learning the software. The choices are: (1) a VXM version recommended for VMware Fusion and VMware Workstation; and (2) an OVF version recommended for VirtualBox "and all other non-VMware virtualization software." Puppet Labs also offers a Puppet Enterprise version of its software that supports up to 10 nodes free.

Along with Linux, Puppet will run on other several platforms, including Windows and Macs,, but you will find little help for those in Arundel's book. You will need to use Puppet Lab's online Mac or Windows documentation. And Windows may not be the greatest of choices. As the documentation notes: "Windows nodes can't act as puppet masters or certificate authorities, and most of the ancillary Puppet subcommands aren't supported on Windows."

It can take a bit of work to get Puppet installed and configured. But once you have it running in a Linux environment, John Arundel's new book can be a solid guide to helping you become both a proficient Puppet user and a more efficient, knowledgeable, and versatile system administrator.

That was a pretty poor review. Giving a summary of the table of contents isn't a review. Additionally it doesn't seem like they recognize that devops and duplicative administration don't fit with a lot of data processing models. There are many organizations that have servers that have a distinct purpose and it doesn't make sense to envision them as just another clone system in "the cloud".

That was a pretty poor review. Giving a summary of the table of contents isn't a review. Additionally it doesn't seem like they recognize that devops and duplicative administration don't fit with a lot of data processing models. There are many organizations that have servers that have a distinct purpose and it doesn't make sense to envision them as just another clone system in "the cloud".

This is why puppet has a very strong inheritance system...
We have it broken down as generic server (2 factor/LDAP configs, nagios configs, etc) and then apache_servers which build out the basic web infrastructure and then more specialized configs for one-off speicalized servers... (admin server versus production web servers)

There is pretty much nothing you can do by hand that puppet can't also do - and often it takes just as much time to update a single puppet config file and run the puppet update pr

There is pretty much nothing you can do by hand that puppet can't also do - and often it takes just as much time to update a single puppet config file and run the puppet update process as it would be to ssh into the server and make the manual change.

Another advantage is what might go into traditional documentation is now just a puppet configuration.. Oh, fuck, this server crashed? Just roll another in 5 minutes... Who cares about the old one..

There is pretty much nothing you can do by hand that puppet can't also do - and often it takes just as much time to update a single puppet config file and run the puppet update process as it would be to ssh into the server and make the manual change.

Another advantage is what might go into traditional documentation is now just a puppet configuration.. Oh, fuck, this server crashed? Just roll another in 5 minutes... Who cares about the old one..

And this is the flaw in your argument. There seems to be some assumption that if it's not under control of Puppet or Chef then it's manual. This is completely untrue. Any competent admin automates their administration. I've been doing it for more than a decade.

Second it's not the host OS and host configuration that makes the servers distinct. It's the data. You can't automate ten years worth of data entry and workflow modules. I suppose it would be unfair of me to hold against you the fact that you don't know anything about my operations, but we're not an internet based company. We're doing stuff other than serving up a bunch of vanity gopro videos. We have several large data centers but we also have hundreds of offices around the world and those offices have their own IT infrastructure. Anyone can stand up a server in 10 minutes in their data center. How long will it take you to stand one up Chengdu given that your primary data centers are in the US and Europe and your network line to the remote facility is 512Kb/s?

The absurdity of the proponents of CFengine, Puppet, Chef, et. al. is that they assume no one has ever solved these problems before. What problems that I have are these products going to solve for me? The emphasis is on "problems that I have". It's not sufficient to tell me what a product does, it's whether it solves my problems.

You are right, there is nothing you can do with puppet that you can't do with SSH, and 10 years ago things like puppet didn't even exist so it makes total sense for you to be in the situation you're in and it wouldn't make alot of sense for you to switch just for the sake of using puppet.

But it's 2013, if you're starting off new - why would you roll your own when many solutions already exist that have been thoroughly tested and extended to have a rich feature set that you probally wouldn't have time to de

I spent a week in the headquarters of one of the vendors mentioned in these comments training on their product. What I discovered is they just can't recognize that a lot of companies are not technology companies. And those companies' IT organizations aren't in the business of standing up or rebuilding hundreds or thousands of servers per week. And those companies have mature IT organizations. And their servers are often not nicely concentrated in a handful of data centers. After about half the week had gone

"An optimisation in configuration management shouldn't be causing such a large impact"

I remember that when Windows 95 came to the party I was using whatever window manager used HP-Ux back then, VUE, or something like that -and it was far from a fair comparation (and that's out of my generosity of not trying to remember when it was not Windows 95 but Windows 3.x the one to compare against). But Windows 95 brought that kind of interfaces to the masses and *that* made a big difference.

This is a slashvertisement..first tells you why you want to learn Puppet and what it's used for, and then gets you to read a book to time invest in learning Puppet which will become your "goto" for all DevOps stuff going forward.
The hope being that -- some point, you end up looking into Puppet Enterprise and get out of open source..or you get to the level where you need enterprise and not the open source version, yadda yadda, more people on puppet, the better for their business model, as eventually some will have to go with enterprise for some type of requirement (additional functionality and/or enterprisey support)

Yes, everything should be provided free of charge in all forms, relying only on donations, or just that good feeling of having provided something useful. No sarcasm, and yes, I have my money where my mouth is. I get paid for my time, not for the code I write, and I would consider it personally immoral to do otherwise. If you think that makin

Why would it be immoral to write code and then charge for it? Aren't you charging for the time of development, except that you are taking the risk that the software you write might not sell well? What's the difference between getting one or more customers to hire you to write code, and then writing the code, vs. writing the code, and then selling to one or more customers?

Slashdot has specific guidelines for structuring and posting book reviews. Anyone is free to write a review and offer it for consideration by Slashdot's editorial staff. I tried out the "Puppet 3 Beginner's Guide" as a Puppet beginner. Then I offered my review as a guide for others who might be (1) considering Puppet and (2) curious about what the author has put inside his book--so they can decide if they want to spend money to buy it or not. You are free to think that somehow rises to the level of a dark,

My comment was pointing out the fact that Slashdot used to have earnest, "Book Reviews", about books on topics like..HTML5, C#, etc...which actually reviewed the book.

Here, we get basically an advertisement for Puppet's functionality. This is going along with all the changes Dice has made to slashdot since its acquisition -- posting a lot of "news" and "book reviews" that's really "marketing." The site already has ads -- they're just

LMAO you can't even argue the point that was made bu tyou cite latin.
Jeez.
The point was that the article isn't a review at all... Puppet may be the next best thing since sliced bread, but this review was supposed to be of the book and we all learned nothing about if its any good.

bad review != advertisement. Especially for a product that isn't commercial. Parent poster is talking about them hoping to eventually make money off of the attention. If that's an advertisement, so is everything on slashdot.

You have a real problem with following logical thoughts, don't you? I made no claim that this had anything to do with commercialization (quite the opposite) and you failed to reply to my comment being that this *is* advertising.

Yea, same here. My team of 4 currently manage about 500 production and production lab (customer integration testing and pre-production testing) systems with another team member managing around 500 dev/sqa systems (we're merging in a few months). We're using OpenView to monitor just the production systems but that leaves production lab systems (which we're still responsible for) and soon the dev/sqa systems to be managed. I have a script framework which manages 15 or 20 scripts per system for data gathering

The sysadmins at work manager the existing servers, but not as well as they could: a UAT environment has a different version of Java, a development server somehow gained a non-GNU tar, one of the two production web servers had a manual quick-fix applied and forgotten about. (They're the best sysadmins we can afford -- we're a charity in London, and it's difficult to compete with the salaries the financial industry can pay.)

By using Puppet, I hope we'll have standardised environments. So far nothing's in p

Make sure you get the official documention from Ahttp://docs.puppetlabs.com/ [puppetlabs.com]. I haven't really seen any books or anything else that is as complete and accurate, especially the reference and learning puppet books.

I find the "puppet resource" command is one of the most useful. You can use it to generate a script for a given configuration and use it to duplicate it elsewhere.

"How well does puppet really help in heterogeneous, custom environments? I have Windows and Linux servers."

Quite fine. Of the tens of different systems you can support, there's only one -you see, one among tens, with poor support. That one is Windows.

But that doesn't come for a surprise: we all know Windows is impossible to be properly managed (and I say that from the standpoint of somebody that supports about 2000 windows servers, so I might know something about the issue).

It requires some agent to be installed on a target server which communicates back to the Puppet Master. It's the same problem I have with other such tools that have agents. Infosec doesn't permit such holes in the various firewalls (I have servers in many locations). So I fall back to what I always do. I write scripts that run on the host gathering data, retrieve the data nightly, and can push changes out on the fly or with a nightly scheduled action.

It's a hell of a way to manage around 1,000 unix servers, but it's the easiest way to get the info I need to manage not just production but lab systems as well.

I'll finish downloading the docs and reading up on it out of curiosity but I don't see this going anywhere for us.

For situations where the agents can't talk back to the Puppet Master, you can push out the manifests (config files) to each host and apply them directly, locally. (As if it were a single, standalone machine)

Not sure if there is a way to push the results back to a Puppet Master for aggregation, but there may be a way to tackle that. (Or just back to a central logging server for parsing)

reports can be sent via http. Use foreman if you want a pretty interface, run passenger, choose any port you want.

For that matter do the same with puppet. You can run it on port 80 or 443 using passenger. You'll just have to customize your client config to use the same... so I don't really know why this is any sort of big deal for you.

Anyway, with foreman you can easily see if systems are out of sync, if they're erroring, etc. Might be a little unusual to do it running the way described in the previous post, but should be manageable if you really really really can't get it to work through the existing firewall configs. I've heard of people doing it when they've hit walls scaling as well. And foreman can do a lot more, though it doesn't have to. We started out using it just for the reports portion and facts searching...

It requires some agent to be installed on a target server which communicates back to the Puppet Master.

You can run puppet in masterless mode, against a local copy of the manifests, either managed locally or checked out from a version control repository.

Likewise with salt (my preferred choice over puppet, but both work), you can run either with a master host, or masterlessly. With salt the nice thing is, you can use the same config for both, just invoke the command differently (salt-call --local vs salt).

Infosec is no reason not to automate, just don't automate with a master server if your policies don't permit it.

Actually just scripts running via SSH. No agents on the hosts listening on ports. SSH is on all systems. The scripts are run on the host, data is saved in a data directory. The central server scps the files to a local, server specific directory for later review and parsing.

Puppet has no agents on ports either the agent grabs the config file every x amount of time and uses pre shared crypto keys to cross validate. If your sending data up from your servers for centralized processing that's probably not something puppet or any other config management bit does.

I'm passing PCI, SAE16, etc. audits without a problem and as a matter of fact use Puppet as part of my control framework, and a sound network topology around the various environments. Not knowing what "infosec doesn't permit such holes" means, the counter argument in a generalized sense is that trusting a host to gather its own data (audit itself) and then retrieving it nightly isn't exactly a stable methodology?

[b]eyond ten or so servers, there simply isn't a choice. You can't manage an infrastructure like this by hand.

Poppycockery and blatherskitism! Eleven isn't some magic number where Puppet suddenly becomes useful. If you only have 11-20 servers, there's a good chance that they're heterogeneous in both hardware and software. Puppet might help with keeping UIDs/GIDs the same on all the servers, but it's overkill for such a small number of machines. Most of the other conf files will be unique to each individual computer.

It is overkill for you but, if your boss has any professional acumen (I know, most don't) it wouldn't be overkill for him. A commonly understated value of automated configuration management is that it allows the company to own the knowledge it paid to acquire instead of letting it live only in some employee's memories.

I strongly disagree. I'd happily deploy configuration management tools or an inventory service in environments with 2 hosts. Configuration management isn't just automation; it's also build documentation and a handy gateway between your RCS and your systems.

Agreed. Plus, I maintain dozens of servers with both consistent and independant user IDs. They belong to multiple companies and what with how I know BASH and git, I don't have any problems keeping it all straight either. Some people are incredibly underskilled at sysadmin life if they think 20 servers is hard to maintain.

We wouldn't have Linux at all without wheel reinvention, and I think that comment is generally silly.
As to system management, installing and maintaining third party software adds trust and management and security issues that didn't exist in the first place, so its not all positive either.
For hundreds of machines I can see the value. However, my point was for 20.

I like the idea of Puppet (and the alternatives) but it's Windows support is way to basic to make it viable over on "my" side of the data center. Why not use one of the established config management programs designed for Windows? Because I hate them all, and I wish they would die, and I do a better job programming an AutoIt script to check configs, registry keys, security, etc, than the big names do in their management programs. - HEX