Assuming you have a puppet configuration as set up according to “development, testing, production” like in Environments en git, een must (Dutch). Basically it’s a setup with three environments based on a single repository.

The article uses no branches in git, which is a shame, also a single user is implied (root).

In this post I’ll attempt to expand this idea to include both, in order to facilitate multiple sysadmins to work on puppet changes and enhancements without bothering each other in the process.

Multiple users

Being a distributed VCS, Git is written for facilitating multiple users to work on (a clone of) the same repository. The workflow that works best for tracking changes by multiple users is to have users clone a repository, work on changes under their own name and when the time comes to exchange the changes, push or pull either to a central or anothers clone of the repository.

If e.g. Chantal wants to work on a specific feature, she can make a feature branch in her clone, work on the feature and push the feature branche back to the origin (the central repo). Or work it out completely and merge it into her master branch and push that back.

The problem is actually a bit more complicated, because we’re working with a puppet configuration. Testing it requires a puppet master and puppet client hosts. What if we could test the branch, without merging it into the main puppet environments (development, testing, production)?

Expanding the three stage puppet config model

In the article mentioned, changes are first developed and tested in the development environment, this requires a specific test machine that nobody relies on for work. When a feature works, the developer pushes the changes to the central repo. then it can be further tested in the testing environment, e.g. on less important servers. This happens when a “git pull” command is done in the clone called testing.
Finally, when the tests are all OK, the changes are merged in the production environment and all agents working in production environment get the new changes.

This doesn’t involve branches and if multiple admins work on new features in develop, they likely interfere with each other and stuff will break!

Now I’m not yet very familiar with puppet, but I understand it is possible to create any number of environments which can be referred to on the clients via their puppet agent configuration.

In general it would have been useful to use branches in the original model so that e.g. master is tracked only on the production environment, testing only in the testing environment and development only in the development environment. Any commits done on the development branch cannot accidentally end up in the testing or production branch/environment without an explicit merge!

Once we have these branches, what’s to stop us from adding more branches and corresponding puppet-environments to test the puppet code while developing it, without interfering with either production or other admins working on different features? Nothing!

For each feature branch we want to test, a new entry must be created in /etc/puppet/puppet.conf to add an environment. The new environment should be in the same base directory as the normal branches, since hiera configuration cannot handle multiple locations for the hiera data.

It looks like Development as a separate environment is not needed anymore, this is taken over by the feature branches and corresponding environments. These branches and environments are temporary and can be deleted after their contents has been integrated in testing.

NB, so far this is a theoretical excercise, I’ll post an update when I’ve had some time testing this model…

update: 4 sept 2013:
I’ve updated the text regarding where the feature’s environment directory can be, as it turns out that hiera.yaml is configured to look under the standard directory /etc/puppet/environments/%{environment}/hieradata.

For this to work easily, it’s probably a good idea to create a group puppetmasters and add all users who can modify/test puppet to this group. Then make /etc/puppet/environments/ writable for this group. This way, there’s no need to become root to create and modify a test environment.

At work, we had a desire to monitor vhosts on a webserver, some webservers have lots of vhosts and they may even change relatively often. The same is true for tomcat servers, though they are usually on a different port…

I looked around but I couldn’t find any specific solution to monitor vhosts in large quantities and be flexible enough to work on a different port as well.

So I wrote my own nagios-plugin. It works well enough and it requires some sort of content-string to match what comes back from the server. This isn’t always easy when you run lots of instances of the same software (like geoserver).

The script uses an input file, so you can change the vhosts to monitor, without restarting nagios.

If you’re running the vhost normally behind a high availability service (pound/relayd/haproxy) you would be able to check the vhost’s availability, but not the backend’s vhost availability. With this script, you can specify the host-address to use (its IPnumber) so any proxy will be bypassed, which is how I’ve configured it here (-H). If you leave this out, the check will use the vhost’s fqdn in the full URL.

As you can see, it uses custom macro’s from the host definition, so you can have one place to manage the service definition and each host defines it’s own (_VHOSTFILE) variable to specify the list of vhosts for that server.

Now v1.8.1 is out (it was released on December 31st, 2012) the code to make a prompt with git’s status using colour is now in a stable release.

To use it, you don’t need to install the entire release, if you just want to use the colour option, just copy the file contrib/completion/git-prompt.sh to a safe place (e.g. ~/.git-prompt.sh) and make sure you source it in your .bashrc, along with some lines to actually active the code. I’ll show here how to do that.

And you’ll get a prompt with colour when inside a directory with a git repository.

The code does the following:

test if ~/.git-prompt.sh exists and source it

set variables to activate showing git’s state in the prompt, when __git_ps1 is called

GIT_PS1_SHOWDIRTYSTATE; show a * to indicate unstaged files or + for staged files

GIT_PS1_SHOWSTASHSTATE; show that something is in the stash ($)

GIT_PS1_SHOWUNTRACKEDFILES; show files in the current directory that are not being tracked by git

GIT_PS1_SHOWUPSTREAM=”auto”; show upstream status (can be further customised

GIT_PS1_SHOWCOLORHINTS; show in colour what the DIRTY state is, must be used in combination with GIT_PS1_SHOWDIRTYSTATE and PROMPT_COMMAND mode

define PROMPT_COMMAND, which is a command ( a function; __git_ps1 in this case) and in order to get __git_ps1 to work properly in this mode, we need to give it two (or three) parameters. The first argument is what is put in the PS1 variable before the status of the git tree, the second argument defines what comes after the status of the git tree in PS1.

By default (with just 2 arguments), __git_ps1 will put the string with branch information in a printf format string ” (%s)”. If that isn’t how you want it, you can add a third parameter with a custom format string.

Examples:

branch before the prompt:

PROMPT_COMMAND="__git_ps1 '' '\u@\h:\w\\$ ' '%s:'"

branch between square brackets:

PROMPT_COMMAND="__git_ps1 '\u@\h:\w' '\\$ ' ' [%s]' "

You may be wondering: “Why aren’t you setting the PS1 variable?”. The thing is that PROMPT_COMMAND is setting this variable, so we don’t have to (it will be overwritten anyway).

Bash will call the function defined for PROMPT_COMMAND every time it is going to prompt the user for a new command.

Something else you could do in PROMPT_COMMAND: flush history before printing a new prompt
You could do this in combination with the __git_ps1 function by setting: PROMPT_COMMAND="history -a; __git_ps1 'arg1' 'arg2' ['arg3']"

The “old” way still works, but colour isn’t possible using command substitution:

A long time ago, when my colleages at at-computing were messing with git, I was playing with putting the branch name and a colour for the different states of a git tree inside the prompt, so whenever I would enter a directory with a git repository it would show up in the prompt and moreover it would show whether there were changed files present (the colour of the branchname in the prompt would change to red and/or yellow if the changes were already staged).

The way this worked was by calling a function using command substitution from the PS1 (since this is evaluated by bash every time the prompt is printed).

PS1='\u@\h:\w $(gitprompt)\$ '

This was good enough for me, and I even accepted that my commandline wrapping got messed up. I didn’t know or cared enough about this and I called it a bug in bash (sorry, it wasn’t).

A few years later, I ran into a file (I forget why or how, but I did) in my debian installation called git in /etc/bash_completion.d/. Basically it did the same, but both better and worse. It had far more sophisticated code to finger the git repository and it was faster and had no wrapping issues, but it had no colours 🙁

I thought: well, it’s free and open, so I can just modify it to print colours.
And I did, it wasn’t that hard, but now it had the wrapping issue again (I didn’t notice, since I was used to it). I proudly posted my solution to the git mailinglist and, of course, people noticed problems…

it had wrapping issues, someone mentioned PROMPT_COMMAND

it had the wrong colours (I should have used only colours already used in git when it uses colours)

did it work with zsh, like the existing version

Ok, back to the drawingboard, searching for a solution to the wrapping problem I found out that bash requires \[ and \] around terminal command codes that produce zero length prompt string output, like beeps and colours.

I figured out how to do it with PROMPT_COMMAND, but I lazily copied the __git_ps1 function to create __git_ps1_pc, since it was so different to set the PS1 using that way.
I posted my “solution” again and this time I got some more comments:

I was duplicating code

it didn’t work with zsh

still wasn’t using the right colours

So again I went coding away, fixed all of the issues and after some good suggestions from Junio C. Hamano (the git maintainer) I managed to get it into an acceptable state.

However, this wasn’t the end of it. A while later, I got a question about the usage of git-prompt.sh in a release candidate. It turned out the documentation wasn’t clear yet. After some e-mails I thought the issue was fixed (but I hadn’t provided a patch). Later, I was showing the colourful prompt to some people and I noticed the improved documentation was not yet in the RC2.

I mentioned this on the mailinglist and in return I found that Junio wasn’t entirely happy with the situation and the code. I further modified the code and documentation to fix the most urgent worries, but in the end, I’m also not entirely happy with the end result in 1.8.1.

The issues that needs fixing is the hacky way to differentiate between command-substition mode, which works in bash and zsh and is more or less as it was, and PROMPT_COMMAND mode. The way the function switches modes is to count the parameters, 0 or 1 for command substition mode, 2 or 3 for PROMPT_COMMAND mode.

I’d rather have different functions for the different modes, but the trick will be to not have duplicated code to maintain.

When working with latex, the book mode has an additional level called chapter, but in article or report mode, this isn’t available, then “section” is the top level. Here’s a small bash+sed script to convert from book (with chapter) to the other styles… (back is a different matter, esp, since subsubsection is usually the lowest level in both modes, so you can’t convert subsubsection to subsubsubsection…)

In our setup we have at least two potential default gateways available for our office network. The office network is behind a pf firewall, which, among other tasks, is doing NAT for the office network.

One of the uplinks is relatively cheap and the other is used to connect our servers in the datacentre. For stability we want to use the server uplink for internal management (remote login) and for the rest of the Internet traffic (you know, people watching youtube etc.) we want to use the cheap cable provider.

As I’m relatively new to pf, I figured I’d be able to find some good examples on how to use pf to facilitate this kind of setup, but no, nobody seems to have put up a good example of how to do this.

After some digging, I found that reply-to is probably the solution, but no example syntax is provided anywhere, so eventually I turned to IRC to figure out a possible way to let pf understand what I want.

Returning back to the problem, basically the default route determines where Internet traffic goes out and this trumps any states kept by pf about incoming connections.

Incoming connections are rare in our setup, but they are used to work on office pc’s from home. In pf this can be done using rdr-to syntax to translate from the external address of the firewall to the internal (rfc1918) address.

When the incoming connection comes in on the firewall’s non-default external port, a nasty thing happens. Returning traffic goes out the other port, and pf cannot match that traffic to the state of the incoming traffic.

That is, unless you tell pf to watch for and reconnect this return traffic to the existing state. (I have no idea why anyone would not want this to happen).