Blog where Leo talks tech

Writing one line shell scripts with bash

April 10, 2012

If you are using ruby with bundler and Gemfiles properly, you probably know about running commands with bundle exec. However, sometimes this does not get you quite the right results, in particular if your Gemfile is not quite precise enough.

For example, I had an issue with cucumber + autotest + rails where I had both rails 3.1 and rails 3.2 apps using the same RVM. Since I was confused, and in a hurry, I figured one brute force option to unconfuse me would be to simply remove all old versions of all gems from the environment. I did just that, and thought I’d explain the process of incrementally coming up with the right shell script one liner.

$ gem uninstall actionmailer -v 3.1.1
You have requested to uninstall the gem:
actionmailer-3.1.1
rails-3.1.1 depends on [actionmailer (= 3.1.1)]
If you remove this gems, one or more dependencies will not be met.
Continue with Uninstall? [Yn] y
Successfully uninstalled actionmailer-3.1.1

Ok, so we need to have it not ask us that question. From studying the command line options, the magic switch is to add -I.

So once we have pinpointed a version to uninstall, our command becomes something like gem uninstall -I $gem_name -v $gem_version. Now we need the list of gems to do this on, so we can run that command a bunch of times.

We’ll now start building our big fancy one-line script. I tend to do this by typing the command, executing it, and then pressing the up arrow to go back in the bash history to re-edit the same command.

Looking at the gem list output again, we can see that any gem with multiple installed versions has a comma in the output, and gems with just one installed version do not. We can use grep to filter the list:

Great. Now we need to extract out of this just the name of the gem and the problematic version. One way of looking at the listing is as a space-seperated set of fields: gemname SPACE (version1, SPACE version2), so we can use cut to pick fields one and three:

Ok, so it has 3 versions. Really in this instance we need to pick out fields 3,4,5,… and loop over them, uninstalling all the old versions. But that’s a bit hard to do. The alternative is to just pick out field 3 anyway, and run the same command a few times. The first time will remove jquery-rails 2.0.1, and then the second time the output will become something like

jquery-rails (2.0.2, 1.0.16)

and we will remove jquery-=rails 1.0.16.

We’re almost there, but we still need to get rid of the ( and , in our output.

Ok. That looks like its the list of commands that we want to run. Since the next step will be the big one, before we actually run all the commands, let’s check that we can do so safely. A nice trick is to echo out the commands.

Note that, as a reusable program or installable shell script, this command really isn’t good enough. For example:

It does not check the exit codes / statuses of the commands ran, instead it just assumes they run successfully

It assumes the output of gem list will always match our expectations (for example, that the output header does not have a comma in it, or that gem names or versions cannot contain space, comma, or -- this may be true but I wouldn't know for sure)

It assumes that -I is the only switch needed to prevent gem list from ever asking us questions

The best approach for a reusable command would probably be to write a script in ruby that used the RubyGems API. However, that would be much more work than writing our one-liner, which is “safe enough” for this use-once approach.

(For the record, this didn’t solve my rails + cucumber woes. Instead, it turns out I had run bundle install --without test at some point, and bundler remembers the --without. So I needed rm Gemfile.lock; bundle install --without ''.)