In case you were still interested, I've updated the [Bad link] for my Penn State MacAdmins Conference 2013 Luggage talk. My examples from the talk are finally on-line, so you can follow along in a little more detail if the information already presented in the slides wasn't enough.

The main repository for the talk contains links to several [Bad link], since I tracked each example separately as its own project.

I was trying to install Dulwich on a Mac OS X Lion system this week and ran into difficulty. I kept getting installation failures that included a missing “python.h” and, eventually, llvm-gcc-4.2 failed to compile the module.

I found the situation frustrating, partly because I pretty much own the top search hits about how to install Dulwich and Hg-Git on Mac OS X Lion, thanks to some earlier article.

It turns out that I had reinstalled Lion about two weeks ago, and had not reinstalled Xcode 4. So, I updated to Xcode 4.2 and this completely eliminated my problem. Presumably, it would also work for you — and for future me, since I’m likely to repeat this — even if Lion hadn’t been reinstalled in between.

My personal goal in doing this is to be able to work with the [Bad link] repository. It is hosted on Google Code, uses a Subversion repo, and depends upon Subversion keyword substitution. It presents an interesting challenge.

So, let’s install Hgsubversion to add Subversion support to the Git support we’ve previously set up. We need to take the following additional steps, going beyond what is done in the previous articles. To continue, you must already have installed:

the Xcode tools (including Subversion), as appropriate for your operating system (probably either Xcode 3.2.6 or 4.1 at this point)

Mercurial 1.9.2 or later.

Hgsubversion also needs Subversion to be installed. The Mercurial extension then has two different ways of working with Subversion. The Hgsubversion docs I read indicated that [Bad link] is preferred over the other method — using Subversion’s SWIG bindings — so we’ll install this additional Python module first. The docs also recommend running tests on Hgsubversion before using it on projects, so we will try that, as well.

Clone the Hgsubversion repository from Bitbucket to a temporary location on your system. Under normal circumstances, I would use easy_install again to get hgsubversion. However, version 1.2.1 in [Bad link] didn’t work for me. So, to get a more current working version, I went to the source repository to get the latest software.

Enable the Mercurial Keywords extension in the cloned repo’s .hg/hgrc. (According to the documentation, you should not enable the Keywords extension on a global basis in your ~/.hgrc, but instead do so on a repo-by-repo basis in .hg/hgrc.) We’ll also set it so it only affects Bash scripts with the right filename extension, because we want keyword expansion on just the “instadmg.bash” file.

[paths]
default = http://instadmg.googlecode.com/svn/trunk/

[extensions]
keyword =

[keyword]# expand keywords in all bash and python files in working dir**.bash =

Set your HGUSER environment variable. The next step will need this. The following steps will do it temporarily for your current terminal session, so when you get a chance, you’ll want to read about how to do this more permanently.

$ HGUSER="anonymous"
$ export HGUSER

Try kwdemo. You must be in the directory for the cloned repository and have enabled the Keyword extension.

Edit the .hg/hgrc file for the cloned repository so that you can set up keyword substitution for the “Revision” keyword. In my experience, you’ll need to set up a map for this, unless you want to get the Hg revision number that you see in the default output of kwdemo above.

[paths]
default = http://instadmg.googlecode.com/svn/

[extensions]
keyword =

[keyword]# expand keywords in all bash and python files in working dir**.bash =

[keywordmaps]
Revision = {svnrev}

Try the demo again. I never get this part to work, but you may have more luck. Actual shrinking/expansion, which we’ll see next, does work for me.

Notice that “Revision: 425” is returned when keywords are expanded, replacing the “$Revision” keyword. This is precisely what we need with InstaDMG, where the revision information is reported when the instadmg.bash script is run.

Hg-Git is the Mercurial extension to use if you want to connect to local or remote Git repositories. I exclusively use Mercurial and Hg-Git for all of my Github transactions, so I can personally vouch that it works.

Now that Hg-Git has been updated to better support Mercurial 1.9, let’s see if we can get an Hg toolchain working on Lion. Since I did that on Snow Leopard a few days ago, Hg-Git has made it into [Bad link]. The installation instructions this time are a bit more streamlined, because we can now use easy_install to get Hg-Git and its dependencies.

To get the toolchain set up, we’ll need Xcode. The Xcode suite includes tools we’ll need to make Python easy_install work, along with Subversion (a prerequisite for Hgsubversion, which I’ll talk about in a later article) and other useful tools.

The Xcode installation is a multi-step install process. Both current download methods — the developer download through connect.apple.com (if you have a paid Mac Developer Account) and the Mac App Store — give you an “Install Xcode” application. That application runs a second, real installer that you have to finish before you actually have the Xcode tools available in a ready-to-use state. This is very similar to the situation for Mac OS X Lion, so you may be developing a sense of familiarity with the situation.

To install Mercurial:

Download Mercurial 1.9.2 or later. The [Bad link] are standard Mac OS X packages; get the one for Lion.

Install Mercurial.

To add Hg-Git to Mercurial on Lion:

Download Xcode 4.1 or later if you don’t already have it. You can do this through connect.apple.com or via the Mac App Store.

Install Xcode if it is not already installed.

Open the developer disk image, run the installer inside it, and then run the “Install Xcode” application that was placed in /Applications.

Run the “Install Xcode” application that was placed in /Applications by the Mac App Store.

Open Terminal. Run the following command, which will install hg-git and its dependencies (including dulwich, of which you’ll want version 0.8.0 or later):

Here is my current installation method for Mercurial 1.9 and Hg-Git on Mac OS X Snow Leopard. I use Hg-Git on a sporadic basis to work with projects on Github. That activity is frequent enough that it’s really helpful to be able to get Hg-Git set up and working quickly.

I had a rough time with Hg-Git after I upgraded to Mercurial 1.9 a few weeks ago. It was notable because that was the first time I can remember having incompatibilities between this stack of software. Since then, Hg-Git has changes available that allow it to install and work with newer versions of Mercurial, including version 1.9, and the most recent Dulwich release. These changes weren’t as readily available at the time I upgraded, so I thought it was worth writing this to remind me how to put the right pieces together.

Install [Bad link] 0.8.0 using easy_install. Here is where I learned you could specify a minimum version or a specific version with easy_install. While doing so is not strictly necessary in this case, I am writing it down to remember the next time I need it.

sudo easy_install 'dulwich>=0.8'

Obtain hg-git 0.3.1 using [Bad link]. We will clone it into /tmp, so that it will exist there only until reboot. It’ll get cleaned up and removed, which is fine, because I typically don’t need this repository stored on my drive.

I don't yet have a working installation method for all of this software on Mac OS X Lion. I ran into a stumbling block getting Dulwich installed there, and Dulwich is a necessity for Hg-Git. Update: The situation changed a few days later, and I have instructions for Mac OS X Lion available now.

This may have been a problem isolated to the particular Lion system I was using. I haven't had the chance to duplicate it elsewhere or troubleshoot the problem on the system where I encountered it.

Update: Since I wrote this, Hg-Git 0.3.1 became available through [Bad link], which means you can install it via easy_install. You no longer need to clone the Hg-Git repository and run the setup process manually. Instead, you can run the following single command to get Hg-Git 0.3.1 and its dependencies (Dulwich 0.8.0 or later):

$ sudo easy_install 'hg-git>=0.3.1'

This simplifies the instructions quite a bit, but I leave the original steps above for reference.

After using Acquia Drupal for a while, I took advantage of a trial subscription to the Acquia Network. The network’s services showed me that I had files present in my install that the agent could not account for.

When the archive is extracted in this way, my repository’s working directory shows modified, unknown, and deleted files. This allows me to treat each category of files individually before I commit the changes for a Drupal update as a revision.

$ hg status

The modified files will be tracked normally because they’ve already been added to the Mercurial repository, so I don’t need to do anything special for them.

The unknown files are ones that are completely new, and have not appeared in the same position in a previous revision. They have yet to be tracked by Mercurial, so I have to add them to the repository. To add just those unknown files, then, I have to pick them out from the status listing:

$ hg status --unknown

In order to operate just on those files to add them to the repository, I run a for loop:

This changes the “?” status to “A,” because the files were successfully being tracked by Mercurial.
I use the “--no-status” flag on the “status” command so that just the file paths are printed; the actual status code is not, which is appropriate for the target of the “add” command in the loop.

I do the same basic steps with deleted files. These are files that were in the previous revisions but have been deleted by the --recursive-unlink option from the tar extraction and not replaced with the extraction of the new Acquia Drupal tar archive. If the deleted files had been replaced by the tar extraction, they would either be unchanged (which would not show up in the “status” output) or marked as modified.

To remove the files that are marked as deleted from the repository’s working directory:

However, that may be the same as simply using the following, which I have to explore further:

$ hg remove --after

So, to follow all of these changes in the repository, I run the loop for the uknown files and the loop for the deleted files. The modified files are already tracked, so I don’t need to do anything additional for them. After that, a “commit” will record all of the changes — modifications, additions, and deletions — in the repo.

These commands are based on my current understanding of Mercurial, and they do work for me right now. There could certainly be another better way to do this in one fell swoop — or at least fewer steps. I would welcome that, so if you’re aware of a way, feel free to comment or contact me.

Update: I found that the “hg addremove” command cleanly replaces all of the shell loops I mentioned above. Therefore, I recommend using it instead of the “for” loops I described.

Here is a sequence of commands and output that show how I keep the Acquia Drupal open source content management system up to date with Mercurial, the open source distributed version control system.

In the example below, my Mercurial repositories for Drupal are located in the “drupal” subdirectory of my “repo” folder. Once I’ve moved into that directory, I download the Acquia Drupal distribution with curl and then extract it into my previously-created Mercurial working directory, “acquia_drupal,” using tar.

After extracting Acquia Drupal my Mercurial working directory, I get the status of the repository. It shows there are changes from the last version I checked in — and this includes new files, denoted by a “?” at the beginning of their line.

Since there are new files, I have to add them so they’ll be tracked by the repository. I only need to add in the parent directory for any changed files, and any new files within it will also be added for tracking.

Excellent; the new files have been added. After this, I just need to accommodate the deleted files that no longer need to be tracked (created when using the “--recursive-unlink” option on tar). For that, see my newer instructions.

Now that the right files are being tracked, I need to commit the changes — modified, added, and deleted files — to the repository. This will create a new revision in the repository’s history, which I’ll tag with the text “Acquia Drupal 1.2.0.”

Once this revision is checked in, I can use it to propagate changes to other repositories. I keep the main Acquia Drupal distribution in its own repository, and then use the “hg fetch” command to pull its changes into one where I track contributed modules. That second repository is then pulled into a third repository which stores just the changes for my production Web site. The use of three repositories in this way modularizes and isolates the updates.

While trying to troubleshoot what I’d done to mess up the [Bad link] repositories managing my Drupal installations last weekend, I really would have liked a way to see what files had changes in specific revisions. Each revision to a Mercurial repository affects some files, of course, but it seems awfully hard to figure what files changed in that check-in.

I have since found a way to do that by [Bad link]. To customize output, you can create templates on the command line (with --template) or for more powerful reformatting, create an output style file.

I struggled for a while to figure out how to use style files, and eventually came up with something that works for me so far.

Since I’ve installed Mercurial from Lee Cantey’s [Bad link], I created the file “map-cmdline.changedfiles” at the “/Library/Python/2.5/site-packages/mercurial/templates” path. (Where you put the file may vary depending on where Mercurial is installed, and I’m sorry but I don’t know where it gets installed on other systems.) The contents of “map-cmdline.changedfiles” are below, along with my possibly inept description of what each line is doing:

# Get all of the files in the selected revision

and stringify them, whatever that means

but do not 'tabindent' or wrap them to 68/76 columns

Without first setting changeset to the list of files

you won't get output from subsequent lines

changeset = '{files|stringify}'

List modified files, one per line

preceded by M to mimic hg status

file = 'M {file}\n'last_file = 'M {file}\n'

List added files, one per line

preceded by A to mimic hg status

file_add = 'A {file_add}\n'last_file_add = 'A {file_add}\n'

List deleted files, one per line

preceded by ! to mimic hg status

file_del = '! {file_del}\n'last_file_del = '! {file_del}\n'

I don’t know why the “map-cmdline.” portion of the filename is there, but as long as I have it, I can call the style file from the command line with what follows the period. So, I can call the style with “--style changedfiles” — and that tiny bit of voodoo seems reasonable enough to me. (The other styles in the directory above, many of which end in “.tmpl” extensions, seem related to the Mercurial Web server, hgweb. I tried, but I couldn’t use their names at the command line, with or without their extensions. Plus, their contents looked HTML-ish.)

With the “map-cmdline.changedfiles” style file saved in that location, I can call Mercurial’s “log” command:

$ hg log --style changedfiles -r tip

… which gives me a list of the files changed in the “tip” (or latest revision) of the repository. I could substitute in any revision identifier for “tip.”

I haven’t actually seen the “file_add” and “file_del” keywords in action; every time I’ve used this style file in the manner described, I’ve only seen files marked as “M” — even if I’m looking at a revision where new files were first checked into the repo. I’m confused by that, but I’m not going to let it sour my day at this point.

There might have been an easier way to do this but I didn’t find one last weekend. It took me some time to figure even this bit out, and I hope writing this post saves someone new to Mercurial from future frustration.

In my [Bad link]-based workflow for updating [Bad link] sites, there is a sequence of commands I need whenever a new version of Drupal comes out. I have a hard time remembering the options for “tar” in this sequence — and my [Bad link] differs from what I need to do on my Web host — so I need to help my memory. The tar command, as constructed below, places its output into the specified destination directory.