Semi-organized collection of thoughts, notes and anything else I feel like writing about

Menu

Tag Archives: DevOps

At times, I have to switch between a few different Heroku accounts. Apart from having to login again, the other annoyance is having the right SSH key active. If you don’t have the right SSH key active, (i.e. if the SSH auth agent has more than one key added to it, or if it has no keys at all), you’ll see errors that look like:

Your key with fingerprint: ... is not authorized to access <application-name>.
fatal: the remote end hung up unexpectedly

The proper solution for this involves configuring them in the SSH configuration files (~/.ssh/config or /etc/ssh/ssh_config). [See this article or ssh_config manpage for details]

But I’m loath to maintain all that configuration just for the sake of occasionally switching between accounts. Here’s what I usually do instead:

Clear any active identities (removing all ambiguity about which SSH key should be picked up for auth)

$ ssh-add -D

$ ssh-add -D

ssh-add key for the account

$ ssh-add ~/.ssh/an_account_key

$ ssh-add ~/.ssh/an_account_key

Push to Heroku

$ git push heroku-remote master

$ git push heroku-remote master

Of course, this assumes that the key is already associated with your Heroku account. If you haven’t, you can do that (after heroku login) with:

$ heroku keys:add ~/.ssh/an_account_key

$ heroku keys:add ~/.ssh/an_account_key

Note: On Linux, if you’re on GNOME, the gnome-keyring-daemon keeps adding keys back to the auth agent as you keep trying to remove them with ssh-add -D. So, it’ll look like the command is not working. The solution is to disable the damn thing (Google for it). I find the daemon annoying for the popups it keeps throwing at me, so personally, I’d be glad to see it gone.

While I use this technique mostly with Heroku, this is useful for any situations involving SSH and multiple keys. It’s useful when switching between SSH accounts, or switching between GitHub accounts (or other Git accounts), and in general, anything involving switching SSH keys for SSH auth. Of course, if you find yourself switching between the same set of accounts (keys) frequently, consider configuring them using ssh_config.

Have you ever wished to have a finer control over your Heroku git repository? There’s a neat little Heroku plugin that gives you just that: heroku-repo.

Around the first release of my current project, things were happening in a frenzy. There were plenty of last minute fixes, and each of them would get built and pushed to our staging instance. On occasions, when people couldn’t wait for the build to finish (wasn’t a particularly long build though), or when the build was a bit flaky, they’d push directly to the Heroku staging repo. At some point, Heroku started rejecting our pushes, because it’s git tree had diverged from ours. It wasn’t nice, but we resorted to doing force pushes to the Heroku git repo from then on.

Once the release frenzy was over, I started investigating the issue. I started by trying to get a local clone of the Heroku repo, to see if I’d find something there, but it kept timing out. I opened a shell on the staging instance and tried searching for a repo somewhere there, but in vain. Heroku doesn’t host the repo from there. I had pretty much decided to blow up the staging app and recreate it, when I found this plugin.

It’s a plugin for the Heroku toolbelt that, in a sense, gives you raw access to the git repo that your Heroku application uses. Using this, I managed to download the git repo locally, which took quite a while because the repo had grown to some insane size (was a few hundred MBs, nearing a GB). Digging into the repo, I found a huge pack file that was causing all the issues. I suspected it to be because of a huge binary accidentally checked in by a team member, but my git-fu isn’t really good enough to say for sure. Running a gc on the repo using the plugin didn’t really help. So, I was still left with having to blow up and recreate the staging app.

I thought it’d be nice if I could just reset the Heroku git repo, and start over instead of having to recreate the app (and then add all the addons to it, which was a bit of a hassle since there were paid addons and I’d have to contact the instance owner to re-enable them). So, to figure out where the repo is hosted and how the plugin manages access it, I went through the plugin source code. Turns out, the repo is hosted on S3, and Heroku toolbelt exposes the S3 URL to it’s plugins. Better yet, the plugin itself had an undocumented command to reset and upload an empty git repo back to S3 🙂

This is pretty amazing. I can now start over with a clean repo in case my Heroku repo is messed up for any reason. On top of that, I can now deploy an entirely new app into my Heroku instance without leaving any dangling commits (not that it’s a common usecase, or even a useful one). See this protip for details: https://coderwall.com/p/okrlzg.

To install the plugin, do:

$ heroku plugins:install https://github.com/lstoll/heroku-repo.git

$ heroku plugins:install https://github.com/lstoll/heroku-repo.git

Here’s a few commands that I found useful:

Download the Git repo as an archive (useful when you can’t clone from Heroku)

$ heroku repo:download -a appname

$ heroku repo:download -a appname

GC the repo (on Heroku)

$ heroku repo:gc -a appname

$ heroku repo:gc -a appname

Reset the repo and upload an empty repo

$ heroku repo:reset -a appname

$ heroku repo:reset -a appname

The plugin has a few more useful commands. Do check it out on GitHub: https://github.com/lstoll/heroku-repo. Also, I’d recommend going through it’s source code to see how it works. I thought it was pretty neat.