Autumn cleaning

As you may have noticed from “scheduled downtime” posts on this blog, the disk space used on gitorious.org has increased quite a bit over the last few years.

All our repositories are stored on large and fast SAN storage, but each time we’ve needed to add more disk to the servers, we’ve had to take down both the physical hosts and the virtual machines to have the hosts discover the increased storage available. This has given us some downtime whenever that happened.

To make matters worse, whenever we reboot our servers a file system check has usually been necessary. This takes quite some time when the file systems grow to be as big as ours, leading to downtime of several hours.

We have just purchased some new servers, and will be setting up new servers for gitorious.org – we expect this to have a nice impact on response times on our services. We have also bought our own SAN, and will start moving all repositories to the new SAN, using smaller file systems for storing repositories.

Looking over the tens of thousands of repositories on gitorious.org we discovered that over half of them have never been pushed to. This can have a number of reasons, but a lot of new Git users instinctively create a server clone of a repository they’re about to fork; not realizing that you don’t need a server-side clone to commit to your repository.

When we start moving the repositories, one of the consequences will be that hard links from the previous file systems are not preserved on the new file systems. Git uses hard links when you clone a repository, so a repository clone with zero commits will take up virtually no disk space at all. The bad news is that moving all the repositories to a new file system will result in an explosive growth in disk use, as a lot of data will be duplicated.

Unused repositories will be removed September 1st

Based on the above, we have decided that repositories that have never been pushed to will be removed from gitorious.org. If you have a repository that you really want to keep, you can keep it from being removed simply by pushing to it. You may push a tag, a single commit, whatever; as long as you push something to it, you’re off the hook.

We will let inactive repositories stay until September 1st, 2012, so please push to your repositories before that date.

We will keep you posted on the migration process over the next few weeks, and hope Christmas will come early this year in the form of a faster, more stable and more beautiful gitorious.org.