Other posts related to time-drive

Innovation happens in fits and starts. No single person, no matter how brilliant, capable or amazing is able to consistently create awesome stuff day in and day out. Whenever engaged in any creative endeavor, there will be hills and valleys in productivity.

These happen for lots of reasons: maybe you’ve hit the bottom of the ideas barrel, or lost the source of your inspiration. But there is another cause: you’ve reached your goal or solved the original problem.

Reaching such a plateau is a good thing. It means you’ve accomplished something. It’s a moment for patting yourself on the back, looking over your accomplishments, and otherwise appreciating that you’ve done something. But at the same time, it doesn’t mean that you should sit still or stagnate. So, in addition to looking at what you’ve done, it’s also important to take stock of the path ahead and start formulating plans. You might even want to find a new challenge.

And this is where Time Drive has been for the better part of six months. Ever since releasing version 0.3, there have been few changes to Time Drive. Sure, both Philippe DeLodder (the other amazing Time Drive Developer) and I have fixed a few bugs and made tweaks; but nothing really major.

My personal reason for this is simple: Time Drive meets my needs. It does a good job of backing up my data. It lets me copy things to an attached hard drive, or across the internet. It makes it easy to search for and restore my files. In short, I’ve reached my original goals. I use Time Drive every day, and it’s saved my digital bacon a couple of times.

But that doesn’t mean that development is finished, or that I don’t have ideas. Because, I do.

Even though much of the original problem has been solved, I’ve found a new one to work on; and given that you use the current version, I thought you might be curious to hear a little about it.

(In addition to introducing the new version of Time Drive and making a few other project announcements.)

In 2007, an engineer at Google named Amit Singh released one of the best programs for Mac ever written. It was amazing, incredible and even a little magical; and most Mac users probably never even realized it existed. It was called MacFuse.

MacFuse was a very interesting program. It took something very old (the computer filesystem) and made it new again, and thereby changed the world.

In the bad old days, writing file systems was the stuff of nightmares. It required blood sacrifice, misery and pain. If you messed up big, it could crash an entire computer system. If you only messed up a little bit, it would “only” result in data corruption and the loss of digital memories.

Fuse/MacFuse changed all that, though. Instead of requiring that an aspiring file systems developer work at the level of the kernel, it allowed for the work to happen in the “user-space”. This meant that the implemented file system code could be run by non-privileged users, and for all intents and purposes the file system was nothing more than a simple program.

By greatly simplifying the process of writing file systems, MacFuse opened the doors to all kinds of innovation. The code got incorporated into virtualization packages such as VMware Fusion and Parallels; and other developers made it available to end-users through projects such as MacFusion. MacFuse made it easier for users to connect their computers in new and interesting ways. If you wanted to connect to Wikipedia, or Google, or Flickr via a MacFuse plugin, you could.

It was file exchange Nirvana; and then, Snow Leopard killed it.

In fairness, I should probably point out that Snow Leopard killed lots of things. It caused old programs to stop working, corrupted data, and destroyed user accounts outright. From its release, Snow Leopard has been a gigantic mess. (Until quite recently, that is.) And though Apple has fixed some of the bugs (since version 10.6.3, things have gotten a lot better), there are some very important programs (like MacFuse) that still don’t work, and this caused problems. In the case of MaFuse, the problems have have come to (somewhat) impact my personal and professional lives. To be more specific, the broken state of has been hampering the release of Time Drive (my backup program). For, you see, the most recent version of Time Drive requires a Fuse plugin for one of the new features to work correctly.

On Linux, there isn’t a problem. The debian package system downloads sshfs and, boom, you’re ready to go. On Mac, though, it doesn’t work at all.

It doesn’t even pretend to work, and I find that to be rather annoying. More than annoying, actually. I find it to be aggravating, infuriating and it makes me want to hit someone. But as much as I would like to engage in violence, that is not going to solve the problem. And I very much wish to fix the problem. So … I did.

With the help of various online postings, I patched the MacFuse source code and created a version that is compatible with Snow Leopard. I also figured out how to build a command line copy of sshfs so that Time Drive will work. The combination of these two actions both made me feel better and even made Mac OS X feel like a magical platform again. (Well, almost.)

The remainder of this post will describe how you can get these things working for you. It will cover where to get the files, how to install them, and a brief guide to life simplification via MacFuse.

Imagine how awesome it would be if this announcement read: “Time Drive has been completely rewritten from scratch (yet again) to take better advantage of the paradigms of modern computing! Version 0.3 has hundreds of updates and new features which will make your life easier and more fulfilled!”

There’s just one little problem … such an announcement wouldn’t necessarily be true. (Marketing hyperbole, I never knew thee!)

The truth is this: Time Drive is a simple backup program that does a good job of backing up your data. It offers a nice list of potential backup options ranging from an attached hard drive to a computer over the network or across the internet. It makes it easy to search for and restore a lost file.

In short, Time Drive seeks to change the world by making an act of computer maintenance more convenient. I’d like to think that it Just Works.

But the real test of a program isn’t how well it works, but how easy it is to fix when broken. A good program does what you want, but a better program helps you get back on track when things go wrong. Back when I was looking at other backup programs available for Linux, this was my number one frustration. Most of the applications would work (for the most part), but I could never troubleshoot or repair problems when they happened. There just wasn’t enough information available.

For an example, let’s take SBackup. It’s a lovely little program, with one horrible flaw. You have absolutely no way of knowing if it is working. It doesn’t keep log files, it doesn’t notify you if a backup job failed. It doesn’t let you know if it is running. Its simplicity is actually symptomatic of a flaw: it’s incomplete.

These were problems that I desperately wanted to avoid with Time Drive. And now that I’m announcing version 0.3 of the program, I’d like to think that I have. So, instead of marketing hyperbole and false promises, here’s the real announcement:

Time Drive 0.3 includes a number of refinements that make it easy to both backup your data and to figure out why a backup might have failed. It’s better, easier and more refined. In the rest of this post, I’ll explain why.

While Mac OS X shares many things in common with other Unix operating systems, it also has a couple of missing parts. This can make installing software more difficult when compared to Linux or other *nix variants, and unfortunately, Duplicity happens to be one of those cases.

But just because it is more difficult to install Duplicity on Mac doesn’t mean that it isn’t worthwhile. Duplicity is one of the best command line backup programs available anywhere. Using the same program, you can backup to a local hard drive, FTP server, over SSH or even to Amazon S3. It uses the rsync algorithm, which means that backups happen quickly and only the parts of files that actually changed get copied. Subsequently, backups are smaller.

Nor does it mean that installing Duplicity is actually all that hard, it’s just a bit tedious. Like many other *nix programs, Duplicity requires a number of additional programs (called dependencies) that you will have to search out and install separately. To make that process easier, this article explains how. Below, you’ll find links to the download pages and instructions for how to compile each of Duplicity’s dependencies on Mac OS X. The instruction set has been tested both on Mac OS X 10.5 (Leopard) and Mac OS X 10.6 (Snow Leopard).

Since version 0.2.2 of Time Drive, installing Time Drive and keeping it up date has gotten a great deal easier. Instead of requiring that you install from source, we now offer Debian based packages through the Time Drive Personal Package Archvie (PPA). This article will briefly describe how to install Time Drive using that PPA.

Background

But before leaping headlong into the nitty gritty, let me provide a bit of background about packages and repositories. Unlike Windows or Mac OS X, software on Linux is organized into a container called a package. The package contains the files needed to run the program in addition to artwork and configuration information. Any given program may require other programs to be installed before it can run. These other programs are referred to as dependencies. Ubuntu and uses a tool for managing packages and dependencies called Apt.

The benefits of using Apt over manually installing things is that Apt figures out all of the dependencies automatically. Further, it keeps everything up to date by periodically scanning the online repository (PPA) and downloading any updates. In practice, Apt works much like the Windows Update service, only better, since it monitors every piece of software installed on your computer.

Installation Instructions

There are two slightly different sets of instructions for installing Time Drive. If you use version 10.04 (Lucid Lynx) or 9.10 (Karmic Koala), follow the first set. If you use version 9.04 (Jaunty Jackolope) or earlier, follow the second. In both, though, the process is similar.

Configure Ubuntu so that it will trust both Time-Drive-Devel and the Duplicity packages.

Install Duplicity and Time Drive

Ubuntu 10.04 (Lucid Lynx) and 9.10 (Karmic Kaoloa)

In the most recent versions of Ubuntu (10.04 and 9.10), it is much easier to add PPAs to your Software Sources list. As a result, you can install Time Drive with three commands from the terminal. The first command adds the Time Drive PPA, the second updates your source list, and the third will download and install Time Drive with all needed dependencies:

Ubuntu 9.04 (Jaunty Jackolope) and Earlier

Step 1: Add the Package Repositories to the Ubuntu Software Sources

Open the Ubuntu Software Sources by going to the System –> Administration –> Software Sources. Then, select the “Third-Party” software tab of the application. “Software Sources” requires root access, therefore, you will be prompted to type in your administrator password.

We need to add two package repositories to the list, one for Duplicity and the other for Time Drive. Depending on the version of Ubuntu that you are using, copy and paste the following lines (one at a time) into the “Add Sources” dialog. After each line, press the “Add Source” button.

When finished, press the “Close” button. Ubuntu should then notify you that the software information is out of date and needs to be updated. Instead of clicking “Reload,” click the “Close” button. We will refresh Apt from the command line in Step 3.

Step 2: Configure Apt to Trust Time Drive and Duplicity Packages

After adding the repository information, you will then need to add the repository key to your system’s list of trusted keys. This is most easily done by running the update-launchpad shell script. (To download, right click on the link and select “Save As”.) When run, the script automatically downloads any needed signing keys and automatically add them to the “Trusted Sources” list.

To run the script, go to Accessories –> Terminal and navigate to where you saved the file, then type:

cd /path/to/folder sudo bash update-launchpad.sh UbuntuVersion

Be sure to substitute the correct Ubuntu code name for UbuntuVersion in the second step – karmic for 9.10, jaunty for 9.04, or intrepid for 8.10 – otherwise, you will get an error. I happen to use Ubuntu 9.04, therefore, for my setup, I would type the following:

sudo bash update-launchpad.sh jaunty

Step 3: Install Duplicity and Time Drive

Finally, we are ready to install Time Drive and Duplicity. From the command line, refresh the package lists by typing:

sudo apt-get update

Then, to install time-drive and duplicity, type:

sudo apt-get install time-drive duplicity

Application Updates

As noted above, whenever a new version of Time Drive, Duplicity, or another dependency is added to the package repository, your system will automatically be updated. However, should you upgrade your system to a newer version of Ubuntu, you will need to return to this page to add the repository for that version.

When it comes to most things, starting fresh is a blessing. The reason for this is rather simple, when starting over you don’t have to worry about baggage. After all, baggage is only valuable when on holiday; otherwise, it just slows everyone down.

This is especially true for software. Over time, computers tend to accumulate a rather potent type of digital baggage that can be very difficult to get rid of. And that digital garbage results in inconsistencies that can cause enormous – and usually unforeseen – problems.

However, even though starting fresh is usually the best option, that isn’t always true. Sometimes, it’s better to risk the problems and incompatibilities. For example, starting over may mean that you destroy hours worth of customization, or that you lose work already created because the older version are not compatible with the new.

Unfortunately, the general rule is also somewhat true of Time Drive. So, if you were one of those stalwart and brave individuals who decided to experiment with Time Drive 0.1, this post is for you.

In the last few days, I have been in touch with a number of people who have experienced a number of said inconsistencies and problems. And while several of these problems ended up providing insight on mistakes made during development, some of the others were changed on purpose. That is to say, the so called “bug” was actually a feature.

After fielding a couple of particularly angry e-mails, however, I thought that it might be good do a formal write up that describes how to work around these incompatibilities. And while no one likes to squash bugs or fix things that previously worked, rather fortunately, every one of these problems can be overcome with a little bit of effort and patience.

More Stable, More Secure, More Settings and Supports Amazon S3

The first time that you attempt to do something, it’s pretty much a guarantee that it’s going to suck. This doesn’t necessarily need to be a bad thing. Shows like America’s Funniest Home Videos and MXC have found dozens of way to cash in on the humiliation of their participants. (And what better exemplifies pure suck than a golf ball to the groin?)

It, therefore, shouldn’t come as any surprise that creative pursuits are no exception to the general rule of suckiness. After all, you have to overcome inexperience and ineptitude to produce anything. The only way to ensure that a release doesn’t suck is to finish a first draft and revise heavily. Which requires a great deal of work.

Yet … as interesting as that might be, this isn’t a post about the creative process. It’s about Time Drive and I should probably admit that Time Drive 0.1 had a few … rough … edges. Sure, it mostly worked, but it was new software and did too many strange things to declare anything other than a “work in progress.” But Time Drive 0.1 was a first release and first releases suck.

Second releases, however, offer a chance to clean things up, refine the bleeding edge, and otherwise deliver the goods. Maybe that’s why I’m so excited to announce the release of Time Drive 0.2. This version of Time Drive is a great improvement over it’s predecessor. So much so that Time Drive 0.2 is hereby dubbed the “More S” release: more stable, more secure, more settings, and Amazon S3 storage.

In the remainder of this post, I’ll attempt to justify such a silly name by taking a look at a few of those new features.

Note (May 26, 2010): The instructions provided here for filing bugs are out of date. Since the time this article was posted, development for both Time Drive and LyX-Outline has moved to Launchpad. The bugtracker below, http://www.oak-tree.us/bugtracker, no longer exists.

Even though software developers try and create a perfect product, every program has its flaws. These can be show stopping, or merely annoying; but they are unavoidable. Even so, unavoidability doesn’t mean that they should be tolerated. Which is where users bear some responsibility. If you want a better program, you need to provide feedback to the developer.

Given how important feedback is, I find it hugely ironic that many of the software programs used to solicit such feedback (called bugtrackers), are horrifically difficult to use. Perhaps the single best (worst?) example is the system used by the Mozilla foundation (famed creators of Firefox and Thunderbird): Bugzilla. The interface is backward, it’s impossible to find anything, and getting to the same page twice requires a minor miracle.

Knowing this upfront, I tried to find a very easy to use program for collecting feedback on LyX-Outline and Time Drive. And I thought I had succeeded when I came across Flyspray, an open source bug tracking system. While still remaining powerful, Flyspray is fairly easy to use for both developers and users. Unfortunately, though, it hasn’t been quite easy enough.

You see, even though it’s pretty simple to open a new bug report, getting to the right place isn’t the most intuitive process in the world. A number of people have expressed confusion about where to go to complain about Time Drive. As a result, I thought i would write a brief guide.

Step 1: Go to the Bug Tracker Website

To get started, you need to head to the Time Drive and LyX-Outline bug tracker website. It can be found at:

Step 2: Go to the Appropriate Project Page

When you load the page for the first time, you will see a list of the currently open bugs for all active projects. Depending how busy I’ve been, this list could be quite long, or quite short. What you won’t find on this page, however, is an easy way to add new bugs. To do that, you’ll need to navigate to the project specific page.

Look at the very left hand side of the tool-bar. Underneath the username box, you will see a drop down list that says “All Projects.” This magical list is the main way that you navigate my Flyspray based bugtracking system.

For example, let’s say that Time Drive just corrupted your vacation photos and you want me to know this and how angry it made you. To communicate these thoughts (bonus points if you can do so without swearing), you need to go to the Time Drive project page. This is as easy as clicking on the down arrow and choosing “Time Drive” from the options. After you change the option, it is likely the page will refresh, and the main navigation menu will change. (Though on some browsers, most notably Opera, you will need to click the “Switch” button.)

Now, instead of the utterly boring, “Overview” and “Tasklist” options, you should be presented with a slightly more exciting menu. This includes the options to take a look at the “Roadmap” and more importantly, “Open a new Task anonymously.”

Step 3: Complain Loudly, Thoroughly and Convincingly

This is where you need to complain. Tell me exactly what you think of me, my creations, and my professionalism. Rant about how Time Drive corrupted your carefully maintained comic book collection, or how LyX-Outline destroyed your thesis. This information is important, it helps me to improve the programs. And getting it off your chest might help you too. Nothing is more soothing than a good rant. Especially when the person on the other end really wants your input.

Squeaky wheels tend to greased. Thus, consider this positing as my official permission to squeak. But to be most effective, you need to make your noise in a place where it will get noticed. For Time Drive and LyX-Outline, this happens to be at their respective project pages.

It’s been an interesting couple of days. I was rather honored to see that Lifehacker did a short highlight of Time Drive, which I thought was pretty cool. It’s always been one of my goals to have something featured in Lifehacker or Gizmodo, and now I’m going to have to scratch that off the list of goals. But that’s okay, I’ve got other things to fill the void. Like … how exactly does one get invited to present at TED?

On another note … while I knew that I would see some kind of traffic bump due to the article in Lifehacker, I wasn’t necessarily prepared for the magnitude. In mathematics, there is this thing called a step function. It’s where you move from one value to another more or less instantaneously. It looks like a step, hence the name. Sure, It may not actually exist, since even very dramatic shifts still have a non vertical slope; but even so, the change in my traffic might as well be a step-function. Between yesterday and today, I’ve had more visits to this site than I’ve had in much of the rest of the year combined. I think that’s kind of cool, though it probably won’t last.

(This might be a good time to say that I am actually rather proud of my “lackluster” web traffic. Though it might not necessarily be that impressive, it is, nevertheless, mine. I’ve worked hard for it, and I revel in the fact that some 40 to 50 people each day find the unorganized garbage of my mind intoxicating. Some of them even come back!)

But as interesting as that might be, traffic stats is probably not why you’re here. Good thing, since I’ve got announcements.

In part 1 of this article, I shared a few of the frustrations and reasons why I decided to write my own backup utility rather than submit to the tyranny of currently available solutions. While some might find those ruminations interesting, the vast majority are probably far more interested in the end result. There is a reason why “Get to the point” is one of the most important sentences in the English language.

Here’s the short version: After becoming tremendously frustrated by the state of backup on Linux, I decided to take matters into my own hands and create my own tool.

And though I only want a few things, I want that tool to do each very well. First, I’m looking for a solution that can incrementally backup over the network and let me restore a file from an existing snapshot. Second, those snapshots should be compressed, encrypted and secure. Third, it should be easy to browse old backups for existing files and restoration should be a one-click affair. Fourth, I want a backup system that can protect me from disaster, carelessness and pathological stupidity.