Design

Endless Projects: Where's the Fun Anymore?

How can you create a non-trivial project that can be completed in a reasonable amount of time?

I got into programming for the same reasons almost all passionate programmers do: the joy of solving problems combined with the deep satisfaction of creating new things. When I started on this journey, programming was undeniably fun. My ambitions ran in all directions at once. And so I pumped out lots of code on my own time: an arbitrary-precision calculator, a B+-tree library, a programmable code generator that could test the limits of compilers. These projects were hugely satisfying, made good articles, and were blessedly bounded. Each one of them could be undertaken and finished in a few weeks to a few months of after-hours work — despite being written in C.

As the GUI interface became the de facto means of interacting with software, programs became significantly bigger. Moreover, the profusion of tools, especially from the open source bazaar, created new demands. There was little point in writing a utility that already had a fully formed version widely available at no cost. Consequently, to write something useful, I was confronted with the need to develop programs with significantly more features (or scalability) than whatever else was out there. Over the period of a few years, even small projects came to require six months to accomplish, frequently more. I had to be much more circumspect about what projects I decided to work on as I increasingly reached the point where the love of the goal tapered off long before the labor. To my distress, I began abandoning projects, even after dedicating significant time to them.

My initial response was to move up from C. The language's low level qualities were fast losing their appeal. The nimbleness, elegance, and performance no longer compensated for the time burden they imposed. I switched to Java, where I thoroughly enjoyed the vast libraries and far better tooling. The OO paradigm, of course, also helped greatly.

I was finally able to undertake and complete exploratory projects again. But, after a couple of years, the old problem of complexity began to return. Projects now needed persistence layers and Web tiers. One programming language was not sufficient. HTML, JavaScript, XML, SQL, and other secondary technologies had to be mastered to do anything interesting. Projects were getting both big and difficult.

Today, I find myself about 30K lines of Java code into a project (counting program and tests), which represents about three years of intermittent consignments of my personal time. While the project has advanced steadily; each year on the anniversary of the founding date, I have come to the same conclusion about my progress: I'm about 50% of the way through.

The love endures (it's typesetting software, after all!), but the desire to explore other projects is getting the better part of me. I note that I am not the only one with these sentiments. The overwhelming majority of open-source projects are abandoned incomplete. In part, this results from the youthful inexperience of project founders who, fresh out of school, want to conquer the world — only to discover the task to be far greater than a one-semester project. And unlike school, no one is paid to assess the results or even care about them.

Anyway, for exploratory programming to be fun once again, it has to avoid seeping into the domain of multi-year projects. So the question is how to create a non-trivial project that can be completed in a reasonable amount of time? For small projects, it's OK to cut back on requirements and design — they can be done in a few hours with a whiteboard and a notebook, rather than with elaborate diagrams and carefully written requirement docs. Other areas, such as tests, however, I won't cut back on. The real bugaboo is not design or testing, it's the (endless) coding. Reducing coding is my mantra going forward. And to fulfill it, I've concluded, I must move up the food chain from Java.

Java's speed, portability, scalability, and extraordinarily good tooling are all pluses, but they don't solve the one problem that bars my way: lack of productivity. To clear this hurdle, I'm willing to sacrifice everything but the good tooling. And so I am considering two principal alternatives: Groovy and Ruby. Groovy has the benefits that come from the JVM (except for speed) and of some very unusual features that make certain problems painless (such as testing, XML processing, and others). Ruby's appeal is that it provides, in some senses, a ready-made universe. It comes with its own Web server and Web framework, hosting platforms, and SCM tools. In addition, most users really like the language. I'll probably do very small projects with both languages to examine them fully and I might even end up choosing JRuby to get the best of all worlds. Hard to know.

However, what I do know is to get out from under the endless projects, I have to change my language and my tools.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!