Frameworks

Frameworks have pretty much been accepted as one of the best solutions to boilerplate by the larger portion of the programming community. At the same time, they're far less effective than domain-specific language. To be fair, they do have lots of good qualities: reducing code reuse, making code more testable, simplifying hard tasks, regulating good program structure, and so on. But is that enough to merit their use?

For things like basic CRUD apps, it's a simple process: grab your favorite web framework, write out a little bit of common code (or import some library to do it for you), and lo-and-behold: you've got yourself a 100 line app that covers your basic use-case.

At this level, a framework does some rather similar to a DSL. Both are just a small toolbox and some scaffolding that allow you to just "drop in" your solution.
So how exactly is using a DSL any different?

DSLs are all about inherently changing the way you think of a task. With a framework, it's perfectly okay to write the same style of code as before without much extra thought. Sure, you need to work it into the right shape to "fit the mold," but other than that there's not much to it.

On the other hand, a DSL is a language; just like people sometimes talk about ideas getting "lost in translation" between human languages, it seems like this can happen in programming languages too. And it's often a good thing. Language changes the way we think. Changing our ability to express one task influences how we think of other tasks. Think of Haskell: learning Haskell, which requires you to think in terms of pure functions, often influences people to solve lots of problems with the same basic tools (like functors, monads, and so on)

Do we need local state? Nah, a monad will work fine. What about null pointers? Nah, monads. Exceptions? Just monads. What about parsers? Mate, there's a monad just FOR that!

This concept is rather astonishing at times: how can a language as bizarre as APL make a seemingly difficult task (like Conway's game of life) look so trivial? The answer is that changing our language to match our goals lets us simplify our ability to create something. Imagine describing everything some program you wrote does without using any code. It's harder than it sounds, especially if you avoid overly technical language. Clearly, English (or whatever language you speak) doesn't work well for programs. You can think of this as a change of basis for vectors. Suddenly, a point which seemed lightyears away is right next to the origin.

By "changing the basis" for your program to the correct paradigm, you can often simplify it greatly. If that paradigm happens to be built specifically for your use-case (which is exactly what DSLs do for you), you're in luck. There's a reason SQL is used for databases and HTML is used for web pages (instead of rendering the page pixel-by-pixel). Likewise, there's a reason we can design languages for building web apps so that our primtives reflect what the code is trying to accomplish (instead of how it's accomplishing it).

DSLs are an open sandbox for new thinking. This is different from the approach of frameworks. While both hand you a set of tools and some basic structure to use, they treat this in different ways. In a DSL, the fundemental building tools are guaranteed to be first-class. They've also got to be powerful enough to construct an entire application with; if they only provided the tools for the 10% of the code that's usually repeated then it would be impossible to use a DSL in any real-world cases without relying on foreign code 90% of the time. This is in contrast to what frameworks do.

In any framework you get your hands on, you're already in a fully supported language for what you're making. You're not attempting to create the tools from scratch, but to simplify the task for creating the final product. Meaning, there's no reason to design ways to do the simple tasks that don't require a special library to implement succinctly or efficiently. Using such a framework necessarily means sacrificing the ability to rethink the basic tasks. More importantly, it means you've got to embed your more imperative business logic inside of the more declarative framework code. This often ends up as an ugly mess.

By just using a DSL, you've got the same language with its same style in these two places. Er, one place, now. All of the code looks the same, so there's no reason to make any distinction, is there?

DSLs don't stop you. Expanding on the previous point, this is a rather interesting conclusion. Because DSLs allow you to program the general purpose logic that you need from within the new language, you no longer have to worry about edge cases in the framework. Of course, high quality frameworks generally make extending and expanding them relatively simple, but can this really compete with not having to extend them at all? That is, is there any reason to force the user to make their own additions to the framework rather than just giving tools general enough for solving any task you can throw at it?

Obviously, this one is a little dependent on the framework and how it's being used. But oftentimes, cases come up where you want to do something special with your framework, and the library just can't handle it. Giant frameworks known for their complexity, like Rails, still require plugins to solve a lot of basic tasks. The closest you could come to this with a DSL is a library within the embedded language. But even that's quite different; an embedded library is still defined in terms of the constructs of the language (whereas the same cannot be said for a plugin that's explicitly escaping the limitations of a framework). Provided that the DSL is general enough for writing real applications, it's probably good enough to replace the need for plugins.

So what good does all of this do? How do you implement these ideas? First of all, pick a good language for defining DSLs. Whether that means good meta-programming abstractions like macros in Scheme or Lisp, or having a powerful type system and malleable syntax (like Haskell or Scala), or maybe even just being able to write and embed a small interpreter quickly (Perl 6, anyone?), what you want is (ironically) a language designed to aid in this specific kind of task. Then, it's important to put thought into the design. Think of what the lowest-level (but most general) actions/structures are for your task. For a web language, these might involve interacting with a database and being able to describe HTTP operations declaratively. In general, the more you can describe in terms of these primitives, the better suited they are for basing a language on. The final step, which, like good framework design, must come after a large amount of careful planning, is putting these ideas you've just come up with in place using the tools your base language provides. So go out and think through your problem, then when you're ready start writing those macros, or those parsers, or designing some new monad. And if your program is really cookie-cutter enough that it fits a framework but not a DSL, you'd probably be best off with a WYSIWYG editor.