Against Monolithic Frameworks

In search of the basic building blocks S V Ramu (2002-09-08)

Prelude

When I saw Apache Cocoon, I loved it. Its view of content generation as a series
of XML-XSL transformation is radical. Seeing Apache Struts, made it clear that
there are areas in XSLT which cannot match the power of a programming language,
and Java could be that language. Who will not enjoy the extreme plug-in
architecture of JEdit! And that Netbeans people's IDE. And now this eclipse,
with its yet another deceptively lean plug-in framework, and the focus on being a
complete IDE, instead of just an editor. How can we miss JBoss from this list,
with its revolutionary vision to make its App Server(!) in a plug-in model.

These and other such
OpenSource
projects are exposing what we missed in the hitherto proprietary model of
software making. When a company, with its rightful motive of making money,
creates a software, it will keep, and has kept its activities closed to outsiders.
Not only the source, but also extensibility of the software is kept closed. This,
they thought, has to be done in order to keep the revenues flowing in. After all,
if they allow users to create their own useful extensions, then how will the
original company sell its next version? But this all changed with the coming of
OpenSource software, which started just as a philanthropic rebellion to the
shooting software prices. When people make software for the sheer fun and beauty
of just making it, they don't worry about the next version, but only its
usefulness. It is definitely not useful for the users of a software, to wait
for the creator to serve all their requests promptly. Moreover why wait for a
small tweak, instead of doing it ourselves. This extensibility, and the
consequent modularity is the hallmark of the OpenSource software development.

Plugins Everywhere

In C (the language) we modeled a software as a collection of functions.
In Java we imagine the same software as a collection of classes, that is, as a
bigger grouping of functionality. In both, the idea is the same: to conquer
complexity by divisioning. The debate is only on the right form of modularity,
and not the usefulness of the modularity itself. Components and
Plugins are just terms denoting the extensions of this business of
breaking a software into manageable and yet meaningful chunks. And only with
Plugins we are squarely dealing with the user based divisioning rather than the
usual programmer based functions and classes (though the OO gurus have been
pleading for a long time now, to make reality based classes).

Plugins epitomizes the axiomatic approach to software engineering. Plugins are
manageable, and simple-to-understand pieces of code, that when arranged suitably
in a framework, can provide a complete solution to the task at hand. Plugins
depend on contracts. The contract stipulates, how a given plug-in will
use the given data, or its output format will be, or else how it can cooperate
with other plug-ins. Though many plug-ins are useless when alone (we'll revisit
this point soon), their sum can be greater than the parts.

More the rules a plug-in requires, less useful it is. And more 'complete' it is,
the less extensible it is. Ideally a plug-in does one thing, and does it well,
following the age old golden rule of functional software. The upcoming
webservices, and the
service based modeling of software as such, are the outcome of this plug-in like
modular visions. It is becoming very clear that pouring millions into the
maintenance of software is plainly unwarranted. Yes, Microsoft does manage its
Windows this way, successfully. But you must be as rich as they, to follow suite.
Not many can keep their product inextensible, and still have the wherewithal to
maintain it. Nowadays OpenSource is like a waiting hound. Either you open
your code and have the initial bite in the market, or leave it to charity.
Also remember, that leaving code source open, makes you vulnerable to peer
reviews, with whom only extensibility and simplicity wins.

The Role of Frameworks

For plug-ins to be useful, and be well orchestrated, we need a framework of
cooperation. It seems, Sun is the first and the best company to use the magnetism
of framework to garner more market share. Of course IBM did this long back with
its open and free PC model, but soon somehow managed to loose its creativity and
dare. Sun took over the baton of being creative by being the author of Java and
the now growing
Java Community Process (JCP). The idea of keeping the framework
as standard and vying in implementation, is daring for sure. This involves the
courage of letting go. Microsoft need not do this, as they already have
their market, and only have to hold on to it. But for how long?

OpenSource is here to stay, and hence the extensibility of Plugins, and hence the
importance of Frameworks. The early bird gets the worm. If
Eclipse is any indicator, only the
strongest early bird gets it. Strongest can mean anything: Maybe the
advertising power, or a complete suite of services, or simplicity, or a mix of
these and more. If Microsoft could be envying Sun (look for Palladium) for its
Liberty Alliance; If IBM et al's Eclipse
can compete with Sun's Netbeans aka Forte; And if every non-Mictosoft company is
flocking to build a competing App server adhering to the J2EE standard (and if
JBoss can win the race with its plug-in model): This only shows the power of
frameworks in both its Openness and the Profitability.

Against Monolithic Frameworks

There is no denying fact that frameworks are central to the present distributed
and open model of development. But what sort of frameworks can win? We must
realize that our understanding of automation is far from complete. Only few,
that too rarely, can claim that they understand a requirement decently. Yes, our
challenge of making a good software is not just the question of technology
alone, but the far subtler question of comprehension itself. Extensibility is
the first causality of poor comprehension of a product. More all-encompassing a
framework is, and more it claims tight integrations with its parts, less chance
does it have of surviving our varying needs.

Why at all we need a framework? To allow the plug-ins to cooperate smoothly,
in spite of each being from different stables. When does a framework cross its
limits of usefulness? When it does anything more than orchestrating the plug-ins.
By keeping the framework loose, we only put the onus on the plug-ins themselves.
The mistake a framework can make, can be made by a plug-in too. But,
plug-ins being designed as replaceable ones from the start, are expected to cause
less problems. But the danger is still there. The question of, what minimum
features a plug-in should have to cooperate, can itself develop into a rigid
framework. It seems the maximum constraints of a framework could not be larger
than the usual Sockets models over TCP/IP. Anything less could be frivolous.
And anything more could be dangerous. Though wire protocol should be the norm,
there should be a way to separate even this dependency through more layers of
indirection. Yes, performance comes into the picture here, and that's where the
compromises start, and that is where we must be careful.

In this light, inverting the starting point might be useful. Where till now we
tried to build the most flexible framework, we can now try to build a set of
most homogenous plug-ins, and then work with them to arrive upon a suitable thin
wrapper framework around. By homogenous, we mean that the number of
concerns addressed by a plug-in (or a component, or a simple block of code), must
be minimal, and ideally just one. Now, 'after separating what is impossible,
whatever remains is however improbable, is the solution'. By no stretch of
imagination can we claim that we know what we have to do, we only know what is
dangerous. Monolithic frameworks are dangerous. Rigid standards are dangerous.
And autonomous and homogenous plug-ins are promising.

How does this translate into few of our day to day uses? If we can see that
transaction are totally separate from pooling - J2EE, EJB are unwanted rigidity
then (don't even talk about .NET here). If XML generators are seen as
sufficiently independent of the framework to generate web pages, then Cocoon is
an unwanted rigidity. If XML and XSL can create 90% of the web pages
generically and openly, then Struts like OO deliberations are superfluous. Think
how independedt a spell checker is, and see
how nicely it fits as a plug-in into jEdit.
Note that Jython or Beanshell are independent scripting engines, and yet they fit
snugly as a plug-in into an IDE. See how Apache Ant, Jalopy like code formatters
are useful and homogenous tools in their own right, and hence make a good
extensions to anything, as a plug-in.

Plugins can themselves expose a thin extensibility hooks, and act as a mini
framework. If a plug-in itself can be made out of smaller plug-ins, or can be
extended or configured by a plug-in, we now have a open network of plug-ins.
A good ideal to keep as our bearing.