visualizing the system

Software is invisible and unvisualizable. Geometric abstractions are powerful
tools. The floor plan of a building helps both architect and client evaluate
spaces, traffic flows, and views. Contradictions become obvious, omissions can
be caught. Scale drawings of mechanical parts and stick-figure models of
molecules, although abstractions, serve the same purpose. A geometric reality is
captured in a geometric abstraction.

The reality of software is not inherently embedded in space. Hence it has no
ready geometric representation in the way that land has maps, silicon chips have
diagrams, computers have connectivity schematics. As soon as we attempt to
diagram software structure, we find it to constitute not one, but several,
general directed graphs, superimposed one upon another. The several graphs may
represent the flow of control, the flow of data, patterns of dependency, time
sequence, name-space relationships. These are usually not even planar, much less
hierarchical. Indeed, one of the ways of establishing conceptual control over
such structure is to enforce link cutting until one or more of the graphs
becomes hierarchical.

In spite of progress in restricting and simplifying the structures of software,
they remain inherently unvisualizable, thus depriving the mind of some of its
most powerful conceptual tools. This lack not only impedes the process of design
within one mind, it severely hinders communication among minds.

I’m not sure what to make of this. It sounds discouraging. It sounds like
Brooks is saying that our efforts to make the program more tractable by making
it more visible are doomed.

Well, one of those “directed graphs” will be better than nothing. How about
something like this?

Figure 1: A graph of the documents

So how do we do that? In the rest of this section, we’ll create that graph from
the program itself.

As Brooks says, a software system constitutes “not one, but several, general
directed graphs.” Willshake is no exception. Interesting graphs would include
the subsystem structure, the global build dependencies, the file structure of
the product, the DOM structure of the web site, and the logical structure of the
web site, just to name the obvious ones. Those also happen to be formal
projections of the system—ones that are expressed by the program, and can be
derived by a program, with no manual effort.

But for the primary view of the system, the map that I first show to visitors,
those formal graphs won’t do. For one thing, they tend to be too complicated,
and too focused on one aspect. Computer systems—formal systems—tend to
homogenize things, with no idea of the relative importance of the parts. In
many cases, the things of importance to people aren’t formally represented in
the system at all. I have tried to combat that by making topical documents the
primary unit of programming. But I’m talking here about the system level.

So while all of those graphs might be useful and interesting, they aren’t
necessarily conceptual maps, since they’re all ruled in some way by technical
considerations. I want at least one map of the system that is governed only by
a wish to make the system as clear as possible for both myself and readers.

What I really want is a map of the system as I see it—an aspirational view of
the system, that allows me to express how I prefer to think of the system,
whether or not that matches reality at the time. Being able to see such changes
before actually making them real is very useful for getting a reaction to them.
If it becomes part of how I think about the system, then it’s succeeded. (Of
course, something can succeed too well, and then I have to destroy it.)

Such a graph is necessarily informal, curated, hand-crafted. To that end, each
document includes a few lines indicating where and how (and whether) it fits
into this graph. This informal graph merely represents the author’s opinion,
which definitely doesn’t represent any objective property of the system at all
(since, if it did, I would use that instead). Also, of course, that metadata
has to be maintained manually.

So although I may eventually create some of those formal graphs—which can be
generated automatically, they will really be subordinate to this one.

it has to be pretty small. Not necessarily small enough to fit on one screen,
but possible to constrain in at least one dimension (width or height).

it has to be self-explanatory. It has to function as a map: an orientation
for newcomers to an unfamiliar place. It can’t be something that itself
requires explanation.

it has to be acyclic. Circular references would not only be confusing here,
they would make no sense.

it should build up cumulatively. In other words, it should be directed.

further to that point, it has to be composed of meaningful units.

Documents are the unit that we’re working with. Any smaller than that would be
too detailed for a complete overview. If this proves at any point to be
inaedequate, we should change the documents until the map is satisfactory.

Nodes are the “objects” in a graph—in this case, the documents.

Note that, although we’re talking about visualizing “software,” the documents in
question are not all part of the program per se. Some documents are merely
philosophical. We include them in this set because they all inform the
decisions made in creating the program.

Which way is up? Good question. There are advantages to either way. If you
think the edges as meaning “depends on,” then top down (like tree roots) is what
you see. If you think of “builds on,” then it’s bottom up (like tree branches).
By default, dot will give you a top-down graph. This (along with using
arrowtail instead of arrowhead above), flips the graph so that it’s top-down.

Graphviz is known for the beautiful, legible graphs that it produces. Even with
the defaults, you generally get good results. But we want the graph to feel
like part of our structure, so we take an interest in its looks.

We can control the appearance of the graph using the many attributes recognized
by Graphviz.2.

This sets the id attribute of the SVG node. This is a very generic
setting… we could almost set it in the command line that generates dot
diagrams. I’m not sure why it’s not a default in dot itself.

We use circular nodes for several reasons. For one thing, it gives them a
pleasant uniformity. As it is, Graphviz sizes the nodes based on their text
content, so they already vary in size arbitrarily based on the length of their
titles. But this way, at least they don’t also vary in aspect ratio.
Rectangular or elliptical nodes are indeed more fitted to their text, and that
efficiency makes them harder to touch—yet without making the whole graph any
smaller. These moon-like discs are also echoes of the site logo, which is
appropriate for this role of reflecting on the site itself.

We also double the default side margins, which gives a little more breathing
room between the label and the bounding shape.

Edges are the connections between graph nodes. They’re represented in the
picture by lines (or curves, when necessary), and since this is a directed
graph, they include arrowheads. Now, as we said above, this is about parts that
build on other parts. So Graphviz will draw an arrow from the new thing to the
older thing. This is a bit counterintuitive, although it is the convention for
“dependency” graphs. As long as there’s no text explaining what the lines mean,
an inverse arrow is a little less suggestive, while still indicating a direction
in the relationship. 3

We use gray edges so that they’ll be visible whether the setting is dark or
light. The default pen width is 1, meaning one pixel. When you’re drawing
angled lines or curves on a screen, a one one-pixel-wide stroke will look weak
and (on most screens) badly aliased. Anything larger helps mitigate this
effect.

Since the graph is created as an SVG graphic, and since SVG graphics recognize
CSS, we could also use stylesheets to control how it looks. But remember that
our documents are supposed to remain portable to different environments, where
SVG may not be available. So the Graphviz attributes that we have been using
are the preferred way to manipulate the output, since they will take effect
regardless of the output type.

Still, we can use a stylesheet for special features in environments where it is
supported.

You might wonder why we can’t just add the necessary rules to our “main”
stylesheets—the ones that we use for the web site itself. This is because an
SVG document is a completely new context—even when it’s “embedded” as an
object. The SVG graphic doesn’t know anything about the styles defined in the
containing document, including any web fonts that we’ve loaded. So if we want
to use those fonts in the graph (and we do), we have to load them into the SVG
document itself:

Most of the document names (such as “the collection” and “the space”) are such
that the emphasis may be placed on the last word. Meanwhile (I think because I
put line breaks between the words), Graphviz is putting each word into a
separate text element. As a result, we can use :last-child to target the last
word.

The word is resized in-place, without any adjustments to the line spacing. So
in some cases there’s a slight overlap with the previous words. This creates a
nice effect which is also reminiscent of the site logo.

.nodeellipsetransitionfill.1s// The border is good for print, but doesn't look good on screens.// // UPDATE: all stylesheets are currently targeting screen, so if you think// this one has some use for print, you may need to link it separately.@mediascreenstrokenone

The .current selector is provisional. Its intention is to mark the node when a
document is opened. At the moment, I don’t have a way to do this using getflow,
since change rules cannot target elements in an embedded document.

Also, we use user_pointing_at because it’s the preferred way to set hover rules,
but I’m pretty sure that the special javascript handling of touch events will
not include SVG elements.

The nodes are links, so they should change a little when they’re pointed at.

I wouldn’t be opposed to a bit of drop-shadow on these, but alas, it’s not as
simple as just adding a box-shadow rule, since box-shadow is not supported on
SVG elements. It can very much be done with SVG, but it would require direct
injection into the SVG exported by dot.4

#about-main-sheet <after><sectionid="more-about"class="about-sheet"><divclass="program-graph-description region-message"><p>Willshake is an experiment in <ahref="/about/literate_programming">literate programming</a>—not because it’s
about literature, but because the program is written for a human audience.</p><p>Following is a <ahref="/about/visualizing_the_system">visualization of
the system</a>. Each circle represents a document that is responsible for
some part of the system. You can open the documents by touching the
circles.</p><p>Starting with the <ahref="/about/the_project">project philosophy</a> as
a foundation, the layers are built up (or down, as it were): the programming
system, the platform, the framework, the features, and so on. Everything
that you see in the site is put there by these documents—even <ahref="/about/visualizing_the_system#sec-6">this message</a>.</p><p>Again, this is an experiment. The documents contain a lot of “thinking
out loud” and a lot of old thinking. The goal is not to make it perfect,
but to maintain a <i>reflective</i> process that supports its own
evolution.</p></div><objectid="program-graph"type="image/svg+xml"data="/static/images/program-graph.svg"><imgsrc="/static/images/program-graph.svg"alt="graph of the program"/></object></section></after>

This new content is added after the main introduction to the about region, and
that introduction takes up the the whole screen. So how would anyone know that
anything is there? There needs a signs that says, “But wait… there’s more!”

@importsigns@importarrows@importcolors@importcentering@importthumb-metrics@importpointing#about-more$width =unit($thumbRems,em)// Occupy that space//font-size 150% only for large screendisplayblockcentered-block(width:$width)text-aligncentercircle($width)sign-text-color()backgroundrgba($aboutColor,.8)+user_pointing_at()backgroundrgba($contextColor,.8)+arrow-after-pointing(down)displayblockmargin0automargin-top.5em// We use a pseudo-element for the sign text because the link only makes// sense in a styled environment.&:beforecontent'more'displayblockpadding-top1.5em

For an example, see “SVG drop shadow using css3.” The answer by
Erik Dahlström (one of the outstanding contributors on the subject of SVG) shows
how this can be done with SVG filters. http://stackoverflow.com/a/6094674

Willshake is an experiment in literate programming—not because it’s
about literature, but because the program is written for a human audience.

Following is a visualization of
the system. Each circle represents a document that is responsible for
some part of the system. You can open the documents by touching the
circles.

Starting with the project philosophy as
a foundation, the layers are built up (or down, as it were): the programming
system, the platform, the framework, the features, and so on. Everything
that you see in the site is put there by these documents—even this message.

Again, this is an experiment. The documents contain a lot of “thinking
out loud” and a lot of old thinking. The goal is not to make it perfect,
but to maintain a reflective process that supports its own
evolution.