It's amazing how easy it is to write simple HTML pages - and
the availability of WYSIWYG HTML editors like
NETSCAPE GOLD lulls one into a mood of "don't
worry, be happy". However, managing multiple,
interrelated pages of HTML rapidly gets very, very
difficult. I recently had a slightly complex set of pages
to put together and it started me thinking - "there has
to be an easier way".

I immediately turned to the WWW and looked up all sorts of
tools - but quite honestly I was rather disappointed. Mostly,
they were what I would call Typing Aids - instead of
having to remember arcane incantations like <a
href="link">text</a>, you are given a button or a
magic keychord like ALT-CTRL-j which remembers the syntax and
does all that nasty typing for you.

Linux to the rescue! HTML is built as ordinary text
files and therefore the normal Linux text management
tools can be used. This includes the revision control tools
such as RCS and the text manipulation tools like
awk, perl, etc. These offer significant help in
version control and managing development by multiple users
as well as in automating the process of extracting from a
database and displaying the results (the classic "grep
|sort |awk" pipeline).

The use of these tools with HTML is documented elsewhere,
e.g. see Jim Weinrich's article in Linux Journal Issue
36, April 1997, "Using Perl to Check Web Links" which I'd
highly recommend as yet another way to really flex those
Linux muscles when writing HTML.

What I will cover here is a little work I've done recently
with using m4 in maintaining HTML. The ideas can
probably be extended to the more general SGML case very
easily.

I decided to use m4 after looking at various other
pre-processors including cpp, the C
front-end. While cpp is perhaps a little too
C-specific to be very useful with HTML, m4 is a
very generic and clean macro expansion program - and it's
available under most Unices including Linux.

m4_include(stdlib.m4)
_HEADER(`This is my header')
<P>This is some plain text<P>
_HEAD1(`This is a main heading')
<P>This is some more plain text<P>
_TRAILER

The format is simple - just HTML code but you can now
include files and add macros rather like in C. I use
a convention that my new macros are in capitals and start
with "_" to make them stand out from HTML language and to
avoid name-space collisions.

The m4 file is then processed as follows to create an
.html file e.g.

m4 -P <file.m4 >file.html

This is especially easy if you create a "makefile" to
automate this in the usual way. Something like:

The most useful commands in m4 include the following
which are very similar to the cpp equivalents (shown in
brackets):

m4_include:

includes a common file into your HTML (#include)

m4_define:

defines an m4 variable (#define)

m4_ifdef:

a conditional (#ifdef)

Some other commands which are useful are:

m4_changecom:

change the m4 comment character (normally #)

m4_debugmode:

control error disgnostics

m4_traceon/off:

turn tracing on and off

m4_dnl:

comment

m4_incr, m4_decr:

simple arithmetic

m4_eval:

more general arithmetic

m4_esyscmd:

execute a Linux command and use the output

m4_divert(i):

This is a little complicated, so skip on first reading. It is a
way of storing text for output at the end of normal processing - it
will come in useful later, when we get to automatic numbering of
headings. It sends output from m4 to a temporary file number
i. At the end of processing, any text which was diverted is then
output, in the order of the file number i. File number -1 is the
bit bucket and can be used to comment out chunks of comments. File
number 0 is the normal output stream. Thus, for example, you can
`m4_divert' text to file 1 and it will only be output at the end.

This is fairly easy to create in each page - the trouble is
that if you make a change in the "standard" button-bar then
you then have the tedious job of finding each occurance of
it in every file and then manually make the changes.

With m4 we can more easily do this by putting the
shared elements into an m4_include statement, just like
C.

While I'm at it, I might as well also automate the naming of
pages, perhaps by putting the following into an include
file, say "button_bar.m4":

It is very troublesome to have items change in multiple HTML
pages. For example, if your email address changes then you
will need to change all references to the new
address. Instead, with m4 you can do something like
this in your stdlib.m4 file:

m4_define(`_EMAIL_ADDRESS', `MyName@foo.bar.com')

and then just put _EMAIL_ADDRESS in your
m4 files.

A more substantial example comes from building strings up
with multiple components, any of which may change as the
page is developed. If, like me, you develop on one machine,
test out the page and then upload to another machine with a
totally different address then you could use the
m4_ifdef command in your stdlib.m4 file (just
like the #ifdef command in cpp):

Note the careful use of quotes to prevent the variable
_LOCAL from being expanded. _HOMEPAGE
takes on different values according to whether the variable
_LOCAL is defined or not. This can then ripple
through the entire project as you make the pages.

In this example, _PLUG is a macro to advertise
Linux. When you are testing your pages, you use the
local version of _HOMEPAGE. When you are ready to
upload, you can remove or comment out the _LOCAL
definition like this:

Styles built into HTML include things like <EM> for emphasis and <CITE> for citations. With m4 you can define your own, new styles like this:

m4_define(`_MYQUOTE',
<BLOCKQUOTE><EM>$1</EM></BLOCKQUOTE>)

If, later, you decide you prefer <STRONG> instead
of <EM> it is a simple matter to change the
definition and then every _MYQUOTE paragraph falls
into line with a quick make.

The classic guides to good HTML writing say things like "It
is strongly recommended that you employ the logical styles
such as <EM>...</EM> rather than the physical
styles such as <I>...</I> in your documents."
Curiously, the WYSIWYG editors for HTML generate purely
physical styles. Using these m4 styles may be a good
way to keep on using logical styles.

I don't depend on WYSIWYG editing (having been brought
up on troff) but all the same I'm not averse to using
help where it's available. There is a choice (and maybe it's
a fine line) to be made between:

Which version you prefer is a matter of taste and convenience
although the m4 macro certainly saves some typing and
ensures that HTML codes are not interleaved. Another example
I like to use (I can never remember the syntax for links) is:

Using m4 allows you to define commonly repeated
phrases and use them consistently - I hate repeating myself
because I am lazy and because I make mistakes, so I find
this feature absolutely key.

A good example of the power of m4 is in building a
table of contents in a big page (like this one). This
involves repeating the heading title in the table of
contents and then in the text itself. This is tedious and
error-prone especially when you change the titles. There are
specialised tools for generating tables of contents from
HTML pages but the simple facility provided by m4 is
irresistable to me.

The advantages of this approach are that if you change
your headings you only need to change them in one place
and the table of contents is automatically regenerated;
also the links are guaranteed to work.

The Table of Contents generator that I normally use is a bit
more complex and will require a little more study, but is
much easier to use. It not only builds the Table, but it
also automatically numbers the headings on the fly - up to 4
levels of numbering (e.g. section 3.2.1.3 - although this
can be easily extended). It is very simple to use as
follows:

Where you want the table to appear, call Start_TOC

at every heading use _H1(`Heading for level 1') or _H2(`Heading for level 2') as appropriate.

Unfortunately, m4 is not unremitting sweetness and
light - it needs some taming and a little time spent on
familiarisation will pay dividends. Definitive documentation
is available (for example in emacs' info documentation
system) but, without being a complete tutorial, here are a
few tips based on my fiddling about with the thing.

m4's quotation characters are the
grave accent ` which starts the quote, and the
acute accent ' which ends it. It may
help to put all arguments to macros in quotes, e.g.

_HEAD1(`This is a heading')

The main reason for this is in case there are commas in an
argument to a macro - m4 uses commas to separate macro
parameters, e.g. _CODE(foo, bar) would print the
foo but not the bar. _CODE(`foo,
bar') works properly.

This becomes a little complicated when you nest macro
calls as in the m4 source code for the examples in this
paper - but that is rather an extreme case and normally you
would not have to stoop to that level.

The worst problem with m4 is that some versions of
it "swallow" key words that it recognises, such as
"include", "format", "divert", "file", "gnu", "line",
"regexp", "shift", "unix", "builtin" and "define". You
can protect these words by putting them in m4 quotes,
for example:

Smart people `include' Linux in their list
of computer essentials.

The trouble is, this is a royal pain to do - and you're
likely to forget which words need protecting.

Another, safer way to protect keywords (my preference) is to
invoke m4 with the -P or
--prefix-builtins option. Then, all builtin macro
names are modified so they all start with the prefix
m4_ and ordinary words are left alone. For example,
using this option, one should write m4_define instead
of define (as shown in the examples in this
article).

The only trouble is that not all versions of
m4 support this option - notably some PC versions under
M$-DOS. Maybe that's just another reason to steer clear of
hack code on M$-DOS and stay with Linux!

Comments in m4 are introduced with the # character -
everything from the # to the end of the line is ignored by
m4 and simply passed unchanged to the output. If you
want to use # in the HTML page then you would need to quote it
like this - `#'. Another option (my preference) is to
change the m4 comment character to something exotic
like this: m4_changecom(`[[[[') and not have to
worry about `#' symbols in your text.

If you want to use comments in the m4 file which do not
appear in the final HTML file, then the macro
m4_dnl (dnl = Delete to New Line) is for you. This suppresses everything
until the next newline.

m4_define(_NEWMACRO, `foo bar') m4_dnl This is a comment

Yet another way to have source code ignored is the
m4_divert command. The main purpose of
m4_divert is to save text in a temporary buffer for
inclusion in the file later on - for example, in building a
table of contents or index. However, if you divert to "-1"
it just goes to limbo-land. This is useful for getting rid
of the whitespace generated by the m4_define
command, e.g.:

"ah ha!", I hear you say. "HTML 3.0 already has an include
statement". Yes it has, and it looks like this:

<!--#include file="junk.html" -->

The problem is that:

The work of including and interpreting the
include is done on the server-side before downloading and
adds a big overhead as the server has to scan files for
`include' statements.

Consequently most servers (especially public ISP's) deactivate this feature.

`include' is all you get - no macro
substitution, no parameters to macros, no ifdef, etc, etc.

There are several other features of m4 that I have not
yet exploited in my HTML ramblings so far, such as regular
expressions and doubtless many others. It might be
interesting to create a "standard" stdlib.m4 for
general use with nice macros for general text processing and
HTML functions. By all means download my version of
stdlib.m4 as a base for your own hacking. I would be
interested in hearing of useful macros and if there is
enough interest, maybe a Mini-HOWTO could evolve from this
paper.

There are many additional advantages in using Linux to
develop HTML pages, far beyond the simple assistance given
by the typical Typing Aids and WYSIWYG tools.

Certainly, this little hacker will go on using m4 until
HTML catches up - I will then do my last make and drop
back to using pure HTML.

I hope you enjoy these little tricks and encourage you to
contribute your own. Happy hacking!