J. R. Mashey
Using a command language
as a high-level programming language
Pdf
(925 KB)
International Conference on Software Engineering
archive Proceedings of the 2nd international
conference on Software engineering
table of contents
San Francisco, California, United States
Pages: 169 - 176 1976

Schwartz, Jacob T., "Set Theory as a Language for Program Specification
and Programming". Courant Institute of Mathematical Sciences, New York
University, 1970. Later book
The SETL Programming
Language by Dewar and Smosna is available.

Scripting languages are higher level than system programming
languages, in the sense that a single statement does more work on average. A
typical statement in a scripting language executes hundreds or thousands of
machine instructions, whereas a typical statement in a system programming
language executes about five machine instructions. Part of this difference is
because scripting languages use interpreters, which are less efficient than
the compiled code for system programming languages. But much of the
difference is because the primitive operations in scripting languages have
greater functionality. For example, in Perl it is about as easy to invoke a
regular expression substitution as it is to invoke an integer addition. In
Tcl, a variable can have traces associated with it so that setting the
variable causes side effects; for example, a trace might be used to keep the
variable's value updated continuously on the screen.

Most people are surprised when I tell
them what language we use in our undergraduate AI programming class.
That's understandable. We use GAWK. GAWK, Gnu's version of Aho,
Weinberger, and Kernighan's old pattern scanning language isn't even viewed as a
programming language by most people. Like PERL and TCL, most prefer to
view it as a "scripting language." It has no objects; it is not
functional; it does no built-in logic programming. Their surprise turns to
puzzlement when I confide that (a) while the students are allowed to use any
language they want; (b) with a single exception, the best work consistently
results from those working in GAWK. (footnote: The exception was a
PASCAL programmer who is now an NSF graduate fellow getting a Ph.D. in
mathematics at Harvard.) Programmers in C, C++, and LISP haven't even been close
(we have not seen work in PROLOG or JAVA).

...Scripting languages such as Perl and Tcl represent
a very different style of programming than system programming languages such
as C or Java. Scripting languages are designed for "gluing" applications;
they use typeless approaches to achieve a higher level of programming and
more rapid application development than system programming languages.
Increases in computer speed and changes in the application mix are making
scripting languages more and more important for applications of the future.

...Scripting languages and system programming languages are
complementary, and most major computing platforms since the 1960's have
provided both kinds of languages. The languages are typically used together
in component frameworks, where components are created with system programming
languages and glued together with scripting languages. However, several
recent trends, such as faster machines, better scripting languages, the
increasing importance of graphical user interfaces and component
architectures, and the growth of the Internet, have greatly increased the
applicability of scripting languages. These trends will continue over
the next decade, with more and more new applications written entirely in
scripting languages and system programming languages used primarily for
creating components.

...The second difference between assembly language and system
programming languages is typing. I use the term "typing" to refer to the
degree to which the meaning of information is specified in advance of its
use. In a strongly typed language the programmer declares how each piece of
information will be used and the language prevents the information from being
used in any other way. In a weakly typed language there are no a priori
restrictions on how information can be used: the meaning of information
is determined solely by the way it is used, not by any initial promises

...Scripting languages such as Perl, Python, Rexx, Tcl, Visual
Basic, and the Unix shells represent a very different style of programming
than system programming languages. Scripting languages assume that there
already exists a collection of useful components written in other
languages. Scripting languages aren't intended for writing applications from
scratch; they are intended primarily for plugging together components. For
example, Tcl and Visual Basic can be used to arrange collections of user
interface controls on the screen, and Unix shell scripts are used to assemble
filter programs into pipelines. Scripting languages are often used to extend
the features of components but they are rarely used for complex algorithms
and data structures; features like these are usually provided by the
components. Scripting languages are sometimes referred to as glue
languages or system integration languages.

In order to simplify the task of connecting components, scripting
languages tend to be typeless: all things look and behave the same so that
they are interchangeable. For example, in Tcl or Visual Basic a variable can
hold a string one moment and an integer the next. Code and data are often
interchangeable, so that a program can write another program and then execute
it on the fly. Scripting languages are often string-oriented, since this
provides a uniform representation for many different things.

The strongly typed nature of system programming languages
discourages reuse. Typing encourages programmers to create a variety of
incompatible interfaces ("interfaces are good; more interfaces are better").
Each interface requires objects of specific types and the compiler prevents
any other types of objects from being used with the interface, even if that
would be useful. In order to use a new object with an existing interface,
conversion code must be written to translate between the type of the object
and the type expected by the interface. This in turn requires recompiling
part or all of the application, which isn't possible in the common case where
the application is distributed in binary form.

...It might seem that the typeless nature of scripting languages
could allow errors to go undetected, but in practice scripting languages are
just as safe as system programming languages. For example, an error will
occur if the font size specified for the button example above is a
non-integer string such as xyz. The difference is that scripting
languages do their error checking at the last possible moment, when a value
is used. Strong typing allows errors to be detected at compile-time, so the
cost of run-time checks is avoided. However, the price to be paid for this
efficiency is restrictions on how information can be used: this results in
more code and less flexible programs.

...Scripting languages are less efficient than system programming
languages, in part because they use interpreters instead of compilers but
also because their basic components are chosen for power and ease of use
rather than an efficient mapping onto the underlying hardware. For example,
scripting languages often use variable-length strings in situations where a
system programming language would use a binary value that fits in a single
machine word, and scripting languages often use hash tables where system
programming languages use indexed arrays.

Scripting languages are higher level than system programming
languages, in the sense that a single statement does more work on average. A
typical statement in a scripting language executes hundreds or thousands of
machine instructions, whereas a typical statement in a system programming
language executes about five machine instructions. Part of this difference is
because scripting languages use interpreters, which are less efficient than
the compiled code for system programming languages. But much of the
difference is because the primitive operations in scripting languages have
greater functionality. For example, in Perl it is about as easy to invoke a
regular expression substitution as it is to invoke an integer addition. In
Tcl, a variable can have traces associated with it so that setting the
variable causes side effects; for example, a trace might be used to keep the
variable's value updated continuously on the screen.

A scripting language is not a replacement for a system programming
language or vice versa. Each is suited to a different set of tasks. For
gluing and system integration, applications can be developed 5-10x faster
with a scripting language; system programming languages will require large
amounts of boilerplate and conversion code to connect the pieces, whereas
this can be done directly with a scripting language. For complex algorithms
and data structures, the strong typing of a system programming language makes
programs easier to manage. Where execution speed is key, a system programming
language can often run 10-20x faster than a scripting language because it
makes fewer run-time checks.

Most of the major computing platforms over the last 30 years have
provided both system programming and scripting languages. For example, one of
the first scripting languages, albeit a crude one, was JCL (Job Control
Language), which was used to sequence job steps in OS/360. The individual job
steps were written in PL/1, Fortran, or assembler language, which were the
system programming languages of the day. In the Unix machines of the 1980's,
C was used for system programming and shell programs such as sh
and csh for scripting. In the PC world of the 1990's, C and C++
are used for system programming and Visual Basic for scripting. In the
Internet world that is taking shape now, Java is used for system programming
and languages such as JavaScript, Perl, and Tcl are used for scripting.

Scripting languages have existed for a long time, but in recent
years several factors have combined to increase their importance. The most
important factor is a shift in the application mix towards gluing
applications. Three examples of this shift are graphical user interfaces, the
Internet, and component frameworks.

The third example of scripting-oriented applications is component
frameworks such as ActiveX, OpenDoc, and JavaBeans. Although system
programming languages work well for creating components, the task of
assembling components into applications is better suited to scripting.
Without a good scripting language to manipulate the components, much of the
power of a component framework is lost. This may explain in part why
component frameworks have been more successful on PCs (where Visual Basic
provides a convenient scripting tool) than on other platforms such as Unix/CORBA
where scripting is not included in the component framework.

Another reason for the increasing popularity of scripting languages
is improvements in scripting technology. Modern scripting languages such as
Tcl and Perl are a far cry from early scripting languages such as JCL. For
example, JCL didn't even provide basic iteration and early Unix shells didn't
support procedures. Scripting technology is still relatively immature even
today. For example, Visual Basic isn't really a scripting language; it was
originally implemented as a simple system programming language, then modified
to make it more suitable for scripting. Future scripting languages will be
even better than those available today.

One final reason for the increasing use of scripting languages is a
change in the programmer community. Twenty years ago most programmers were
sophisticated programmers working on large projects. Programmers of that era
expected to spend several months to master a language and its programming
environment, and system programming languages were designed for such
programmers. However, since the arrival of the personal computer, more and
more casual programmers have joined the programmer community. For these
people, programming is not their main job function; it is a tool they use
occasionally to help with their main job. Examples of casual programming are
simple database queries or macros for a spreadsheet. Casual programmers are
not willing to spend months learning a system programming language, but they
can often learn enough about a scripting language in a few hours to write
useful programs. Scripting languages are easier to learn because they have
simpler syntax than system programming languages and because they omit
complex features like objects and threads. For example, compare Visual Basic
with Visual C++; few casual programmers would attempt to use Visual C++, but
many have been able to build useful applications with Visual Basic.

Scripting languages have been mostly overlooked by experts in
programming languages and software engineering. Instead, they have focused
their attention on object-oriented system programming languages such as C++
and Java. Object-oriented programming is widely believed to represent the
next major step in the evolution of programming languages. Object-oriented
features such as strong typing and inheritance are often claimed to reduce
development time, increase software reuse, and solve many other problems
including those addressed by scripting languages.

...How much benefit has object-oriented programming actually
provided? Unfortunately I haven't seen enough quantitative data to answer
this question definitively. In my opinion objects provide only a modest
benefit: perhaps a 20-30% improvement in productivity but certainly not a
factor of two, let alone a factor of 10. C++ now seems to be reviled as much
as it is loved, and some language experts are beginning to speak out against
object-oriented programming
[2]. The rest of this section explains why objects don't improve
productivity in the dramatic way that scripting does, and it argues that the
benefits of object-oriented programming can be achieved in scripting
languages.

...The reason why object-oriented programming doesn't provide a
large improvement in productivity is that it doesn't raise the level of
programming or encourage reuse. In an object-oriented language such as C++
programmers still work with small basic units that must be described and
manipulated in great detail. In principle, powerful library packages could be
developed, and if these libraries were used extensively they could raise the
level of programming. However, not many such libraries have come into
existence. The strong typing of most object-oriented languages encourages
narrowly defined packages that are hard to reuse. Each package requires
objects of a specific type; if two packages are to work together, conversion
code must be written to translate between the types required by the packages.

Scripting languages, on the other hand, have actually generated
significant software reuse. They use a model where interesting components are
built in a system programming language and then glued together into
applications using the scripting language. This division of labor provides a
natural framework for reusability. Components are designed to be reusable,
and there are well-defined interfaces between components and scripts that
make it easy to use components. For example, in Tcl the components are custom
commands implemented in C; they look just like the builtin commands so they
are easy to invoke in Tcl scripts. In Visual Basic the components are ActiveX
extensions, which can be used by dragging them from a palette onto a form.

Scripting languages represent a different set of tradeoffs than
system programming languages. They give up execution speed and strength of
typing relative to system programming languages but provide significantly
higher programmer productivity and software reuse. This tradeoff makes more
and more sense as computers become faster and cheaper in comparison to
programmers. System programming languages are well suited to building
components where the complexity is in the data structures and algorithms,
while scripting languages are well suited for gluing applications where the
complexity is in the connections. Gluing tasks are becoming more and more
prevalent, so scripting will become an even more important programming
paradigm in the next century than it is today.

I and just about every designer of Common Lisp
and CLOS has had extreme exposure to the MIT/Stanford style of design. The
essence of this style can be captured by the phrase ``the right thing.'' To
such a designer it is important to get all of the following characteristics
right:

Simplicity -- the design must be
simple, both in implementation and interface. It is more important for the
interface to be simple than the implementation.

Correctness -- the design must be
correct in all observable aspects. Incorrectness is simply not allowed.

Consistency -- the design must not
be inconsistent. A design is allowed to be slightly less simple and less
complete to avoid inconsistency. Consistency is as important as
correctness.

Completeness -- the design must
cover as many important situations as is practical. All reasonably
expected cases must be covered. Simplicity is not allowed to overly reduce
completeness.

I believe most people would agree that these
are good characteristics. I will call the use of this philosophy of design
the ``MIT approach.'' Common Lisp (with CLOS) and Scheme represent the MIT
approach to design and implementation.

The worse-is-better philosophy is only
slightly different:

Simplicity -- the design must be
simple, both in implementation and interface. It is more important for the
implementation to be simple than the interface. Simplicity is the most
important consideration in a design.

Correctness -- the design must be
correct in all observable aspects. It is slightly better to be simple than
correct.

Consistency -- the design
must not be overly inconsistent. Consistency can be sacrificed for
simplicity in some cases, but it is better to drop those parts of the
design that deal with less common circumstances than to introduce either
implementational complexity or inconsistency.

Completeness -- the design must
cover as many important situations as is practical. All reasonably
expected cases should be covered. Completeness can be sacrificed in
favor of any other quality. In fact, completeness must be
sacrificed whenever implementation simplicity is jeopardized. Consistency
can be sacrificed to achieve completeness if simplicity is retained;
especially worthless is consistency of interface.

Early Unix and C are examples of the use of
this school of design, and I will call the use of this design strategy the
``New Jersey approach.'' I have intentionally caricatured the worse-is-better
philosophy to convince you that it is obviously a bad philosophy and that the
New Jersey approach is a bad approach.

However, I believe that worse-is-better, even
in its strawman form, has better survival characteristics than
the-right-thing, and that the New Jersey approach when used for software is a
better approach than the MIT approach.

The concept known as “worse is better”
holds that in software making (and perhaps in other arenas as well) it is better to start with a minimal creation and grow it as needed.
Christopher Alexander might call this “piecemeal growth.” This is the
story of the evolution of that concept.

From 1984 until 1994 I had a Lisp company
called “Lucid, Inc.” In 1989 it was clear that the Lisp business was not
going well, partly because the AI companies were floundering and partly
because those AI companies were starting to blame Lisp and its
implementations for the failures of AI. One day in Spring 1989, I was
sitting out on the Lucid porch with some of the hackers, and someone asked
me why I thought people believed C and Unix were better than Lisp. I
jokingly answered, “because, well, worse is better.” We laughed over it
for a while as I tried to make up an argument for why something clearly
lousy could be good.

A few months later, in Summer 1989, a small
Lisp conference called EuroPAL (European Conference on the Practical
Applications of Lisp) invited me to give a keynote, probably since Lucid
was the premier Lisp company. I agreed, and while casting about for what
to talk about, I gravitated toward a detailed explanation of the
worse-is-better ideas we joked about as applied to Lisp. At Lucid we knew
a lot about how we would do Lisp over to survive business realities as we
saw them, and so the result was called “Lisp: Good News, Bad News, How to
Win Big.” [html]
(slightly abridged version) [pdf]
(has more details about the Treeshaker and delivery of Lisp applications).

I gave the talk in March, 1990 at Cambridge
University. I had never been to Cambridge (nor to Oxford), and I was quite
nervous about speaking at Newton’s school. There were about 500-800 people
in the auditorium, and before my talk they played the Notting Hillbillies
over the sound system - I had never heard the group before, and indeed,
the album was not yet released in the US. The music seemed appropriate
because I had decided to use a very colloquial American-style of writing
in the talk, and the Notting Hillbillies played a style of music heavily
influenced by traditional American music, though they were a British band.
I gave my talk with some fear since the room was standing room only, and
at the end, there was a long silence. The first person to speak up was
Gerry Sussman, who largely ridiculed the talk, followed by Carl Hewitt who
was similarly none too kind. I spent 30 minutes trying to justify my
speech to a crowd in no way inclined to have heard such criticism -
perhaps they were hoping for a cheerleader-type speech.

I survived, of course, and made my way home
to California. Back then, the Internet was just starting up, so it was
reasonable to expect not too many people would hear about the talk and its
disastrous reception. However, the press was at the talk and wrote about
it extensively in the UK. Headlines in computer rags proclaimed “Lisp
Dead, Gabriel States.” In one, there was a picture of Bruce Springsteen
with the caption, “New Jersey Style,” referring to the humorous name I
gave to the worse-is-better approach to design. Nevertheless, I hid the
talk away and soon was convinced nothing would come of it.

About a year later we hired a young kid
from Pittsburgh named Jamie Zawinski. He was not much more than 20 years
old and came highly recommended by Scott Fahlman. We called him “The Kid.”
He was a lot of fun to have around: not a bad hacker and definitely in a
demographic we didn’t have much of at Lucid. He wanted to find out about
the people at the company, particularly me since I had been the one to
take a risk on him, including moving him to the West Coast. His way of
finding out was to look through my computer directories - none of them
were protected. He found the EuroPAL paper, and found the part about worse
is better. He connected these ideas to those of Richard Stallman, whom I
knew fairly well since I had been a spokesman for the League for
Programming Freedom for a number of years. JWZ excerpted the
worse-is-better sections and sent them to his friends at CMU, who sent
them to their friends at Bell Labs, who sent them to their friends
everywhere.

Soon I was receiving 10 or so e-mails a day
requesting the paper. Departments from several large companies requested
permission to use the piece as part of their thought processes for their
software strategies for the 1990s. The companies I remember were DEC, HP,
and IBM. In June 1991, AI Expert magazine republished the piece to gain a
larger readership in the US.

However, despite the apparent enthusiasm by
the rest of the world, I was uneasy about the concept of worse is better,
and especially with my association with it. In the early 1990s, I was
writing a lot of essays and columns for magazines and journals, so much so
that I was using a pseudonym for some of that work:
Nickieben Bourbaki.
The original idea for the name was that my staff at Lucid would help with
the writing, and the single pseudonym would represent the collective, much
as the French mathematicians in the 1930s used “Nicolas Bourbaki” as their
collective name while rewriting the foundations of mathematics in their
image. However, no one but I wrote anything under that name.

In the Winter of 1991-1992 I wrote an essay
called “Worse
Is Better Is Worse” under the name “Nickieben Bourbaki.” This piece
attacked worse is better. In it, the fiction was created that Nickieben
was a childhood friend and colleague of Richard P. Gabriel, and as a
friend and for Richard’s own good, Nickieben was correcting Richard’s
beliefs.

In the Autumn of 1992, the Journal of
Object-Oriented Programming (JOOP) published a “rebuttal” editorial I
wrote to “Worse Is Better Is Worse” called “Is
Worse Really Better?” The folks at Lucid were starting to get a little
worried because I would bring them review drafts of papers arguing (as me)
for worse is better, and later I would bring them rebuttals (as Nickieben)
against myself. One fellow was seriously nervous that I might have a
mental disease.

In the middle of the 1990s I was working as
a management consultant (more or less), and I became interested in why
worse is better really could work, so I was reading books on economics and
biology to understand how evolution happened in economic systems. Most of
what I learned was captured in a presentation I would give back then,
typically as a keynote, called “Models
of Software Acceptance: How Winners Win,” and in a chapter called “Money
Through Innovation Reconsidered,” in my book of essays, “Patterns of
Software: Tales from the Software Community.”

You might think that by the year 2000 I
would have settled what I think of worse is better - after over a decade
of thinking and speaking about it, through periods of clarity and periods
of muck, and through periods of multi-mindedness on the issues. But, at
OOPSLA 2000, I was scheduled to be on a panel entitled “Back to the
Future: Is Worse (Still) Better?” And in preparation for this panel, the
organizer, Martine Devos, asked me to write a position paper, which I did,
called “Back
to the Future: Is Worse (Still) Better?” In this short paper, I came
out against worse is better. But a month or so later, I wrote a second
one, called “Back
to the Future: Worse (Still) is Better!” which was in favor of it. I
still can’t decide. Martine combined the two papers into the single
position paper for the panel, and during the panel itself, run as a
fishbowl, participants routinely shifted from the pro-worse-is-better side
of the table to the anti-side. I sat in the audience, having lost my voice
giving my Mob Software talk that morning, during which I said,
“risk-taking and a willingness to open one’s eyes to new possibilities and
a rejection of worse-is-better make an environment where excellence is
possible. Xenia invites the duende, which is battled daily because there
is the possibility of failure in an aesthetic rather than merely a
technical sense.”

FAIR USE NOTICE This site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
in our efforts to advance understanding of environmental, political,
human rights, economic, democracy, scientific, and social justice
issues, etc. We believe this constitutes a 'fair use' of any such
copyrighted material as provided for in section 107 of the US Copyright
Law. In accordance with Title 17 U.S.C. Section 107, the material on
this site is distributed without profit exclusivly for research and educational purposes. If you wish to use
copyrighted material from this site for purposes of your own that go
beyond 'fair use', you must obtain permission from the copyright owner.

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no
less then 90 days. Multiple types of probes increase this period.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.

Original materials copyright belong
to respective owners. Quotes are made for educational purposes only
in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development
of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with.We do not warrant the correctness
of the information provided or its fitness for any purpose.