Programming methodology deals with the analysis, design and implementation of
programs. One important methodology involves what is referred to as a
"top-down" approach to analysis, design and implementation.

Top-down-design starts with an description of the overall system
and usually consists of a hierarchical structure which contains
more detailed descriptions of the system at each lower level. The lower
level design details continue until further subdivision is no longer possible, i.e.,
until the system is described in terms of its "atomic" parts.

This method involves a hierarchical or tree-like structure for a system as
illustrated by the following diagram:

At the top level, we have that part of the system which deals with the overall
system; a kind of system overview or main top-level module.

An alternate design metholodogy, Bottom-up design, starts with description of the lowest level system
components together with description of how such components are assembled
to form higher level system components. This design methodology continues
until all levels of the system are connected in a hierarchical structure
to form the entire system.

Top-down design is preferred because, at the beginning of an analysis and
design cycle, it is usually not possible to know the lower level design
details which are determined by a process of successive refinement.

As an example of top-down design, consider the design of an interactive system. The top level program
will be the part of the system which ties together the key system components.
One of these components might be the part of the system which reads a command;
another component might evaluate the command just entered. Still another
component might be the part of the system which displays the results of
executing the command just entered. The overall structure of such a system is
shown from the top down in the following diagram:

We wish to design and implement a simple data base system which allows commands
to be entered interactively and results to be displayed.

We will attempt to illustrate this top-down methodology with an example. As
with all of the topics covered so far in this course, we will model this system
using the J programming notation. One might actually use some language other
than J (such as Scheme or C) to implement a database system. However, the
approach we use is valid since one might use J to implement a prototype system
and then use the C programming language for the production system. This is an
example of an important technique known as software prototyping. Just
as an engineer always builds at least one prototype machine before giving the
go-ahead to begin production of the machine, many software engineers also use
this same technique. J is a particularly good language for prototyping as we
shall soon see.

Since most J systems are already interactive systems, we can simplify our
prototype design by using the J top-level as our prototype's top-level.

Similarly, we can use the J syntax for our data base command language. This
means that we can use the J system's read-eval-print loop (the second level in
our system diagram) as our prototype's second level and third level. It should
now begin to be obvious why J is a good prototyping language for interactive
systems. We are already down to the fourth level of our design and we have yet
to begin to design or implement any programs!

Another aspect of this design method of building a system from the top down in
distinct layers is that it is possible (and desirable) to attempt to make each
layer of software independent from the layers above and below it. This
involves design of some sort of abstract interface between the layers.

The most important benefit of design layers is that it should then be possible
to change the software in one layer (provided the new version of that layer
adhears to the same interface above and below) without having to change any
other part of the software design.

This is a lofty design goal. In practice, it is often very difficult to
achieve.

We define a database to be of a collection of records. Each record is a
collection of related items. Each record will have a key field which we
will use as an identifier of that record. Since we are implementing our
prototype in J we will use a list as our data structure for the database. Each
record, itself, will be a list. The key field will be the first item of each
record and will often (but not necessarily always) be a character list. The
remaining items in each record may be any valid J object. Following is an
example database (admittedly small) containing some information on people.

Notice that by the way we have constructed the data associated with each key in
the people database, that the result of using lookup-all is to produce a new
database which might be used as the database for a lookup operation.

Next
we consider the problem of adding items to an existing database. The result of
this operation is a new database. Therefore to actually add an item to an
existing database we need to re-bind the name of the existing database to be
the result of add-data.

You have probably noticed that so far, each database operation requires two
arguments. The first is a key and the second is the database to which the
operation is applied. This was a concious design choice which gives additional
flexibility to the database operations. These operations are more general
because they are not tied to a specific database; they may be used with any
appropriately structured database.

We now consider the problem of more permenant storage of our database in a
filesystem. As it stands now, our database is stored as a J list in the main
memory of our computer. It is vulnerable to loss if the power to the computer
goes down for example or if we exit from the J system.

There are four utility verbs which are needed to construct two verbs,
write_list and read_list which will allow J lists to be
permenantly saved as files. Two of the four utility verbs (read and
write) will not be fully explained at this time. We will still be
able to make effective use of read and write even though we may not fully
understand their J definitions. Their definitions are:

read =: monad def '1 !: 1 < y.'
write =: dyad def 'x. 1 !: 2 < y.'

J
objects are stored in files as strings of characters. The dyad write
requires a character list for its left input and a quoted filename for its
right input; the character list is written to the indicated file. For example,

'Hi there' write 'junk'

creates
a file named junk containing the characters 'Hi there'. A
file may only contain characters as data. The monad read requires a quoted
filename as its right input; the indicated file is read into the J environment.
For example:

read 'junk' ==>
Hi there

J
lists may consist of arbitrary J items (numbers of various types, characters,
functions, etc.). To write and read J lists to and from files it is necessary
to convert these lists to and from character strings. Two utilities are
available to do this, however, it should be noted that files created this way
on one type of computer system (Unix) may not be readable on another type of
computer system (Intel architecture). The utility functions are:

to_char =: 3 !: 1
to_internal =: 3 !: 2

These
verbs are used to define the following verbs which may be used to store and
retrieve J objects (including lists) as files.