KnowledgeBase 00009: Developing a Web Site Using OpenQM

Introduction

This article describes how an interactive web site can be constructed using
QM. The technique described here is how the openqm.com web site operates
though some of the site specific detail has been omitted.

Although we strongly recommend several third party web development packages,
the openqm.com web site was developed with a totally "home grown" approach,
mainly because the combination of features that we wanted was not available in
any one toolset at that time.

CGI

This technique is based on the CGI (Common Gateway Interface) and should
work with all operating systems. A simple C program that handles incoming web
transactions on a Windows system is shown below. It routes the transaction to
a QM process via the QMClient API.
The SERVER_xxx tokens should be replaced by the
appropriate values for the system on which it will run.

The QMCall() function returns the text of the web page to be displayed via
its Response argument. If the page data is more than 32kb, the QMBasic
part of the application writes the data to a temporary file and returns the
pathname of this file prefixed by an exclamation mark. The C program detects
this as a special case, emits the data to the web client and deletes the
temporary file.

The QMBasic component of this web interface is a catalogued subroutine
declared as

$qmcall
subroutine cgi(input.data, params, reply)

The $qmcall compiler directive permits this subroutine to be called when
QMClient security is at its highest level. This is the only subroutine in the
entire application compiled with this option and is therefore the only
available entry point.

The input.data argument receives the incoming web request,
params is a multivalued list of environment variable names and
values as collected by the C program (only REMOTE_ADDR and HTTP_HOST in this
version) and the reply argument is used to pass the constructed web
page back to the CGI program.

Handling Web Transactions

This application has a mix of traditional "text with hyperlink" pages and
form filling. The elements of a form and other incoming data are passed to the
server as an extension to the URL, for example,

www.mysite.com/cgi/cgi.exe?t0=m&t1=links&x=jlfo9d9pqn

Each ampersand separated element of the text following the question mark
represents a web transaction parameter. The naming of these is entirely
controlled by the application and our usage is such that text fields are named
as the case insensitive letter T followed by a number. Because of the way in
which this application evolved, T0 is used to identify the program that
processes the screen or a special code such as M in this example to indicate
that it is a menu action. For other T items, the number is the field position
in a dynamic array that will receive the associated text value. In the same
way, we use B for buttons, C for checkboxes and R for radio buttons. The X
item in this example is the session id which we will discuss later.

The process of parsing the parameters into the relevant dynamic arrays is
simple except that it is necessary to decode some special HTML data constructs.
Imaginatively, the T, B, C and R parameter items are parsed into common
variable dynamic arrays named T, B, C and R. Perhaps such terse names are not
a good idea but their use is well understood by the developers involved with
this application.

To make things a little more complex, the parser used by the openqm.com
web site also allows multi-valued
tokens (for example, T5.2) but we will not need to look further at those here.

Page Layout

The openqm.com web site page as a banner heading, a menu bar with
expanding sub-menus and a page body that might be simple static HTML text,
dynamically generated text, or a form with input boxes, etc.

Whatever style of page is in use, every element of it is constructed
by QM. There are no static web pages stored by the web server.

Session Ids

Web transactions are inherently totally separate. There is no automatic
persistence of data from one transaction to the next for the same user. There
are various ways in which an application can provide its own persistence and
the approach that we take is to assign a unique id to the user's session. This
is used as the record id to a SESSIONS file in which we can record whatever
session related persistent data we need. Examples of such data for this
application include the user's access level and details of the menus that are
expanded on the menu bar.

When a user first connects, he will not have a session id and there will be
no X parameter in the incoming data. In this case, or if the session id has
expired, the application creates a new session id from a random sequence of
ten characters and writes a new session record for an unauthenticated user.

If a session id is provided in the incoming data, the application
checks whether it is still valid. We timeout a session after a given period
of inactivity, each new transaction for that session resetting the timer.
There is also a mechanism to flush timed out sessions from the file.

Handling the Incoming Request

Once we have parsed the incoming data and, perhaps, linked it to an existing
session, we are ready to process the action. The T0 parameter is used to
identify what we are doing. For the purposes of this discussion we will look
only at three codes.

A T0 value of H requests display of a template HTML page where T1 identifies
the actual page to be displayed. These pages are stored in a file as simple
HTML data but they can contain special substitution tokens that will get
replaced before the data is emitted. The openqm.com home page works in this
way, with the current QM release number automatically substituted into the
text.

A T0 value of M indicates a menu action where T1 identifies the menu name.
If the session record shows that the named menu is not currently displayed,
it is added to the list of menus to be expanded. Conversely, if it is
currently displayed, it is removed from the list. Actual display of the menu
comes later.

Other T0 values identify subroutines to be called to process the
request. In each case, the subroutine name is formed by adding an internally
defined prefix to the name in T0 so that it is only possible to call specific
subroutines. The subroutine builds an HTML data for the body of the page image
in a common variable that will be merged with the rest of the page structure
later.

Emitting the Basic Page Structure

Now that we have determined the menu bar content and built the page body, the
time has come to emit the page structure. We do this by building up a variable
that will contain the HTML headers, style definitions, the page banner, the
menu bar and the page body. Only the menu bar element is evaluated at this
stage; everything else has already been built.

The final page text is then returned to the CGI program via the reply argument
to the parser subroutine. If this text is over 32kb, it is written to a
temporary file and the reply argument is set to the name of this file prefixed
by an exclamation mark.

HTML Generation

So far, we have not really looked at how the HTML for form filling operations
is constructed or how multi-screen sequences are handled.

All the form filling operations within this web site are built using HTML
tables. We have written a set of simple functions that will return the
relevant HTML constructs to display table elements. For example, a slightly
simplified version of the function that emits a table element containing an
input text box is shown below.

This function shows that highlighting of errors is controlled by a dynamic
array named T.ERR with a field by field correspondence to the T dynamic array
that holds the input data. There are similar error arrays for the other input
elements.

The program to generate an HTML page becomes nothing more that a series of
function calls to each of the data elements to appear on the screen.

Multi-screen sequences are handled by adding a numeric suffix to the screen
program name in the T0 variable. This is stripped out by the input data parser
and stored in a variable that identifies the screen number in the sequence.
Thus, for example, generation of an evaluation licence starts with T0 set to
"generate" and then goes on to "generate2", "generate3" and so on. Because it
would be possible for a user to construct a URL that dived in part way through
a screen sequence, any data validation has to be repeated for each phase of
the screen.

Security

This application has several distinct levels of access corresponding to
different classes of user. The session record holds the access level
associated with the session. The menu building elements of the application
attached an access level to each menu item and only display it where it is
valid. Again, because a user could construct a URL of his own in an attempt
to bypass this mechanism, every program begins with a check that the user
really should be allowed in, displaying an error at a security violation.