Saturday, January 20, 2007

An Introduction to Web Development with PLT Scheme

Introduction

From time to time people ask how to develop for the web with PLT Scheme on the PLT mailing list. The quick answer is "Just as in any other language", but that's not how to get people hooked on Scheme. To write a decent web-application require knowledge of a range of subjects such as HTML, databases, servlets, and web-servers. For some reason there is a lack of tutorials on these subjects, so I have decided to make an attempt at writing, if not a complete tutorial, then an elaborate get-started example.

The example application will be a mini version of Reddit called ListIt. The front page consists of a list of links to interesting articles, users can vote the articles up and down, and submit new articles. The hope is that the example is small enough to be easily understood, but on the other hand large enough to illustrate as many aspects as possible. Please leave comments on the blog: Did the example hit home? Is a paragraph in need of a rewrite? Did I skip something?

Model-View-controller

First things first. How should the program be organized? There is no need to reinvent the wheel, so I have chosen to use the Model-View-Controller architecture, which works just as well for web applications as it does for graphical user interfaces.

In a nutshell the Model-View-Controller architecture works like this: The model holds the data, the view displays data. User interactions goes through the controller in order to keep a separation between the model and the view. (See section 22.3 of HTDP or Wikipedia for more on MVC ).

In our case we will represent the model, the view and the controllers as three separate Scheme modules. The model will use a database to hold the links, the view will consists of functions generating HTML and the controller will the web-servlet that reacts on the user actions.

The Model

Today we will concentrate on the model. Each entry in our database consists of an entry-id , a title to display, an url to the article and a score representing the votes. Since we expect many entries in our database, we will think of them as divided into page. The number of entries in each page is given by the parameter PAGE-LIMIT.

The interface to our model consists of the following functions:

insert-entry: title url score -> entry-id Insert a new entry into the database.

increase-score : entry-id -> Increase the score of an existing entry

decrease-score : entry-id -> Decrease the score of an existing entry

top : natural -> (list (list entry-id title url score)) Return the given number of entries with the highest scores

page : natural -> (list (list entry-id title url score)) Return the list of entries in the given page.

url-in-db? : url -> boolean Is the url already listed?

These functions are the only ones to be exposed to the controller.

Implementation of the model

To implement these functions we will use an SQLite database. It wouldn't be unreasonable to argue that it would be easier to use a hash-table, but I want to illustrate how to use SQLite.

SQLite is small database engine, which comes in the form of a single self-contained, zero-configuration DLL-file on Windows or a a so-file on other platforms. We will use Jay McCarthy and Noel Welsh's PLT Scheme bindings sqlite.plt . On top of these binding we'll use a S-expression to SQL-string library written by me (it will appear on PLaneT soon - it has been submitted). On Windows you download SQLite by pasting the following into the DrScheme interaction window (the REPL):

The macro sql converts an S-expression representation of an SQL-statement into a string, which is then handed to SQLite by insert. The string produced by the sql macro from(insert-entry "Everything Scheme" "http://www.scheme.dk/blog/" 42)becomes"INSERT INTO entries (title, url, score) VALUES ('Everything Scheme', 'http://www.scheme.dk/blog/', '42')".

The remaining functions from the interface are all simple SQL-statements, which can be studied in the full program below.

A loose end: In the source below the parameter current-database is used to hold the database. As a convenience I have with the help of syntax-id-rules defined the identifier db to expand to (current-database). But in order make everything work also as when the database isn't created yet, the actual definition below is a little more involved.

Testing

To test the model, open the "model.scm" in DrScheme. In the "Language" menu use "Choose Language" to choose the "Module" language. Click and "Run" and you are ready to test it:

(definePAGE-LIMIT (make-parameter50)) ; number of entries on each page (defineDATABASE-PATH (string->path"listit.db")); initialization of the db happens at the first run, see bottom of this file (definecurrent-database (make-parameter#f))

Oh so readable! You make it look easy. If I could use your code for my PostgreSQL and MySQL databases, I think I could switch my personal DB webapps from Java/Python to Scheme in no time. I especially appreciate your sql macro for constructing SQL strings. Does your macro handle embedded single quotes and double quotes? What if I have a string that looks like an sexp (is surrounded by parens)? Are there problems in the encoding or decoding of those?

Thanks. The intention is that the sql macro handles converting strings automatically. However you can use prepare to "compile" your SQL-statement with parameters marked with "?" and then later apply the statement to actual parameters. In this way, there is no risk of SQL injections.

Thanks for the suggestion. I might write about file upload and sessions. Since I'm using the PLT web-server, I have no need for FastCGI. Do I guess correctly that you want to use Apace? If so note, that you can use Apache and the PLT Scheme web-server at the same time. See for example this discussion on using the PLT Web Server and Apache at the same time in the mailing list archive.

I have just stumbled on this while looking for info on web dev withplt scheme.

Some things I noticed while overcoming bumps in getting it to run:1. here-strings: It is great to see these in the plt language as they are so useful. But as with here strings in other languages making sure that the terminator is not indented, nor has trailing spaces is crucial. I was bitten by this when getting the example up and running. Emacs indenting is a blessing most of the time. ;-)

2. I am guessing, but it would seem that you must use absolute paths for the path to the sqlite file. I tried something like this: (string->path "~/devel/scheme/database.sb") and it failed, but hardcoding an absolute path seemed to do the trick.

But, now it is running fabulously, and I love the schemey sqlnotation; very nice.

I have been really enjoying your blog, so many useful bits and pieces,and thanks for all the work you put into it.

Looking forward to reading the next installment of the plt webdev. tutorial.

Rohan

P.S. I am very glad you included using a relational database backend,makes this example much more relevant for all those people who need touse such things for getting work done, or who are just very used tousing them. It also nicely demonstrated how easily you can have plt up and runningwith a SQL system. :-)

I love your programming style, I intend to try to learn from it. I've been coding scheme for a couple of years, but I still haven't quite got the hang of picking exactly the right mix of macros and runtime. I think emulating this style of yours will help me.