Revision as of 12:44, 19 August 2007

HAppS is a framework for developing Internet services quickly, deploying them easily, scaling them massively, and managing them effortlessly. Web, persistence, mail, DNS and database servers are all built-in so you can focus on app development rather than integrating and babysitting lots of different servers/services (the Haskell type system keeps everything consistent).

The quick way to see what's missing is to get the darcs repository, change into that directory, and run runghc Setup.hs configure. If you don't get an error, try runghc Setup.hs build and then as root runghc Setup.hs install.

2 Overview

The application model in HAppS is to help separate state, application logic, wire formats, protocols, and presentation layer:

2.1 State

State is just a haskell data type you define (deriving (Read, Show, Typeable)). If you have several pieces of state, you'll probably want to define the state as a Haskell record of these pieces.

2.2 ACID

Atomicity: Guarantees that every single one of the operations is successfully performed or none of them are. This prevents unfinished operations in the system.

Consistency: Guarantees that the system is in a valid state before and after every operation. This is enforced by Haskell's type system.

Isolation: Guarantees that all operation happen isolated from the other operations in the system. This means that outsider operations will never see operational values while they're still being processed.

Durability: Guarantees that after an operation has succesfully exited, it's value will remain in the system as long as needed (it will not dissappear in the case of system failure). This is handled by MACID write-ahead logging and check-pointing.

2.3 Application

Incoming events are gathered in individual haskell threads and then pushed onto a single application queue for processing. The queue model gives you ACID Atomicity and Isolation and lets your app be simply a set of functions with types like:

SomeInputType -> MACID SomeOutputType

The MACID monad lets you update your state and *schedule* side-effects. To be clear, MACID is not in the IO monad so you cannot execute side effects, you can only schedule them. The framework takes care of making sure they are executed at-least-once (if they can be completed by a deadline you specify).

2.4 Wire formats

Since your app consists of a set of functions with various haskell input and output types, somewhere you need a place to convert between those internal haskell types and external protocol event types; e.g. from URL Encoded HTTP requests to SomeInputType and from SomeOutputType to XML encoded HTTP responses.

2.5 Protocols

HAppS currently provides support for HTTP Requests/Responses and SMTP Envelopes. To be clear HAppS provides ACID Atomicity at the protocol event level. So if you write a protocol with SMTP envelopes being the arriving event type then your app will have atomicity in processing incoming SMTP envelopes. If you write a protocol with SMTP commands being the arriving event type, then your app will have atomicity at the level of individual smtp commands.

2.6 Presentation

If your application outputs XML as its wire format, HAppS provides a lot of support for using XSLT to transform it for presentation purposes. For example, you can send XML mail and HAppS will take care of applying the relevant XSLT stylesheet before it is delivered. If you output XML HTTP responses, HAppS takes care of applying the XSLT stylesheet server side for user-agents that don't support doing so on the client. The value here is that you can have designer types who know XSLT modify presentation stuff without touching your application code.

3 First-step examples

This chapter will run you through some first simple programs written in HAppS. For other programs have a look at the directory named 'examples'.

First of all, default HAppS applications run their own webserver on port 8000, so you probably want to try out these examples at http://localhost:8000/

If you'd rather access these applications on some other port, use ./myapp --default-port=8001 obviously substituting the name of your binary for myapp.

3.1 How to build these examples

Cut'n'paste this into a file named Hello.hs and run ghc --make Hello.hs -o hello to compile and then ./hello to execute the resulting binary.

Handlers are functions that produce either a request or a response. stdHTTP runs forward through the list
of handlers transforming requests into requests until it hits a handler that produces a response.
It then runs backward up the list transforming responses into responses.

debugFilter actually consists
of two handlers, one that prints the request to console and then returns it and another that prints the
response to console and then returns it. It is defined in HAppS.Protocols.SimpleHTTP2 as

Note: The darcs version of 2007-07 uses hslogger, and is no longer able to log the incoming request.

h is a wrapper around Handle that simplifies matching on uris and methods and structuring responses.
It only executes the handler if the URI matches the regex in its first argument and the method specification in its second.
A "^" is automatically added to the URI because that is the 99% case.

noState is just there to properly establish the state type for the
MACID monad, since nothing else is doing so.

Notice in this example that any request other than GET / will produce an error!

3.2.2 Add "val" for simplicity

The concept of just returning a value is so common that we defined a function "val" so you don't have
to define a function just to return a simple value.

3.2.3 Methods and paths

The first argument to h must be a suitable type to be used by the class FromReqURI that is in charge of parsing the URI. Whatever a match
returns is then passed on as the first argument of the method, so the
type of this argument also controls what happens. Note the use of the Prefix constructor below, whose corresponding class instance dumps the rest of the URI into the lst argument.

Method arguments can be individual methods, lists of methods or () to mean all methods.

import HAppS
main = stdHTTP $ debugFilter : -- we want to see debug messages in the console
noState : -- our application has no state[
h [""] GET $ ok $ val "Hello World",h ["getPost"][GET,POST]$ ok $ val "either GET or POST will result in this response",h (Prefix ["dir"])()$ ok $ \lst ()-> respond (unwords lst)-- any method to /dir/sub/dir will return "sub dir",h ["methods"]()$ ok $ val "Hello"-- any method to /methods will return "Hello",h ()()$ ok $ val "default"-- any method and any reqURI not matched above gets this--these two are automatically added by stdHTTP so you don't have to unless you want to override--notice that the responses are not "ok" they are notFound and notImplemented!,h ()[GET,POST]$ notFound $ val "not found",h ()()$ notImplemented $ val "not implemented"]

Note that to try this out with some static files you should create a directory named "static" in the directory where you are running the tutorial code, and put any files you wish to serve in there.

3.2.5 Block dot files

But observe that we don't want to serve all paths in the filesystem. So we want to preempt certain
requests that reach the fileServe line:

Now we observe that we actually want to block dot files as well so we do. (There's probably a nicer way to do this
using regex). Notice that the fileServe code actually does IO. So you can write responses that do IO. Conceptually
you can serve content out of an external database or a proxy server.

3.3 Saved state examples

when monkeying with happs code involving state. This seems to happen when you add state, remove state, or change the way state is being use. At startup, happs attempts to read state information from the state cache (a subdirectory of your working directory) and if this disagrees with what it is is expecting you get that error. I clear my state cache (and logs) with the following command.

rm -rf '<interactive>_error.log''<interactive>_state'

Of course if you do this you will lose state information, so this is not recommended for a production application. Probably okay for while you are learning though.

3.3.1 Getting the URL itself

Now lets add some state and a function that does something with state. Notice that we now get rid of the noState
directive. In this example, we write an instance for FromReqURI that tries to read the next part after the url as an
value of type Int.

{-# OPTIONS -fglasgow-exts -fth #-}import HAppS
import HAppS.Protocols.SimpleHTTP2
import Data.Monoid
import Control.Monad.State (get, put)--import Data.Typeable--data MyState = MySt { appVal :: Int } deriving (Read, Show, Typeable)data MyState = MySt { appVal ::Int}deriving(Read,Show)instance Serialize MyState where
encodeStringM = defaultEncodeStringM
decodeStringM = defaultDecodeStringM
-- Question: why does my state have to be a monoid?-- instance Monoid MyState where-- mempty = MySt 0-- mappend (MySt x) (MySt y) = MySt (x+y)-- State needs to be an instance of the class StartState, to define-- a default initial value to be used when there is no saved state.-- There seems to be a default instance for Monoid => StartState,-- meaning that a warning about Monoid is produced if no StartState instance-- is given. I guess this is a bug. Also, the below template Haskell-- is broken too.-- code will work without this line-- $(inferStartState ''MyState) -- boilerplate that will eventually be SYBinstance StartState MyState where
startStateM =return$ MySt 0-- You wouldn't normally expect to get an entire state type out-- as the return value from fromReqURI, and would instead-- use a separate type to represent the information extracted from-- the URI.instance FromReqURI [String] MyState where
fromReqURI expr uri =do[val]<- fromReqURI (Prefix expr) uri
fmap MySt $ mbRead val
main = stdHTTP
[debugFilter -- we want to see debug messages in the console,h [""] GET $ ok $ val "GETting root hello"-- /val shows us the current value,h ["val"] GET $ ok $ \()()->do(MySt val)<- get; respond (show val)-- /set/56 would set the value to 56,h ["set"] GET $ ok $ \newVal ()->do put newVal; respond ("New value is "++show newVal)-- notice that newVal here gets type MyState which invokes the FromReqURI instance above.]

3.3.2 Getting from a POST'd value

Here, we replace parsing information from the URI with parsing information
from the headers and content. This uses the class FromMessage to extract the information, and its return value turns into the second argument to the method.

{-# OPTIONS -fglasgow-exts -fth #-}import HAppS
import HAppS.Protocols.SimpleHTTP2
import Data.Monoid
import Data.Typeable
import Control.Monad.State (get, put)data MyState = MySt { appVal ::Int}deriving(Read,Show, Typeable)instance Serialize MyState where
encodeStringM = defaultEncodeStringM
decodeStringM = defaultDecodeStringM
instance Monoid MyState where
mempty = MySt 0
mappend (MySt x)(MySt y)= MySt (x+y)$(inferStartState ''MyState)-- boilerplate that will eventually be SYB-- You wouldn't normally expect to get an entire state type out-- as the return value from fromMessageM, and would instead-- use a separate type to represent the information extracted from-- the post.instance FromMessage MyState where
fromMessageM m =do
val <- maybeM $ lookMbRead m "val"return$ MySt val
-- Note that fromMessageM is monadic, and can fail. If it fails, then-- the entire parse is counted as failing, and we drop through to-- the next handler. This can lead to spurious 404s, when what-- it really means is badly formatted form data. (or parse code)
main ::IO()
main = stdHTTP
[debugFilter -- we want to see debug messages in the console,h [""] GET $ ok $ val "GETting root hello",h (Prefix ["s"]) GET $ respIO $ fileServe staticPath
--, h (Prefix ["s"]) GET $ respIO $ fileServe staticPath -- 0.8.4, hs (Prefix ["s"]) GET $ basicFileServe staticPath -- 0.8.8-- /val shows us the current value,h ["val"] GET $ ok $ \()()->do(MySt val)<- get; respond (show val)-- /set with the POST data "val"=56 would set the value to 56,h ["set"] POST $ ok $ \() newVal ->do put newVal; respond ("New value is "++show newVal)-- The first one is FromReqURI and the second one is FromMessage-- The cryptic comment about is referring to the arguments () and newVal-- to the method. The type of newVal being MyState is what-- invokes our custom FromMessage instance above.]

1. view source to see server output.
2. make templates for each of your output types.
3.use the xsl lib that handles all sorts of standard template issues to make it all nice!
(The value of the element <something> is <xsl:value-of select="something"/>)

And then change one line and add one line for the example code above.

,h "/status" GET $ ok plain_xml examplePostVal'

to

,h "/status" GET $ ok xml exampleDumpVal

and add this to get the state and dump it to the browser:

exampleDumpVal ()()= get >>= respond

3.3.5 Redirection

Next we'll demonstrate redirection by creating a static error page and redirection to it in the fallthrough case.

You can use this shockingly complex error page.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>

If you want to dynamically choose the page to redirect to, as from
a form posting you might want code like

seeOther $ \fromurlarg frommessagearg ->do-- Do MACID stuff here.
respond ("/somewhere","redirecting")-- respond is just return.Right. seeOther picks up the pair-- and converts it to a proper response.

3.3.6 Send email

Sending email is straightforward. Create the message value, then hand it to send, which first tries to send it via SMTP_RELAY then directly. Alternatives are the more specific autoSend for direct delivery, envSend for relay via SMTP_RELAY, and sendHost host port for relay via a given server. Often in fear of spam, the recipient will accept your message only if you relay it via your ISP's outgoing SMTP server.

The contents field of the Envelope is the actual message, which should conform to the RFC 2822 SMTP standard. That is, the message body is preceded by header lines and a blank line (lines separated by "\r\n"). According to the standard, Date and From lines are obligatory. Other lines that probably should be in the messages are To and Subject. For attachments, and messages that are not plain US-ASCII text, see the RFC 2046 MIME standard.

If the SMTP server is temporarily unavailable or uses graylisting, the message should be saved for a retry later. Sending the message using queueMessage (uses send) achieves this.

3.3.7 Get and set cookies

At some point you'll want to get and set cookies in the browser.

To set a cookie, modify the response as in setsomecookie. To get a cookie, look in the request as in showallcookies.

To see output from this example, first go to /setcookie, and then to /showcookie.

3.3.8 Sessions

3.3.9 Blocking IO

There's a way to do blocking IO within HAppS, ...

4 How to structure your handlers?

As we have discussed before, HAppS programs are structured as lists of server parts that handle events. The type system stops you from making some mistakes, but the types involved are somewhat complex to understand at first. You can get a long way by re-using the example code, but here are some tips for when it gets more elaborate. We've already seen examples of nesting with 'multi' and 'hs'.

The list you give to stdHTTP is a list of ServerParts. Each ServerPart is a Handle which takes a Request, a ModResp which post-processes a further Result, or a Multi which nests more ServerParts. Of course the ServerParts can be constructed by functions so you don't have to write them manually.

The normal way to construct a Handle is 'h' which makes the handler list useful by choosing which requests are handled by which handlers. For ModResp, there's a corresponding 'hOut'. 'multi' isn't very useful, it just lets you define a part as a list of parts. 'multiIf' works more like 'h', letting you filter the requests that reach the nested list.

When you use 'h' to construct a ServerPart, you give it two expressions: one for matching the URI paths and another for matching the HTTP methods that you want the handler to handle. The third parameter is the handler itself: it gets one argument based on the URI and another based on the query parameters. The type classes FromMessage, FromReqURI and MatchMethod ensure that there are various types uri_match, req_match, uri_msg and req_msg can take, and you can define more.

As can be seen from the extremely generic type, 'hs' doesn't really do anything. From the earlier examples we have learned that it lets us include in our part list a part which does its own filtering and message decoding, such as basicFileServe, so that it looks like a call to 'h'. It doesn't do any actual request filtering, it just passes the filtering expressions to the part.

Here the effect of multiIf is that 'basicAuth' and 'handleAdminIndex' aren't applied to all requests. Notice how "admin" still needs to be included in the delete handlers, since 'multiIf' only filters.

(Implicit parameters are useful when you don't want to pass the parameters explicitly in a deep hierarchy of function calls. 'getTime' needs to be run in the monad but ?time is available in pure functions. ?root provides the right amount of '..' needed in the beginning of link URI references to get to the root directory of myApp even when the HTTP server hosts more than one application, perhaps even behind an Apache reverse proxy.)

5 How to deploy your application?

You need a server which has enough RAM to hold the state of your HAppS application and while they are transmitted the requests and responses. Of course you need to be able to run HAppS as a continuous background process on the server. Typically this is achieved by starting 'screen' and starting HAppS within that.

If you compile on the server, you need to have the compiler and all the required libraries installed, and the compiling and linking process easily takes a hundred megabytes of extra RAM. If you have a work station computer which is similar to the server, you can compile on the work station and upload the resulting binary to the server. Note that you don't necessarily need root (administration) privileges on the server, as HAppS can use unprivileged ports, such as the port 8000 by default. Similarily, you can install the compiling environment to your home directory by setting prefix as that directory.

At least if you want to run more than one HAppS process on one server, you need to change the default port numbers. This is done with for example './myApp --http-port=8001'.

5.1 Backgrounding

If you want to run the application in the background without 'screen', you still need a way to provide it with standard input as it waits for an 'e' there before it exits. One way to accomplish this is to background a shell pipeline which sleeps (practically) forever before outputting the 'e':

In your HAppS app, you need to take care of the links in HTML being interpreted correctly by web browsers. The links either need to be all relative, or the absolute paths need to include "/myhappsapp/". However, you needn't worry about URIs in HTTP headers such as a redirect Location, as ProxyPassReverse takes care of those.

(note : Starting from a base apache installation , I had to enable mod_proxy and mod_proxy_http, ie in apache2, cd /etc/apache2/mods-enabled, link ln-s ../mods-available/proxy_http.load . and idem for proxy.load .. ( i needed not proxy.conf, but check for you)