1 Preface: DONT PANIC!

Recent experiences from a few of my fellow C++/Java programmers
indicate that they read various Haskell tutorials with "exponential
speedup" (think about how TCP/IP session starts up). They start slow
and cautious, but when they see that the first 3-5 pages do not
contain "anything interesting" in terms of code and examples, they
begin skipping paragraphs, then chapters, then whole pages, only to
slow down - often to a complete halt - somewhere on page 50, finding
themselves in the thick of concepts like "type classes", "type
constructors", "monadic IO", at which point they usually panic, think
of a perfectly rational excuse not to read further anymore, and
happily forget this sad and scary encounter with Haskell (as human
beings usually tend to forget sad and scary things).

This text intends to introduce the reader to the practical aspects of Haskell
from the very beginning (plans for the first chapters include: I/O, darcs,
Parsec, QuickCheck, profiling and debugging, to mention the few). The reader
is expected to know (where to find) at least the basics of haskell: how to run
"hugs" or "ghci", that layout is 2-dimensional, etc. Other than that, we do
not plan to take radical leaps, and will go one step at a time in order not to
lose the reader along the way. So DONT PANIC, take your towel with you and
read along.

Oh, almost forgot: author is very interested in ANY feedback. Drop him a line
or a word (see Adept for contact info)

2 Chapter 1: Ubiquitous "Hello world!" and other ways to do IO in Haskell

Each chapter will be dedicated to one small real-life task which we will
complete from the ground up.

So here is the task for this chapter: in order to free up space on
your hard drive for all the haskell code you are going to write in the
nearest future, you are going to archive some of the old and dusty
information on CDs and DVDs. While CD (or DVD) burning itself is easy
these days, it usually takes some (or quite a lot ot) time to decide
how to put a several Gb's of digital photos on CD-Rs, when directories
with images range from 10 to 300 Mb's in size, and you dont want to
burn half-full (or half-empty) CD-Rs.

So, the task is to write a program which will help us to put a given
collection of directories on the minimum possible amount of media,
while packing the media as tight as possible. Let's name this program
"cd-fit".

Oh. Wait. Let's do the usual "hello world" thing, before we forget about it,
and then move on to more interesting things:

-- put this in hello.hs
module Main where
main = putStrLn "Hello world!"

Run it:

$ runhaskell ./hello.hs
Hello world!

OK, we've done it. Move along now, nothing interesting here :)

Any serious development must be done with the help of version control
system, and we will not make an exception. We will use modern
distributed version control system "darcs". "Modern" means that it is
written in Haskell, "distributed" means that each working copy is
repository in itself.

First, lets create an empty directory for all our code, and invoke
"darcs init" there, which will create subdirectory "_darcs" to store
all version-control-related stuff there.

Fire up your favorite editor and create new file "cd-fit.hs" in our
working directory. Now lets think for a moment about how our program
will operate and express it in pseudocode:

main = read list of directories and their sizes
decide how to fit them on CD-Rs
print solution

Sounds reasonable? I thought so.

Lets simplify our life a little and assume for now that we will
compute directory sizes somewhere outside our program (for example,
with "du -sb *") and read this information from stdin.
Now let me convert all this to Haskell:

Not really working, but pretty close to plain English, eh? Let's stop
for a moment and look closer at whats written here line-by-line

Let's begin from the top:

input <- getContents

This is an example of the Haskell syntax for doing IO (namely, input). This
line is an instruction to read all the information available from the stdin,
return it as a single string, and bind to the symbol "input", so we can
process this string any way we want.

How did I know that? Did I memorize all the functions by heart? Of course not!
Each function has a type, which, along with function's name, usually tells a
lot about what function will do.

Let's fire up interactive Haskell environment and examine this function
up close:

We see that "getContents" is a function without arguments, that will return
"IO String". Prefix "IO" meant that this is an IO action. It will return
String, when evaluated. Action will be evaluated as soon as we use "<-" to
bind its result to some symbol.

Note that "<-" is not a fancy way to do assignment to variable. It is a way to
evaluate (execute) IO actions, in other words - to actually do some I/O and
return its result (if any).

We can choose not to evaluate action obtained from "getContents", but to carry
it around a bit and evaluate later:

let x = getContents
-- 300 lines of code here
input <- x

So, as you see, IO actions can act like an ordinary values. Suppose that we
have built a list of IO actions and found a way to execute them one by one.
This would be a way to simulate imperative programming with its notion of
"order of execution".

Haskell allows you to do better than that.

Standard language library (named "Prelude", by the way) provides us with lots
of functions that return useful primitive IO actions. In order to combine them
to produce more complex actions, we use a "do":

When all this will actually be executed? Answer: as soon as we evaluate "c"
using the "<-" (if it returns result, as "getContents" does) or just
by using it as a function name (if it does not return a result, as "print"
does):

process = do putStrLn "Will do some processing"
c
putStrLn "Done"

Notice that we took a bunch of functions ("someAction", "someOtherAction",
"print", "putStrLn") and using "do" created from them a new function, which we
bound to symbol "c". Now we could use "c" as a building block to produce even
more complex function, "process", and we could carry this on and on.
Eventually, some of the functions will be mentioned in the code of function
"main", to which the ultimate topmost IO action any Haskell program is bound.

When the "main" will be executed/evaluated/forced? As soon as we run the
program. Read this twice and try to comprehend:

Execution of the Haskell program is an evaluation of the symbol "main" to
which we have bound an IO action. Via evaluation we obtain the result of that
action.

Readers familiar with advanced C++ or Java programming and arcane body of
knowledge named "OOP Design Patterns" might note that "build actions from
actions" and "evaluate actions to get result" is essentially a "Command
pattern" and "Composition pattern" combined. Good new: in Haskell you get them
for all your IO, and get them for free :)

We see that "main" is indeed an IO action which will return nothing
when evaluated. When combining actions with "do", the type of the
result will be the type of the last action, and "putStrLn something" has type
"IO ()":

Oh, by the way: have you noticed that we actually compiled our first
Haskell program in order to examine "main"? :)

Lets celebrate that by putting it under version control: execute
"darcs add cd-fit.hs" and "darcs record", answer "y" to all questions
and provide a commit comment "Skeleton of cd-fit.hs"

Let's try to run it:

$ echo "foo" | runhaskell cd-fit.hs
DEBUG: got input foo

Exercises:

Try to write program that takes your name from the stdin and greets you (keywords: getLine, putStrLn);

Try to write program that asks for you name, reads it, greets you, asks for your favorite color, and prints it back (keywords: getLine, putStrLn).

3 Chapter 2: Parsing the input

OK, now that we have proper understanding of the powers of Haskell IO
(and are awed by them, I hope), lets forget about IO and actually do
some usefull work.

As you remember, we set forth to pack some CD-Rs as tightly as
possible with data scattered in several input directories. We assume
that "du -sb" will compute the sizes of input directories and output
something like:

Our next task is to parse that input into some suitable internal
representation.

For that we will use powerful library of parsing combinators named
"Parsec" which ships with most Haskell implementations.

Much like the IO facilities we have seen in the first chapter, this
library provides a set of basic parsers and means to combine into more
complex parsing constructs.

Unlike other tools in this area (lex/yacc or JavaCC to name a few),
Parsec parsers do not require separate preprocessing stage. Since in
Haskell we can return function as a result of function and thus
construct functions "from the thin air", there is no need for separate
syntax for parser description. But enough advertisements, lets actually
do some parsing:

Just add those lines into "cd-fit.hs". Here we see quite a lot of new
things, and several those that we know already.

First of all, note the familiar "do" construct, which, as we know, is
used to combine IO actions to produce new IO actions. Here we use it
to combine "parsing" actions into new "parsing" actions. Does this
mean that "parsing" implies "doing IO"? Not at all. Thing is, I must
admit that I lied to you - "do" is used not only to combine IO
actions. Is is used to combine any kind of so-called monadic
actions or monadic values together.

Think about monad as of "" in the functional world.
Monad is a way to hide from the user (programmer) all the machinery
required for complex functionality to operate.

As you might have heard, Haskell has no notion of "assignment",
"mutable state", "variables", and is a "pure functional language",
which means that every function called with the same input parameters
will return exactly the same result. Meanwhile "doing IO" requires
hauling around file handles and their states and dealing with IO
errors. "Parsing" requires to track position in the input and dealing
with parsing errors.

In both cases Wise Men Who Wrote Libraries cared for our needs and
hid all underlying complexities from us, exposing the API of their
libraries (IO and parsing) in the form of "monadic action" which we
are free to combine as we see fit.

Think of programming with monads as of doing the remodelling with the
help of professional remodelling crew. You describe sequence of
actions on the piece of paper (that's us writing in "do" notation),
and then, when required, that sequence will be evaluated by the
remodelling crew ("in the monad") which will provide you with end
result, hiding all the underlying complexity (how to prepare the
paint, which nails to choose, etc) from you.

Lets use the interactive Haskell environment to decipher all the
instructions we've written for the parsing library. As usually, we'll
go top-down:

Assuming (well, take my word for it) that "GenParser Char st" is our
parsing monad, we could see that "parseInput", when evaluated, will
produce a list of "Dir", and "dirAndSize", when evaluated, will
produce "Dir". Assuming that "Dir" somehow represents information
about single directory, that is pretty much what we wanted, isn't it?

Let's see what a "Dir" means. We defined datatype Dir as a record,
which holds an Int and a String:

, which would define datatype "Dir" with data constructor "D".
However, traditionally name of the datatype and its constructor are
chosen to be the same.

Clause "deriving Show" instructs the compiler to make enough code "behind
the curtains" to make this datatype conform to the interface of
the type class Show. We will explain type classes later, for
now lets just say that this will allow us to "print" instances of
"Dir".

Exercises:

examine types of "digit", "anyChar", "many", "many1" and "manyTill" to see how they are used to build more complex parsers from single ones.

compare types of "manyTill", "manyTill anyChar" and "manyTill anyChar newline". Note that "anyChar `manyTill` newline" is just another syntax sugar. Note that when function is supplied with less arguments that it actually needs, we get not a value, but a new function, which is called partial application.

OK. So, we combined a lot of primitive parsing actions to get ourselves a
parser for output of "du -sb". How can we actually parse something? the Parsec library supplies us with function "parse":

At first the type might be a bit cryptic, but once we supply "parse" with the parser we made, the compiler gets more information and presents us with a more concise type.

Stop and consider this for a moment. The compiler figured out type of the function without a single type annotation supplied by us! Imagine if a Java compiler deduced types for you, and you wouldn't have to specify types of arguments and return values of methods, ever.

OK, back to the code. We can observe that the "parser" is a function, which,
given a parser, a name of the source file or channel (f.e. "stdin"), and
source data (String, which is a list of "Char"s, which is written "[Char]"),
will either produce parse error, or parse us a list of "Dir".

Datatype "Either" is an example of datatype whose constructor has name, different
from the name of the datatype. In fact, "Either" has two constructors:

data Either a b = Left a | Right b

In order to undestand better what does this mean consider the following
example:

*Main> :t Left 'a'
Left 'a' :: Either Char b
*Main> :t Right "aaa"
Right "aaa" :: Either a [Char]
*Main>

You see that "Either" is a union (much like the C/C++ "union") which could
hold value of one of the two distinct types. However, unlike C/C++ "union",
when presented with value of type "Either Int Char" we could immediately see
whether its an Int or a Char - by looking at the constructor which was used to
produce the value. Such datatypes are called "tagged unions", and they are
another in the Haskell toolset.

Did you also notice that we provide "parse" with parser, which is a monadic
value, but receive not a new monadic value, but a parsing result? That is
because "parse" is an evaluator for "Parser" monad, much like the GHC or Hugs runtime is an evaluator for the IO monad. The function "parser" implements all monadic machinery: it tracks errors and positions in input, implements backtracking and lookahead, etc.

Lets extend our "main" function to use "parse" and actually parse the input
and show us the parsed data structures:

As you might already have recognized, our problem is a classical one. It is
called a "knapsack problem" (google it up, if you don't know already what it
is. There are more than 100000 links).

Lets start from the greedy solution, but first let's slightly modify our "Dir"
datatype to allow easy extraction of its components:

data Dir = Dir {dir_size::Int, dir_name::String} deriving Show

Exercise: examine types of "Dir", "dir_size" and "dir_name"

From now on, we could use "dir_size d" to get a size of directory, and
"dir_name d" to get its name, provided that "d" is of type "Dir".

The Greedy algorithm sorts directories from the biggest down, and tries to puts
them on CD one by one, until there is no room for more. We will need to track
which directories we added to CD, so lets add another datatype, and code this
simple packing algorithm:

I'll highlight the areas which you could explore on your own (using other nice
tutorials out there, of which I especially recommend "Yet Another Haskell
Tutorial" by Hal Daume):

We choose to import a single function "sortBy" from a module Data.List, not the whole thing.

Instead of coding case-by-case recursive definition of "greedy_pack", we go with higher-order approach, choosing "foldl" as a vehicle for list traversal. Examine its type. Other useful function from the same category are "map", "foldr", "scanl" and "scanr". Look them up!

To sort list of "Dir" by size only, we use custom sort function and parameterized sort - "sortBy". This sort of setup where the user may provide a custom "modifier" for a generic library function is quite common: look up "deleteBy", "deleteFirstsBy", "groupBy", "insertBy", "intersectBy", "maximumBy", "minimumBy", "sortBy", "unionBy".

To code the quite complex function "maybe_add_dir", we introduced several local definitions in the "let" clause, which we can reuse within the function body. We used a "where" clause in the "greedy_pack" function to achieve the same effect. Read about "let" and "where" clauses and the differences between them.

Note that in order to construct a new value of type "DirPack" (in function "maybe_add_dir") we haven't used the helper accessor functions "pack_size" and "dirs"

In order to actually use our greedy packer we must call it from our "main"
function, so lets add a lines:

Now it is time to test our creation. We could do it by actually running it in
the wild like this:

$ du -sb ~/DOWNLOADS/* | runhaskell ./cd-fit.hs

This will prove that our code seems to be working. At least, this once. How
about establishing with reasonable degree of certainty that our code, parts
and the whole, works properly, and doing so in re-usable manner? In other
words, how about writing some test?

Java programmers used to JUnit probably thought about screens of boiler-plate
code and hand-coded method invocations. Never fear, we will not do anything as
silly :)

QuickCheck is a tool to do automated testing of your functions using
(semi)random input data. In the spirit of "100b of code examples is worth 1kb of
praise" lets show the code for testing the following property: An attempt to pack directories returned by "greedy_pack" should return "DirPack" of exactly the same pack:

We've just seen our "greedy_pack" run on a 100 completely (well, almost
completely) random lists of "Dir"s, and it seems that property indeed holds.

Lets dissect the code. The most intriguing part is "instance Arbitrary Dir
where", which declares that "Dir" is an instance of typeclass "Arbitrary". Whoa, that's a whole lot of unknown words! :) Let's slow down a
bit.

What is a typeclass? A typeclass is a Haskell way of dealing with the
following situation: suppose that you are writing a library of usefull
functions and you dont know in advance how exactly they will be used, so you
want to make them generic. Now, on one hand you dont want to restrict your
users to certain type (e.g. String). On the other hand, you want to enforce
the convention that arguments for your function must satisfy a certain set of
constraints. That is where typeclass comes in handy.

Think of typeclass as a contract (or "interface", in Java terms) that
your type must fulfill in order to be admitted as an argument to certain
functions.

It could be read this way: "Any type (let's name it 'a') could be a member of the class Arbitrary as soon as we define two functions for it: "arbitrary" and "coarbitrary", with signatures shown. For types Dir, Bool, Double, Float, Int and Integer such definitions were provided, so all those types are instance of class Arbitrary".

Now, if you write a function which operates on its arguments solely by means
of "arbitrary" and "coarbitrary", you can be sure that this function will work
on any type which is an instance of "Arbitrary"!

Lets say it again. Someone (maybe even you) writes the code (API or library),
which requires that input values implement certain interfaces, which is
described in terms of functions. Once you show how your type implements this
interface you are free to use API or library.

Conside the function "sort" from standard library:

*Main> :t Data.List.sort
Data.List.sort :: (Ord a) => [a] -> [a]

We see that it sorts lists of any values which are instance of typeclass
"Ord". Let's examine that class:

We see a couple of interesting things: first, there is an additional
requirement listed: in order to be an instance of "Ord", type must first be an
instance of typeclass "Eq". Then, we see that there is an awful lot of
functions to define in order to be an instance of "Ord". Wait a second, isn't
it silly to define both (<) and (>) when one could be expressed via another?

Right you are! Usually, typeclass contains several "default" implementation
for its functions, when it is possible to express them through each other (as
it is with "Ord"). In this case it is possible to supply only a minimal
definition (which in case of "Ord" consists of any single function) and others
will be automatically derived. If you supplied fewer functions than are required
for minimal implementation, the compiler/interpreter will say so and
explain which functions you still have to define.

Once again, we see that a lot of types are already instances of typeclass Ord, and thus we are able to sort them.

Now, lets take a look back to the definition of "Dir":

data Dir = Dir {dir_size::Int, dir_name::String} deriving Show

See that "deriving" clause? It instructs the compiler to automatically derive code to make "Dir" an instance of typeclass Show. The compiler knows about a bunch of standard typeclasses (Eq, Ord, Show, Enum, Bound, Typeable to name a few) and knows how to make a type into a "suitably good" instance of any of them. If you want to derive instances of more than one typeclass, say it this way: "deriving (Eq,Ord,Show)". Voila! Now we can compare, sort and print data of
that type!

Sidenote for Java programmers: just imagine java compiler which derives code
for "implements Storable" for you...

Sidenote for C++ programmers: just imagine that deep copy constructors are
being written for you by compiler....

Exercises:

Examine typeclasses Eq and Show

Examine types of (==) and "print"

Try to make "Dir" instance of "Eq"

OK, back to our tests. So, what we have had to do in order to make "Dir" an
instance of "Arbitrary"? Minimal definition consists of "arbitrary". Let's
examine it up close:

*Main> :t arbitrary
arbitrary :: (Arbitrary a) => Gen a

See that "Gen a"? Reminds you of something? Right! Think of "IO a" and "Parser
a" which we've seen already. This is yet another example of action-returning
function, which could be used inside "do"-notation. (You might ask yourself,
wouldn't it be useful to generalize that convenient concept of actions and
"do"? Of course! It is already done, the concept is called "Monad" and we will talk about it in Chapter 400 :) )

Since 'a' here is a type variable which is an instance of "Arbitrary", we could substitute "Dir" here. So, how we can make and return an action of type "Gen Dir"?

We have used the library-provided functions "choose" and "elements" to build up
"gen_size :: Gen Int" and "gen_name :: Gen String" (exercise: don't take my
word on that. Find a way to check types of "gen_name" and "gen_size"). Since
"Int" and "String" are components of "Dir", we sure must be able to use "Gen
Int" and "Gen String" to build "Gen Dir". But where is the "do" block for
that? There is none, and there is only single call to "liftM2".