At 16:53 22/06/04 +0300, Max Ischenko wrote:
>Hi all,
>>I'm going to try to implement a version of Markdown tool[1] in Haskell.
>The application is rather simple: take a text file with some (simple)
>mark-up embedded in it and turn it into another text file, this time with
>XHTML markup.
Cool project!
>I need some guidelines on how to get started. I'll have to struggle with
>both the language itself (as I am a newcomer) plus with libraries that I
>may need.
Ah, how to get started? I find myself wrestling with this each time I
start a new Haskell project, or phase of a project.
>If I to do this in some convenient language, like Python, I'd use a lexer
>to parse the input, a state machine to build some internal representation
>and some serializer to write this into XHTML.
These choices are all available in Haskell. The way you pit them together
may be a little different (see below).
It's bee a while since I peered at MarkDown, and I forget how the language
looks, except that I recall that, by design, it's kept very simple and has
very little "superfluous" syntactic structure.
I think the first choice is whether to go for a separately identifiable
lexing phase, rather than working directly from the raw text. Either might
work, I think. The HaXml XML parser has a separate lexer, but it turns out
that it's not always easy to get the tokenization right without having
contextual information (e.g. from the syntax analyzer). (XML is rather
messy in that way.)
In Haskell, it's often reasonably efficient to construct a program as a
composition of "filters", rather like a Unix command pipeline; lazy
evaluation often means that data can "stream" through the filters and never
exist in its entirety in an intermediate form. This immediately allows the
program structure to be resolved into a number of smaller, independent
pieces; e.g.
tokenize :: String -> [Token]
parse :: [Token] -> DocModel
createXHTML :: DocModel -> Document -- (cf. HaXml)
Then HaXml provides function that can generate textual XML. Thus the
overall conversion function might look like:
markdownToXHTML :: String -> String
markdownToXHTML = show . document . createXHTML . parse . tokenize
(where "document" is from the HaXml module Text.XML.HaXml.Pretty).
For parsing of any complexity, I recommend Parsec: it has the advantage of
being very well documented, and it helps to show how monads can be used to
handle state information.
The outline sketched above has at least one weakness, it doesn't provide
any way to handle errors. This could be overcome by using Either as an
error monad (see Control.Monad and Control.Monad.Error in the standard
hierarchical libraries), and then using >>= in place of function
composition (noting the reversal of component order):
-- Get definition of (Either String) as an instance of Monad:
import Control.Monad()
import Control.Monad.Error()
tokenize :: String -> Either String [Token]
parse :: [Token] -> Either String DocModel
createXHTML :: DocModel -> Either String Document
markdownToXHTML :: String -> Either String String
markdownToXHTML s =
tokenize s >>= parse >>= createXHTML >>= (Right . show . document)
Or, using the do-notation:
markdownToXHTML s = do
{ ts <- tokenize s
; dm <- parse ts
; xd <- createXHTML
; return $ show (document xd)
}
>Which approaches could you suggest for that kind of application in
>Haskell? I'd especially glad to hear about some "idiomatic" or "native"
>ways to solve the problem.
>I've made a quick search and found tools like Parsec, HaXML, Happy and
>WASH/HTML. Not sure which ones I'll need.
I recommend Parsec (if it fits your needs, of course).
It's not clear to me whether you want to generate text directly from the
document data, or via something like HaXml. HaXml does have a pretty
printing function (though I have noticed some minor problems with it,
though probably nothing you'd encounter with this application.)
...
Just to check out my use of >>= and do-notation, I constructed a trivial
complete program using both.
[[
-- spike-composition.hs
--
-- composition of functions; using Either as an error monad
-- Get definition of (Either String) as an instance of Monad:
import Control.Monad()
import Control.Monad.Error()
data Token = T { unT::Char }
data DocModel = D String
instance Show DocModel where
show (D s) = s
tokenize :: String -> Either String [Token]
tokenize = Right . map T
parse :: [Token] -> Either String DocModel
parse = Right . D . reverse . map unT
docToString1 s =
tokenize s >>= parse >>= (Right . ("docToString1: "++) . show)
docToString2 s = do
{ ts <- tokenize s
; dm <- parse ts
; return $ (("docToString2: "++) . show) dm
}
t1 = docToString1 "abc" -- == Right "docToString1: cba"
t2 = docToString2 "def" -- == Right "docToString2: fed"
]]
#g
--
>[1] - http://daringfireball.net/projects/markdown/>>TIA,
>max.
>_______________________________________________
>Haskell-Cafe mailing list
>Haskell-Cafe at haskell.org>http://www.haskell.org/mailman/listinfo/haskell-cafe
------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact