Overview

What is Markup?

We like to call it a “toolkit for stream-based generation of markup for the web”. The largest feature provided by Markup is an XML-based template engine that is heavily inspired by Kid.

Why yet another template engine?

We'll let Ryan Tomanko, the author of Kid, answer this one:

“There's at least four billion and nine text based template languages for
Python but there aren't a lot of options that fit nicely into the XML
tool-chain. Or, if they do fit nicely into the XML tool-chain, they don't
fit nicely with Python.”

Why XML-based?

Most template engines for web applications are character-stream based: they know nothing about the format of the text that is being generated. They simply substitute variable expressions, and provide some kind of directives looping, conditionals, etc. Thus they can be used to generate any kind of textual output, be it HTML, plain text emails, program code, or really anything else.

However, 99% of the templates used by web applications generate some kind of XML/HTML-based markup. So we believe that web applications can benefit from a template engine that “knows what it's doing” when it comes to markup. You don't need to worry about generating output that is not well-formed, nor do you need to worry about accidentially not escaping some data, thereby greatly reducing the risk for introducing XSS attack vectors. Furthermore, your templates look a lot more like the targetted output format: an HTML template looks like HTML, a template for an RSS feed looks like RSS. Directives in text-based template languages often result in rather messy templates, or produce excessive amounts of unnecessary white space.

In addition, text-based templates don't even work all that well for many text formats. Imagine you want to generate a plain text email or an iCalendar file. How do you deal with important concerns such as line-wrapping and white-space in your templates? You may be better off using a specialized formatter.

So then why not just use Kid?

We think that Kid represents a huge step forward for XML-based templating in Python. Match templates and the generator-based processing model are extremely powerful concepts.

But arguably Kid also has some basic design problems. For exampe, Kid generates Python code from templates, which adds a lot of complexity to the code and can make the process of locating and fixing template errors a true nightmare. A syntax error in a template expression will cause an exception that points somewhere in the generated code. In addition, as Kid is based on ElementTree, and the ElementTree API doesn't provide location information for parse events, exceptions reported by Kid often don't include information about what part of the template caused the error.

We felt these problems would best be addressed by developing a new engine form scratch, as opposed to trying to “fix” Kid.

What are the main differences between Kid and Markup?

Markup executes templates directly, there's no code generation phase. Expressions are evaluated in a more forgiving way by walking the AST. Template variables are stored on a stack, which means that some variable set in a loop deep in the template won't leak into the rest of the template. And even though Markup doesn't generate Python code for templates, it generally performs better than Kid (up to 2x in our tests).

Markup does not depend on ElementTree. It uses Expat for parsing XML, and is based on streaming slightly abstracted parse events through the processing pipeline. It uses XInclude (instead of Kids' py:extends) to allow template authors to factor out common bits. For match templates, it uses XPath expressions instead of the ElementTree API.

What other stuff is in the toolkit?

Beyond the template engine, Markup provides:

a unified stream-based processing model for markup, where

streams can come from XML or HTML text, or be generated programmatically using a very simple syntax.