Some text, like tabs and spaces, are removed altogether. JS-style comments (both // line comments and /* */ block) are also removed. If there are multiple newlines in a row, they are consolidated into a single line token; any opening and closing lines are removed as well.

Numbers and quoted strings become value tokens; all interpolated values are inserted as value tokens as well. Words that match JavaScript's rules for identifiers become identifier tokens. Most punctuation becomes operator tokens.

Grouping symbols ( ) [ ] { } are specially handled and become startToken and endToken, and these have a special purpose in the next stage of processing.

Skeleton syntax trees

Next, the grouping tokens are matched, and the tokens between them are collected into a single structure token:

This step allows Zebu to target visibly pushdown languages -- essentially this means that Zebu grammars can be as (computationally) simple as regular expressions, but still allow recursive structures in brackets. This also makes it easier to provide good error messages & avoid some performance issues. This technique was adapted from Owl and Dylan's D-Expressions.

Parsing

In the next step, we try to match the tokens to the top rule of the grammar.

Combining grammars

Operator grammars

Operator precedence is rather tedious to implement in this grammar, so zebu also includes a separate mini-language just for defining operators. Operators in the same clause have the same precedence level; clauses nearer the top of the list are higher in precedence.

Future work

Better Documentation. It took me a long time to understand how parsing works, and I suspect that Zebu's documentation is quite opaque to someone who is new to parsing. How can Zebu be a good introduction to parsing?

Babel macro support. Zebu is designed such that most of the parsing, both in grammars and in the languages they create, can be done at compile time. Zebu doesn't currently take advantage of this, but it would be considerably more useful if it did.

Interactive parsing workbench.Nearley and Owl both have interactive browser examples that were immensely helpful for understanding how they work. Zebu should also support online/interactive examples.