For those new to this blog
Marpa is something new in parsing --
it parses anything you can write in BNF and,
if your grammar is in one of the classes currently in practical use,
parses it in linear time.
Marpa::R2's parse engine is written in optimized C,
so that Marpa::R2's speed is competitive with parsers of far less power.

Marpa knows at every point in the parse,
which rules are possible,
how far into those rules the parse has already gotten,
and where those rules might start.
This degree of situational awareness is unusual in a parser,
and the more "production quality" the parser,
the more unusual it is.
This seems paradoxical.
Situational awareness seems like a
common sense thing to want in any parser,
and certainly in one that you'd hope would be able to
produce quality error messages.
But the practice has been for parsers,
as a way to achieve efficiency,
to turn their grammar into an abstraction.
This can be OK if all goes as planned, but if anything
goes wrong it means these parsers don't really have a clue
what it was.

In
stable distribution, Marpa::XS,
Marpa already made situational awareness available in
a form that allowed Ruby Slippers parsing --
at any point in the parse,
it could return a list of expected tokens.
A full description of the parse was also available in
Marpa::XS's progress reports,
which gave this information in text form.
This was wonderful for tracing parsing issues,
but it was not a convenient form for
post-processing.