We evaluate our Bitwise Data Parallel implementation against GNU grep version 2.10 and nrgrep version 1.12. GNU grep is a popular open-source grep implementation that uses DFAs. NR-grep is one of the strongest competitors in regular expression matching performance. It uses an NFA-based approach.

Each GREP implementation is tested with the five regular expressions in Table \ref{RegularExpressions}. Xquote matches any of the representations of a quote in xml. It is run on roads-2.gml, a 11,861,751 byte gml data file. The other four expressions are taken from Benchmark of Regex Libraries[?] and are all run on a concatenated version of the linux howto[?] which is 39,422,105 bytes in length. @ simply matches the "@" character. It demonstrates the overhead involved in matching the simplest regular expression. Date, Email, and URIOrEmail provide examples of common uses for regular expression matching.

For the @ expression, Grep had very slightly better performance than the bitstreams implementation. The bitstreams implementation has a fair amount of overhead for transposition that hurts relative performance for very simple expressions.

For the Date expression, NRGrep is able to skip large portions of the input file since every time it encounters a character that can't appear in a date, it can skip past six characters. For the more compicated expressions, it loses this advantage.

The bitstreams implementation has fairly consistent performance. As the regular expression complexity increases, the cycle count increases slowly. The largest difference in cycles per byte for the bitstreams implementation is a ratio of 2 to 1. The same cannot be said of Grep or NRGrep. NRGrep uses more than 10 times the cycles per byte for Xquote than for Date. The number of cycles per byte that Grep uses for URIOrEmail is almost 900 times as many as it uses for @.

Figure \ref{fig:SSEInstructionsPerByte} shows instructions per byte. The relative values mirror cycles per byte. The bitstreams implementation continues to show consistent performance. This is especially noticeable in Figure \ref{fig:SSEInstructionsPerCycle}, which shows instructions per cycle. The bitstreams implementation has almost no variation in the instructions per cycle. Both Grep and NRGrep have considerably more variation based on the input regular expression.

Overall, our performance is considerably better than both NRGrep and Grep for the more complicated expressions that were tested. Also, our performance scales smoothly with regular expression complexity so it can be expected to remain better for complicated expressions in general.