A basic TokenStream.Tokenizer implementation that ignores whitespace but includes tokens for individual symbols, the period
('.'), single-quoted strings, double-quoted strings, whitespace-delimited words, and optionally comments.

Note this Tokenizer may not be appropriate in many situations, but is provided merely as a convenience for those situations
that happen to be able to use it.