String Tokenizer

When you are writing a lexical analyzer it would be helpful to have a class like the
StreamTokenizer class from Sun's Java, so I've done something like that and here it is the
CStringTokenizer class, the usage of this class is the same as by the StreamTokenizer from
Java, there are a few additional functionality's and the function names are slightly
different

The CStringTokenizer class is contained in the files StringTokenizer.h and
StringTokenizer.cpp

This class is writen to be used at many types of lexical analyzers, you can inherit
your own lexical analyzer class from this CStringTokenizer class.

NEW!!!

Bug corections:

1. String memory alocation error corected

2. Pascal comments bug corected

Sample project:

The sample project shows how you can use the String Tokenizer Class, and how you can
adjust it to your needs, the project also makes some pseudo Pascal, sintactical and some
semantical analisis, the String Tokenizer should be now bugfree, but the PascalLexical,
sintactical or semantical analizer should have bugs (I know 2 of them).

Top White Papers and Webcasts

Migrating away from Windows Server 2003 is an investment in your organization's future, and there has never been a better time to begin the migration process. Take the next step to transform your datacenter by upgrading your server platform with leading edge Windows Operating Systems and SanDisk flash solutions.

Data integrity and ultra-high performance dictate the success and growth of many companies.
One of these companies is BridgePay Network Solutions, a recently launched and rapidly growing financial services organization that allows merchants around the world to process millions of daily credit card transactions. Due to the nature of their business, their IT team needed to strike the perfect balance between meeting regulatory-mandated data security measures with the lowest possible levels of latency and …