I took the csv file and converted it into an XML file that tokens could handle, and here's the output of detokenizing that g3m file:
http://pastebin.com/L6376zxs
Note all the "NOT FOUND"s. This is where it was unable to find a token.

I haven't fully compiled the list of the last tokens for _player to parse, but there are tokens with values of ## and #### already.

Nifty. I'm hoping that they follow the same patterns as the TI-BASIC tokens, namely that no one-byte token has the same value as the first half of any two-byte sequence. If that holds true, then I can do tokenization and detokenization with no changes to my existing routines for the process.

I've sent five programs off to _player to be HEX'd for me. Until I get those back, heres a photo for each program showing all the tokens. Not the best representations, as I made them so I could easily spot missing tokens. It actually works out, that I was able to line up the characters exactly to the menus I got these from with little editing in the program. I got rid of spaces and breaks under "MATH" so I could fit it one screen.

I'm 50% sure I mixed up Greek & Latin here.

The code block above.

There are two types of people in the world: those who can extrapolate data from incomplete data

Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.