On Dec 11, 6:56 pm, Kaz Kylheku <k...@kylheku.com> wrote:> On 2011-12-07, Andrew Tomazos <and...@tomazos.com> wrote:>> > Summary: We want to find out how often a given token appears in a> > random stream formed by concatenating randomly chosen strings from a> > given set of strings.> > (Note hits can overlap each other)>> But tokens do not overlap, so you're not actually extracting tokens. Using> C tokens as an example, the C token >>= is one hit, not four. The longest> match calls for extracting three characters and moving on.

Substitute occurrences of the word "token" in my post for "key
string" (or just "string") and reinterpret.
-Andrew.
[I suppose, but finding tokens would be more interesting. -John]