string - Code golf: "Color highlighting" of repeated text

Build a program that finds all substrings and "tags" them with color attributes (effectively highlighting them in XML).

The rules:

This should only be done for substrings of length 2 or more.

Substrings are just strings of consecutive characters, which may include non-alphabetic characters. Note that spaces and other punctuation do not delimit substrings.

Character casing cannot be ignored.

The "highlight" should be done by tagging the substring in XML. Your tagging should be of the form <TAG#>theSubstring</TAG#> where # is a positive number unique to that substring and identical substrings.

The priority of the algorithm is to find the longest substring, not how many times it matches within the text.

Note: The order of the tagging shown in the example below is not important. Its just used by the OP for clarity.

The criteria remains "elegance", which admittedly IS slightly vague, but it also encapsulates simple character/line counts as well. Comments by others and/or upvotes are also indicative of how the SO community views the challenge

And the result of running this on the example input, it picks the following words ('LoremIpsum', 'dummytext', 'industry', 'print', 'types', 'oft', 'ing', 'and', 'the', 'ook', 'ss', 'im', 'he', 'tt', 'en', 'er', 'le', 'pe') and the result is:

PS. Somebody complained so I added input and output statements. To the confused I apologize - it seemed obvious to me. Apparently not, so I added prefix/trailer statements, which are not required by the problem spec and should not be counted to the code length.

网友答案:

Haskell: 343/344 403420445485 characters

Character count is 343 while using an exponential algorithm, whereas it is 344 when using a quadratic algorithm.

The code posted is the quadratic one. For the exponential algorithm, change the one occurrence of inits=<<tails to subsequences in the code.

I think you can use back references to do this. See this post : Regular Expression to detect repetition within a string
I've done many attempts and for the moment I have this expression: #([a-zA-Z ]+).*\1#, but I think it finds the first repeated string, not the largest...
This was before I knew you didn't care about words...
What you should do is:

find the largest sequence of characters repeated in the text

remove it from the text where it appears

iterate until you find nothing repeated

use tomit's method to colorize the strings you have memorized

the step is described on this page: http://en.wikipedia.org/wiki/Longest_common_substring_problem
And here is a php implementation : http://www.davidtavarez.com/archives/longer-common-substring-problem-php-implementation/ (you'll have to fix it, it contains html entities, and the comment says it returns an integer but we don't know what it represents...), if it still does not work, you can try to implement wikipedia's pseudo-code.

网友答案:

Python, 236281 chars, no REGEX :)

Makes a set M of all 2+ character strings then iterates through them to assign them in greedy-length order

Many thanks to Dennis Williamson who helped me arrive at this approach by answering a few related questions I had on shell scripting - here and here.

Known issues with the below:

It works only for ascii files and not binary ones

It works only if there are no newlines in the file

It takes exponentially longer as the file gets longer

It breaks easily on files more than a few kb long (runs out of tmp disk space)

As you can see, its a huge brute-force method - not a smart algorithm at all. I've recorded the time taken for a few sample files.

bytes time(s)
204 1.281
407 24.916
610 269.302

The point of even doing this as I did below was more about a "thought-challenge" for me - to do this in a shell environment and in a manner as "complete" as possible. Nothing more. Of course, as the results show, its grossly inefficient, so its completely unsuited for a real world application.