Let me start by saying that while I understood in theory what was happening in the Schwartzian and Guttman Rosler Transformations, I had a difficult time understanding the the details of how the magic "map sort map" incantations actually worked.

I copied the snippet, filled in the XFORM function, and ran the code manually through the debugger printing what various parts of the sort transformation did, and finally "got it".

Now, I'm pretty intelligent, so I figured that if I hadn't gotten it after reading bunches of the "sort" technique articles around the catacombs, that it would probably help a number of others if I could explain the magic a bit so they might understand what was going on with these transformation techniques.

The basic theory

We take the data we want to sort some particular way, and figure out how to mangle that data such that a standard Perl ascii sort will do the job for us, then unmangle the data such that we get back our original information, except now in sorted order. If you understand this, and have inklings about how to transform the data, but don't understand the details about how it all gets put together, this node is intended specifically for you.

How this Transformation works

As is true for all transformations, it helps a great deal to first understand that everything within this construct: (directly from Tye's node above)

is layed out in reverse order from which the execution is done. So, start at the end, you'll see the 0..$#list, which is just a list of all the indexes within the array @list. The indexes are what we're iterating over, and $_ is set to each index in turn.

Next we have :map { XFORM($list[$_]) . pack "N", $_ } which simply calls XFORM(), which is your transformation of the data into a sortable string, and then appends to that string an encoding of the index value packed (in an architecturally independant format) into 4 bytes. That's the pack "N", $_ portion. The map statement just says to do that for every index.

The resulting array of transformed, stringified data with the index values packed and glued to the end of each string is then passed to the sort routine. Assuming your transformation works as it is supposed to, the correctly sorted array of transformed, stringified data is then passed to the first map statement. This map just grabs the last 4 bytes of each string, which must be our encoded index value from the original array, and decodes it back into a usable index value; and it does this for all the entries. To be completely clear, the transformed, stringified stuff is thrown away at this point; only the index values are left.

What we have at this point is an array of index values from the original array, but these index values are in the desired sorted order! At this point we've simplified the original transformation into this: my @sorted=@list[@array_of_sorted_indexes]; Execute that and we finally get @sorted which has the original array's information, but in sorted order, and @list is unmodified.

If you're fuzzy about the @list[@array] notation, this is perfectly legal. Try this in a debugging session:print @{[a..z]}[0,4,8,14,20]; and you'll get the list 'a,e,i,o,u' back.

If all that was still not clear enough and you need a concrete example with the sort magic broken out into individual statements, the following sorts dotted IP address information:

It'll read from an input file or Standard In, and grab the first IP address on each line. If you want to manually run through the split up sort steps within a debugger, the script's first argument should be any positive number. Or just set $DEBUG yourself while in the debugger...

As I was reading through the article, I was anxiously awaiting the appearance of the Schwartzian Transform.

But alas, even though you mention it in the title, and once in the body, you never got to it. Just the GRT, and Tye's particular trick.

I still think that the ST is always clearer than the GRT, and that jumping
to the GRT without careful consideration is a case of premature optimization—except for those rare cases where the work required to wiggle your data into a single sortable string isn't all that unobvious.

Ah, yes, you're right. When I started, I had intended to better show that understanding this one instance, which is a specific form of the GRT, which is in turn a specialization of the Schwartzian Transform, it's fairly easy to generalize the understanding to cover the more general cases.

But, I never quite made it that far.

Thanks for the vote of confidence anyway!

Update: Let's attempt to rectify this situation.

The analysis on the root node is a very specific, sort transformation technique. I've only just scratched the surface, but I believe this technique is very powerful and should be useful for nearly any sorting job. However, it's not the only technique out there. I think that by understanding this technique, you should be able to easily generalize this to better understand the other transformations.

Guttman Rosler Transformations

The next step up are the Guttman Rosler Transformations. The core idea of the GRT is to use the built in perl ascii sorting. In order to use this, we need to create stringified representations of our data first, then sort, then get the original data back. In a lot of cases, the map functions do a lot of work to put the data in the correct form, and then undo that work after the sort to get the data back. An example of this can be found at this node by jdporter.

Slightly modified to replace original @a array with more descriptive @files array

In this case the OP wanted to sort the files by date. So, the above code took the array of text data in @files, simply grabbed the date string as-is, and appended that onto the front of the original string, sorted, and then extracted the original string. At first glance the map after the sort (the encoding map) doesn't do enough work to sort properly since it doesn't appear to align the days when dealing with single digits. However, it works because the sprintf("%10s"...) right justifies the text which does align the dates.

Schwarzian Transformations

In the same thread, there are multiple versions of a Schwartzian Transformation. A Schwartzian Transformation generally creates an array of 'stuff'; part of that array is the original data, other parts are the pieces to sort against. And the call back sorting routine is specified. Probably the best ST answer given is this one by RazorbladeBidet.

This is slightly modified from the original to show the intent of the original author. The original had sprintf("%02d", $3), or against the 4 digit year, which ends up to not do anything. I believe he'd intended to use that on the date itself.

Here, he goes through the trouble to map the month abreviations to numbers via a generated hash, and makes sure to sort by year then month then day. Let's see how this works.

Again the map statement after the sort is executed first, and it looks pretty complicated. The split is matching the periods in the filenames, and is only keeping the 2nd part which is the date information. It then matches and returns the individual parts of the date in $1, $2 and $3 corresponding to the day, month, year. Then, an array of 4 items is created, the original string, the year, the month in numeric format, and finally the date with a leading zero if date is a single digit. This is done for every member of the @files array.

The resulting array of arrays is sent to the sort routine which uses the supplied call back function to compare first the year, then the numeric version of the month, and finally the date.

The map function before the sort then gets called to pull out only the original version of the data, which is then printed rather than stored.

Conclusion

Hopefully between the original node, and this subsequent analysis of the other transformations, this has given you enough understanding to use these powerful sorting techniques for your own needs.

You are using $1, $2, $3 without testing the success of the match. This means that you might possibly be getting the previous round's data, resulting in duplicated output. What you should be doing instead is skipping over the erroneous entries, or perhaps dieing. Just to be different from the previous node, let's do the die thing:

I agree with merlyn that it's better to try to grok the ST
before the GRT, which, after all, is a special case of the ST.
I'm not sure that the ST is always clearer, however.

Interesting that the example of sorting IP addresses is used here,
because that was the original scenario in which the GRT was first explored
(by Michal Rutka, Uri Guttman, Larry Rosler and myself).
We could probably say that it is a canonical example: The data itself is
very simple, the string representation cannot be (usefully) used for sorting,
and the transformation to and from a meaningfully sortable representation is
trivial.

When putting a smiley right before a closing parenthesis, do you:

Use two parentheses: (Like this: :) )
Use one parenthesis: (Like this: :)
Reverse direction of the smiley: (Like this: (: )
Use angle/square brackets instead of parentheses
Use C-style commenting to set the smiley off from the closing parenthesis
Make the smiley a dunce: (:>
I disapprove of emoticons
Other