This is a post about a silly (and mostly pointless) optimization. To motivate this, consider the following problem which you see all the time in mesh processing:

Given a collection of triangles represented as ordered tuples of indices, find all topological duplicates.

By a topological duplicate, we mean that the two faces have the same vertices. Now you aren’t allowed to mutate the faces themselves (since orientation matters), but changing their order in the array is ok. We could also consider a related problem for edges in a graph, or for tetrahedra in mesh.

There are of course many easy ways to solve this, but in the end it basically boils down to finding some function or other for comparing faces up to permutation, and then using either sorting or hashing to check for duplicates. It is that comparison step that I want to talk about today, or in other words:

How do we check if two faces are the same?

In fact, what I really want to talk about is the following more general question where we let the lengths of our faces become arbitrarily large:

This follows the standard C/JavaScript convention for comparison operators, where it returns -1 if a comes before b, +1 if a comes after b and 0 if they are equal. For large enough sequences, this isn’t a bad solution. It takes time and is pretty straightforward to implement. However, for smaller sequences it is somewhat suboptimal. Most notably, it makes a copy of a and b, which introduces some overhead into the computation and in JavaScript may even trigger a garbage collection event (bad news). If we need to do this on a large mesh, it could slow things down a lot.

An obvious way to fix this would be to try inlining the sorting function for small values of n (which is all we really care about), but doing this yourself is pure punishment. Here is an `optimized` version for length 2 sets:

If you have any aesthetic sensibility at all, then that code probably makes you want to vomit. And that’s just for length two arrays! You really don’t want to see what the version for length 3 arrays looks like. But it is faster by a wide margin. I ran a benchmark to try comparing how fast these two approaches were at sorting an array of 1000 randomly generated tuples, and here are the results:

compareSimple: 5864 µs

compareBrutal: 404 µs

That is a pretty good speed up, but it would be nice if there was a prettier way to do this.

Symmetry

The starting point for our next approach is the following screwy idea: what if we could find a hash function for each face that was invariant under permutations? Or even better, if the function was injective up to permutations, then we could just use the `symmetrized hash` compare two sets. At first this may seem like a tall order, but if you know a little algebra then there is an obvious way to do this; namely you can use the (elementary) symmetric polynomials. Here is how they are defined:

Given a collection of numbers, the kth elementary symmetric polynomial, is the coefficient of in the polynomial:

For example, if , then the symmetric polynomials are just:

And for we get:

The neat thing about these functions is that they contain enough data to uniquely determine up to permutation. This is a consequence of the fundamental theorem for elementary symmetric polynomials, which basically says that these polynomials form a complete independent basis for the ring of all symmetric polynomials. Using this trick, we can formulate a simplified version of the sequence comparison for :

Running the sort-1000-items test against the simple method gives the following results:

compareSimple: 7637 µs

compare3Sym: 352 µs

Computing Symmetric Polynomials

This is all well and good, and it avoids making a copy of the arrays like we used in the basic sorting method. However, it is also not very efficient. If one were to compute the coefficients of a symmetric polynomial directly using the naive method we just wrote, then you would quickly end up with terms! That is because the number of terms in , and so the binomial theorem tells us that:

A slightly better way to compute them is to use the polynomial formula and apply the FOIL method. That is, we just expand the symmetric polynomials using multiplication. This dynamic programming algorithm speeds up the time complexity to . For example, here is an optimized version of the case:

For comparison, the first version of compare3 used 11 adds and 10 multiplies, while this new version only uses 9 adds and 6 multiplies, and also has the option to early out more often. This may seem like an improvement, but it turns out that in practice the difference isn’t so great. Based on my experiments, the reordered version ends up taking about the same amount of time overall, more or less:

compare3SymOpt: 356 µs

Which isn’t very good. Part of the reason for the discrepancy most likely has something to do with the way compare3Sym gets optimized. One possible explanation is that the expressions in compare3Sym might be easier to vectorize than those in compare3SymOpt, though I must admit this is pretty speculative.

But there is also a deeper question of can we do better than asumptotically? It turns out the answer is yes, and it requires the following observation:

Using a fast convolution algorithm, we can multiply two polynomials together in time. Combined with a basic divide and conquer strategy, this gives an algorithm for computing all the elementary symmetric functions. However, this is still worse than sorting the sequences! It feels like we ought to be able to do better, but further progress escapes me. I’ll pose the following question to you readers:

Question: Can we compute the n elementary symmetric polynomials in time or better?

Overflow

Now there is still a small problem with using symmetric polynomials for this purpose: namely integer overflow. If you have any indices in your tuple that go over 1000 then you are going to run into this problem once you start multiplying them together. Fortunately, we can fix this problem by just working in a ring large enough to contain all our elements. In languages with unsigned 32-bit integers, the natural choice is , and we get these operations for free just using ordinary arithmetic.

But life is not so simple in the weird world of JavaScript! It turns out for reasons that can charitably be described as “arbitrary” JavaScript does not support a proper integer data type, and so every number type in the language gets promoted to a double when it overflows whether you like it or not (mostly not). The net result: the above approach messes up. One way to fix this is to try applying a modulus operator at each step, but the results are pretty bad. Here are the timings for a modified version of compare2Sym that enforces bounds:

compare2Masked: 1750 µs

That’s more than a 5-fold increase in running time, all because we added a few innocent bit masks! How frustrating!

Semirings

The weirdness of JavaScript suggests that we need to find a better approach. But in the world of commutative rings, it doesn’t seem like there are any immediately good alternatives. And so we must cast the net wider. One interesting possibility is to extend our approach to include semirings. A semiring is basically a ring where we don’t require addition to be invertible, hence they are sometimes called “rigs” (which is short for a “ring without negatives”, get it? haha).

Just like a ring, a semiring is basically a set with a pair of operators that act as generalized addition and multiplication. You also have a pair of elements, which act as the additive and multiplicative identity. These things then have to satisfy a list of axioms which are directly analogous to those of the natural numbers (ie nonnegative integers). Here are a few examples of semirings, which you may or may not be familiar with:

In many ways, semirings are more fundamental than rings. After all, we learn to count stuff using the natural numbers long before we learn about integers. But they are also much more diverse, and some of our favorite definitions from ring theory don’t necessarily translate well. For those who are familiar with algebra, probably the most shocking thing is that the concept of an ideal in a ring does not really have a good analogue in the language of semirings, much like how monoids don’t really have any workable generalization of a normal subgroup.

Symmetric Polynomials in Semirings

However, for the modest purposes of this blog post, the most important thing is that we can define polynomials in a semiring (just like we do in a ring) and that we therefore have a good notion of an elementary symmetric polynomial. The way this works is pretty much the same as before:

Let be a semiring under ; then we have the symmetric functions. Then for two variables, we have the symmetric functions:

And for we get:

And so on on…

Rank Functions and (min,max)

Let’s look at what happens if we fix a particular semiring, say the (min,max) lattice. This is a semiring over the extended real line where:

Now, here is a neat puzzle:

Question: What are the elementary symmetric polynomials in this semiring?

Here is a hint:

And…

Give up? These are the rank functions!

Theorem: Over the min,max semiring, is the kth element of the sorted sequence

In other words, evaluating the symmetric polynomials over the min/max semiring is the same thing as sorting. It also suggests a more streamlined way to do the brutal inlining of a sort:

Slick! We went from 25 lines down to just 5. Unfortunately, this approach is a bit less efficient since it does the comparison between a and b twice, a fact which is reflected in the timing measurements:

The Tropical Alternative

Using rank functions to sort the elements turns out to be much simpler than doing a selection sort, and it is way faster than calling the native JS sort on small arrays while avoiding the overflow problems of integer symmetric functions. However, it is still quite a bit slower than the integer approach.

The final method which I will now describe (and the one which I believe to be best suited to the vagaries of JS) is to compute the symmetric functions over the (max,+), or tropical semiring. It is basically a semiring over the half-extended real line, with the following operators:

There is a cute story for why the tropical semiring has such a colorful name, which is that it was popularized at an algebraic geometry conference in Brazil. Of course people have known about (max,+) for quite some time before that and most of the pioneering research into it was done by the mathematician Victor P. Maslov in the 50s. The (max,+) semiring is actually quite interesting and plays an important role in the algebra of distance transforms, numerical optimization and the transition from quantum systems to classical systems.

This is because the (max,+) semiring works in many ways like the large scale limit of the complex numbers under addition and multiplication. For example, we all know that:

But did you also know that:

This basically a formalization of that time honored engineering philosophy that once your numbers get big enough, you can start to think about them on a log scale. If you do this systematically, then you eventually will end up doing arithmetic in the (max,+)-semiring. Masolv asserts that this is essentially what happens in quantum mechanics when we go from small isolated systems to very big things. A brief overview of this philosophy that has been translated to English can be found here:

But enough of that digression! Let’s apply (max,+) to the problem at hand of comparing sequences. If we expand the symmetric polynomials in the (max,+) world, here is what we get:

And for :

If you stare at this enough, I am sure you can spot the pattern:

Theorem: The elementary symmetric polynomials on the (max,+) semiring are the partial sums of the sorted sequence.

This means that if we want to compute the (max,+) symmetric polynomials, we can do it in by sorting and folding.

Working the (max,+) solves most of our woes about overflow, since adding numbers is much less likely to go beyond INT_MAX. However, we will tweak just one thing: instead of using max, we’ll flip it around and use min so that the values stay small. Both theoretically and practically, it doesn’t save much but it gives us a maybe a fraction of a bit of extra address space to use for indices. Here is an implementation for pairs:

24 Responses to Comparing Sequences Without Sorting

It’s true that [0, 1, 2] == [2, 0, 1], but we can’t say that say that [0, 1, 2] == [0, 2, 1], right?. This makes the problem of finding duplicate polygons a whole lot easier (and faster). I’m assuming you know this and therefor I’m asking:
Why do you want the ordering to be invariant under permutation? For finding duplicate polygons within a mesh you don’t want this condition.

How these generalize to semirings though is less clear, since it seems like you need to be able to subtract in order to get the `alternating` property to work, but the idea should work in integer arithmetic.

Semirings indeed sound problematic for use with alternating polynomials, but I could imagine using a complemented lattice. I haven’t checked whether this actually “works” (in the sense that it gives useful results), but the Vandermonde polynomial for n=2 would be something like min(x_1,x_2^C) (which is indeed “alternating” if one replaces negation by complementation). However, I don’t think the trick with “multiplying” with a symmetric polynomial works here, so a generalization would not be immediate.

Okay, firstly I don’t understand this post. Secondly, it’s pretty cool. Thirdly, I have a curious thought. A bad (because it wastes memory) but interesting alternative to sorting an array is to not sort the array but return a new array which carries the information of how the array should be sorted. For example, the array [‘a’, ‘c’, ‘b’] would result in the array [0, 2, 1]. So the comparison function would also have the bad alternative of a function which takes two arrays to compare, and two arrays showing how those arrays are sorted. I’m curious, and wonder is there a similar bad alternative for the comparison function which does not sort the lists?

Actually, the language J does that exact thing in sorting lists. The function “/:” when it is monadic, or takes a single argument, returns the sequence that shows the order of the elements. The dyadic “/:” is defined to be “(/: y ) { x “, where “{” is “nth”, “x” is the left argument and “y” is the left.
/: ‘edfbac’
4 3 5 1 0 2
/:~ ‘edfbac’
abcdef
“~” is a function modifier which takes the left argument and makes it both the left and right argument for the function.http://www.jsoftware.com/help/dictionary/d422.htm

Quick implementation note – consider adding | 0 to make addition and subtraction (but not multiplication) faster, since JS engines can recognize that the result is being casted to an int32 and remove overflow checks.

(max,+) isn’t really a tropical semiring. It’s the dual of a tropical semiring; namely, it’s the log-domain Viterbi semiring. The actual tropical semiring is (min,+) whereas the standard Viterbi semiring is (max,*). I discuss these and other alternatives over here.

I just started reading the Symmetry section, but if I understand this correctly, it seems that you have S(2,0) and S(2,1) reversed.

If n = 2, you get (x+a0)(x+a1) = S(2,0) + S(2,1)x + x^2
the first part becomes a0*a1 + (a0 + a1)x + x^2
so it looks to me like S(2,0) = a0*a1 and S(2,1) = (a0 + a1)
so I think you want to switch that in your post.

Thanks! I apologize for the nasty formatting, and I hope I understand the rest of your post. 🙂

Nice Read. How about this: make a list with a large enough number of primes, say primeList. Then a and b are equal iff primeList[a[0]] * primeList[a[1] * primeList[a[2]] == primeList[b[0]] * primeList[b[1] * primeList[b[2]]

Two unordered sequences A=(a1, .., an) and B=(b1,…,bn) are equal
iff the polynomial f(x)=(x-a1)(x-a2)…(x-an) and g(x) = (x-b1)..(x-bn) are equal.
iff f(x) and g(x) have the same coefficients w.r.t some basis (of vector space of polynomials)

The approaches explore different choice for +,* and different choice of basis.

For the question: Is there any method which can test if two unordered sequences are equal in linear time and at most log space?

f(x) = g(x) are equal as polynomials iff f(k)=g(k) for all k=1..n

If you don’t mind probabilistic algorithms, check if f(k) = g(k) for a random k in (1,..,n).
In practice if you expect f and g to be different, you can try a few values of k, and if they cannot distinguish between f and g, do the O(n log n) approach. I won’t be surprised if this has expected linear running time.

Fun post! Really makes me want to read more about the (max,+)-semiring.

This was a good idea: “The starting point for our next approach is the following screwy idea: what if we could find a hash function for each face that was invariant under permutations? Or even better, if the function was injective up to permutations […]”

We can build a hash like that out of ordinary hash functions in a familiar way: for each element in the original array, insert it as a key into a hash table with initial value 1, and increment the value each time that same element appears in the array. (It’s possible to build a hash table so that its natural order is invariant to permutions of insertion order, just by generating more hash bits to resolve collisions.) So if we had [2, 0, 2], we’d create the hash table {2: 2, 0: 1}.

Interpret a serialization of the hash table as a big int, and you’ve got a permutation-invariant hash (actually an injective function up to permutations) in I think amortized linear time.

Comparing two arrays this way does consume linear space worst-case, so it’s not quite an answer to your question, but maybe there’s some way to optimize from here. (Though my money is on linear time log space being impossible.)

Implementing this efficiently in js is maybe kind of a pain. Objects iterate in insertion order and even if you can get access to hash order somehow, the underlying hash table might not be invariant to insertion order. I think you’d need to write a careful hash table implementation in js.

I know this is years after the fact, but I found this article interesting. If I’m not mistaken, don’t strings of bits form a ring (if not a field) under bitwise-and, bitwise-xor? With this ring, there’d be no chance of overflow, and you could support non-arithmetic types. Though it still doesn’t solve the JavaScript problem.