(We discussed part I of “interlacing families” in this post about new Ramanujan graphs. Looks like a nice series.)

The Kadison-Singer Conjecture

The Kadison-Singer conjecture refers to a positive answer to a question posed by Kadison and Singer: “They asked ‘whether or not each pure state of is the extension of some pure state of some maximal abelian algebra’ (where is the collection of bounded linear transformations on a Hilbert space.”) I heard about this question in a different formulation known as the “Bourgain-Tzafriri conjecture” (I will state it below) and the paper addresses a related well known discrepancy formulation by Weaver. (See also Weaver’s comment on the appropriate “quantum” formulation of the conjecture.)

Update: A very nice blog post on the new result was written by Nikhil Srivastava on “Windows on theory.” It emphasizes the discrapancy-theoretic nature of the new result, and explains the application for partitioning graphs into expanders.

The Bourgain-Tzafriri theorem and conjecture

Jean Bourgain and Lior Tzafriri considered the following scenario: Let be a real number. Let be a matrix with norm 1 and with zeroes on the diagonal. An by principal minor is “good” if the norm of is less than .

Consider the following hypergraph :

The vertices correspond to indices . A set belongs to if the sub-matrix of is good.

Bourgain and Tzafriri showed that for every there is so that for every matrix we can find so that .

Moreover, they showed that for every nonnegative weights there is so that the sum of the weights in is at least times the total weight. In other words, (by LP duality,) the vertices of the hypergraph can be fractionally covered by edges.

The “big question” is if there a real number so that for every matrix can be covered by good sets. Or, in other words, if the vertices of can be covered by edges. This question is known to be equivalent to an old conjecture by Kadison and Singer (it is also known as the “paving conjecture”). In view of what was already proved by Bourgain and Tzafriri what is needed is to show that the covering number is bounded from above by a function of the fractional covering number. So if you wish, the Kadison-Singer conjecture had become a statement about bounded integrality gap. Before proving the full result, Marcus, Spielman and Srivastava gave a new proof of the Bourgain-Tzafriti theorem.

Great achievement and very clear exposition! Roughly, the main approach, which will certainly produce many more cool results and PROOFS, can be summarized as follows: associate with
a stability preserving operation OPER, possibly from some restricted class, a functional F(p) on polynomials such that there is
a bound F(OPER(p)) \geq Const(…)F(p) and iterate this bound over the products of those preservers.
The main problem to overcome: this functional F should be close(or better equal)
on “small” degree polynomials to the thing you are really interested in. If all this done then
an easy induction proof results. When I was dealing with the Van Der Waerden- Schrijver
like inequalities, the preservers are partial differentiations with zeroing and the best functional happened to be what I called the capacity. Actually,
the capacity is essentially a measure of stability. But I understood this fact after coming up with the
idea. Interestingly, the capacity was suggested by the (numerical) approach in STOC-2000 paper
with Alex Samorodnitsky(itself motivated by two Technion’s papers on the SCALING: Nemirovski-Rothblum and David London). The functional in MSS paper is what they call “Barrier Function”, in a way a “numerical’
thing as well. I envision many more algorithmic applications of the approach as well. An interesting
extension, barely touched, is to go beyond stable polynomials. For instance, the Mixed Discriminant
is related to a nice stable polynomial(the determinant) but the Mixed Volume is not. I did some
work in this direction in http://arxiv.org/abs/0812.3687(with CONJECTURES!) and http://arxiv.org/abs/cs/0702013.
Gil, your blog and https://www.facebook.com/ellina.gurvits?fref=ts are my main two WEB favorites.
Toda Roba!

It is worth point out that, not only did they solve the Weaver’s conjecture, but they solve it with optimal parameters (up to constants).

Of course, the fact that they have two sets in Corollary 1.3 (i.e. r=2) is optimal, and the bound of 1/2 + O(sqrt(alpha)) that they achieve in equation (5) is also optimal, as is shown in Section 3 of Weaver’s paper.

Gil, the statement of the original form of Kadison-Singer is a bit garbled. It should be: every pure state on an atomic maximal abelian subalgebra of B(H) has a unique extension to a pure state on B(H).

Also, +1 to Anonymous for the comment about optimal parameters. Indeed, this is the best we could have hoped for!

Actually my other comment isn’t quite accurate. The question they quote was in fact asked by Kadison and Singer, but it is not what we now know as the Kadison-Singer problem. (This other question was later sharpened by Anderson, and Akemann’s and my result falsifies both versions of the question.)

Tip my hat, again, to Adam,Dan and Nikhil! Let me stress why their proof looks so magically effortless and why some many experts failed before: the proof is not (random) matrix theoretic: matrices disappeared after the first application of the elementary first order differential operator,
but the proved Lax Conjecture(a hermitian solution due to Boris Dubrovin(1983) is sufficient)
allows to use (simple) matrix stuff to prove the needed convexity and monotonicity.
A similar observation applies to the comparison of A. Schrijver’s and mine proofs(no Lax though!,
but it was used in the earlier attempt http://arxiv.org/abs/math/0404474 and in STOC-2006 version).
On the other hand, in the context of the permanent, Lex proved much more in his paperhttp://homepages.cwi.nl/~lex/files/countpms2.pdf and it has been heavily used inhttp://arxiv.org/abs/1106.2844 to nail finally down Friedland’s conjecture on the monomer-dimer entropy(about the coefficients of the matching polynomial!).

I was thinking about possible extensions and generalizations. Of course, one natural question,
related to the description of maximizers is MSS paper, is to incorporate the ranks of A_i to
the (main) bound of Theorem 5.1. It looks like this body of work, starting with sparsifiers,
can be applied to an approximation of homogeneous real-stable polynomials, at least
multilinear ones, by polynomials with “few” variables. I noticed that the monotonicity/convexity
(Lemma 5.7) really needs just log-concavity, at least in one variable: take any log-concave function
F(t) that is
positive on [a, \infty) together with derivatives up to the third order. Then the ratio (F^{(1)}(t)/F(t))
is decreasing and convex on [a, \infty). With that in mind, I wonder about possible generalization
of the main (balancing) result of MSS to the Minkowski sums (aka zonotopes) of vector intervals \{a X_i: |a| \leq 1, X_i \in R^n\}. Of course, one will need to deal with mixed volumes instead of mixed discriminants,
the luxury of real roots and interlacing is gone, there are no known proved analogues of the Lax
conjectures…yet the Van der Waerden like bound was proved by an induction based
on the multiple partial differentiations.

Dear Gil, roughly something of the kind:
Consider m symmetric vector intervals and the Minkowski(volume) polynomial. Let be the maximum coordinate of the gradient of
the logarithm of the volume polynomial
at (1,…,1). Then there exists a partition (1,2,…,m) = such that
and .

Needless to say, it is a very preliminary statement. Can be easily rephrased in terms of
sums of support functions.

Dear Leonid, That’s very interesting! On another matter you might be interested in the recent added comments at the end of my old post on BosonSamplin. It looks that Gaussian permanents, and perhaps even general outcomes of Boson machines are very noise-sensitive in the sense of Benjamini, Schramm and me. It would be nice to prove a general theorem of this kind (for permanents it is easy), and relate it to classical simulability perhaps via your old results on Boson machines.

Dear Gil, thanks for the pointers on noise-sensitivities. I am not very familiar with this line of research. Yet, it the context of bosons I made this observation: quantum optical distributions
are much worse concentrated than their classical approximations. To be more precise:
consider a square complex $ n \times n$ unitary matrix $U$ and associate with it two homogeneous
polynomials with non-negative coefficients:
(quantum) Q(x_1,…,x_n) = Per(UDiag(x_1,…,x_n)U^{*});
(classical) C(x_1,…,x_n) = \prod_{1 \leq i \leq n} ( \sum_{1 \leq j \leq n} |U(i,j|^2 x_j).
The sum of coefficients of both of them is 1. So they both give some prob. distributions
on integer vectors \{(r_1,…,r_n): r_1+…+r_n = n \}. Moreover, both random vectors have the same
mean, namely $(1,1,…,1)$.

Now, the coefficients of Q satisfy the
upper bound $q_{r_1,…,r_n} \leq \prod_{1 \leq i \leq n} \frac{r_i !}{r_i^{r_i}};
and the coefficients of C satisfy much smaller upper bound
$c_{r_1,…,r_n} \leq \prod_{1 \leq i \leq n} \frac{1}{r_i^{r_i}}, which is due to the simple inequality
$C(x_1,…,x_n) \leq n^{-n}(x_1+….+x_n)^n$. I do conjecture that in general
$q_{r_1,…,r_n} \leq (\prod_{1 \leq i \leq n} r_i ! ) c_{r_1,…,r_n}$.

Both bounds are sharp. Also, the distribution of $C$ can be exactly efficiently classically simulated,
even though the individual probabilities are the permanents.

Actually, the polynomial $C$ is log-concave(even stable, like those in MSS paper) on the positive orthant and Q is not.

Charles Akemann and Nik Weaver note in their 2008 PNAS paper that confirmation of the Kadison-Singer conjecture implies (S1) and (S2) are equivalent. The statement (S1) is that ‘‘every pure state on $\mathcal{B}(H)$ restricts to a pure state on some atomic masa’’, and (S2) is Anderson’s conjecture that every pure state on $\mathcal{B}(H)$ is diagonalizable, that is, of the form $f(A) =lim_{U}\langle Ae_{n}, e_{n}\rangle$ for some orthonormal basis $(e_{n})$ and some ultrafilter $U$ over $\mathbf{N}$. Prior to showing that the Continuum Hypothesis (CH) refutes (S1), they remark: “It seems likely that the statement ‘‘every pure state on $\mathcal{B}(H)$ restricts to a pure state on some atomic masa’’ is also consistent with standard set theory.”

Has there been any further progress on whether (S1) (and hence now (S2)) is relatively consistent with ZFC, e.g. under a forcing axiom? Or has CH been eliminated or weakened?