Let $A$ be an $m\times n$-matrix and $B$ an $n \times m$-matrix over the same field. Consider the matrices $C=AB$ and $D=BA$. It is probably well known (and not difficult to show) that the only difference between the canonic rational forms of $C$ and $D$ are nilpotent blocks (blocks with minimal polynomial $x^k$). (In particular, these compensate the different dimensions of $C$ and $D$.)
I'm interested in the converse question:

Given an $m\times m$-matrix $C$ and an $n\times n$-matrix $D$, what are necessary and sufficient conditions that there exist matrices $A$ and $B$ such that $C=AB$ and $D=BA$?

One may assume without loss of generality that $C$ and $D$ are both nilpotent. I'm thinking of a characterization in terms of the Jordan normal forms of $C$ and $D$. These in turn are characterized by their block sizes. In fact, an equivalent version can be stated as a question on partitions:

Suppose $\lambda=(\lambda_1 \geq\lambda_2\geq \dotsc )$ and $\mu= (\mu_1 \geq \mu_2 \geq \dotsc)$ are partitions of the integers $m$ and $n$. When are there matrices $A$ and $B$ such that the blocks in the jordan normal forms of $AB$ and $BA$ belonging to the eigenvalue $0$ have sizes $\lambda_1, \dotsc$ and $\mu_1, \dotsc$, respectively?

These are not arbitrary, for example, the quotient of the minimal polynomials must be in $\{1, x^{\pm 1}\}$, meaning that $|\lambda_1-\mu_1| \leq 1$.
This problem seems so natural that I think it has been addressed somewhere (not in the linear algebra books I looked into, however), so in particular I would appreciate a reference.

EDIT: I have now seen that $|\lambda_i -\mu_i|\leq 1$ for all $i$ is sufficient (and this is easy, since we may assume $C$ and $D$ in Jordan form, and then reduce to the case of one Jordan block). I guess it's necessary, too. Does anyone know a reference for this? And are there any nontrivial mathematical applications of this situation?

4 Answers
4

"And are there any nontrivial mathematical applications of this situation?"

Yes, this is a very important construction in algebraic geometry and representation theory!

Algebraic geometry. The papers of Kraft and Procesi used this construction to analyze singularities of the closures of nilpotent orbits in the classical Lie algebras $\mathfrak{gl}_n, \mathfrak{sp}_{2n}, \mathfrak{o}_n.$ In particular, they proved that, in the case of $\mathfrak{gl}_n,$ these closures are normal varieties. Their proof is based on the relation
$$
\bar{\mathcal{O}}_{\lambda}=r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu}). \qquad (*)
$$
Here $\bar{\mathcal{O}}_{\lambda}$ is the closure of the conjugacy class of nipotent $n\times n$ matrices with partition $\lambda$ and $\bar{\mathcal{O}}'_{\mu}$ is the closure of the the conjugacy class of nipotent $m\times m$ matrices with partition $\mu,$ where $\mu_i=\operatorname{max}(\lambda_i-1,0)$; the diagram of $\mu$ is obtained from the diagram of $\lambda$ by removing the first column, so that $n-m=\lambda'_1.$ The maps $r$ and $\ell$ are
$$
r((A,B))=AB, \quad l((A,B))=BA
$$
in the notation of the question. The papers of Daskiewicz, Kraskiewicz and Przebinda considered a more general situation: starting with $m$ and $n$ and a partition $\mu$ of $m,$ they proved that the formula (*) holds for a certain partition $\lambda$ of $n$. In other words, the algebraic variety $r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu})$ is the closure of a single nilpotent orbit, without imposing additional assumptions on $m$ and $n;$ the proof involves careful combinatorial analysis, especially when $m>n.$

Representation theory. Without going into too much detail, this construction emerges in Roger Howe's theory of reductive dual pairs. The nilpotent orbits in question arise as the wave front sets or the associated varieties of representations of two classical groups occurring in the Howe duality correspondence. This is considered and exploited in various papers of J.-S. Li, T. Przebinda, P. Trapa, Nishiyama, Oshiai, and Taniguchi, and my own.

Thank you for this very interesting answer! Using your notation, my question can be formulated as what $r\circ l^{-1} ( \mathcal{O}_{\mu} )$ is, where $\mathcal{O}_{\mu}$ is the conjugacy class of nilpotent $m\times m $-matrices with partition $ \mu $. I'm (almost, I didn't check completely) sure now that it's those $\mathcal{O}_{\lambda}$ for which $|\lambda_i-\mu_i|\leq 1$ for all $i$. I guess this must be contained in the papers of Daszkiewicz et. al. you mention. I must confess, however, that I'm not familiar enough with the language used there to understand their results.
–
Frieder LadischMar 1 '11 at 12:07

By the way, reading your first point, I remembered that in Brieskorn's linear algebra(!) book (Lineare Algebra und analytische Geometrie, Band 2) there is some discussion of $\mathcal{O}_{\mu}$ and its closure (over the complex numbers with the usual topology of $\mathbb{C}^{m\times m}$). I don't have the book at hand now to see what exactly it's saying, however.
–
Frieder LadischMar 1 '11 at 12:16

Your criterion is basically correct, but needs to be stated with care: `$\mathcal{O}_\lambda\subseteq r\circ\ell^{-1}(\mathcal{O}'_\mu)$' if and only if, after extending the partitions $\lambda$ and $\mu$ by a number of zero parts, $|\lambda_i-\mu_{\pi(i)}|\leq 1$ for all $i$ and some permutation $\pi$ of the indices. This follows from the description of the nilpotent orbits of the pairs $(A,B)$ and their behavior under the maps $\ell$ and $r$: these orbits are enumerated by the "$ab$-diagrams", which consist of strings of alternating $a$s and $b$s ($a, b, ab, ba, aba, bab, abab$, etc).
–
Victor ProtsakMar 1 '11 at 20:27

The letter $a$ occurs $m$ times and the letter $n$ occurs $n$ times in the diagram; the map $\ell$ counts the $a$s in each string, discarding the $b$s, and the map $r$ counts the $b$s, discarding the $a$s. For example, $\ell(a, ab, bab, ababa)=(aaa, a, a, a)=(3,1,1,1)$ and $r(a, ab, bab, ababa)=(bb, bb, b)=(2,2,1,0);$ here $m=6, n=5.$ It seems plausible that the permutation $\pi$ is, in fact, not needed, but I'd need to think about it more carefully. By the way, the closures of $\mathcal{O}_{\lambda}$ in the usual topology and in the Zariski topology are the same.
–
Victor ProtsakMar 1 '11 at 20:42

By accident, I came accross a note in the American Mathematical Monthly (C. R. Johnson and E. A. Schreiner, The Relationship between AB and BA, vol. 103), where this is treated. It seems that this was first treated in the following article:

In fact, $|\lambda_i-\mu_i|\leq 1$ is the exact condition, and this is sometimes called "Flanders' theorem". This gives a keyword to search for on mathreviews etc., but probably this has been rediscovered more than once. (Note that the titles above don't contain very specicifc keywords.)

This reminds me of the notion of strong shift equivalence in symbolic dynamics. Here one works with nonnegative integral matrices. Consider the equivalence relation on the set of nonnegative integral square matrices generated by saying that $AB$ is equivalent to $BA$ where $A$ is an $m\times n$ nonnegative integral matrix and $B$ is an $n\times m$ nonnegative interal matrix. This equivalence relation is called strong shift equivalence. The problem of deciding whether two matrices are strong shift equivalent is open to the best of my knowledge. This problem is equivalent to the conjugacy problem for shifts of finite type.

I realize that the problem has a lot of subtleties, but, first, we can naively count the number of constraints vs. the number of unknowns. For the former we have $n^2+m^2$ which come from the equations $C=AB, D=BA$, whereas, the number of unknowns is $2mn$ as both $A$ and $B$ have $nm$ elements. So we have $n^2+m^2 \geq 2mn$ and the equality saturates only for m=n. Thus unless any special assumptions are taken on the matrices C and D the decomposition does not take place.

I know how to decide whether $C=AB$ has solutions when $C$ and $A$ are given, but that is not the question here. I'm interested in the case where quadratic matrices $C$ and $D$ are given, and I want to know how to decide whether the two equations $C=AB$ and $D=BA$ have a simultanous solution $(A, B)$. Most appealing would be a criterion in terms of the canonic rational forms of $C$ and $D$.
–
Frieder LadischFeb 25 '11 at 17:32

sorry for misunderstanding, but let's keep counting the number of constraints vs. the number of unknowns. For the former we have $n^2+m^2$ which come from the equations $C=AB, D=BA$, whereas, the number of unknowns is $2mn$ as both $A$ and $B$ have $nm$ elements. So we have $n^2+m^2 \geq 2mn$ and the equality saturates only for $m=n$. Thus unless any special assumptions are taken on the matrices $C$ and $D$ the decomposition does not take place.
–
PeterFeb 27 '11 at 3:42