Abstract: The map which takes a square matrix $A$ to its polytrope is piecewise linear. We show that cones of linearity of this map form a fan partition of $\{R}^{n \times n}$, whose face lattice is anti-isomorphic to the lattice of complete set of connected relations. This fan refines the non-fan partition of $\R^{n \times n}$ corresponding to cones of linearity of the eigenvector map. Our results answer open questions in a previous work with Sturmfels and lead to a new combinatorial classification of polytropes and tropical eigenspaces.