It should be noted that "vector" here means "element of a vector space" which can include many mathematical entities. Ordinary vectors are elements of a vector space, and multiplication by a matrix is a linear transformation on them; smooth functions "are vectors", and many partial differential operators are linear transformations on the space of such functions; quantum-mechanical states "are vectors", and observables are linear transformations on the state space.

An important theorem says, roughly, that certain linear transformations have enough eigenvectors that they form a basis of the whole vector states. This is why Fourier analysis works, and why in quantum mechanics every state is a superposition of eigenstates of observables.