I'm trying to figure out if there is a direct relationship between these concepts. Strictly from the definitions, they appear to be different concepts in general. The more I think about it, however, the more I think they are very similar.

Let $X,Y$ be WSS random vectors. The covariance, $C_{XY}$, is given by $$C_{XY}=E\left[(X-\mu_x)(Y-\mu_y)^H\right]$$ where $H$ stands for the Hermitian of the vector.

Let $Z$ be a WSS random vector. The autocorrelation function, $R_{XX}$, is given by $$R_{ZZ}(\tau)=E\left[\left(Z(n)-\mu_z\right)\left(Z(n+\tau)-\mu_z\right)^H\right]$$

Edit Note There is a correction to this definition as applied to signal-processing, see Matt's Answer below.

The covariance does not involve a concept of time, it assumes each element of the random vector is a different realization of some random generator. The autocorrelation assumes a random vector is the time evolution of some initial random generator. Yet in the end, they are both the same mathematical entity, a sequence of numbers. If you let $X=Y=Z$, then it appears $$C_{XY}=R_{ZZ}$$ Is there something more subtle that I am missing?

$\begingroup$If you look at $\tau$ as a variable, then the autocorrelation becomes a function of that "time gap" which can yield very interesting information about the data set. Look at the relation between autocorrelation, discrete Fourier transforms and the Wiener–Khinchin theorem.$\endgroup$
– PhilMacKayFeb 28 '17 at 16:36

$\begingroup$@PhilMacKay: Sure, but that only works for WSS processes. I gave the definitions for the general case, where processes are not necessarily stationary.$\endgroup$
– Matt L.Feb 28 '17 at 16:43

$\begingroup$Yes indeed non-stationary processes can be annoying for data analysis, which is why I always try to de-trend a data before using my beloved statistical tools! It's not always possible, though...$\endgroup$
– PhilMacKayFeb 28 '17 at 20:05

Notice how your definition of Autocorrelation includes an additional term $\tau$, which specifies an offset from the two sequences of number $Z(n)$ and $Z(n+\tau)$. In fact, the notation suggest that $R_{ZZ}(\tau)$ is a continuous function defined for any $\tau \in \mathbb{R}^+$, while $C_{XY}$ is a scalar.

As you mentioned, if you let $X=Y=Z$, then you are implying that $\tau = 0$, which is one special case of $R_{ZZ}(\tau)$.

In my personal experience (astrophysics, various sensor processing), the covariance was used as a coefficient to check the similarity of two datasets, while the autocorrelation was used to characterize the correlation distance, that is, how quickly a data evolves to become another data entirely.