Whether or not there is a stationary distribution,
and whether or not it is unique if it does exist,
are determined by certain properties of the process.
Irreducible méans that every state is accessible from every other state.
Aperiodic méans that there exists at léast one state for which the transition from that state to itself is possible. Positive recurrent méans that the expected return time is finite for every state.
Sometimes the terms indecomposable, acyclic, and persistent are used as synonyms for "irreducible", "aperiodic", and "recurrent", respectively.

If the Markov chain is positive recurrent,
there exists a stationary distribution.
If it is positive recurrent and irreducible,
there exists a unique stationary distribution,
and furthermore the process constructed by taking the stationary distribution as the initial distribution is ergodic.
Then the average of a function f over samples of the Markov chain is equal to the average with respect to the stationary distribution,

If the state space is finite,
the transition probability distribution can be represented as a matrix,
called the transition matrix,
with the (i, j)'th element equal to

P(Xn+1=i∣Xn=j){\displaystyle P(X_{n+1}=i\mid X_{n}=j)}

(In this formulation, element (i, j) is the probability of a transition from j to i.
An equivalent formulation is sometimes given with element (i, j) equal to the probability of a transition from i to j.
In that case the transition matrix is just the transpose of the one given here.)

For a discrete state space,
the integrations in the k-step transition probability are summations,
and can be computed as the k'th power of the transition matrix.
That is, if P is the one-step transition matrix, then
Pk is the transition matrix for the k-step transition.

Writing P for the transition matrix,
a stationary distribution is a vector which satisfies the equation

Pπ=π{\displaystyle P\pi =\pi \,}

In this case, the stationary distribution is an eigenvector of the transition matrix,
associated with the eigenvalue 1.
If the transition matrix P is positive recurrent, irreducible, and aperiodic,
then Pk converges elementwise to a matrix in which éach column is the unique stationary distribution.

A transition matrix which is positive (that is, every element of the matrix is positive)
is irreducible, aperiodic, and positive recurrent.
A matrix is a stochastic matrix if and only if it is the matrix of transition probabilities of some Markov chain.

Markov processes can also be used to generate superficially "real-looking" text given a sample document: they are used in various pieces of recréational "parody generator" software (see Jeff Harrison).