Let $X,Y,Z$ represent random variables and let $p_X(x)$ represent the probability density for the random variable $X$. Similarly, we write $p_{X,Y}(x,y)$ for the joint probability distribution of two random variables.

Let $f(x)$ be any function of $x$. Consider the random variable $Z$ defined by the equation

(1)

\begin{align} Z=f(X)\ . \end{align}

What is the probability density of the random variable $Z$ given the probability density of $X$? Let us pick a value $z$ and let $x_1,x_2,\ldots$ be the set of points that solve the equation $z=f(x)$. Thus, the probability density $p_Z(z)$ will be determined by the probability density of the random variable $X$ at $x_1,x_2,\ldots$. We need to add the different contributions with some weight that has to be determined. How do we do that? We claim that the following holds:

The above formula satisfies all the required conditions: (i) given a value $z$, it gets contributions only from the set of points $x_1,x_2,\ldots$ that solve the equation $z=f(x)$; (ii) it is suitably normalized. A simple way to prove the above is to first consider the case when the random variables take values in a discrete set. Then, one deals with probabilities rather than densities and the Dirac delta function is replaced by the Kronecker delta. Of course, the integral gets replaced by a sum.

Further simplification of the above formula can be carried out by using the following identity involving delta functions:

The most common function that is considered is the sum i.e., $z=x+y$. In such situations, one of the integrals in the above equation can be carried out to obtain the probability density as the convolution of the two probability densities.