Emanuel Parzen [125] invented this approach in the early 1960s, providing a rigorous
mathematical analysis. Since then, it has found utility in a wide spectrum of areas and applications
such as pattern recognition [48], classification [48], image
registration [170], tracking, image segmentation [32], and image
restoration [9].

Parzen-window density estimation is essentially a data-interpolation
technique [48,171,156]. Given an instance of the random sample, ,
Parzen-windowing estimates the PDF from which the sample was derived. It essentially
superposes kernel functions placed at each observation or datum. In this way, each
observation contributes to the PDF estimate. There is another way to look at the estimation
process, and this is where it derives its name from. Suppose that we want to estimate the value of
the PDF at point . Then, we can place a window function at and determine how
many observations fall within our window or, rather, what is the contribution of each
observation to this window. The PDF value is then the sum total of the contributions
from the observations to this window. The Parzen-window estimate is defined as

(53)

where is the window function or kernel in the -dimensional space such that

(54)

and is the window width or bandwidth parameter that corresponds to the
width of the kernel. The bandwidth is typically chosen based on the number of available
observations . Typically, the kernel function is unimodal. It is also itself a PDF,
making it simple to guarantee that the estimated function satisfies the properties of a
PDF. The Gaussian PDF is a popular kernel for Parzen-window density estimation, being infinitely
differentiable and thereby lending the same property to the Parzen-window PDF estimate .
Using (2.53), the Parzen-window estimate with the Gaussian kernel becomes

(55)

where is the standard deviation of the Gaussian PDF along each dimension.
Figure 2.5 shows the Parzen-window PDF estimate, for a zero-mean unit-variance
Gaussian PDF, with a Gaussian kernel of and increasing sample sizes. Observe that
with a large sample size, the Parzen-window estimate comes quite close to the Gaussian PDF.

Figure 2.5:
The Parzen-window PDF estimate (dotted curve), for a Gaussian PDF (solid curve)
with zero mean and unit variance, with a Gaussian kernel of
and a sample size of
(a) 1,
(b) 10,
(c) 100, and
(d) 1000.
The circles indicate the observations in the sample.