4.5 False alarm and detection probabilities – Gaussian case

4.5.1 Statistical properties of the -statistic

We first present the false alarm and detection pdfs when the intrinsic parameters of the signal
are known. In this case the statistic is a quadratic form of the random variables that are
correlations of the data. As we assume that the noise in the data is Gaussian and the correlations
are linear functions of the data, is a quadratic form of the Gaussian random variables.
Consequently -statistic has a distribution related to the distribution. One can show (see
Section III B in [49]) that for the signal given by Equation (14), has a distribution with 4
degrees of freedom when the signal is absent and noncentral distribution with 4 degrees of
freedom and non-centrality parameter equal to signal-to-noise ratio when the signal is
present.

As a result the pdfs and of when the intrinsic parameters are known and when
respectively the signal is absent and present are given by

where is the number of degrees of freedom of distributions and is the modified
Bessel function of the first kind and order . The false alarm probability is the
probability that exceeds a certain threshold when there is no signal. In our case we have

The probability of detection is the probability that exceeds the threshold when the
signal-to-noise ratio is equal to :

The integral in the above formula can be expressed in terms of the generalized Marcum -function [94, 44],
. We see that when the noise in the detector is Gaussian and the intrinsic
parameters are known, the probability of detection of the signal depends on a single quantity: the optimal
signal-to-noise ratio .

4.5.2 False alarm probability

Next we return to the case when the intrinsic parameters are not known. Then the statistic
given by Equation (35) is a certain generalized multiparameter random process called the random field (see
Adler’s monograph [4] for a comprehensive discussion of random fields). If the vector has one
component the random field is simply a random process. For random fields we can define the
autocovariance function just in the same way as we define such a function for a random process:

where and are two values of the intrinsic parameter set, and is the expectation value when
the signal is absent. One can show that for the signal (14) the autocovariance function is given by

where

We have .

One can estimate the false alarm probability in the following way [50]. The autocovariance function
tends to zero as the displacement increases (it is maximal for ). Thus we can divide
the parameter space into elementary cells such that in each cell the autocovariance function is
appreciably different from zero. The realizations of the random field within a cell will be correlated
(dependent), whereas realizations of the random field within each cell and outside the cell are almost
uncorrelated (independent). Thus the number of cells covering the parameter space gives an estimate
of the number of independent realizations of the random field. The correlation hypersurface
is a closed surface defined by the requirement that at the boundary of the hypersurface the
correlation equals half of its maximum value. The elementary cell is defined by the equation

for at cell center and on cell boundary. To estimate the number of cells we perform the Taylor
expansion of the autocorrelation function up to the second-order terms:

As attains its maximum value when , we have

Let us introduce the symmetric matrix

Then the approximate equation for the elementary cell is given by

It is interesting to find a relation between the matrix and the Fisher matrix. One can show (see [60],
Appendix B) that the matrix is precisely equal to the reduced Fisher matrix given by
Equation (43).

Let be the number of the intrinsic parameters. If the components of the matrix are constant
(independent of the values of the parameters of the signal) the above equation is an equation for a
hyperellipse. The -dimensional Euclidean volume of the elementary cell defined by Equation (57)
equals

where denotes the Gamma function. We estimate the number of elementary cells by dividing the
total Euclidean volume of the -dimensional parameter space by the volume of the
elementary cell, i.e. we have

The components of the matrix are constant for the signal
when the phase is a linear function of the intrinsic parameters .

To estimate the number of cells in the case when the components of the matrix are not
constant, i.e. when they depend on the values of the parameters, we write Equation (59) as

This procedure can be thought of as interpreting the matrix as the metric on the parameter
space. This interpretation appeared for the first time in the context of gravitational-wave data
analysis in the work by Owen [74], where an analogous integral formula was proposed for the
number of templates needed to perform a search for gravitational-wave signals from coalescing
binaries.

The concept of number of cells was introduced in [50] and it is a generalization of the idea of an effective
number of samples introduced in [36] for the case of a coalescing binary signal.

We approximate the probability distribution of in each cell by the probability when the
parameters are known [in our case by probability given by Equation (46)]. The values of the statistic
in each cell can be considered as independent random variables. The probability that does not exceed
the threshold in a given cell is , where is given by Equation (48).
Consequently the probability that does not exceed the threshold in all the cells is
. The probability that exceeds in one or more cell is thus given by

This by definition is the false alarm probability when the phase parameters are unknown. The number of
false alarms is given by

A different approach to the calculation of the number of false alarms using the Euler characteristic of level
crossings of a random field is described in [49].

It was shown (see [29]) that for any finite and , Equation (61) provides an upper bound for
the false alarm probability. Also in [29] a tighter upper bound for the false alarm probability was derived by
modifying a formula obtained by Mohanty [68]. The formula amounts essentially to introducing a suitable
coefficient multiplying the number of cells .

4.5.3 Detection probability

When the signal is present a precise calculation of the pdf of is very difficult because the presence of
the signal makes the data random process non-stationary. As a first approximation we can estimate
the probability of detection of the signal when the parameters are unknown by the probability of detection
when the parameters of the signal are known [given by Equation (49)]. This approximation assumes that
when the signal is present the true values of the phase parameters fall within the cell where has
a maximum. This approximation will be the better the higher the signal-to-noise ratio
is.