4.7 Suboptimal filtering

To extract signals from the noise one very often uses filters that are not optimal. We may have to choose
an approximate, suboptimal filter because we do not know the exact form of the signal (this is almost
always the case in practice) or in order to reduce the computational cost and to simplify the analysis. The
most natural and simplest way to proceed is to use as our statistic the -statistic where the filters
are the approximate ones instead of the optimal ones matched to the signal. In general the
functions will be different from the functions used in optimal filtering,
and also the set of parameters will be different from the set of parameters in optimal
filters. We call this procedure the suboptimal filtering and we denote the suboptimal statistic by
.

We need a measure of how well a given suboptimal filter performs. To find such a measure we calculate
the expectation value of the suboptimal statistic. We get

where

Let us rewrite the expectation value in the following form,

where is the optimal signal-to-noise ratio. The expectation value reaches its maximum equal to
when the filter is perfectly matched to the signal. A natural measure of the performance of a
suboptimal filter is the quantity FF defined by

The fitting factor is the ratio of the maximal signal-to-noise ratio that can be achieved with suboptimal
filtering to the signal-to-noise ratio obtained when we use a perfectly matched, optimal filter. We note that
for the signal given by Equation (76), FF is independent of the value of the amplitude . For the
general signal with 4 constant amplitudes it follows from the Rayleigh principle that the fitting factor is the
maximum of the largest eigenvalue of the matrix over the intrinsic parameters of the
signal.

For the case of a signal of the form

where is a constant phase, the maximum over in Equation (77) can be obtained analytically.
Moreover, assuming that over the bandwidth of the signal the spectral density of the noise is constant and
that over the observation time oscillates rapidly, the fitting factor is approximately given by

In designing suboptimal filters one faces the issue of how small a fitting factor one can accept. A
popular rule of thumb is accepting . Assuming that the amplitude of the signal and
consequently the signal-to-noise ratio decreases inversely proportional to the distance from
the source this corresponds to 10% loss of the signals that would be detected by a matched
filter.

Proposals for good suboptimal (search) templates for the case of coalescing binaries are given in [26, 91]
and for the case spinning neutron stars in [49, 16].