XMM-Newton EPIC Background Analysis

Please note: These pages and the software and files within them are no
longer supported. A much more comprehensive study of the EPIC background
is being undertaken by the EPIC Background Working Group (BGWG), and these
pages, including newer software and files for the general user, are
available from
the official XMM-Newton site.
The user is strongly recommended to visit
this site.

XMM-Newton background Events files for the 3 EPIC instruments in their different instrument
mode/filter combinations have been constructed using a superposition of many pointed observations.
Background maps in several different instrument/mode/filter combinations and in several energy
bands have also been constructed. On these pages, details can be found on how to obtain these
background products together with related software and the paper on their construction and
usage.

Latest updates : -

Exposure maps for the thick filter events files are now available (8th Oct 2004)

Thick filter background events files are now available (4th Oct 2004)

XMM-Newton background subtraction

When performing detailed XMM-Newton EPIC analysis, a good knowledge of the background is required.
Sometimes it may be possible to extract the background from a region close to the particular source
one is interested in (using a so-called `local' background). For a large or extended source
however, one may have to extract the background far from the target source (the source may in fact
be so extended, that no local background is visible within the field of view). Here, a number of
effects can cause the extracted local background to be highly inappropriate in analysing the target
source.

The effective area of the mirrors is a function of off-axis angle. This vignetting effect can
be seen in
this figure, taken from the XMM-Newton
Users' Handbook. As the off-axis angle increases, less of the photons entering the telescope
reach the focal plane. The EPIC background however, is made up (primarily) of photons and
particles, and particles are not vignetted. One can see that, even at only 3 arcminutes off-axis
angle, a 10% difference is introduced between the change in effective area of the photon and
particle components.

Instrumental fluorescence, due to the interaction of high-energy particles with the detector
gives rise to strong features at particular energies, only seen in certain areas of each detector
(see the explanatory notes). Mismatch of source
and background regions can therefore introduce inappropriate background features.

The spectral response can depend on the position on the detector. Useful information on the
spectral responses can be found here
(taken from the EPIC Current Calibration File
pages). The change in redistribution matrix as a function of off-axis angle however, is thought
to predominantly affect low energies, and is quite small. The response of the PN is actually a
function of Y-position rather than angle - hence the different response files for each of 10
different distances from the readout node. For the MOS, it is currently thought that the response
has little or no dependence on off-axis angle.

If one believes that the use of a local background is impossible or will give rise to incorrect
results, then the alternative is to use background data from a different observation, or better
still, from several different observations. Hence the background products available here, which are
described in more detail below.

Available XMM-Newton background files

The Background Maps and Event files can be obtained from this ftp site.

Currently there are background events files available for the MOS1 and MOS2 in full-frame mode,
and for the PN in both full-frame and full-frame-extended mode. Thin, medium and thick filter
versions of these files are available for each of these instrument/mode combinations. Exposure maps
are also available for each of these cases. For the thin and medium filters a set of background
maps (photon maps and particle maps) are also available in a few predefined energy bands. In
general the events files should be of more use to most people -- The events files are needed for
spectral background subtraction, or for imaging analysis in a user selected energy band (this is
also the only option for imaging analysis with the thick filter).

In the generation of the background events files and maps, sources detected in the standard SSC
pipeline analysis of the constituent datasets were removed. Datasets with extended sources were
also avoided.

The various files available are named as described in the explanatory notes, and the naming is also recapped
below:

Event Files
Event files are named in the form: E1_"mode/filter"_"instrument".fits
where:

"mode/filter" is a 6 character code which gives the instrumental mode and filter for the file.
The first two digits refer to the MOS1, the second pair refer to the MOS2 and the final pair to the
PN. In each pair of digits the first character identifies the mode and the second identifies the
filter. The table below shows the meaning of the letters.

Thus the events file E1_0000ft_PN.fits is a pn event file obtained in full-frame mode and using
thin filter.
Thick filter versions of these events files were added on 04/10/04.

Event Files with exposure extensions
As of 10/06/03, event files also exist with the exposure extensions merged together into the files.
These files, named in the same manner as the above event files but with a leading "E2", are
substantially larger than the "E1" files, and are available from the same ftp site.
e.g. E2_00fh00_M2.fits is a MOS2 events file obtained in full-frame mode using the thick filter,
which has exposure extensions merged into the file.
These E2 files were last updated on 30/10/03. Thick filter versions were added on
04/10/04.

Exposure Maps
For each event file, 4" resolution exposure maps are also available. These are useful in evaluating
the degree of the source and bad pixel removal. They are named in a similar manner to the event
files: expmap_"mode/filter"_"instrument".fits
e.g. expmap_0000em_PN.fits is the exposure map for the PN events file in full-frame-extended mode
with the medium filter.
Also available are the equivalent exposure maps, but without the effects of source removal. These
are named in the form: expmap_noSRC_"mode/filter"_"instrument".fits
Exposure maps for the thick filter were added on 08/10/04.

Background Maps (Images)
Individual background maps are available for the thin and medium filter. These were produced by a
process of clipping and averaging over many datasets, as described in the explanatory notes. They are named in the
form: A1_"mode/filter"_"type""energy-band""instrument".fits
where:

"mode/filter" and "instrument" are the same as described in the Events file section above.

For example, A1_ft0000_cphim4M1.fits is a corrected photon background map in energy-band 4
(4500-7500 eV) for MOS1 in full-frame mode with thin filter.

Source lists
These files contain information regarding the times [START/STOP] and the positions of each of the removed
sources (from the original datasets). They are named in a similar manner to the other files
available on this web page: src1_"instrument"_"mode/filter".fits

Software available relating to background files

All related software can be obtained from the main
background ftp site. As described in the explanatory notes, the user will usually work in sky
coordinates (RA & Dec), not detector coordinates, and to a much finer scale than the 1
arcminute scale used in the production of the background maps. With this in mind, software has been
developed to rebin and reproject onto the sky any of the above background maps to the spatial scale
and sky position of a user-input image.

BGrebinimage2SKY is a shell script wrapper for the fortran routine BGrebinimage2SKY_interp. The
shell script (BGrebinimage2SKY) and the fortran executable (either BGrebinimage2SKY_interp_solaris
or BGrebinimage2SKY_interp_linux, depending on your system - this must be renamed to
`BGrebinimage2SKY_interp') can again be obtained from the main background ftp site.

Both files, BGrebinimage2SKY & BGrebinimage2SKY_interp, should be placed in the user's
directory (alternatively, one can edit the script so that it calls the fortran routine, wherever it
is placed). One may need to change the protection of the fortran routine so that it is executable.
Typing the name of the shell script wrapper BGrebinimage2SKY gives help on usage. See also the
explanatory notes.

There is also (as of 29/05/03) a shell script wrapper to rebin and reproject the 4" resolution
exposure maps. This is called BGrebinimage2SKY_4arcs, and can be obtained from the same ftp site. It uses the same BGrebinimage2SKY_interp
(29/05/03 +) routine. The script is experimental, and because of the larger nature of the images
involved, is very CPU-intensive, and may take some time.

There may also be some scripts of interest in the Background
Analysis Scripts web page on this site (In particular the skycast script, also discussed
briefly below).

Production of Background Maps and Event Files

(The explanatory notes referred to in other parts of this document point to a local copy of this paper)

For the thick filter files (which were added after this paper was published) information on the
original datasets making up these files can be found on the main ftp site in the file entitled 01_thick_filter_extra_notes.txt.
The information in this file complements the information found in tables 3 and 4 of the
main paper.

Using these Background Files

A brief introduction

Reading the paper about these files first is
probably a good idea, but a brief introduction to using these files is given below.

There are 2 sets of background products available: Background events files and background maps
(which are images). In general the events files should be of more use to most people. The
background images will be discussed first below.

Background maps (images)

The background images are 1 arcmin resolution maps in a number of different energy bands. These
images are exposure corrected for any removed sources and in units of
counts per second. The coordinate system used in these maps is detector coordinates (DETX/DETY).
Usually, of course, the user will want to work in sky coordinates, not detector coordinates, and
often the users data will be at a higher resolution than the 1 arcminute resolution of the
maps.

To convert one of the background maps to a file appropriate for your observation use the script
BGrebinimage2SKY (see the software section above). This program will rebin and
reproject the 1 arcmin resolution detector images to the spatial scale and sky position of a
user-input image. See the software section above and section 4.3 of the
paper for a few more details.

The final product of this process will be an appropriate background image (photon map, particle
map, or both combined) in the selected energy band, which is projected to the same resolution and
sky position of your data.

Background Events files

Matching the events file to your data

To reproject one of the background events files to match your dataset, the script, skycast,
can be used. This is available from, and described on the Background
Analysis Scripts web page on this site. Basically this script will cast an XMM EPIC background
dataset (or indeed any EPIC event dataset) onto the sky, at the position given by an input template
event dataset or image.

Once the background events file has been reprojected to match the sky position of your data, one
can extract either images or spectra from it. However, a quick look at any of the background events
files will show partial holes in the data where sources were removed from the original files making
up the combined background events file. To make any use of these backround files some allowance must
be made for this effect.

Generating an exposure corrected image from an events file

The available exposure maps can be used to correct the uneven exposure time across
an image extracted from the background events file.

NB - you may notice that for some of the exposure maps the peak values (units are seconds)
are somewhat above the total livetime values given in table 4 of the paper (particularly for the
PN). This is due to Out-of-Time Events (OoT) -- see the XMM-Newton
Users' Handbook for more details on OoT events. Basically, these OoT events are events recorded
while the ccd is being read out, so the total exposure time at any particular position is a
combination of the total live time plus a fraction of time in which OoT events may be recorded.

The exposure maps provided are in DETX/DETY coordinates and are of 4 arcsec resolution. One
possibility is that you could do all your work using this coordinate system and binning. Hence you
could use the exposure maps directly to exposure-correct any (DETX/DETY 4") image created from the
BG event files. This would be a bit restrictive though.

As listed above in the software section there is a shell script wrapper to
rebin and reproject the 4" resolution exposure maps. This is called BGrebinimage2SKY_4arcs. It uses
the same BGrebinimage2SKY_interp routine (as used by BGrebinimage2SKY). This script is still
slightly experimental, and because of the larger nature of the images involved, is very
CPU-intensive, and may take some time to run.

If any problems come up whilst using the BGrebinimage2SKY_4arcs script, another possibility is
to rebin the exposure map images to a 1 arcmin resolution image and then use the much more robust
BGrebinimage2SKY (described in the background images section above).

NB - BGrebinimage2SKY is currently hardwired to a 1 arcmin image resolution and
BGrebinimage2SKY_4arcs is hardwired to a 4 arcsecond resolution image. It should be possible to
write a generic, any resolution version, and the guts of it should be in these two scripts. It
doesn't exist yet though.

Whichever route is taken, the end results should be an exposure map which matches the sky
position and resolution of your dataset.

By this stage you should be able to take a background event dataset and reproject it to match
the sky coordinates of your observation dataset (using skycast). You should also be able to
generate an appropriate exposure map image for the background file (using BGrebinimage2SKY or
BGrebinimage2SKY_4arcs). This should cover what is needed for an imaging analysis.

NB - You will want to use the events file to generate images rather than the background maps
if you want to use an energy band which doesn't match any of the bands used for the generation of
the background maps, or if you want to generate any sort of image for the thick filter (background
maps don't exist for the thick filter).