One supplementary aspect of extraction deserves its own description and is covered in the the section that follows, namely, the various ways in which time selections can be made.

Finally, before extracting data from a given configuration, it's a good idea to review just what options the data format actually allows. PCA Standard 1 data, for example, come in only one spectral bin, so extracting a spectrum would only yield the mean count rate. The reduction options are described in the PCA Issues chapter, in the section on EDS configurations.

Creating GTI files based on filter files is covered in the Screening chapter. Here's a briefer, alternative example: a GTI file generated from a light curve containing the start and stop times corresponding to when the count rate in the light curve is between 10,000 and 20,000 counts per second. As in the case of filter files, the ftool maketime is used to generate the GTI file:

The times in the GTI file, which you can examine with the ftool fdump, are relative to the value of TSTART in the header. The GTI file itself is applied by the extractors in response to the question Input GTI file to be AND'd with INFILE.

After the example and explanatory notes comes a list of various "reduction scenarios" motivated by scientific priorities and accompanied by the corresponding saextrct inputs - scenarios such as "creating a light curve with 0.125-second bins from Binned Burst Catcher data in the 2-6 keV band."

Input file name: The name of a single input science array file or, when prefaced with "@", the name of an ASCII file containing a list of several science array files. In the above example, the file std2.list was generated by XDF and contains the lines:

At present, up to 100 files can be listed in this way. Please note that the list should end with a carriage return.

Input GTI files to be OR'd: Whether to apply the GTI information in the science array files themselves. For practical purposes, there are two alternative inputs to this prompt: "APPLY" (the default) instructs saextrct to apply the GTI in the GTI extensions of each of the input files. If more than one file is supplied, the files will
be OR'ed
together but are specific for each input file.
A blank or "-" instructs saextrct not to apply the GTI.

Input GTI file to be AND'd: The name of a FITS GTI file containing your selected good times, e.g. as constructed from a filter file using maketime. In this example, elv_gt_10.gti has GTI corresponding to when the elevation of the satellite was 10 degrees above the horizon. The current modus operandi of this type of file is:

More than one file may be entered by using "@" followed by the name of a list of up to 100 GTI files, one per line, ending with a carriage return. The times contained in these files will be AND'ed.

We recommend, however, if you want to combine more than one FITS GTI file, that you should use the ftool mgtime to merge the individual FITS GTI files into a single file.

If your time selections comprise simply a start time and an end time, you can enter them instead at the timemin and timemax prompts.

Root name for output file: Here, since "pca_std2" is entered, the extracted lightcurve and spectrum will be called "pca_std2.lc" and "pca_std2.pha", respectively.

Accumulate (ONE) or (MANY): You can decide whether you want the output lightcurve and spectrum to contain either one column of binned events (combined from data extracted from the input columns prompted for below) or many columns (one per input column). Our default of one column is the more usual, but the choice depends on what you want to do with downstream software. Both xronos and xspec can handle many-column files. But of course, many-column files occupy more disk space.

Name of TIME column:
Self-explanatory. But note that unlike some other ftools, saextrct is insensitive to case: even though the column is called Time, the default of TIME is acceptable.

Name of COLUMNS to be accumulated: Most science array files have more than one column containing valid science data (HEXTE Archive files, for example, have five). The default "GOOD", which we entered here, will cause saextrct to extract data from all the valid columns. If, instead, you want to extract data from a subset of columns, you can either list the columns at the prompt (using up to 80 characters) or input an ASCII file listing the columns, one per line, preceded with "@". For example, the list of columns in PCA Standard-2 files corresponding to the top layer of PCU0-2 comprises:

If you enter a binsize smaller than the intrinsic resolution of the data (of the first data file, to be precise), the binsize will be reset to that value. The intrinsic time resolution of data in, say, column N is given by the value of the TIMDEL keyword in the header of the XTE_SA extension - however, this value is superseded by theCDLTN keyword if present.

Binsizes and time stamps have a minimum time resolution of 0.00000001 seconds (0.01 microseconds). This is because 8 of the 16 available digits in double precision are taken up by the number of seconds in Mission Elapsed Time. As a result, binsizes cannot be smaller than 0.01 microseconds nor be specified with an accuracy requiring more than 8 digits after the decimal point.

Minimum acceptable fractional exposure: Allows the option for light curves of accumulating partially filled bins. By default, no partially filled bins are included in the output light curve. If, say, the minimum acceptable fractional exposure is set to 0.75, then all bins more than three quarters full will be included. Please note that if you do include partially filled bins, you should make sure that your analysis software, like Xronos, can interpret the FRACEXP column in the output light curve. Note, too, that this option is only active when the type of binning is "RATE".

Chose print option: Specifies whether to output a light curve, a spectrum or both (the default). Since light curve files can be large, you might want to specify "spectrum" on those occasions when you want only a spectrum.

Type of binning for LIGHTCURVE: This option determines how the binned values in light curves are calculated: "RATE" (the default) divides the total accumulated counts by the binsize; "SUM" simply gives the total accumulated counts in each bin; "MEAN" gives the mean value of counts in each bin, i.e. takes the average of the intrinsic bins in each time bin.

Type of binning for SPECTRUM: This option determines how the binned values in spectra are calculated: "SUM" (the default) gives the total accumulated counts per channel in each bin; "RATE" divides the total accumulated counts per channel by the temporal binsize; "MEAN" gives the mean value of counts per channel in each bin, i.e. takes the average of the intrinsic bins in each time bin.

Starting time for summation: Gives the option of setting a later time for the beginning of the extraction than the default - specified with "INDEF" - which is to start with the first valid time stamp (before GTI are applied, that is). Times must be given in absolute time, i.e. in seconds with an offset of TSTART + TIMEZERO, the values of which are in the header of the data file.

Ending time for summation: Gives the option of setting an earlier time for the end of the extraction than the default - specified with "INDEF" - which is to end with the last valid time stamp (before GTI are applied, that is). Times must be given in absolute time, i.e. in seconds with an offset of TSTART + TIMEZERO, the values of which are in the header of the data file.

If your time selections comprise simply a start time and an end time, you can enter them instead at the timemin and timemax prompts.

Minimum energy bin to include in Spectra: Allows the user to set the energy channel at which extraction begins. Please note that channels are specified here with respect to the full 256-channel PCA or 256-channel HEXTE pass bands. Be careful when dealing with input data files in which the channels are binned. For example, in PCA Standard-2 files, the first five channels
(0-4) are combined into one: setting the minimum energy bin to 3 would cause this first combined bin to be included in the extraction because it encompasses unbinned energy channel 3. To convert binned channels to unbinned channels, use the Perl script chantrans.

Although the prompt refers to spectra, this option can be used to restrict the pass band for a light curve.

Maximum energy bin to include in Spectra: Allows the user to set the energy channel at which extraction stops. Please note that channels are specified here with respect to the full 256-channel PCA or 256-channel HEXTE pass bands. Be careful when dealing with input data files in which the channels are binned. For example, in PCA Standard-2 files, the last six channels
(250-255) are combined into one: setting the minimum energy bin to 250 would cause this last combined bin to be included in the extraction because it encompasses unbinned energy channel 250. To convert binned channels to unbinned channels, use the Perl script chantrans.

Although the prompt refers to spectra, this option can be used to restrict the pass band for a light curve.

This option is best used for light curves, since it does not cause any rebinning of the output spectrum (cf. the prompt Input channels for each bin below). For example, specifying channels 5-25, 45-70 and 76-180 for PCA Standard 2 data would lead to the output of a spectrum with 129 channels (the resolution of the configuration) but with zero counts in those channels outside the specified ranges.

Channels are specified here with respect to the full 256-channel PCA or 256-channel HEXTE pass bands. Be careful when dealing with input data files in which the channels are binned. For example, in PCA Standard-2 files, channels 54-135 are binned by a factor of 2 while channels 136-237 are binned by a factor of 3: setting channel ranges 54-134 and 138-237 would cause data to be extracted from channels 54-135 and 136-237 because channel 134 cannot be separated from channel 135 and channel 138 cannot be separated from channels 136 and 137. To convert binned channels to unbinned channels, use the Perl script chantrans.

If entering channel ranges directly at the prompt, the format should be, e.g. 5-25, 45-50, 76-180

If entering channels in a file, the filename should be preceded by "@" and file format should be, e.g.

5 25
45 70
76 180

where the last character is a carriage return. The Perl script chantrans can be used to generate such a list.

Input channels for each bin: Allows the user to specify how input channels are to be binned into output channels. This option has two main practical uses:

Combining input data files with different channel binning.

Provided the input channels and output channels are compatible, it is possible to combine differently binned input files and extract a single light curve and/or spectrum.

Rebinning an output spectrum.

Provided the input channels and output channels are compatible, it is possible to produce an output spectrum with coarser binning than the input data. In effect, this is like running the ftool rbnpha.

In addition, please note

Channels are specified here with respect to the full 256-channel PCA or 256-channel HEXTE pass bands. Be careful when dealing with input data files in which the channels are binned, especially if your input files are binned differently. Make sure that the output bins can include all the valid input bins from all the files. To convert binned channels to unbinned channels, use the Perl script chantrans.

If entering channel ranges directly at the prompt, the format should be, e.g. 5-25, 45-50, 76-180.

If entering channels in a file, the filename should be preceded by "@" and file format should be either like that of chantrans output, e.g.

5 25
45 70
76 180

where the last character is a carriage return. Or it can be in the form of the CPIX keyword (without the CONTINUE keywords, e.g., for the SpecDet0 (the ninth) column in HEXTE Archive data files:

1CPIX9 = '0~63;2,64~127;4,128~255;8'

Perform a dryrun: A dry run differs from the real thing in two respects: the on-screen error and warning messages are more extensive, and no output files are produced. Obviously, a dry run is a good idea if you are unsure of what your inputs will produce or if you want to find out why it did not produce what you expected.

Verify the names of the columns containing the spectral data by running the ftool flcol on one of the Archive mode data files. The columns, one per detector, are:

SpecDet0
SpecDet1
SpecDet2
SpecDet3

which should be entered at the prompt:

Name of COLUMNS to be accumulated (GOOD):[GOOD] SpecDet0 SpecDet1
SpecDet2 SpecDet3

Run saextrct on the on-source files, calling the output spectrum, e.g., herx1_on.pha.

Run saextrct on the off-source files, calling the output spectrum, e.g., herx1_off.pha.

Adjust the exposure of the off-source spectrum. This is necessary because the time spent by the cluster moving between on-source and off-source positions (2 seconds) is not accounted for in the data files, i.e. the structure of the files implies that on-source and
off-source rows each contain 16 seconds of data. To make the adjustment:

First verify the cluster dwell time by running fdump on one of the original HEXTE archive mode files to see how often, in terms of the 16-s rows, the cluster changes position:

Here, we see that the cluster position changes from 0 (on-source for 1.5-degree rocking) to 1 or 5 (+1.5-degree & -1.5-degree offsets, respectively) every row, i.e. every 16 seconds. This means that each off-source row really contains 16 - 2 x 2 = 12 seconds of data, and, consequently, the total exposure for a spectrum extracted from
off-source data in this case is overestimated by a factor of 16/12 = 1.33.

Before making the exposure correction itself, first use the ftool fkeyprint (or fdump) to determine the current value of the exposure in the off-source spectrum:

As described in the Data Files chapter, science event files contain irregularly spaced
time-stamped event words which, in the case of the PCA, are interspersed at regular time intervals with various EDS flags (also known as clock events). This means that extracting light curves and spectra from an event file involves removing the clock events, and, if desired, picking a subset of the valid events, e.g. data from PCU0 only. In practice, this entails defining a bitmask: a binary string (or set of strings) which the event words in the file must match in order pass through extraction.

Seextrct applies but does not define bitmask selections, so another program, sefilter, must be run before the extraction. This caveat applies to:

All PCA and HEXTE data if you want to apply a particular bitmask, e.g. to pick out events from a single PCU or HEXTE detector.

This section is split into two. The first, immediately following, covers running sefilter. The second covers running seextrct. As for the corresponding section on saextrct, a fully worked example is given along with some data reduction scenarios.

Sefilter is a Perl script that helps you create an ASCII bitmask filter. The script then uses the bitmask to run either of the following:

fselect to create a single bitmask-filtered event file.

This is the only valid option if the bitmask includes alternatives, e.g. if you want to select events from either PCU0 anode X1L or PCU0 anode X1R (i.e. the top layer of PCU0). It may also be used if you want to create an intermediate filtered event file.

seextrct to create bitmask-filtered light curves and spectra.

Seextrct, unlike fselect, cannot implement a bitmask that contains alternatives. However, for simple bitmasks, e.g. events from PCU0 and PCU1, sefilter offers the choice of creating the filtered event file with fselect or of going straight to seextrct to create light curves and spectra.

An example of using sefilter follows, but please note the following:

Bitmask files are simple, containing only the coded selection criteria and no record of their origin. Moreover, neither fselect nor seextrct can check whether a given bitmask is sensible. Fortunately, sefilter contains built-in safeguards: after creating a valid bitmask, sefilter initiates either fselect or seextrct, whichever is appropriate, to apply the filter.

Bitmasks contain a prefix (b for binary, h for hex, o for octal - all RXTE bitmasks are binary) followed by a string of zeroes, ones or x's. The x is a wild card: for binary bitmasks it means "0 or 1". When you run sefilter, not selecting a particular token is equivalent to leaving all its values as x's. Practically, this means that if, say, you want events from all five PCU, you should not make a selection based on the D-token which would otherwise enable you to single out a particular PCU.

By default, and if no other selections are specified, sefilter creates a bitmask for PCA data which will select only good science events, i.e. those with the M[1]{1} marker or M-token. It is an ASCII file containing a line like

Event == b1xxxxxxxxxxxxxxx.

Although, for some configurations, channel information is contained in the bitmask, selection by channel is not performed via sefilter. Seextrct is used instead.

Example of using sefilter

Here, we'll use sefilter to generate a bitmask corresponding to the top layers of all the PCU. The input file, E_1ms_128M_0_8s.list is a list of data files in the E_1ms_128M_0_8s configuration, hence the @ sign. In this example, user input is in boldface.

rufus [41] [day] test: sefilter
Enter SE FITS file for filtering:[]@E_1ms_128M_0_8s.list
Enter the M-token to be processed (M[1]{1}):[M[1]{1}]
Enter the column name to be operated upon (Event):[Event]
Enter the output bitfile name to contain processed boolean expression:[]top.bit

Defaults are entered for the
M-token and column name. We've named the filter file itself top.bit.

After reading the datafile headers, sefilter prints on the screen the part of the TEVTB2 keyword describing the selectable parts of the template. For this particular configuration, Z-tokens enumerate the various combinations of PCU and anode. We'll be selecting the ones corresponding to the X1L and X1R anodes of PCU0-5.

The file we've elected to delete, outfile_368, contains the lines ( Z[] etc. It is not essential. The more important file, top.bit, contains the bitmask itself, as shown below (sebitmask is the actual ftool that creates it).