combine_spectra

Synopsis

Combine multiple PHA imaging spectra and associated responses

combine_spectra src_spectra outroot [src_arfs] [src_rmfs] [bkg_spectra]
[bkg_arfs] [bkg_rmfs] [method] [clobber] [verbose]

Description

The combine_spectra script sums multiple imaging source PHA spectra, and optionally, associated background PHA spectra and source and background ARF and RMF instrument responses. To run the script, it is only necessary to enter a list of PHA files into the 'src_spectra' parameter and specify a root name for output files in the 'outroot' parameter; in this case, background spectra and instrument response files will be combined based on information contained in the headers of the input source spectra. Please read the "Caveats" section below before conducting scientific analysis with the output of this script.

combine_spectra is a Python script which may be downloaded from the CIAO contributed scripts package, and executed on the Unix command line within the CIAO environment. The CIAO tools dmcopy, dmpaste, dmtcalc, dmhedit, dmappend, and addresp are invoked by combine_spectra.

combine_spectra calculates:

• a simple sum of input source PHA spectra to produce a single combined source spectrum;
• an exposure-weighted sum of input or located source ARF responses to produce a single combined source ARF;
• an ARF- and exposure-weighted sum of input or located source RMF responses to produce a single combined source RMF;
• an area- and exposure-weighted sum of input or located background PHA spectra to produce a single combined background spectrum;
• an exposure-weighted sum of input or located background ARF responses to produce a single combined background ARF;
• an ARF- and exposure-weighted sum of input or located background RMF responses to produce a single combined background RMF.

Counts in the individual input source spectra are simply summed, without weighting, to produce the output combined source spectrum. Background spectra are weighted before being summed in order to account for differences in exposure time and spectral extraction region area between each source and background spectrum. A combined source or background ARF instrument response output by the script is an exposure-weighted sum of the individual ARFs. Input RMF responses are weighted by the SPECRESP values and the exposure time of the corresponding ARFs, to create an exposure-weighted, ARF-weighted combined RMF.

When 'method=sum', combined spectra are output with the total PHA exposure time recorded in the header (the behaviour of the script in all previous versions); when 'method=mean' (new as of the CIAO 4.4 release), the mean exposure time is recorded. The output combined responses are appropriately adjusted for the current 'method' setting.

The required input to combine_spectra includes a list of at least two OGIP-compliant FITS PHA spectral data files, entered into the 'src_spectra' parameter, and a root name for the output combined file(s), specified in the 'outroot' parameter. If the remaining file input parameters are left at the default value of "None", the script will inspect the headers of the input source spectral files for associated ARF and RMF files to combine, as well as background PHA spectra and their associated instrument responses. Otherwise, the script will combine the files explicitly entered into the 'src_arfs', 'src_rmfs', 'bkg_spectra', 'bkg_arfs', and 'bkg_rmfs' fields. If the source response files and background spectral files recorded in the headers of the input source spectral files cannot be located - or the corresponding header keywords have null values - these files will not be combined by the script. At a minimum, the input source PHA spectra will be combined.

In order to combine background ARF files - either by explicitly listing filenames in the 'bkg_arfs' parameter or by having the code automatically locate the files when this parameter is left empty - a list of background spectra must be available for combining. This may be done by explicitly entering background spectra into the 'bkg_spectra' parameter, or by ensuring that the BACKFILE header keyword in all input source spectral files contains the appropriate filename. This also applies to the 'bkg_rmfs' parameter.

Format of Input Files

The combine_spectra script should be compatible with any OGIP-compliant PHA spectral file containing at least one FITS HDU (CXC Data Model block) associated with an HDUCLAS1 header key value equal to 'SPECTRUM', which contains CHANNEL, PI, and COUNTS columns. It is also required that EXPOSURE and BACKSCAL header keywords exist in all input spectra, and that any ARF or RMF files to be combined contain an EXPOSURE header keyword and having matching energy grids.

The script is incompatible with ARF files which do not contain an EXPOSURE header keyword; ARFs created with XMM SAS tools may cause the script to exit with an error. In this case, the appropriate exposure value should be added to the header of the ARF file.

RSP-style responses which have been premultiplied by an ARF are not supported.

Header Keywords of Output Files

By default, the sum of the exposure times of all input source spectra is recorded in the EXPOSURE header keyword of the combined source spectrum. The BACKSCAL value is set to 1.0.

By default, the sum of the exposure times of all input or located backgroud spectra is recorded in the EXPOSURE header keyword of the combined background spectrum. The BACKSCAL header keyword value is computed according to the formula in the section "Output Combined Background Spectrum".

The ANCRFILE and BACKFILE header keywords of the output combined source spectrum are updated with the names of the combined source ARF and combined background spectrum, if these files were created; will be "NONE" if not.

The ANCRFILE header keyword of the combined background spectrum is updated with the name of the combined background ARF, if this file was created; will be "NONE" if not.

The RESPFILE header keyword of the combined source spectrum is updated with the name of the combined source RMF, if this file was created; will be "NONE" if not.

The RESPFILE header keyword of the combined background spectrum is updated with the name of the combined background RMF, if this file was created; will be "NONE" if not.

By default, the EXPOSURE header keyword of the combined ARF file records the total exposure time of the input source PHA files.

Caveats

• Any grouping flags which may be present in input source or background PHA spectra will be ignored by the script.
• Combining background spectra with wildly varying spectral extraction region areas may yield misleading uncertainty estimates; i.e., some extractions will be over-represented while others will be under-represented.
• If the background rates contributing to a source are significantly different in the individual spectra to be combined, it is recommended that these spectra remain separate and be modeled simultaneously - otherwise, the modeling results of the combined source spectrum could be biased towards the observation(s) with the highest background rate(s).

Output Combined Source Spectrum

A combined PHA source spectrum output by the script consists of CHANNEL, PI, COUNTS, and COUNT_RATE columns contained within a SPECTRUM block, with the COUNTS column containing the total, unweighted sum of the counts of the individual input source spectra. The COUNT_RATE column is computed as the total summed counts divided by the total summed exposure time of the individual spectra. The total exposure time is recorded in the EXPOSURE header keyword of the combined source spectrum, and the BACKSCAL value is set to 1.0.

SOURCE_COUNTS = src_counts1 + src_counts2 + ... + src_countsN

SOURCE_EXPOSURE = src_exp1 + src_exp2 + ... + src_expN

Output Combined Background Spectrum

The counts in the individual background spectra to be combined are weighted before being summed in order to account for differences in exposure time and spectral extraction region area between each source and background spectrum. The scale factor, f_N, used for weighting each background spectrum prior to combining, is computed according to the following formula:

f_N = {(bkg_exp1 + ... + bkg_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]}*{(src_expN/bkg_expN)*(src_backscalN/bkg_backscalN)}

BGD_COUNTS = f_1*bkg_counts1 + f_2*bkg_counts2 + ... f_N*bkg_countsN

By default, rhe exposure times of the individual background spectra are summed and recorded in the EXPOSURE header keyword of the combined background spectrum. The BACKSCAL header keyword value is computed according to the following formula:

BGD_BACKSCAL = (src_exp1 + ... + src_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]

The filename of the combined background spectrum is recorded in the BACKFILE header keyword of the combined source spectrum.

Combining multiple source and background spectra according to the steps outlined above allows for simultaneous fitting of the combined source and combined background spectrum - or background subtraction - in a spectral fitting program such as Sherpa or XSpec. Refer to the HEASARC documentation for a detailed explanation of how to derive the formulae shown above.

Statistical Errors

Statistical errors on the combined counts in the combined source and background spectra are not computed by the script, as it is left to the X-ray fitting program chosen to fit the data to assign uncertainties to each spectral bin. When Sherpa, for example, fits a PHA data set, the statistical errors on the counts are automatically calculated using the chosen fit statistic. Users are warned against combining background spectra with wildly varying spectral extraction region areas, as this may yield misleading uncertainty estimates; i.e., some extractions will be over-represented while others will be under-represented.

Output Combined ARF and RMF Files

Source and background ARF and RMF files input to combine_spectra are combined using the CIAO tool addresp, which computes the exposure-weighted sum of the individual ARF SPECRESP values and writes them to a single output ARF file, and weights each input RMF by the SPECRESP and exposure time values of the corresponding ARF prior to combinining and writing to a single output RMF file; see the addresp ahelp file for details. The combined ARF and RMF files are compatible with a combined PHA spectrum containing the simple sum of the counts of its constituent PHA spectra, and either the summed or mean source exposure time (depending on the 'method' setting). Refer to the ACIS Extract documentation for a detailed explanation.

The filenames of combined ARF and RMF files are recorded in the ANCRFILE and RESPFILE header keywords of the corresponding combined spectrum (source or background), respectively.

Example 1

combine_spectra src_spectra=obs1843.pi,obs1842.pi outroot="combined"
method=mean

The counts columns in the PHA source imaging spectral files obs1843.pi and obs1842.pi are summed and written to the combined output spectrum 'combined_src.pi'. The source ARF files, RMF files, and associated background spectral files are read from the ANCRFILE, RESPFILE, BACKFILE keywords in the headers of the input source spectra and combined, producing the output files 'combined_src.arf', 'combined_src.rmf', and 'combined_bkg.pi'. If background spectra are located for each input source spectrum, their headers are searched for background ARF and RMF files to combine. If the ARF and RMF filenames recorded in the header of a background spectrum are identical to those recorded in the corresponding source spectrum header, the combined responses for the source are copied to the filenames 'combined_bkg.arf' and 'combined_bkg.rmf'. The mean PHA source exposure will be recorded in the output combined source spectrum and ARF.

Example 2

combine_spectra src_spectra=/data/pha1.pi,/data/pha2.pi
outroot="summed" bkg_spectra=pha1_bkg.pi,pha2_bkg.pi
bkg_arfs=@pha_arf.lis

The input source spectra 'pha1.pi' and 'pha2.pi' are combined to produce the summed counts spectrum 'summed_src.pi'. The script locates the associated source ARF and RMF files for these spectra and combines them, producing output files 'summed_src.arf' and 'summed_src.rmf'. Instead of having the script search for the associated background spectra and background ARFs to combine, they are explicitly entered into the 'bkg_spectra' and 'bkg_arfs' parameters.

Example 3

combine_spectra src_spectra=@pha3.lis outroot="combined" verbose=2

An ASCII file containing a list of Level=3 PHA spectral filenames is input to combine_spectra for combining. The script locates the associated source responses, background spectra, and background responses, and combines these files as well. The verbosity level is changed from the default value of 1 to 2, in order to print to the screen each step of the code as it is carried out.

Parameters

Detailed Parameter Descriptions

Parameter=src_spectra (file required filetype=input)

The list of filenames of source PHA imaging spectra to combine.

A comma-separated list of OGIP-compliant PHA spectral files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra PHA spectra with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script.

Parameter=outroot (string required filetype=input default=None)

The root name to use for output files.

The string entered into this parameter is prepended to the filenames of combined spectra and responses output by the script. The combined spectrum file is named '<outroot>_src.pi', the combined source ARF is named '<outroot>_src.arf', and so on.

Parameter=src_arfs (file not required default=None)

The list of filenames of source ARF files to combine.

A comma-separated list of ARF response files may be entered directly, or as a stack (see "ahelp stack" for more information). All files should have the same energy range and binning. Enter non-Chandra ARF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate source ARF files to combine by reading the ANCRFILE header keyword in each of the input source PHA spectra. If the ANCRFILE header keyword in each source spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=src_rmfs (file not required default=None)

The list of filenames of source RMF files to combine.

A comma-separated list of RMF response files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra RMF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate source RMF files by reading the RESPFILE header keyword in each of the input source PHA spectra. If the RESPFILE header keyword in each source spectrum is not null, and the referenced files exist, these files will be combined.

RSP-style responses which have been premultiplied by an ARF are not supported.

Parameter=bkg_spectra (file not required default=None)

The list of filenames of background PHA spectra to combine.

A comma-separated list of background PHA spectral files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra PHA files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background PHA files to combine by reading the BACKFILE header keyword in each of the input source PHA spectra. If the BACKFILE header keyword in each source spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=bkg_arfs (file not required default=None)

The list of filenames of background ARF files to combine.

A comma-separated list of ARF response files may be entered directly, or as a stack (see "ahelp stack" for more information). All files should have the same energy range and binning. Enter non-Chandra ARF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background ARF files to combine by reading the ANCRFILE header keyword in each of the input (or located) background PHA spectra. If the ANCRFILE header keyword in each background spectrum is not null, and the referenced files exist, these files will be combined.

Parameter=bkg_rmfs (file not required default=None)

The list of filenames of background RMF files to combine.

A comma-separated list of RMF response files may be entered directly, or as a stack (see "ahelp stack" for more information). Enter non-Chandra RMF response files with caution, as the expected header keywords, FITS HDUs, or FITS table column names may be incompatible with the CIAO tools invoked by the script. If this parameter is left null, the script will try to locate background RMF files by reading the RESPFILE header keyword in each of the input (or located) background PHA spectra. If the RESPFILE header keyword in each background spectrum is not null, and the referenced files exist, these files will be combined.

RSP-style responses which have been premultiplied by an ARF are not supported.

Parameter=method (string required default=sum)

The method by which to combine PHA exposures, either by summing or averaging.

The method parameter may be set to either 'sum' (default) or 'avg' to specify how the individual PHA exposure times should be combined for the output combined spectrum. When 'method=sum', combined spectra are output with the total PHA exposure time recorded in the header (the behaviour of the script in all previous versions); and when 'method=mean' (new as of the CIAO 4.4 release), the mean exposure time is recorded. The output combined responses are appropriately adjusted for the current 'method' setting.

Parameter=clobber (boolean not required default=no)

Overwrite existing output files?

Parameter=verbose (integer not required default=1 min=0 max=5)

Controls the amount of information printed to the screen.

Verbose can be from 0 to 5, generating different amounts of output. The default value is 1, which will print a minimal amount of information to the screen, e.g., which files have been located, which will be combined, and any warning or error messages which may occur from incompatibilities between the user input and the script. A verbosity level of 2 or higher will print each step of the script to the screen as it is being carried out, e.g., calls to the CIAO tools which are invoked by the script.

About Contributed Software

This script is not an official part of the CIAO release but is made available as "contributed" software via the CIAO scripts page. Please see the installation instructions page for help on installing the package.

Changes in the December 2010 Release

combine_spectra in CIAO 4.3 invokes the new tool addresp for combining ARF and RMF responses. Prior to this release, the script combined ARFs via the dmarfadd tool - with results equivalent to the output of addresp - and did not combine RMFs.

Changes in the April 2011 Release

The script prints the version number when verbosity is greater than 0.

Temporary output files are written to the 'outroot' directory instead of the current working directory, before being moved to the /tmp directory.

Changes in the December 2011 Release

New 'method' parameter for selecting between a summed and mean exposure value to be recorded in the header of output combined spectra and ARFs.

Bug fix: the ANCRFILE header keyword in an output combined background spectrum was incorrectly being set to "NONE" when the input backround response files exactly matched the input source responses.

Internal updates to the addresp step of the script to support the new addresp 'phafiles' and 'method' parameters.

Changes in the February 2012 Release

The script was updated to work around a CIAO 4.4 bug in the tool addresp, used for combining ARF and RMF responses: the output combined RMF file is missing the TLMIN header keyword for the F_CHAN column; combine_spectra now adds the TLMIN4 keyword to the combined RMF file.

The script now cleans up several temporary files which were being written to the outroot directory, instead of to /tmp/.

Bugs

There are no known bugs for this tool.

See Also

calibration

ardlib

psf

psf

tools

acis_bkgrnd_lookup, acis_fef_lookup, acis_set_ardlib, add_grating_orders, add_grating_spectra, addresp, asphist, dither_region, dmarfadd, dmextract, eff2evt, fluximage, fullgarf, hrc_bkgrnd_lookup, make_instmap_weights, mean_energy_map, merge_all, mkacisrmf, mkarf, mkexpmap, mkgarf, mkgrmf, mkinstmap, mkpsf, mkpsfmap, mkrmf, mkwarf, psextract, psf_project_ray, rmfimg, sky2tdet, specextract