Coadding Spectra and Responses
![[CXC Logo]](../../imgs/cxc-logo.gif){kind=logo}
CIAO 4.5 Science Threads
Overview
Synopsis:
The combine_spectra script sums multiple imaging source PHA spectra, and the associated background PHA spectra and ARF files.
Purpose:
To coadd imaging source and background spectra and the related response files for ACIS imaging observation or the 0th-order of a grating observation.
Software & Calibration Updates:
This thread requires the following updates to the standard CIAO 4.5 installation
- Scripts Package version 16 Feb 2012
Last Update: 3 Dec 2012 - Review for CIAO 4.5; no changes
Contents
- Getting Started
- Running the Script
- Fitting
- Caveats
- Parameter files:
- History
Getting Started
This thread assumes that you have already extracted spectra and responses by running specextract. The examples use spectra and responses from five observations of NGC 6300 (ObsIds 10289-10293).
Note that combine_spectra can also be run from specextract. When extracting multiple spectra, set combine=yes and the spectra and responses will be coadded.
About combine_spectra
combine_spectra runs the CIAO tools dmcopy, dmpaste, dmtcalc, dmhedit, dmappend, and addresp to create:
- a summed source spectrum
- an exposure-weighted source ARF
- an ARF- and exposure-weighted source RMF
- an area- and exposure-weighted background spectrum
- an exposure-weighted background ARF
- an ARF- and exposure-weighted background RMF
When method=sum (the default), the sum of the exposure times from the input source spectra is recorded in the EXPOSURE header keyword of the combined source spectrum. If method=average, the average exposure time is used. The source extraction region area (BACKSCAL header keyword) is set to 1.0.
When method=sum (the default), the sum of the exposure times from the input background spectra is recorded in the EXPOSURE header keyword of the combined background spectrum. If method=average, the average exposure time is used. The background BACKSCAL value is computed according to the following formula:
BGD_BACKSCAL = (src_exp1 + ... + src_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]
Further technical details about how the output files are available in the combine_spectra help file.
Running the Script
The thread shows two sample runs of combine_spectra: letting the script read the filenames from the headers of the spectra and providing a list of all the input files.
A. Reading from the Spectrum File Header
When a list of source spectra is the only input provided, the script attempts to read the associated background and response file names from the spectra headers.
In this example, the input files are provided as a comma-separated list:
unix% punlearn combine_spectra unix% pset combine_spectra src_spectra="10289.pi,10290.pi,10291.pi,10292.pi,10293.pi" unix% pset combine_spectra outroot=ngc6300
Running the script with verbose=1 shows which files are being combined:
unix% combine_spectra verbose=1 Source PHA files to combine; enter list or '@stack' (10289.pi,10290.pi,10291.pi,10292.pi,10293.pi): Root name for output files (ngc6300): Running: combine_spectra Version: 14 February 2012 WARNING: The grouping flags in spectrum '10289.pi' will be ignored. WARNING: The grouping flags in spectrum '10290.pi' will be ignored. WARNING: The grouping flags in spectrum '10291.pi' will be ignored. WARNING: The grouping flags in spectrum '10292.pi' will be ignored. WARNING: The grouping flags in spectrum '10293.pi' will be ignored. Source PHA files to be combined: 10289.pi 10290.pi 10291.pi 10292.pi 10293.pi Checking headers of source PHA files for ARF files to combine. Found ARF file 10289.arf Found ARF file 10290.arf Found ARF file 10291.arf Found ARF file 10292.arf Found ARF file 10293.arf Checking headers of source PHA files for RMF files to combine. Found RMF file 10289.rmf Found RMF file 10290.rmf Found RMF file 10291.rmf Found RMF file 10292.rmf Found RMF file 10293.rmf Checking headers of source PHA files for background files to combine. Found background file 10289_bg.pi Found background file 10290_bg.pi Found background file 10291_bg.pi Found background file 10292_bg.pi Found background file 10293_bg.pi Checking headers of background PHA files for ARF files to combine. WARNING: The ANCRFILE header keyword value in 10289_bg.pi is 'none'; no ARF files to combine. WARNING: The ANCRFILE header keyword value in 10290_bg.pi is 'none'; no ARF files to combine. WARNING: The ANCRFILE header keyword value in 10291_bg.pi is 'none'; no ARF files to combine. WARNING: The ANCRFILE header keyword value in 10292_bg.pi is 'none'; no ARF files to combine. WARNING: The ANCRFILE header keyword value in 10293_bg.pi is 'none'; no ARF files to combine. Wrote file ngc6300_src.pi. Wrote file ngc6300_bkg.pi. WARNING: The sum of the exposure values of the individual source ARFs is not equal to the total exposure of the source spectra being combined. The CIAO tool addresp() writes the total PHA exposure to the output combined ARF. Wrote files ngc6300_src.arf and ngc6300_src.rmf.
The warning indicates that there is a difference in the total EXPOSURE time in the input ARF and PHA file; users should take this into account in subsequent spectral analysis with the combined data files. The output files both have the total PHA exposure time in the header.
The background response filenames are not included in the header of the *_bg.pi files, so they are not added. The next example, Listing the Input Files, shows how to input these files.
Check the parameter file that was used with plist combine_spectra.
B. Listing the Input Files
If the file names are not available in the spectrum headers, or if you simply wish to list the files explicitly, each input parameter accepts a comma-separated list or a stack file.
In this run, a number of stack files are created. Each one contains the five filenames for that data product; for example:
unix% cat src.lis 10289.pi 10290.pi 10291.pi 10292.pi 10293.pi unix% cat src_arf.lis 10289.arf 10290.arf 10291.arf 10292.arf 10293.arf
The stack file is preceded by an "@" symbol so that the software parses the list of filenames:
unix% punlearn combine_spectra unix% pset combine_spectra src_spectra=@src.lis unix% pset combine_spectra src_arfs=@src_arf.lis unix% pset combine_spectra src_rmfs=@src_rmf.lis unix% pset combine_spectra bkg_spectra=@bg.lis unix% pset combine_spectra bkg_arfs=@bg_arf.lis unix% pset combine_spectra bkg_rmfs=@bg_rmf.lis unix% pset combine_spectra outroot=ngc6300_stacks
The script is run with verbose=1 again:
unix% combine_spectra verbose=1 Source PHA files to combine; enter list or '@stack' (@src.lis): Root name for output files (ngc6300_stacks): Running: combine_spectra Version: 14 February 2012 WARNING: The grouping flags in spectrum '10289.pi' will be ignored. WARNING: The grouping flags in spectrum '10290.pi' will be ignored. WARNING: The grouping flags in spectrum '10291.pi' will be ignored. WARNING: The grouping flags in spectrum '10292.pi' will be ignored. WARNING: The grouping flags in spectrum '10293.pi' will be ignored. Source PHA files to be combined: 10289.pi 10290.pi 10291.pi 10292.pi 10293.pi Source ARF files to be combined: 10289.arf 10290.arf 10291.arf 10292.arf 10293.arf Source RMF files to be combined: 10289.rmf 10290.rmf 10291.rmf 10292.rmf 10293.rmf Background PHA files to be combined: 10289_bg.pi 10290_bg.pi 10291_bg.pi 10292_bg.pi 10293_bg.pi Background ARF files to be combined: 10289_bg.arf 10290_bg.arf 10291_bg.arf 10292_bg.arf 10293_bg.arf Background RMF files to be combined: 10289_bg.rmf 10290_bg.rmf 10291_bg.rmf 10292_bg.rmf 10293_bg.rmf Wrote file ngc6300_stacks_src.pi. Wrote file ngc6300_stacks_bkg.pi. WARNING: The sum of the exposure values of the individual source ARFs is not equal to the total exposure of the source spectra being combined. The CIAO tool addresp() writes the total PHA exposure to the output combined ARF. Wrote files ngc6300_stacks_src.arf and ngc6300_stacks_src.rmf. Wrote files ngc6300_stacks_bkg.arf and ngc6300_stacks_bkg.rmf.
The warning indicates that there is a difference in the total EXPOSURE time in the input ARF and PHA file; users should take this into account in subsequent spectral analysis with the combined data files. The output files both have the total PHA exposure time in the header.
Check the parameter file that was used with plist combine_spectra.
Fitting
The RESPFILE and ANCRFILE header keywords have been updated in the combined spectrum file to point to the combine background and responses:
unix% dmlist ngc6300_src.pi header | grep FILE 0102 BACKFILE NONE String 0103 CORRFILE none String 0104 RESPFILE ngc6300_src.rmf String 0105 ANCRFILE ngc6300_src.arf String
If the background files are available, the BACKFILE keyword is also updated:
unix% dmlist ngc6300_stacks_src.pi header | grep FILE 0102 BACKFILE ngc6300_stacks_bkg.pi String 0103 CORRFILE none String 0104 RESPFILE ngc6300_stacks_src.rmf String 0105 ANCRFILE ngc6300_stacks_src.arf String
The header of the combined background file is also updated, when the responses are available:
unix% dmlist ngc6300_stacks_bkg.pi header | grep FILE 0104 BACKFILE none String 0105 CORRFILE none String 0106 RESPFILE ngc6300_stacks_bkg.rmf String 0107 ANCRFILE ngc6300_stacks_bkg.arf String
When the spectrum is read into Sherpa, the background file (when applicable) and source-related responses will automatically be read in as well.
To fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit it. See the Introduction to Fitting PHA Spectra thread for details.
If the background rates contributing to a source are significantly different in the individual spectra to be combined, it is recommended that these spectra remain separate and be modeled simultaneously - otherwise, the modeling results of the combined source spectrum could be biased towards the observation(s) with the highest background rate(s).
To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread.
Caveats
-
Any grouping flags which may be present in input source or background PHA spectra will be ignored by the script. This is indicated in the script output at verboset > 0:
WARNING: The grouping flags in spectrum '10289.pi' will be ignored.
The data can be grouped after being read into Sherpa, as shown in the Changing the grouping scheme of a data set thread.
-
Combining background spectra with wildly varying spectral extraction region areas may yield misleading uncertainty estimates; i.e., some extractions will be over-represented while others will be under-represented.
Parameters for /home/username/cxcds_param/combine_spectra.par
src_spectra = 10289.pi,10290.pi,10291.pi,10292.pi,10293.pi Source PHA files to combine; enter list or '@stack'
outroot = ngc6300 Root name for output files
(src_arfs = ) Source ARF files to combine; enter list or '@stack'
(src_rmfs = ) Source RMF files to combine; enter list or '@stack'
(bkg_spectra = ) Background PHA files to combine; enter list or '@stack'
(bkg_arfs = ) Background ARF files to combine; enter list or '@stack'
(bkg_rmfs = ) Background RMF files to combine; enter list or '@stack'
(method = sum) Sum or average PHA exposures?
(clobber = yes) OK to overwrite existing output file?
(verbose = 3) Debug Level(0-5)
(mode = ql)
Parameters for /home/username/cxcds_param/combine_spectra.par
src_spectra = @src.lis Source PHA files to combine; enter list or '@stack'
outroot = ngc6300_stacks Root name for output files
(src_arfs = @src_arf.lis) Source ARF files to combine; enter list or '@stack'
(src_rmfs = @src_rmf.lis) Source RMF files to combine; enter list or '@stack'
(bkg_spectra = @bg.lis) Background PHA files to combine; enter list or '@stack'
(bkg_arfs = @bg_arf.lis) Background ARF files to combine; enter list or '@stack'
(bkg_rmfs = @bg_rmf.lis) Background RMF files to combine; enter list or '@stack'
(method = sum) Sum or average PHA exposures?
(clobber = yes) OK to overwrite existing output file?
(verbose = 3) Debug Level(0-5)
(mode = ql)
History
| 13 Jan 2011 | new for CIAO 4.3: the thread uses the combine_spectra script in place of the old acisspec script |
| 25 Feb 2011 | updated for 25 Feb scripts package release: combine_spectra correctly handles duplicate input filenames, e.g. source spectra all having the same name, located in different directories. |
| 04 Apr 2011 | updated for 04 Apr scripts package release: combine_spectra temporary files are written to the 'outroot' directory before being moved to /tmp; script prints version at verbose > 0. |
| 20 Jul 2011 | required software updates are listed in Synopsis |
| 11 Jan 2012 | reviewed for CIAO 4.4: removed reference to psextract, which is no longer in CIAO; added warning about summed times being different in PI and ARF files |
| 16 Feb 2012 | a combine_spectra update was released in the 16 Feb 2012 scripts package: works around a bug in addresp (no TLMIN4 keyword in the output RMF file) so that combined RMF files can be used in Sherpa and X-Spec. |
| 03 Dec 2012 | Review for CIAO 4.5; no changes |
