Coadding Spectra and Weighted Responses
![[CXC Logo]](../../imgs/cxc-logo.gif){kind=logo}
CIAO 4.1 Science Threads
OverviewLast Update: 6 May 2009 - check the version of the CIAO scripts package instead of the individual script Synopsis: Users are encouraged to use specextract in place of acisspec for new analyses, noting that specextract does not yet offer the option to coadd spectra as acisspec does; refer to the specextract thread for more information. In this thread, acisspec is used to weight the spectra for their respective exposures and (optionally) group the output spectra. It also creates weighted responses (WARF and WRMF). Purpose: To describe two ways to run the acisspec script on a point source (the zero-order source of two HETG/ACIS-S observations):
Read this thread if: you are working with an ACIS imaging observation or the 0th-order of a grating observation and would like to perform one of the analyses described in the Purpose (above). |
Contents
- Getting Started
- Downloading acisspec
- Defining Source & Background Regions
- Calculating the Weights
- Running the Script
- Fitting
- Common Warnings
- Analysis Caveats
- Parameter files:
- History
- Images
Getting Started
Sample ObsIDs used: 9100 (ACIS-S, SNR 0104-72.3); 9810 (ACIS-S, SNR 0104-72.3)
File types needed: evt2
ACIS CTI_APP Keyword Required
CIAO 4.1 and CALDB 4.1 require that ACIS event files have a CTI_APP header keyword to indicate whether the CTI correction has been applied. The older CTI_CORR keyword is no longer used.
To check for CTI_APP:
unix% dmkeypar input.fits CTI_APP echo+ # dmkeypar (CIAO 4.1): ERROR: Keyword 'CTI_APP' was not found in file 'input.fits'.
If CTI_APP is not found, follow the instructions in the ACIS CTI_APP Keyword Required section of the ACIS CTI Correction why topic to add the keyword before continuing.
This thread may produce incorrect results without issuing an error if the keyword is missing.
Using Consistent Calibration: mkrmf vs mkacisrmf
The tool mkacisrmf is used to create RMFs for:
- all -120 ACIS data taken in (V)FAINT mode that has the time-dependent gain adjustment and CTI correction applied
- -120 ACIS GRADED mode data on the back-illuminated chips (ACIS-S1 and S3) only
- -110 ACIS data taken on the back-illuminated chips (ACIS-S1 and S3) only
All new analyses with these types of data should be done with mkacisrmf instead of mkrmf. Although this script runs mkrmf, it can still be used to create the spectrum and ARF files for the data. Then follow the Creating ACIS RMFs with mkacisrmf thread to generate new RMFs.
It is important that the calibration applied to the event file is consistent with the RMF tool chosen, as explained in the "Using Consistent Calibration" section of the why topic. If necessary, reprocess your data with the correct gain file before beginning this thread.
The ACIS dead area correction
The "dead area" is a fractional area loss per unit time due to cosmic ray flux incident on the ACIS detector. Calibration to account for this ACIS "dead area" was included in CALDB 3.3.0 on 15 December 2006. The correction is non-zero for the 8 front-illuminated (FI) ACIS chips; the effect is not detectable for the back-illuminated (BI) chips, so there is no correction applied to them.
Refer to the ACIS Dead Area Correction why topic for more information, including how to "turn off" the correction, e.g. if you would like to compare results with and without it applied.
When coadding files with acisspec it is not appropriate to apply the ACIS dead area correction. Since coadded data is provided as input to mkwarf, there is not an accurate single file to provide for the pbkfile parameter. Therefore, the correction is turned off in the examples in this thread.
Downloading acisspec
This thread uses the acisspec script, which is part of the CIAO Scripts distribution. The CIAO scripts package should be the following version or newer:
unix% cat $ASCDS_CONTRIB/VERSION.CIAO_scripts 17 Apr 2009
Please check that you have at least this version of the scripts package installed before continuing. If you do not have the scripts installed or need to update to a newer version, refer to the Scripts page.
Defining Source & Background Regions
First we need to define regions from which the source and background spectra will be created. In this example, the same source and background regions are used for both files:
unix% cat src.reg # Region file format: CIAO version 1.0 circle(4473.5,3611.5,40) unix% cat bgd.reg # Region file format: CIAO version 1.0 circle(4353.0001,3680.7499,20) circle(4343,3599.7499,20) circle(4422.5001,3737.2499,20) circle(4530.2501,3728.4999,20)
Make sure that you save the regions in CIAO format (Regions → File Format → CIAO) so that they are fully compatible with the analysis tools; see the Using CIAO Regions thread for more information.
The regions are shown displayed on the event files in Figure 1.
![[Thumbnail image: The two event files used for spectral extraction are displayed in ds9 with the source and background regions overlaid.]](regions.png)
[Version: full-size]
Figure 1: Extraction regions on the event files
ObsID 9100 is displayed in the top frame and ObsID 9810 is displayed in the bottom frame. Both event files have a white source region (src.reg) and green background regions (bgd.reg) overlaid.
Calculating the Weights
In order to apply a weighting factor to the spectra before coadding them, we need to calculate the ratio of each observation's exposure to the total exposure (sum of the two).
First, find the individual exposures:
unix% dmkeypar acisf09100N001_evt2.fits EXPOSURE echo+ 54466.269883243 unix% dmkeypar acisf09810N001_evt2.fits EXPOSURE echo+ 55815.36769867
Now calculate the total exposure and the ratios:
total exposure: 24527.46+24845.44 = 110281.64 weight1: 54466.27/110281.64 = 0.49 weight2: 55815.34/110281.64 = 0.51
Running the Script
The thread shows two sample runs of acisspec: one beginning at the event files and the other using spectra and WMAPs as input.
Beginning at the event files
This run of the script takes the original event files as input, creating and coadding the spectra and weighted responses. In this example, we opt to create both source and background spectra.
We need to set the input source and background files - including the regions with the appropriate DM filter - as well as the root and weight parameters. The pbkfile and dafile are set such that the ACIS dead area correction is not applied.
unix% punlearn acisspec unix% pset acisspec soufile1="acisf09100N001_evt2.fits[sky=region(src.reg)]" unix% pset acisspec soufile2="acisf09810N001_evt2.fits[sky=region(src.reg)]" unix% pset acisspec bgfile1="acisf09100N001_evt2.fits[sky=region(bgd.reg)]" unix% pset acisspec bgfile2="acisf09810N001_evt2.fits[sky=region(bgd.reg)]" unix% pset acisspec pbkfile="" dafile=NONE unix% pset acisspec root=events unix% pset acisspec weight1=0.49 weight2=0.51
Now execute the script:
unix% acisspec Primary source file: events or spectrum (acisf09100N001_evt2.fits[sky=region(src.reg)]): Root name for output files (events): NONE, or name of the parameter block file (): BTIMDRFT values are different...FAIL... BTIMNULL values are different...FAIL... BTIMRATE values are different...FAIL... warning: DS_IDENT has different value...Merged... warning: OBS_ID has different value...Merged... (messages are repeated for second file) # mkwarf (CIAO 4.1): WARNING: Input image name was "events_sou12.wmap" instead of "WMAP". Will attempt to use. # mkwarf (CIAO 4.1): WARNING: Input image name was "events_bgd12.wmap" instead of "WMAP". Will attempt to use. Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "events_sou12.wfef", is found. Total 5 regions to be processed: --- Region #1 processed --- Region #2 processed --- Region #3 processed --- Region #4 processed --- Region #5 processed Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "events_bgd12.wfef", is found. Total 13 regions to be processed: --- Region #1 processed --- Region #2 processed --- Region #3 processed ... --- Region #12 processed --- Region #13 processed
See the Common Warnings section for an explanation of the messages printed to the screen.
Check the parameter file that was used with plist acisspec.
There are a couple things to note about this run:
-
The default values of the binwmap (det=8) and ewmap (300:2000) were used.
-
The spectra were not grouped in this case. There are four parameters that can be used to specify grouping for the source (gtype, gspec) and background (bggtype, bggspec) spectra, if desired.
You may now skip ahead to the Using the Ouput section.
Spectra and WMAPs as input
This run of the script takes source spectra as input to create and coadd the weighted spectra. When using a spectrum for either soufile parameter, it is required that the corresponding WMAP (an image of the field in detector coordinates) be supplied. This can easily be made with dmcopy; for details on this process, see the Weighting ARFs and RMFs thread, specifically the Create the WMAP: Using dmcopy section.
In this example, we opt to create only source spectra, so background WMAPs are not needed:
unix% dmcopy "acisf09100N001_evt2.fits[sky=region(src.reg)][energy=300:2000][bin det=8]" \ src_1.wmap unix% dmcopy "acisf09810N001_evt2.fits[sky=region(src.reg)][energy=300:2000][bin det=8]" \ src_2.wmap
The input spectra were created with a basic dmextract command:
unix% dmextract "acisf09100N001_evt2.fits[sky=region(src.reg)][bin pi]" src_1.pi unix% dmextract "acisf09810N001_evt2.fits[sky=region(src.reg)][bin pi]" src_2.pi
We can now set the necessary parameters. Again, the pbkfile and dafile are set so that the ACIS dead area correction is not applied.
unix% punlearn acisspec unix% pset acisspec soufile1=src_1.pi soufile2=src_2.pi unix% pset acisspec souwmap1=src_1.wmap souwmap2=src_2.wmap unix% pset acisspec pbkfile="" dafile=NONE unix% pset acisspec root=spectra unix% pset acisspec weight1=0.49 weight2=0.51
And execute the script:
unix% acisspec Primary source file: events or spectrum (src_1.pi): Root name for output files (spectra): NONE, or name of the parameter block file (): BTIMDRFT values are different...FAIL... BTIMNULL values are different...FAIL... BTIMRATE values are different...FAIL... warning: DS_IDENT has different value...Merged... warning: OBS_ID has different value...Merged... # mkwarf (CIAO 4.1): WARNING: Input image name was "spectra_sou12.wmap" instead of "WMAP". Will attempt to use. Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "spectra_sou12.wfef", is found. Total 5 regions to be processed: --- Region #1 processed --- Region #2 processed --- Region #3 processed --- Region #4 processed --- Region #5 processed
See the Common Warnings section for an explanation of the messages printed to the screen.
Check the parameter file that was used with plist acisspec.
There are a couple things to note about this run:
Fitting
The BACKFILE, RESPFILE, and ANCRFILE header keywords have been updated in the source spectrum files:
unix% dmlist events_sou12.pi header | grep FILE 0110 BACKFILE events_bgd12.pi String 0111 CORRFILE none String 0112 RESPFILE events_sou12.wrmf String 0113 ANCRFILE events_sou12.warf String
Since no background spectrum was created for the second run, the BACKFILE keyword is unaffected:
unix% dmlist spectra_sou12.pi header |grep FILE 0110 BACKFILE none String 0111 CORRFILE none String 0112 RESPFILE spectra_sou12.wrmf String 0113 ANCRFILE spectra_sou12.warf String
When the source spectra are read into Sherpa, the background file (when applicable) and source-related responses will automatically be read in as well. Note that the RESPFILE and ANCRFILE header keywords are not updated in the background spectrum; to fit the background, the background response files will have to be loaded into Sherpa separately.
If you would like to fit the background-subtracted source spectrum using a common RMF and ARF for source and background, simply read the source spectrum FITS file into Sherpa, subtract the background, and fit it. See the Introduction to Fitting PHA Spectra thread (S-Lang or Python) for details.
To fit source and background spectra simultaneously with distinct RMFs and ARFs, follow the Independent Background Responses thread (S-Lang or Python).
Common Warnings
There are several warning messages that you may see printed to the screen during the acisspec run:
-
Originating tool: dmmerge
BTIMDRFT values are different...FAIL... BTIMNULL values are different...FAIL... BTIMRATE values are different...FAIL... omit - DEC_NOM values different more than 0.000300 warning: DETNAM has different value...Merged... warning: OBJECT has different value...Merged... warning: OBSERVER has different value...Merged... warning: OBS_ID has different value...Merged... omit - RA_NOM values different more than 0.000300 omit - ROLL_NOM values different more than 1.000000 warning: SEQ_NUM has different value...Merged... warning: TITLE has different value...Merged...
The messages are related to how the tool merges the header information in the input files. The merging_rules ahelp file explains the rules and how they affect the output file header.
-
Originating tool: mkwarf
# mkwarf (CIAO 4.1): WARNING: Input image name was "spectra_sou12.wmap" instead of "WMAP". Will attempt to use.
There are two ways to create a WMAP, with dmextract or with dmcopy; acisspec uses the dmcopy method. In this process, the primary block of the WMAP file is name EVENTS_IMAGE, as opposed to WMAP. Since we know that sources.wmap8 was created as a WMAP image (i.e. the detector coordinate system was used), the warning message can be ignored.
-
Originating tool: mkwarf
Couldn't determine chip position for pixel: (4076.500000,4052.500000) with value=3.000000. Skipping pixel
If the source is near a chip boundary, mkwarf may not be able to determine the chip location of every pixel in the WMAP. This has to do with the lack of SIM info in the image; details are available in this FAQ.
In the vast majority of cases, the number of counts ignored (i.e. the sum of the "value" of each ignored pixel) is much smaller than the total signal in the WMAP. However, this may not be the case for small regions near chip boundaries. dmstat may be used to determine the sum of all pixels in the WMAP for comparison to the sum of the ignore pixel values.
-
Originating tool: mkrmf
Atten: grid input,"energy=0.3:11.0:0.01", is ignored as a weights file, "es_1447_sou.wfef", is found.
The energy axis grid is ignored since the grid is read from the weights file. Since the value entered for the energy axis (axis1 in this case) must be syntactically correct for the tool to parse over it properly, acisspec uses "energy=0.3:11.0:0.0" as a placeholder.
Analysis Caveats
Users should be cautious about analyzing the data for sources near the edges of the ACIS CCDs.
-
For X-rays passing through the mirrors, the very bottom of each CCD is obscured by the frame store. As a result, some of the events in rows with CHIPY <= 8 are not detected. (The set of rows affected varies from CCD to CCD.) Since the CIAO tools do not compensate for this effect, the ARFs and exposure maps for sources in these regions may be inaccurate.
-
For sources within about thirty-two pixels of any edge of a CCD, the source may be dithered off the CCD during part of an observation. The aspect histogram, which is used to create ARFs and exposure maps, is designed to compensate for this effect.
-
An ARF calculated at the edge of a chip will not be accurate. The response tools for spectral extraction (specifically the ARF) assume that 100% of the PSF is enclosed - i.e. on the chip - all the time, which may not be the case. The amount of error introduced depends on how close the source is to the edge, the morphology of the source, and the characteristics of the PSF, which depends on the source spectrum.
-
A contaminant has accumulated on the optical-blocking filters of the ACIS detectors, as described in the ACIS QE Degradation why topic. Since there is a gradient in the temperature across the filters (the edges are colder), there is a gradient in the amount of material on the filters. (The contaminant is thicker at the edges.) Within about 100 pixels of the outer edges of the ACIS-I and ACIS-S arrays, the gradient is relatively steep. Therefore, the effective low-energy (' 1 keV) detection efficiency may vary within the dither pattern in this region. The ARF and instrument map tools are designed to read a calibration file which describes this spatial dependence.
Parameters for /home/username/cxcds_param/acisspec.par
soufile1 = acisf09100N001_evt2.fits[sky=region(src.reg)] Primary source file: events or spectrum
root = events Root name for output files
pbkfile = NONE, or name of the parameter block file
(bgfile1 = acisf09100N001_evt2.fits[sky=region(bgd.reg)]) Primary background file: events or spectrum
(soufile2 = acisf09810N001_evt2.fits[sky=region(src.reg)]) Secondary source file: events or spectrum
(bgfile2 = acisf09810N001_evt2.fits[sky=region(bgd.reg)]) Secondary background file: events or spectrum
(souwmap1 = ) Primary source WMAP
(bgwmap1 = ) Primary background WMAP
(souwmap2 = ) Secondary source WMAP
(bgwmap2 = ) Secondary background WMAP
(binwmap = det=8) Binning factor for WMAPs
(ewmap = 300:2000) Energy Range for WMAPs
(weight1 = 0.49) Weight for the primary source spectrum
(weight2 = 0.51) Weight for the secondary source spectrum
(gtype = ) Source spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
(gspec = ) Source spectrum grouping specs (NONE,10,etc)
(bggtype = ) Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
(bggspec = ) Background spectrum grouping specs (NONE,10,etc)
(dafile = NONE) CALDB, NONE, or name of ACIS dead-area calibration file
(clobber = no) Clobber existing file
(verbose = 0) Debug Level(0-5)
(mode = ql)
|
Parameters for /home/username/cxcds_param/acisspec.par
soufile1 = src_1.pi Primary source file: events or spectrum
root = spectra Root name for output files
pbkfile = NONE, or name of the parameter block file
(bgfile1 = ) Primary background file: events or spectrum
(soufile2 = src_2.pi) Secondary source file: events or spectrum
(bgfile2 = ) Secondary background file: events or spectrum
(souwmap1 = src_1.wmap) Primary source WMAP
(bgwmap1 = ) Primary background WMAP
(souwmap2 = src_2.wmap) Secondary source WMAP
(bgwmap2 = ) Secondary background WMAP
(binwmap = det=8) Binning factor for WMAPs
(ewmap = 300:2000) Energy Range for WMAPs
(weight1 = 0.49) Weight for the primary source spectrum
(weight2 = 0.51) Weight for the secondary source spectrum
(gtype = ) Source spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
(gspec = ) Source spectrum grouping specs (NONE,10,etc)
(bggtype = ) Background spectrum grouping type (NONE, BIN, SNR, NUM_BINS, NUM_CTS, or ADAPTIVE)
(bggspec = ) Background spectrum grouping specs (NONE,10,etc)
(dafile = NONE) CALDB, NONE, or name of ACIS dead-area calibration file
(clobber = no) Clobber existing file
(verbose = 0) Debug Level(0-5)
(mode = ql)
|
History
| 14 Dec 2004 | updated for CIAO 3.2: created Using Consistent Calibration section; can be run on 0th order grating in CIAO 3.2 |
| 25 Feb 2005 | acisspec updated to version 3.4: better error handling (see the script update history for a complete description of the changes) |
| 23 Jun 2005 | CIAO 3.2.2 patch: new calibration for mkacisrmf is available; see the Using Consistent Calibration: mkrmf vs mkacisrmf section |
| 16 Dec 2005 | updated for CIAO 3.3: the acisspec script has been updated to version 3.5; note about specextract in the Overview |
| 01 Feb 2006 | added link to specextract thread |
| 14 Feb 2006 | correction: weight1 was given in some places as 0.58, when it should be 0.59 |
| 16 Feb 2006 | acisspec updated to version 3.6: improved the conditional statement for recognizing weighted vs. unweighted cases. |
| 05 Apr 2006 | specextract use update added to Overview |
| 14 Jun 2006 | corrected link in "Calibration Updates"; clarified information on GRADED mode data |
| 01 Dec 2006 | updated for CIAO 3.4: acisspec updated to version 3.7 (removed "punlearn mkwarf" to allow new CIAO 3.4 mkwarf parameters (asolfile, pbkfile, dafile) to be set before running the script, if desired); CIAO version in warnings |
| 02 Feb 2007 | updated for CALDB 3.3.0.1 patch |
| 26 Feb 2007 | acisspec v4.0: Dead area correction parameters added to the acisspec.par file: pbkfile and dafile (added ACIS dead area correction section). Script prints a warning if RMF should be remade with mkacisrmf. |
| 23 Jan 2008 | updated for CIAO 4.0: acisspec v4.1 (parameter updates for applying ACIS dead area correction, which is turned on by default); links point to Sherpa Beta website; removed outdated calibration updates; filename and screen output updated for reprocessed data (version N002 event file for 2463) |
| 31 Mar 2008 | updated for CALDB 3.4.3: use mkacisrmf for -110 BI chips if TGAIN calibration has been applied |
| 03 Feb 2009 | updated for CIAO 4.1: thread rewritten with ObsIDs 9100 and 9810; image converted to inline; "ARDLIB warning ... Assuming the first "interesting" extension." no longer printed; input data must have a CTI_APP keyword |
| 19 Feb 2009 | renamed "Using the Output" section to "Fitting" and included additional links to Sherpa threads |
| 06 May 2009 | check the version of the CIAO scripts package instead of the individual script |
