|AHELP for CIAO 4.5||
Generate a weighted ARF for Chandra ACIS data
mkwarf infile outfile weightfile spectrumfile egridspec pbkfile [threshold] [feffile] [mskfile] [asolfile] [mirror] [detsubsysmod] [dafile] [ardlibpar] [geompar] [clobber] [verbose]
`mkwarf' creates a weighted ARF file based on an input weight map (WMAP). It also produces a set of weights that are used by `mkrmf' to create the associated weighted RMF. The use of a weighted response may allow one RMF and ARF to be used to analyze a spectrum extracted from a large area that covers multiple FEF regions (and hence a range of spectral responses).
Please note that the algorithm used by mkwarf and mkrmf for calculating the weighted responses assumes that there is no spatial variation in the source spectrum across the extraction region(s) used. In addition, it should only be used to create "PI" responses - ie for the analysis of spectra binned over the PI column and not the PHA column.
The WMAP should be created by running the tool sky2tdet. sky2tdet projects an image for a region of interest in sky(x,y) coordinates to TDET (tiled detector) coordinates. This WMAP properly accounts for chip edges and bad pixels in the response file. Users should no longer use the dmextract tool to create WMAPs for use with mkwarf.
Ideally, the response function for a given region should be weighted by the fraction of the incident flux from the source that falls within that region. However, this is generally what we are looking to obtain from the observation! We instead know the distribution of counts (i.e. the incident flux after it has passed through the telescope mirror and been absorbed by the detector) across the regions. Below we discuss two possible approaches to the problem; the choice of which - if either - is better depends on the characteristics of the data and the science objectives of the analysis.
The simplest approach is to use the raw counts to weight the response (in which the spectrumfile parameter should be set to none). This means that the weights will include the effect of the mirror response and detector QE (including bad pixels) - ie the effective area term - which may bias the spectral fits. One way to minimise this is to pick a restricted energy range over which to make the weight map, one over which the effective area (or, more correctly, the product of the source spectrum and the effective area) does not vary strongly.
Another approach is to use the spectrum of the source to correct the detected counts for the effective area of the telescope/instrument combination. Of course, we don't know the source spectrum to use to correct the model. One approach is therefore to use an iterative scheme, where the spectral model used to weight the WMAP is updated after each fit. The spectrumfile parameter is used to supply the an ASCII source model file to use in weighting the WMAP. Note that the energy range of the spectrum should be adjusted to match that used to create the WMAP.
Examples of creating weighted ARFs can be found in the CIAO science threads.
mkwarf "spectrum1.fits[wmap]" spectrum1.warf spectrum1.wgt none 0.5:9.5:0.02
This will create a weighted ARF, called spectrum1.warf, over the energy grid of 0.5 to 9.5 keV in 20 eV bins. The weighting uses the pixel values in the WMAP extension of the input PHA file (spectrum1.fits), with no spectral weighting.
The weights file created by mkwarf would then be used in mkrmf by setting:
unix% pset mkrmf weights=spectrum1.wgt unix% pset mkrmf infile=CALDB unix% pset mkrmf axis1="energy=0:1"
where the infile parameter is set to CALDB since we do not need to use acis_fef_lookup in this case, and the value for the energy grid is irrelevant (as long as something is supplied) since it's overridden by the energy grid in the weights file (here spectrum1.wgt).
mkwarf "spectrum2.fits[wmap]" spectrum2.warf spectrum2.wgt none 0.5:9.5:0.02 threshold=0.01
As with example 1, but this time only those FEF regions which contain at least 1 percent of the total counts are included in the calculation of the weighted ARF. This option can be used to speed up mkrmf by removing the need to calculate RMFs for regions which do not significantly contribute to the response. The choice of thresehold depends on both the distribution of counts across the WMAP and the science objectives of your analysis.
mkwarf "sources.pi[WMAP]" sources.warf sources.wgt egridspec=0.01:11:0.01 pbkfile=acisf063875928N002_pbk0.fits dafile=CALDB
Create a weighted ARF over the energy grid of 0.01 to 11 keV in 0.01 eV bins. The pbkfile and dafile parameters are set to apply the dead area correction.
mkwarf csmooth_out.fits spectrum3.warf spectrum3.wgt spec.txt "grid(spectrum1.rmf[MATRIX][cols ENERG_LO,ENERG_HI])"
This will use the (supposedly) smoothed image in 'csmooth_out.fits' as the WMAP with the spectral weights in the ASCII file 'spec.txt' to create the ARF and RMF weights on an energy grid that matches that of the RMF stored in spectrum1.rmf.
Parameter=infile (file required filetype=input)
Input WMAP file name. Be sure to include the DM filter "[WMAP]" if you are using a WMAP attached to a PHA file.
The input WMAP parameter name. The WMAP should be binned in DET coordinates and include the SIM positions in its header. If the data has been CTI corrected then the data must also contain the CTI_CORR keyword with a value of TRUE (CIAO tools will retain this information in the output file if the original event file contained the keyword, although it may be lost if non-CIAO tools have created/manipulated the image).
The sky2tdet tool should be used to create a WMAP in TDET coordinates.
Parameter=outfile (file required filetype=output)
The output file name for the weighted ARF file.
The weighted ARF created by mkwarf is OGIP compliant.
Parameter=weightfile (file required filetype=output)
The output file name for the weights file.
This file is needed by mkrmf to create the weighted RMF (the weights parameter). It contains a list of FEF regions used to create the weighted ARF, along with the weight used for each region.
Parameter=spectrumfile (file required filetype=input)
Input spectral weights file
If the raw counts in the input weight map are to be used to weight the spectral response then set this parameter to "none" (or left blank). Otherwise this parameter should be set to the name of an ASCII file containing columns for energy (in keV) and a weight value (such as the spectral model in photon/cm^2/s/keV).
The energy range of the spectrum should be adjusted to match that used to create the WMAP. Note that the weights column is normalized before being used (unlike the spectrumfile parameter for mkinstmap).
Parameter=egridspec (string required)
Output energy grid specification
This parameter defines the energy grid of the output ARF. It accepts the same syntax as the engrid parameter of mkarf.
The ACIS detector is calibrated over the range 0.224004 - 12 keV; choosing values outside this range may result in errors from the tool.
Parameter=pbkfile (string required filetype=input)
The parameter block file, which defines ACIS pixel clocking parameters, is a standard product of pipeline processing and is available for every observation. This file contains information which defines how long any pixel is exposed before being read-out, which is related to the probability that any pixel will be disabled ("deadened") by cosmic rays. See the description of the "dafile" parameter for more information on ACIS dead area.
Parameter block files have names of the form, "acisf146860615N001_pbk0.fits". The long string of digits refers to the time of observation (seconds since reference date) and "N001" is a revision number.
Since the pbkfile contents are observation-dependent, there is no default other than "NONE". In this case, no dead area correction is applied and the dafile should also be set to "NONE". If pbkfile is set to a valid file, then the dafile parameter must also refer to a calibration file.
The pbkfile parameter is ignored for HRC data.
Parameter=threshold (float default=0 min=0 max=1)
Fractional threshold cut for FEF regions
Any FEF region that has a weight (counts or spectrum weighted counts) below the fractional threshold of the total will be ignored from the calcuation of the ARF and will not go into the weights file. This can be used to exclude regions that only minially contribute to the over-all spectrum/flux and thus speed up the generation of the RMFs by mkrmf.
The final fractional contribution (in the 'FRACTION' column in the weight file output) can be less than the threshold parameter since the thresholding is done and then the fractions of the remaining regions are re-normalized.
The total weight/flux can be reduced by more than the threshold percent if, for example, there are many low counts regions and a single bright region. The low count regions individually could be less than the threshold of the total, but the sum of the flux/counts from these regions could be more (considerably more) than the threshold.
Parameter=feffile (file filetype=ARD default=CALDB)
The FEF file containing the calibration areas.
In most case this parameter should be left at its default value of CALDB. The name of the FEF file selected by the tool is output if the verbose value is set to 2 or greater and is also included in the header of the output files as the FEFFILE keyword.
The verbose=2 output when running mkwarf looks like:
Mapping response regions to FEF regions FEF File: /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD2000-01-29fef_pha_ctiN0001.fits Mapping response regions to FEF regions. Done
and the FEFFILE value can be found from the header using the dmkeypar tool:
unix% dmkeypar spectrum.wgt feffile echo+ /soft/ciao/CALDB/data/chandra/acis/fef_pha/acisD2000-01-29fef_pha_ctiN0002.fits
Parameter=mskfile (file default=)
The detector mask file
The mask file (msk1.fits) for the observation. The mask file is needed in particular when a window or subarray was used. A value of "NONE" indicates that no mask file will be applied.
More information on the mask file is available in the CIAO Dictionary.
Parameter=asolfile (file stacks=yes)
If the aspect solution file(s) - pcad*_asol1.fits - for the observation are provided, the average dy, dz, and dtheta values are computed and are used to adjust the SIM alignment. This reduces problems seen when the WMAP is at the edge of the chip, causing the mapping from detector to chip coordinates to fail.
Multiple aspect solution files may be provided as a comma-separated list or as a stack; see "ahelp stack" for more information.
Parameter=mirror (string default=HRMA)
The ARDLIB mirror parameter; can use additional qualifiers to change mirror response.
Parameter=detsubsysmod (string default=)
A modifier that is added to the detector name sent into ARDLIB.
Unlike some other ARDLIB enabled tools; mkwarf runs on multiple chips and as such does not have a detsubsys parameter. This parameter allows one to modify the internal detsubsys value to allow the response product to be modified. Things such as detector QE/U can be overridden (see "ahelp ardlib" for more information).
Parameter=dafile (string filetype=input default=CALDB)
ACIS "dead area" coefficients file, which may have the values "NONE" (no dead area computation), CALDB (for automatic lookup), or an explicit file reference to an ACIS "dead area" coefficients FITS table.
The ACIS dead area refers to a slight decrease in detector efficiency due to the background cosmic ray flux which temporarily renders some pixels useless. The calibration product is a coefficient (per CCD) which gives the fractional area lost (or "deadened") per second. Since the area lost increases with time, the magnitude of the effect depends upon the ACIS clocking parameters (number of rows, window location, frame-time) which determine how long a pixel was exposed to the background cosmic ray flux during the primary exposure and during electronic readout from the frame-store area. For full-frame timed-exposure, the dead area is about 4% at maximum CHIPY and about 2% at the readout. It is smaller for subarrays.
The ACIS clocking parameters required to scale the coefficients in the dafile are contained in the observation-specific parameter block file, which can be set by the associated parameter of this tool, "pbkfile".
The dafile parameter is ignored for HRC data.
Parameter=ardlibpar (file default=ardlib)
The name of the ARDLIB parameter file.
Parameter=geompar (file default=geom)
The name of the Pixlib Geometry parameter file.
Parameter=clobber (boolean default=no)
Overwrite output files if they already exist?
Parameter=verbose (integer default=0 min=0 max=5)
Controls amount of debugging output is sent to screen.
The ACIS chips are split up into 32 by 32 pixel regions - referred to as chip regions below - and the WMAP examined to see how many counts fall into each of these regions. Since the mapping uses the center of each WMAP pixel, the WMAP should be binned at a scale smaller than 32 pixels; we suggest a value of "det=8". Using a much smaller binning value than this will lead to larger files (the WMAP) and longer processing times. Note that these chip regions may be smaller than the FEF tile size for the chip (the size of a region over which the spectral response is assumed to be constant).
The conversion from detector to chip coordinate systems requires knowledge of the SIM position. The value stored in the WMAP header - the SIM_X/Y/Z keywords - corresponds to the average value during the observation; although the change in SIM position during an observation is normally small, it is possible that a small fraction of WMAP events at the edges of the chips are not mapped onto any chip region. In these situations mkwarf ignores these counts, and prints out a warning message giving the location and number of counts. Note that the coordinate conversion is made using the center of each chip region.
If a spectrumfile is supplied, then the counts detected in each chip region are weighted by the expected number of events given the the input spectral model and the ARF for the region.
Once all the chip regions have been analysed, the fractional contribution of each region is calculated - this is referred to as the region weight below. If the threshold parameter is not set to 0, then those regions with weights below the threshold are removed from the list and the weights re-calculated (this pass happens only once).
An ARF is calculated for each chip region using the energy grid specified by the egridspec parameter and multiplied by the weight for the region. The chip regions are then accumulated up into FEF regions (i.e. the weights for all the chip regions within a single FEF region are added), and the resultant set of weights is output as the weights file. The weighted ARF is calculated as the sum of these weights.
- The algorithm assumes that the source spectrum is spatially uniform, so that only the normalisation of the spectrum varies with position.
- The algorithm is designed to represent the weighted ARF over a large region. If the exposure varies strongly over most of the extraction region then the ARF may not fully account for this variation. This is mainly of concern for regions at the edge of an ACIS chip.
- The weights file must only be used with mkrmf to create a PI RMF; PHA RMFs are not supported.
- Response regions are currenly fixed to be 32x32 pixel regions on each ACIS chip.
Bug Fix: the tool only opens grating modules once, decreasing the run time and preventing 1000's of history records from being written in the header.
The tool has been upgraded to support CALDB4. The changes should be transparent to the user.
- Issue with the WMAP in DET coordinates when the source is at a chip edge
The preferred WMAP will be in TDET coordinates so this will not be a problem.
If the observation has large dy & dz offsets in the aspect solution file and they are quite variable during the observation, the tool will fail with a CALDB error. The large (and varying) offsets cause the mapping from DET to CHIP coordinates to fail and the tool cannot determine which response calibration file to use in creating the RMF.
- # mkwarf (CIAO): ERROR: Could not map response region to FEF region (04 Jun 2008)
This issue is often related to a mkwarf change in CIAO 3.4: using single pixel resolution when evaluating the wmap. The edge of the wmap (in detector coordinates) ends up getting mapped to chip coordinates at 1024.5 which is rounded to 1025. It then tries to find the FEF region that converts chipy=1025 (for which there are none) and thus fails.
Supply the aspect solution in the asolfile parameter when running mkwarf. The aspect shifts the coordinate transform back to where the pixels all map to good chip coordinates.
- ERROR: Max egridspec energy=10 above max FEF energy=9.886
mkwarf is required to compute and write a "weightfile" output file which contains FEF regions for use by mkrmf. If the energy range in the input RMF is greater than that in the FEF files, you get an error like:
ERROR: Max egridspec energy=10 above max FEF energy=9.886
Although the comparison to the FEF files is unnecessary in this case, there is currently no way to turn it off (e.g. set the weightfile to "NONE").
In order to avoid the error, it is necessary to define an energy range for mkacisrmf that falls within the boundaries of the FEF files, i.e. approximately 0.28 - 9.8 keV.
The mkwarf tool may run out of memory for large regions.
# mkwarf (CIAO): dsALLOCERR -- ERROR: Could not allocate memory.
If a weighted ARF is being created for a large region, on order of a full chip (or larger), mkwarf may hit the intrinsic memory limit of a 32-bit application.
If you encounter this bug, increase the bin value to create a coarser WMAP.
Couldn't determine chip position for pixel: (1013.000000,0.000000) with value=1.000000. Skipping pixel
The issue is that the conversion from detector coordinates (i.e. those used to create the WMAP) to chip coordinates needs the SIM_Z values. In reality, SIM_Z varies with time, but as we have an image (hence no time information) and the SIM_Z variations are usually small, a single value stored in the image header is used for the transformation.
This means that there's actually some ambiguity in the mapping from the WMAP to chip location. In reality this isn't a concern, since we already bin on larger scales than introduced by the SIM_Z variation (i.e. we are only concerned with 32x32 pixel regions on the chip). However, it does mean that those pixels close to the chip boundaries can end up as apparently not falling on a chip. In this case, we issue the warning shown and ignore the counts from this pixel in the WMAP.
If the ignored counts are small compared to the total signal in the WMAP, as in the vast majority of situations, then it's not a problem. It can be a problem if most of the counts in the WMAP end up being ignored. In reality, this situation is unlikely to occur; two possible situations would be where the source region is narrow and lies along the chip boundary (this is a truly extreme case) or if you are using mkwarf to calculate "point source" responses near chip boundaries.
- acis_bkgrnd_lookup, acis_fef_lookup, acis_set_ardlib, add_grating_orders, add_grating_spectra, addresp, aprates, asphist, combine_spectra, dither_region, dmarfadd, dmextract, eff2evt, fullgarf, hrc_bkgrnd_lookup, make_instmap_weights, mean_energy_map, mkacisrmf, mkarf, mkexpmap, mkgarf, mkgrmf, mkinstmap, mkpsfmap, mkrmf, psextract, psf_project_ray, rmfimg, sky2tdet, specextract