Synopsis
Automate the steps to apply a fine astrometric correction
Syntax
fine_astro infile outroot [src_lists] [ref_src_list] [det_filter] [det_scales] [src_filter] [stop] [parallel] [nproc] [tmpdir] [cleanup] [clobber] [verbose]
Description
`fine_astro' automates the typical steps needed to apply a fine astrometric correction to a set of Chandra observations. This script automates the steps shown in the Correcting Absolute Astrometry thread. By default the script will run the wavdetect tool to create a source list for each observation, or users can choose to input a stack of source lists. Users can either apply a relative correction to the World Coordinate System (WCS) by matching source positions in multiple Chandra observations or apply an absolute correction by matching source positions against a reference catalog. The script will compute the cross matches using the wcs_match tool and then applies the translation-only solution to each event file and aspect solution file using the wcs_update tool. Users can then optionally run merge_obs to reproject the events files to the same tangent point and create co-added images.
Examples
Example 1
fine_astro infile="10309,26897" out="QSO_B1311-270"
fine_astro (23 October 2024)
infile = 10309,26897
outroot = QSO_B1311-270
src_lists =
ref_src_list =
det_filter =
det_scales = 1.4 2 4
src_filter =
stop = fineastro
parallel = yes
nproc = INDEF
tmpdir = /tmp
clobber = no
cleanup = yes
verbose = 1
mode = ql
Found 10309/repro/acisf10309_repro_evt2.fits
Found 26897/repro/acisf26897_repro_evt2.fits
Verifying 2 observations.
Gathering source lists
Running pre-detect merge_obs
Running wavdetect
Running wavdetect on 10309
Running wavdetect on 26897
Running cross matches using wcs_match
Using detect/QSO_B1311-270_10309.src for the reference source list
detect/QSO_B1311-270_10309.src nsrc=6 xoff= 0.000[pix] yoff= 0.000[pix]
detect/QSO_B1311-270_26897.src nsrc=2 xoff=-0.348[pix] yoff= 0.470[pix]
Updating astrometry
The following event files were created:
QSO_B1311-270_10309_fa_evt.fits
QSO_B1311-270_26897_fa_evt.fits
The following aspect solution files were created:
pcadf10309_000N001_fa_asol.fits
pcadf26897_000N001_fa_asol.fits
In the most simple form, the user supplies as input the directory names containing the event files and auxiliary data products and the output file root name (including directory). The script will run merge_obs to create the images it needs to run wavdetect. It will then cross match those source lists with the longest observation, in this example OBS_ID 10309, to compute the astrometric corrections. Finally, the event files and aspect solution files are updated.
Example 2
fine_astro infile="10309,26897" out="QSO_B1311-270" ref_src_list=csc2.1.fits
... Running cross matches using wcs_match detect/QSO_B1311-270_10309.src nsrc=4 xoff= 0.049[pix] yoff=-0.077[pix] detect/QSO_B1311-270_26897.src nsrc=1 xoff=-0.566[pix] yoff= 0.691[pix] Updating astrometry ...
This is the same as the previous example except now the crossmatch is done against an externally supplied source list. Note that the pixel offsets are different, and in particular that the offsets for OBS_ID=10309 are now non-zero.
Example 3
fine_astro infile=10309/repro/acisf10309_repro_evt2.fits out=obsid10309 ref=csc2.1.fits clob+
fine_astro (23 October 2024)
infile = 10309/repro/acisf10309_repro_evt2.fits
outroot = obsid10309
src_lists =
ref_src_list = csc2.1.fits
det_filter =
det_scales = 1.4 2 4
src_filter =
stop = fineastro
parallel = yes
nproc = INDEF
tmpdir = /tmp
clobber = yes
cleanup = yes
verbose = 1
mode = ql
Verifying one observation.
Gathering source lists
Running pre-detect merge_obs
Running wavdetect
Running wavdetect on 10309
Running cross matches using wcs_match
detect/obsid10309_10309.src nsrc=4 xoff= 0.130[pix] yoff=-0.060[pix]
Updating astrometry
The following event files were created:
obsid10309_10309_fa_evt.fits
The following aspect solution files were created:
pcadf10309_000N001_fa_asol.fits
This is similar to the previous example, except now we have explicitly specified a single event file, by name. When using a reference source list we can update the astometery of a single OBS_ID.
Example 4
fine_astro @obi.lis QSO_B1311-270 src_lists=@srcs.lis ref_src_list=csc2.1.fits clob+
fine_astro (23 October 2024)
infile = @obi.lis
outroot = QSO_B1311-270
src_lists = @srcs.lis
ref_src_list = csc2.1.fits
det_filter =
det_scales = 1.4 2 4
src_filter =
stop = fineastro
parallel = yes
nproc = INDEF
tmpdir = /tmp
clobber = yes
cleanup = yes
verbose = 1
mode = ql
Found 10309/repro/acisf10309_repro_evt2.fits
Found 26897/repro/acisf26897_repro_evt2.fits
Verifying 2 observations.
Gathering source lists
Running cross matches using wcs_match
obsid10309_cell.src nsrc=2 xoff=-0.314[pix] yoff=-0.607[pix]
Warning: Failed to identify matching sources for OBS_ID 26897. Proceeding with no astrometric corrections applied.
obsid26897_cell.src nsrc=0 xoff= 0.000[pix] yoff= 0.000[pix]
Updating astrometry
The following event files were created:
QSO_B1311-270_10309_fa_evt.fits
QSO_B1311-270_26897_fa_evt.fits
The following aspect solution files were created:
pcadf10309_000N001_fa_asol.fits
pcadf26897_000N001_fa_asol.fits
In this example we have used stacks to specify the list of observations and the list of source lists to use. When specifying the "src_lists" parameters, users must also specify the "ref_src_list" parameter. The contents of the file "obi.lis" looks like
# obsid 10309 26897
(note: lines that begin with "#" are ignored) and the contents of the file "srcs.lis" looks like
obsid10309_cell.src obsid26897_cell.src
which in this example are source lists created with the celldetect tool.
In this example, OBS_ID 26897, failed to find any matching sources between the input source list and the reference source list. In this case a warning is shown and the script proceeds by applying a WCS update with no offsets (equal to 0.0).
Example 5
fine_astro @obi.lis "QSO_J1016+2037" clob+
...
Running cross matches using wcs_match
Using detect/QSO_J1016+2037_25858.src for the reference source list
detect/QSO_J1016+2037_20411.src nsrc=2 xoff= 2.188[pix] yoff= 0.472[pix]
detect/QSO_J1016+2037_25382.src nsrc=2 xoff= 2.944[pix] yoff= 0.422[pix]
detect/QSO_J1016+2037_25857.src nsrc=2 xoff= 3.630[pix] yoff= 0.753[pix]
Warning: X shift = 3.630 exceeds warning limit of 3.0 pixels
detect/QSO_J1016+2037_25858.src nsrc=2 xoff= 0.000[pix] yoff= 0.000[pix]
detect/QSO_J1016+2037_26339.src nsrc=2 xoff= 0.688[pix] yoff=-0.204[pix]
detect/QSO_J1016+2037_27711.src nsrc=2 xoff= 0.875[pix] yoff= 0.853[pix]
detect/QSO_J1016+2037_27728.src nsrc=2 xoff= 3.972[pix] yoff= 1.221[pix]
Warning: X shift = 3.972 exceeds warning limit of 3.0 pixels
Warning: Total offset = 4.155 exceeds warning limit of 4.0 pixels
detect/QSO_J1016+2037_27729.src nsrc=2 xoff= 3.621[pix] yoff= 1.159[pix]
Warning: X shift = 3.621 exceeds warning limit of 3.0 pixels
Updating astrometry
...
In this example with 8 observations there are some observations with large offsets. This script will produce a warning if the offset in either the X or Y direction exceeds 3 pixels and/or if the total offset exceeds 4 pixels. Users may want to more carefully examine the results to ensure that the automatic crossmatch worked correctly.
Example 6
fine_astro @/home/user/Chandra/Abell3562/obi.lis abell3562/out \ src_filter="[pos=circle(203.4117720d,-31.6794249d,10)]"
...
Running cross matches using wcs_match
Using abell3562/detect/out_25861_filtered.src for the reference source list
out_25494_filtered.src nsrc=1 xoff= 0.274[pix] yoff=-0.509[pix]
out_25859_filtered.src nsrc=1 xoff= 0.991[pix] yoff= 1.912[pix]
out_25860_filtered.src nsrc=1 xoff= 0.896[pix] yoff= 1.894[pix]
out_25861_filtered.src nsrc=1 xoff= 0.000[pix] yoff= 0.000[pix]
out_26404_filtered.src nsrc=1 xoff= 0.741[pix] yoff= 2.149[pix]
out_27965_filtered.src nsrc=1 xoff= 0.380[pix] yoff=-1.140[pix]
out_4167_filtered.src nsrc=1 xoff= 0.592[pix] yoff=-1.276[pix]
Updating astrometry
...
A filter, src_filter, is added to the source list to identify a single source to be used for all the cross matches. Note: The detect tools contain a "pos" column, not a "sky" column. Also note the trailing 'd' in the used in the coordinates of the circle to indicate that these are celestial coordinates, ie RA and Dec values, rather than physical coordinates; alternatively users could also have used sexagesimal format to indicate that the values are celestial rather than physical coordinates. The output source list has been filtered as is indicated by the '_filtered' suffix addd to the file names.
Example 7
fine_astro @/home/user/Chandra/Abell3562/obi.lis abell3562/out \ det_filter="ccd_id=0:3"
This is similar to the previous example example but now we have applied detect filter, det_filter. The det_filter is added to the input event files when creating the images with merge_obs that will be used to detect sources. In this example we are restricting the images to just the 4 ACIS-I chips.
Parameters
| name | type | ftype | def | min | max | reqd | stacks |
|---|---|---|---|---|---|---|---|
| infile | file | input | yes | yes | |||
| outroot | file | output | yes | ||||
| src_lists | file | input | yes | ||||
| ref_src_list | file | input | |||||
| det_filter | string | ||||||
| det_scales | string | 1.4 2 4 | |||||
| src_filter | string | ||||||
| stop | string | fineastro | |||||
| parallel | boolean | yes | |||||
| nproc | integer | INDEF | |||||
| tmpdir | string | ${ASCDS_WORK_PATH} | |||||
| cleanup | boolean | yes | |||||
| clobber | boolean | no | |||||
| verbose | integer | 1 | 0 | 5 |
Detailed Parameter Descriptions
Parameter=infile (file required filetype=input stacks=yes)
Input event files
Can be stack of event files, or a stack of directories. For more details see the merge_obs "infile" parameter description or from the command line:
ahelp -b PARAM -t infiles merge_obs
Parameter=outroot (file required filetype=output)
The output directory and root file name
The root of the output file names to create (including directory). The script creates two type of files
- For each input event file, the script will create an output event file with the name: ${outroot}_${obi}_fa_evt.fits; "fa" indicates fine-astrometry correction has been applied. ${obi} will most often be the OBS_ID of the observation. For multi-obi observations ${OBI} will be the OBS_ID followed by an underscore and the OBI_NUM. For interleaved mode, ${obi} will be the OBS_ID followed by "e1" or "e2" for the primary and secondary exposures.
- For each observation, the script will update all the aspect solution files associated with the OBI. Most OBS_IDs now contain a single aspect solution file; however there are a few early datasets that still may contain multiple aspect solutions files per OBI. The script will use the same base file names as the input aspect solution file, often pcadf${obs_id}_${obi_num}N00${n}, and will replace the suffix "asol1" with "fa_asol".
Each output event file will have the ASOLFILE keyword updated with the new aspect solution file names.
Parameter=src_lists (file filetype=input stacks=yes)
A stack of source lists to be used for the cross match.
Users may supply their own source list to be used to cross match against the reference source list. This could for example be the output from another CIAO detect tool (eg celldetect) or could be from running wavdetect in a different energy band. Or it could be generated using some other detect tool/algorithm outside of CIAO.
Format of src_lists files
The source lists must contain the columns RA and DEC, in decimal degrees; sexagesimal formats are not supported. The source lists must be in numeric order based on the OBS_ID. If the source list has an OBS_ID keyword it will be checked against the corresponding event file to make sure that they match.
The most simple example of a valid src_lists file is an ASCII file with columns RA and DEC. For example:
$ cat src_list_01.dat #RA DEC 203.4117720 -31.6794249
FITS binary tables, eg as created by the CIAO detect tools, are also valid. If the table has columns with names other than "RA" and "DEC", for example "RAJ2000" and "DECJ2000", then you can rename the columns on the file using CIAO datamodel syntax:
... src_lists="my_file.fits[cols ra=raj2000,dec=decj2000]" ...
If the source lists contain the columns "RA_ERR" and "DEC_ERR", then wcs_match will use those when determining source matches and when computing the transform parameters.
If users supply their own src_lists, then they must also supply the reference source list: ref_src_list.
Parameter=ref_src_list (file filetype=input)
The source list to use as a reference
Users can choose to supply their own source list to be used as the reference for the astrometric correction. If users supply their own src_lists, then they must also supply the reference source list: ref_src_list. The ref_src_list must contain the columns RA and DEC, in decimal degrees; sexagesimal formats are not supported. It may for example be from some external catalog (eg GAIA) to provide an "absolute" reference or it can be another Chandra source list to generate a "relative" correction.
If no ref_src_list is supplied, then the tool will use the source list for the observation with the longest EXPOSURE time.
The format of the ref_src_list is the same as the src_lists parameters as discussed in the "Format of src_lists files" section above.
Parameter=det_filter (string)
Filter to apply to event files when detecting sources
There may be times when it is useful to supply a filter to the input event file when doing the source detection. For example, users may want to restrict the source detection to a certain single source, or to sources some distance from the optical axis, etc. The contents of this parameter are added as a filter to each event file when the images are created to used for detect.
Parameter=det_scales (string default=1.4 2 4)
wavdetect wavelet detect scales
The size of the wavelets to use to search for sources. Units are in logical pixel size. See wavdetct ahelp page for more details. From the command line:
ahelp -b PARAM -t scales wavdetect
Parameter=src_filter (string)
Source list filters
Users can choose to apply a filter the detected source lists. This could for example be used to specify a minimum number of net-counts (ie only use bright sources for the cross match). This filter is only applied to the internally detected sources; it is not applied to the files specified by the src_lists nor ref_src_list parameters.
Parameter=stop (string default=fineastro)
Stop processing after the fine astrometric correction or continue to run merge_obs?
There are two "stop" options:
- stop=fineastro : The script will stop after computing and applying the fine astrometric correction to the event files and aspect solution files.
- stop=mergeobs : After the fine astrometric corrections are applied, the script will run merge_obs. This will reproject the event files to a common tangent point and will dmmerge them. It will create the counts image, exposure map, and PSF map in the default energy band.
Parameter=parallel (boolean default=yes)
Run processes in parallel?
When run on a multi-processor system, many of the tasks can be run in parallel to reduce the execution time of the script. Since the tasks are likely to be memory or I/O-bound, the reduction in run time will be less than the number of cores on a machine. When parallel=yes, the default behaviour is to use all the CPU processors, but this can be changed with the nproc parameter.
This option can be ignored when run on a single-processor system.
Parameter=nproc (integer default=INDEF)
Number of processors to use
This parameter is only used when parallel=yes. It determines the number of processors to use. If maxproc is the actual number of processors on your machine, then a value of INDEF - the default value - means that all maxproc processors will be used. A positive value means to use that number of processors (any value larger than maxproc will be set to maxproc). A negative value is added to maxproc (and any value less than one is set to one).
Parameter=tmpdir (string default=${ASCDS_WORK_PATH})
Directory for temporary files.
Directory for storing temporary files that require further processing before becoming useful. If the directory does not exist then it will be created for use by the script, and then deleted.
Parameter=cleanup (boolean default=yes)
Cleanup intermediary files on exit.
If set to "yes", the intermediate data products are deleted when the script ends, leaving only: the reprojected event files; merged event file; per-observation images, exposure maps, and exposure-corrected images; and the combined images, exposure maps, and exposure-corrected images.
Parameter=clobber (boolean default=no)
Overwrite existing files?
Parameter=verbose (integer default=1 min=0 max=5)
Output verbosity.
The default verbosity value of 1 prints status messages as the script runs. Higher verbosity settings print the commands that are being run. Setting verbose=0 turns off most of the screen output (some output is currently unavoidable).
Processing Step-by-Step
The script will try to locate the event files in each input directory. First it looks in the directory itself, then in a subdirectory named "repro" and finally in a sub directory named "primary".
The script will then run the merge_obs script to create the input files needed to run wavdetect: counts image, exposure map, and PSF map. The script uses the default energy band which is 'broad' band for ACIS and 'wide' band for HRC. Users who require a source list created using a different energy band must create the source lists separately and use the "src_lists" parameter to specify them. If the "det_filter" parameter is set, it is used to filter the event files during this initial merge. This can be used to restrict the source detection to a single CCD_ID, to a specific spatial region, etc. The output from the initial merge_obs run are in the 'pre' output directory.
wavdetect is run on each separate observation (separately for each OBS_ID, multi-obi observations, and interleaved mode observations). wavdetect is run in parallel based on the number of CPUs available and the parameter+nproc parameter settings. Users can specify the wavelet scales using the "det_scales" parameter. If the "src_filter" parameter is specified, then each source list is filtering using the value provided. This can be used for example to include on high count/significance sources, to excludes sources detected in certain regions, etc. The detect outputs are stored in the 'detect' output directory.
Next the sources are cross matched. If no reference source list is supplied, then the observation with the longest EXPOSURE time is used. The script runs the wcs_match tool using a match radius of 4 arcsec and generates a translation only solution. If there are no matching sources, then the script will issue a warning and will proceed by using a 0-offset solution.
The script then applies each fine astrometric correction to each event file and all associated aspect solution files using the wcs_update tool. The ASOLFILE header keyword in the updatd event file is also updated to point to the new aspect soltuion file(s).
Finally, if the stop parameter is set stop=mergeobs, then the script will run merge_obs to reproject the event astrometrically adjusted event files to the same tangent point and to create images using the default energy band.
About Contributed Software
This script is not an official part of the CIAO release but is made available as "contributed" software via the CIAO scripts page . Please see this page for installation instructions - such as how to ensure that the parameter file is available.
Bugs
There are no known bugs for this tool.
See Also
- tools::coordinates
- reproject_aspect, reproject_events, sso_freeze, wcs_match, wcs_update