Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: 16 Dec 2014

URL: http://cxc.harvard.edu/ciao/threads/combine/

Merging Data from Multiple Imaging Observations

CIAO 4.9 Science Threads


Overview

Synopsis:

When merging event files from observations with similar, but not necessarily identical, target coordinates, the event sky coordinates (X,Y in the event list) must be recomputed. This is often referred to as "regridding" or "reprojecting events to a different tangent point." It is necessary because the sky coordinates are computed relative to the nominal R.A. and Dec. for each individual observation, which will likely differ between observations of the same target.

Purpose:

To combine ACIS or HRC imaging data from multiple observations, e.g. to search for faint sources or low-surface-brightness extended features.

The merged event list should not be used for spectral analysis, since it does not contain sufficient information to generate correct response files. The recommended technique for the spectral analysis case is to generate separate PHA, RMF, and ARF files for each observation and to analyze them simultaneously.

Related Links:

Last Update: 16 Dec 2014 - Reviewed for CIAO 4.7. Added a link to the merged datasets spectrum caveat.


Contents


Getting Started

Download the sample data: 1842 (ACIS-I, G21.5-09); 1843 (ACIS-I, G21.5-09)

unix% download_chandra_obsid 1842,1843 
unix% chandra_repro 1842,1843 outdir=

Merging about a Common Tangent Point

Each event list contains all the World Coordinate System (WCS) information required to transform between sky coordinates and celestial coordinates in its header. If the WCSs are accurate, one may merge event lists simply by converting sky coordinates in one event list to celestial, using that list's WCS, and then converting those celestial coordinates to sky coordinates in the second event list using the second WCS (see the ds9 thread coordinate definitions for more information).

The level=2 event files may be reprojected to a common tangent point in one of two ways:

Using WCS Information to Merge

In this case we choose to map ObsId 1842 onto 1843, so we set the refcoord parameter of reproject_obs to 1843/repro/acisf01843_repro_evt2.fits:

unix% punlearn reproject_obs
unix% reproject_obs 1842,1843 merged1/ refcoord=1843/repro/acisf01843_repro_evt2.fits
Running reproject_obs
Version: 13 November 2013

Found 1842/repro/acisf01842_repro_evt2.fits
Found 1843/repro/acisf01843_repro_evt2.fits
Verifying 2 observations.
Tangent point is taken from the file 1843/repro/acisf01843_repro_evt2.fits
New tangent point: RA=18h 32m 11.496s Dec=-10d 34' 13.44"

Observations to be reprojected:

  Obsid  Obs Date   Exp    DETNAM     SIM_Z    FP   Sepn   PA
                   (ks)                (mm)    (K)   (')  (deg)
---------------------------------------------------------------
1 1843  2000-09-02   7.9 ACIS-012367 -245.857 153.4   0.0    +0
2 1842  2000-09-02   7.4 ACIS-012367 -231.239 153.4   5.1  +101

Running tasks in parallel with 4 processors.
Reprojecting 2 event files to a common tangent point.
Merging reprojected events files to merged1/merged_evt.fits

The following files were created:

The reprojected event files:
     merged1/1843_reproj_evt.fits
     merged1/1842_reproj_evt.fits

The merged event file:
     merged1/merged_evt.fits

Warning: the merged event file merged1/merged_evt.fits
   should not be used to create ARF/RMF/exposure maps because
      the RA_NOM keyword varies by 0.08513503515 (limit is 0.0003)
      the DEC_NOM keyword varies by 0.015586294773 (limit is 0.0003)
      the ROLL_NOM keyword varies by 2.47035457541 (limit is 1.0)
      the SIM_Z keyword varies by 14.6182291633 (limit is 0.1)

The contents of the parameter file may be checked using dmhistory:

unix% dmhistory merged1/merged_evt.fits reproject_obs 
reproject_obs infiles="1842,1843" outroot="merged1/" asolfiles="" merge="yes"
   refcoord="1843/repro/acisf01843_repro_evt2.fits" parallel="yes" nproc="INDEF" 
   linkfiles="yes" tmpdir="/tmp" clobber="no" verbose="1" 

and the script created the following files:

unix% ls -1 merged1/
1842.asol@
1842.bpix@
1842.mask@
1842_reproj_evt.fits
1843.asol@
1843.bpix@
1843.mask@
1843_reproj_evt.fits
merged_evt.fits

The merged event file can be displayed in ds9:

unix% ds9 -cmap b merged1/merged_evt.fits -bin filter 'energy=500:7000' -bin about 3815 3975 -zoom 8

as shown in Figure 1.

[Thumbnail image: The source photon distribution in the merged event list appears as slightly double-peaked.]

[Version: full-size]

[Print media version: The source photon distribution in the merged event list appears as slightly double-peaked.]

Figure 1: Data merged by WCS information

The appearance of the merged source - which has two peaks - suggests that there is a small systematic offset in the two WCSs.

The source photon distribution in the merged event list appears as slightly double-peaked, indicating that there is in fact a small (2-3 pixels) systematic offset in the two WCSs. This may be minimized by first correcting the aspect, as described in the Correcting Aspect Prior to Merging section.


Merging about a Specific R.A. and Dec.

It is also possible to reproject the events to a new tangent point that is different than any of the input observations. In this example, both event lists are reprojected to the arbitrarily chosen point (R.A.,Dec) = (278.086,-10.58625) prior to merging.

unix% punlearn reproject_obs
unix% reproject_obs 1842,1843 merged2/ refcoord="278.086,-10.58625"
Running reproject_obs
Version: 13 November 2013

Found 1842/repro/acisf01842_repro_evt2.fits
Found 1843/repro/acisf01843_repro_evt2.fits
Verifying 2 observations.
New tangent point: RA=18h 32m 20.64s Dec=-10d 35' 10.5"

Observations to be reprojected:

  Obsid  Obs Date   Exp    DETNAM     SIM_Z    FP   Sepn   PA  
                   (ks)                (mm)    (K)   (')  (deg)
---------------------------------------------------------------
1 1843  2000-09-02   7.9 ACIS-012367 -245.857 153.4   2.4   -67
2 1842  2000-09-02   7.4 ACIS-012367 -231.239 153.4   2.8   +90

Running tasks in parallel with 4 processors.
Reprojecting 2 event files to a common tangent point.
Merging reprojected events files to: merged2/merged_evt.fits

The following files were created:

The reprojected event files:
     merged2/1843_reproj_evt.fits
     merged2/1842_reproj_evt.fits

The merged event file:
     merged2/merged_evt.fits

Warning: the merged event file merged2/merged_evt.fits
   should not be used to create ARF/RMF/exposure maps because
      the RA_NOM keyword varies by 0.08513503515 (limit is 0.0003)
      the DEC_NOM keyword varies by 0.015586294773 (limit is 0.0003)
      the ROLL_NOM keyword varies by 2.47035457541 (limit is 1.0)
      the SIM_Z keyword varies by 14.6182291633 (limit is 0.1)

The contents of the parameter file may be checked using dmhistory:

unix% dmhistory merged2/merged_evt.fits reproject_obs 
reproject_obs infiles="1842,1843" outroot="merged2/" asolfiles="" merge="yes"
   refcoord="278.086,-10.58625" parallel="yes" nproc="INDEF" 
   linkfiles="yes" tmpdir="/tmp" clobber="no" verbose="1" 

The data looks the same as Figure 1 apart from a different SKY to Ra,Dec mapping; the choice of tangent point means that the source is close to 4090,4095 rather than 3815,3975 above:

unix% ds9 -cmap b merged2/merged_evt.fits -bin filter 'energy=500:7000' -zoom 8

Correcting Aspect Prior to Merging

In order to improve on the results of the previous examples, we must address the problem of the systematic shift in the coordinate systems of the two event lists. To do this, we will shift the aspect solution of one to make it consistent with the other.

[WARNING]
Warning

If you choose to use this method, the shifted aspect solution should then be used throughout the rest of the analysis in order to obtain correct results. This is why we update the ASOLFILE keyword in the reprojected event file.

First we determine what shift to apply to the data. In this case we use dmstat to calculate the centroid of the source in the two observations, but other approaches, such as using the positions calculated by wavdetect are also possible.

unix% punlearn dmstat
unix% dmstat "1842/repro/acisf01842_repro_evt2.fits[energy=500:7000][sky=circle(4429,4093,5)][cols eqpos]"
EQPOS(RA, DEC)[deg]
    min:	( 278.08616228 -10.587086496 )	      @:	( 63 70 )
    max:	( 278.08718721 -10.58613893 )	      @:	( 34 12 )
   mean:	( 278.08680308 -10.586482825 )
  sigma:	( 0.00023073756038 0.00015688307705 )
    sum:	( 22246.944246 -846.91862601 )
   good:	( 80 80 )
   null:	( 0 0 )

unix% set p1 = `pget dmstat out_mean`
unix% dmstat "1843/repro/acisf01843_repro_evt2.fits[energy=500:7000][sky=circle(3815,3981,5)][cols eqpos]"
EQPOS(RA, DEC)[deg]
    min:	( 278.08649393 -10.586581351 )	      @:	( 14 42 )
    max:	( 278.08732648 -10.585561927 )	      @:	( 26 92 )
   mean:	( 278.08692376 -10.58610396 )
  sigma:	( 0.00014081515996 0.00019400577118 )
    sum:	( 31145.735461 -1185.6436435 )
   good:	( 112 112 )
   null:	( 0 0 )

unix% set p2 = `pget dmstat out_mean`
unix% echo $p1
278.08680308,-10.586482825
unix% echo $p2
278.08692376,-10.58610396

We decide to shift ObsId 1842 onto 1843 since it is the shorter observation (although in this case the difference is not large). To find the shift:

  1. calculate the difference between the two centroids:

    unix% chips -n
    chips-1> ra1 = 278.08680308
    chips-2> ra2 = 278.08692376
    chips-3> dec1 = -10.586482825
    chips-4> dec2 = -10.58610396
    chips-5> dra = ra2 - ra1
    chips-6> ddec = dec2 - dec1
    chips-7> dra
             0.00012068000000908796
    chips-8> ddec
             0.00037886499999828516
    
  2. correct the delta RA by "cos(Dec)" to get a reasonable approximation of the distance between the two points.

    chips-9> mean_dec = ( dec1 + dec2 )/2.0
    chips-10> dra2 = dra * np.cos(mean_dec * np.pi / 180.0)
    chips-11> dra2
              0.00011862594517488127
    
  3. convert the values to sky pixels. The delta x conversion is done with -1.3667E-4 deg/pixel because the x value is opposite in sign from RA. Dec and y have the same sign, so 1.3667E-4 deg/pixel is used for that calculation. (note that 0.492/3600 = 1.3667e-4, where 0.492 is the ACIS pixel size)

    chips-12> conv = 0.492 / 3600
    chips-13> dx = dra2 * -1 / conv
    chips-14> dy = ddec / conv
    chips-15> dx
              -0.86799472079181417
    chips-16> dy
              2.7721829268167206
    

The tool wcs_update is used to apply the offsets. The shifts are input to the deltax and deltay parameters in units of sky pixels. Additionally, the WCS information from the event file for the match observation (1843/repro/acisf01843_repro_evt2.fits) is supplied in the wcsfile parameter. Note that the transformfile parameter is left empty.

unix% punlearn wcs_update
unix% pset wcs_update infile=1842/repro/pcadf084280882N003_asol1.fits
unix% pset wcs_update outfile=1842/repro/new_asol1.fits
unix% pset wcs_update wcsfile=1843/repro/acisf01843_repro_evt2.fits
unix% pset wcs_update deltax=-0.87 deltay=2.77
unix% wcs_update
Input coordinate transform file (): 
Either input asol file, or file with WCS to be updated (1842/repro/pcadf084280882N003_asol1.fits): 
Output asol file (1842/repro/new_asol1.fits): 

We then use the new aspect solution to re-calculate the WCS positions of events in the original event file with the reproject_events tool:

unix% punlearn reproject_events
unix% pset reproject_events infile=1842/repro/acisf01842_repro_evt2.fits 
unix% pset reproject_events outfile=1842/repro/new_evt2.fits 
unix% pset reproject_events aspect=1842/repro/new_asol1.fits
unix% reproject_events
Input dataset/block specification (1842/repro/acisf01842_repro_evt2.fits): 
Output dataset/block specification (1842/repro/new_evt2.fits): 
Match file (none): 

and adjust the ASOLFILE keyword in the new events file to point to the new aspect solution:

unix% dmhedit 1842/repro/new_evt2.fits filelist= op=add key=ASOLFILE value=new_asol1.fits
[IMPORTANT]
The sub-pixel algorithm

reproject_events does NOT preserve the sub-pixel improvements made by the EDSER algorithm in acis_process_events when the aspect parameter is not empty, as above.

The acis_process_events tool can be used to preserve the EDSER improvements, but care needs to be taken if you have used non-standard settings previously with this tool to create the level-2 event file.

unix% punlearn acis_process_events
unix% pset acis_process_events apply_cti- apply_tgain- doevtgrade-
unix% pset acis_process_events calculate_pi- rand_pha-
unix% acis_process_events 1842/repro/acisf01842_repro_evt2.fits 1842/repro/ape_evt2.fits 1842/repro/new_asol1.fits
# acis_process_events (CIAO): WARNING: The values of PHA, ENERGY, PI, FLTGRADE and GRADE may be inaccurate because doevtgrade=no.
# acis_process_events (CIAO): WARNING: The values of ENERGY and PI may be inaccurate because calculate_pi=no.

Manually reprojecting the data

We can now reproject ObsId 1842 onto 1943 before merging them (making sure to list the inputs to dmmerge in time order):

unix% punlearn reproject_events
unix% reproject_events 1842/repro/new_evt2.fits 1842/repro/reproj_evt2.fits 1843/repro/acisf01843_repro_evt2.fits
unix% punlearn dmmerge
unix% dmmerge 1843/repro/acisf01843_repro_evt2.fits,1842/repro/reproj_evt2.fits merged.fits
omit - DEC_PNT values different more than 0.000300
warning: DS_IDENT has different value...Merged...
warning: OBJECT has different value...Merged...
warning: OBS_ID has different value...Merged...
omit - RA_PNT values different more than 0.000300
omit - ROLL_NOM values different more than 1.000000
omit - ROLL_PNT values different more than 1.000000
warning: SEQ_NUM has different value...Merged...
omit - SIM_Z values different more than 0.100000

The merged event file can be displayed in ds9:

unix% ds9 -cmap b merged.fits -bin filter 'energy=500:7000' -bin about 3815 3975 -zoom 8

(as shown in Figure 2).

[Thumbnail image: The source now has a single peak, rather than the double peak visible in previous versions.]

[Version: full-size]

[Print media version: The source now has a single peak, rather than the double peak visible in previous versions.]

Figure 2: Aspect solution corrected before reprojection

The shift in the peak emission visible in earlier versions - such as Figure 1 - is no longer seen.


Using reproject_obs

When combining data from multiple observations - or you wish to reproject the data to a different position - it is easier to run reproject_obs on all the observations, rather than running reproject_events and dmmerge manually. For example, we could have said

unix% punlearn reproject_events
unix% reproject_obs 1842/repro/new_evt2.fits,1843/repro/acisf01843_repro_evt2.fits merged3/
Running reproject_obs
Version: 13 November 2013

Verifying 2 observations.
Calculating new tangent point.
New tangent point: RA=18h 32m 21.696s Dec=-10d 34' 41.484"

Observations to be reprojected:

  Obsid  Obs Date   Exp    DETNAM     SIM_Z    FP   Sepn   PA  
                   (ks)                (mm)    (K)   (')  (deg)
---------------------------------------------------------------
1 1843  2000-09-02   7.9 ACIS-012367 -245.857 153.4   2.6   -79
2 1842  2000-09-02   7.4 ACIS-012367 -231.239 153.4   2.6  +101

Running tasks in parallel with 4 processors.
Reprojecting 2 event files to a common tangent point.
Merging reprojected events files to: merged3/merged_evt.fits

The following files were created:

The reprojected event files:
     merged3/1843_reproj_evt.fits
     merged3/1842_reproj_evt.fits

The merged event file:
     merged3/merged_evt.fits

Warning: the merged event file merged3/merged_evt.fits
   should not be used to create ARF/RMF/exposure maps because
      the RA_NOM keyword varies by 0.08513503515 (limit is 0.0003)
      the DEC_NOM keyword varies by 0.015586294773 (limit is 0.0003)
      the ROLL_NOM keyword varies by 2.47035457541 (limit is 1.0)
      the SIM_Z keyword varies by 14.6182291633 (limit is 0.1)

unix% ds9 -cmap b merged3/merged_evt.fits -bin filter 'energy=500:7000' -bin about 4121 4038 -zoom 8

which creates an image similar to Figure 2, although with different mapping between SKY and WORLD coordinates, because of the different tangent point.


History

10 Jan 2012 reviewed for CIAO 4.4: clarified the order of the steps in the Correcting Aspect Prior to Merging section
15 Oct 2012 Updated to use reproject_obs rather than merge_all, which is now deprecated. The Correcting Aspect Prior to Merging section has been updated.
03 Dec 2012 Review for CIAO 4.5
15 Mar 2013 Fixed links in manually merging section.
27 Jun 2013 Tweaked information in the distance equations.
02 Dec 2013 Review for CIAO 4.6. Noted that the PBK file for ACIS observations is no-longer copied/linked by reproject_obs.
16 Dec 2014 Reviewed for CIAO 4.7. Added a link to the merged datasets spectrum caveat.


Last modified: 16 Dec 2014
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.