Using MARX to Create an Event File from ChaRT Rays
CIAO 4.9 Science Threads
The output of ChaRT is a set of rays (a PSFRAYS table) that cannot be used directly in your analysis; it must be converted into a suitable format, i.e. a pseudo-events file. This thread describes how to use the MARX software to do this conversion by projecting the rays onto the detector-plane, and then accounting for the detector (ACIS or HRC).
Using MARX allows the user to take into account all changes to the photon distribution emerging from the HRMA due to the detector response. In particular, the detector QE & QEU and the roll are accounted for. In addition to simulating the detector response, MARX uses the ray weights to account for the mirror effects, i.e. different efficiency of different shells at different angles/energies.
This thread is not a definitive guide to using MARX. There are many more options and parameters discussed in the MARX manual.
- Caveats page: details on what is not included when using MARX.
- The MARX Frequently Asked Questions webpage
- Using a PSF Image as the Convolution Kernel
Last Update: 28 Jul 2016 - Added information about suggested ACIS-S AspectBlur.
- Get Started
- Setting up MARX
- Get Source Coordinates
- Using simulate_psf script
- Step by Step
- Projecting the Rays
- Create an Events File
- Common Mistakes
This example uses HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits, which was created in the Introduction to ChaRT Data Files thread. In addition to the ray files, the actual observation data are also retrieved
Download the sample data: 942 (ACIS-S, NGC 4244)
unix% download_chandra_obsid 942
The rest of this thread assumes the data have been reprocessed with chandra_repro.
This thread requires the MARX software. If you do not have MARX installed you can install it using the install_marx script or manually by following these instructions (may be necessary for some OSX users).
The default C compiler in OSX is clang, which is aliased to gcc. While MARX compiles successfully with clang, several problematic issues have been seen in the resulting MARX build. The MARX developers suggest using a true gcc compiler, obtainable at http://hpc.sourceforge.net/ for Max OS X, as noted in Known Bugs and Limitations—Max OS X. The real gcc needs to be manually installed and put early in the PATH environment variable, otherwise the broken Mac OS X version will be picked up. Prior to installing MARX, it may also be worthwhile to set the CC environment variable to the gcc
For example, if the new gcc is installed in /usr/local/bin/, then set the C compiler variable CC:
(t)csh% setenv CC /usr/local/bin/gcc
bash% export CC=/usr/local/bin/gcc
Set up your environment to access all the files and directories which MARX needs:
unix% setenv MARX_DIR <local_marx_directory> unix% set path = ($path $MARX_DIR/bin) unix% setenv MARX_DATA_DIR $MARX_DIR/share/marx/data
Alternatively, the MARX DataDirectory parameter can be defined to point to the data files. By default, this parameter is set to the contents of the $MARX_DATA_DIR environment variable. At this point, your system is ready to run simulations
Before running the simulation, it is necessary to determine the nominal position of the detector during the observation. These values are stored in the header of the level=2 events file:
unix% dmlist acisf00942_repro_evt2.fits header | grep _NOM 0055 RA_NOM 184.3430399825 Real8 Nominal RA 0056 DEC_NOM 37.7808853898 Real8 Nominal Dec 0057 ROLL_NOM 230.8753656982 Real8 Nominal Roll
Additionally, the source coordinates for the PSF are required. These were determined in the Determine the Off-axis Angle section of the Preparing to Run ChaRT thread. The values are stored in the ChaRT header
unix% dmlist HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits header | egrep "SRC_(RA|DEC)" 0010 SRC_RA 184.237458333 [deg] String Input source Right Ascension 0011 SRC_DEC 37.7265805556 [deg] String Input source Declination
Lastly we check if the ChaRT simulation used an aspect solution by inspecting the value of the ASOLFILE keyword. If present, this keyword must match the one used with MARX.
unix% dmkeypar HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits ASOLFILE echo+ pcadf075214790N004_asol1.fits.gz
The gzip, .gz, extension can be omitted.
The simulate_psf script automates the steps needed to run the ChaRT generated ray files, run them through marx, and produce an image of the PSF.
unix% pset simulate_psf infile=acisf00942_repro_evt2.fits unix% pset simulate_psf outroot=chart unix% pset simulate_psf ra=184.237458333 unix% pset simulate_psf dec=37.7265805556 unix% pset simulate_psf simulator=file unix% pset simulate_psf rayfile=HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits unix% pset simulate_psf projector=marx unix% simulate_psf mode=h simulate_psf infile = acisf00942_repro_evt2.fits outroot = chart ra = 184.237458333 dec = 37.7265805556 spectrumfile = monoenergy = INDEF flux = INDEF simulator = file rayfile = HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits projector = marx random_seed = -1 blur = 0.07000000000000001 readout_streak = no pileup = no ideal = yes extended = yes binsize = 1 numsig = 7 minsize = INDEF numiter = 1 keepiter = no asolfile = marx_root = /soft/marx-5.3.0 verbose = 1 mode = h Started check_setup Finished check_setup Performing iteration 1 of 1 Started run_marx Finished run_marx Started create_psf_image Finished create_psf_image Started create_average_image Finished create_average_image Final output PSF image is : chart.psf
Users may need to adjust the blur parameter as is discussed below.
This completes this thread. Users may wish the review the information about some common mistakes. Otherwise they may use the PSF image created in this step for their analysis.
The next alternative section is the step-by-step instructions showing how to setup and run marx.
Setting up Ray Projection
In order to analyze the spatial distribution of the PSF, we need to project the PSF rays in HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits onto the detector plane. MARX enables us to do so.
The steps in this thread need to be repeated for each ChaRT ray file individually. The individual images are combined in the Creating an Image of the PSF thread.
Since all the simulation parameters are the same, users only need to change the name of the rays file and the output directory for each run.
Copy over the parameter file locally:
unix% cp $MARX_DIR/share/marx/pfiles/marx.par ./marx.par unix% chmod +w ./marx.par
and set the necessary parameters:
unix% pset ./marx SAOSACFile=HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits unix% pset ./marx OutputDir=marx_output_i0000.dir unix% pset ./marx SourceType=SAOSAC unix% pset ./marx RA_Nom=184.34304 unix% pset ./marx Dec_Nom=37.78089 unix% pset ./marx Roll_Nom=230.87537 unix% pset ./marx SourceRA=184.237458333 unix% pset ./marx SourceDEC=37.7265805556 unix% pset ./marx DitherModel=FILE unix% pset ./marx DitherFile=pcadf075214790N004_asol1.fits unix% pset ./marx DetectorType=ACIS-S unix% pset ./marx GratingType=NONE unix% pset ./marx ExposureTime=0.0
There are a few things to note in the pset commands:
- Make sure to use the correct detector for the DetectorType parameter; valid options are HRC-S, ACIS-S, HRC-I, or ACIS-I.
The Exposuretime is set to 0.0 to ensure that the simulated output has the desired exposure length. The "Simulation Control: Exposure Time" section of the MARX manual [PDF] has more information on setting this parameter.
MARX defines the ACIS DetectorType values to be specific sets of CCDs:
- ACIS-I = ACIS-0123
- ACIS-S = ACIS-456789
If you have data that is taken at the imaging aimpoint, but on the spectroscopic array (e.g. aimpoint on ACIS-I3/3, but data on ACIS-S3/7) or vice-versa (aimpoint on ACIS-7, but data on ACIS-3), the DetectorType selected must match which CCD the data falls on, i.e. not the aimpoint array. Subsequently, a large DetOffsetZ value is to be expected.
Observations are often made with the science instrument module (SIM) is often offset from its nominal position. If these offsets are not included in the MARX run, the simulated PSF will not be placed at the correct location on the detector. The nominal MARX SIM positions are at:
|Detector||SIM_X [mm]||SIM_Y [mm]||SIM_Z [mm]|
and can be also looked up in the $MARX_DATA_DIR/caldb/telD1999-07-23aimptsN0002.fits file or by running the MARX ray projection with the default offset values of zero.
To determine the offset, first, compare the SIM values in the original Chandra events file to the ACIS-S values in the table.
unix% dmlist acisf00942_repro_evt2.fits header | grep SIM_ 0039 SIM_X -0.68282252473119 Real8 SIM focus pos (mm) 0040 SIM_Y 0 Real8 SIM orthogonal axis pos (mm) 0041 SIM_Z -190.1400660499 Real8 SIM translation stage pos (mm) MARX SIM_X: -0.68426746699586 MARX SIM_Y: 0 MARX SIM_Z: -190.1325231040
Some observations are done with an offset of several millimeters. The difference, calculated as event_file_value - marx_value, is small in this case:
SIM_X: -0.68282252473119 - (-0.68426746699586) = 0.00144494 SIM_Z: -190.1400660499 - (-190.1325231040) = -0.00754295
The SIM offset generally has a negligible effect on the off-axis angle, but often manifests itself by placing the source at a different position angle relative to the detector aimpoint. The determined SIM offsets can then be set:
unix% pset ./marx DetOffsetX=0.00144494 unix% pset ./marx DetOffsetY=0 unix% pset ./marx DetOffsetZ=-0.00754295
At the start of the mission, the SIM was located on-axis, but has since drifted due to changes in telescope geometry, with effects that can no longer be neglected. Fortunately, if an aspect solution is used for the observation's dither model, MARX will account for this drift using the available positional information at all points in time from the aspect solution file.
When simulating real Chandra data, to avoid pixelization effects in the pseudo-events file, there is an AspectBlur parameter which should be set. Users may experiment with the appropriate values of AspectBlur; we suggest values of 0.2 arcsec for ACIS-I, 0.25 arcsec for ACIS-S, and 0.07 arcsec for HRC as suitable for typical data, based on a limited set of simulations.
For ACIS-I: unix% pset ./marx AspectBlur=0.20 For ACIS-S: unix% pset ./marx AspectBlur=0.25 For HRC: unix% pset ./marx AspectBlur=0.07
MARX v4 has a similar parameter—with a physically different underpinning—called DitherBlur, which has since been supplanted by AspectBlur in MARX v5. In MARX v4, the aspect reconstruction process introduces small positional errors in the derived sky positions for events, roughly 0.35 arcsec for ACIS and 0.18 arcsec for HRC.
For ACIS: unix% pset ./marx DitherBlur=0.35 For HRC: unix% pset ./marx DitherBlur=0.18
unix% marx @@./marx.par MARX version 5.0.0, Copyright (C) 2002-2012 Massachusetts Institute of Technology ... screen output omitted ... Opening ASPSOL fits file pcadf075214790N004_asol1.fits [Using ASPSOL dither model] Initializing source type SAOSAC... Opening SAOSAC fits file HRMA_ra184.23746_dec37.72658_source_flux_chart.dat_dithered_i0000_rays.fits System initialized. Starting simulation. NumRays to collect = 1000000, dNumRays = 100000 Collecting 100000 photons... 8387 collected. Reflecting from HRMA Detecting with ACIS-S Writing output to directory 'marx_output.dir' ... Total photons: 8387, Total Photons detected: 2193, (efficiency: 0.261476) (efficiency this iteration 0.261476) Total time: 50307.129827
It should not be expected that the SIM_* header keywords of the observed events file and projected pseudo-events file be identical.
MARX creates a number of ASCII files in the specified directory (marx_output_i0000.dir):
unix% ls marx_output_i0000.dir b_energy.dat det_theta.dat obs.par sky_roll.dat xpos.dat zcos.dat det_dy.dat energy.dat pha.dat time.dat ycos.dat zpos.dat det_dz.dat marx.par sky_dec.dat xcos.dat ypixel.dat detector.dat mirror.dat sky_ra.dat xpixel.dat ypos.dat
These .dat files need to be converted to a FITS events file before they can be used in CIAO. The MARX tool marx2fits does this, given the directory name and the output filename.
unix% marx2fits --pixadj=EDSER marx_output_i0000.dir marx_output_i0000.fits Examining marx_output_i0000.dir/time.dat Examining marx_output_i0000.dir/detector.dat Examining marx_output_i0000.dir/energy.dat Examining marx_output_i0000.dir/b_energy.dat Examining marx_output_i0000.dir/xpos.dat Examining marx_output_i0000.dir/ypos.dat Examining marx_output_i0000.dir/zpos.dat Examining marx_output_i0000.dir/xcos.dat Examining marx_output_i0000.dir/ycos.dat Examining marx_output_i0000.dir/zcos.dat Examining marx_output_i0000.dir/xpixel.dat Examining marx_output_i0000.dir/ypixel.dat Examining marx_output_i0000.dir/pha.dat Examining marx_output_i0000.dir/mirror.dat Examining marx_output_i0000.dir/sky_ra.dat Examining marx_output_i0000.dir/sky_dec.dat Examining marx_output_i0000.dir/sky_roll.dat Examining marx_output_i0000.dir/det_dy.dat Examining marx_output_i0000.dir/det_dz.dat Examining marx_output_i0000.dir/det_theta.dat
Users should only use --pixadj=EDSER option when they supply ChaRT an aspect solution -- either via specifying OBS_ID or by uploading a file.
Using EDSER without the aspect solution will lead to rectangular artifacts that will no match the data. For simulation done without an aspect solution users should use --pixadj=EXACT.
If the objective of running ChaRT and MARX is to simulate an observation, as opposed to generate a model PSF, then you may want to include pileup effects. First you need to be sure that your simulation parameters exactly matched the observation (for example the spectrum flux was not artifically increased to increase sampling of the PSF). If these are not correct, the observed pileup cannot be matched to the simulation.
To include pileup, you must run the marxpileup tool:
unix% cp $MARX_DIR/share/marx/pfiles/marxpileup.par . unix% marxpileup MarxOutputDir=marx_output_i0000.dir
and to apply the pileup effects on the pseudo-events file, include the --pileup flag to process the pileup simulation with marx2fits.
unix% marx2fits --pixadj=EDSER --pileup marx_output_i0000.dir/pileup marx_output_i0000.fits
Note: With pileup enabled, it is unclear how to apply an ad-hoc normalization to get the simulation to match the data. One can try to use a region away from the core of the PSF; however, if the observed data does have any extended emission (dust halo, PWN, etc) then the result will be incorrect.
*In MARX 5.0.0, marxpileup introduces an offset on the source position, placing the source between 0.2-0.4 arcsec at 40 degrees from the readout streak, in the direction of the streak. A future MARX release will fix this issue.
The file may be viewed in ds9:
unix% ds9 acisf00942_repro_evt2.fits marx_output_i0000.fits &
This section will provide some examples of mistakes users will commonly make when attempting to use ChaRT and MARX
If ChaRT was run using an aspect asolution -- either by specifying the OBS_ID or by uploading a file, and the user fails to use that file when running MARX, the result will be a dithered PSF as shown in Figure 2.
In ChaRT 2, the exposure time of the simulation is limited by the length of the aspect solution. Users may therefore be tempted to increase the photon flux in the spectrum in order to generate more rays. This strategy can backfire especially if trying to model pileup.
The increased photon flux over the same time interval will be seen by MARX as an increase in the number of piled up events. Spatially this has the effect of depressing the core of the PSF as shown in Figure 3.
Users who require a large number of events in their PSF should generate multiple realizations and combine the image as discussed in the Creating an Image of the PSF thread.
Another common mistake is to attempt to simulate the PSF using the EDSER subpixel algorithm, marx2fits --pixadj=EDSER, without having supplied ChaRT an aspect solution.
The output file (marx_output_i0000.fits) is a pseudo-events file that can be used, for example, to create an image of the PSF.
|27 Jun 2003||original version, updated for CIAO 3.0: layout|
|02 Dec 2004||added note about setting ExposureTime parameter|
|16 Feb 2005||reviewed for CIAO 3.3: no changes|
|31 Jul 2007||updates for ciao3.4|
|18 Aug 2008||updated for CIAO 4.0: MARX v4.2.0; minor changes to screen output; image converted to inline|
|09 Oct 2008||updated to MARX v4.3.0|
|18 Feb 2009||updated to MARX v4.4.0|
|19 Mar 2009||updated MARX v4.4.0 syntax to find modified parameter file first|
|18 Feb 2010||updated to MARX v4.5.0; added Correcting for an Offset section; added mention of the MARX DataDirectory parameter|
|15 Dec 2010||reviewed for CIAO 4.3: no changes|
|22 Feb 2012||reviewed for CIAO 4.4: point to MARX 4.5 page|
|01 Oct 2012||reviewed for PSF site overhaul. Updates for some MARX 5.0 syntax needed for ChaRT v1 compatibility|
|29 Mar 2013||Updated file version from N003 to N004.|
|01 May 2013||Added a note about pileup and normalization.|
|10 Jul 2014||Added information about using the DY_AVG and DZ_AVG header values.|
|20 Aug 2014||Updated to use MARX v5.|
|11 May 2015||Updated for ChaRT v2. New information about aspect. New section about combining images from multiple iterations.|
|29 Feb 2016||Added a note about using install_marx script.|
|06 Apr 2016||Revised AspectBlur values.|
|11 Apr 2016||Added warning about installing MARX on OSX with default C compiler.|
|10 Jun 2016||Revised information about handling the SIM offset and added information about the MARX AspectBlur parameter.|
|16 Jun 2016||simulate_psf can now be used to project rays using marx.|
|28 Jul 2016||Added information about suggested ACIS-S AspectBlur.|