CIAO 4.1 Release Notes

(27 Mar 2009) CALDB 4.1.2 has been released.

(15 Jan 2009) CIAO 4.1 has been patched to CIAO 4.1.1.

CIAO 4.1.1 Release Notes

CIAO 4.1 features a complete, non-beta version of Sherpa, the modeling and fitting package. There are several new features in this release of Sherpa and many improvements since Sherpa Beta; the About the Sherpa Release page highlights new items and outlines a few features still missing in comparison to Sherpa 3.4.

Other notable changes in CIAO 4.1 are:

• enhancements and bug fixes in the Data Model ASCII kernel. Both FITS and ASCII files are now accepted by most CIAO tools.

• several new tools, which were initially developed for use with the Chandra Source Catalog: aprates, dmellipse, eff2evt, lim_sens, mkpsfmap

• significant work has been done on the tool wavdetect to fix the bugs identified in CIAO 4.0: changing the falsesrc parameter value, a lower limit on the sigthresh value, and limitations in the PSF information

• the CIAO GUIs - prism, peg, and taskmonitor - have been rewritten in Gtk. The prism rewrite includes several enhancements, such as expanding vectors and arrays inline, tabbed frames to load multiple files into a single prism window, and a basic GUI for customizing plots.

• version 4 of the CALDB library, caldb4-module, and calquiz tool. CIAO tools have been upgraded to use the new library and calibration database.

• all the enhancements and bug fixes to the CIAO tools.

• due to simplifications made to the behavior of lines, labels, regions, and points in CIAO 4.1, the binary state files created by ChIPS in CIAO 4.0 can not be read into CIAO 4.1 (an error message about an incompatible version will be returned by when trying to do so).

• How CALDB 4.1.2 Affects Your Analysis
• How CALDB 4.1.1 Affects Your Analysis
• How CIAO 4.1 and CALDB 4.1 Affect Your Analysis
• Tools
• Parameter Files
• Data Model
• ChIPS
• Sherpa
• GUIs (Graphical User Interfaces)
• Libraries
• Environment
• Analysis Scripts
• Documentation

How CALDB 4.1.2 Affects Your Analysis

ACIS -120 C Imaging and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 November 2008 - 31 January 2009 (Epoch 36):

  acisD2008-08-01t_gainN0005.fits
  acisD2008-08-01t_gainN0006.fits
  acisD2008-11-01t_gainN0002.fits
  acisD2008-11-01t_gain_biN0002.fits
  

  The files are applicable to -120 C focal plane temperature only. The acisD2008-11-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0006 files showing T_GAIN energy shifts
  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

How CALDB 4.1.1 Affects Your Analysis

ACIS Imaging Data

• HRMA Effective Area (version N0008)

  The latest HRMA effective areas have been generated as a result of revisting the Chandra X-ray Calibration Facility (XRCF) ground calibration data and raytrace model, specifically because of known cross-calibration issues that have arisen within Chandra and between Chandra and XMM, when analyzing observations of high-temperature galaxy clusters. The new model predicts plasma temperatures that are more in agreement with XMM for simultaneous cluster observations. In addition it brings analysis of ACIS-I and ACIS-S observations of similar clusters more into agreement.

  The new file will be used by default when creating instrument response files (ARFs) or exposure maps. The CIAO tools and scripts affected by this are:

  • mkexpmap
  • mkarf
  • mkwarf
  • mkgarf
  • merge_all (uses mkexpmap)
  • specextract (uses mkwarf)
  • psextract (uses mkarf)
  • fullgarf (uses mkgarf)

  This plot shows the percent difference between the old N0007 (black line) and the new N0008 (red line) HRMA files:

  

  Users working in the 0-2 keV range will see a ~9% change in spectral fitting that uses an ARF file created with the new HRMA effective area.

  For instructions on how to continue using the vN0007 file, read the CALDB 4.1.1: HRMA Effective Area why topic.

ACIS Grating Data

• HRMA Effective Area (version N0008)

  The latest HRMA effective areas have been generated as a result of revisting the Chandra X-ray Calibration Facility (XRCF) ground calibration data and raytrace model, specifically because of known cross-calibration issues that have arisen within Chandra and between Chandra and XMM, when analyzing observations of high-temperature galaxy clusters. The new model predicts plasma temperatures that are more in agreement with XMM for simultaneous cluster observations. In addition it brings analysis of ACIS-I and ACIS-S observations of similar clusters more into agreement.

  The new file will be used by default when creating instrument response files (ARFs) or exposure maps. The CIAO tools and scripts affected by this are:

  • mkexpmap
  • mkarf
  • mkwarf
  • mkgarf
  • merge_all (uses mkexpmap)
  • specextract (uses mkwarf)
  • psextract (uses mkarf)
  • fullgarf (uses mkgarf)

  This plot shows the percent difference between the old N0007 (black line) and the new N0008 (red line) HRMA files:

  

  Users working in the 0-2 keV range will see a ~9% change in spectral fitting that uses an ARF file created with the new HRMA effective area.

  For instructions on how to continue using the vN0007 file, read the CALDB 4.1.1: HRMA Effective Area why topic. Note that if you choose to continue using the N0007 HRMA EA file, you must also use the N0006 HETG GREFF file.

• HETG Grating Efficiency (version N0006)

  The HETG grating efficiency file (GREFF) was updated to be consistent with the new N0008 HRMA EA file. The new file (N0006) will be used by default when creating HETG response files (ARFs). The CIAO tools and scripts which use this file are:

  • mkgarf
  • fullgarf (uses mkgarf)

  For instructions on how to continue using the vN0005 file, read the CALDB 4.1.1: HETG Grating Efficiency why topic. Note that if you choose to continue using the N0006 HETG GREFF file, you must also use the N0007 HRMA EA file.

How CIAO 4.1 and CALDB 4.1 Affect Your Analysis

Compatibility

• CALDB 4.1 is required in order for CIAO 4.1 to work correctly.

  It is not backward-compatible: CALDB 4.1 cannot be used with any CIAO software release prior to v4.1.

  A number of CIAO tools have been updated to be work with CALDB 4.1. Aside from the CTI_APP keyword issue discussed in the next section, the changes to these tools should be transparent to the user:

• ChIPS state files from CIAO 4.0 - i.e. the files created by "save_state (S-Lang or Python help)" and loaded by "load_state (S-Lang or Python help)" can not be loaded into CIAO 4.1. Please contact the CXC Helpdesk if this is a serious problem for you.

  Most ChIPS scripts written for CIAO 4.0 will continue to run in CIAO 4.1. However some code may need updating, as described in the ChIPS upgrade guide. Contact the CXC Helpdesk if you need help in updating CIAO 4.0 ChIPS scripts to CIAO 4.1.

ACIS CTI_APP Keyword Required

• CALDB 4.1 requires that all ACIS data files have a CTI_APP header keyword to indicate whether the ACIS CTI correction has been applied. The older CTI_CORR keyword is no longer used.

  The following CIAO tools and scripts are affected by this change:

  • acis_process_events
  • mkacisrmf
  • mkarf (via ardlib)
  • mkgarf (via ardlib)
  • mkgrmf (via ardlib)
  • mkinstmap (via ardlib)
  • mkrmf
  • mkwarf
  • psf_project_ray (via ardlib)
  • specextract
  • tg_resolve_events
  • acis_bkgrnd_lookup (contributed script)
  • acis_fef_lookup (contributed script)
  • psextract (contributed script, calls acis_fef_lookup)

  These tools will either fail or return incorrect results if the CTI_APP header keyword is missing.

  Adding the CTI_APP keyword

  If your dataset is old enough that it doesn't have a CTI_APP keyword, consider running the Reprocessing Data to Create a New Level=2 Event File thread to be sure that the newest calibration has been applied to the file.

  If you don't wish to reprocess, run the CIAO contributed script, check_ctiapp.sh, to update the file header:

  unix% check_ctiapp.sh acis_evt2.fits
  Setting CTI_APP value to PPPPPNPNPP
  

  It is also simple to edit the file header to add a CTI_APP keyword; the following is the same as running check_ctiapp.sh.

  The CTI_CORR value is a boolean: "TRUE" (or "1") if the CTI has been applied and "FALSE" (or "0") if it hasn't.

  • If CTI_CORR=yes, CTI_APP=PPPPPNPNPP:

    unix% dmkeypar acis_evt2.fits CTI_CORR echo+
    1
    
    unix% dmhedit acis_evt2.fits file= op=add key=CTI_APP value=PPPPPNPNPP
    

  • If CTI_CORR=no or the keyword doesn't exist in the file, CTI_APP=NNNNNNNNNN:

    unix% dmkeypar acis_evt2.fits CTI_CORR echo+
    # dmkeypar (CIAO 4.1): ERROR: Keyword 'CTI_CORR' was not found in file 'acis_evt2.fits'.
    
    unix% dmhedit acis_evt2.fits file= op=add key=CTI_APP value=NNNNNNNNNN
    

ACIS -120 C Imaging and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2008 - 31 October 2008 (Epoch 35):

  acisD2008-05-01t_gainN0005.fits
  acisD2008-05-01t_gainN0006.fits
  acisD2008-08-01t_gainN0002.fits
  acisD2008-08-01t_gain_biN0002.fits 
  

  The files are applicable to -120 C focal plane temperature only. The acisD2008-08-01t_gain_biN0002.fits file is the new default processing file in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0006 files showing T_GAIN energy shifts
  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

Change in Directory Structure

• The directory structure has changed in CALDB 4 to eliminate the "bcf" and "cpf" subdirectories in each instrument directory. Any scripted analysis task which contains a CALDB path will fail unless it is updated for the new structure.

  For example, in CALDB 3, the acis directory contains:

  unix% ls $CALDB/data/chandra/acis/
  bcf/  caldb.indx@  cpf/  index/
  
  unix% ls $CALDB/data/chandra/acis/bcf/
  badpix/  contam/  dead_area/  evtsplt/  grade/   qe/
  bkgrnd/  cti/     disp_reg/   gain/     gtilim/  t_gain/
  
  unix% ls $CALDB/data/chandra/acis/cpf/
  2dpsf/  fefs/  osip/  p2_resp/  pimms/  rmf/
  

  and in CALDB 4, the acis directory structure is:

  unix% ls $CALDB/data/chandra/acis/
  2d_psf/  caldb.indx@  dead_area/  evtsplt/  gti_lim/    lsfparm/  qe/
  badpix/  contam/      det_gain/   fef_pha/  index/      osip/     qeu/
  bkgrnd/  cti/         disp_reg/   grade/    key.config  p2_resp/  t_gain/
  

Tools

acis_build_badpix

• The ability to specify "calibfile="CALDB"" has been removed from this version. Users must specify the name of the pipeline-created badpixel file or query the CALDB via calquiz to set the filename.

acis_find_hotpix

• Bug Fix: the tool would fail if the badpixfile parameter is set to "NONE" or "CALDB". Now a warning is issued.

acis_process_events

• Two separate pulse-height randomizations are performed in acis_process_events. One randomization is performed when the value of PHA is calculated after the time-dependent gain adjustment is applied. The other randomization is performed when the value of ENERGY is calculated from the value of PHA. In previous versions of acis_process_events, only the second of these two randomizations was controlled by the parameter rand_pha. Now both randomizations are controlled by this parameter.

• The parameters cclev1 and ccgrdlev1, which are the default eventdefs for the regular (CC33_FAINT) and graded (CC33_GRADED) continuous-clocking modes, have been expanded to include "d:time_ro." As a result, the output event files include the read-out times (TIME_RO) in addition to the times of arrival at the detector (TIME).

• The following warning message is no longer printed at verbose=0:

  # acis_process_events: The following error occurred NNNNNN times:
  dsAFEBADPCNTERR -- WARNING: Event island contains 1 or more bad pixels.
  

  since these occurrences are routine and should not be a concern in data analysis.

• A number of warning messages have been upgraded to be more informative.

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

acisreadcoor

• The tool no longer copies additional extensions (e.g. the REGION block from an evt1a.fits file). This behavior is consistent with other CIAO tools.

aprates

• New Tool!

  The aprates tool computes values and bounds for source intensity quantities (net counts, source rate, photon flux, energy flux) using counts and exposure data obtained in source and background apertures. The Compute Net Counts, Rate, or Flux for Point Sources thread shows how to use the tool.

  This tool was initially developed for use with the Chandra Source Catalog.

axbary

• The leapsec file has been updated

calquiz

• New Tool!

  The calquiz tool is the CALDB 4 version of the tool quizcaldb. It is used to query the CALDB for calibration files which match the given set of conditions.

destreak

• If there is no EXPTIME header keyword, the value "TIMEDEL-0.04104" is used.

dmellipse

• New Tool!

  The dmellipse tool finds the best-fit ellipse (or rotated box) that encloses the specified fraction of flux to within a given tolerance.

  This tool was initially developed for use with the Chandra Source Catalog.

dmgti

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

dmimg2jpg

• Bug Fix: if the input image doesn't have a WCS and showgrid=yes, the tool will print an error message.

dmreadpar

• Bug Fix: file parameters that are "(f)ile" type are written as strings into the header.

• If dmreadpar cannot convert a value, it skips that value and prints a warning. Specifically, unsigned long values will trigger the warning.

dmregrid

• In approximation mode, i.e. "npts!=0", dmregrid would skip pixels with negative values. The tool now allows values < 0 to be regridded.

dmstat

• A verbose parameter has been added to dmstat. The default, verbose=1, replicates the previous tool behavior. Setting verbose=0 turns off all screen output.

eff2evt

• New Tool!

  The eff2evt tool calculates the energy flux in ergs/cm**2/sec for each input event, taking the quantum efficiency (QE) and effective area (EA) into account. The QE, EA, and flux values are recorded in new columns in the output file. The Calculate the Flux for a Position thread shows how to use the tool.

  This tool was initially developed for use with the Chandra Source Catalog.

hrc_build_badpix

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

hrc_process_events

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

lim_sens

• New Tool!

  The lim_sens tool creates a limiting sensitivity map from background and PSF files. Limiting sensitivity is the flux of a point source that meets but does not exceed the flux significance threshold for detection. An exposure map may also be supplied, in which case lim_sens also takes into account the ratio of background to source.

  The units of the map are [photons/cm^2/sec] if an exposure map is input and [counts] if the exposure map is not provided.

  This tool was initially developed for use with the Chandra Source Catalog.

mkacisrmf

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

mkarf

• The tool has been upgraded to support CALDB4 (via the ardlib). The changes should be transparent to the user.

mkexpmap

• Bug Fix: the tool would produce slightly different results on OS-X Intel due to a double to unsigned integer cast.

mkgarf

• The tool has been upgraded to support CALDB4 (via the ardlib). The changes should be transparent to the user.

mkgrmf

• The tool has been upgraded to support CALDB4 (via the ardlib). The changes should be transparent to the user.

mkinstmap

• The tool has been upgraded to support CALDB4 (via the ardlib). The changes should be transparent to the user.

mkpsfmap

• New Tool!

  The mkpsfmap tool computes the PSF size for each pixel in the input image for a given energy and enclosed counts fraction (ECF) value.

  This tool was initially developed for use with the Chandra Source Catalog.

mkrmf

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

mkwarf

• Bug Fix: the tool only opens grating modules once, decreasing the run time and preventing 1000's of history records from being written in the header.

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

mtl_build_gti

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

psf_project_ray

• The tool has been upgraded to support CALDB4 (via the ardlib). The changes should be transparent to the user.

quizcaldb

• The quizcaldb tool has been replaced in CIAO 4.1 by the calquiz tool.

reproject_aspect

• reproject_aspect is a script which runs wcs_match and wcs_update; refer to the release notes for those tools as well.

• Bug Fix: the error status is now set to 0 when wcs_update fails.

• Bug Fix: the output file was missing a CHECKSUM keyword.

reproject_events

• Bug Fix: the tool is able to run on HRC data without a TIME column. reproject_events will now look for CHIP_ID instead of CCD_ID in this case.

specextract

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

tg_create_mask

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

tg_resolve_events

• New columns may be used in custom eventdefs, if desired:

  • individual l:rawx and l:rawy
  • l:pha_ro (propagated from infile)
  • f:chipy_tg (cc mode chipy along grating arm)
  • f:chipy_zo (cc mode chipy for zeroth order)

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

• The following warning message is no longer printed:

  # tg_resolve_events: The following error occurred 47000 times:
  dsTREUNKNOWNINCOLERR -- WARNING: Not loading data from unrecognized level 1.5 input column. 
  

  since this is not a problem for the data analysis.

tgextract

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

tgextract2

• The tool has been upgraded to support CALDB4. The changes should be transparent to the user.

wavdetect

• wavdetect is a script which runs wrecon and wtransform; refer to the release notes for those tools.

wcs_match

• Bug Fix: the A21 column in the output file has the correct description "Transform Matrix element A21".

• Bug Fix: the tool sets the status value to non-zero if unable to create the output file.

wcs_update

• Bug Fix: wcs_update looks for the block ASPSOL (uppercase), but writes it out as aspsol (lowercase). The tool wouldn't recognize the aspect solution block name when run on that file again. The tool is now case-insensitive regarding the extension name.

wrecon

• If the psffile parameter is unspecified for Chandra data, the smallest wavelet scale is used. This is useful for merged datasets where the DETNAM value is "Merged".

• If the source region determined by wavdetect is only one pixel wide, then the axis in that direction is equal to 0. If the entire region is one pixel square, then both the semi-major and minor axes are (0,0). These radii are now output as a float to preserve the major and minor axes.

• If the source is more than 20 arcminutes off-axis (theta > 20'), the PSF size at 20' is used to compute source properties.

wtransform

• The previous root-finding routine was replaced with the GNU Scientific Library (GSL) make the "falsesrc" parameter more stable.

• Bug Fix: a bug was fixed where the number of iterations was exceeded and a warning was generated.

• Bug Fix: fixed a problem when a sigthresh values off the table resulted in an error.

Parameter Files

ciao.par

• The ciao.par parameter file has been removed from CIAO 4.1.

dmstat

• A verbose parameter has been added to dmstat. The default, verbose=1, replicates the previous tool behavior. Setting verbose=0 turns off all screen output.

Data Model

Bug Fixes

• Region values which used to be padded with 0 (e.g. POS, R) are now padded with NaN. This change may be seen in detect source lists, skyfov output, dmcontour output, etc.

• Block names may contain blank spaces (e.g. "SPECRESP MATRIX").

• Subspace order of the original component set is preserved. Previously, the order would change to reflect that of the new filter.

• If the specified filter file is missing, e.g. "sky=region(fov1.fits[ccd_id=7])", the error message will indicate that this why the command failed.

• Bug Fix: ability to update the BUNIT keyword.

• Bug Fix: a message about string columns being too long and being truncated was being issued when this was not the case.

• Bug Fix: SegV seen when accessing 1D image data.

• Bug Fix: possible memory overflow when generating long comment key.

• Bug Fix: SegV when trying to access a virtual vector array column as scalars. (e.g. through dmkeypar).

• Bug Fix: SegV seen when writing a very large image file. When preparing the output buffer, the total size of the requested buffer exceeded the limit of the datatype used for the calculation (unsigned long), resulting in insufficient buffer space being allocated.

• DM now supports the definition of String arrays using TDIM keywords in the FITS kernel. This change required the release of cfitsio version (>= v3.04) which supports this syntax:

   TFORM  A
   TDIM
  

  Previously was:

   TFORM  A "
  

ASCII Kernel

• New: support for binning table columns into ascii image.

• New option: "nullstr" allows the user to define what the string representation for NULL is in the ASCII file (input and output). This option recognizes TNULL keywords in DTF files.

• Modified the representation of "bit" type data in ascii files. The bit type has been read/write as an array of byte values. However, user expectation (and dmlist presentation) is to show them as a bit pattern of 1s and 0s. The ASCII kernel now reads/writes bit data as a bit string.

• Bug Fix: if there was whitespace surrounding a value and the "white" option is not specified, the ASCII kernel set the column datatype to "string".

  Whitespace in a colspec line with a non-space separator is also handled correctly.

• Either single or double quotes may be used within option strings (e.g. [opt sep=","] or [opt sep=',']).

• Bug Fixes for DTF-format files:

  • keyword comments are copied to the output file.
  • recognize the "END" keyword even if it contains trailing blanks.
  • recognize unit information stored in keyword comment section
  • recognizes TDNULL value for string type columns.
  • PHYSICAL WCS information is not written to the header if it does not exist in the input file.
  • The header keyword "NAXIS1" is written to the output header of a DTF-FIXED Table block.
  • Added support for FIXED format tables to define columns where there is no separator between them (i.e. abutting).

• Check for and report duplicate keywords when reading the header.

• ASCII files with no data (only header) are handled correctly.

• Bug Fix: "colnames=none" for text/simple input file no longer overridden by "colnames=first".

Crates Library

• Change to the defaults of print_col_names(), get_colnames(), and C-level get_col_names(): only regular columns are shown by default; need to indicate the virtual columns should be displayed as well.

• New function: is_virtual() checks if the specified column is virtual.

• New function: copy_values() returns a copy of CrateData values.

• Crates sets an Error message and status when attempting to read a non-existent file. IMAGECrate exits with an error if given a non-image file.

• write_rmf produces variable length array columns as appropriate for the file according to standards.

• Added support for vectors of array columns.

• Enable users to extract values directly from a single component of a virtual vector column. Formerly, could only extract values for the full vector.

• An optional boolean flag was added to the PHACrate constructor which enables a user to distinguish the source and background spectrum of a Chandra Source Catalog level=3 PHA file.

• nD array values are now indexed as C-style arrays.

• The LINEAR2D was integrated into Crates.

• The TRANSFORM type check is now case-insensitive.

• Bug Fix: RMFCrate writes the MATRIX extension before EBOUNDS extension.

• Bug Fix: corrected a problem writing array column data, particularly to ASCII format output.

• Bug Fix: vector columns whose names were obtained from get_colnames() could not be retrieved using get_column(). Crates now accomodates the syntax produced by get_colnames for vectors, e.g. "pos(x,y)" for the vector column "pos".

• Bug Fix: properly handle empty keyword strings in get_key_value()

• Bug Fix: added get_data_values(long index) to Crate API.

• Bug Fix: corrected handshaking problem for string and logical type data between Python and the underlying DM code.

• Improved Python to C-Layer memory management for vector data. Vector data returned by Crates now retains linkage to the underlying C memory.

Transform Library

• Added support for the TAN-P projection

• Corrected display of Boolean type TransParam objects on Mac PPC.

  • improved interface to LINEAR2D::get_transform_matrix() so that properly updated/(re)calculated matrix array is always returned.
  • removed the delete matrix commands from the LINEAR2DTransform class.
  • occasional segV from freeing uninitialized memory.

• Bug Fix: enable copy of generic Transform object to the specialized WCSTAN or LINEAR2D Transform types.

ChIPS

Startup Options

• There are four new command-line options which may be specified when ChIPS is started:

  • -x : launch chips/sherpa shell in separate display terminal.
  • -b : run in batch mode
  • -l : specifies which language to use, "slang" or "python".
  • -n : do not print banner

  The same options may be used with Sherpa.

CIAO 4.0 State Files

• Due to simplifications made to the behavior of lines, labels, regions, and points in CIAO 4.1, the binary state files created by ChIPS in CIAO 4.0 can not be read into CIAO 4.1 (an error message about an incompatible version will be returned by load_state). The threads have been updated to describe this issue.

GUI for printing plots

• If the preference "export.printdialog" is set to true, ChIPS will open a printing dialog box when print_window is called. The dialog allows you to decide whether to send the figure to the printer or to a file, and allows you to set all the printing options available in the print_window (S-Lang or Python help) command (e.g. output format, color scheme, page margins).

  An example of the GUI is shown in the Introductory ChIPS thread (S-Lang or Python).

Interactively change the limits of a plot

• A new command, pick_limits (S-Lang or Python help), allows the user to interactively set the axes ranges for the x and/or y dimensions of a plot. After issuing the command, the user left-clicks on the plot and drags to draw a bounding box (i.e. keeps the mouse button pressed down). When the mouse button is released, the plot axes limits are filtered to match the x and/or y ranges contained within the box.

Fill style for regions and histograms

• The attribute "fill.style" replaces the "fill.visible" attribute for regions and histograms. There are multiple fill styles available, instead of a boolean filled/not filled; see the Fill Pattern section of "ahelp chipsopt" (S-Lang or Python help). A value of 0 (chips_nofill) is equivalent to fill.visible=0; a value of 1 (chips_solidfill) is equivalent to fill.visible=1 .

  The new command load_fill (S-Lang or Python help) enables users to define a custom fill pattern.

• Bug Fix: the histogram and region fill colors when set to "default" were not being changed from a display foreground color to a postscript foreground color.

New symbols in labels

• Chips now includes support for the following latex symbols: \sun, \odot, \mercury, \mercury, \venus, \earth, \mars, \jupiter , \saturn , \uranus, \neptune, \pluto, \circ, \textdegree, \int , \pentagon, \hexagon , \ScissorRight, \Asterisk, \FiveStar, \SixStar , \leftmoon, \rightmoon, \maltese , \rightturn, \smiley, \eighthnote , \hbar

  Additionally, grave, acute, circumflex, umlaut, tilde, and cedilla accents are supported.

Contours

• The default contouring algorithm has been changed to be marching squares since it is a faster algorithm than the standard.

• The attribute "levels" which refers to the requested number of levels for a given contour in count mode has been changed to "numlevels." When setting the attribute, "levels" now refers to a list of exact levels the user wants the contour to display (when the mode is set to "arbitrary"). When using get_contour(), the "levels" attribute contains the numeric values of the contour levels.

Annotations

• A change in the annotation hierarchy means that lines, labels, regions, and points now belong to a plot and not a frame.

• Added the ability to set the coordinate system from the attribute list for annotations and axes when they are first created.

• clear_plot (S-Lang or Python help) now clears all annotations created in a particular plot, instead of just those in plot_norm and data coordinates.

Undo command: bug fixes

• If the dataset contained negative numbers, undoing a limits command which is in log scale would result in an incorrect axis range.

• When creating a strip chart with at least three plots - where the middle plots are in log scale - undoing then redoing the strip_chart command would result in incorrect axis ranges. A similar issue would happen with certain bound axes.

• When undoing the creation of a plot, the counter in the plot id wasn't being reset.

• Setting an undo tag in rare circumstance would cause chips to act strangely when the tag is undone. It would either lock the display of the window or return an error message saying that there were no more commands to be undone.

• This change addresses an issue dealing with preference values not being written out to the state files. The issue becomes apparent when the undo/redo commands are flushed out to file and then the file is reaccessed via undo commands.

Running ChIPS in batch mode

• If ChIPS is invoked with the batch-mode option, e.g. "chips -b", the "window.display" preference will be set to false. To have the display on, include a set_preference call at the beginning of the script to set "window.display" to true. In addition, if a connect command is issued within the script, it will automatically disconnects from the chips server if it is attached to one.

Loading Advanced ChIPS Functions

• A number of ChIPS functions were moved into a separate module of advanced commands. These functions are primarily those whose capability is available from a higher-level ChIPS command; for instance, set_line_color() was moved because its functionality is available from the set_line() function. More advanced undo/redo commands (e.g. using undo and redo buffers) and software error handling commands were moved to this module as well.

  Users who wish to access the advanced commands simply have to load the additional module into ChIPS.

  • In Python:

    from pychips.advanced import *
    

  • In S-Lang:

    require("chips_advanced");
    

Additional Enhancements & Bug Fixes

• The default page margins were change to .2 to account for the margin which normally exists on printers.

• The connect function has been updated so that if no server id is specified it will attempt to connect to an existing server. If multiple servers or no servers are running, an error is returned.

• Bug Fix: a failure would occur if an invalid value was supplied to set_point_style. The command now errors out correctly.

• The ability to set border axes as the current axis has been removed, as it will cause problems with coordinate systems.

• Bug Fix: the erase command would fail to work if an erase after a previous erase was executed and undone.

• Bug Fix: an id specified in the attribute list during an add_line call would not be used, resulting in lines with autogenerated ids.

Sherpa

Startup Options

• There are four new command-line options which may be specified when Sherpa is started:

  • -x : launch chips/sherpa shell in separate display terminal.
  • -b : run in batch mode
  • -l : specifies which language to use, "slang" or "python".
  • -n : do not print banner

  The same options may be used with ChIPS.

GUIs (Graphical User Interfaces)

prism

• prism has migrated from Motif to gtk. The gui now utilizes gtk themes to control the appearance and color of windows and various aspects of the gui. Preferences for prism originally loaded from $HOME/.CXCdefaults are no longer used. $HOME/.gtkrc.prism is now used to provide prism specific preference directives.

• prism now expands vectors and arrays inline. Previously, vectors and arrays were expanded into separate gui windows.

• Tabbed frames allow a single instance of prism to load multiple files. Previously, prism could only load a single file at a time.

• Toolbars have been added to the header and data windows to provide quick access to related functionality.

• The mouse interactions have been reworked so the left button is now used to select or deselect a row, column, or cell for editing. The original prism mouse interactions used the left button for column selection, the middle button for row selection, and <control> plus the left mouse button for selecting a cell to edit.

• The resource file has been renamed from .prism.prefs to .prismrc. The format of the file is unchanged.

• The Session pulldown menu is no longer available in prism.

peg

• The peg GUI has been rewritten in GTK, but functions the same as it did in CIAO 4.0.

taskmonitor

• The taskmonitor GUI has been rewritten in GTK, but functions the same as it did in CIAO 4.0.

ds9 and dax

• ds9 v5.4 is packaged with CIAO 4.1.

• New Tool!

  dax is a suite of scripts that sets up several CIAO tasks to be run directly from the ds9 Analysis menu. The commands are stored in the file $ASCDS_INSTALL/config/ciao.ds9 and are automatically loaded when ds9 is launched within CIAO.

  The following tools are used by dax:

  • dmellipse
  • dmimgadapt
  • dmimgblob
  • dmimgfilt
  • dmimglasso
  • dmimgproject
  • dmimgreproject
  • dmmaskbin
  • dmmaskfill
  • dmnautilus

  Note that dax and all the tools it uses are being released as beta in CIAO 4.1.

Libraries

caldb4 and caldb4-module

• The caldb4 library and the caldb4-module used to interface with that library have both been updated to be compatible with CALDB4. Previous versions were named cxccaldb and caldb-module.

dmtlib

• The support library "dmtlib" has been removed from CIAO 4.1.

dslib

• The "setting FITS kernel" message is no longer printed when the event file doesn't end in .fits or .qp.

• If there is only one block in the file, just the primary header is updated. This fixes some problems seen with the DM ASCII kernel.

pixlib-module_slang

• The new function pix_get_fp_scale_in_asec returns the focal-plane pixel size in arcsec for the current detector.

• If the location is off the chip, the pix_fpc_to_chip function uses extended coordinates.

regionlib

• A polygon is closed when x[n] == x[0] && y[n] == y[0]. Previously, there was the extra constraint that x[n+1] == 0 and y[n+1] == 0. This change allows NaN-filled regions to be used.

tcd-module

• This new module is a S-Lang wrapper for tcdCorrelate.

Environment

IPython

• CIAO 4.1 includes IPython in the CIAO OTS directory which is used by Sherpa and ChIPS to provide command-line user interfaces. These programs create an IPython profile in the directory $HOME/.ipython-ciao.

  If IPython users want any personal customizations to be available when running CIAO, they will have to copy them from $HOME/.ipython to $HOME/.ipython-ciao.

Eliminated ots/bin from the users path

• By using wrapper scripts for the command line tools needed by CIAO in ots/bin, ots/bin is not longer added to the users PATH variable when CIAO is sourced.

  The executables in ots/bin are also no longer wrapped in scripts.

.ciaorc configuration file

• The new $HOME/.ciaorc file allows users to control the versions of Python, iPython, ds9, slsh, and xpa used with CIAO. If the users do not want to use the copy of these tools packaged with CIAO, they now have the option of using their own local copies

  By default, the versions packaged with CIAO are used. Refer to "ahelp ciaorc" for more information.

Building from source and building against CIAO

• Updated configure to be more robust in finding X libraries and includes when building from source.

• A new script, ciaorun, allows users who develop their own tools with the CIAO infrastructure to run without creating a wrapper script. This script can also be used to run the binaries packaged in ots/bin if desired.

• A new script, ciao-python-fix, allows users to re-generate python modules in their build to allow faster run-time execution.

Off-the-Shelf (OTS) Package Versions

• The following OTS packages are included with CIAO 4.1. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.

  • cfitsio v3.10
  • DS9 v5.4
  • fftw v3.1.2
  • ghostscript v8.56
  • gtkglext v1.0.6
  • GTK v2.12 *
  • iPython v0.8.4
  • jpeg v6b
  • numpy v1.2.0
  • pysl v0.9.0
  • Python v2.5.2
  • readline v5.2
  • Slang v2.1.4
  • VTK v5.0.2
  • wcssubs v3.7.0
  • xerces-c v2.8.0
  • xpa v2.1.9
  • Xspec v12.4.0ao

  * GTK consists of several sub-packages: pkg-config v0.18.0, gettext v0.17, glib v2.16.6, ATK v1.20.0, pixman v0.12.0, FreeType v2.3.7, expat v2.0.1, libxml v2-2.6.31, tiff v3.8.2, png v1.2.32, fontconfig v2.3.92, cairo v1.6.4, Pango v1.19.4, GTK+ v2.12.12

Analysis Scripts

acis_bkgrnd_lookup vCIAO 4.1 - 1.0

• The script has been rewritten to be compatible with CALDB 4.1. Images are now supported as the input file; the script functionality is otherwise unchanged.

acis_fef_lookup vCIAO 4.1 - 1.0

• The script has been rewritten to be compatible with CALDB 4.1. A new "quality" parameter indicates whether the RMF should be remade with mkacisrmf (no= use mkacisrmf). Otherwise the script functionality is unchanged.

check_ctiapp.sh v1.0

• A new script, check_ctiapp.sh, adds the CTI_APP header keyword to a file, using the value of the CTI_CORR header keyword to set the correct value.

  unix% check_ctiapp.sh acis_evt2.fits
  Setting CTI_APP value to PPPPPNPNPP
  

lightcurve scripts vCIAO 4.1 - 1.0

• The scripts lc_clean and analyze_ltcrv have been updated and combined into a single script for CIAO 4.1. The new script is available for both Python and S-Lang: lightcurves.sl and lightcurves.py.

  The two analysis routines are named lc_clean and lc_sigma_clip (formerly analyze_ltcrv). The functionality is the same as in CIAO 4.0, and the ChIPS plotting has been restored.

Documentation

CALDB 4 Website

• The CALDB website has been revised for the CALDB 4 release: http://cxc.harvard.edu/caldb.

  For information on CALDB 3: http://cxc.harvard.edu/caldb3.

Ahelp

• The ahelp files for Sherpa and ChIPS contain information for both the supported languages: python and s-lang. The value of the CIAO_SCRIPT_LANG environment variable is used to determine which version of a help file to display when calling ahelp from the CIAO command line (i.e. not from withing ChIPS or Sherpa).

  Refer to "ahelp ciaorc" for information on how the CIAO_SCRIPT_LANG environment variable is set.

  If no ahelp file is found, the resulting message indicates what context was searched:

  unix% ahelp dmilst
  CXC Help: No info found for subject 'dmilst' context '/py[.]*/'
  

Analysis Threads

• The CIAO Science Threads are in the process of being tested and updated for the CIAO 4.1 release. Please check the "Last Update" information at the top of each thread to see if it has been reviewed yet.

Obsolete Threads

• The following obsolete threads have been removed from the website. The information in these threads is available in the Reprocessing Data to Create a New Level=2 Event File thread, as well as the CIAO Why Topics.

  • Apply the Time-Dependent ACIS Gain Correction
  • Apply the ACIS CTI Correction
  • Apply an ACIS Gain Map
  • Remove ACIS Pixel Randomization
  • Apply/Remove ACIS PHA Randomization
  • Calculate CC-mode Times of Arrival
  • Destreak the ACIS-S4 Chip
  • HRC AMP_SF Correction and Reducing Tap-Ringing Distortions
  • HRC-I Degap Correction
  • HRC-S Degap Correction