About the Sherpa Beta Release

There are several new features in this Beta release of Sherpa. However, since this is the initial phase of the Sherpa redesign, not all of the functionality of the CIAO 3.4 version is implemented yet. This document highlights new items and outlines missing features.

This Beta release supports fitting 1D X-ray spectra from Chandra, as well as some 1D non X-ray data, including ASCII data arrays and lightcurves. Simple 2D spatial fits should work with limited support for analyses that include PSFs, but have not been exhaustively tested. Grating data analysis are not yet supported.

The CIAO 3.4 version of the application will be available and fully supported until missing functionality is implemented in Sherpa 4.0.

Please send feedback and questions on Sherpa Beta to the Helpdesk.

New Features

Python and S-Lang environments

The Sherpa interface is available through both S-Lang and Python. Users can select in which language they prefer to do data analysis.

Python and S-Lang allow scripting, mathematical calculations, and access to libraries of scripts written by the community. The new ChIPS plotting package has a compatible interface, so that both applications can be used within the same environment. The Crates interface provides a way to read FITS and ASCII data into Python or S-Lang from files, retaining all the relevant metadata and providing CIAO Data Model filtering and binning capabilities.

Accessing and manipulating data structures

In addition to the normal mode of loading data from files, Sherpa allows you to manipulate the data arrays internally at the Python/S-Lang prompt and then "re-load" them before fitting and modelling further. The resulting data may be manipulated further without writing it out to a file, making it easy to incorporate into scripts.

Optimization methods

Three new optimization methods - LMDIF (a modification of the Levenberg-Marquardt algorithm, and so name "levmar"), neldermead (Nelder Mead Simplex direct search), moncar (moncar method based on the paper by Hedar & Fukushima) - are a complete replacement of the OPTIM routines. The algorithms are more robust and converge faster.

However, overhead in the Beta release makes the total run time longer, so users will not yet see the full benefit of this change.

Statistics

The XSPEC variance method has been added to the available statistics in Sherpa. Named "chi2xspecvar", it is a chi^2 with data variance = 1.0 if data less than or equal to 0.0.

Missing Functionality

A number of features not yet implemented in the current beta release of Sherpa. The CXC has not yet determined priorities or schedules for implementing and releasing this functionality.

Commands and algorithms

guess: for simple models, make an initial guess at parameters to ensure convergence.

projection: an initial implemention of project is available, based on the CIAO 3.4 approach, but may be improved in a later release.

Statistics

The "chi primini" and "bayes" statistics are not implemented.

Fitting

There is no support for RMFs and ARFs whose energy grids are not the same.

Support for simultaneous fitting with multiple different RMFs and ARFs is not correctly supported.

Support will be added for controlling whether certain models are calculated as integrated values over a bin or point values at a bin center (e.g. turning integration on or off).

Multiplication of two integrated models, each with its own normalization, is not handled correctly. It results in incorrect fit parameters and an incorrect number of counts in "fake_pha" simulations.

The sigma rejection method is not available.

Sherpa calculates the count errors internally using methods which depend on the chosen fit statistic. Using precalculated statistical errors provided in the PHA file is not supported.

The "ignore_bad" command does not currently ignore the data values with nonzero quality flags.

User interface

Sherpa will have the grouping functionality of dmgroup, but implemented so that the user can repeatedly try different grouping on their data "on the fly" to see how the fit is affected.

The routines in sherpa_utils.sl - e.g. calc_kcorr(), groupByCounts(), get_source_components() - will be implemented in the future.

An interface to the GUIDE atomic transition catalog and software will be provided.

The fit() command will automatically save the initial parameter values so that a new command, reset(), can restore them to those initial values after the fit.

Model parameter interface

In CIAO 4.0, the user must set soft limits and initial parameter values by hand. In the future, a default set of soft limits can be set automatically from the data for simple models, as in CIAO 3.4.

Plotting

Overplotting of fit and model components: the ability to overplot the contribution of individual model components on top of the best fit model will be implemented.

Sherpa and ChIPS no longer have the ability to do surface plots of 2D data. However, the support for contour plots is improved in the new version of ChIPS.

General infrastructure

The "save all" command creates a binary file that can be restored in Sherpa. In the future, it will be written as an ASCII file which can be inspected and edited by the user.

A simple function to list all Sherpa commands will augment the existing capabilities of the scripting environments to list all available functions (whether Sherpa or not).

Provide the ability to save fits in a format that may be manipulated by the user, similar to the CIAO 3.4 MDL files. The support in CIAO3.4 was not entirely satisfactory, and that approach will be reviewed.

Performance

The CXC will continue to work to improve performance, especially in the projection code, which is currently limited by algorithmic inefficiency and internal overheads.