Sherpa Statistics

The following statistics are available in Sherpa:

• cash
• χ² statistics:
  • chi2constvar (CHI PARENT | CVAR)
  • chi2datavar (CHI DVAR)
  • chi2gehrels (CHI GEHRELS)
  • chi2modvar (CHI MVAR)
  • chi2xspecvar
• cstat

cash

A maximum likelihood function.

Counts are sampled from the Poisson distribution, and so the best way to assess the quality of model fits is to use the product of individual Poisson probabilities computed in each bin i, or the likelihood :

where M_i =S_i +B_i  is the sum of source and background model amplitudes, and D_i  is the number of observed counts, in bin i.

The Cash statistic (Cash 1979, ApJ 228, 939) is derived by (1) taking the logarithm of the likelihood function, (2) changing its sign, (3) dropping the factorial term (which remains constant during fits to the same dataset), and (4) multiplying by two:

The factor of two exists so that the change in Cash statistic from one model fit to the next, ΔC, is distributed approximately as Δχ² when the number of counts in each bin is high (> 5). One can then in principle use ΔC instead of Δχ² in certain model comparison tests. However, unlike χ², the Cash statistic may be used regardless of the number of counts in each bin.

The magnitude of the Cash statistic depends upon the number of bins included in the fit and the values of the data themselves. Hence one cannot analytically assign a goodness-of-fit measure to a given value of the Cash statistic. Such a measure can, in principle, be computed by performing Monte Carlo simulations. One would repeatedly sample new datasets from the best-fit model, and fit them, and note where the observed Cash statistic lies within the derived distribution of Cash statistics. (The ability to perform Monte Carlo simulations is a feature that will be included in a future version of Sherpa.)

Note on Background Subtraction

The background should not be subtracted from the data when this statistic is used. It should be modeled simultaneously with the source.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa-1> set_stat('cash')
   sherpa-2> print get_stat_name()
   cash
   

χ² statistics:

Chi-square statistic.

The chi-square statistic is

where N_i,S  is the total number of observed counts in bin i of the on-source region; B_i (x_i , p_B ) is the number of predicted background model counts in bin i of the on-source region (zero for background-subtracted data), rescaled from bin i of the off-source region, and computed as a function of the model argument x_i  (e.g., energy or time) and set of background model parameter values p_B ; S_i (x_i , p_S ) is the number of predicted source model counts in bin i, as a function of the model argument x_i  and set of source model parameter values p_S ; and σ_i  is the error in bin i.

N_i,B  is the total number of observed counts in bin i of the off-source region; A_B  is the off-source "area," which could be the size of the region from which the background is extracted, or the length of a background time segment, or a product of the two, etc.; and A_S  is the on-source "area."

In the analysis of PHA data, A_B  is the product of the BACKSCAL and EXPTIME FITS header keyword values, provided in the file containing the background data. A_S  is computed similarly, from keyword values in the source data file.

Note that in the current version of Sherpa, it is assumed that there is a one-to-one mapping between a given background region bin and a given source region bin. For instance, in the analysis of PHA data, it is assumed that the input background counts spectrum is binned in exactly the same way as the input source counts spectrum, and any filter applied to the source spectrum automatically applied to the background spectrum. This means that currently, the user cannot, e.g., specify arbitrary background and source regions in two dimensions and get correct results. This will be changed in a future version of Sherpa.

(However, this limitation only applies when analyzing background data that have been entered with the load_bkg command. One can always enter the background as a separate dataset and jointly fit the source and background regions.)

chi2constvar

Chi-square statistic with constant variance computed from the counts data. (CHI PARENT | CHI CVAR in CIAO 3.4)

In some applications, analysts have seen fit to assume that the variance is constant for each bin. For this choice of statistic, the variance is assumed to be the mean number of counts, or

where N is the number of on-source (and off-source) bins included in the fit. The background term appears only if a background region is specified and background subtraction is done.

See the section on χ² statistics for more information, including definitions of the additional quantities shown in the equation.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa> set_stat('chi2constvar')
   sherpa> print get_stat_name()
   chi2constvar
   

chi2datavar

Chi-square statistic with variance computed from the data. (CHI DVAR in CIAO 3.4)

If the number of counts in each bin is large (> 5), then the shape of the Poisson distribution from which the counts are sampled tends asymptotically towards that of a Gaussian distribution, with variance

The background term appears only if a background region is specified and background subtraction is done. See the section on χ² statistics for more information, including definitions of the additional quantities shown in the equation.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa-1> set_stat('chi2datavar')
   sherpa-2> print get_stat_name()
   chi2datavar
   

chi2gehrels

Chi-square statistic with the Gehrels variance function.

This is the Sherpa default statistic.

If the number of counts in each bin is small (< 5), then we cannot assume that the Poisson distribution from which the counts are sampled has a nearly Gaussian shape. The standard deviation (i.e., the square-root of the variance) for this low-count case has been derived by Gehrels (1986):

Higher-order terms have been dropped from the expression; it is accurate to approximately one percent. If one does not perform background subtraction, then σ_i =σ_i,S ; otherwise, one may use standard error propagation to estimate that

The background term appears only if a background region is specified and background subtraction is done. See the section on χ² statistics for more information, including definitions of the additional quantities shown in the equation.

Note on Background Subtraction

We have not determined the accuracy of the latter expression, thus the user should proceed with caution when subtracting background from the raw data when using this statistic. An approach preferable to background subtraction is to model the background and data simultaneously.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa-1> set_stat("chi2gehrels")
   sherpa-2> print get_stat_name()
   chi2gehrels
   

chi2modvar

Chi-square statistic with variance computed from model amplitudes. (CHI MVAR in CIAO 3.4)

This statistic is equivalent to chi2datavar , except that the variance is estimated using the background and source model amplitudes rather than the observed counts data:

where B_i,off  is the background model amplitude in bin i of the off-source region. See the section on χ² statistics for more information, including definitions of the additional quantities shown in the equation.

Note on Background Subtraction

The background should not be subtracted from the data when this statistic is used. chi2modvar underestimates the variance when fitting background-subtracted data.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa-1> set_stat("chi2modvar")
   sherpa-2> print get_stat_name()
   chi2modvar
   

chi2xspecvar

Chi-square statistic with variance computed from data amplitudes.

This statistic is equivalent to chi2datavar , except that when the number of counts in a channel bin is less than 1 the variance is set to 1.

cstat

A maximum likelihood function.

The CSTAT statistic is equivalent to the XSPEC implementation of the Cash statistic.

Counts are sampled from the Poisson distribution, and so the best way to assess the quality of model fits is to use the product of individual Poisson probabilities computed in each bin i, or the likelihood :

where M_i =S_i +B_i  is the sum of source and background model amplitudes, and D_i  is the number of observed counts, in bin i.

The CSTAT statistic (Cash 1979, ApJ 228, 939) is derived by (1) taking the logarithm of the likelihood function, (2) changing its sign, (3) dropping the factorial term (which remains constant during fits to the same dataset), (4) adding an extra data-dependent term, and (4) multiplying by two:

The factor of two exists so that the change in CSTAT statistic from one model fit to the next, ΔC, is distributed approximately as Δχ² when the number of counts in each bin is high (> 5). One can then in principle use ΔC instead of Δχ² in certain model comparison tests. However, unlike χ², the CSTAT statistic may be used regardless of the number of counts in each bin.

The advantage of CSTAT over Sherpa's implementation of CASH is that one can assign an approximate goodness-of-fit measure to a given value of the CSTAT statistic, i.e., the observed statistic, divided by the number of degrees of freedom, should be of order 1 for good fits.

Note on Background Subtraction

The background should not be subtracted from the data when this statistic is used. It should be modeled simultaneously with the source.

Examples

1. Specify the fitting statistic and then confirm it has been set.

   sherpa-1> set_stat("cstat")
   sherpa-2> print get_stat_name()
   cstat