Chandra X-Ray Observatory
	(CXC)
Skip to the navigation links
Last modified: December 2013

URL: http://cxc.harvard.edu/sherpa/ahelp/calc_ftest.html
Jump to: Description · Example · Bugs · See Also


AHELP for CIAO 4.9 Sherpa v1

calc_ftest

Context: utilities

Synopsis

Calculate the significance using the F test

Syntax

calc_ftest(dof_1, stat_1, dof_2, stat_2)

Description

The calc_ftest command computes the significance using the F test with the degrees of freedom of the simple model (dof_1) and its best-fit statistic (stat_1), along with the degrees of freedom of the complex model (dof_2) and its best-fit statistic (stat_2).

  • dof_1: degrees of freedom of the simple model
  • stat_1: best-fit statistic of the simple model
  • dof_2: degrees of freedom of the complex model
  • stat_2: best_fit statistic of the complex model

The F-test is a model comparison test. Model comparison tests are used to select from two competing models which best describes a particular data set. A model comparison test statistic, T, is created from the best-fit statistics of each fit; as with all statistics, it is sampled from a probability distribution p(T). The test significance is defined as the integral of p(T) from the observed value of T to infinity. The significance quantifies the probability that one would select the more complex model when in fact the null hypothesis is correct. A standard threshold for selecting the more complex model is significance < 0.05 (the "95% criterion" of statistics).

calc_ftest uses the ratio of the reduced chi2 which follows the F-distribution, (chi2_1/dof_1)/(chi2_2/dof_2). When calculating the significance, Sherpa uses the incomplete Beta function to obtain the integral of the tail of the F-distribution. The significance, or p-value, is returned by calc_ftest. If significance is < 0.05, the more complex model is selected.

The F-test may be used if:

  • the simpler of the two models is nested within the other, i.e., one can obtain the simpler model by setting the extra parameters of the more complex model to default values, often zero or one;
  • those normal distributions are not truncated by parameter space boundaries;
  • and the best-fit statistics for each fit individually are sampled from the chi-square distribution.

If these conditions are fulfilled, then the observed F statistic is sampled from the F distribution, whose shape is a function of dof_1 and dof_2. (The tail integral may be computed analytically using an incomplete beta function; see any basic statistics text for details.) If these conditions are not fulfilled, then the F-test significance may not be accurate.

Example

sherpa> calc_ftest(2, 20.28, 34, 33.63)

Calculate the F statistic for where the simple model has 2 degrees of freedom and a best-fit statistic of 20.28 and the complex model has 34 degrees of freedom and a best-fit statistic of 33.63.

Bugs

See the bugs pages on the Sherpa website for an up-to-date listing of known bugs.

See Also

data
copy_data, dataspace1d, dataspace2d, datastack, delete_data, fake, get_axes, get_bkg_plot, get_counts, get_data, get_data_plot, get_dep, get_dims, get_error, get_quality, get_specresp, get_staterror, get_syserror, group, load_ascii, load_data, load_grouping, load_quality, set_data, set_quality, ungroup, unpack_ascii, unpack_data
filtering
get_filter, load_filter, set_filter
info
get_default_id, list_data_ids, list_response_ids
modeling
clean
plotting
plot_data, set_xlinear, set_xlog, set_ylinear, set_ylog
saving
save_error, save_filter, save_grouping, save_quality, save_staterror, save_syserror
utilities
calc_data_sum, calc_data_sum2d, calc_kcorr, calc_mlr, calc_model_sum2d, calc_source_sum2d, get_rate
visualization
contour, contour_data, contour_ratio, get_ratio, get_resid, histogram1d, histogram2d, image_data, rebin

Last modified: December 2013
Smithsonian Institute Smithsonian Institute

The Chandra X-Ray Center (CXC) is operated for NASA by the Smithsonian Astrophysical Observatory. 60 Garden Street, Cambridge, MA 02138 USA.   Email:   cxchelp@head.cfa.harvard.edu Smithsonian Institution, Copyright © 1998-2017. All rights reserved.