Computes significance using the Maximum Likelihood Ratio test.
sherpa> MLR <delta_dof> <delta_stat>
The command arguments are:
MLR Command Arguments
<delta_dof> |
The difference in the number of degrees of freedom (dofs) between
the fits of the null and alternative (more complex) hypotheses.
|
<delta_stat> |
The difference in the best-fit statistics between the two fits.
|
The Maximum Likelihood Ratio (MLR) test is a model comparison test.
Model comparison tests are used to select from two competing models
that which best describes a particular dataset. A model comparison
test statistic T is created from the best-fit
statistics of each fit;
as with all statistics, it is sampled from a probability distribution
p(T). The test significance is defined as the
integral of p(T)
from the observed value of T to infinity. The
significance
quantifies the probability that one would select the more complex
model when in fact the null hypothesis is correct. A standard
threshold for selecting the more complex model is significance <
0.05 (the "95% criterion" of statistics).
The MLR test may be used if:
-
the simpler of the two models is nested within the other, i.e.,
one can obtain the simpler model by setting the extra
parameters of the more complex model to default values, often
zero;
-
the extra parameters have values sampled from normal distributions
under the null hypothesis (i.e., if one samples many datasets given
the null hypothesis and fits these data with the more complex
model, the distributions of values for the extra parameters must
be Gaussian);
-
those normal distributions are not truncated by parameter space
boundaries; and
-
the best-fit statistics for each fit individually are
sampled from the chi-square distribution.
If these conditions are fulfilled, then the change in statistic from
one fit to the other (<delta_stat>)
is sampled from the chi-square
distribution for <delta_dof> degrees of freedom.
If these conditions
are not fulfilled, then the MLR test significance may not be accurate.
The MLR test significance can also be retrieved
using the Sherpa/S-Lang module function
get_mlr.
Perform the Maximum Likelihood Ratio test.
Fit two models to the data, one with two more parameters than the
other. The improvement in
chi-square (or the Cash statistic) is
20.0. Determine the significance by computing the tail integral
of the chi-square
distribution for 2 degrees of freedom from 20.0 to infinity:
sherpa> MLR 2 20.00
significance = 4.53999e-05
If this significance is smaller than the predefined threshold for
accepting the more complex model (e.g., 0.05), then the more
complex model is selected. Otherwise, the null hypothesis is selected.
- sherpa
-
berrors,
bsyserrors,
compute_errors,
compute_statistic,
covariance,
errors,
ftest,
get_paramest,
get_paramestint,
get_paramestlim,
get_paramestreg,
goodness,
interval-projection,
interval-uncertainty,
list_paramest,
projection,
region-projection,
region-uncertainty,
restore_paramest,
run_paramest,
run_paramestint,
run_paramestlim,
run_paramestreg,
set_errors,
set_syserrors,
staterrors,
syserrors,
uncertainty
|