acis_run_hotpix

Synopsis

Identify and flag "hot" pixels and cosmic-ray "afterglows."

acis_run_hotpix  infile outfile badpixfile biasfile maskfile pbkfile
[biasthresh] [pbkext] [berrfile] [berrext] [maxerr] [usrfile] [bitflag]
[procbias] [evt0file] [evtsext] [writebias] [outbias] [probthresh]
[regwidth] [expnothresh] [tempdir] [verbose] [clobber]

Description

NOTE: this tool has been superseded by acis_find_afterglow in CIAO 4.4.

acis_run_hotpix is a script which executes, in sequence, the tools acis_build_badpix, acis_find_hotpix, acis_classify_hotpix and acis_build_badpix. The tool acis_build_badpix is executed twice. Collectively, these tools (1) search for pixels where the bias value is too low or too high (acis_build_badpix), (2) search for suspicious pixels where there is an unusually large or small number of events (acis_find_hotpix), (3) classify the events on suspicious pixels as being associated with cosmic-ray afterglows, hot pixels or astrophysical sources (acis_classify_hotpix) and (4) write the bad pixels and their neighbors to the output bad-pixel file (acis_build_badpix).

If a pixel in the input bad-pixel file is identified as bad because it is included in a CALDB list of known bad pixels, then the pixel is copied to the output bad-pixel file. If a pixel in the input bad-pixel file is identified as bad because it is hot, is affected by an afterglow or has a bad bias value, then the pixel is copied to the output bad-pixel file if and only if the pixel is reidentified as bad when the tool is rerun. The output bad-pixel file may also contain pixels that are not included in input bad-pixel file if they are found to be hot, to be affected by an afterglow or to have a bad bias value.

Since astrophysical sources can produce an unusually large number of events in a pixel, some care is used to try to avoid misidentifying these events as hot-pixel or afterglow events. Unlike afterglows or hot-pixel events, events from astrophysical sources are dithered. Therefore, if the number of events on the pixels surrounding a suspicious pixel is large compared to the mean number of events per pixel (see probthresh), the events are most likely associated with an astrophysical source. The events on these pixels are not flagged and the pixels are not identified as bad. Of course, this means that the tool may not find hot pixels or afterglows if they are in the dither pattern of an astrophysical source.

An "afterglow" is produced when a large amount of charge is deposited on a CCD by a cosmic ray. Most of the charge is clocked off of the CCD in a single frame. However, a small amount can be captured in charge traps, which release the charge relatively slowly. As a result, a sequence of events can appear in a single pixel over a few to a few dozen frames as the trapped charge is released. The events need not occur in consecutive frames. There can be gaps of a few frames with no events for the pixel. In general, the amount of charge released per frame declines with time. However, the trend need not be monotonic, especially near the end of an afterglow. If the median number of frames between consecutive events on a suspicious pixel is small (see expnothresh), then the events are flagged as afterglow events. The inferred start and stop times of an afterglow are copied to the columns TIME and TIME_STOP, respectively, in the output bad-pixel file. Legitimate x-ray events may be identified as afterglow events if the x-ray source is relatively faint except during some time interval which is short compared to the period of the dither.

If the events on a suspicious pixel are not associated with an astrophysical source or an afterglow, then the pixel is identified as a hot pixel.

This tool was designed to replace the tool acis_detect_afterglow because acis_run_hotpix (1) misidentifies far fewer legitimate x-ray events as afterglows than acis_detect_afterglow, (2) identifies pixels that are hot or that have bad bias values, which acis_detect_afterglow does not do and (3) permits afterglows to have gaps, which acis_detect_afterglow does not do. However, there are legitimate afterglows that the tool acis_detect_afterglow identifies, but that acis_run_hotpix does not. These afterglows typically have less than eight events (i.e. too few to be considered statistically significant by acis_run_hotpix). It is possible to change the statistical threshold by using the parameter probthresh, but such a change may increase the chances that real x-ray events are discarded. It may be desirable to use acis_detect_afterglow in addition to acis_run_hotpix to remove the afterglows with small numbers of events, but some care is required to ensure that a significant fraction of real x-ray events are not discarded. A new, more powerful, afterglow-detection algorithm is being developed to replace acis_run_hotpix (and acis_detect_afterglow).

This tool should only be run on data that has been taken in a TIMED exposure mode. Do not use acis_run_hotpix on continuous-clocking (CC) mode data. This is because the code is designed to look for bad pixels, not bad columns, and only the column of an event is known in CC mode.

Status Bits

acis_find_hotpix identifies 3 different kinds of anomalies: hot pixels, afterglow events, and pixels with bright bias. These are represented by the following status bits in the bad pixel file:

When the bad pixel and event files are input to acis_process_events, different bits in are set in the event file to record these same anomalies:

For more information on the status bits in the bad pixel file, see "ahelp acis_build_badpix". Technical information on status bits in bad pixel and event files is available from the memos on the MIT CXC Documents page.

Example

acis_run_hotpix acisf00732_000N002_evt1.fits new_bpix1.fits
pbkfile="acisf079650447N002_pbk0.fits"
badpixfile="acisf00732_000N002_bpix1.fits" biasfile=@bias_list.txt
maskfile="acisf00732_000N002_msk1.fits"

This example shows the default use of the tool acis_run_hotpix. The files acisf00732_000N002_evt1.fits, acisf079650447N002_pbk0.fits, acisf00732_000N002_bpix1.fits, bias_list.txt and acisf00732_000N002_msk1.fits are read and processed to create the file new_bpix1.fits. The input bad-pixel file should be an observation specific file (e.g. acisf00732_000N002_bpix1.fits) instead of a file from the calibration database (e.g. acisD2002-10-05badpixN0001.fits). The output file new_bpix1.fits should be used when making an imaging or grating ARF or when making an instrument (i.e. exposure) map. The file bias_list.txt contains a list of the bias maps for the observation, with one file name per line.

Parameters

Detailed Parameter Descriptions

Parameter=infile (file required filetype=input stacks=yes)

The name(s) of the input event data file(s) (e.g. acisf00732_000N002_evt1.fits). acis_run_hotpix can read Level 0, Level 1 and Level 2 files, but users are urged to use Level 1 files because these files contain all of the events. Some of the events are excluded from Level 2 files.

As mentioned before, this tool should only be run on data that has been taken in a TIMED exposure mode. If you are unsure of the mode of your data, check the READMODE header keyword.

Parameter=outfile (file required filetype=output)

The name of the output bad-pixel file. This file contains the all the identified bad pixels; note that the bad pixels in the input bad pixel file are not all necessarily included in the output bad pixel file.

Parameter=badpixfile (file required filetype=input)

The name of the observation-specific input bad-pixel file (e.g. acisf00732_000N002_bpix1.fits). Do not use the name of a file in the calibration database (CALDB).

Parameter=biasfile (file required filetype=input stacks=yes)

The name(s) of the optional input bias map(s) for the observation (e.g. acisf079649146N002_0_bias0.fits). Each bias map is searched for pixels whose bias values are either too low or too high (see biasthresh).

Parameter=maskfile (file required filetype=input default=NONE)

The mask file (msk1.fits) for the observation.

The name of the optional input mask file which defines the "active" region(s) of the CCD(s) used for the observation. The mask file is needed in particular when a window or subarray was used. A value of "NONE" indicates that no mask file will be applied.

More information on the mask file is available in the CIAO Dictionary.

Parameter=pbkfile (file required filetype=input)

The name of the input parameter-block file (e.g. acisf079650447N002_pbk0.fits). The parameter-block file is used to determine which CCDs are active, the READMODE and DATAMODE and the values of CCD_ID, FEP_ID, ROWCNT, STARTROW, TSTART, TSTOP etc.

Parameter=biasthresh (integer not required default=6 min=3 max=100)

This parameter is used to determine which pixels in a bias map have suspiciously high or low values. Before performing the test, the median bias value of a column of a CCD is subtracted from the bias values of all the pixels in the column to obtain the adjusted biases. Pixels identified as bad in the calibration, bias-err and bias-map files are excluded from the computation of the median. If the adjusted-bias value of a pixel > biasthresh adu or < -biasthresh adu, then the pixel is identified as bad and written to the output file. The default value of this parameter should be adequate in most cases. Be cautious about using some other value.

Parameter=pbkext (string not required default=PBK)

The name (EXTNAME) of the extension in the parameter-block file that includes information about the active CCDs.

Parameter=berrfile (file not required filetype=input stacks=yes)

The name(s) of the optional input bias-parity error file(s), if any (see example 1). The bias-parity error data is useful for two reasons. First, pixels that have bias-parity errors are written to the output bad-pixel file. Due to the manner in which TIMED VFAINT mode data is handled onboard, occasionally the pixels (CHIPX,CHIPY) = (1023,1) and (1024,1) are incorrectly included in the bias-parity error files. In such cases, the output bad-pixel file does not include bias-parity errors for these pixels. The problem does not occur for other observing modes. A second reason for using bias-parity error files is that they provide a means of identifying observations that suffer from the rare "FEP0" problem (see maxerr).

Parameter=berrext (string not required default=BERR)

The name (EXTNAME) of the extension in the bias-parity error file(s) that includes information about bias-parity errors.

Parameter=maxerr (integer not required default=10)

The upper limit on the number of "normal" bias-parity errors for a CCD. If a CCD has more than this number, then the CCD is assumed to be suffering from the rare "FEP0" problem. In such cases, the entire top half of the CCD is marked as bad in the output bad-pixel file. The default value of this parameter was chosen with care. Be cautious about using another value.

Parameter=usrfile (file not required filetype=input default=none)

The name of an optional user-specified input file. Such files provide a means of including one or more additional bad pixels or columns in the output bad-pixel file and of excluding one or more bad pixels from the output file (see example 7).

The first two rows of the file must be

#ccd_id chipx_lo chipx_hi chipy_lo chipy_hi time time_stop bit action
#------ -------- -------- -------- -------- ---- --------- --- ------

Every row thereafter must contain nine space- or tab-delimited columns of data. From left to right, these columns are

Table 2

If there is more than one region specified in the usrfile, then the coordinates of the regions cannot overlap.

Parameter=bitflag (string not required default=00000000000000022221100020022222)

A 32 character string with one character for each of the 32 bad-pixel STATUS bits (see Table 1). The character on the right-hand side is associated with STATUS bit 0. The character on the left-hand side is associated with STATUS bit 31. The default and allowed values for each character are listed in Table 3.

Table 3

The actions associated with the characters "0", "1", and "2" are described in Table 4.

Table 4

Parameter=procbias (boolean not required default=yes)

If procbias=yes, then the bias map(s) specified by the parameter biasfile is(are) searched for pixels that have suspiciously high or low bias values (see biasthresh). Pixels that have one or more of the STATUS bits 0-6, 11, or 13 set are excluded from the search. Pixels with suspicious bias values are written to the output file.

Parameter=evt0file (file not required filetype=input default=none stacks=yes)

The name(s) of the optional input Level 0 event-data file(s). This parameter should only be used if the observation was performed using the rare faint-with-bias mode (i.e. DATAMODE = FAINT_BIAS) and if an output bias map(s) is desired (i.e. writebias=yes). The prefix of the name(s) of the output bias map(s) is specified using the parameter outbias.

Parameter=evtsext (string not required default=EVENTS)

The name (EXTNAME) of the extension in the Level 0 event-data file(s) that includes the bias information.

Parameter=writebias (boolean not required default=no)

If writebias=yes and the parameter evt0file is used to specify the input Level 0 faint-with-bias mode event-data file(s), then a bias map will be created for each CCD from the bias information in the event file(s). The file name prefix for each bias is specified by the parameter outbias.

Parameter=outbias (file not required filetype=output)

The file name prefix of the output bias map(s) created if writebias=yes and the parameter evt0file is used to specify the input Level 0 faint-with-bias mode event-data file(s).

Parameter=probthresh (real not required default=1e-3 min=1e-10 max=0.1)

This parameter specifies the minimum significance of suspicious pixels. For example, if P is the probability of obtaining S events on a pixel for an expected number of events R, then the pixel is identified as suspicious if P < (probthresh / T), where T is the total number of pixels examined. A pixel is also suspicious if P > 1 - (probthresh / T). The default value of 0.001 corresponds to 3.09 sigma for a normal distribution. The one, two and three sigma confidence values are 0.159, 0.0228 and 0.00135, respectively. The default value of this parameter should be adequate in most cases. Be cautious about using some other value.

Parameter=regwidth (integer not required default=7 min=3 max=255)

This parameter specifies the size of the reference region used to determine the expected number of events for a pixel. The default value of seven means that the forty-eight pixels in a 7 pixel x 7 pixel region surrounding a pixel are used. The actual number of pixels used is smaller than forty-eight if the region contains pixels known to be bad, contains inactive pixels or overlaps the edge of a node. If no events are in the reference region, then the mean number of events per pixel for the node is used as the expected number of events. The default value of regwidth should be adequate in most cases. Be cautious about using some other value.

Parameter=expnothresh (integer not required default=10 min=2 max=10000)

This parameter is used to distinguish between events associated with cosmic-ray afterglows and hot pixels. Events on a suspicious pixel are identified as afterglow events if the median number of frames between consecutive events is less than expnothresh. Otherwise, the suspicious pixel is identified as a hot pixel. The default value of this parameter should be adequate in most cases. Be cautious about using some other value.

Parameter=tempdir (string not required default=${ASCDS_WORK_PATH})

The name of the directory to which temporary files are written.

Parameter=verbose (integer not required default=0 min=0 max=5)

This parameter determines the amount of messages that is generated by acis_run_hotpix. If verbose=0, very few messages are reported. If verbose=5, the largest amount of messages is produced.

Parameter=clobber (boolean not required default=no)

If clobber=yes and a file exists that has the same name as the name of the output file, then the existing file is overwritten. If clobber=no, then the existing file remains unchanged.

Changes in CIAO 4.5

• Additional acis_build_badpix parameters may be set from the acis_run_hotpix parameter file: pbkext, berrfile, berrext, maxerr, usrfile, bitflag, procbias, evt0file, evtsext, writebias, and outbias.

Bugs

Tool does not add a history entry to the header.

The tool does not add a history entry to the output file header, which means the dmhistory tool cannot be used to recall the command.

Caveats

# acis_classify_hotpix (CIAO): WARNING: File /tmp/hotpixel_list.fits does not have any rows. 

Some datasets will return this warning when acis_classify_hotpix is run, including when it is called by acis_run_hotpix. It warning means that no hot pixels were identified in the observation, which is a normal occurrence.

See Also

chandra

eventdef

tools

acis_build_badpix, acis_classify_hotpix, acis_clear_status_bits, acis_detect_afterglow, acis_find_afterglow, acis_find_hotpix, acis_process_events, acis_streak_map, acisreadcorr, destreak