Synopsis
Split up an array column into multiple files (e.g. PHA type 2 to PHA type 1).
Syntax
dmtype2split infile outfile [clobber] [verbose]
Description
CIAO extends the concept of type 1 and type 2 PHA files to general tables. A type 1 file of size N is a simple table consisting of scalar columns each with N rows. A type 2 file of size N x M is a table with M rows in which each column is either scalar or an array of size N. Each row of the type 2 file corresponds to a whole type 1 file, and `dmtype2split' performs this mapping.
The mapping consists of turning each array column entry from `sideways' to `vertical', so that an array of 4096 values becomes a column with 4096 rows. Scalar columns map to keywords in the output file.
Each of the specified rows in the type 2 file is copied to a type 1 file. Each output file has the basic header keywords and subspace copied from the input. If the input file is a pha file, keywords required by the OGIP format are also written; otherwise, generic keywords are written. For each scalar column in the input, the coordinate keyword information is written, and the datatype, value, and unit are written as a keyword. For each input column that is an array, a column is created from the datatype, value, and unit; the row of input data is read and written to the output column.
Specifying Rows
The rows can be specified in any order, but the Data Model sorts them numerically before processing. If supplying several input rows and a stack of output filenames, it is recommended that the rows be given in order so that the mapping to the filename is clear. The rows may be comma- or colon-delimited, e.g., "1:4,6,9:12". A colon-delimited interval is inclusive (e.g., "2:5" means rows 2, 3, 4, 5). If the number of rows specified does not equal the number of output files specified, the program exits with an error message. If no rows are specified, rows 1:n are processed, where n is the number of files in the outfile stack.
Specifying Output Block Name
By default, the output block name will be "SPECTRUM" since the most common use of this tool is the TYPE:II PHA files. Users can choose a different output block name by specifying it as part of the outfile file name.
Examples
Example 1
In this example we are going to take the following ASCII file, which contains two colums - the scalar x and vector y with 4 elements - and split it into two FITS files:
unix% cat vals.dat
# x y[4]
1.2 2.3 4.5 6.2 -3.4
4.5 0.1 2.3e2 -2301 43.9
unix% dmlist vals.dat blocks
--------------------------------------------------------------------------------
Dataset: vals.dat
--------------------------------------------------------------------------------
Block Name Type Dimensions
--------------------------------------------------------------------------------
Block 1: vals.dat Table 2 cols x 2 rows
unix% dmlist vals.dat cols
--------------------------------------------------------------------------------
Columns for Table Block vals.dat
--------------------------------------------------------------------------------
ColNo Name Unit Type Range
1 x Real8 -Inf:+Inf
2 y[4] Real8(4) -Inf:+Inf
unix% dmtype2split vals.dat v1.fits,v2.fits
unix% dmlist v1.fits blocks
--------------------------------------------------------------------------------
Dataset: v1.fits
--------------------------------------------------------------------------------
Block Name Type Dimensions
--------------------------------------------------------------------------------
Block 1: PRIMARY Null
Block 2: SPECTRUM Table 1 cols x 4 rows
unix% dmlist v1.fits cols
--------------------------------------------------------------------------------
Columns for Table Block SPECTRUM
--------------------------------------------------------------------------------
ColNo Name Unit Type Range
1 y Real8 -Inf:+Inf
unix% dmlist v1.fits data,clean
# y
2.30
4.50
6.20
-3.40
The first file, v1.fits, contains a single column, y, with 4 rows, containing the values of the first y[4] array in vals.dat. As the x column in vals.dat is a scalar, the value for the first row is written as a keyword rather than a column; e.g.
unix% dmkeypar v1.fits x echo+ 1.2
Example 2
unix% cat out.stk v1.dat v2.dat unix% dmtype2split vals.dat @out.stk"[opt kernel=text/dtf]"
This is a repeat of the previous example, but this time the output format has been changed to one of the ASCII formats supported by CIAO, rather than FITS. The dmascii ahelp page contains more information on the ASCII support in the CIAO Data Model. An external stack file was used so that the opt specifier only needed to be given once, rather than once per file; ahelp stack contains more information on how stacks are used in the CIAO tools.
In this case the "DTF" flavor was chosen because it is the only format that supports keywords. If "text/simple" had been used for the kernel instead, the output files would not contain any information on the value of the x column.
Example 3
unix% dmtype2split pha2.fits @outlist
Create output files (names in the stack outlist, (see ahelp stack for more information), one per row of the input file pha2.fits.
Example 4
unix% dmtype2split "pha2.fits[#row=1:3,6]" @outlist
Extract the first four rows and the sixth row from pha2.fits. The output stack file outlist must contain five file names.
Example 5
unix% dmmakereg "polygon(1,2,3,4,5,6)" poly.fits unix% dmtype2split "poly.fits[cols x,y]" poly_table.fits"[REGION]"
In this example we create a FITS region file with 1 row where the X and Y columns are 4 element arrays.
unix% dmlist poly.fits cols -------------------------------------------------------------------------------- Columns for Table Block REGION -------------------------------------------------------------------------------- ColNo Name Unit Type Range 1 POS(X,Y)[4] pixel Real8(4) -Inf:+Inf Position 2 SHAPE String[16] Region shape type 3 R[2] pixel Real8(2) -Inf:+Inf Radius 4 ROTANG[2] deg Real8(2) -Inf:+Inf Angle 5 COMPONENT Int2 - Component number
We then use dmtype2split to transpose that single row into a single table with scalar X and Y columns. The output block name, "REGION", is specified as part of the output file name. The output file then looks like
unix% dmlist poly_table.fits blocks,cols
--------------------------------------------------------------------------------
Dataset: poly_table.fits
--------------------------------------------------------------------------------
Block Name Type Dimensions
--------------------------------------------------------------------------------
Block 1: PRIMARY Null
Block 2: REGION Table 2 cols x 4 rows
--------------------------------------------------------------------------------
Columns for Table Block REGION
--------------------------------------------------------------------------------
ColNo Name Unit Type Range
1 X pixel Real8 -Inf:+Inf
2 Y pixel Real8 -Inf:+Inf
Parameters
| name | type | ftype | def | min | max | reqd | stacks |
|---|---|---|---|---|---|---|---|
| infile | file | input | yes | ||||
| outfile | file | output | yes | yes | |||
| clobber | boolean | no | |||||
| verbose | integer | 0 | 0 | 5 |
Detailed Parameter Descriptions
Parameter=infile (file required filetype=input)
Input type II file
The input file should contain at least one array column; that is, a dmlist opt=cols call on the file name returns at least one column with "[n]" after its name, where n is an integer.
Parameter=outfile (file required filetype=output stacks=yes)
Enter output file name(s)
The number of file names must equal the number of rows in the input file.
Parameter=clobber (boolean default=no)
OK to overwrite existing output file(s)?
Parameter=verbose (integer default=0 min=0 max=5)
Verbosity level
Bugs
- Corrupted error message
The error message that dmtype2split uses to report a problem opening the input file can contain random characters.
Caveats
- Does not support files with string columns
dmtype2split does not support files with string columns. They must be removed from the file for dmtype2split to reformat the remaining columns.
- The extension name for the output file must be included as part of the output file name(s).
-
Since dmtype2split is used for many different types of files it does not set the extension name, rather leaving it up to the user to specify. If the block name is omitted, the basename (no path) of the file name is used.
unix% dmtype2split "pha2[#row=1]" pha1 unix% dmlist pha1 blocks -------------------------------------------------------------------------------- Dataset: pha1 -------------------------------------------------------------------------------- Block Name Type Dimensions -------------------------------------------------------------------------------- Block 1: PRIMARY Null Block 2: pha1 Table 7 cols x 8192 rows -
Workaround:
As some analysis systems require certain files to contain specific block names to comply with different standards, the block name may need to be set as shown below
unix% dmtype2split "pha2[#row=1]" "pha1[SPECTRUM]" unix% dmlist pha1 blocks -------------------------------------------------------------------------------- Dataset: pha1 -------------------------------------------------------------------------------- Block Name Type Dimensions -------------------------------------------------------------------------------- Block 1: PRIMARY Null Block 2: SPECTRUM Table 7 cols x 8192 rows
See Also
- chandra
- eventdef
- dm
- dmfiltering, dmopt
- tools
- combine_grating_spectra, detilt, dewiggle, dmappend, dmgroup, dmgti, dmjoin, dmmerge, dmpaste, dmsort, dmtabfilt, dmtcalc, dmtype2split, mktgresp, symmetrize, tg_bkg, tg_choose_method, tg_create_mask, tg_findzo, tg_resolve_events, tgdetect, tgdetect2, tgextract, tgextract2, tgidselectsrc, tgmask2reg, tgmatchsrc, tgsplit