Synopsis
Create an index file or append data on an existing one of calibration database (CALDB) products.
Syntax
calindex index keyconf calfile [--clobber y] [--verbose #]
Description
calindex generates an "index" file from calibration data source directed to "calfile" with user's specification in "key.conf". The index then will be served in CALDB product retrievals by other applications. The index has a single FITS extension "CIF" with multi-columns, including a set of mendatory columns of predefines and the optional ones of run-time specification. A header-key configuration table, "keyconf", allows user to extend the index table with defined column data types, data formats, and other entries. The name field, "calfile", sets access of the calibration data products for the indexing.
calindex appends new index data on an existing "CIF" extension or appends new index extension on an existing file if "clobber" is default or set to no|n|0.
Examples
Example 1
calindex xfile.1.fits keys_1.cfg datadir --clobber yes
Create index file named as "xfile.1.fits" to index all FITS files in directory, "datadir" and its recursively down directories. The header-key config file, "keys_1.cfg", is a text file and has following columns and rows. As clobber is set to "yes" (yes|y|1), "xfile.1" will be removed first if existed.
#keys_01.cfg
#colName dataType format hdrKey query null
TELESCOP string a10 TELESCOP yes ""
INSTRUME string a10 INSTRUME yes ""
CAL_VSD string a10 DATE-OBS no ""
CAL_VST string a8 DATE-OBS no ""
CAL_DESC string a70 CDES no ""
CAL_CLAS string a4 FOO no ""
The lines starting with '#' are comments and will be ignored at run time. Only the first three columns (colName, dataType, and format) are interesting for index creation and the last 3 columns are served for CALDB product retrieval in other applications. What it says here is that, for example, "TELESCOP", a column name entry, has "string" data type and data format of 10 ASCII characters ("a10"). The rows of the "TELESCOP" values are extracted from all FITS files under "datadir" and then indexed in "xfile.1.fits". The other five rows of "keys_1.cfg" follow the same suit. calindex gets right CALDB FITS files by checking the file postfix, "fits" or "fit", which is case-insensitive.
In this example, the output of "xfile.1.fits" with "CIF" block is created with 14 columns, as shown below. Column #1,2,11-14 are user's addition as specified in "keys_1.cfg" above and the rest are defaults, common to any index file.
-------------------------------------------------------------------------------- Columns for Table Block CIF -------------------------------------------------------------------------------- ColNo Name Unit Type Range Comment 1 TELESCOP String[10] Telescope 2 INSTRUME String[10] Instrument 3 CAL_DEV String[20] CALDB Device 4 CAL_DIR String[70] CALDB Directory 5 CAL_FILE String[40] CALDB File 6 CAL_CNAM String[20] CALDB Product 7 CAL_CBD[9] String(9) Condition 8 CAL_XNO Int2 - Extension # 9 CAL_QUAL Int2 - Quality 10 CAL_DATE String[10] date 11 CAL_VSD String[10] Start Date 12 CAL_VST String[8] Start Time 13 CAL_DESC String[70] description 14 CAL_CLAS String[4] class
Example 2
calindex xfile.1a.fits keys_1.cfg mycaldb1+ --clobber y or calindex xfile.1a.fits keys_1.cfg mycaldb1/+ --clobber y
Create index, "xfile.1a.fits", with the same optional columns as above example but having different data products indexed. The index rows of the output are filled from all FITS files in the directory "mycaldb1" and in its recursive down directories. The symbol, '+', following directory name and or '/' indicates a recursive readout of calibration files from the current location.
Example 3
setenv CALDB mycaldb7 calindex xfile.7.fits keys_7.cfg CALDB --clobber y
Create index, "xfile.7.fits" with "CALDB" to "calfile". The "CALDB" is an environmental variable and it, in the current example, is pointed to "mycaldb7+" by pre-commanding "setenv CALDB mycaldb7". In the current cae, mycaldb7 contains 3 FITS files for feeding calindex.
"keys_7.cfg" has two more rows, "CAL_VED" and "CAL_VET", added over the "keys_1.cfg", as shown below
#keys_7.cfg #colName dataType format hdrKey query null TELESCOP string a10 TELESCOP yes "" INSTRUME string a10 INSTRUME yes "" CAL_VSD string a10 "" yes "" CAL_VST string a8 "" yes "" CAL_VED string a10 "" yes "" CAL_VET string a8 "" yes "" CAL_DESC string a70 CDES no "" CAL_CLAS string a4 FOO no ""
Example 4
setenv CALDB mycaldb6 calindex xfile_6b.fits keys_1.cfg "CALDB,mycaldb2" --clobber y
Index file, "xfile_6b.fits", is created on directory list, CALDB and "mycaldb2". CALDB is aliased to "mycaldb6" in which there are 3 FITS files and one sub-dir containging one file. "mycaldb2" has only one FITS file within. So, the output has, at least, 4 FITS files indexed.
Example 5
calindex xfile.5.fits keys_1.cfg @cfile_5.lst --clobber y
This example shows an alternative to assign calfile value with named file list, marked by '@' before the named file, "cfile_5.lst". Below is what the list looks like
#cfile_5.lst
cfile_5A.fits
cfile_5B.fits
When calindex detects the first char, '@', of the calfile string and it opens up the file list, "cfile_5.lst", and feeds "cfile_5A.fits" and "cfile_5B.fits" in to the array of calibration files.
Example 6
calindex xfile.1g.fits keys_1.cfg ./*.foo --clobber y
Create index, "xfile.1g.fits", to all files ending with ".foo", in the current directory.
Example 7
calindex xfile.1h.fits keys_1.cfg '' --clobber y or calindex xfile.1h.fits keys_01.cfg "NONE" --clobber y
Create index, "xfile.1h.fits", to all FITS files in and under the CALDB directory tree. Note that calindex, with any empty or 'NONE' input to 'calfile', will look into CALDB directory tree.
Example 8
setenv CALDB /data/regression_test/development/CALDB/data/chandra calindex xfile.8a.fits keys_1.cfg $CALDB/acis/cpf+ --clobber y
Create index, "xfile.8a.fits" to all FITS files in "$CALDB/acis/cpf" directory and its resursive subdirectories.
Example 9
calindex xfile.9.fits keys_9.cfg "cfile.9A.fits,cfile.9B.fits" --clobber y
As usual, calindex creates "xfile.9.fits" to index two calibration files, "cfile.9A.fits" and "cfile.9B.fits". However "keys_9.cfg" adds a "MYDETNAM" column name, an incoventional entry, as shown below. With this in check, calindex geneartes the output containing "MYDETNAM" column.
#keys_9.cfg #colName dataType format hdrKey query null TELESCOP string a10 TELESCOP yes "" INSTRUME string a10 INSTRUME yes "" CAL_VSD string a10 "" yes "" CAL_VST string a8 "" yes "" MYDETNAM string a20 "dnam" yes "NONE" CAL_DESC string a70 CDES no "" CAL_CLAS string a4 FOO no "DATA"
Example 10
calindex xfile.9.fits ./acis/key.conf ./acis/det_gain
As "clobber" is default(to 0 ), the tool appends new index rows, extracted from ./acis/det_gain directory, on the extension, "CIF", of the disk file, "xfile.9.fits" if existed or it creates "xfile.9.fits" if not existed.
Parameters
| name | type | ftype | def | min | max | reqd |
|---|---|---|---|---|---|---|
| index | file | output | yes | |||
| keyconf | file | input | yes | |||
| calfile | file | input | yes | |||
| --clobber | boolean | no | ||||
| --verbose | integer | 0 | 0 | 5 |
Detailed Parameter Descriptions
Parameter=index (file required filetype=output)
Output file name
The named file, created either from scratch or from an existing one, has an extension named after "CIF", Calibration Index File. The "CIF" table contains columns internally defined (mandatory) as defaults and optionally defined by users. The mandatory columns are
CAL_DEV, CAL_DIR, CAL_FILE, CAL_XNO,
CAL_CNAM,
CAL_CBD,
CAL_QUAL,
CAL_DATE
conforming with the current CALDB version 4. The set of optional columns are specified in an external text file, an input to 'keyconf'. The default name of 'index' will be given to ./caldb.indx' if the value of 'index' is empty or 'NONE' or 'none'.
Parameter=keyconf (file required filetype=input)
Input file name
The key config file is used for user to define optional columns of the Index. The frequent conventional columns in the CALDB category include
TELESCOP,
INSTRUME,
CAL_VST, CAL_VSD,
CAL_VET, CAL_VED,
DETNAM
In addition, user can adds on any names other than the above, like FIDELITY, MYDETNAM, etc.
There are two special notes about the creation of the keyconf file. The column names of "TELESCOP" and "INSTRUME" must appear in keyconf file, so do in index file, as they will be served as the starting point of CALDB query later. Though the first three columns, or colName, dataType, and format, will be only interesting to index generation, the format of the entire set of columns have to be carried on as required by the software.
Parameter=calfile (file required filetype=input)
Input file name
"calfile" file, as the name explains, serves as a source directories of calibration data products. Following entries are allowed by the tool.
1> CALDB (the root of calibration data tree), 2> single file, 3> file stack, and 4> named file list.
The stacking files, as usual, are separated in comma, ','. The file list is indicated by '@' symbol in front of the named file string. Below is to summarize the possible syntaxes for the input string to 'calfile':
[directory] [directory]+ or [directory]/+ CALDB [file1, file2, ..., filen] [directory]/*.fits [directory]/*.[foo] @[file-list] NONE, none or empty
Parameter=--clobber (boolean default=no)
The prefix, "--", to "clobber" is an optional tag, followed by a given value. If the value is set to 1 or "yes", calindex removes the existing file and create a new file with an index, "CIF", extension. Otherwise for the default (no|n|0), calindex either 1) appends new data to an existing "CIF" extension of the said output file, or 2) creates a "CIF" extension appending on to the said output file which does not have CIF extension yet, or 3) creates a "CIF" index output while the file does not exit in the disk.
Parameter=--verbose (integer default=0 min=0 max=5)
Verbosity level of terminal display information to user.
There is no output message for default but more details printed out in ascending levels.
Bugs
There are no known bugs for this tool.