merging_rules

Synopsis

A description of the merging rules used when combining header information.

Description

CIAO tools, such as dmmerge, have a number of rules for how the keywords from the header files should be combined. Some of these tools allow the rules to be configured; an example being the lookupTab parameter of dmmerge.

Default behavior

The name of the keyword is used to determine what rule to follow (see the "About the lookup table" section below for examples). When no rule exists the values is taken from the first file that is being combined. This means that

unix% dmmerge a.fits,b.fits combined-ab.fits mode=h
unix% dmmerge b.fits,a.fits combined-ba.fits mode=h
unix% dmlist combined-ab.fits,combined-ba.fits keys

can have different keyword values.

MAX

Take the maximal value among the keys that are present in the input files. If the key is not found in any of the input files, then no output and warning for the key.

Example:

In this case, the output is the maximal value '4'.

MIN

Take the minimal value among the keys that are present in the input files. If the key is not found in any of the input files, then no output and warning for the key.

Example:

In this case, the output is the minimal value '2.5'.

FAIL

If the key is not present in the first file, or it is present in the first file but the value is different from others, then display a warning and don't write the key to the output. If the key is present in the first file and the value is same as others, then write the value to the output file without a warning.

Example:

In this case, KEY is present in the first file, but the value is different from others. Therefore display a warning: 'KEY values are different...FAIL...', but no output for KEY.

FAIL;Default-[value]

First, go through each input file. If the key is not present, set the key value to the default (i.e. [value]). Once each file is checked, compare the values with one another. If they are all same, write it to the output; otherwise, display a warning but no output for the key.

Example using the rule 'FAIL;Default-3':

In this case, KEY is not present in the infile1 and infile2, so the rule sets both of them to be the default value '3', which happens to be same as the one in infile3. Therefore, it writes '3' to the output without a warning.

calcForce

Calculate the value with a special hard-coded rule and write it to the output file. Time keywords with a calcFORCE rule are generally set to "sum the values from the input files". For other keys, this rule will simply copy the first value to the output file.

calcGTI

Calculate the value in the output file from the time subspace (e.g. the good time intervals [GTIs]). This rule is currently applied to ONTIME, LIVETIME, LIVTIMEn, and EXPOSURn keywords.

Note: DATE-OBS DATE-END are special keys whose merging rules are always treated like 'calcForce' in the dmmerge tool. In other words, they are always computed from the special hard-coded rules.

FORCE

First, go through each input file. If the key is not present, set the key value to the default (null, 0, blank). Once each file is checked, compare the values with one another. If they are different, write the first one to the output with a warning message; otherwise, output the value without warning.

Example:

In this case, the KEY of infile2 is not present, therefore set it to the default (i.e. '0') first. Since KEY values are different, write the first one (i.e. '3') to the output with a warning: 'KEY has 2 different values.'

FORCE-[value]

First, go through each input file. If the key is not present, set the value to the default (i.e. [value]). Once each file is checked, compare the values with one another. If they are different, write the first one to the output with a warning message; otherwise, output the value without warning.

Example using the rule 'Force-3':

In this case, the KEY of infile2 is not present, therefore set it to [value] (i.e. '3') first. Since KEY values are all same, write it to the output without warning.

WarnFirst

If the key is not present in the first file, give a warning and don't write it to the output. Otherwise, copy the first value to the output and display a warning message only if the first value is different from others.

Example:

In this case, the KEY value of infile1 is not present, therefore no output for KEY except a warning message: "omit - keyword KEY not present in 1st file".

WarnFirst;Force-[value]

First, go through each input file. If the key is not present, set the value to the default (i.e. [value]). After each file is checked, copy the first value to the output; if it is different from others, display a warning message.

Example using the rule 'WarnFirst;Force-5':

In this case, the KEY value of infile1 is not present, therefore force it to [value] (i.e. '5') first. Since all KEYs have same value, write it to the output without a warning.

WarnOmit-[tol]

If the key is not present in the first header, or if it is present in the first header but different from others more than [tol], then don't write the key and give a warning. Otherwise, take the first value.

Example using the rule 'WarnOmit-0.3':

In this case, the KEY value of infile1 is present but is different from others more than the given tolerance (i.e. 0.3), therefore no output for KEY except a warning: 'omit - KEY values different more than 0.3'.

WarnPrefer-[value]

If the key is not present in the first file, or if it is present in the first file but different from others, then write the [value] to the output and display a warning. If all the key values are same, then write it to the output with no warning.

Example using the rule 'WarnPrefer-9.0':

In this case, KEY is not present in infile1, therefore set it to [value] (i.e. '9.0'). Since the KEY values are different, write the first one to the output with a warning: 'warning: KEY has 2 different values'.

Merge-[value1];Force-[value2]

First, go through each input file. If the key is not present, force it to [value2]. Once each file is checked, compare the values with one another. If different, then write [value1] to the output with a warning. If same, write the first value to the output.

Example using the rule 'MERGE-99;FORCE-10':

In this case, the KEY is not present in infile1, therefore set it to [value2] (i.e. '10') first. Since the KEY values are different, write [value1] (i.e. '99') to the output and display a warning: 'KEY has different value..merged..'.

SKIP

Don't write the key to the output file.

Example:

In this case, always omit KEY from the output file.

PUT_STRING-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'char'.

Example using the rule 'PUT_STRING-TEST':

In this case, force the KEY value to be 'TEST'.

PUT_DOUBLE-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'DOUBLE'.

Example using the rule 'PUT_DOUBLE-99.9':

In this case, force the KEY value to be 99.9.

PUT_LONG-[value]

Force the output value to be [value]. It applies for the key that does not belong to the event header and has the data type of 'LONG'.

Example using the rule 'PUT_LONG-100':

In this case, force the KEY value to be 100.

Special merging rules and keywords

All merging rules described in the above section can only be applied for the keys that belong to the event header except the following rules of 1.a, 1.b, and 1.c.

These three rules should be used by the keys that do not belong to the event header:

• PUT_STRING-[value]
• PUT_DOUBLE-[value]
• PUT_LONG-[value]

This rule can be used by any key, regardless whether or not it belongs to the event header:

• SKIP

The above description for this rule is only applied to an event key. For a non-event key, it will be same as the rule 'PUT_STRING-[value]':

• FORCE-[value]

The merging rules of the following keys are always treated like 'calcForce' in the dmmerge tool. In other words, they are always computed from the special hard-coded rules:

• DATE-OBS
• DATE-END

About the lookup table

A lookup table is required by the dmmerge tool. It assigns a merging rule to each keyword. The current table used by dmmerge can be checked by calling:

unix% punlearn dmmerge
unix% cat `pget dmmerge lookupTab`

Example of lookup table

Lookup Table Format

Each row consists of two columns: the keyword and the associated merging rule.

The two columns are separated by blanks or tabs.

If a default value is assigned to a merging rule, use "-" to separate them, e.g.

mergeRule-defValue

where "defValue" can be a number or a character string

If a keyword has two merging rules together, use ";" as a delimiter between the merging rules. For example:

mergeRuleOne;mergingRuleTwo
mergeRuleOne-defValueOne;mergeRuleTwo-defValueTwo

Lookup Table Restriction

For the first column:

• It has to be in capitals.
• The keyword may be either an exact match or will use the or longest substring match. Thus using 'ONTIME' will apply the same rule to each of the 'ONTIME0', 'ONTIME1', etc. keywords.

For the second column:

• It has to be one word without any space
• If a default value is assigned to a merging rule, its data type needs to be same as the keyword. Otherwise, a warning message will be displayed.
• Currently not able to have a negative number as the numerical default value.
• For those keywords not found in the user's lookup table, the code will copy the 1st header to the output.
• Merging rules are limited to those defined in Sec. 3.
• For any merging rule not found in Sec. 3, the 1st value will be copied to the output.
• Merging rules are only applied to the values of the keyword, not to the 'comment' of the keyword.

See Also

tools

addresp, dmimgcalc, dmimgfilt, dmmerge, dmregrid2, hrc_dtfstats, reproject_image, reproject_image_grid