A list of bugs fixed in CIAO 4.2 is available.

Bugs

General

1. You may get a seg fault if you try to create a very large image. What constitutes "very large" depends on the data type, but for long and float images, 8192x8192 pixels seems to be the threshold.

2. # DMCOPY (CIAO 4.2): Bad data type in filter string formatting

Filtering Data

1. Multi-valued string filters don't work

2. Creating a vector on-the-fly when region filtering

3. Applying a bit-filter expression to an integer column does not work, nor does it cause an error.

4. Using incorrect syntax with the rectangle shape does not fail when filtering.

5. Filtering an image on logical coordinates causes problems when the short cut of omitting a number (i.e. to indicate the default value) is used.

6. Trying to exclude a region filter with update=no will cause the image to be filtered by the region's bounding box.

7. There is a bug in region and range filters.

8. Using "!=" with string filtering causes dmlist to fail. (05 Mar 2007)

9. The "or" syntax ("||") doesn't work on string columns (17 Jul 2009)

ASCII Kernel

1. Virtual columns are not supported. (11 Dec 2007)

2. DTF-FIXED keyword comments may be truncated. (15 Jan 2008)

Binning & Rebinning Images

1. Rebinning an image with different values for the two axes causes the coordinate information to be lost

2. The DM doesn't catch incorrect binning syntax (17 Jul 2009)

3. Running a DM tool on an image where one of the axes has been filtered results in an error. (17 Jul 2009)

Bugs

General

1. You may get a seg fault if you try to create a very large image. What constitutes "very large" depends on the data type, but for long and float images, 8192x8192 pixels seems to be the threshold.

   (The image doesn't have to be square; it just needs to have 8192^2 pixels.)

   This condition may be met when the "update=no" option is used. Normally, when you filter a dataset, the data subspace (which describes the boundaries of each column's data and therefore is the intersection of the initial minima and maxima with any subsequent filters) gets updated to reflect the filtering. However, when you give the "update=no" option, you instruct the DM not to update the subspace to reflect the current filter. Therefore, the full ranges for x and y are used in the binning, and you get a 8192x8192 image (and a seg fault, for the reason described above).

2. # DMCOPY (CIAO 4.2): Bad data type in filter string formatting

   If a logical column exists in the file, the DM will generate this warning. The output is unaffected by this bug.

Filtering Data

1. Multi-valued string filters don't work

   "col=foo" is okay, but "col=foo,bar" isn't.

   Workaround:

   Use "col=foo,col=bar" instead.

2. Creating a vector on-the-fly when region filtering

   When region-filtering images, you can create a vector on the fly from any two axes by using a filter like "(#1,#3)=circle(...)". Although the image is filtered correctly with a temporary vector, the region filter isn't recorded in the subspace. Hence, tools that use the filtered file don't know that pixels outside the filter region are invalid. As a result, dmstat reports no nulls in the filtered image (unless you explicitly tell the DM to set pixels outside the filter to null by using "opt null=...").

3. Applying a bit-filter expression to an integer column does not work, nor does it cause an error.

4. Using incorrect syntax with the rectangle shape does not fail when filtering.

   For example, setting xmax > xmin and/or ymax > ymin. Instead it appears that the Data Model simply swaps the min and max values.

5. Filtering an image on logical coordinates causes problems when the short cut of omitting a number (i.e. to indicate the default value) is used.

   The exit status of dmcopy is also incorrectly set to 0 (success):

   unix% dmcopy "image.fits[#1=1:20,#2=:]" delme.fits
   # DMCOPY (CIAO 4.2): [ftColRead]: FITS error 308 bad first element number in 
   dataset image.fits Block 1 PRIMARY
   
   unix% echo $status
   0
   

   Workarounds:

   1. Omit the "#2=:" from the filter

   2. Specify a range for both elements: [#1=1:20,#2=1:20]

6. Trying to exclude a region filter with update=no will cause the image to be filtered by the region's bounding box.

   For example:

   unix% dmcopy "acis_img.fits[exclude sky=region(src.fits)][opt full,update=no]" filtered.fits 
   

   The regions are correctly excluded; however, the image is also clipped at the bounding box around all the excluded shapes, so the corners of a few chips are removed.

   Workarounds:

   1. Remove update=no. In this case, the Data Model internally inverts all exclude filters to be an inclusive filter, and correctly filters the image.

      Be aware that this process is much slower if the region is large. In that case, it will also add a large region keyword to the file's header, noticeably slowing down any operation on that file.

   2. For ASCII region files, it is also possible to manually invert the filter in the file. The "field()" region syntax is used to include the entire field, then remove the undesired sources. For instance,

      # Region file format: CIAO version 1.0
      circle(1635.5,4113.5,135.11408)
      circle(3975,4233,20)
      circle(2565.5,4129.5,40)
      circle(2129.5,4007.5,40)
      

      would become

      # Region file format: CIAO version 1.0
      field()
      -circle(1635.5,4113.5,135.11408)
      -circle(3975,4233,20)
      -circle(2565.5,4129.5,40)
      -circle(2129.5,4007.5,40)
      

      and the dmcopy filtering command would be

      unix% dmcopy "acis_img.fits[sky=region(src.ascii)][opt full,update=no]" filtered.fits 
      

7. There is a bug in region and range filters.

   This command:

   unix% dmcopy "input.fits[sky=circle(4096,4096,100),y=4020:4100,4250,4350]" \
         output.fits
   

   fails to do the y filter altogether. This bug also applies to exclude filters.

8. Using "!=" with string filtering causes dmlist to fail. (05 Mar 2007)

   For example, the following commands both fail:

   unix% dmlist "region.fits[shape!=Annulus]" data
   
   unix% dmlist catalog.fits"[COMMENT!='weak'][cols COMMENT]" data
   

9. The "or" syntax ("||") doesn't work on string columns (17 Jul 2009)

   For example, this command does not find all instances of "PMterm" in the selected columns:

   unix% dmlist stat.fits"[src=PMterm||det=PMterm||mst=PMterm]" counts
   13
   

   Compare to

   unix% dmlist stat.fits"[cols det,src,mst]" data,clean | grep PMterm | wc -l
   27
   

ASCII Kernel

1. Virtual columns are not supported. (11 Dec 2007)

   In the DM, you can normally do

   unix% dmlist evt.fits"[cols ra,dec]"  data
   

   even though RA and Dec are just coordinate systems defined on the X and Y columns in the file; the DM applies the transform on the fly. This doesn't work yet for ASCII files.

2. DTF-FIXED keyword comments may be truncated. (15 Jan 2008)

   DTF-FIXED header lines may be up to 1024 characters long. However, if the keyword is longer than the FITS standard, the comment is truncated.

   unix% input.txt output.dtf'[opt kernel=text/dtf-fixed]'
   

   In input.txt:

   TTYPE14 = 'Class' / LV Class Exo: M = missile [B = tactical ballistic  
   missile (except Redstone) apo=80:200]
   R = research rocket O = orbital LV V = RTV Y = Exo weather rocket X =  
   Big test rocket D= Deep space launch
   

   In output.dtf:

   TTYPE14 = "Class " / LV Class Exo: M = missile [B = tactical  
   ballistic missile (except Redstone) apo
   

Binning & Rebinning Images

1. Rebinning an image with different values for the two axes causes the coordinate information to be lost

   For example:

   unix% dmcopy acis.img"[bin x=::5,y=::6]" acis5x6.img
   

   Using the same value for both axes works correctly:

   unix% dmcopy acis.img"[bin (x,y)=::5]" acis5.img
   

2. The DM doesn't catch incorrect binning syntax (17 Jul 2009)

   unix% dmcopy foo.lis"[bin col1=0.5:1024.5:1,col2=0.5:1024:5:1]"
   foo.img
   

   The bad syntax in the col2 binning - 0.5:1024:5:1 - should produce an error. Instead it ignores the ":1" and bins col2 to a length of 205.

3. Running a DM tool on an image where one of the axes has been filtered results in an error. (17 Jul 2009)

   unix% dmextract
   Input event file (ccd3.sky4.fits[y=3767:][bin sky=annulus(3786,3767,0:380:4)]):
   Enter output file name (rprof.fits):
   # dmextract (CIAO 4.0 Beta 2): WARNING: Input file, "ccd3.sky4.fits[y=3767:]", 
   has no rows in it.
   
   Bus error
   

Bugs fixed in CIAO 4.2

The following is a list of bugs that were fixed in the CIAO 4.2 software release.

1. If you use "[opt null=nan]" on an integer image, the BLANK keyword is set to 0, since nan isn't a valid integer.

   Users should specify "[opt null=-999]", or some other number that is not likely to be a valid pixel value.

2. Header comment values may not exceed 81 characters. (17 Jul 2009)