Using a CSC Save File in CIAO

CSC Threads

Overview

Synopsis:

The results of a catalog search may be written to a file by selecting the "Save" toolbar button while the the Results tab of CSCView is open, or by clicking the "Save Results to File" box in the Query tab before actually submitting a query. The save files output in Tab Separated Values (TSV) format consist of headers commented with '#', which contain definitions of the CSC columns in the table, as well as data type and format information. There is a row of CSC column names beneath the header, and beneath that are fixed format, tab-separated columns of data values. TSV files are compatible with Vizier and Aladin, and also with the CIAO Data Model after following the steps in this thread.

The tsv2ciao.pl script makes several small edits to a TSV save file, creating an output file which can be filtered and edited in CIAO.

Last Update:

11 Aug 2010 - updated for CSCview version 1.1: conversion script updated to accept TSV format save files in place of RDB format; 'cone_distance' catalog parameter has been changed to 'separation'.

About the Script

Download the tsv2ciao.pl script and make sure it is executable.

The script takes two input arguments:

unix% ./tsv2ciao.pl
usage: ./tsv2ciao.pl input output

input - the TSV format save file output by CSCview
output - name for the output CIAO-compatible file

When run, tsv2ciao.pl performs two operations on the input file:

any lines which begin with "#" are removed.
a "#" is prepended to the line of column names. This enables CIAO to identify the column names and use them in data filters.

Note that CIAO cannot correctly list the column values if there any blank spaces representing null cells; these empty cells must be populated with some value, e.g. "Null" or "-9999". The desired null value for empty cells in a table of query results may be specified in the Query tab of CSCview with the menu option "Edit->Preferences->Null Value Representation", before a query is submitted and results are returned (note that the CSCview default value is a blank space). You may also do so in the command-line interface by following Example 4 on the CSC Command-Line Interface page.

Running the Script

This example is run using csc.in, a sample save file, and the output is named out.ciao:

unix% ./tsv2ciao.pl csc.in out.ciao

The first few lines of the output file out.ciao are:

unix% less out.ciao
# separation name    ra      dec     err_ellipse_r0  conf_flag       sat_src_flag    significance    flux_aper_b     flux_aper_lolim_b       flux_aper_hilim_b       flux_aper_w     flux_aper_lolim_w       flux_aper_hilim_w
2.393124e-02    CXO J004244.3+411608     10.68484        41.26892          0.06  TRUE   FALSE     37.70 2.215e-13       2.176e-13
       2.255e-13             nul             nul             nul
6.937175e-02    CXO J004244.3+411605     10.68482        41.26813          0.10  TRUE   FALSE     14.58 1.434e-13       1.374e-13
       1.494e-13             nul             nul             nul
7.605652e-02    CXO J004244.2+411613     10.68454        41.27055          0.16  TRUE   FALSE      8.55 1.724e-14       1.560e-14
       1.890e-14             nul             nul             nul
9.451914e-02    CXO J004243.8+411611     10.68264        41.26979          0.16 FALSE   FALSE      9.39 2.285e-14       2.082e-14
       2.491e-14             nul             nul             nul
1.266645e-01    CXO J004243.8+411603     10.68273        41.26772          0.10  TRUE   FALSE     11.49 4.231e-14       4.035e-14
       4.429e-14             nul             nul             nul
1.601792e-01    CXO J004244.6+411618     10.68611        41.27170          0.08  TRUE   FALSE     17.36 5.088e-14       4.893e-14
       5.285e-14             nul             nul             nul

Using the File in CIAO

When using the output file in CIAO, it is necessary to tell the software that it is a tab-separated file. The default separator includes the whitespace character, which means fields that contain spaces (e.g. name, ra, dec) will be split incorrectly.

To specify the tab separation, include the "sep" option with the filename:

ciao% dmlist "out.ciao[opt sep=\t]" cols
 
--------------------------------------------------------------------------------
Columns for Table Block out.ciao
--------------------------------------------------------------------------------
ColNo  Name                 Unit        Type             Range
   1   separation                        Real8          -Inf:+Inf
   2   name                              String[1024]
   3   ra                                Real8          -Inf:+Inf
   4   dec                               Real8          -Inf:+Inf
   5   err_ellipse_r0                    Real8          -Inf:+Inf
   6   conf_flag                         String[1024]
   7   sat_src_flag                      String[1024]
   8   significance                      Real8          -Inf:+Inf
   9   flux_aper_b                       Real8          -Inf:+Inf
  10   flux_aper_lolim_b                 Real8          -Inf:+Inf
  11   flux_aper_hilim_b                 Real8          -Inf:+Inf
  12   flux_aper_w                       String[1024]
  13   flux_aper_lolim_w                 String[1024]
  14   flux_aper_hilim_w                 String[1024]

The Using ASCII Files in CIAO thread show how to filter and copy ASCII files with the CIAO Data Model.

History

06 Oct 2008	original version
01 Jun 2009	updated to include recent changes to CSCview
11 Aug 2010	updated for CSCview version 1.1: conversion script updated to accept TSV format save files in place of RDB format; 'cone_distance' catalog parameter has been changed to 'separation'.