Retrieving Data Products

Retrieving Data Products through CSCview

Establish search criteria

To begin our query, we first clear any example entries which may be present in the interactive windows of the Query tab by selecting the "File->New->Empty Form" menu option - or by highlighting these items and selecting the '-' button next to the appropriate window. (You may choose to have the query form appear empty upon startup, or populated with an example query with the "Startup Query" option in "Edit->Preferences"; the latter is the default startup option.) To locate supernova SN 1993 in the catalog, we can either do a cone search around its position coordinates, or we can search on the target name in the Search Criteria window of CSCview. The former method involves searching for all sources within a specified radius of the Equatorial or Galactic source position coordinates. The coordinates may be entered directly into the Cone Search window of the Query tab, or located and entered automatically by the "By Name->Name/Resolver" feature.

Cone search ("By Name") in the Query tab of CSCview

We can also search on the target name of a Chandra observation by selecting the 'targname' source property from the "Observation-Specific Information->Observation Target and Pointing->Observation Target" category of the Source Observations list of properties and dragging it into the Search Criteria window (though this method is slower than the cone search). Here we would set 'o.targname LIKE SN 1993%' as the source property search condition for the query; the special character '%' will locate source names starting with 'SN 1993', followed by any set of characters.

Observation target name search in the Query tab of CSCview

The metadata display at the bottom of the CSCview Query tab lists the name, data type, units, and description of each source property selected in the Source Properties window. For extended, high-level descriptions of each source property included in the catalog, see the "Catalog Columns" page of the CSC website.

Select desired results

After establishing the search condition for the catalog query, we specify in the Result Set window the properties we would like returned by the query for each source. Since we are only interested in retrieving data products for SN 1993, and not necessarily any derived source properties contained in the CSC, we decide to have the query return only the IDs of the Chandra observations in which it has been detected, as well as the associated catalog names and best-estimate equatorial coordinates (in addition to the requisite 'dataset_id' property from the Data Products Category). To do this, we drag the 'name', 'ra', and 'dec' master source properties, and the 'obsid', 'obi', 'targname', 'ra_targ', and 'dec_targ' source observation properties into the Result Set window.

Finally, we change the 'Select' option from the default 'top 1000' to 'top 50' to retrieve data for the first fifty sources found in the search.

Complete query in the Query tab of CSCview

Submit query and view search results

The database query is now complete; we may submit the query by selecting the "Search" toolbar button. When the catalog search finishes, the query results interface appears with a table of results displayed in the Results tab. Since we entered nine source properties in the Result Set window of the Query tab, and set the number of desired search results to 'top 50', the query results table contains these nine columns and fifty rows. (Note, that the 'dataset_id' column is automatically entered into the result set of CSCview startup and standard queries, so that data products associated with sources returned by the search may be accessed. In this example, we added it ourselves since we cleared the CSCview startup query initially). Since we requested a mix of master source and source observation properties for SN 1993, there are multiple rows in the query results table, one for each observation in which it was detected.

Query results interface of CSCview (Results tab)

Save search results and make data product selections

The results may now be saved to a text file by selecting the "Save" toolbar button while the Results tab is open. (Note that the results of a catalog search can be saved to a file without ever having to leave the Query tab by clicking the box "Save Results to File" in the upper-right corner of the query form). To convert a Tab Separated Values (TSV) format save file output by CSCview to a CIAO-compatible text file, refer to the thread Using a CSC Save File in CIAO.

At this point, we may browse data products associated with sources found in the search. However, before proceeding to the Products tab to download selected data products, we may preview source region and full-field images of sources in selected rows of the results table using the Source Preview in the Results tab, accessible via the clickable "View" buttons in the table rows. The soure region images available include a "tricolor" soft-medium-broad events image, a single-color events image in any of the selected CSC energy bands (broad, hard, medium, soft, ultrasoft, wide), and a PSF image in any CSC energy band. The full-field broad-band events image is available at different blocking factors (1, 2, and 4). The full-field events image of the source in the second row of the results table is shown below, at a blocking factor of 2.

Preview of the CSC broad-band full-field image of a selected source in the results table (Results tab)

The image displayed in Source Preview may be saved to a JPG file by selecting File->Save within the Source Preview window. Note that Source Preview is only available when the 'dataset_id' catalog column is included in the search results, as this property is used to uniquely identify the set of data products associated with each source.

Download data products

The full list of data files provided by the CSC for each source appear in the Data Products window in the Results tab. We decide to retrieve the Level-3 event file, source region and full-field images, and the background and exposure maps corresponding to the observation of SN 1993J in the second row of the query results table. To do this, we select the second row of the table, then each of the desired file types and energy bands before selecting the "Search" toolbar button.

Selecting data products for download in the Results tab of CSCview

The filenames of all selected data products appear in the Products tab after the "Search" option has been selected, where they may be downloaded together as a tar file (by selecting the "Download" toolbar button) or via a download script for a batch download on the Unix command line (by selecting the "Script" toolbar button). Decompressing the tar file produces a directory named "cdapackage...", which contains the selected data products, compressed by gzip (Windows users should refer to the GNU zip website for .gz decompression options). The download script output by the "Script" option contains a list of GNU Wget commands, one for each file.

Downloading a tar file of selected data products in the Products tab of CSCview

Retrieving Data Products on the Command Line

The CSC data products may be accessed on the command line, but only after interactively querying the catalog within CSCview. After submitting a query in the Query tab and selecting the desired data products in the Results tab, you are brought to the Products tab, where you are given the option of downloading the selected data products to a single tar file, or via a download script. The download script contains a list of Wget commands, one for each file, which can be executed on the Unix command line for a batch download of the selected files. An example of such a file is shown below:

Data product download script available through the CSCview Products tab