CSCview Help

CSCview is a GUI application which provides direct access to the contents of the catalog via user-specified queries. Search criteria and desired results are specified using the source properties contained in the catalog, which are presented in two table views: the Master Sources Table and the Source Observations Table. In each table, a row represents a source and each column a quantity or parameter that is officially part of the catalog. Bi-directional links within the catalog connect source properties contained in both tables (referred to as Master Sources/Source Observations associations), as well as associated source region and full-field data products, so that all known properties of a source are presented. The current database view of the catalog may be accessed with CSCview, which includes the most recently processed data sets which have not yet been included in the next catalog release. As of March 2009, the Release 1 catalog view is also available. A catalog release view is a carefully reviewed, static version of the CSC.

Software Requirements: CSCview is designed to run in a web browser with Java version 1.5 or higher. To check your current Java configuration, visit the Java help site, which performs an automatic check of your system.

The face of CSCview is reminiscent of that of a web browser, with standardized menus like "File", "Edit", and "Help", as well as icons representative of the most commonly used menu items. However, the tabbed pages are not independent of one another as they are in a web browser; the Results tab remains empty until a query is submitted in the Query tab, and the Products page is not populated with information until data products are selected on the Results page. Furthermore, the menus are adapted to the different tabs; whereas some menu options apply to all the tabs, others are active in only some of the tabs. For ease of use, first-time users of CSCview are encouraged to begin a search by reading the "Getting Started" help guide which pops up when CSCview is opened, as well as browsing the pull-down menus at the top of the GUI to become familiar with all of the available options.

Questions about CSCview should be submitted to the CXC Helpdesk.

• Catalog View (Catalog tab)
• Query Interface
• Query tab
• Standard Queries
• Source Properties
• Search Criteria
• Result Set
• Cone Search
• ADQL 2.0 interface
• within CSCview
• cURL and Wget
• Results Interface
• Results tab
• Products tab
• Menus
• File
• Edit
• View
• Tools
• Help
• Status Bar

Catalog View

Before querying the catalog, you must specify in the "Catalog" tab which version of the catalog to access: the current database view ("Current Database") or the current release view ("Release"). A catalog release view is a carefully reviewed, static version of the CSC. It is appropriate for the user who requires a detailed characterization of the statistical properties of the catalog, such as limiting sensitivity, completeness, false source rates, astrometric and photometric accuracy, and variability information. The database view provides direct access to the active CSC database for a chosen date and time, with the default being the date and time of the query. This view includes the most recently processed data sets which have not yet been included in a catalog release. See the CSC web page Catalog Release Views and Database Access Views for more information.

To begin a query, select the desired catalog view and then the "Search" icon; this will open the Query tab.

Query Interface

You can query the catalog database in one of three ways in the Query tab:

• by selecting and dragging the provided source properties into the Search Criteria, Result Set, and Sort Order windows (and/or by defining a cone search around a given sky position);
• by building a query from the example "Standard Queries" provided; or
• by writing query expressions in the Astronomical Data Query Language (version 2.0) in the ADQL view of the Query tab, which is accessible via the menu option "View->Query->Show Language".

The Getting Started guide provides a brief overview of the features on each tab of CSCview, as well as a set of instructions for loading example queries and building custom queries. Selecting the "Edit->Preferences->Startup Help->None" menu item will prevent this guide from automatically popping up when CSCview is opened.

Query Tab

To quickly start a query in the Query tab of CSCview, simply select one of the "standard queries" provided and drag it to any area on the query form; this will automatically populate the interactive windows with example entries. Otherwise, you can build a custom query by selecting and dragging the provided "source properties" into the Search Criteria window to define the search conditions for a catalog query, and into the Result Set window to specify the desired quantities to be returned by the search ("Edit->Add" may also be used for entering source properties). Multiple source properties may be selected simultaneously by pressing and holding either the keyboard Control, Shift, or Command key while selecting the desired source properties with a mouse cursor. The sort order of the table of search results to be returned may be specified by dragging one of the source properties listed in the Result Set window to the Sort Order window. A "cone search" around a sky position may be entered as the sole search criterion, or used in conjunction with other constraints entered into the Search Criteria window. The query can be saved to a text file by selecting the "Save" icon, and re-opened once saved with "File->Open".

Standard Queries

Many example queries are available as search templates in the Standard Queries window of the Query tab. To populate the query form with one of these queries, simply select one from the list and click "Edit->Add->Add Query", or drag and drop the selection into the query form. These queries can be combined, or you can clear the query form between each selection with "File->New".

The standard queries consist of a Master Source Basic Summary, Master Source Summary, Master Source Photometry, Master Source Variability, Source Observation Summary, Source Observation Photometry, and Source Observation Variability. The "Master" queries return properties from only the Master Sources Table, while the "Observation" queries include a mix of properties from the Master Sources Table and Source Observations Table.

The Standard Search Criteria section of the Standard Queries window consists of two options: Search by Observation Identification and Search by Variable Sources. When the first option is added to the query form, the 'o.obsid' source observation property is entered into the Search Criteria window so that you may enter a Chandra ObsID by which to search. When the second search criterion is selected, a set of search conditions specific to source variability analysis are entered into the Search Criteria window. To run this query, be sure to add at least one source property to the Result Set window.

Source Properties

The Master Sources/Source Observations Associations group contains the 'match_type' column, which labels a source observation found in a query search as either ambiguously ('a') or uniquely ('u') matched to a master source. If a source observation is ambiguously matched to a master source, it lies within the same area of the sky as the master source, but it can be matched to more than one distinct source in the area. Such a source is characterized as confused, therefore it is ambiguously matched to at least two master sources, and its properties do not contribute to any master source properties recorded in the Master Sources Table. A source observation that is uniquely matched to a master source is one that clearly corresponds to the same source in the same region of sky in multiple observations; its properties contribute to the master source properties recorded in the Master Sources Table. (Note, however, that if a uniquely matched source observation is piled-up, its properties will not contribute to the corresponding master source properties UNLESS it is an ACIS observation and all other ACIS observations of that source are also piled-up.) See the thread "Using Source Property Associations" for a demonstration of the use of the 'match_type' catalog column.

The Master Sources/Source Observations Associations group also contains the 'is_primary' column, which returns a Boolean value of True or False to indicate if a source observation represents the 'primary' observation chosen from multiple observations of the same source. A primary source observation represents the 'best' observation of a source, therefore its properties are key in determining the master source properties for that source. If a master source is associated with only one unique source observation, this observation is the primary.

The collective group of source properties is presented in a hierarchy in which each category - Master Sources, Source Observations, and Master Sources/Source Observations Associations - may be sorted by scientific group (e.g. Spectral Properties, Source Fluxes, etc.), science energy band, or alphabetically by name. The ordering may be set by expanding the "View-->Properties" menu item and choosing either "By Group", "By Band", or "By Name". For extended descriptions of each source property in the CSC, see the catalog column descriptions.

Search Criteria

Source observation properties are prefixed with 'o.' in the Search Criteria window to be distinguished from master source properties. Once the source properties have been added, you have the option to set each property 'equal to', 'not equal to', 'greater than', 'greater than or equal to', 'less than', or 'less than or equal to' a specified quantity. Additionally, the Boolean clauses IN, BETWEEN, LIKE, NULL, NOT NULL, and TRUE/FALSE may be used to set the search criteria for the database query. The Boolean clause 'IN' accepts a comma-delimited list, such as 'acis_num IN 2,3' ('m.acis_num IN (2,3)' in the ADQL interface). The Boolean clause 'LIKE' accepts a case-sensitive string, with the following special characters:

• % matches any set of characters

For example, 'o.targname LIKE SN%' matches source target names which begin with "SN", followed by any set of characters ('o.targname LIKE "SN%"' in the ADQL interface).

• _ matches exactly one character

For example, 'o.targname LIKE NGC 001_' matches source target names which begin with "NGC 001", followed by any single character ('o.targname LIKE "NGC 001_"' in the ADQL interface).

• [p-s] matches single characters in the range 'p' through 's'

For example, 'o.targname LIKE [p-s]%' matches source target names which start with 'p' through 's', followed by any set of characters. ('o.targname LIKE "[p-s]%"' in the ADQL interface).

• [aghz] matches exactly one of a, g, h, or z

For example, 'o.targname LIKE [aghz]%' matches source target names which start with 'a', 'g', 'h', or 'z', followed by any set of characters ('o.targname LIKE "[aghz]%"' in the ADQL interface).

• [^p-s] matches single character that is NOT 'p' through 's'

For example, 'o.targname LIKE [^p-s]%' matches source target names which do not start with 'p' through 's', followed by any set of characters ('o.targname LIKE "[^p-s]%"' in the ADQL interface).

• [^agkm] matches single character that is NOT a, g, k, or m

For example, 'o.targname LIKE [^agkm]%' matches source target names which do not start with 'a', 'g', 'k', or 'm', followed by any set of characters ('o.targname LIKE "[^agkm]%"' in the ADQL interface).

A given source property may be entered into the Search Criteria window multiple times, e.g., to define a source property range such as 'src_cnts_aper > 100' and 'src_cnts_aper <= 50'. Though the 'match_type' column can only be entered once; it is used to specify how to combine source observation and master source properties. Each of the source property conditions listed in the Search Criteria window may be related by an appropriate 'AND'/'OR' logic statement to specify if the sources found in the database search are to satisfy all or a subset of the source conditions, with a corresponding set of parentheses for delimitation; the pull-down menus located on either side of each source property condition in the Search Criteria window may be utilized for this purpose. All searches are case-sensitive.

Result Set

The table of search results to be returned may be sorted on any of the source properties entered into the Results Set window, in ascending or descending order; the sort order is specified by dragging one of the Result Set source properties into the Sort Order window. The column ordering of the table will reflect the order specified in the Result Set window. The default units in the query results table for any ra* / dec* columns selected is sexagesimal notation, hh:mm:ss.s, ±d:mm:ss.s; the output coordinate format may be changed to decimal degrees with the "Edit->Preferences->Output Coordinate Format" menu item.

The "Select" option above the Result Set window allows the user to specify as few as 10 rows of results to be displayed at a time in the query results interface, and as many as "all", once the query has been submitted. It also features the "count" item, which will return only one number when the search finishes: the total number of search results found ("total count"). Re-visiting the Query tab, changing "Select: count" to "Select: all", and re-submitting the query will return this number of rows of data in the Results tab. If multiple pages of data have been retrieved in the search, it will be necessary to use the Page Up/Page Down icons to the right of the query results table on the Results tab to view all of the rows.

The "rows"/"distinct rows" option of the "Select" feature can be used to display or hide duplicate source entries in the query results table to be returned, as certain types of queries can (correctly) return multiple entries for a given source. For example, confused sources in the catalog (match_type='a') are associated with at least two different master source names since they cannot definitively be resolved into one or the other. As a result, a query including a mix of master source and source observation properties may return the same set of source observation properties for multiple master source names (though the confused source properties do not contribute to the calculation of the corresponding master source properties). "Select: rows" is the default setting for a blank query, but note that "Select: distinct rows" is used in all of the master source standard queries provided.

Cone Search

-- within the GUI:

Note that a specified cone search will be carried out in conjunction with any search conditions set forth in the Search Criteria window (i.e., the Search Criteria conditions are connected to the cone search conditions by an 'AND'). For example, if a user has entered "obsid=617" in the Search Criteria box to locate all source observations associated with "ObsID 617", but has also entered a cone search of an area of the sky which overlaps many ObsIDs, including ObsID 617, the query results returned will consist only of sources matching ObsID 617, not all sources matching all ObsIDs returned by the cone search.

-- using command-line tools:

A VO Cone Search service is also available for conducting a search on source position in the CSC. It follows the IVOA cone search recommendations, with a fixed-call syntax which does not get combined with CSC cone searches in ADQL.

unix% curl 'http://cda.cfa.harvard.edu/cscvo/coneSearch?RA=83.77333&DEC=-5.68464&SR=.233&VERB=1'

unix% wget -O out.vot 'http://cda.cfa.harvard.edu/cscvo/coneSearch?RA=83.77333&DEC=-5.68464&SR=.233&VERB=1'

In the VO Cone Search, one searches on RA, DEC and SR (search radius) in demical degrees, and receives one of the three available result sets, based on the value of the parameter VERB (verbosity):

• verbosity = 1 reports the source name, RA and DEC values from the Master Sources Table for each matching source resulting from the cone search
• verbosity = 2 reports the master source summary results
• verbosity = 3 reports all the columns in the Master Sources Table

See example output in 'out.vot' produced by the Wget command above.

ADQL 2.0 command-line interface

-- within the GUI:

CSCview allows you to query the catalog by transmitting an Astronomical Data Query Language (ADQL) query expression. ADQL is a database language designed to enable sophisticated queries of an astronomical database from the command line, such as with the SELECT, TOP, FROM, WHERE, ORDER BY statement supported by the ADQL view of the Query tab. This view is accessed by selecting the menu item "View->Query->Show Language" while the Query tab is open.

ADQL 2.0 (case-sensitive) query expressions can be constructed to establish database search criteria and produce query results equivalent to those which result from the main view of the Query tab (the query in the ADQL view need not match the query defined in the Query tab in a given session of CSCview). If a query has been defined in the main view of the Query tab, upon entering the ADQL view the user will find the ADQL translation of this query (this does not work in reverse - a query expression defined in the ADQL view cannot be imported into the standard form of the Query tab). If a query has not been previously defined, the ADQL view appears with "SELECT top 1000 m.productid,' ' FROM master_source m "; you may edit the statement by dragging the provided source properties into the command-line window. To view examples of full ADQL query expressions, simply drag one of the seven standard queries to the ADQL query window. An ADQL query can be saved to a text file by selecting the "Save" option from the "File" pull-down menu.

An ADQL SELECT statement returns a result set of records from one or more tables of astronomical data, located by the FROM clause - fixed to be 'master_obi_assoc a', and/or 'master_source m', and/or 'obi_source o' for the CSC - and has many optional clauses.

• TOP specifies the number of rows to retrieve.
• WHERE specifies which rows to retrieve, according to the search criteria
• ORDER BY specifies an order in which to return the rows.

For example, submitting the following query in the ADQL tab will retrieve these results: the master source name, master source significance, master source broad band energy flux, and master source power law model photon index for the first 1000 catalog sources found which have a master source significance greater than 10.0, pile-up fraction smaller than ~10%, and a hard-to-soft hardness ratio greater than 0.7:

SELECT TOP 1000 m.name, m.significance, m.flux_aper_b, m.alpha FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)  

See the CSCview threads for more examples.

-- using command-line tools:

To non-interactively access the properties of the catalog through a URL, command-line tools such as cURL and GNU Wget may be used with the same query syntax as CSCview. cURL and Wget are utilities which allow the user to retrieve files with URL syntax from the command line, simulating the user's actions at a web browser. For example:

1. To perform a basic property search query from the command line:

unix% curl --form query='SELECT TOP 1000 m.name, m.significance, m.flux_aper_b, m.alpha FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)' 'http://cda/csccli/getProperties'

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT TOP 1000 m.name, m.significance, m.flux_aper_b, m.alpha FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7)'

2. To perform a basic cone search query from the command line:

unix% curl --form query='SELECT m.name, m.ra, m.dec, m.flux_aper_b FROM master_source m WHERE dbo.cone_distance(m.ra,m.dec,83.77333,-5.68464)<=10' 'http://cda/csccli/getProperties'

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT m.name, m.ra, m.dec, m.flux_aper_b FROM master_source m WHERE dbo.cone_distance(m.ra,m.dec,83.77333,-5.68464)<=10'

3. To upload a query to the URL:

% curl --form query=@/tmp/cscquery.adql 'http://cda/csccli/getProperties'

4. To specify how a catalog value of NULL should appear in a table of query results (e.g., instead of a blank space):

unix% curl --form query="SELECT TOP 50 o.obsid, o.obi, o.region_id, o.theta, o.mjr_axis_raw_s FROM obi_source o WHERE (o.instrument = 'ACIS' AND o.theta < 0.5 AND o.edge_code = 0 AND o.detect_significance_b > 10.0 AND o.pileup_warning < 0.1) ORDER BY theta DESC" 'http://cda/csccli/getProperties?nullAppearance=NULL'

5. To change the output coordinate format for the RA and DEC columns to decimal degrees (from the default sexagesimal format):

unix% wget -O out.file 'http://cda.cfa.harvard.edu/csccli/getProperties?query=SELECT TOP 1000 m.name, m.ra, m.dec FROM master_source m WHERE (m.significance > 10.0 AND m.pileup_flag = 0 AND m.hard_hs > 0.7) &coordFormat=decimal'

Results Interface

Once a catalog query has been entered and the "Search" button selected in the Query tab, the query results interface appears, displaying a table of search results and a list of available data products in the Results tab.

Results Tab

The cell widths of the query results table may be adjusted by dragging the cell boundaries with a mouse cursor; double-clicking a boundary resets the width to the default size.

Note: Query results stored in the Results tab will be lost if you re-visit the Query tab and modify the original query which produced those results, even if a new search hasn't been conducted; this is a security measure put in place to ensure the user always has a set of results consistent with the query form. Consider a likely scenario in which this feature is very useful: a user submits a query and its results are visible on the Results tab. He decides to return to the Query tab to make a small change. He then gets distracted by an important phone call before he can click "Search" to modify the results. When the telephone call finishes, he is no longer sure if the results match the original or modified query.

Products Tab

The filenames of all selected data products, along with file type and size, appear in the Products tab after the "Search" option has been selected, where they may be downloaded together as a tar file (by selecting the "Download" button beneath the pull-down menus) or batch file (by selecting the "File-->Generate Download Script" option). Decompressing the tar file produces a directory named "cdapackage...", which contains the downloaded data products compressed with gzip (Windows users should refer to the GNU zip website for .gz decompression options). The batch file contains a list of GNU Wget commands (one for each file) which can be used to download the batch of selected files directly through a URL from the command line.

Menus

There are five pull-down menus at the top of CSCview (File, Edit, View, Tools, Help), the contents of which apply to whichever view is currently active: either the query interface (the main view or ADQL view of the Query tab) or the query results interface (Results tab or Products tab). For example, the "Save" option in the File menu will save to a text file either the query specified in the Query tab or the results contained in the Results tab, depending upon which tab is open.

File

The Open item allows you to load a text file to which a previous CSCview query has been saved (from either the standard or ADQL view of the Query tab). In the standard view, the loaded query appears as a web-style query, and in the ADQL view, the ADQL command-line version of a query is displayed.

The Save item saves to a text file either a CSCview query or a table of query results, depending upon which tab is open. If this item is selected from the standard or ADQL view of the Query tab, the current query is saved to a file with a view-specific format (file extension .adql or .prop); note that the two query views need not contain the same query. If selected from the Results tab, the "Save" item saves to a text file the table of query results that appears after query submission. The file format is an extension of RDB, consisting of tab-separated columns of data beneath a header commented by '#', with a single line of dashes between the column titles and column data values. To convert the save files produced by CSCview to CIAO-compatible text files, refer to the thread Using a CSC Save File in CIAO.

Note: Source properties with a catalog value of 'NaN'/'NULL' appear as blank entries in the RDB-style file to which query results are saved. You may specify how 'Nan'/NULL' is represented in the output results file with the "Edit-->Preferences" menu item (or by using a command-line tool such as cURL).

The New item clears the Query tab of its current selections so that a new query may be entered. If the default startup query has been set to "Edit->Preferences->Startup Query->None", then the query form will appear empty after selecting "File->New". If "Edit->Preferences->Startup Query->Master Source Basic Summary" has been set, then a standard query will display.

The Search item queries the catalog database for source properties when selected from the Query tab, and data products when selected from the Results tab. While a search is in progress, a small window appears which reads "Searching..." and a "Cancel" option to stop the search.

The Download item allows you to download a tar file of the data products listed on the Products tab.

The Generate Download Script item saves a batch file containing a list of GNU Wget commands, one for each data product on the Products tab. This file can be used to download the batch of selected files directly through a URL from the command line.

The Reset item reloads CSCview, regardless of which tab is currently open; it effectively quits and restarts the application. It clears the Query tab of its current selections and loads the startup query, and resizes any vertical and horizontal bars which were adjusted. The default startup query form can either be empty, or display a standard query; this preference is set with the "Edit->Preferences->Startup Query" menu item.

The "Quit" item allows the user to exit the CSCview session; the user is prompted with the message "Are you sure you wish to quit this application?" before the GUI is closed.

Edit

The Preferences-->Null Value Representation item allows you to specify how a catalog value of 'NaN'/'NULL' should appear in a saved table of query results, instead of the blank entries which appear by default in the RDB-style file to which query results are saved.

You may select "Sexagesimal" or "Decimal" from the Preferences-->Output Coordinate Format item to specify the output coordinate format for ra* / dec* source property values in a table of query results.

The Preferences-->Floating Point Format item is used to specify how source property values should appear, in "default" or "native" format. The "native" catalog format includes numbers with many significant digits, exactly as they are output by the catalog processing pipeline, while numbers in the default format are truncated.

The Preferences-->Toolbar Appearance feature provides user control over the style of the toolbar display for ease of use; the choices are Icons and Text, Icons, and Text.

The Preferences-->Startup Query item provides control over the contents of the query form upon startup: either it should be empty ("None"), or display the "Master Source Basic Summary" standard query.

The Preferences->Startup Help menu item provides the option to either hide or display the Getting Started guide when CSCview is opened.

The Add item may be used in lieu of the drag-and-drop method to enter source properties into the appropriate interactive window on the Query tab to define a query, either the Search Criteria window, Result Set Window, Sort Order window. It also has the "Add Query" option to populate the query form with a selected standard query. The Remove option deletes highlighted source properties in any of the interactive windows on the Query tab; it can be used after dragging a standard query to the query form to undo the addition.

The Move Up and Move Down items may be used in lieu of the drag-and-drop method to move highlighted items up or down in a list, e.g. for re-ordering lists of source properties in the interactive windows of the Query tab.

The Deselect item deselects the current selection of source properties in the Query tab, or rows of data and/or data products in the Results tab.

View

The Query-->Show Language option allows the user to switch from the standard view of the Query tab to the ADQL view, where ADQL 2.0 query expressions may be entered; Query-->Return to Form restores the standard view of the Query tab.

The Properties and Products items allow the user to specify the visual sorting of the source properties and data products listed in the Source Properties window of the Query tab and Data Products window of the Results tab, respectively. The sorting can be done "By Group", e.g. "Source Fluxes", "Source Variability," etc., for source properties and "Images", "Exposure Maps", etc., for data products; "By Band", science energy band u, s, m, h, b, or w; or "By Name", alphabetically by source property or data product name.

The Status Bar box may be checked or cleared to bring to view or hide the dialog box at the bottom of CSCview (which records key actions and any errors that occur during a query session).

Tools:

Tip: Hovering the mouse cursor over a filename will show where the file was saved.

Help

Status bar

The status bar at the bottom of CSCview records a dialog of key actions and any errors which have taken place before, during, and after query submission, such as "Query loaded", "Running query", "Query completed: 50 rows found", "Query validation failed", and so on. The status bar may be hidden from view by clearing the "Status Bar" check box in the "View" pull-down menu.