copy_colvals

Context: crates

Syntax

copy_colvals(crate, colname)
copy_colvals(crate, colnum)

Description

crate - input table crate
colname - column name (case insensitive)
colnum - column number, where the first column is numbered 0

The copy_colvals command retrieves a numpy array of the values of the specified column within a table crate. The data values are copied into an array and that array is returned.

The get_col_names() routine ("ahelp get_col_names") can be used to find the columns in a Crate.

The get_col command ("ahelp get_col") returns a CrateData object for the column, which is useful if you are interested in the metadata associated with it.

Example 1

>>> cr = read_file("evt2.fits")
>>> t = copy_colvals(cr, "time")
>>> print(t[0:3])
[ 52379718.80918065 52379718.80918065 52379718.80918065]

Copy the values of the "time" column from the crate "cr", then print out the first three elements of the array.

Example 2

>>> cr = read_file("evt2.fits")
>>> print(get_col_names(cr))
['time' 'ccd_id' 'node_id' 'expno' 'chip(chipx,chipy)'
'tdet(tdetx,tdety)'
'det(detx,dety)' 'sky(x,y)' 'pha' 'pha_ro' 'energy' 'pi' 'fltgrade'
'grade' 'status']
>>> vals = copy_colvals(cr, 7)
>>> print(vals.shape)
(603495, 2)
>>> print(vals[0:3])
[[ 3892.52172852 4165.99365234]
[ 3908.08300781 4142.32275391]
[ 3849.05786133 3993.52783203]]

The values from column number 7 - the sky(x,y) column in this file - are copied to the variable "vals".

Example 3

>>> cr = read_file("evt2.fits")
>>> sky = copy_colvals(cr, "sky")
>>> y = copy_colvals(cr, "Y")
>>> print(sky.shape)
(124940, 2)
>>> print(y.shape)
(124940,)
>>> print(get_number_rows(cr))
124940

Use copy_colvals to get the values of vector column SKY(X,Y) from the crate "cr". Then get the Y component values as a separate array.

Should I use copy_colvals or get_colvals?

The general rule of thumb is to use copy_colvals rather than get_colvals, particularly for interactive use from the ChIPS, Sherpa or IPython shells.

The return value from get_colvals reflects the contents of the Crate, so that changes to the return value also change the crate, whereas copy_colvals returns a copy of this data. This can be seen in the example below, which uses as an input file:

unix% cat tbl.dat
# x y
1  2
3  5
9  10

The steps taken below read in the file into a crate, extract the x and y column values, change the middle value of each column, check to see what values are stored in the crate, and then write out the crate to a new file. The x column is accessed using copy_colvals whereas the y column data is retrieved using get_colvals to show the difference in behavior.

>>> cr = read_file("tbl.dat")
>>> x = copy_colvals(cr, "x")
>>> y = get_colvals(cr, "y")
>>> print(x)
[ 1. 3. 9.]
>>> print(y)
[ 2. 5. 10.]
>>> x[1] = 5
>>> y[1] = 9
>>> print(x)
[ 1. 5. 9.]
>>> print(get_colvals(cr, "x"))
[ 1. 3. 9.]
>>> print(y)
[ 2. 9. 10.]
>>> print(get_colvals(cr, "y"))
[ 2. 9. 10.]
>>> write_file(cr, "tbl2.dat[opt kernel=text/simple]")

The output file contains:

unix% cat tbl2.dat
#TEXT/SIMPLE
# x y
1.0 2.0
3.0 9.0
9.0 10.0

So changing the values in the x array - which was created using copy_colvals - does not change the values stored in the crate, whereas changes to the values in the y array - which was created with get_colvals - are propogated to the crate, and hence into the output file tbl2.dat.

Note that the input file - here tbl.dat - is not changed by these operations.

It is generally going to be safer to use copy_colvals rather than get_colvals unless:

you do want to be able to change values in the crate for either future processing or to write out to disk
you have read in a large file and do not want to waste memory by creating another copy of the array

If the column being read in is a virtual one then the data is always going to be copied, so in this particular case there is no difference between copy_colvals and get_colvals.

Accessing 'virtual' columns

The copy_colvals() and get_colvals() commands can be used to access "virtual" columns - that is a column defined via a coordinate transform on an existing column, such as the CEL_BRI column from a radial profile created by dmextract or the EQPOS values in an Chandra event file. It is best to expliclty include the column names of any such column you wish to use when reading in the file - e.g. say

cr = read_file("profile.fits[cols cel_r, cel_bri]")
x = copy_colvals(cr, "cel_r")
y = copy_colvals(cr, "cel_bri")
add_histogram(x[:,0], x[:,1], y)

to avoid a bug in the CIAO Data Model in which the values of the virtual columns are calculated incorrectly.

Bugs

See the bug pages on the CIAO website for an up-to-date listing of known bugs.

Refer to the CIAO bug pages for an up-to-date listing of known issues.