Installing CIAO 4.1

CIAO 4.1 Science Threads

Overview

Last Update: 20 Apr 2009 - updated to include CIAO 4.1.2 patch instructions; CALDB version updated to 4.1.2; added the test the CALDB installation step; there is a new tool - ciao-install - that automates the installation of CIAO and CALDB

Synopsis:

How to install version 4.1 of CIAO, the Chandra Interactive Analysis of Observations software package. This information is also available in the INSTALL_BINARY file that is packaged with CIAO.

[New] There is a new tool, ciao-install, which automates the installation and upgrade of CIAO and CALDB. Use of this tool, which is a beta release for CIAO 4.1.2, is described in the ciao-install thread.

This thread does not explain how to re-compile CIAO yourself. Refer to the INSTALL_SOURCE file for information on that process.

Related Links:

Use ciao-install: use the ciao-install tool to automate the installation of CIAO and the CALDB
Patching CIAO 4.1 to CIAO 4.1.1: how to upgrade to CIAO 4.1.1 if you already have CIAO 4.1 installed.
Patching CIAO 4.1 or 4.1.1 to CIAO 4.1.2: how to upgrade to CIAO 4.1.2 if you already have CIAO 4.1 or 4.1.1 installed.

Proceed to the HTML or hardcopy (PDF: A4 | letter) version of the thread.

System Requirements
Downloading the Software
Where should CIAO and CALDB be installed?
Installing the CIAO Software
Overlay the CIAO 4.1.1 patch (Optional)
Overlay the CIAO 4.1.2 patch
Contributed Science Scripts
Create the Help File Indexes
Run the Smoke Tests
- Known Smoke Test Issues
Test the CALDB installation
Clean Up the Tarfiles
Notes
Summary
History

System Requirements

CIAO is supported on a number of operating systems, which are listed on the Platform Support page. Certain system requirements are necessary to successfully install and run CIAO; review these system requirements before continuing with the installation.

Downloading the Software

To install CIAO, you need the following packages from the CIAO download pages:

Binaries: ciao-4.1-bin-<component>-<OS>.tar.gz

Contains the CIAO tools, libraries, parameter files, and help files. Also contains the pre-compiled off-the-shelf (OTS) programs required to run CIAO. There are five components in CIAO 4.1: core, tools, chips, sherpa, and obsvis.

The relationship between <OS> and the supported operating systems is given in the following table:

<OS> Operating system

fc8 Fedora Core 8

fc4 Fedora Core 4

osxi OS-X with an Intel processor

osxppc OS-X with a PPC processor

sun Solaris 8 and 9

sun10 Solaris 10

CIAO 4.1.1 (optional, all platforms): there is a platform-independent patch which contains Cycle 11 proposal planning calibration files: ciao-4.1.1-tools.tar. This is required for users of PIMMS or if you are planning to use the ciao-install tool to install CIAO.

CIAO 4.1.2: the platform-specific binary patch file: ciao-4.1.2-patch-<OS>.tar.gz.

Solaris 10 Note: the CIAO 4.1.2 release for Solaris 10 is a full release, so there are no 4.1.1 or 4.1.2 patch files for that platform and the binary files are called ciao-4.1.2-bin-xxx rather than ciao-4.1-bin-xxx.
Chandra Calibration Database (CALDB)
Contributed science scripts and modules: CIAO_4.1_scripts.tar

`<OS>`	Operating system
`fc8`	Fedora Core 8
`fc4`	Fedora Core 4
`osxi`	OS-X with an Intel processor
`osxppc`	OS-X with a PPC processor
`sun`	Solaris 8 and 9
`sun10`	Solaris 10

Save the tarfiles to a location that is accessible from where you wish to install CIAO. These files were used to install CIAO on a Fedora Core 8 machine:

unix% ls -1
acis_bkgrnd_4.1.2.tar.gz
caldb_4.1.2_main.tar.gz
ciao-4.1.1-tools.tar.gz
ciao-4.1.2-patch-fc8.tar.gz
ciao-4.1-bin-chips-fc8.tar.gz
ciao-4.1-bin-core-fc8.tar.gz
ciao-4.1-bin-obsvis-fc8.tar.gz
ciao-4.1-bin-sherpa-fc8.tar.gz
ciao-4.1-bin-tools-fc8.tar.gz
CIAO_4.1_scripts.tar

Where should CIAO and CALDB be installed?

The CIAO software package and the Chandra Calibration Database (CALDB) can be installed in any location, and does not need to be installed with root or super-user privileges.

CIAO

As of CIAO 4.0, CIAO unpacks into a directory called ciao-<version>/, so for the current release this is ciao-4.1/; prior to CIAO 4.0 the files unpacked into the working directory. So if CIAO were to be installed into /usr/local/, it would be placed into the directory /usr/local/ciao-4.1/. This directory (/usr/local/ciao-4.1/) is referred to as the root directory of CIAO in the rest of this thread. It is also the value that the $ASCDS_INSTALL environment variable is set to once CIAO has been started.

In the rest of this thread we will assume CIAO is being installed into /soft/.

CALDB

You can choose to install the CALDB within the CIAO installation - in the directory $ASCDS_INSTALL/CALDB or in a separate directory, in which case $ASCDS_INSTALL/CALDB should be a soft link to the actual location.

For this thread, we assume that the database will be installed in the /soft directory:

/soft/CALDB/

It is suggested that you install the CALDB before starting the CIAO installation, as CIAO has to be linked to the location of those files. Installation instructions are available on the CALDB website:

Installing (or upgrading) the CALDB

Installing CIAO as the root user

It is is strongly suggested that CIAO be installed as a non-privileged user and not as the root user. If CIAO must be installed in a directory owned by root and you are following the manual process, then

unpack CIAO as root (so creating the ciao-4.1/ directory);
use the chown utility to change ownership of this directory to the installer's username.

Installing the CIAO Software

1. Unzip the files

First we have to uncompress all the gzip'ed files (note that some browsers automatically uncompress files after they are downloaded):

unix% gunzip *gz

This process will take a few minutes. When gunzip is done running, there will only be .tar files in the directory:

unix% ls -1
ciao-4.1.1-tools.tar
ciao-4.1.2-patch-fc8.tar.gz
ciao-4.1-bin-chips-fc8.tar
ciao-4.1-bin-core-fc8.tar
ciao-4.1-bin-obsvis-fc8.tar
ciao-4.1-bin-sherpa-fc8.tar
ciao-4.1-bin-tools-fc8.tar
CIAO_4.1_scripts.tar

2. Change to the CIAO installation directory

The CIAO 4.1 tarfiles create a directory named ciao-4.1 for the software installation. It is not necessary to create the CIAO directory before starting the installation.

Move to the directory in which you would like the root of the CIAO installation to be located. If you are installing the software for a local network, this directory must also be visible to all potential CIAO users.

This installation will be done in the /soft directory:

unix% cd /soft/

3. Unpack CIAO 4.1

From the CIAO location just chosen, untar the CIAO 4.1 binary packages. (If you prefer more screen output, add the "v" option to the tar command.)

Solaris users must use GNU tar, as shown in this example (see the system requirements page). Users on other platforms who don't have gtar installed can use tar with the same "xf" options.

The CIAO patch files are not unpacked in this step.

unix% pwd
/soft

unix% gtar xf /some/path/ciao-4.1-bin-chips-fc8.tar 

unix% gtar xf /some/path/ciao-4.1-bin-core-fc8.tar 

unix% gtar xf /some/path/ciao-4.1-bin-obsvis-fc8.tar

unix% gtar xf /some/path/ciao-4.1-bin-sherpa-fc8.tar 

unix% gtar xf /some/path/ciao-4.1-bin-tools-fc8.tar

When all the files are unpacked, the CIAO directory will look like:

unix% ls ciao-4.1
bin/           data/           ots/
binexe/        doc/            param/
config/        include/        README
config.guess*  INSTALL_BINARY  README.cycle9
config.h.in    install-sh*     README.obsvis
config.sub*    INSTALL_SOURCE  share/
configure*     lib/            test/
configure.in*  LICENSE         VERSION
COPYING*       LICENSE.SAO     VERSION.prop_tk
COPYING.LIB*   Makefile

4. Run the configuration script

Run the CIAO configuration script:

unix% cd ciao-4.1/

unix% ./configure
Running CXC configure, v4.3.4
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for more... /bin/more
checking for main in -lpthread... no
--with-imagerdir not specified, defaulting to /soft/ciao-4.1/ots/bin
configure: creating ./config.status
config.status: creating bin/ciao_setup.sh
config.status: creating ots/lib/python2.5/config/Makefile
config.status: creating bin/ciao.sh
config.status: creating bin/ciao.csh
config.status: creating bin/ciao.ksh
config.status: creating bin/ciao.bash
config.status: creating bin/setup_obsvis.sh
config.status: creating bin/setup_obsvis.csh
config.status: creating bin/setup_chips.sh
config.status: creating bin/setup_chips.csh
config.status: creating bin/setup_sherpa.sh
config.status: creating bin/setup_sherpa.csh
config.status: creating bin/setup_tools.sh
config.status: creating bin/setup_tools.csh
config.status: creating bin/ciao-python-fix
config.status: creating test/smoke/bin/setup.sh
config.status: creating test/Makefile
config.status: creating ots/etc/pango/pangorc
config.status: creating ots/etc/pango/pango.modules
config.status: creating ots/etc/gtk-2.0/gtk.immodules
config.status: creating ots/etc/gtk-2.0/gdk-pixbuf.loaders

5. Link to CALDB

Create a link to the calibration data, changing the path to match your system installation:

unix% pwd
/soft/ciao-4.1/

unix% ln -s /soft/CALDB CALDB

At this point, your directory should look like:

unix% ls
bin/            COPYING*        Makefile
binexe/         COPYING.LIB*    ots/
CALDB@          data/           param/
config/         doc/            README
config.guess*   include/        README.cycle9
config.h.in     INSTALL_BINARY  README.obsvis
config.log      install-sh*     share/
config.status*  INSTALL_SOURCE  test/
config.sub*     lib/            VERSION
configure*      LICENSE         VERSION.prop_tk
configure.in*   LICENSE.SAO

6. Create a CIAO alias

It is strongly suggested that you create a CIAO alias before continuing.

The alias syntax depends on the shell (tcsh, csh, bash); see this FAQ for help in determining which shell you are using.

csh/tcsh users should add the following to their $HOME/.cshrc file.
```
alias ciao "source /soft/ciao-4.1/bin/ciao.csh"
```
bash users should add the following to their $HOME/.bashrc file.
```
alias ciao=". /soft/ciao-4.1/bin/ciao.bash"
```
ksh users should add the following to their $HOME/.login file.
```
alias ciao=". /soft/ciao-4.1/bin/ciao.ksh"
```

More information is given in the Running CIAO section of the Starting CIAO thread.

Once the alias is defined, you will be able to simply type

unix% ciao

and CIAO will be setup and ready to use in that window.

Solaris 10 users should now skip to the Contributed Science Scripts step.

Overlay the CIAO 4.1.1 patch (Optional)

You are not required to install the CIAO 4.1.1 patch before updating to CIAO 4.1.2. Version 4.1.1 may be skipped if you do not need the Cycle 11 proposal planning calibration files used by PIMMS.

The patch unpacks into the "ciao-4.1" directory where CIAO is installed. In our example, this directory is located within /soft/; if you do not recall where your root directory is, start CIAO and type:

unix% cd $ASCDS_INSTALL

unix% cd ..

unix% pwd
/soft/

From this directory, untar the patch file:

unix% tar xf /some/path/ciao-4.1.1-tools.tar

Next, run the patch script:

unix% cd ciao-4.1/

unix% ./ciao_patch.sh

The script does not print any output. It moves the updated files to the $ASCDS_INSTALL/config/pimms/data/ directory and renames the old files to ".<filename>_CIAO_4.1".

The patch is now installed. When you start CIAO, the version number indicates this:

CIAO configuration is complete...
  CIAO version     :  CIAO 4.1.1 Wednesday, January 14, 2009
  Proposal Toolkit version :  Cycle 11 Friday, December 5, 2008
  Python path      :  CIAO

If disk space is an issue, you can run the ciao_cleanup.sh script to delete the files replaced by CIAO 4.1.1.

unix% pwd
/soft/ciao-4.1/

unix% ./ciao_cleanup.sh

Overlay the CIAO 4.1.2 patch

unix% cd $ASCDS_INSTALL

unix% cd ..

unix% pwd
/soft/

From this directory, untar the patch file:

unix% tar xf /some/path/ciao-4.1.2-patch-fc8.tar

Next, run the patch script:

unix% cd ciao-4.1/

unix% ./ciao_patch.sh

The script does not print any output. It installs the CIAO 4.1.2 files in the correct directories and renames the old files to ".<filename>_CIAO_4.1.1".

Run the CIAO configuration script again:

unix% ./configure
Running CXC configure, v4.3.4
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for more... /bin/more
checking for main in -lpthread... no
--with-imagerdir not specified, defaulting to /export/junk_dir/ciao-4.1/ots/bin
configure: creating ./config.status
config.status: creating bin/ciao_setup.sh
config.status: creating ots/lib/python2.5/config/Makefile
config.status: creating bin/ciao.sh
config.status: creating bin/ciao.csh
config.status: creating bin/ciao.ksh
config.status: creating bin/ciao.bash
config.status: creating bin/setup_obsvis.sh
config.status: creating bin/setup_obsvis.csh
config.status: creating bin/setup_chips.sh
config.status: creating bin/setup_chips.csh
config.status: creating bin/setup_sherpa.sh
config.status: creating bin/setup_sherpa.csh
config.status: creating bin/setup_tools.sh
config.status: creating bin/setup_tools.csh
config.status: creating bin/ciao-python-fix
config.status: creating test/smoke/bin/setup.sh
config.status: creating test/Makefile
config.status: creating ots/etc/pango/pangorc
config.status: creating ots/etc/pango/pango.modules
config.status: creating ots/etc/gtk-2.0/gtk.immodules
config.status: creating ots/etc/gtk-2.0/gdk-pixbuf.loaders

The patch is now installed. When you start CIAO, the version number indicates this:

CIAO configuration is complete...
  CIAO version     :  CIAO 4.1.2 Thursday, April 2, 2009
  Proposal Toolkit version :  Cycle 11 Friday, December 5, 2008
  Python path      :  CIAO

If disk space is an issue, you can run the ciao_cleanup.sh script to delete the files replaced by CIAO 4.1.2.

unix% pwd
/soft/ciao-4.1/

unix% ./ciao_cleanup.sh

Contributed Science Scripts

The contributed science scripts are written and maintained by local CIAO users at the CXC. More information on each of the scripts is available from the CIAO 4.1 Contributed Scripts Page. Check that page for announcements about new scripts and updates to existing ones.

The CIAO_4.1_scripts.tar file is designed to be unpacked in $ASCDS_INSTALL, the root directory of the CIAO installation. In our example, this directory is /soft/ciao-4.1; if you do not recall where your root directory is, start CIAO and type "cd $ASCDS_INSTALL" to get there.

unix% cd /soft/ciao-4.1

Since this is a new CIAO directory (i.e. not a patch to an existing version of CIAO), we do not need to worry about uninstalling old scripts.

Unpack the script tarfile:

unix% tar xvf /some/path/CIAO_4.1_scripts.tar
contrib/Changes.CIAO_scripts
contrib/README_CIAO_scripts
contrib/VERSION.CIAO_scripts

...

contrib/bin/tg_bkg
contrib/doc/xml/tg_bkg.xml
contrib/uninstall_4.1_scripts

The tarfile contains a VERSION.CIAO_scripts file. Compare the timestamp in this file to the "Last Updated" date on the Scripts page to determine when you need to update your local scripts installation.

Create the Help File Indexes

In order for CIAO to be able to access the help files packaged with the software, it is necessary to create the ahelp database indexes. This step must be run by the user who owns the CIAO installation.

Start CIAO and run ahelp -r:

unix% ciao
CIAO configuration is complete...
  CIAO version     :  CIAO 4.1.2 Thursday, April 2, 2009
  Proposal Toolkit version :  Cycle 11 Friday, December 5, 2008
  bin dir          :  /soft/ciao-4.1/bin
  Python path      :  CIAO

unix% ahelp -r

Processing file '/soft/ciao-4.1/doc/xml/ChipsId.xml'...OK

Processing file '/soft/ciao-4.1/doc/xml/Attributes.xml'...OK

Processing file '/soft/ciao-4.1/doc/xml/ChipsDictionary.xml'...OK

....

Processed Files  : 916
Succeeded Files  : 916
Failed Files     : 0
Purged Duplicates: 0

Creating Key Indexes...DONE
Subject DB Size: 1577

Creating Context Indexes...DONE
Context DB Size: 1577

Creating Crossreference DB...DONE
Crossreferences DB Size: 17888

Creating Seealso  DB...DONE
Seealso DB Size: 2125

Run the Smoke Tests

A set of smoke tests are packaged with CIAO 4.1. The tests are designed to spot-check the installation for critical problems, not to run a complete regression test of the system. The output created by the new installation of CIAO is compared against a baseline set of files included in the package.

The runtime for the smoke tests varies based on platform: 5-10 minutes on a 3GHz Linux machine and on order of 30 minutes on a 1GHz Solaris machine. Some of the tests launch ChIPS windows and GUI applications, such as prism; these applications will be closed automatically when the test is complete.

unix% cd $ASCDS_INSTALL/test
unix% pwd
/soft/ciao-4.1/test

The smoke test syntax depends on the shell (tcsh, csh, bash); see this FAQ for help in determining which shell you are using.

csh/tcsh users:

unix% make -k test |& tee ~/ciao_test.log

bash or ksh users:

unix% make -k test 2>&1 | tee ~/ciao_test.log

This example is run using the tcsh syntax:

unix% make -k test |& tee ~/ciao_test.log

PATH=${PWD}/smoke/bin:${PATH} && \
export PATH && \
make test-ui test-obsvis test-chips test-sherpa \
test-dm test-crates test-tools
make[1]: Entering directory `/soft/ciao-4.1/test'

Running test prism-smoke001
prism-smoke001 completed SUCCESSFULLY

...

Running test tools-dmstat3
PASS: Test tools-dmstat3 completed successfully

make[1]: Leaving directory `/soft/ciao-4.1/test'

A successful test prints a message:

  <testname> completed SUCCESSFULLY

  PASS: Test <testname> completed SUCCESSFULLY

An unsuccessful test will report:

  <testname> FAILED
  Review /tmp/smoke/<testname>/diff.log for details

The smoke tests do not currently print a summary of successful and failed tests. Review the log file or screen output to be certain that none of the tests failed.

Known Smoke Test Issues

There is a bug page for known smoke test issues. If you experience a problem that isn't listed there, submit it to Helpdesk.

Test the CALDB installation

[New] (20 Apr 2009)

The contributed scripts package now contains a tool - check_ciao_caldb - that performs a simple check of the Chandra CALDB installation. Running the tool should produce screen output like the following:

unix% check_ciao_caldb
CALDB environment variable = /soft/ciao-4.1/CALDB
        which is a link to = /soft/CALDB/
             CALDB version = 4.1.2
             relelase date = 2009-03-26T18:00:00
CALDB query completed successfully.

For a more-thorough validation please follow the chkcif steps described on the download CALDB page.

Clean Up the Tarfiles

Finally, you may delete the tarfiles from the original download directory:

unix% rm /some/path/ciao*tar

unix% rm /some/path/CIAO_4.1_scripts.tar

Notes

Solaris: "@LongLink" file

If a file named "@LongLink" is created when the tarfiles are unpacked on Solaris, it is an indication that tar was used instead of GNU tar ("gtar").

As noted in the System Requirements section, Solaris users must use gtar to unpack the files. If native Solaris tar is used, some of the test filenames will be truncated.

Known Conflicts

There are some known conflicts between CIAO and other software or system libraries and tools. They are documented in the following webpages:

Usage and Licensing

All non-OTS source code will be freely available in support of any CIAO release. The software is subject to Smithsonian Institution Copyright, a statement of which can be found in the headers of the source files, as well as at $ASCDS_INSTALL/LICENSE.SAO.

CIAO 4.1 is further governed by the GNU General Public License ($ASCDS_INSTALL/COPYING) and/or the GNU Library General Public License ($ASCDS_INSTALL/COPYING.LIB), both of which are bundled with the software.

Summary

The CIAO installation is now complete. For help getting started with the software, read the Introductory science threads.

History

12 Aug 2005	original version, new for CIAO 3.2.2
14 Nov 2005	updated for CIAO 3.3: added Linux8 package: renaming libgcc_s.so.1 and Mac OS X 10.2: Using ds9 v3.0.3 notes.
12 Jan 2006	updated to include CIAO 3.3.0.1 patch instructions
01 Dec 2006	updated for CIAO 3.4 and CALDB 3.3.0
29 Jan 2007	updated to include CIAO 3.4.1[.1] patch instructions
14 Dec 2007	updated for CIAO 4.0: tarfile creates a directory into which it unpacks; CIAO is shipped with gzip support enabled (no longer need to specify an option in the configure step); smoke tests are included in the AUX tarfile; CALDB 3.4.2 is used
21 Feb 2008	updated to include CIAO 4.0.1 patch instructions
26 Feb 2008	consistently use "/soft" as the installation directory
05 Mar 2008	renamed "Reindexing the Help Files" section to "Create the Help File Indices"
10 Mar 2008	system requirements moved to a separate page; "Create the Help File Indices" should be "Create the Help File Indexes"; CIAO does not need to be installed by the root user; "`ahelp -r`" must be run as user who owns installation; Mac users need to add xrdb to the PATH; FC7 expected smoke test failure
16 Apr 2008	updated to include CIAO 4.0.2 patch instructions; location of CIAO alias updated for bash users
08 Sep 2008	corrected ChIPS test number in Fedora Core 7 smoke test error (test 3 prints the error, not test 5)
15 Dec 2008	updated for CIAO 4.1: CALDB 4.1.0 is used, new supported platform (FC8 instead of FC7); added Known Smoke Test Issues section; removed the "setting xrdb path" section (needed for .CXCdefaults file, which the GUIs no longer use)
15 Jan 2009	updated to include CIAO 4.1.1 patch instructions; include full alias syntax in CIAO alias section; added bash syntax in Run the Smoke Tests section
29 Jan 2009	added "Installing CIAO as root user" information to Choose a CIAO installation directory section
20 Apr 2009	updated to include CIAO 4.1.2 patch instructions; CALDB version updated to 4.1.2; added the test the CALDB installation step; there is a new tool - `ciao-install` - that automates the installation of CIAO and CALDB