This page is for the old CIAO 4.15 release. Please see the CIAO home page for the latest release.

CIAO "Watch Out" Page

This page lists noteworthy items and issues about the CIAO release. For the full list of known issues please review the:

• Bug List for CIAO Tools
• Data Caveats

Problem plotting with matplotlib on macOSX

macOSX users who installed CIAO using ciao-install are likely to have problems when plotting in sherpa and using various CIAO scripts due to a compatibility problem with the Tk libraries. Users will need to override the default plotting back end by modifying the matplotlib configuration file to use MacOSX instead of TkAgg.

$ cp $ASCDS_OTS/lib/python3.9/site-packages/matplotlib/mpl-data/matplotlibrc \
     $ASCDS_OTS/lib/python3.9/site-packages/matplotlib/mpl-data/matplotlibrc.bak

$ sed "s,backend: TkAgg,backend: MacOSX," \
  $ASCDS_OTS/lib/python3.9/site-packages/matplotlib/mpl-data/matplotlibrc.bak > \
  $ASCDS_OTS/lib/python3.9/site-packages/matplotlib/mpl-data/matplotlibrc

Now when we run sherpa we see

$ sherpa
-----------------------------------------------------
Welcome to Sherpa: CXC's Modeling and Fitting Package
-----------------------------------------------------
CIAO 4.15 Sherpa version 1 Monday, November 21, 2022

Python 3.9.9 (main, Mar 29 2022, 12:33:31) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.0.0 -- An enhanced Interactive Python. Type '?' for help.

IPython profile: sherpa
Using matplotlib backend: MacOSX

Problems using CIAO Tools (dax) Sherpa fitting tasks from ds9's Analysis menu

The same ciao-install Tcl/Tk issue affecting matplotlib on macOS (above) also affects the "DAX model editor". The model editor is used by each of the Sherpa fitting tasks available in the ds9 CIAO Tools (dax) analysis menu. The Tcl/Tk issue may cause the tasks to fail (Abort) or to display an unusable blank/black window. Unfortunately there is no work around for this problem with the ciao-install version.

macOS users that wish to use the Sherpa fitting tasks from ds9 must install CIAO using conda.

Problems with Xquartz 2.8.5

Some users have reported problems related to disappearing menus using ds9 with Xquartz version 2.8.5 which was released on January 26, 2023. This appears to be a bug in that version of Xquartz; users are advised to downgrade to version 2.8.4 to mitigate the problem.

Files and directories/folders with spaces or other special characters

Users are reminded that CIAO does not work with file names or directories/folders with spaces, " ", or other certain special characters.

SAOImage DS9

Help, I am unable to create regions in ds9!

By default, ds9 will no longer create regions when it starts; that is, clicking in the ds9 window will not create a region. This is because the ds9 interface has changed in version 7.3.2, as noted above, so that it no longer starts in region editing mode.

To create or edit regions, users must now manually switch to Region mode by selecting

Edit → Region

from the menu bar or the button bar.

Users can make this the default by editing their preferences:

Edit → Preferences ...
Menus and Buttons                  (left panel)
Edit → Menu → Region               (right panel)
Save

Unable to plot table columns with prism with DS9 8.4

As mentioned on the DS9 bug page, DS9 8.4 - which is included in CIAO 4.15 - can not be used to plot columns via the prism table viewer. Version 8.4.1 can be downloaded from the DS9 web site and an update for CIAO is planned for January.

Preserve During Load preferences

CIAO users should not enable any of the following options

Menus and Buttons → File → Menu → Preserve During Load → Pan
Menus and Buttons → File → Menu → Preserve During Load → Region

preferences. Due to the way CIAO starts ds9, having these preferences enabled leads to a half-pixel shift in regions. This shift accumulates if the region file is saved and re-loaded.

Installing & Starting CIAO

Using conda

Items that are relevant for people who install CIAO with conda:

Problems with NumPy 1.24

Resolved in CIAO 4.15.1

This issue has been resolved in the CIAO 4.15.1 patch.

NumPy 1.24 has just been released to conda-forge and CIAO does not work with it. A patch is planned for January, but users can replace it with an earlier version. If CIAO has already been installed with conda and the NumPy version is greater than 1.23 - e.g.

% conda list numpy
# packages in environment at /home/ciaouser/miniconda3/envs/ciao-4.15:
#
# Name                    Version                   Build  Channel
numpy                     1.24.0          py310h08bbf29_0    conda-forge

then say:

$ conda install -c "numpy<1.24"

or, if CIAO has not been installed yet, add numpy<1.24 to the conda call when installing CIAO:

$ conda create -n ciao-4.15 \
  -c https://cxc.cfa.harvard.edu/conda/ciao \
  -c conda-forge \
  ciao sherpa ds9 ciao-contrib caldb_main marx "numpy<1.24"

Examples of the problems with NumPy 1.24 include being unable to import crates

$ python -c 'import pycrates'
/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/pycrates/cratedata.py:31: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  from numpy import object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/pycrates/__init__.py", line 32, in <module>
    from pycrates.cratedata import *
  File "/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/pycrates/cratedata.py", line 31, in <module>
    from numpy import object
ImportError: cannot import name 'object' from 'numpy' (/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/numpy/__init__.py)

or being unable to start sherpa

$ sherpa
-----------------------------------------------------
Welcome to Sherpa: CXC's Modeling and Fitting Package
-----------------------------------------------------
Sherpa 4.15.0

Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:26:04) [GCC 10.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.7.0 -- An enhanced Interactive Python. Type '?' for help.

IPython profile: sherpa
Using matplotlib backend: QtAgg
/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/pycrates/cratedata.py:31: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  from numpy import object
[TerminalIPythonApp] WARNING | Unknown error in handling startup files:
...
... more traceback
...
ImportError: cannot import name 'object' from 'numpy' (/home/ciaouser/miniconda3/envs/ciao-4.15/lib/python3.10/site-packages/numpy/__init__.py)

sherpa In [1]: 

Use the Intel conda build on Mac

On macOS, support for the ARM version is provided by Rosetta2 emulation rather than with native code generation. This means that conda users must use the Intel x86-64 MacOS distribution of Python, and not the ARM64 version.

Depending on the version/architecture of conda that is installed, some users may get an error message that looks like this:

PackagesNotFoundError: The following packages are not available from current channels:

- sherpa
- ciao
- ds9
- marx

Current channels:

- https://cxc.cfa.harvard.edu/conda/ciao/osx-arm64
...

Note the reference to the osx-arm64. If this error occurs, then users can tell conda to install the Intel x86-64 version using the CONDA_SUBDIR environment variable

$ env CONDA_SUBDIR=osx-64 conda create -n ciao-4.15 \
  -c https://cxc.cfa.harvard.edu/conda/ciao -c conda-forge \
  ciao sherpa ds9 ciao-contrib caldb_main marx python=3.10

$ conda activate ciao-4.15
$ conda config --env --set subdir osx-64

The first command tells conda to install the Intel x86-64 binaries. The second and third commands ensure that any updates or additional packages added to that environment will continue to use the Intel x86-64 versions.

Using ciao-install

Items that are relevant for people who install CIAO with ciao-install:

bash, ksh, and zsh use ciao.sh

In CIAO 4.15, the obsolete CIAO setup scripts for bash (ciao.bash) and ksh (ciao.ksh) shells have been removed. All users who installed CIAO with ciao-install that use any of the Bourne shell derivatives including bash, zsh (for macOS Catalina and later), and ksh, must source the ciao.sh script to setup for CIAO.

$ alias ciao="source /soft/ciao-4.15/bin/ciao.sh"
$ ciao

Users who have installed CIAO with conda will continue to activate the environment as described on the download page.

Python 3.9 support only

In CIAO 4.15 installed using ciao-install only Python 3.9 is provided.

The Python 3 for Scientists website provides guidance on updating code written for Python 2, and the CXC Helpdesk can provide support.

Users of the conda installation can select python 3.10, 3.9, or 3.8.

Other issues

CIAO on Mac needs recent XQuartz and XCode

OSX users, particularly those that have upgraded from an earlier OSX version, should ensure that compatible versions of XQuartz (at least version 2.7.11) and XCode (also comes as part of Apple Developer Tools) are installed on the system before proceeding with the CIAO installation.

macOSX Catalina: ls: Operation not permitted

Since macOSX Catalina, Apple has restricted access to various folders in the user's home directory including Documents, Desktop, and Downloads. By default the Terminal application and Xquart's xterm application do not have access to these folders. Attempting to list the contents from the command line yeilds the following error message:

$ cd Desktop
$ ls
ls: Operation not permitted

This restriction may impede CIAO users who commonly use these folders.

Users can still navigate and display the contents of these folders using the Finder application. CIAO users who want to use these restricted folders need to grant the Terminal and|or xterm applications "Full Disk Access". The setting can be found in the "Security" settings application.

No support for 32-bit Linux, nor OSX 10.14 (Mojave) and earlier.

CIAO is no longer available for 32-bit Linux, nor is it available for OSX 10.14 (Mojave) or earlier versions of OSX. The platform support page describes the operating systems that can be used with CIAO.

Fontconfig error: Cannot load default config file

During the smoke tests some users will see the following message during the obsvis test

Fontconfig error: Cannot load default config file

This error is not critical. As long as the test reports a PASS, then users can ignore that message.

Parameter file conflicts with HEAsoft

Both CIAO and FTOOLS use parameter files to store parameter values for tools, and provide tools for manipulating these files (e.g. plist, pset, punlearn; they have the same name in both systems). The location of the parameter files is taken from the $PFILES environment variables for CIAO and FTOOLS (or $UPARM if $PFILES does not exist).

Common errors are:

• pget: could not get parameter "foo"
  

• The following parameter:
      "foo,b,l,no,,,"Keyword""
  has the following error(s):
    o Mode field "l" is invalid.
  

• The following parameter:
  "foo,i,h,INDEF,INDEF,INDEF,"description""
  has the following error(s):
  o Problem converting minimum field "INDEF" to parameter type "i".
  o Problem converting maximum field "INDEF" to parameter type "i".
  

We recommend that you start FTOOLS before CIAO to minimize conflicts; more information on this problem is available in the CIAO and FTOOLS section of the Starting CIAO thread. Also refer to the related FAQ "The tool is failing because ")sval" has somehow shown up in the parameter value.".

Conflicts with Other Software Packages

There are some known conflicts between CIAO and other software or system libraries and tools. Issues not covered in this "Watch Out" page are listed on the "Other Software Packages" bug page.

Analysis

Incorrect GTIs when time-filtering grating data with dmcopy opt=all

When a time filter is applied to a level=1.5 (evt1a.fits) or level=2 (evt2.fits) grating file and the dmcopy opt=all option is used, the GTI block in the output file is not updated to reflect the correct time range. Time-related header keyword values - such as ontime, livetime, and exposure - will be incorrect because they are calculated from the time ranges in the GTI block.

This bug affects all ACIS and HRC grating data analysis. Read the bug report for details and a workaround.

Python

Installing 3rd party modules with pip3

Users must be careful when using pip3 to install 3rd party modules such as Astropy and Scipy into CIAO. The latest version of these packages MAY require updated versions of packages that CIAO needs; numpy being the most likely conflict.