Last modified: 22 Aug 2023

URL: https://cxc.cfa.harvard.edu/sherpa/threads/xspec_user_model/

Using XSpec User Models in Sherpa

Sherpa Threads (CIAO 4.16 Sherpa)


Overview

Synopsis:

The experimental script convert_xspec_user_model provides support for using some XSpec contributed models—whether written youself or obtained from the new XSpec models page—from Sherpa.

Last Update: 22 Aug 2023 - Rewritten for CIAO 4.15; examples using binaries and Conda distribution of CIAO on Linux and MacOS platforms with different user models.


Contents


Preamble

The convert_xspec_user_model script is provided as part of the CIAO contributed-scripts package. The script underwent significant enhancements and 'under-the-hood' re-working that was introduced alongside the CIAO 4.15 release.

For the purposes of the thread,

  • Version 2.3 of the relxill collection of relativistic and non-relativistic reflection models will be used for the example of building the models for the pre-compiled CIAO binaries installed with the ciao-install script.
  • Version 1.0.1 of the RELTRANS reverberation mapping models will be used for the example of building with a Conda distribution of CIAO.
  • Version 2.47 of the WARMABS XSpec package—a collection of warm absorption and photoionized emission models derived from the XSTAR program that calculates physical conditions and spectra of photoionized gasses.

will be used for the examples herein. The convert_xspec_user_model script is similar to the XSpec initpackage command, so follow the model instructions to make any changes that would be needed before running initpackage, and then run convert_xspec_user_model instead.

[TIP]
MacOS and the XCode SDK

Often, compilers require access to system files to correctly build programs and a standardized set for MacOS is available through XCode's Software Development Kit (SDK).

The CIAO binary installation already requires XCode presently, so you may set:

(t)csh
unix% setenv SDKROOT `xcrun --show-sdk-path`

bash/zsh
unix% export SDKROOT=`xcrun --show-sdk-path`

For Conda, it may be also necessary to set CONDA_BUILD_SYSROOT,

(t)csh
unix% setenv CONDA_BUILD_SYSROOT `xcrun --show-sdk-path`

bash/zsh
unix% export CONDA_BUILD_SYSROOT=`xcrun --show-sdk-path`

but if XCode is not available on your system for Conda, the SDK may be externally acquired. The environment variable may then be explicitly set to where it has been saved to, for example if it is located in /opt/MacOSX10.10.sdk:

(t)csh
unix% setenv CONDA_BUILD_SYSROOT /opt/MacOSX10.10.sdk

bash/zsh
unix% export CONDA_BUILD_SYSROOT=/opt/MacOSX10.10.sdk

Compiling a user model for the standard CIAO binaries distribution

In this example, the relxill models package will be used. The file relxill_model_v2.3.tgz is assumed to have been downloaded and unpacked, for example:

unix% cd /data/models/relxill_v2
unix% tar -xzf relxill_model_v2.3.tgz

unix% ls -F
common.h            ModelParams.h   Relprofile.h             Relxill.cpp
compile_relxill.sh  README.txt      Relreturn_BlackBody.cpp  Relxill.h
config.h            Relbase.cpp     Relreturn_BlackBody.h    writeOutfiles.c
donthcomp.c         Relbase.h       Relreturn_Corona.cpp     writeOutfiles.h
IonGradient.cpp     Relcache.cpp    Relreturn_Corona.h       Xillspec.cpp
IonGradient.h       Relcache.h      relreturn_datastruct.c   Xillspec.h
LICENSE             Rellp.cpp       relreturn_datastruct.h   xilltable.c
lmodel_relxill.dat  Rellp.h         relreturn_table.c        xilltable.h
LocalModel.cpp      Relmodels.cpp   relreturn_table.h        XspecSpectrum.h
LocalModel.h        Relmodels.h     reltable.c               xspec_wrapper_lmodels.cpp
ModelDatabase.h     relphysics.c    reltable.h               xspec_wrapper_lmodels.h
ModelInfo.h         relphysics.h    relutility.c
ModelParams.cpp     Relprofile.cpp  relutility.h

Setting up the model

For version 2.x of the relxill model, in the directory with the unpacked source code, the line #include "fftw/fftw3.h" needs to be changed to #include "fftw3.h" since the fftw3 header file is in a different relative location when it is built for CIAO/Sherpa than where it is located for HEASoft/XSpec.

Linux (GNU):
unix% sed -i 's,#include "fftw/fftw3.h",#include "fftw3.h",' common.h
unix% sed -i 's,#include "fftw/fftw3.h",#include "fftw3.h",' Relbase.cpp

MacOS (BSD):
unix% sed -i '' -e 's,#include "fftw/fftw3.h",#include "fftw3.h",' common.h
unix% sed -i '' -e 's,#include "fftw/fftw3.h",#include "fftw3.h",' Relbase.cpp
[NOTE]
Note

Many models require no set up or code changes.


Compile the model

After any setup, the convert_xspec_user_model script is used to compile the model. For many cases you just need a name for the Python module containing the model, or models, and the name of the "model.dat" file. In this case we use 'relxill_v2' for the model name; this will be used to load in the model for testing later.

(t)csh
unix% setenv CC `which gcc`
unix% setenv CXX `which g++`

bash/zsh
unix% export CC=`which gcc`
unix% export CXX=`which g++`


unix% convert_xspec_user_model relxill_v2 lmodel_relxill.dat
convert_xspec_user_model: 21 April 2023
  name:       relxill_v2
  modelfile:  lmodel_relxill.dat
  clobber:    False

Found no Fortran files.
Found 8 C files.
Found 13 C++ files.
Processing 12 models.
Using language interface: C style
Using XSPEC version: 12.12.1c

..............................
... a lot of screen output ...
..............................

Successfully built relxill-v2
Installing collected packages: relxill-v2
Successfully installed relxill-v2-1.0

Testing out importing the model ...

Error: unable to import the module; possible reasons are:
  - incompatible version of gcc/gfortran used.
  - the model uses the udmget set of routines which are currently
    unsupported
  - missing a library or include directory

# convert_xspec_user_model (21 April 2023): ERROR Unable to import model:
${ASCDS_OTS}/lib/python3.9/site-packages/relxill_v2/_models.cpython-39-x86_64-linux-gnu.so: undefined symbol: fftw_plan_dft_c2r_1d

The initial attempt to build the model fails with an error associated with a fftw-related library. Examining the available fftw library files installed on the system with ldconfig (on Linux):

unix% ldconfig -p | grep fftw
	libfftw3_threads.so.3 (libc6,x86-64) => /lib64/libfftw3_threads.so.3
	libfftw3_omp.so.3 (libc6,x86-64) => /lib64/libfftw3_omp.so.3
	libfftw3.so.3 (libc6,x86-64) => /lib64/libfftw3.so.3

(t)csh
unix% setenv LDFLAGS "-l:libfftw3.so.3"

bash/zsh
unix% export LDFLAGS="-l:libfftw3.so.3"

*or*

(t)csh
unix% setenv LDFLAGS "-lfftw3"

bash/zsh
unix% export LDFLAGS="-lfftw3"

we then set a linker flag in the LDFLAGS environment variable for the compiler to explicitly look for the libfftw3.so.3 file (prepended by -l:) in the usual library directory locations (specific syntax use of the '-l' flag will be dependent on the compilers/OS used; the use of '-lfftw3' is more generalized and would not necessarily find the version specific .so.3 library).

[TIP]
Tip

In some Linux distributions, for example Ubuntu 18.04 LTS, it may be necessary to also specify the library directories too. For example, ldconfig points to the use /usr/lib/x86_64-linux-gnu and /usr/lib on the system:

unix% ldconfig -p | grep fftw
    libfftw3f_threads.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f_threads.so.3
    libfftw3f_omp.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f_omp.so.3
    libfftw3f.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3f.so.3
    libfftw3_threads.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3_threads.so.3
    libfftw3_omp.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3_omp.so.3
    libfftw3.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfftw3.so.3

bash/zsh
unix% export LDFLAGS="-l:libfftw3.so.3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"

(t)csh
unix% setenv LDFLAGS "-l:libfftw3.so.3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"

*or*

bash/zsh
unix% export LDFLAGS="-lfftw3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"

(t)csh
unix% setenv LDFLAGS "-lfftw3 -L/usr/lib/x86_64-linux-gnu -L/usr/lib"

where the -L linker flag is used to specify additional directories to look for library dependencies, in addition to the specified library using -l.

This example is done on a Red Hat Enterprise Linux 8.6 system with the C/C++ compilers from the GNU Compiler Collection (GCC) 8.5.0; however, for other operating systems and/or compilers, setting LDFLAGS/CFLAGS/CXXFLAGS/FFLAGS may be completely unnecessary. For this particular version of the relxill model and the compiler used, a lot of warnings are emitted, which can be suppressed by setting CFLAGS to use the set of flags:

  • -Wno-unused-value
  • -Wno-unused-variable
  • -Wno-array-bounds

for example,

(t)csh
unix% setenv CFLAGS '-Wno-unused-value -Wno-unused-variable -Wno-array-bounds'

bash/zsh
unix% export CFLAGS='-Wno-unused-value -Wno-unused-variable -Wno-array-bounds'

and re-run the script to build the model, also including --clobber flag:

unix% convert_xspec_user_model relxill_v2 lmodel_relxill.dat --clobber
convert_xspec_user_model: 21 April 2023
  name:       relxill_v2
  modelfile:  lmodel_relxill.dat
  clobber:    True

Found no Fortran files.
Found 8 C files.
Found 13 C++ files.
Processing 12 models.
Using language interface: C style
Using XSPEC version: 12.12.1c

...
... a lot of screen output 
...

Installing collected packages: relxill-v2
  Attempting uninstall: relxill-v2
    Found existing installation: relxill-v2 1.0
    Uninstalling relxill-v2-1.0:
      Successfully uninstalled relxill-v2-1.0
Successfully installed relxill-v2-1.0

Testing out importing the model ...
Import succeeded

------------------------------------------------------------------

Finished. You should be able to now say
    import relxill_v2.ui
from Sherpa.

The following 12 models are available:
  xsumrelline
  xsumrelconv
  xsumrelline_lp
  xsumrelconv_lp
  xsumrelxill
  xsumrelxilllp
  xsumxillver
  xsumxillvercp
  xsumrelxillcp
  xsumrelxilllpcp
  xsumxillverns
  xsumrelxillns

pip can then be used to confirm that the package is installed into CIAO for use with Sherpa.

unix% pip list | grep relxill
relxill-v2           1.0

unix% pip show relxill-v2
Name: relxill-v2
Version: 1.0
Summary: XSPEC user models in Sherpa
Home-page: 
Author: 
Author-email: 
License: 
Location: ${ASCDS_INSTALL}/ots/lib/python3.9/site-packages
Requires: 
Required-by: 
[TIP]
Setting up to compile on MacOS

In principle the compilers used to build XSpec models should be the same as that used to build the CIAO/Sherpa distribution installed on your system. The Clang/LLVM compilers that come packaged with XCode should be used to build C/C++ software for CIAO/Sherpa distributed through the binaries (e.g. via ciao-install) on MacOS and that on a typical system, gcc and g++ are linked to use clang and clang++ respectively by default. GNU C and C++ may sometimes be used successfully in lieu of Clang. Pre-built distributions of GNU Fortran compilers are also available for MacOS.

It is worth mentioning that as of relxill 2.3, the native Clang compiler has trouble building the model. There has been limited success building with the x86-64 GNU (gcc/g++) compilers into the CIAO binary distribution and buildable into the Conda CIAO distribution using the Conda compilers for MacOS.

An important step to using this collection of models is to make sure they have access to the tabulated data the models are dependent upon. The relxill documentation notes that the RELXILL_TABLE_MODEL environment variable should be used to define this location. Assuming the table files are downloaded and unpacked to /path/to/directory/with/relxill/table/files, then within a Sherpa or Python environment with the models imported, the os.environ Python object should be used to internally set this environment variable:

sherpa> import relxill_v2.ui
Adding additive       XSPEC local model: xsumrelline
Adding convolution    XSPEC local model: xsumrelconv
Adding additive       XSPEC local model: xsumrelline_lp
Adding convolution    XSPEC local model: xsumrelconv_lp
Adding additive       XSPEC local model: xsumrelxill
Adding additive       XSPEC local model: xsumrelxilllp
Adding additive       XSPEC local model: xsumxillver
Adding additive       XSPEC local model: xsumxillvercp
Adding additive       XSPEC local model: xsumrelxillcp
Adding additive       XSPEC local model: xsumrelxilllpcp
Adding additive       XSPEC local model: xsumxillverns
Adding additive       XSPEC local model: xsumrelxillns

sherpa> os.environ["RELXILL_TABLE_MODEL"] = "/path/to/directory/with/relxill/table/files"

to set the location for the models to look in. If you would like to avoid declaring the os.environ["RELXILL_TABLE_PATH"] whenever the relxill model package is initially imported, the path can be hardcoded into the common.h file before compiling the package, changing the line (line 117 as of v2.3):

#define RELXILL_TABLE_PATH "./"

to

#define RELXILL_TABLE_PATH "/path/to/directory/with/relxill/table/files"

Compiling a user model for the Conda CIAO distribution

Compiling software compatible with a Conda environment and other Conda packages should not be anymore involved than building with the system's default compilers for the CIAO binaries, but requires a different tack.

In this example, the RELTRANS models package will be used. The file reltrans_v1.0.1.tar.gz is assumed to have been downloaded and unpacked; for example:

unix% cd /data/models/reltrans_v1
unix% tar -xzf reltrans_v1.0.1.tar.gz
unix% cd reltrans_v1.0.1/

unix% ls -F 
amodules.f90       lmodel_relxill.dat  rellp.c      reltable.h     revmakefile_laptop
clean.xcm*         load.xcm            rellp.h      relutility.c   srevtrans.f90
common.h           README.txt          relmodels.c  relutility.h   subroutines/
compile_xspec.sh*  relbase.c           relmodels.h  revmakefile    xilltable.c
donthcomp.c        relbase.h           reltable.c   revmakefile_g  xilltable.h

Out of the box no additional file modifications are needed for the the RELTRANS v1 models. Please note that the minor modifications to the unpacked relxill source code, illustrated earlier in this thread, are applicable for a Conda build too.

Setting up the Conda environment for model compilation

Packages on a Conda distribution channel are built using a standardized set of compilers and build dependenices, also available on that given channel. The Conda distribution of CIAO are compiled using the Conda compiler packages:

Linux:
gcc, gxx, gfortran

MacOS:
clang, clangxx, gfortran

Since the Conda distribution of CIAO was built using the compilers from the conda-forge channel, we install them into the environment:

Linux:
unix% conda install -n ciao-4.15 -c conda-forge gcc gxx gfortran

MacOS:
unix% conda install -n ciao-4.15 -c conda-forge clang clangxx gfortran

where the Conda environment with CIAO installed is called 'ciao-4.15' in this example. The respective compiler environment variables are also set:

unix% conda activate ciao-4.15

Linux:
(t)csh
(ciao-4.15) unix% setenv CC ${CONDA_PREFIX}/bin/gcc
(ciao-4.15) unix% setenv CXX ${CONDA_PREFIX}/bin/g++
(ciao-4.15) unix% setenv FC ${CONDA_PREFIX}/bin/gfortran

bash/zsh
(ciao-4.15) unix% export CC=${CONDA_PREFIX}/bin/gcc
(ciao-4.15) unix% export CXX=${CONDA_PREFIX}/bin/g++
(ciao-4.15) unix% export FC=${CONDA_PREFIX}/bin/gfortran

MacOS:
(t)csh
(ciao-4.15) unix% setenv CC ${CONDA_PREFIX}/bin/clang
(ciao-4.15) unix% setenv CXX ${CONDA_PREFIX}/bin/clang++
(ciao-4.15) unix% setenv FC ${CONDA_PREFIX}/bin/gfortran

bash/zsh
(ciao-4.15) unix% export CC=${CONDA_PREFIX}/bin/clang
(ciao-4.15) unix% export CXX=${CONDA_PREFIX}/bin/clang++
(ciao-4.15) unix% export FC=${CONDA_PREFIX}/bin/gfortran

and the C compiler flag specifying the CIAO C header files might also be needed in this example:

(t)csh
(ciao-4.15) unix% setenv CFLAGS "-I${ASCDS_OTS}/include"

bash/zsh
(ciao-4.15) unix% export CFLAGS="-I${ASCDS_OTS}/include"

Note that depending on the model package, defining every compiler environment variable (e.g. CC, CXX, FC) may be unnecessary and even detrimental.


Compile the model

(ciao-4.15) unix% convert_xspec_user_model reltrans_v1 lmodel_relxill.dat --clobber
convert_xspec_user_model: 21 April 2023 
  name:       reltrans_v1
  modelfile:  lmodel_relxill.dat
  clobber:    True

Found 2 Fortran files.
Found 7 C files.
Found no C++ files.
Processing 15 models.
Using language interfaces: C style, Fortran - single precision
WARNING: reltrans needs to be re-calculated per spectrum; this is untested.
WARNING: reltransCp needs to be re-calculated per spectrum; this is untested.
Using XSPEC version: 12.12.1c

..............................
... a lot of screen output ...
..............................

Successfully built reltrans-v1
Installing collected packages: reltrans-v1
Successfully installed reltrans-v1-1.0

Testing out importing the model ...
Import succeeded

------------------------------------------------------------------

Finished. You should be able to now say
    import reltrans_v1.ui
from Sherpa.

The following 15 models are available:
  xsumrelline
  xsumrelconv
  xsumrelline_lp
  xsumrelconv_lp
  xsumrelxill
  xsumrelxilllp
  xsumxillver
  xsumrelxilld
  xsumrelxilllpd
  xsumxillverd
  xsumrelxillcp
  xsumrelxilllpcp
  xsumxillvercp
  xsumreltrans
  xsumreltranscp

Please note the following 2 problems:
  model reltrans needs to be re-calculated per spectrum; this is untested.
  model reltransCp needs to be re-calculated per spectrum; this is untested.

If the script gets this far—that is, the model can be imported and a message listing the available model, or models, is displayed—then the model has compiled and you can check it out in Sherpa. Otherwise, follow on to the errors section.


What to do if the build fails

There are a number of reasons why the build might fail; please contact the CXC Helpdesk if you are unable to solve the problem. The --clobber flag may be needed if convert_xspec_user_model is called multiple times, and sometimes directly deleting the src and build sub-directories generated by the script in the directory with the model source code may clear things up.

  1. In CIAO 4.8, Only additive and multiplicative models are supported. Convolution model support was added in CIAO contributed scripts package 4.9.2 and in CIAO 4.13, XSpec convolution models are directly supported by Sherpa.

  2. Support for models that require the -udmget or -udmget64 flag when calling XSpec's initpackage command was added in the CIAO contributed scripts package 4.15.2 using the --udmget or --udmget64, respectively.

  3. There can be problems if your version of gcc and gfortran is much different than the version used to build CIAO 4.15 (GCC version 7.4.0 for the binaries and 12.1.0 for Conda on Linux CentOS 7.9 and Clang 11.0.3/gfortran 11.2.0 for the binaries and Clang 14.0.4/gfortran 11.3.0 for Conda on MacOS 1.15). This can lead to link errors such as:

    undefined symbol: _gfortran_copy_string
    
  4. The script does not guarantee to compile files in the same order as XSpec does, so if an ordering is required, then the build may fail.

  5. If there are link or build issues, then the flags --no-xspec and --cfitsio may be useful: --no-xspec means that no XSpec libraries are linked to and --cfitsio can be added if the model only uses the CFITSIO routines.

    There may also be a version conflict between the XSpec version needed by the model and that provided by Sherpa, which uses version 12.12.1c in CIAO 4.15.

  6. If there is a problem when building the interface the resulting error message is unlikely to be useful. In this case try running

    unix% pip install . --verbose
    

    as it should provide more information.

    Possible problems are models that refer to XSPEC code from older (or newer) versions of XSPEC than used by CIAO, as well as access to certain parts of the XSPEC build system. Some of these can be worked around—such as removing the problematic models or tweaking how the extension module is built in setup.py.


Testing the model in Sherpa

Once the model has been compiled, it can be used from Sherpa. For the relxill model, we can try it without any data or response information using the following. First, load in the module and set the RELXILL_TABLE_PATH environment variable /path/to/directory/with/relxill/table/files (replace with the location of your model's tabulated data dependency):

sherpa> import relxill_v2.ui
Adding additive       XSPEC local model: xsumrelline
Adding convolution    XSPEC local model: xsumrelconv
Adding additive       XSPEC local model: xsumrelline_lp
Adding convolution    XSPEC local model: xsumrelconv_lp
Adding additive       XSPEC local model: xsumrelxill
Adding additive       XSPEC local model: xsumrelxilllp
Adding additive       XSPEC local model: xsumxillver
Adding additive       XSPEC local model: xsumxillvercp
Adding additive       XSPEC local model: xsumrelxillcp
Adding additive       XSPEC local model: xsumrelxilllpcp
Adding additive       XSPEC local model: xsumxillverns
Adding additive       XSPEC local model: xsumrelxillns
  
sherpa> os.environ["RELXILL_TABLE_PATH"] = "/path/to/directory/with/relxill/table/files"

Now we create a simple data space over which the model can be evaluated and the model expression (note that the user model used in the same manner as the built-in models). The default 'xsum' prefix of the model names is shorthand for "XSpec user model"; the "--prefix" flag can be used to alter the prefix, taking care to avoid using the lower case 'xs' since it is already used for the standard set of XSpec models in Sherpa and risk the possibility of overwriting the models if it has the same name. The xsumrelconv convolution model will be used in this example:

sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt)

sherpa> set_source(xsumrelconv.rel1(const1d.c0))

sherpa> show_source()
Model: 1
xsumrelconv.rel1(const1d.c0)
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   rel1.Index1  frozen            3          -10           10           
   rel1.Index2  frozen            3          -10           10           
   rel1.Rbr     frozen           15            1         1000           
   rel1.a       thawed        0.998       -0.998        0.998           
   rel1.Incl    frozen           30            3           87        deg
   rel1.Rin     frozen           -1         -100           -1           
   rel1.Rout    frozen          400            1         1000           
   rel1.limb    frozen            0            0            2           
   c0.c0        thawed            1 -3.40282e+38  3.40282e+38           
[NOTE]
Parameter names

The parameter names specified in the XSpec "model.dat" file are not always valid for Python, so a transformation is made, converting unusable characters—such as + and /—into the underscore character (_).

Two versions of the model are now plotted on the same graph; first for the default parameter values and then after changing the inclination angle, Incl, parameter, which creates Figure 1:

sherpa> plot_source()
 *** loading RELXILL model (version 2.3) ***

sherpa> rel1.Incl = 80
sherpa> plot_source(overplot=True)
sherpa> plt.loglog()

Figure 1: Comparison of two model plots

[comparison of the relconv model convolving a constant model component by altering the inclination angle.]
[Print media version: comparison of the relconv model convolving a constant model component by altering the inclination angle.]

Figure 1: Comparison of two model plots

The blue line shows the model using the default parameters, whereas the orange line shows the model after changing the Incl parameter to 80 degrees.

Since there is no response information here—as no RMF has been loaded—then the axes have no units.

[NOTE]
External files for configuration

Many models require, or allow, external configuration. The set_xsxset command is the equivalent of XSpec's xset command. As an example:

sherpa> set_xsxset('CARBATM', '/data/models/data/carbatm')

sets the XSpec CARBATM variable.

The equivalent location to XSpec's spectral/ directory is ${ASCDS_INSTALL}/ots/spectral/.

Other models will access data dependencies through a specified shell environment variable, which may be set in Sherpa using the os.environ Python object, as demonstrated earlier in this thread,

sherpa> os.environ["RELXILL_TABLE_MODEL"] = "/path/to/directory/with/relxill/table/files"

Use of set_xsxset versus os.environ will be completely model dependent, which should be informed based on the specific model's user-documentation.


Example: a model that successfully compiles to completion but fails at execution

The WARMABS Fortran model readily compiles on Conda without setting any specific compiler or linker flags, but is problematic when building it against the CIAO 4.15 Linux binary distribution on a RHEL 8 system and its native compilers (GCC 8.5.0).

In this example, (t)csh syntax is used to set environment variables—which can be easily substituted with the equivalent export call in bash/zsh—and the WARMABS model package is assumed to have been downloaded and unpacked,

unix% mkdir warmabs_v247
unix% tar -xzf warmabs247.tar.gz -C warmabs_v247

unix% ls -F warmabs_v247
atdbwarmabs.fits   PARAMX            xo01_detah4.fits  xo01_detal3.fits
coheatwarmabs.dat  README            xo01_detahl.fits  xo01_detal4.fits
fphotems.f90       xo01_detah2.fits  xo01_detail.fits
lmodel.dat         xo01_detah3.fits  xo01_detal2.fits

For organizational purposes, the data tables used by the models are separated from the source code and moved to a separate directory:

unix% ls -F /path/to/warmabs_v2.4x/data/files
atdbwarmabs.fits   xo01_detah3.fits  xo01_detail.fits  xo01_detal4.fits
coheatwarmabs.dat  xo01_detah4.fits  xo01_detal2.fits
xo01_detah2.fits   xo01_detahl.fits  xo01_detal3.fits

The model package will be built with the module name 'warmabs_v247' herein. It seemingly compiles successfully using convert_xspec_user_model with no build failure messages but an error is thrown upon using the model with Sherpa:

sherpa> import warmabs_v247.ui
Adding additive       XSPEC local model: xsumhotemis
Adding multiplicative XSPEC local model: xsumhotabs
Adding additive       XSPEC local model: xsumphotemis
Adding multiplicative XSPEC local model: xsumwarmabs
Adding multiplicative XSPEC local model: xsummultabs

sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files"

sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt)

sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1)

sherpa> show_model()
Model: 1
(xsumwarmabs.abs1 * xspowerlaw.p1)
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   abs1.column  thawed            0           -3            2      cm^-2
   abs1.rlogxi  thawed            0           -4            5           
   abs1.Cabund  frozen            1            0         1000           
   abs1.Nabund  frozen            1            0         1000           
   abs1.Oabund  frozen            1            0         1000           
   abs1.Fabund  frozen            1            0         1000           
   abs1.Neabund frozen            1            0         1000           
   abs1.Naabund frozen            1            0         1000           
   abs1.Mgabund frozen            1            0         1000           
   abs1.Alabund frozen            1            0         1000           
   abs1.Siabund frozen            1            0         1000           
   abs1.Pabund  frozen            1            0         1000           
   abs1.Sabund  frozen            1            0         1000           
   abs1.Clabund frozen            1            0         1000           
   abs1.Arabund frozen            1            0         1000           
   abs1.Kabund  frozen            1            0         1000           
   abs1.Caabund frozen            1            0         1000           
   abs1.Scabund frozen            1            0         1000           
   abs1.Tiabund frozen            1            0         1000           
   abs1.Vabund  frozen            1            0         1000           
   abs1.Crabund frozen            1            0         1000           
   abs1.Mnabund frozen            1            0         1000           
   abs1.Feabund frozen            1            0         1000           
   abs1.Coabund frozen            1            0         1000           
   abs1.Niabund frozen            1            0         1000           
   abs1.Cuabund frozen            1            0         1000           
   abs1.Znabund frozen            1            0         1000           
   abs1.write_outfile frozen            0            0            1           
   abs1.outfile_idx frozen            0            0        10000           
   abs1.vturb   frozen            0            0        10000       km/s
   abs1.Redshift frozen            0            0           10           
   p1.PhoIndex  thawed            1           -3           10           
   p1.norm      thawed            1            0        1e+24           

sherpa> plot_source()
 doing setup:           0           1           0           0           0
 photemis/warmabs v2.47d
Fortran runtime error: Insufficient number of elements in TARRAY.

Taking a quick look at the library dependencies of the built module with ldd

unix% ldd build/lib.linux-x86_64-cpython-39/warmabs_v247/_models.cpython-39-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffdc97db000)
	libXSModel.so => not found
	libXSFunctions.so => not found
	libXSUtil.so => not found
	libXS.so => not found
	libhdsp_6.30.so => not found
	libCCfits_2.6.so => not found
	libcfitsio.so.9 => not found
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fca97e26000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fca97aa4000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fca9788c000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fca974c7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcad9d7e000)

which is found in the temporary 'build' sub-directory that convert_xspec_user_model generates in the unpacked source code directory.

All of the "not found" libraries are CIAO OTS libraries (which upon further inspection with the 'ldd -u' flag shows they are ultimately unused by the model). In this case, the linker flag is set before attempting to do another re-build of the package:

sherpa> setenv LDFLAGS -Wl,-rpath,${ASCDS_OTS}/lib

sherpa> convert_xspec_user_model warmabs_v247 lmodel.dat --clobber

..............................
... a lot of screen output ...
..............................



unix% ldd build/lib.linux-x86_64-cpython-39/warmabs_v247/_models.cpython-39-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007ffd5ccd2000)
	libXSModel.so => ${ASCDS_OTS}/lib/libXSModel.so (0x00007f05fb703000)
	libXSFunctions.so => ${ASCDS_OTS}/lib/libXSFunctions.so (0x00007f05f9d5f000)
	libXSUtil.so => ${ASCDS_OTS}/lib/libXSUtil.so (0x00007f05f98eb000)
	libXS.so => ${ASCDS_OTS}/lib/libXS.so (0x00007f05f96cc000)
	libhdsp_6.30.so => ${ASCDS_OTS}/lib/libhdsp_6.30.so (0x00007f05f92b4000)
	libCCfits_2.6.so => ${ASCDS_OTS}/lib/libCCfits_2.6.so (0x00007f05f8e47000)
	libcfitsio.so.9 => ${ASCDS_OTS}/lib/libcfitsio.so.9 (0x00007f05f8943000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f05f85ae000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f05f822c000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f05f8014000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f05f7c4f000)
	libtinfo.so.5 => ${ASCDS_OTS}/lib/../lib/libtinfo.so.5 (0x00007f05f7a1c000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f05f7818000)
	libgfortran.so.4 => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4 (0x00007f05f72fe000)
	libquadmath.so.0 => ${ASCDS_OTS}/lib/../lib/libquadmath.so.0 (0x00007f05f70b7000)
	libwcs-7.7.so => ${ASCDS_OTS}/lib/../lib/libwcs-7.7.so (0x00007f05f6d24000)
	libhdutils_6.30.so => ${ASCDS_OTS}/lib/../lib/libhdutils_6.30.so (0x00007f05f6a1f000)
	libfgsl.so.1 => ${ASCDS_OTS}/lib/../lib/libfgsl.so.1 (0x00007f05f676c000)
	libgsl.so.25 => ${ASCDS_OTS}/lib/../lib/libgsl.so.25 (0x00007f05f61f9000)
	libgslcblas.so.0 => ${ASCDS_OTS}/lib/../lib/libgslcblas.so.0 (0x00007f05f5fa6000)
	libreadline.so.8 => ${ASCDS_OTS}/lib/../lib/libreadline.so.8 (0x00007f05f5d4e000)
	libz.so.1 => ${ASCDS_OTS}/lib/../lib/libz.so.1 (0x00007f05f5b30000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f063da90000)
	libncurses.so.5 => ${ASCDS_OTS}/lib/../lib/../lib/libncurses.so.5 (0x00007f05f5902000)

unix% ldd -u build/lib.linux-x86_64-cpython-39/warmabs_delme/_models.cpython-39-x86_64-linux-gnu.so
Unused direct dependencies:
	${ASCDS_OTS}/lib/libXSModel.so
	${ASCDS_OTS}/lib/libXSFunctions.so
	${ASCDS_OTS}/lib/libXSUtil.so
	${ASCDS_OTS}/lib/libXS.so
	${ASCDS_OTS}/lib/libhdsp_6.30.so
	${ASCDS_OTS}/lib/libCCfits_2.6.so
	${ASCDS_OTS}/lib/libcfitsio.so.9
	/lib64/libm.so.6
	/lib64/libgcc_s.so.1
	/lib64/libpthread.so.0

unix% ldd -v build/lib.linux-x86_64-cpython-39/warmabs_delme/_models.cpython-39-x86_64-linux-gnu.so | grep fortran
	libgfortran.so.4 => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4 (0x00007fd10738b000)
		libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4
		libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4
	${ASCDS_OTS}/lib/../lib/libgfortran.so.4:
		libgfortran.so.4 (GFORTRAN_7) => ${ASCDS_OTS}/lib/../lib/libgfortran.so.4

Since the CIAO 4.15 Linux binaries were built using GCC 7.9 and the ligbfortran.so.4 library, there are incompatibilities with the version of GCC 8.x that comes with the OS, which does not support compiler backwards compatiblity. Using ldconfig to find the libgfortran that comes with the operating system,

unix% ldconfig -p |grep gfortran
	libgfortran.so.5 (libc6,x86-64) => /lib64/libgfortran.so.5

the linker flag is then modified to use the libgfortran.so.5 version of the library that is compatible with the compiler, and then attempt another build of the models:

unix% setenv LDFLAGS "-l:libgfortran.so.5 -Wl,-rpath,/lib64 -Wl,-rpath,${ASCDS_OTS}/lib"

sherpa> convert_xspec_user_model warmabs_v247 lmodel.dat --clobber

..............................
... a lot of screen output ...
..............................

Installing collected packages: warmabs-v247
Successfully installed warmabs-v247-1.0

Testing out importing the model ...
Import succeeded

------------------------------------------------------------------

Finished. You should be able to now say
    import warmabs_v247.ui
from Sherpa.

The following 5 models are available:
  xsumhotemis
  xsumhotabs
  xsumphotemis
  xsumwarmabs
  xsummultabs

and once again, the model seems to successfully compile. Trying to use the model, strangely, it will seemingly run, and sometimes will run to completion with the model displayed,

sherpa> import warmabs_v247.ui
Adding additive       XSPEC local model: xsumhotemis
Adding multiplicative XSPEC local model: xsumhotabs
Adding additive       XSPEC local model: xsumphotemis
Adding multiplicative XSPEC local model: xsumwarmabs
Adding multiplicative XSPEC local model: xsummultabs

sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files"

sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt)

sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1)

sherpa> plot_source()
 doing setup:           0           1           0           0           0
 photemis/warmabs v2.47d
 xstar version 2.58e
 Loading Atomic Database...
  Loading Atomic Database...                                                                                             
 Atomic Data Version: 2021-08-03T18:36:21
 number of pointers=     1216828
 number of reals=      199199716
 number of integers=     6205342
 number of chars=         753844
 initializng database...
  initializng database...                                                                                                
 number of lines=          736256
 number of rrcs=           301301
 max levels per ion:         744
habund      0.1000E+01  0.1000E+01
heabund     0.1000E+01  0.1000E+00
liabund     0.0000E+00  0.0000E+00
beabund     0.0000E+00  0.0000E+00
babund      0.0000E+00  0.0000E+00
cabund      0.1000E+01  0.3700E-03
nabund      0.1000E+01  0.1100E-03
oabund      0.1000E+01  0.6800E-03
fabund      0.1000E+01  0.3980E-07
neabund     0.1000E+01  0.2800E-04
naabund     0.1000E+01  0.1780E-05
mgabund     0.1000E+01  0.3500E-04
alabund     0.1000E+01  0.2450E-05
siabund     0.1000E+01  0.3500E-04
pabund      0.1000E+01  0.3310E-06
sabund      0.1000E+01  0.1600E-04
clabund     0.1000E+01  0.3980E-06
arabund     0.1000E+01  0.4500E-05
kabund      0.1000E+01  0.8910E-07
caabund     0.1000E+01  0.2100E-05
scabund     0.1000E+01  0.1660E-08
tiabund     0.1000E+01  0.1350E-06
vabund      0.1000E+01  0.2510E-07
crabund     0.1000E+01  0.7080E-06
mnabund     0.1000E+01  0.2510E-06
feabund     0.1000E+01  0.2500E-04
coabund     0.1000E+01  0.1260E-06
niabund     0.1000E+01  0.2000E-05
cuabund     0.1000E+01  0.3160E-07
znabund     0.1000E+01  0.1580E-07
  
 Reading pop files
 using files xout_detail.fits etc.         219
 constructing energy grid        9999
 constructing energy grid        7571        6377
${ASCDS_INSTALL}/binexe/sherpa: line 187: 2822848 Segmentation fault      (core dumped) ipython --profile sherpa

but the model execution is inconsistent, and more often than not, a segmentation fault is thrown with a core dump.

Exploring the various GFortran compiler flags for code generation,

unix% setenv FFLAGS -fcheck=all # insert runtime error checking

...

At line 9599 of file src/_xspeclmodels/fphotems.f90
Fortran runtime error: Index '-1' of dimension 1 of array 'nlbin' below lower bound of 1

setting -fmax-stack-var-size=10 (or using the -fno-automatic flag) to avoid a stack overflow can work around the error by avoiding to write variables/arrays to stack memory (at the expense of a minor performance penalty for the flexibility offered by writing to heap memory).

unix% setenv FFLAGS -fmax-stack-var-size=10

unix%  convert_xspec_user_model warmabs_v247 lmodel.dat
convert_xspec_user_model: 21 April 2023
  name:       warmabs_v247
  modelfile:  lmodel.dat
  clobber:    False

Found one Fortran file.
Found no C files.
Found no C++ files.
Processing 5 models.
Using language interface: Fortran - single precision
Using XSPEC version: 12.12.1c

...
... a lot of screen output 
...

Successfully built warmabs-v247
Installing collected packages: warmabs-v247
Successfully installed warmabs-v247-1.0

Testing out importing the model ...
Import succeeded

------------------------------------------------------------------

Finished. You should be able to now say
    import warmabs_v247.ui
from Sherpa.

The following 5 models are available:
  xsumhotemis
  xsumhotabs
  xsumphotemis
  xsumwarmabs
  xsummultabs

Testing out the multiplcative xsumwarmabs absorption model in Sherpa with a constant model component of unity, we see that the model plots without an error being thrown:

sherpa> import warmabs_v247.ui
Adding additive       XSPEC local model: xsumhotemis
Adding multiplicative XSPEC local model: xsumhotabs
Adding additive       XSPEC local model: xsumphotemis
Adding multiplicative XSPEC local model: xsumwarmabs
Adding multiplicative XSPEC local model: xsummultabs

sherpa> os.environ["WARMABS_DATA"] = "/path/to/warmabs_v2.4x/data/files"

sherpa> dataspace1d(0.1, 10, step=0.01, id=1, dstype=Data1DInt)

sherpa> set_source(xsumwarmabs.abs1 * xspowerlaw.p1)

sherpa> p1.PhoIndex = 0

sherpa> show_source()
Model: 1
(xsumwarmabs.abs1 * xspowerlaw.p1)
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   abs1.column  thawed            0           -3            2      cm^-2
   abs1.rlogxi  thawed            0           -4            5           
   abs1.Cabund  frozen            1            0         1000           
   abs1.Nabund  frozen            1            0         1000           
   abs1.Oabund  frozen            1            0         1000           
   abs1.Fabund  frozen            1            0         1000           
   abs1.Neabund frozen            1            0         1000           
   abs1.Naabund frozen            1            0         1000           
   abs1.Mgabund frozen            1            0         1000           
   abs1.Alabund frozen            1            0         1000           
   abs1.Siabund frozen            1            0         1000           
   abs1.Pabund  frozen            1            0         1000           
   abs1.Sabund  frozen            1            0         1000           
   abs1.Clabund frozen            1            0         1000           
   abs1.Arabund frozen            1            0         1000           
   abs1.Kabund  frozen            1            0         1000           
   abs1.Caabund frozen            1            0         1000           
   abs1.Scabund frozen            1            0         1000           
   abs1.Tiabund frozen            1            0         1000           
   abs1.Vabund  frozen            1            0         1000           
   abs1.Crabund frozen            1            0         1000           
   abs1.Mnabund frozen            1            0         1000           
   abs1.Feabund frozen            1            0         1000           
   abs1.Coabund frozen            1            0         1000           
   abs1.Niabund frozen            1            0         1000           
   abs1.Cuabund frozen            1            0         1000           
   abs1.Znabund frozen            1            0         1000           
   abs1.write_outfile frozen            0            0            1           
   abs1.outfile_idx frozen            0            0        10000           
   abs1.vturb   frozen            0            0        10000       km/s
   abs1.Redshift frozen            0            0           10           
   p1.PhoIndex  thawed            0           -3           10           
   p1.norm      thawed            1            0        1e+24           

   
sherpa> plot_source()
 doing setup:           0           1           0           0           0
 photemis/warmabs v2.47d
 xstar version 2.58e
 Loading Atomic Database...
  Loading Atomic Database...                                                                                             
 Atomic Data Version: 2021-08-03T18:36:21
 number of pointers=     1216828
 number of reals=      199199716
 number of integers=     6205342
 number of chars=         753844
 initializng database...
  initializng database...                                                                                                
 number of lines=          736256
 number of rrcs=           301301
 max levels per ion:         744
habund      0.1000E+01  0.1000E+01
heabund     0.1000E+01  0.1000E+00
liabund     0.0000E+00  0.0000E+00
beabund     0.0000E+00  0.0000E+00
babund      0.0000E+00  0.0000E+00
cabund      0.1000E+01  0.3700E-03
nabund      0.1000E+01  0.1100E-03
oabund      0.1000E+01  0.6800E-03
fabund      0.1000E+01  0.3980E-07
neabund     0.1000E+01  0.2800E-04
naabund     0.1000E+01  0.1780E-05
mgabund     0.1000E+01  0.3500E-04
alabund     0.1000E+01  0.2450E-05
siabund     0.1000E+01  0.3500E-04
pabund      0.1000E+01  0.3310E-06
sabund      0.1000E+01  0.1600E-04
clabund     0.1000E+01  0.3980E-06
arabund     0.1000E+01  0.4500E-05
kabund      0.1000E+01  0.8910E-07
caabund     0.1000E+01  0.2100E-05
scabund     0.1000E+01  0.1660E-08
tiabund     0.1000E+01  0.1350E-06
vabund      0.1000E+01  0.2510E-07
crabund     0.1000E+01  0.7080E-06
mnabund     0.1000E+01  0.2510E-06
feabund     0.1000E+01  0.2500E-04
coabund     0.1000E+01  0.1260E-06
niabund     0.1000E+01  0.2000E-05
cuabund     0.1000E+01  0.3160E-07
znabund     0.1000E+01  0.1580E-07
  
 Reading pop files
 using files xout_detail.fits etc.         219
 constructing energy grid        9999
 constructing energy grid        7571        6377

Figure 2: warmabs curve

[plot of the warmabs absorption model with default settings]
[Print media version: plot of the warmabs absorption model with default settings]

Figure 2: warmabs curve

Plot of the [xsum]warmabs warm absorber model, expressed by multiplying the absorption component with a constant model component expressed as a powerlaw with photon index of 0.


History

30 Sep 2014 original version
15 Dec 2014 Updated for CIAO 4.7
29 Feb 2016 Updated for CIAO 4.8 and contributed scripts 4.8.2
25 Jan 2017 Updated for CIAO 4.9; convolution models are still not supported.
13 Jul 2018 Updated for CIAO 4.10; convolution models supported again.
22 Aug 2023 Rewritten for CIAO 4.15; examples using binaries and Conda distribution of CIAO on Linux and MacOS platforms with different user models.