Using Sherpa in a notebook

Sherpa can be used with notebooks, whether the classic notebook interface or the newer lab environment. The What is Jupyter? page provides a lot of information on why people may want to use "computional notebooks". There are, however, some adjustments that need to be made when using the notebook or lab interfaces:

• the environment is not the same as provided by the sherpa tool;

• the display output may be different;

• and the Python environment used by the lab or notebook can get out of sync with that used by CIAO, in particular if the Jupyter installation does not match the CIAO environment.

The sections of this document are:

• Starting the notebook environment
• Loading Sherpa
• A quick tour
• Potential issues
  • Extra output
  • Image display

Starting the notebook environment

The lab or notebook environments can be used. For example, after ensuring that the Jupyter environment is installed:

unix% jupyter lab
...
[I 2025-05-01 15:41:55.575 ServerApp] Jupyter Server 2.14.2 is running at:
[I 2025-05-01 15:41:55.575 ServerApp] http://localhost:8888/lab?token=big-long-hexadecimal-string
[I 2025-05-01 15:41:55.575 ServerApp]     http://127.0.0.1:8888/lab?token=big-long-hexadecimal-string
[I 2025-05-01 15:41:55.575 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2025-05-01 15:41:55.671 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///...a path
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=big-long-hexadecimal-string
        http://127.0.0.1:8888/lab?token=big-long-hexadecimal-string
...

The screen output should include the web page to navigate to, if it hasn't already been selected. The locations used may well differ to those listed in the example above.

Loading Sherpa

As well as Sherpa, it is often useful to ensure that NumPy and Matplotlib are loaded:

import numpy as np
from matplotlib import pyplot as plt  

from sherpa.astro import ui

Note that the versions used in CIAO 4.17 are not necessarily the latest (in particular NumPy has not been updated to version 2):

from importlib import metadata

metadata.version("numpy"), metadata.version("matplotlib")

('1.26.4', '3.9.1')

Note

This document imports Sherpa as ui rather than using "from sherpa.astro.ui import *" as the Sherpa command-line tool does. This will make some differences below - in particular when creating a model.

An advantage to using the explicit name is that it makes it easier to use tab-completion - e.g. after entering ui.plot tab completion will only look for Sherpa commands that start with plot.

A quick tour

Using Sherpa in the notebook environment is, for the most part, the same as when using it from the command line, but there are a few additions.

ui.load_arrays(1, [10, 20, 40], [12, 13, 47])

ui.plot_data()

The first difference to the command-line version is that a number of Sherpa objects can take advantage of the "rich text formatting" support to automatically create "enhanced" output. In this case, the data object will automatically be displayed as an image:

ui.get_data()

The return value can be stored as a variable and then accessed directly (such as printing its contents):

d = ui.get_data()
print(d)  

name      = 
x         = Int64[3]
y         = Int64[3]
staterror = None
syserror  = None

When creating model components, the model name must be preceeded by ui. (this is not needed if Sherpa was loaded with from sherpa.astro.ui import *):

ui.set_model(ui.polynom1d.mdl)

print(mdl)

polynom1d.mdl
   Param        Type          Value          Min          Max      Units
   -----        ----          -----          ---          ---      -----
   mdl.c0       thawed            1 -3.40282e+38  3.40282e+38           
   mdl.c1       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c2       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c3       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c4       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c5       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c6       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c7       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.c8       frozen            0 -3.40282e+38  3.40282e+38           
   mdl.offset   frozen            0 -3.40282e+38  3.40282e+38           

Models are also displayed differently, as shown below (unfortunately the display does not let users changed the thawed or frozen state of each parameter):

ui.get_source()

Sherpa behaves as normally - that is the model can be tweaked, the statistic changed, and the data fit:

ui.thaw(mdl.c1)

ui.set_stat("leastq")

ui.fit()

Dataset               = 1
Method                = levmar
Statistic             = leastsq
Initial fit statistic = 2721
Final fit statistic   = 0.285714 at function evaluation 22
Data points           = 3
Degrees of freedom    = 1
Change in statistic   = 2720.71
   mdl.c0         -5.33706e-12 +/- 1.06297     
   mdl.c1         1.17143      +/- 0.0223607

ui.get_fit_results()

ui.plot_fit(yerrorbars=False)

WARNING: The displayed errorbars have been supplied with the data or calculated using chi2xspecvar; the errors are not used in fits with leastsq

Potential issues

Extra output

XSpec models can produce screen output depending on the chatter setting and whether the model has been evaluated before. This output can be displayed both in the notebook and in the terminal running the Jupyter environment. In the example below a dataset is created just so that it can be used to evaluate a model (xsapec) which is known to create screen output the first time it is evaluated (in this case when creating the model plot).

egrid = np.arange(0.1, 10, 0.1)
elo = egrid[:-1]
ehi = egrid[1:]

ui.load_arrays("x", elo, ehi, np.zeros(len(elo)), ui.Data1DInt)

ui.set_source("x", ui.xsapec.xmdl)

ui.get_source("x")

oldchatter = ui.get_xschatter()
ui.set_xschatter(20)

ui.plot_model("x", xlog=True, ylog=True)

Reading APEC data from 3.0.9

Thermal broadening : F
Velocity broadening : 0

For continuum interpolating between 0.966887 and 1.02418 for 1
with coefficients 0.422019 and 0.577981

For lines interpolating between 0.966887 and 1.02418 for 1
with coefficients 0.422019 and 0.577981

ui.set_xschatter(oldchatter)

Image display

Sherpa can fit 2D data within a notebook, but the image display routines - such as image_data and image_fit - rely on SAOImageDS9, which is not integrated into the lab environment (unless you are using the HEASARC SciServer environment). This may mean that calls to image_data will either not succeed, or use an SAOImageDS9 instance that does not appear with the notebook.