Monitoring and alerts.
ACIS Ops Response to SCS107.
When to call a telecon.
How to call a telecon.
The ACIS Ops team learned in the first weeks of theChandra mission that the
camera's front illuminated (FI) chips are highly vulnerable to damage
from low energy protons at roughly 100 keV. Such protons cause
displacement damage in the Si lattice producing charge traps which
significantly increase the CTI. A single belt passage increased
the FWHM by 20 eV in FI chips at -120C. As detailed in
MIT's report, the expected damage from belt passages
varies markedly as Chandra's orbit changes. 1999-2001, and 2010-2015
are the worst periods.
Since then, the ACIS Ops team has continuously monitored space weather
in an attempt to gauge Chandra's present and near future radiation
environment. Radiation levels of concern can arise from two sources:
Warning of a high radiation environment may come from several sources,
most of them handily gathered at cxc's
The following summary moves from data with the longest to the shortest
Trapped radiation belts.
The ACIS instrument must be protected through each perigee passage
through the radiation belts:
- The SIM must be translated to the HRC-S position while the
Chandra Radiation Model (CRM) predicts proton fluxes above
threshold. The CRM determines the radiation belt entry and exit
times, which often but not always align with the EEF1000 and
XEF1000 orbital events, as determined by NSDDC's AE-8 electron
model. Translations to and from HRC-S occur at pad times before
entry and after exit. CTI measurements are taken during these pad
- The "backstop pad time" runs from Radmon disable to EEF 1000,
and from XEF 1000 to Radmon enable. The "CRM pad time" runs from
the time of soft proton entry of the belts according to the CRM
model (lengthened up to a maximum of 10 ks if ACIS Ops consents) to
the AE8 predicted time of electron radiation belt entry.
- At all points in the orbit, the radiation monitor must be enabled
whenever ACIS is in the focal plane. During radiation belt
crossings, the radiation monitor should be disabled once the SIM
translation to HRC-S is complete.
- Video boards must be powered down between CRM pad times and
perigee. There should be a minimum of five hours between powerdown
This requirement is enforced by diligent review of perigee passages in
weekly command loads.
- High solar background.
In order to limit increase in the full width half max at the top of an
FI chip to an annual
0.1%, the ACIS operations team has established a radiation exposure budget for
for ACIS: an annual accumulated
fluence of 2e^10 protons/cm2-s-ster-MeV.
As a rule of thumb, expecting 8 to 10 major events a year, we allow
ourselves to accumulate a fluence of about 2e+9 for any one
Our best long look-ahead comes from tracking sunspots
on the near and far faces of the Sun. Spots take about 26 days to
make an (apparent, from the Earth's viewpoint) rotation across the
Sun; a couple days less at the equator and more near the poles.
Solar X-rays and CMEs
In the event of an X class solar flare, NOAA's Space Weather Prediction
Center will send an email alert to acisdude with
the subject line "SUMMARY: X-Ray Event exceeded X1". That's our cue to
jump to Today's Space Weather,
which displays the last three days of solar X-ray flux in a logarithmic
graph. Each horizontal line represents a factor of 10 increase in
energy. Flares below M-7 or so are unlikely to give ACIS any
grief. If they're above that level, we need to start keeping an eye on
ACE and GOES rates. The most energetic particles may arrive in 0.5 to
3 days, depending in part on where the flare was directed.
In the past, Shanil received email notices of Coronal Mass Ejections (CMEs)
from email@example.com, which he forwarded to the rest
of us. (TBD: what is the current mechanism for learning as rapidly as
possible whether there has been a CME and what its angle was with
Earth's line of sight?) The
velocity of CMEs is variable, but if one is
earth directed, it typically takes two to five days for the ejected
mass to arrive, and several hours for it to sweep past earth.
The warnings most indicative of current Chandra conditions are the
alerts from MTA based on ACE fluence monitoring. The ACE
instrument orbits at Earth-Sun L1, so that it experiences solar winds
and storms about an hour in advance of the Earth. The ACE page provides tables
and charts in realtime of electron
and proton fluxes at a variety of energies. We are interested in the
P3 proton channel (115 - 195 keV)>. So as not to rely on a single
detector, the two derived P3 numbers, conservatively scaled from the P5 and P6
channels can also trigger messages. Alerts are sent when:
- Three sot_red_alerts, spaced half an hour apart, when fluence for
the orbit exceeds 1.0e^9. Additional alerts are
sent on each tripling of the fluence.
- The fluence integrated over two-hours exceeds 3.6e^8. This more
urgent condition will send pager alerts in addition to the
The GOES satellites are in geosynchronous orbits about 38K
kilometers up, with GOES-15 and GOES-12 over the eastern and
GOES-14 over the
western hemisphere. They indicate near-earth proton levels, bearing
in mind that Chandra's inclination exposes us to a different cross
section of the magnetosphere.
GOES-14 replaced GOES-10 on December 1,
2009. Unlike its predecessor, GOES-14 will have no proton detection
At the end of February 2011, GOES-11 will be decommissioned, and
GOES-15 will become the SWPC Secondary GOES satellite for particle
measurements. Several of the high energy proton channels will
disappear for a while. In September 2011, a new GOES satellite, to be
located at 135
degrees longitude, is planned to begin transmitting data for all the
MTA sends a yellow alert when GOES-11 P2 > 30 and P5 > 0.25; and
red alerts for P2 > 90.9 and P5 > 0.70. The red limits are designed to
indicate a probable trip of the old P4GM (P2) and P41GM (P5) EPHIN
channels. (NB: GOES definitions of P2 and P5 channels differ from the
ACE definitions of P3, etc.)
MTA will send notification through sot_yellow_alert when Earth's
near-term Kp index as projected by the
Costello model is high.
Kp is a measure of the magnetic
field at the surface of the Earth, averaged from data at various
ground stations around the globe. Increases in the Kp indicates that
the particle density in the radiation belts is changing.
EPHIN In 2008, EPHIN's detector A, because of a high
proportion of dead times during which the detector was saturated, the
failure mode A was set to "ON".
Since then, EPHIN no
longer has a channel devoted exclusively to protons. (It was always
unable to detect protons at the low energies which are most
problematic.) EPHIN now reports just two channels: E150 and E1300, and
limits are scaled to account for the fact that proton counts are
also included. 174,216.03 E150 counts/s/cm2/sr will trigger an SCS-107.
The E1300 limit, still specific to electrons, stands at at 19.296079 counts/s/cm2/sr. (These limits account for the update to the geometric
factor in December 2009.)
HRC As EPHIN heats up and grows less reliable, we will
probably begin to rely on the High Resolution Camera's anticoincidence
shield as our on-board radiation detector. This has severe drawbacks:
the HRC can detect only high energy protons, and it saturates before
it reaches levels corresponding to current EPHIN red limits. The prospects were
discussed in the 2005 Flight
Note 443 (pdf), with a breakdown of results of adopting the
highest possible threshold value in a 2006 memo
from Mike Juda.
SCS 107 notification
When an SCS-107 is commanded, or EPHIN executes one spontaneously and
the fact becomes apparent on the next contact, MTA sends an email to
sot_red_alert indicating the time at which the SCS107 DISA status was
noted in realtime telemetry. (This will not ordinarily be the same as
the time at which the SCS-107 occurred. The OC will usually promulgate
that actual time before the contact finishes.)
As a cross-check that the SCS-107 is real, our Realtime page will show
DPA-A and DPA-B currents of about 0.43 and 0.28 respectively, and the
SIM at HRC-S.
Events that had been scheduled in the daily load before the SCS-107
will continue to show up on the Replan Central page,
but greyed out.
Confirm that ACIS is safe via our realtime page:
If any of these checks fail, be diligent and
make sure that SOP_UNSAFE_ACIS PHASE 1 is executed.
SIM is at the HRC-S position -99616
DPA input currents indicate that the FEPs have been powered down:
1DPICACU < 0.4 A, 1DPICBCU < 0.3 A
DEA time-averaged power is less than 50 W
Status bits indicate that a science run is not active, the last boot
was a normal boot and the flight SW is still running:
1STAT1ST = 1(green) science not active
1STAT2ST = 1(green) normal boot
1STAT0ST = 0 or 1 toggles every 64s to indicate SW is running
Check the Fluence monitors to determine how much ACIS has detected
already. These are based on the ACE Flux.
with the SCS 107 time. (The
webpage instructions for radiation replan reviews are less
detailed but more up-to-date.)
Be ready for the replan. Review the CTI RTS and be ready to write one
up if there is enough time.
In the event of an SCS-107, ACIS is safe, and the Engineering Manager
or the Flight Director will call a telecon.
An ACE Alert has come through and P3 channel rates have been
rising. The default is to call a telecon if there's been an ACE
alert. In case the alert is at the 3.6e^8 threshold, do the following
- Determine exposureTimeToRadzone (i.e., to Radmon Disable). (If we're already
in the radzone, this is the RadmonDisable for the next orbit.)
Subtract from these times any period spent at HRC-S or HRC-I, and
divide by 2 or 5 respectively when the LETG or HETG is in place.
- Determine exposureTimeToContact, and
exposureTimeTo2ndContact (i.e., the next
commanding opportunity, and the one following that.) Make the same
adjustments for SIM position and gratings as in (1); namely, subtract
from these times any period spent at HRC-S or HRC-I, and divide by 2
or 5 respectively when the LETG or HETG is in place.
- Is the ACE flux clearly falling?
If so, set "CalcFlux" to the
current reading. Use the real P3 channel.
Otherwise, extrapolate the flux to the next UTimeToRadzone or
UTimeToContact (whichever is first) and set "CalcFlux" to that value.
RadZFlu = current orbital fluence + CalcFlux * timeToRadzone
ContFlu = current orbital fluence + CalcFlux * timeToContact
NxtContFlu = current orbital fluence + CalcFlux *
[We really should have a script to pop out all these numbers. I'll
volunteer to write it, once we've agreed on policy and scaling parameters.]
If there's been an alert, the default is to call a telecon via
sot_red_alert before the next contact. However, if the alert comes
outside of reasonable ( 7pm to 7am) hours and
then, via email to sot_yellow_alert, call for the telecon the next
morning, between 7 am and the first contact opportunity.
- ContFlu < 1.0e^9 and the first contact occurs after 7 am or
- NxtContFlu < 1.0e^9 and the following contact occurs after 7 am
Finally, if ACE flux is clearly falling, and both RadZFlu and NxtContFlu are
below 1.0e^9, you may defer a decision on calling a telecon and continue to
Send an email to sot_red_alert. Radiation telecons will ordinarily be
on the contingency number 877-521-0441 (111165#)
Required personnel to make a decision on whether to shut down are a
flight director, and representatives from FOT engineering and FOT
MP. Desirable are people from PCAD (either Eric or Tom), SOT MP, SOT
lead, and Steve O'Dell from NASA.
SCS-107 EXCUTION AND NON-LOAD TRACKING
If an SCS-107 occured - either manually or automatically - you want
to record that Non-Load
Event. When execution of the SCS-107 begins, be sure to note the time.
Once the SCS-107 execution is complete, you need to add this event to
the "Non-Load Tracking File". This is a file that is used by
various programs to incorporate the effects of of the shutdown
into the Thermal Models.
The instructions for running the Non-Load Event Tracker can be found
CAP for CTI using an RTS (pdf)
Joe's Summary on radiation response doc,
Rad response Ex. 1 doc,
Rad response Ex. 2 doc,
Rad response Ex. 3 txt,