ACIS Shift report H342 June 28 2210, GMT 180:22:10:00, EDT June 28, 2000 18:00 PLAN: Execute CAP 626 to dump diagnostic information from ACIS to understand the FEP 0 and FEP 3 anomaly and power down the FEPs and restart a normal CTI run to confirm the problem has been cleared. SUMMARY: Given the new ACIS anomaly CAP 625 to run Squeegy 7 was postponed to run CAP 626 to diagnose and recover from the FEP 0 and 3 anomaly. The ACIS braintrust (Peter, Bev, Royce and Catherine) developed the hypothesis that the threshold crossing plane (memory) had become stuck. This would cause ACIS to report events from the same pixels frame after frame as valid events regardless of the data in those pixels. It was presumed that this condition was trigerred by the bias only run which was run from the daily load (OBSID 62015). Apparently a bias-only run with the FEP mode set to "raw" and the BEP packing mode set to "faint" can cause the same problem seen at XRCF when we were trying to trickle bias while telemetering event data. The problem is that two processes from the flight SW are accessing the same piece of memory too close together in time, confusing the HW and preventing further updates to this memory. Once the threshold crossing plane is locked up, it will stay the same frame after frame, until the FEP is powered down. We had a horrendous time getting COM on the spacecraft and data to the EGSE. The first problem was due to unexplained difficulties at Madrid and the second was due to a change in the firewall configuration at the OCC. We eventually gave up on Madrid and switched to Goldstone. We got 45 minutes of COM on Goldstone so we had to run a truncated version of the CAP. The original CAP was intended to: 1. Dump Flight SW 2. Dump volatile memory, steps 1-4,5L-5W,7L, \& 10 of SOP 61024 3. Power down the FEPs, WSFEPALLDN 4. Warm Boot the BEP, SOP 61070 5. Execute normal CTI I array (rerun of CAP 574) Given the short COM pass, we accomplished steps 3, 1, and 5 in that order. The data from the normal CTI run showed that FEP 0 and FEP 3 were behaving nominally. The event rate was nominal on FEP 0 and 3 and the number of threshold crossings was changing from frame to frame indicating that the threshold crossing plane was updating on every frame as it should. Nevertheless, we should examine the data from the next pass to check that FEP 0 and FEP 3 are still producing valuable data. We believe the FEP 0 and FEP 3 problem has been cleared by tonight's actions. We thank the rapid response of the ACIS team in developing the appropriate contingency plan and understanding the anomaly. We also thank the persistent efforts of the OC to find us COM on Goldstone so that we could correct the problem and save the upcoming science observations. 1) Detailed Timeline: GMT 180:22:10 AOS on DSS 66 180:22:11 1CR[AB]T = -123.1/-120/7 C 180:22:12 SQUIRT not sending data to GRETA or ACIS EGSE 180:22:30 execute step 1 of CAP 626 to uplink 1A_RB009_196.CLD 180:23:08 COM becomes extremely ratty from DSS 66 180:23:10 SQUIRT back up FP temp = -116.6 C ** very warm !!! ** 180:23:12 OC and DSS 66 start switching equipment to improve quality of TLM 181:00:15 Abandon DSS 66 and Switch to DSS 24 FP temp = -118.6 C 181:00:50 AOS on DSS 24 181:00:58 switch to 1024 bit rate to get the dump down faster 181:01:05 Command capability 181:01:06 execute steps 5 and 6 of CAP 626 to power down the FEPs 181:01:07 ACIS confirms FEPs are down execute step 7 of CAP 626 to clear and disable SCS 165 181:01:09 execute steps 10 of CAP 626 which is steps 1 and 2 of CAP 574 to load and enable and activate 1A_CTI00_137.CLD to start a normal ACIS I CTI measurement OBSID is 62010 set by the daily load 181:01:12 ACIS confirms science run in progress 181:01:17 execute step 2 of CAP 626 to enable and activate SCS 196 which dumps the ACIS flight SW 181:01:23 ACIS flight SW dump complete 181:01:25 execute step 3 of CAP 626 to clear and disable SCS 196 181:01:27 Bias data start from this run, looks normal 181:01:38 first event data, rates look normal on FEP 0 and FEP 3, threshold crossings also look normal 181:01:39 executed step 4 of CAP 574 to clear and disable SCS 137 181:01:46 LOS on Goldstone 2) Chandra state at the end of pass: UTC 2000:181:02:39:01 (Jun29) f_ACE 6.53e+01 F_ACE 1.88e+05 Kp 2.7 R km 74119A OBT 2000:181:01:45:13 CTUVCDU 10953343 OBC s/w NRML FMT2_NORM CPEstat OBT 78630313.21 ONLVCDU 10953328 OBC Errs 0 OBSID 62010 EPState SUN SIM TTpos -99616 HETG Angle 80.2 PCADMODE NPNT RA 142.393 Bus V 31.02 SIM FApos -467 LETG Angle 78.3 PCONTROL NPNT Dec -62.260 Bus I 33.30 AOFSTAR GUID Roll 219.890 ACA Object SSSSSSSS Dither ENAB HRC-I HV OFF ACA ImFunc TTTTTNNN Dith Yang -5.02 Yaw Rate 0.19 HRC-S HV ON ACA CCD Temp -10.4 Dith Zang -2.62 Pitch Rate 0.10 OBSMode NEXT ACA Int Time 1.698 Roll Rate 0.10 Door OP UNAC AOACSTAT OK FSS SunBeta SUN Door CL UNAC FSS Alfa -0.40 Batt 1 SOC 100.00% Avg HRMA Temp 70.58 FSS Beta -2.67 Batt 2 SOC 100.00% ACIS Stat7-0 64 Avg OBA Temp 50.97 SA Resolv 92.52 Batt 3 SOC 100.00% Cold Rad -127.9 OBA Tavg Fault NFLT SA Sun Pres ILLM Warm Rad -82.0 OBA Trng Fault NFLT +Y SA Amps 15.85 HRMA power 15.50 SCS 128 -Y SA Amps 17.03 EPHIN Geom LARG OBA power 111.00 SCS 129 +Y SA Temp 107.64 E150 74 SCS 130 -Y SA Temp 107.37 E300 15 Roll Mom. 2.361 E1300 3 Pitch Mom. 17.920 UpL Cmd Acc 30394 EPH A-Leak 0.4720 P4GM 2 Yaw Mom. 0.488 Cmd Rej A 168 EPH B-Leak 0.0760 P41GM 0 SOT LEAD: Dan Schwartz ACIS: Paul Plucinsky, Peter Ford, Bev Lamarr, Royce Buehler, Catherine Grant