Transcription

1 WEB-BASED AUTOMATED REPORTING: SAVING TIME, MONEY, AND TREES Jeffrey A. Fox* Pacific Northwest National Laboratory 901 D Street, SW, Suite 900 Washington, DC Cindy Starr, Paul Baker, Kai-Dee Chu Global Science and Technology, Inc Ivy Lane Suite 300 Greenbelt, Maryland Julie Breed NASA/Goddard Space Flight Center Greenbelt Road Greenbelt, Maryland Mick Baitinger 8403 Colesville Road, Suite 915 Silver Spring, Maryland NEXGEN Solutions, Inc. Abstract In the current "smaller, faster, cheaper" world of space programs, there is increased pressure to dramatically reduce costs. This has led to the design of smaller spacecraft and to a reduction in mission personnel. One of the major obstacles in reducing staff in all phases of a mission is the significant amount of paperwork that the staff must complete. For example, during routine operations, the Spacecraft Control Team (SCT) may need to complete a variety of reports including anomaly reports, pass summaries, and weekly summary reports. Boxes of these reports are archived indefinitely. The Spacecraft Emergency Response System (SERS) is an innovative Web-based suite of tools created to support lights-out operations. SERS automates many of the reporting activities required during both operations and integration and test (I&T). Currently, SERS is being used by several Small Explorer (SMEX) missions. In *Pacific Northwest National Laboratory is operated for the U.S. Department of Energy by Battelle under Contract DE- AC06-76RLO addition, SERS will be used by the Hubble Space Telescope (HST), and missions from Middle Explorers (MIDEX) and the New Millennium Program (NMP). Key words: Lights-out operations, Automation, SERS, Anomaly management. Introduction At NASA s Goddard Space Flight Center (GSFC) and at space centers elsewhere, the trend is to reduce mission operations costs through automation, as demonstrated by a number of papers presented at the 1998 SpaceOps Conference Some organizations are even moving to lights-out operations for unmanned spacecraft, where many of the standard operational activities are automated. In this operational setting, the spacecraft control team (SCT) no longer continuously occupies the mission operations center (MOC). The goal of lights-out operations is to reduce the overall cost of spacecraft operations by reducing the total number of hours that personnel are directly involved in day-to-day

2 activities. Instead, a reduced number of team members work a standard shift (perhaps 9-5, Monday through Friday) performing those tasks that cannot be automated in a cost-effective fashion. The rest of the time, the SCT acts as on-call team members who respond to notifications of anomalous conditions. The effectiveness of this approach depends on a number of factors, including: How well anomalous conditions can be identified How much of the former in-moc activities can be automated, and How well the alert system works. NASA/GSFC's Advanced Architectures and Autonomy Branch (Code 588) is working to build technologies to help facilitate the transition to lights-out operations. One such effort is a research project called the Virtual Mission Operations Center (VMOC). The goal of the VMOC project is to work closely with operations staff to develop new technologies and workgroup computing concepts to more effectively meet the new objectives for mission operations. The VMOC project is exploring the feasibility and effectiveness of technologies such as computersupported collaborative work (groupware), wireless communications devices, and distributed operations in order to meet these goals. One result of the VMOC effort is the development of the Spacecraft Emergency Response System (SERS). SERS is a Web-based suite of tools created to monitor and summarize spacecraft operations and notify the SCT when anomalous conditions arise. During operations, SERS provides autonomous logging of events that require human attention (as defined by the SCT). SERS also automatically generates summaries of each pass. In addition, SERS supports automatic schedule changes and the manual reporting of anomalies. Upon logging anomalies, SERS' software agents perform customizable workflow processes that automatically generate reports and alert the appropriate SCT member(s) and/or engineer(s). This paper provides more detail on SERS' reporting capabilities. Automation During Operations While SERS supports anomaly logging and tracking during I&T, its capabilities are best utilized during the operational phase of a spacecraft. During routine operations, SERS receives notifications from expert systems and automatically examines ground system log files for anomalous conditions. When such a condition is detected, SERS creates an Event Report to document the condition(s) and automatically notifies the appropriate on-call team members. SERS also autonomously generates Pass Summary Reports that summarize the contents of each ground system log file that it examines. In addition, SERS handles schedule changes submitted by the Deep Space Network (DSN). Many ground systems generate log files for each contact between the ground station and a spacecraft. These contacts occur throughout the day; so in lightsout operations, many of these contacts are performed autonomously. The log file that is generated contains documentation on the status of the spacecraft and its subsystems (i.e., health and safety data), along with any commands or procedures initiated by the SCT. SERS examines these log files with two different processes (1) to determine if a condition has arisen that requires the SCT to be notified and (2) to summarize notable messages and errors that were detected during the pass. These two processes are independently configured and executed, so that changes in the configuration of one will not affect the processing of the other. Event Reports & Notifications In SERS, an "event" is defined as a condition that merits the attention of the SCT and that may require some human intervention. When a new event occurs, SERS determines whom to notify based upon the event's characteristics, user-defined filters, an on-line schedule, and profiles of the SCT. SERS then mediates the response to the alert by notifying back-up personnel if the initial individuals notified of the problem do not respond to the alert. SERS uses a variety of mechanisms (pagers, , Web, and telephone) to communicate with the SCT. More detailed accounts of how SERS performs these functions are described in Fox, et. al. 13 and Baker, et al. 14 A high-level diagram of the process is shown in Figure 1. Events in SERS are generated in two ways: (1) by receipt of a formatted message sent from an expert system or another outside process and (2) by the SERS Regular Event Processor (REP). In both cases, SERS receives the details of the event in a formatted e- mail message and autonomously generates a Pass Event Report, which is available immediately via the Web. By providing Web access, SERS allows the SCT to view reports from any computer that has a Web browser. A typical Pass Event Report is shown in Figure 2. SCT personnel who are notified of the problem can access the Web page to obtain complete information about the event. If the spacecraft's front-end is an expert system, like Altairis, 15 the expert system sends SERS an

3 Spacecraft Dump H&S Data Front-End System Generate Log File Expert System Process Data FTP SERS REP Parse Data Generate & Manage Reports (Web) Pager, Phone, Web SCT Read Alert Notification Respond to Alert Annotate Reports Identify Event Notify SERS Issue Alert Notification Web Migrate Event to Anomaly Report Figure 1: SERS-Mediated Workflow for Operational Event Handling message depicting the state of the spacecraft when it determines that an anomalous condition has arisen. In addition, other external processes can automatically e- mail SERS when problems are detected. These currently include weather alerts, along with various processes that monitor the status of the other system hardware and software. Each alert causes a Pass Event Report to be generated and designated SCT personnel to be notified. If a mission is not using an expert system as part of its front-end, the SERS REP parses the log files from ground systems, like ITOS 16 and ASIST, 17 to determine if they contain any conditions requiring SCT notification. The SCT defines the items of interest over the Web by completing forms that define the elements of interest in the log file. These are known as filters. The SCT then defines the criteria for those filters that will either trigger an alert, provide contextual information, or inhibit an alert notification. When SERS finds the filter criteria in a log file, it automatically generates a Pass Event Report that includes information about the identified problem and any pre-selected contextual data. Alternatively, inhibit filters can be defined to prevent unnecessary alert notifications from being sent. Inhibit filters are generally narrowly defined to match certain recurring messages that may contain data that are not nominal, but are nonetheless not of interest to the SCT. This reduces the number of unnecessary notifications to a minimum. Pass Summaries SERS generates Pass Summary Reports based on data from two sources: (1) the Deep Space Network (DSN) schedule and (2) the ground system log file. SERS first uses DSN's 7-day pass plan, or schedule, to automatically generate skeleton reports containing schedule data for each pass for the spacecraft it monitors. A typical DSN Schedule for SMEX missions is shown in Figure 3. As the schedule is revised throughout the week, these skeletal Pass Summary

4 Figure 2: Typical SERS Pass Event Report

5 Figure 3: Typical DSN Schedule for SMEX Missions Reports are deleted, created, or modified to reflect changes in the schedule. Next, SERS generates a temporary report for each ground system log file once the REP has completed its examination of the file. Information that is determined to be relevant by the SCT is extracted from the log file and stored in a temporary summary. The temporary summary is then matched to the previously generated Pass Summary Report based on date, time, spacecraft, and orbit number. Relevant log file information from the temporary summary is posted automatically into the Pass Summary Report and the temporary summary is subsequently deleted. A typical Pass Summary Report is shown in Figure 4. At certain times, matches cannot be automatically determined between the temporary reports and the Pass Summaries generated by the schedule. For example, passes may be taken that are not on the schedule. In this case, the SCT may manually generate an "Unscheduled Acquisition." This report functions in the same way as a Pass Summary Report generated from a schedule. Log file information will automatically propagate the form provided the time/date and orbit number match. Another case is when the information in the log file may not match the Pass Summary Report that was generated from the schedule. This may be due to a data entry error during the pass or a time shift from the original schedule. When this occurs, unmatched log information is posted to the Web. When the unmatched fields are corrected by the SCT, the unmatched log data will post automatically to the correct Pass Summary Report.

6 Figure 4: Typical SERS Pass Summary Report (Top Portion)

7 Schedule Changes SERS also handles schedule changes from the DSN. As shown in Figure 5, a person from the DSN can notify SERS of a slip or cancellation of a spacecraft contact via the Web or by calling into SERS with a phone. When using a phone, the user is guided through a series of menus to select (1) the appropriate spacecraft, (2) the contact that must be modified, and (3) the type of modification (e.g., slip or cancel). All menu selections are made via the keypad. Users can enter comments about the modification by speaking into the phone. SERS records the voice as an attachment to the report. SERS then automatically generates a Schedule Modification Report and notifies the appropriate SCT member(s). In the future, SERS will allow on-call SCT members to accept or reject schedule changes via 2-way pager. SERS will then pass the information about the changes on to the front-end expert system so that it may make the appropriate modifications to handle the pass. Similarly, DSN personnel can send more general communications alerts to the SCT via SERS. VMOC Vision VMOC's ultimate goal is to provide a suite of integrated collaboration tools to support the design and documentation of the entire life cycle of a spacecraft. In the VMOC program, new longer-range technologies and concepts are being explored. One such concept is an expert system that analyzes the health and safety, or state data, of a spacecraft, as recorded in SERS Pass Event Reports. The system then automatically generates hypertext (Web) links from the SERS report to the appropriate on-line support information. A first step in this process will be to provide tools that increase SERS' utility in diagnosing and resolving anomalies. For example, if SERS logs limit violations in the attitude control system (ACS) of a spacecraft, the expert system would automatically insert links (as part of the Web report) to schematics of the ACS, documentation on the operation of ACS, descriptions of the out-ofrange mnemonics, other anomaly reports similar to that one, and other diagnostic tools. Conclusion Moving to a lights-out approach to operations requires the automation of many tasks. SERS is a system designed to facilitate this new paradigm. SERS provides tools to automate report generation for events, passes, and contact schedule changes. It also manages the alert notification and response process for anomalous events. In addition to the capabilities described in this paper, new features are being investigated as part of the VMOC effort and will be integrated into SERS, as they become available. DSN SERS SCT Replan Contact Schedule Telephony or Web UI Receive and Parse Message Generate Report Issue Alert Notification Pager View Notification Figure 5: SERS-Mediated DSN Notification

Oracle Fixed Scope Services Definitions Effective Date: October 14, 2011 "You" and "your" refers to the individual or entity that has ordered Advanced Customer Services from Oracle or an authorized distributor.

THE TOWN OF GROTON IS REQUESTING PROPOSALS FOR A WIRELESS ALARM MONITORING AND CONTROL SYSTEM WITH INTERNET BASED DATA ACCESS The proposal shall be for supplying pre-wired cellular ready to install field

TIMED Mission System Engineering and System Architecture David Y. Kusnierkiewicz Aspace mission consists of more than just a spacecraft and scientific instruments. It also includes a ground system to support

reserves the right to change the specifications, functions, or features, at any time, without notice. has prepared this document for the use by its employees and customers. The information contained herein

USER GUIDE Voice Verizon Business National Unified Messaging Service Enhanced Service Guide What Is Unified Messaging? Verizon Business National Unified Messaging Service is an interactive voicemail system

1. Definitions & Interpretations Conditions for ICT Partner Solutions Service Schedule for The following definitions shall apply to the provisions of the Service, in addition to those in the General Terms

Engineering Change Control Systems (atecc) Product Life Cycle Management Enterprise information boundaries are disappearing as corporations open their networks to allow external access by manufacturing

System i Systems Management Advanced job scheduler Version 5 Release 4 System i Systems Management Advanced job scheduler Version 5 Release 4 Note Before using this information and the product it supports,

A SYSTEMS UNDERSTANDING A 1.0 Organization Objective: To ensure that the audit team has a clear understanding of the delineation of responsibilities for system administration and maintenance. A 1.1 Determine

Version 4.61 or Later Copyright 2013 Interactive Financial Solutions, Inc. All Rights Reserved. ProviderPro Network Administration Guide. This manual, as well as the software described in it, is furnished

SOFTWARE DEVELOPMENT STANDARD FOR SPACECRAFT Mar 31, 2014 Japan Aerospace Exploration Agency This is an English translation of JERG-2-610. Whenever there is anything ambiguous in this document, the original

Software Engineering Architectural Design 1 Software architecture The design process for identifying the sub-systems making up a system and the framework for sub-system control and communication is architectural

The Big Data Mining Company THE GLOBAL EVENT MANAGER When data is available and reachable, it has to be processed and decrypted using multiple heterogeneous tools, if these are available. Each of these

v.2016.02 PROFESSIONAL SERVICES Training Services Computrition offers a wide array of training services to streamline the implementation process. Of all the factors critical to the success of an automation

System i and System p Customer service, support, and troubleshooting System i and System p Customer service, support, and troubleshooting Note Before using this information and the product it supports,

Open EMS Suite by Nokia Software Development Kit Functional Overview Version 1.3 Nokia Siemens Networks 1 (21) Software Development Kit The information in this document is subject to change without notice

CHAPTER 24 Setting Up Broadcast Messaging in Cisco Unity 8.x System broadcast messages are recorded announcements that are sent to everyone in an organization (or to particular locations within an organization).

Streamlining Communications at Medical Facilities Across the Globe 2015 Cleo. All rights reserved. Cleo is a trademark of Cleo Communications US, LLC. All other marks are the property of their respective

Chapter 14 Working with SQL Server Agent Jobs Microsoft SQL Server features a powerful and flexible job-scheduling engine called SQL Server Agent. This chapter explains how you can use SQL Server Agent

SPRINT MANAGED SECURITY SERVICES PRODUCT ANNEX The following terms and conditions, together with the Sprint Master or Custom Services Agreement or Domestic Sprint Services Sales Application Form ("Agreement"),

BridgeConnex Statement of Work Managed Network Services (MNS) & Network Monitoring Services (NMS) 1. Introduction This Statement of Work (SOW) is an appendix to the existing Master Services Agreement between

3.11 The functional area is intended to contribute to the overall flexibility, efficiency, and security required for operating and maintaining the system. Depending on the architecture of the system, system

Managing Enterprise-wide Printing: The IT Challenge Presented by September 2002 There is a way To inject true efficiency into enterprisewide printing To have enterprise-wide visibility, total control and

SQL Server Automated Administration To automate administration: Establish the administrative responsibilities or server events that occur regularly and can be administered programmatically. Define a set

Advanced User s Guide Version 2.0 Contents Introduction to the Documentation... 3 About the Documentation... 3 Ifbyphone on the Web... 3 Logging in to your ifbyphone Account... 3 Setting Up a Voice Mailbox...

Microsoft Windows SharePoint SharePoint Basics Introduction What is Microsoft SharePoint? SharePoint is a tool to connect people and information. It provides a central site for sharing information with