Study of Hubble Archive & Reprocessing Enhancements (SHARE)
Introduction:
The calibration of HST data is one of the primary areas where the STScI
adds value to the Hubble science program. The Hubble Archive has become, in
itself, a major resource for the astronomical community. The imminent
operational debut of the On-the-fly Reprocessing(OTFR) system will radically
change the paradigm that we have applied to the calibration and storage of
HST data. This system will open up avenues for the STScI to further enhance
the scientific value and impact of the HST data sets stored in the archive.
Progress in the field of astronomical surveys and catalogs has been great in
the past few years and suggests possibilities for increaasing the scientific
scope of our archive beyond that originally envisioned when the system was
designed. This is an appropriate time for us to evaluate the scientific
potential of such enhancements and develop a roadmap for implementing the
most promising of them.
In the past, the requirement to store uniformly calibrated data has
generally prevented adoption of algorithms requiring user input, or
knowledge of the astronomical scene. The OTFR concept removes those
restrictions and could allow selection of algorithms or processing paths by
the user. Our initial pipelines dealt with only one data set at a time.
Later, pre-defined associations of data sets were developed to allow
processing of related data sets, such as wavelength calibration of
astronomical spectra via internal calibration exposures. The OTFR concept
allows for post facto definition of associations of data, either permanent
or on-the-fly. This could allow application of simple associations to SIs
for which this was not originally available (i.e. WFPC2). This could also
allow processing of larger groups of data sets to provide more
scientifically valuable products, such as summed data sets or mosaics. The
archive catalog was originally conceived as simply an index into individual
observations. Already, its scientific value has been increased by
incorporation of pre-view images, and we are working in the direction of
seamless access to data across missions. At this point, it would be
technically feasible to extend the Hubble archive to include scientific
services such as object catalogs, their generation from HST data sets, and
direct cross-references to other catalogs and databases. Many of these
ideas are also under contemplation for the NGST era, and it is worth
considering how the archive might smoothly evolve to provide similar
services for HST, NGST, and MAST holdings.
Charter:
A cross-divisional committee is hereby established to evaluate the
scientific potential of a wide variety of possible enhancements to the
scientific data products provided by the STScI via the Hubble Archive and
the OTFR system. This committee will consist of members from the Hubble
Division, the Archive Catalog and Data Services Division, and the
Engineering and Software Services Division and the Next Generation Space
Telescope Division. The committee is encouraged to arrange participation of
staff from the ECF and CADC, and should feel free to solicit contributions
from others at STScI and in the astronomical community.
1.) The committee is requested to evaluate and recommend general
capabilities, services and enhancements to these systems.
2.) The committee is requested to evaluate and recommend specific new
scientific services and enhancements to these systems. These
should be augmentations with clear and substantial added benefit
for the research community.
3.) The committee is requested to provide a rough roadmap or order for
the implementation of the recommendations made in item 1 and 2.
4.) The committee is requested to recommend a process for encouraging
astronomical community participation in the development of such
enhancements.
5.) The committee is requested to recommend a process for regularly
assessing and prioritizing enhancements of this type in the future.
The committee is encouraged to have a preliminary report by
June 15, 2001 and a final report by October 15, 2001.