11 February 2008

This release includes new versions of everything, in that anything that used to
use numarray now uses numpy. Furthermore, there are changes to PyRAF, PyFITS,
pytools, Multidrizzle and related tasks.

Last release to support Numarray

This release is the last release that supports numarray, even as a deprecated
package. Numarray is no longer used within STScI_Python. Support for
numarray will be eliminated entirely for the next release. Support for numarray
now is limited to critical problems or major platform support issues. The
threshold for 'critical' or 'major' is continually becoming harder to meet;
you are advised to migrate to numpy as soon as possible. Information on
making the switch to numpy from numarray may be found at this
link.

Platform Support

Normally, platform support issues do not apply to Python tasks, as most Python
code will run on all platforms on which Python has been installed. This
distribution was tested to correctly support installation on Linux, Mac OS X,
and Solaris, while also being provided for installation under Windows. The
single exception is that the text-based epar functionality now available in
PyRAF (in addition to the already existing GUI-based epar) is not available
under Solaris, and likely will never be.

Documentation

Documentation for these tasks has been consolidated into a single location
complete with an index viewable as a local web page. The documentation
includes any available user guides and API documentation generated using
'EPYDOC' for the modules or packages. This index can be accessed using:

--> import
stscidocs-->
stscidocs.viewdocs()-->
stscidocs.help()

This will automatically bring up the default web browser application to
display the index page. All the available documentation for software packaged
with this distribution can be accessed through this interface, as this
documentation resides locally upon installation.

Python Environment

Python 2.3 or later is required. We recommend using the latest version of
Python if there is a choice (2.5.x)

PyRAF 1.5

Changes since the PyRAF 1.4 release:

The following enhancements were made:

Some general refactoring of PyRAF's GKI-related classes was performed in
preparation for the coming Matplotlib graphics kernel. The first look
at this new capability is expected in PyRAF 1.6. (ticket #76)

The PyRAF 'time' command now matches the format of the associated cl
command. (ticket #67)

Two graphics-related test modules were added to PyRAF for automated
regression testing. (rev 802, no ticket)

Internal GkiKernel behavior was changed so that communication pipe
constructs are able to handle special redirection cases. This is
useful for the embedding of PyRAF within external applications.
(ticket #74)

The following bugs were fixed:

There was a bug which allowed the wrong package to be assigned to a task in
cases where the package script loaded a different package. (ticket #68)

The algorithm used by PyRAF to search for task parameters was changed to
more exactly match that used by the cl. (rev 796, no ticket)

The sscanf module's buffer size in PyRAF was made to be consistent with the
same buffer size in the cl. (ticket #65)

The ability to append graphics (e.g., via "append+") was not working
correctly due to a problem in the logic of when to initialize a new graphics
engine. (ticket #70)

As a performance improvement, element checks for IRAF arrays were
simplified, otherwise the whole array was being checked every time a single
element was assigned. For large arrays (thousands of elements), this
resulted in a performance increase of orders of magnitude. (rev 803, no
ticket)

On Linux (Ubuntu) a user was seeing a traceback with "cd .." which turned
out to have to do with poor handling of Unicode in PyRAF commands. Such
PyRAF commands are now being handled correctly. (ticket #73)

PyFITS 1.3

This release contains PyFITS Version 1.3 (24-Jan 2008). Updates described
in this release are only supported in the NUMPY version of pyfits.

The following enhancements were made:

Provided support for a new extension to pyfits called stpyfits.

The stpyfits module is a wrapper around pyfits. It provides
all of the features and functions of pyfits along with some STScI specific
features. Currently, the only new feature supported by stpyfits is
the ability to read and write fits files that contain image data quality
extensions with constant data value arrays. See:

http://stsdas.stsci.edu/pytools/stpyfits

for more details on stpyfits.

Added a new feature to allow trailing HDUs to be deleted from a fits file
without actually reading the data into the file.

This supports a JWST requirement to delete a trailing HDU from a file
whose primary Image HDU is too large to be read on a 32 bit machine.

Updated pyfits to use the warnings module to issue warnings. All warnings
will still be issued to stdout, exactly as they were before, however, you
may now suppress warnings with the -Wignore command line option. For
example, to run a script that will ignore warnings use the following command
line syntax:

python -Wignore yourscript.py

Updated the open convenience function to allow the input of an already
opened file object in place of a file name when opening a fits file.

Updated the writeto convenience function to allow it to accept the
output_verify option.

In this way, the user can use the argument output_verify='fix' to allow
pyfits to correct any errors it encounters in the provided header before
writing the data to the file.

Updated the verification code to provide additional detail with a
VerifyError exception.

Added the capability to create a binary table HDU directly from a
numpy.ndarray. This may be done using either the new_table convenience
function or the BinTableHDU constructor.

The following performance improvements were made:

Modified the import logic to dramatically decrease the time it takes to
import pyfits.

Modified the code to provide performance improvements when copying and
examining header cards.

The following bugs were fixed:

Corrected a bug that occurs when reading the data from a fits file that
includes BZERO/BSCALE scaling. When the data is read in from the file,
pyfits automatically scales the data using the BZERO/BSCALE values in the
header. In the previous release, pyfits created a 32 bit floating
point array to hold the scaled data. This could cause a problem when
the value of BZERO is so large that the scaled value will not fit into the
float 32. For this release, when the input data is 32 bit integer, a
64 bit floating point array is used for the scaled data.

Corrected a bug that caused an exception to be raised when attempting to
scale image data using the ImageHDU.scale method.

Corrected a bug in the new_table convenience function that occurred when a
binary table was created using a ColDefs object as input and supplying an
nrows argument for a number of rows that is greater than the number of rows
present in the input ColDefs object. The previous version of pyfits
failed to allocate the necessary memory for the additional rows.

Corrected a bug in the new_table convenience function that caused an
exception to be thrown when creating an ASCII table.

Corrected a bug in the new_table convenience function so as to allow the
input of a ColDefs object that was read from a file as a binary table with a
data value equal to None.

Corrected a bug in the construction of ASCII tables from Column objects that
are created with noncontinuous start columns.

Corrected bugs in a number of areas that would sometimes cause a failure to
raise an exception when an error occurred.

Corrected a bug where attempting to open a non-existent fits file on a
windows platform using a drive letter in the file specification caused a
misleading IOError exception to be raised.

numdisplay 1.4

Removed all support for numarray objects so it only works with numpy objects
as input.

Added a reset method to allow for better user control over parameters
controlling how the array gets transformed for display.

A new method was added to allow the user to get the values related to the
properties of the display buffer.

Scaling of arrays where all values are much less than 1.0 now handled
correctly.

imagestats 1.2

Modified to remove all compatibility with numarray, relying on native numpy
C-api.

Pytools

This set of modules has been used to create a formal package for use under
Python. All code from this package now need to be accessed using:

--> import pytools
or
--> from pytools import <module>

In addition, the following changes were made to the code in the (new) package:

A new module, stpyfits, was added. This module is an extension to the pyfits
module which offers additional features specific to STScI; specifically, the
handling of Constant Data Value Arrays.

A new function 'isFits' has been added to the fileutil module to determine
whether an input file is a FITS file or not, and if so, what format of FITS
file; namely, 'waiver' FITS (primary header with data cube and binary table
extension containing parameter values related to each slice of the data
cube), multi-extension FITS (MEF), or 'simple' FITS (primary header only,
with or without data).

Updates were made to several modules to support automatic conversion and use
of GEIS images through the 'openImage' function in fileutil. The
changes to 'readgeis.py' included allowing it to work under win32, and
populating the PyFITS object with the extname and extver so that the
'[sci,1]' syntax will work just as if it were read in from a FITS file.

Error messages are now reported when using 'readgeis' or 'openImage'
to read in a GEIS image suspected of having a byteorder problem.

Docstring was also heavily updated for the 'openImage' function.

Changed the 'irafglob' function to raise an IOError exception instead of an
empty list when no valid input files are found.

The 'parseinput' function can now support IRAF style @files which may
include a second column of filenames.

Applications

MultiDrizzle 3.1

The following changes were made:

Automatic generation of IVM files for use with NICMOS data has been added.

Error states now generate exceptions with error messages more directly
related to the cause of the problem.

Now updates WFPC2 DQ image (c1h files as converted to FITS format) with
cosmic-ray mask identifications made by MultiDrizzle. If no c1h files are
present on input, the cosmic-ray identifications do not get reported to the
user.

Simplified the creation of a NICMOS dark image for use in IVM
generation. Multidrizzle assumes a constant dark rate for the
detectors.

The following bugs were fixed:

Fixed a bug in how the output name gets determined for a single input image.

Fixed a bug in the handling of flat field images in the NICMOS IVM
generation routines. Multidrizzle now uses NICMOS flats that are
properly gain corrected.

The algorithm for creating the IVM files automatically was corrected.

Removed all remaining references to use of older cubic distortion
coefficients for WFPC2 data, replacing them with references to IDCTAB
reference file.

Fixed a problem with the error messages generated when the boxcar
convolution in the minmed algorithm fails.

PyDrizzle 6.1

The following changes were made to improve support for WFPC2 data:

DQ information for WFPC2 data now correctly recognized for all input formats

PyDrizzle creates an empty DQ array for use by PyDrizzle if none are found.

The following bugs were fixed:

Processing an image which does not have any distortion coefficients now
works as expected; specifically, no changes get imposed on the image.

Corrected a problem in the definition of the default DQ array

CALCOS 1.2

The COS pipeline calibration processing software, calcos, has been added to this
release. This represents the first public release of this code in support
of the upcoming installation of COS on HST during Servicing Mission 4 currently
scheduled for September 2007. This code performs all the standard
calibrations to COS data for use in pipeline processing or reprocessing by an
observer.

NICMOS Data Analysis

The tasks RNLINCOR, SAACLEAN, and PUFTCORR have been reorganized so that they
are now part of a larger set of tasks designed for NICMOS data analysis and
calibration, the NICTOOLS package. As before, they can each be imported
independently and can be used on NICMOS data directly from Python without any
STSDAS interface available to run them. This package also includes new
tasks; namely,

nic_rem_persist: This is the first version of a tool to remove
persistence from NICMOS data. It is being provided on an experimental
basis only, with extensive testing and development still being
required.

temp_from_bias: This task computes the detector temperature based on
the measurements of the bias values in NICMOS data. It is being provided on
an experimental basis only, with extensive testing and development still
being required.

Bug fixes included in the release:

Handling of the sky values corrected in rnlincor.

Version numbers no longer printed out when running saaclean.

STIS Data Analysis

The STIS data analysis tasks were reorganized into a single package,
stistools. This package still contains modules for supporting data
analysis tasks developed for post-pipeline processing of STIS data; namely,
wx2d, stisnoise, mktrace, sshift. They can each be imported independently and
used on STIS data directly from Python.

No changes were implemented to any of the tasks themselves in this package
for this release.

WFPC2 Data analysis

A new package, wfpc2tools, has been added to provide new tools for working
specifically with WFPC2 data. This new package currently contains the
following modules:

wfpc2cte: computes estimates of the effects of CTE on an input WFPC2
image, and reports the change in magnitude the CTE would impose on point
sources with 3 values of total flux. The algorithm used by this module is
explained in the help for the module, and can be obtained using the built-in
help() command:

--> import wfpc2tools;wfpc2tools.wfpc2cte.help()

wfpc2destreak: An experimental module which attempts to identify when
low bias readout of WFPC2 data causes uneven bias values across the chip,
and attemps to correct the uneven background caused by this bias readout
problem. This module is provided on an as-is basis as it still
requires additional testing and development.