Flatfielding has been added for line scans only to the standard product generation (SPG) pipeline.

Spectrometer RSRF band R1 provides correct line fluxes in red leak region (beyond 190 microns). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within Hipe.

Update of the spectrometer telescope background model with new parameter set representing the aging model of the optical surface.

SPIRE

Photometer:

Update to flux calibration to use new Neptune model (the ESA4 model instead of the ESA2 model).

Spectrometer:

The telescope and instrument RSRFs have been updated to produce 20% lower instrument noise.

The beam FWHM inside the SCalSpecBeamParam product has been updated.

Core system

HIPE

Batch mode

The hipe executable can be used as a replacement of jylaunch to run scripts in batch mode.

In HIPE 11, jylaunch will still be available, although it will print a warning informing that it shall be removed in HIPE 12.

Plug-ins

If the latest version of a plug-in is not compatible with an older version of HIPE, when running the older version of HIPE, it will not pick up an older version of the plug-in that is compatible (if available).

Tasks defined in plug-ins now have their source code available from the Task GUI.

Navigator

Compressed files can be opened with either double click or right click and Open With.

The default folder that is offered in file dialogues when opening or saving files is now customizable. The behaviour can be set in the General > Directories preferences panel.

Data Access & HSA

There is a new Data Access Perspective, bundling all functionality needed to get data in and out of HIPE.

The HIPE status bar now indicates the log-in status with the HSA (logged in as...). If the connection to the HSA is lost, for example because a WiFi connection is dropped, it will be shown as well.

HIPE now shows the currently logged in user on the status bar, next to a lock icon. If no user is logged in, one can double-click the icon to log in. Also, if no user is logged in and log-in information is required, rather than failing, HIPE will now pop up a log-in window, allowing the user to specify username and password.

Editors & Viewers

Edition of product's metadata through the GUI does not allow to remove mandatory parameters (product's attributes).

The search dialogue for text editors is now more user-fiendly:

It focuses the text pane once opened, so the user doesn't need to click on it.

It shows an info label saying how to trigger the search with the keyboard: Ctrl+Enter.

It does not lose the focus, so that multiple entries can be searched with the keyboard one after the other.

Undo/redo is available in the search text pane, by means of the usual key combinations.

Console

Commands in the history are indexed so that the first executed command in the current session has index 0. Commands in previous sessions have negative indices.

As previously, the complete command history can be obtained by running !history, and a specific command can be recovered by its index through !<index>.

Plotting

A new axis type TAI

Bug fixes

TablePlotter

OverPlotter

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

sourceExtractorSimultaneous now has a region of interest (roi) parameter

Images

Display

Speed improvements.

A lot of bug fixes.

Getting the distance between two points can be done using left mouse click, combined with SHIFT (and CONTROL).

Contour drag and drop also shows the color settings on the console.

photApertureCorrectionPointSource can be drawn on an Image.

Coordinates can be shown as decimal degrees.

It is possible to show extra datasets added to the image or the cube in Display.

It is possible to select which flags should be overlayed on the display.

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

New parameter 'resolution' used internally to speed up the fitHifiFringe task. The default is 10 MHz and should be reduced for processing of spectra with standing wave periods less than about 30 MHz (non-HIFI spectra).

Defaults for input parameters fitHifiFringe (nfringes, typical_period, start_period, and end_period) are automatically set to optimal values, matching standing waves in each HIFI band.

PACS

Pipeline

This gives an approximately 20% improvement in processing time over the previous version.

Improvements in the multi-threaded running of tasks on slices.

This can now be turned on/off without needing to restart HIPE.

Stability improved: all tasks have been checked for thread safety.

Improved memory handling.

Photometry

Error estimate for Level 2.5 MADmap maps. MADmap maps now include an error estimate per map pixel. The error is computed as for the photProject pipeline, but is then modified to reflect the same relative sensitivity between the High-pass filter branch and extended emission branch as determined for HSpot. For the naivemap error is the formal error of the mean. This is unchanged. The MADmap error is formal and likely optimistic than reality in that user diligence in preprocessing PACS pipelines and subsequent execution of MADmap pipeline with proper parameters is not accounted for. These could introduce systematical errors in the maps that are significantly larger than formal errors. We will try to account for such systematics in subsequent versions of the error analysis code for HIPE 12.

Masked out matrix 9 after OD 1375.

Level 3 processing.

Deglitching improvements.

Many improvements to scanamorphos.

Better treatment of removal of calibration blocks.

Improved noise computations.

Improved memory handling.

Spectroscopy

Simplified fitting cubes and support projecting fitted parameters by using the new (Pacs)ParameterCube.

Introduced a new script for fitting and then projecting.

Tasks specRespCalToTelescope and specRespCalToBackground are now merged.

Improved extracting spectra from a cube.

Improved the range scan flatfielding, including better diagnostic plots and avoiding noise with new RSRF.

Flatfielding has been added for line scans only to the standard product generation (SPG) pipeline.

The level-2 product will now contain flatfielded PACS cubes.

Improvements to the script to combine PACS and SPIRE spectra.

Improved the pseudo WCS coordinates of PACS (rebinned) cubes, which contain just the row/column coordinates of their RA and Dec cubes.

Improved the background subtraction schemes in the unchopped pipelines.

Error propagation added to specProject.

Calibration

Photometry

Improved cooler recycling times calibration file.

Spectroscopy

Improved telescope background calibration file.

New calibration files with high resolution beams per spaxel.

Improved telescope emissivity model calibration file.

Relative spectral response updates to cover the edges of the spectral bands

Relative spectral response function for the red leak region (for use in hipe)

Analysis tools

Extension to tree viewers now available for all PACS sliced products. This means that also the outline viewer shows information about the slice content.

Photometry

Spectroscopy

Display shows now properly the special pseudo WCS of PACS rebinned (5x5) cubes.

The PACS extensions of the Spectrum Explorer have been improved:

Orientation of spaxel selector in Frames panel now represents the real orientation of the cube spaxels.

Excluding NaNs when creating the average image for the cube selection panel.

Fixed some problems when multiple products are loaded in the SE.

Product changes

An observation summary has been added to the browse product of an observation context (currently only available for the spectrometer).

Amount of details in the product history reduced: E.g., it no longer goes back to where a calibration file has been read by its creator.

Meta data sliceNumber added, which is used to add a slice number to the FITS filenames.

Documentation

Improved use cases and options description for interactive pipeline scripts.

Improved explanation of wavelength grid parameters. * Flatfielding has been added for line scans only to the standard product generation (SPG) pipeline.

New section on spectrometer drizzling.

Better documentation on how to fit cubes and then project them.

Many improvements to the data reduction guides.

SPIRE

Calibration Products

Photometer

Update to flux calibration to use new Neptune model (the ESA4 model instead of the ESA2 model).

The temperature drift correction and extended gains calibration products are also updated to be consistent with the new flux conversion.

Spectrometer

The telescope and instrument RSRFs have been updated to have much higher signal-to-noise. This is due to a new method that makes it possible to increase the number of observations used to derive them. The consequence of this is that long observations have up to 20% lower noise level.

The beam FWHM inside the SCalSpecBeamParam product has been updated to be consistent with the 2D beam profiles contained in the SCalSpecBeamProf product, and to contain numbers taken from the beam profile paper (Makiwa et al., Applied Optics, in press 2013).

The telescope model correction was improved by including recent days in the derivation of the calibration product - this improves the continuum shape of recent observations of faint targets.

Calibration products were updated to allow bright mode observations to be processed in the same pipeline as nominal mode observations:

All bright mode specific editions of the RSRFs and beam parameter file were removed - both bright and nominal pipelines now use the same products.

The channel time constant product was updated to allow bright mode specific time constants to be stored - this functionality is not yet used, but is ready for a future HIPE version.

Calibration Framework

The way that jiggled RSRF files are selected for spectrometer mapping observations was improved. This may affect (improve) maps made before OD1011.

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

New definition of Level 2.5 and Level 3 products. Level 2.5 is now much more homogenised with PACS and contains single pairs of nominal and orthogonal SPIRE destriped parallel maps. The new Level 3 products are similar to the old Level 2.5 maps, but with different units (MJy/ster rather than Jy/beam). They contain mosaics of overlapping regions from a given program using the absolute calibrated maps from the Planck zero point.

Spectrometer Pipeline

Level-1 spectra are now averaged over scans by the pipeline (although level-1 interferograms remain unaveraged). This reduces the size of the observation context. The mapping algorithm has been updated to correctly propagate the errors when averaged scans are used. The individual scans can still be examined by running the pipeline script (either from the beginning or starting with the interferograms).

Point source calibrated data are now included in the Level-2 product for unvignetted detectors - previously only the central detectors were included.

Bright mode observations are now processed using the same pipeline as nominal mode. This leads to higher accuracy in bright mode observations (although the overall change from HIPE v10 is <10%).

The way that the WCS is calculated for mapping observations was optimised (in SpireWcsCreator) - this means that the grid used for mapping observations is different to previous versions (although pixel sizes remain unchanged). The exact pixel position of NaN values in the final Level-2 cube does change due to the shift in the grid compared to the observed sky positions.

The temperature drift correction task was removed from the pipeline - this task made no difference to the output spectra.

Spectral cubes now have the correct final resolution in their metadata

Second level deglitching parameters were optimised for observations with greater than 30 repetitions

Interactive Analysis and Tools

Spectrometer

Spectrometer Level1 Mask Editor is available.

A new semiExtendedCorrector task is available for correcting point source spectra for slightly-extended sources

A new "convolutionProjection" mapper is available for spectrometer data. The map maker used by the standard pipeline is still the Naive Map maker. The new convolution mapper is very similar to the HIFI gridding projection (distributing the signal from each detector across multiple pixels using a kernel), but allows a frequency dependent kernel.

Update to the SDS/SDI Explorer - now a plot is launched with a single mouse click on a detector in the footprint. A second click removes the spectrum from the plot.

Minor updates/corrections to useful scripts

spireProduct2SimpleSpectrum is now more flexible, and can combine two spectra into one SimpleSpectrum

A new task to extract point sources from mapping observations has been added (specPointSourceExtractor).

Photometer

convolutionMapper has a new weightKernel parameter, so that the signal from the bolometers can be distributed across multiple pixels when making a map