README version 06/02/04
EIS/MVM SOFTWARE RELEASE VERSION 1.0.1; 2004-02-06
This is the second release (version 1.0.1) of the stand-alone EIS/MVM image
reduction system. The present product release should continue to be
considered as a work in progress. Users are encouraged to send comments,
suggestions, verification results and requested to report any errors/features
found with the released software via electronic mail to:
eis-help@eso.org
indicating explictly the software version in use and date of release
(e.g. Version: 1.0.1, date of release: 6 February 2004).
NOTES ON VERSION 1.0.1 (PRESENT) RELEASE WITH RESPECT TO PREVIOUS SOFTWARE
RELEASE (VERSION 1.0; 2003-12-17)
This release completely supersedes the data release of the EIS/MVM
software release 1.0 performed on 17 December 2003. Differences exist
between the two releases. In the present release, compilation errors
and miscellaneous bugs reported by users have been resolved, as
reported in the following log of (technical) changes:
* mvmjitter.cc MvmJitter::get_mvmamer add in xml log the noise of
images node . Futur harmonization in node name should occur.
* mvmamer.cc MvmAmmer object: add option Match_min_match_amer to specify
minimum number of anchor for the astrometric solution. This notion
asserts a minimum absolute number of matches.
* config/reduction-mode.xml update definition for jitter-base
match-search-relative set to 0.5 match-min-match-relative set to 0.1
match-min-match-amer set to 4
* mvmjitter.cc MvmJitter::jitter_norm bug fix in trim_value computation.
The trim-min-image option is now working properly.
* cube.cc cube::set_max_image bug fix with _max_image.
* scr/src_eis/Makefile set CC=g++ instead of gcc for gcc > 3 compatibility
from Thomas Erben.
* add_mosaic.cc add option --cosmic to switch on the cosmic rejection.
* mosaic.cc mosaic.h add option to select sigma-cliping rejection.
* normcalib.cc bug fix in stat computation that make the fluxscale
computation wrong.
* rbspec.cc add option to specify the min number of RB This help for
multi-chip instrument in order to remove residual cosmic rays between
chips.
* config/reduction-mode.xml change definition jitter-base such as the
sigma-cliping is done at 2.5 sigma before stacking 3 or more images.
* mvmjitter.cc triming is done with the context map instead of computed
from the weight map. This give a true triming. Option trim-sensitive
disappeared.
* deflat.cc new option flag-remove-bad-column to detect bad aligned pixels.
The algorithm is based on minimum in gray level with direction x and y.
This algo remove bad columns not observed in bias and flat. Note that the
remaining bad columns could be masked out from dark. But no dark are
observed in optic in general. This algo should more to mvmcalib in the
future. This algo is quite heavy and add 10% of the total time to reduce
an image.
* fringing.cc the detected tracks of the first fringe estimations are now
removed also to the original data. This prevent bad fringing estimation
when a track is present.
* config/reduction-mode.xml finishing clean up of old options. new
definitions common for most Reduction Mode.
* rbspec.c implementation of the new step to automaticly
create RBs from slipted images from their object name.
Since this present EIS/MVM software release incorporates improvements to
completely supersede the first release, to the best of our knowledge the
natural recommendation would be that the most recently available, updated
release be adopted whenever possible. There has been no difference to the
content of the user manual between release 1.0 performed on 17 December 2003
and the present release.
INTRODUCTION
Since June 2002 the EIS (ESO Imaging Survey) and the PSG (Public Survey Group)
teams of DMD (Data Management Division) of ESO have been working on the
development of an end-to-end, integrated, high-throughput software system. The
EIS survey system benefits from the existing ESO data flow system (DFS) and
archive infrastructure and consists of several elements which include:
- a Python wrapper responsible for the administration of the entire system
consisting of over 250,000 lines of code
- a specially designed C++ based image processing system, consisting of over
100,000 lines of code, to reduce optical/infrared images from single- and
multi-chip instruments, referred to hereafter as the EIS/MVM
(Multi-resolution-Vision-Model) system (see BACKGROUND)
- over 100 tailored tables in "Sybase" database
- a WEB service based on the internal database and making extensive use of XML
(extensible mark-up language) and SVG (scalable vector graphics)
technologies
and many other features that will be presented elsewhere.
While developed primarily to support public surveys the EIS survey system can
also be used as a front-end to the science archive as well as to support
individual end-users. Some of the highlights of the system have been presented
by da Costa (2003 a, b), in the Workshop of "Large Programmes and Public
Surveys", in Garching (May 2003) and in demonstrations presented to the Survey
Working Group for Public Surveys.
While both Python and C code are open source, the integrated nature of the
survey system prevents its distribution as a whole. However, a stand-alone
version of the EIS/MVM image reduction system is being made publicly available
in the present distribution.
It is important to note that this system has been extensively tested using
SOFI and ISAAC infrared data as well as optical multi-chip data from high-
and low-galactic latitude fields taken with the wide-field imager (WFI) at
the 2.2m MPG/ESO telescope at La Silla. These tests, carried out by the
EIS/PSG team as well as other users, have demonstrated that the system leads
to cosmetically enhanced images, including the automatic removal of satellite
tracks, robust astrometric calibration and reliable relative photometric
measurements. The system is also remarkably efficient as attested by
benchmarks carried out on different platforms. For instance, the system
reduces one WFI image per minute (or about 1 Mpix/sec) using a 4 CPU INTEL
Pentium 4 machine.
CONTENTS
Herewith please find a "tarball" for the distribution of the EIS/MVM image
reduction software. The tarball contains:
- A draft version of the EIS/MVM Image Pipeline User's Manual (date)
- The source code and the required XML files of the most recent
version of the pipeline
- This README file prepared by the EIS/PSG team
- An INSTALL file describing the installation procedure
BACKGROUND
The EIS/MVM image processing pipeline is a software package developed by B.
Vandame, funded by the EIS project at ESO. The development of the pipeline
started in June 2000 using as the basis a C++ based library known as the
Multi-resolution-Vision-Model (MVM) developed by B. Vandame and others at the
Nice observatory under the supervision of A. Bijoui. The pipeline has been
assembled to address the data coming from ESO optical/infrared imagers. It is
also used as the image processing engine of the end-to-end EIS survey system
mentioned above.
DISCLAIMER
Even though best effort has been made to ensure the accuracy of the package
and manual, both still remain a work in progress. Therefore, we encourage
potential users to send their comments, suggestions and verification results
via email to:
eis-help@eso.org
Current users of the system include: S. Arnouts (Marseille); C. Benoist
(Nice Observatory); C. Mullis (ESO Fellow/AVO); S. Zaggia (Trieste
Observatory). The system is also being evaluated by the DFS/DMD of ESO.
Users of this preliminary version accept the following conditions:
- the code in its present form is the property of ESO and as such follows the
regulations described in the ESO Software Policy document of March 7, 2003
(GEN-POL-ESO-00000-3011 V1.0)
- the transfer of the code will be made using the survey system framework so
as to register the distribution for future reference
- all errors/bugs/suggestions should be reported via email to:
eis-help@eso.org
- all reports should explictly refer to software version in use and date
of release (e.g. Version: 1.0.1, date of release: 6 February 2004).
- the code may change without prior warning. It is the responsibility of the
user to check for updates
- the EIS team cannot guarantee immediate action on problems reported by
external users. The PSG reserves the right to establish priorities
- the EIS team takes no responsibility for the scientific use of the package
in its current form
- the EIS team takes no responsibility for any changes made to the source code
by the users
- communication among users is strongly urged to exchange experiences
COPYRIGHTS
The EIS/MVM is a copyright-protected software product of the European Southern
Observatory for image processing. It is available under the GNU General Public
License as published by the Free Software Foundation.
Users of this package are kindly requested to acknowledge the European
Southern Observatory and the EIS Project for the use of the software in any
publication or presentation of results prepared using this package. They are
also requested to reference Vandame et al. (2003, in preparation).
REFERENCES
da Costa, L., 2003, ``Making Ground-based optical/infrared imaging surveys
VO-compatible'', "Toward an International Virtual Observatory - Scientific
Motivation, Roadmap for Development and Current Status,
ESO Conference Series, in press
da Costa, L., 2003, Astronomical Data Analysis Software & Systems XIII
October 12-15, 2003
Vandame, B. 2002, "New algorithms and technologies for the un-supervised
reduction of Optical/IR images"
Proceedings of the SPIE, Volume 4847, pp. 123-134 (2002)