The new geostationary satellites, G-16 and Himawari-8 carry high-resolution advanced baseline imagers, ABI and AHI. The ABI onboard G-16 provides imagery in two narrow visible bands (red, blue), while ABI’s twin sensor AHI onboard Himawari-8 also has a green band, which allows the direct production of true color (RGB) images for AHI. Since natural color images are easier for both meteorologists and the public to interpret, it is important to provide true color imagery from geostationary orbit.
In this paper we present a method to estimate green band for ABI from available visible and near-IR bands by building a statistical predictor trained on AHI data. Simple approaches such as look-up-table or simple linear regression on the multi-spectral input parameters may produce satisfactory results globally, but will fail to correctly estimate green band in some cases due to the underlying non-linearity of the data.
We will present an approach which uses piecewise multi-linear regression on the multi-spectral input to train the green channel predictor. Our predictor is built from the combination of a classifier followed by a multi-linear function. Based on the values from the ABI bands, the classifier assigns each pixel to a class. Each class as an associated set of coefficients determining a multi-linear predictor mapping the ABI multi-spectral values to a predicted green value. This combination of a categorical classifier with per-class multi-linear function combines the efficiency of linear map while still preserving flexibility and accuracy by adjusting the number of classes.

Following the launch of the Suomi National Polar-orbiting Partnership satellite in October 2011 with the Visible Infrared Imager Radiometer Suite (VIIRS) sensor onboard, National Oceanic and Atmospheric Administration (NOAA) started generating a global level 2 preprocessed (L2P) sea surface temperature (SST) product. The NOAA Advanced Clear-Sky Processor for Ocean (ACSPO) L2P data are organized into 144 10-min granules per day, with a total volume of ∼27 GB. The L2P product has been successfully assimilated in several level 4 (L4) analyses. At the same time, some other users requested a gridded level 3 (L3) product with a reduced data volume. An L3U “uncollated” product (in which multiple passes over the same grid are independently saved) was produced by mapping the L2P product into equal 0.02° grids. Similar to the L2P, the L3U data are also reported in 10-min granules, but with a daily volume <1 GB. Currently, the NOAA VIIRS L3U SST product is operationally used or tested in several major international numerical weather prediction centers. The L3U shows comparable performance with L2P, suggesting that both products can be used interchangeably as input into L4 analyses. The original L2P pixel-level swath data continue to be produced and available to interested users from NOAA (NCEI) and JPL (Physical Oceanography) data archives.

The ultimate objective of this work is to improve characterization of the ice cover distribution in the polar areas, to improve sea ice mapping and to develop a new automated real-time high spatial resolution multi-sensor ice extent and ice edge product for use in operational applications. Despite a large number of currently available automated satellite-based sea ice extent datasets, analysts at the National Ice Center tend to rely on original satellite imagery (provided by satellite optical, passive microwave and active microwave sensors) mainly because the automated products derived from satellite optical data have gaps in the area coverage due to clouds and darkness, passive microwave products have poor spatial resolution, automated ice identifications based on radar data are not quite reliable due to a considerable difficulty in discriminating between the ice cover and rough ice-free ocean surface due to winds. We have developed a multisensor algorithm that first extracts maximum information on the sea ice cover from imaging instruments VIIRS and MODIS, including regions covered by thin, semitransparent clouds, then supplements the output by the microwave measurements and finally aggregates the results into a cloud gap free daily product. This ability to identify ice cover underneath thin clouds, which is usually masked out by traditional cloud detection algorithms, allows for expansion of the effective coverage of the sea ice maps and thus more accurate and detailed delineation of the ice edge. We have also developed a web-based monitoring system that allows comparison of our daily ice extent product with the several other independent operational daily products.

Discriminating clear-ocean from cloud in the thermal IR imagery is challenging, especially at night. Thresholds in automated cloud detection algorithms are often set conservatively leading to underestimation of the Sea Surface Temperature (SST) domain. Yet an expert user can visually distinguish the cloud patterns from SST. In this study, available pattern recognition methodologies are discussed and an automated algorithm formulated. Analyses are performed with the SSTs retrieved from the VIIRS sensor onboard S-NPP using the NOAA ACSPO system. Based on the analyses of global data, we have identified low-level spectral and spatial features potentially useful for discriminating cloud from clear-ocean. The algorithm attempts to mimic the visual perception by a human operator such as gradient information, spatial connectivity, and high/low frequency discrimination. It first identifies contiguous areas with similar features, and then makes decision based on the statistics of the whole region, rather than on a per pixel basis. Our initial objective was to automatically identify clear sky regions misclassified by ACSPO as cloud, and improve coverage of dynamic areas of the ocean and coastal zones.

We describe an algorithm for creating a virtual, statistically estimated 13.3-μm band for the Visible Infrared Imaging Radiometer Suite (VIIRS), an instrument aboard the National Oceanic and Atmospheric Administration’s (NOAA’s) operational satellite, Suomi NPP. VIIRS does not have a 13.3-μm band, although this band has important applications such as estimating cloud-top pressure. We demonstrate that a reliable estimate of the missing data can be created with a multisensor approach, using other VIIRS bands at 4, 9, 11, and 12 μm, as well as input from the Cross-track Infrared Sounder, on board the same satellite, which produces data at a much finer spectral resolution but lower spatial resolution. In addition, we evaluate the algorithm by applying it to data from the Moderate Resolution Image Spectroradiometer (MODIS) and the Atmospheric Infrared Sounder (AIRS), both on the Aqua satellite. MODIS and AIRS provide a benchmark for measuring the accuracy of the algorithm since, unlike VIIRS, MODIS makes measurements in the 13.3-μm band.

The ABI on GOES-R will provide imagery in two narrow visible bands (red, blue), which is not sufficient to
directly produce color (RGB) images. In this paper we present a method to estimate green band from a simulated
ABI multi-spectral image. To address this problem we propose to use statistical learning to train and update
functions that estimate the value for the 550 nm green channel using the values that will be present in other
bands of the ABI as input parameters. One challenge is that in order to exploit as many bands as possible,
we cannot use straightforward non-parametric methods such as a look-up tables because the number of entries
in look-up tables grows exponentially with the number of input parameters. Other simple approaches such as
simple linear regression on the multi-spectral input parameters will not produce satisfactory results due to the
underlying non-linearity of the data. For instance, the relationship among different spectra for cloud footprints
will be radically different from that of a desert surface. The approach we propose is to use piecewise multi-linear
regression on the multi-spectral input to train the green channel predictor. Our predictor is built from the
combination of a classifier followed by a multi-linear function. The classifier assigns each pixel to a class based
on the array of values from the simulated (or proxy) ABI bands at that pixel. To each class is associated a set
of coefficients for a multi-linear predictor for 550 nm green channel to be predicted. Thus, the parameters of
the predictor consist of parameters of the classifier, as well as coefficients defining the approximating hyperplane
for each class. To determine these classifiers we will use methods based on K-means clustering, as well as
multi-variable piecewise linear approximation.

The broad goal of GEOSS-V, creating a unified system of systems that encompasses all relevant atmospheric and
environmental remote sensing data, accesses social and economic impact information, and integrates all relevant
analysis and decision-making tools, is a monumental task. This is made more difficult in that as technology
and algorithms change at an ever-increasing pace, the ability to test, prototype, and integrate new technology
is often difficult in large production systems. We have been developing Graphyte, a very flexible lightweight
integration framework which is aimed at augmenting GEOSS-V technology by providing agile development tools
for research and development.

The Advanced Baseline Imager (ABI) on GOES-R will help NOAA's objective of engaging and educating the
public on environmental issues by providing near real-time imagery of the earth-atmosphere system. True color
satellite images are beneficial to the public, as well as to scientists, who use these images as an important
"decision aid" and visualization tool. Unfortunately, ABI only has two visible bands (cyan and red) and does
not directly produce the three bands (blue, green, and red) used to create true color imagery.
We have developed an algorithm that will produce quantitative true color imagery from ABI. Our algorithm
estimates the three tristimulus values of the international standard CIE 1931 XYZ colorspace for each pixel of the
ABI image, and thus is compatible with a wide range of software packages and hardware devices. Our algorithm
is based on a non-linear statistical regression framework that incorporate both classification and local multispectral
regression using training data. We have used training data from the hyper-spectral imager Hyperion.
Our algorithm to produce true color images from the ABI is not specific to ABI and may be applicable to other
satellites which, like the ABI, do not have the ability to directly produce RGB imagery.

Currently, the MODIS instrument on the Aqua satellite has a number of broken detectors resulting in unreliable
data for 1.6 micron band (band 6) measurements. Damaged detectors, transmission errors, and electrical failure
are all vexing but seemingly unavoidable problems leading to line drop and data loss. Standard interpolation can
often provide an acceptable solution if the loss is sparse. Interpolation, however, introduces a-priori assumptions
about the smoothness of the data. When the loss is significant, as it is on MODIS/Aqua, interpolation creates
statistically or physically implausible image values and visible artifacts.
We have previously developed an algorithm to recreate the missing band 6 data from reliable data in the
other 500m bands using a quantitative restoration. Our algorithm uses values in a spectral/spatial neighborhood
of the pixel to be estimated, and proposes a value based on training data from the uncorrupted pixels. In this
paper, we will present extensions of that algorithm that both improve the performance and robustness of the
algorithm. We compare with prior work that just restores band 6 from band 7, and present statistical evidence
that data from bands 3, 4, and 5 are also pertinent. We will demonstrate that the increased accuracy from our
multi-band statistical estimate has significant consequences at the product level. As an example we show that
the restored band 6 has potential benefit to the NASA snow mask for MODIS/Aqua when compared with using
band 7 as a replacement for the damaged band 6.

Even with the most extensive precautions and careful planning, space based imagers will inevitably
experience problems resulting in partial data corruption and possible loss. Such a loss occurs, for
example, when individual image detectors are damaged. For a scanning imager this results in missing
lines in the image. Images with missing lines can wreak havoc since algorithms not typically designed
to handle missing pixels. Currently the metadata stores the locations of missing data, and naive
spatial interpolation is used to fill it in.
Naive interpolation methods can create image artifacts and even statistically or physically implausible
image values. We present a general method, which uses non-linear statistical regression
to estimate the values of the missing data in a principled manner. A statistically based estimate
is desirable because it will preserve the statistical structure of the uncorrupted data and avoid the
artifacts of naive interpolation. It also means that the restored images are suitable as input for
higher-level statistical products.
Previous methods replaced the missing values with those of a single closely related band, by
applying a function or lookup table. We propose to use the redundant information in multiple
bands to restore the lost information. The estimator we present in this paper uses values in a
neighborhood of the pixel to be estimated, and propose a value based on training data from the
uncorrupted pixels. Since we use the spatial variations in other channels, we avoid the blurring
inherent spatial interpolation, which have implicit smoothness priors.

The detection and monitoring of harmful algal blooms using in-situ field measurements is both labor intensive and is
practically limited on achievable temporal and spatial resolutions, since field measurements are typically carried out at a
series of discrete points and at discrete times, with practical limitations on temporal continuity. The planning and
preparation of remedial measures to reduce health risks, etc., requires detection approaches which can effectively cover
larger areas with contiguous spatial resolutions, and at the same time offer a more comprehensive and contemporaneous
snapshot of entire blooms as they occur. This is beyond capabilities of in-situ measurements and it is in this context that
satellite Ocean Color sensors offer potential advantages for bloom detection and monitoring. In this paper we examine
the applications and limitations of an approach we have recently developed for the detection of K. brevis blooms from
satellite Ocean Color Sensors measurements, the Red Band Difference Technique, and compare it to other detection
algorithm approaches, including a new statistical based approach also proposed here. To achieve more uniform standards
of comparisons, the performance of different techniques for detection are applied to the same specific verified blooms
occurring off the West Florida Shelf (WFS) that have been verified by in-situ measurements.

To efficiently use the limited bandwidth available on the downlink from satellite to ground station, imager data
is usually compressed before transmission. Transmission introduces unavoidable errors, which are only partially
removed by forward error correction and packetization. In the case of the commonly used CCSD Rice-based
compression, it results in a contiguous sequence of dummy values along scan lines in a band of the imager data.
We have developed a method capable of using the image statistics to provide a principled estimate of the missing
data. Our method outperforms interpolation yet can be performed fast enough to provide uninterrupted data
flow. The estimation of the lost data provides significant value to end users who may use only part of the data,
may not have statistical tools, or lack the expertise to mitigate the impact of the lost data. Since the locations of
the lost data will be clearly marked as meta-data in the HDF or NetCDF header, experts who prefer to handle
error mitigation themselves will be free to use or ignore our estimates as they see fit.

Data from satellites and model simulations is increasing exponentially as observations and model computing
power improve rapidly. Not only is technology producing more data, but it often comes from sources all over the
world. Researchers and scientists who must collaborate are also located globally. This work presents a software
design and technologies which will make it possible for groups of researchers to explore large data sets visually
together without the need to download these data sets locally. The design will also make it possible to exploit
high performance computing remotely and transparently to analyze and explore large data sets.
Computer power, high quality sensing, and data storage capacity have improved at a rate that outstrips
our ability to develop software applications that exploit these resources. It is impractical for NOAA scientists
to download all of the satellite and model data that may be relevant to a given problem and the computing
environments available to a given researcher range from supercomputers to only a web browser.
The size and volume of satellite and model data are increasing exponentially. There are at least 50 multisensor
satellite platforms collecting Earth science data. On the ground and in the sea there are sensor networks,
as well as networks of ground based radar stations, producing a rich real-time stream of data. This new wealth of
data would have limited use were it not for the arrival of large-scale high-performance computation provided by
parallel computers, clusters, grids, and clouds. With these computational resources and vast archives available, it
is now possible to analyze subtle relationships which are global, multi-modal and cut across many data sources.
Researchers, educators, and even the general public, need tools to access, discover, and use vast data center
archives and high performance computing through a simple yet flexible interface.

This paper reports a comparative study of current lossless compression algorithms for data from a representative
selection of satellite based earth science multispectral imagers. The study includes the performance of compression
algorithms on Advanced Very High Resolution Radiometer(AVHRR), SEVIRI, the Moderate Resolution
Imaging Spectroradiometer(MODIS) imager, as well as a subset of MODIS bands as a proxy for the upcoming
GOES-R series. SEVIRI aboard the ESA/EUMETSAT operated Meteosat Second Generation (MSG) satellites
is a geostationary imager. The AVHRR aboard the NOAA Polar Orbiting Environmental Satellites and MODIS
aboard the NASA Terra and Aqua satellites have polar orbits. Thus this study will present representatives
from both polar and geostationary orbiting imagers. The imagers we include have sensors for both reflected
and emissive radiance. We also note that the older satellites have coarser quantizations and present our conclusions
on the impact on compression ratios. Faced with a enormous growing large volume of data on a new
emerging current generation images from faster scanning, finer spatial resolution, and greater spectral resolution,
this study provides a comparison of current compression algorithms as a baseline for future work. With
growing satellite Earth science multispectral imager volume data, it becomes increasingly important to evaluate
which compression algorithms are most appropriate for data management in transmission and archiving. This
comparative compression study uses a wide range standard implementations of the leading lossless compression
algorithms. Examples include image compression algorithms such as PNG and JPEG2000, and widely-used file
compression formats such as BZIP2 and 7z. This study includes a comparison with the Consultative Committee
for Space Data Systems (CCSDS) recommended Szip software which uses the extended-Rice lossless compression
algorithm as well as the most recent recommended compression standard which relies on a wavelet transform
followed by an entropy coder. To establish statistical significance of our analysis, we have developed a system to
acquire and manage a large number of imager granules: currently over 1000 MODIS granules, over 2400 AVHRR
granules, and over 220 SEVIRI granules.

Multispectral imaging is becoming an increasingly important tool for monitoring the earth and its environment
from space borne and airborne platforms. Multispectral imaging data consists of visible and IR measurements
from a scene across space and spectrum. Growing data rates resulting from faster scanning and finer spatial and
spectral resolution makes compression an increasingly critical tool to reduce data volume for transmission and
archiving. Research for NOAA NESDIS has been directed to finding for the characteristics of satellite atmospheric
Earth science Imager sensor data what level of Lossless compression ratio can be obtained as well as appropriate
types of mathematics and approaches that can lead to approaching this data's entropy level. Conventional
lossless do not achieve the theoretical limits for lossless compression on imager data as estimated from the
Shannon entropy. In a previous paper, the authors introduce a lossless compression algorithm developed for
MODIS as a proxy for future NOAA-NESDIS satellite based Earth science multispectral imagers such as GOES-R.
The algorithm is based on capturing spectral correlations using spectral prediction, and spatial correlations
with a linear transform encoder. In decompression, the algorithm uses a statistically computed look up table to
iteratively predict each channel from a channel decompressed in the previous iteration. In this paper we present
a new approach which fundamentally differs from our prior work. In this new approach, instead of having a
single predictor for each pair of bands we introduce a piecewise spatially varying predictor which significantly
improves the compression results. Our new algorithm also now optimizes the sequence of channels we use for
prediction. Our results are evaluated by comparison with a state of the art wavelet based image compression
scheme, Jpeg2000. We present results on the 14 channel subset of the MODIS imager, which serves as a proxy
for the GOES-R imager. We will also show results of the algorithm for on NOAA AVHRR data and data from
SEVIRI. The algorithm is designed to be adapted to the wide range of multispectral imagers and should facilitate
distribution of data throughout globally. This compression research is managed by Roger Heymann, PE of OSD
NOAA NESDIS Engineering, in collaboration with the NOAA NESDIS STAR Research Office through Mitch
Goldberg, Tim Schmit, Walter Wolf.

Multispectral, hyperspectral and ultraspectral imagers and sounders are increasingly important for atmospheric
science and weather forecasting. The recent advent of multipsectral and hyperspectral sensors measuring radiances
in the emissive IR are providing valuable new information. This is due to the presence of spectral channels
(in some cases micro-channels) which are carefully positioned in and out of absorption lines of CO2, ozone, and
water vapor. These spectral bands are used for measuring surface/cloud temperature, atmospheric temperature,
Cirrus clouds water vapor, cloud properties/ozone, and cloud top altidude etc.
The complexity of the spectral structure wherein the emissive bands have been selected presents challenges
for lossless data compression; these are qualitatively different than the challenges offered by the reflective bands.
For a hyperspectral sounder such as AIRS, the large number of channels is the principal contributor to data size.
We have shown that methods combining clustering and linear models in the spectral channels can be effective
for lossless data compression. However, when the number of emissive channels is relatively small compared to
the spatial resolution, such as with the 17 emissive channels of MODIS, such techniques are not effective. In
previous work the CCNY-NOAA compression group has reported an algorithm which addresses this case by
sequential prediction of the spatial image. While that algorithm demonstrated an improved compression ratio
over pure JPEG2000 compression, it underperformed optimal compression ratios estimated from entropy. In
order to effectively exploit the redundant information in a progressive prediction scheme we must, determine a
sequence of bands in which each band has sufficient mutual information with the next band, so that it predicts
it well.
We will provide a covariance and mutual information based analysis of the pairwise dependence between
the bands and compare this with the qualitative expected dependence suggested by a physical analysis. This
compression research is managed by Roger Heymann, PE of OSD NOAA NESDIS Engineering, in collaboration
with the NOAA NESDIS STAR Research Office through Mitch Goldberg, Tim Schmit, Walter Wolf.

Despite tremendous efforts to avoid them, stripes are a re-occurring problem for many remote imaging sensors.
Much work has focused on suppressing or eliminating them in order to recover accurate observed radiances.
Beyond the obvious need to eliminate stripes to obtain accurate scientific measurements, stripes can also significantly
impact the performance of compression algorithms. Many compression algorithms are based on linear
representations of image space or assume the data to be relatively smooth. In contrast stripes produce nonlinearities
in the data as well as sharp discontinuities which make it seem necessary to describe the images with
many parameters. Yet the sources and nature of the stripes are often not well known, they could come from
specific irregularities with the sensors. If the a priori construction of the sensor is accounted for, and the stripe
statistically modeled, it is possible to transmit the stripe parameters separately along with de-striped images.
The de-striped images have image statistics whose assumptions are much closer to those for which standard
compression algorithms are optimized. As an example, we show this yields a significant boost in the performance
of these algorithms when applied to the de-striped MODIS images.

This paper reports a comparative study of lossless compression algorithms for MODIS data. MODIS, The
Moderate Resolution Imaging Spectroradiometer, is a 36 band Visible and IR multispectral imager aboard the
Terra and Aqua satellites, having spatial resolution ranging from 0.250 to 1 kilometer and spectral resolution
ranging from 0.405 -0.420 to 4.482-4.549 microns. MODIS data rates are 10.6 Mbps (peak daytime); and 6.1
Mbps (orbital average). Faced with such an enormous volume of data on a current generation imager, this study
provides a comparison of current compression algorithms as a baseline for future work. The Hierarchical Data
Format (HDF) is standard format selected for data archiving and distribution within the Earth Observing System
Data and Information System (EOSDIS). Currently this system handles over one terabyte of data daily, and this
volume continues to increase over time. With growing satellite Earth science multispectral imager volume data
compression, it becomes increasingly important to evaluate which compression algorithms are most appropriate
for data management in transmission and archiving. This comparative compression study uses a wide range
standard implementations of the leading lossless compression algorithms. Examples include image compression
algorithms such as PNG and JPEG2000, and widely-used file compression formats such as BZIP2 and 7z. This
study includes a comparison with the Consultative Committee for Space Data Systems (CCSDS) most recent
recommended compression standard. by a significant margin.

Multispectral imaging is becoming an increasingly important tool for monitoring the earth and its environment
from space borne and airborne platforms. Multispectral imaging data consists of visible and IR measurements
from a scene across space and spectrum. Growing data rates resulting from faster scanning and finer spatial
and spectral resolution makes compression an increasingly critical tool to reduce data volume for transmission
and archiving. Examples of multispectral sensors we consider include the NASA 36 band MODIS imager,
Meteosat 2nd generation 12 band SEVIRI imager, GOES R series 16 band ABI imager, current generation
GOES 5 band imager, and Japan's 5 band MTSAT imager. Conventional lossless compression algorithms are
not able to reach satisfactory compression ratios nor are they near the upper limits for lossless compression
on imager data as estimated from the Shannon entropy. We introduce a new lossless compression algorithm
developed for the NOAA-NESDIS satellite based Earth science multispectral imagers. The algorithm is based
on capturing spectral correlations using spectral prediction, and spatial correlations with a linear transform
encoder. Our results are evaluated by comparison with current sattelite compression algorithms such the new
CCSDS standard compression algorithm, and JPEG2000. The algorithm as presented has been designed to
work with NOAA's scientific data and so is purely lossless but lossy modes can be supported. The compression
algorithm also structures the data in a way that makes it easy to incorporate robust error correction using FEC
coding methods as TPC and LDPC for satellite use. This research was funded by NOAA-NESDIS for its Earth
observing satellite program and NOAA goals.

As new instruments are developed, it is becoming clear that our ability to generate data is rapidly outstripping our ability to transmit this data. The Advanced Baseline Imager (ABI), that is currently being developed as the future imager on the Geostationary Environmental Satellite (GOES-R) series, will offer more spectral bands, higher spatial resolution, and faster imaging than the current GOES imager. As a result of the instrument development, enormous amounts of data must be transmitted from the platform to the ground, redistributed globally through band-limited channels, as well as archived. This makes efficient compression critical. According to Shannon's Noiseless Coding Theorem, an a upper bound on the compression ratio can be computed by estimating the entropy of the data. Since the data is essentially a stream, we must determine a partition of the data into samples that capture the important correlations. We use a spatial window partition so that as the window size is increased the estimated entropy stabilizes. As part of our analysis we show that we can estimate the entropy despite the high-dimensionality of the data. We achieve this by using nearest neighbor based estimates. We complement these a posteriori estimates with a priori estimates based on an analysis of sensor noise. Using this noise analysis we propose an upper bound on the compression achievable. We apply our analysis to an ABI proxy in order estimate bounds for compression on the upcoming GOES-R imager.

Errors due to wireless transmission can have an arbitrarily large impact on a compressed file. A single bit error appearing in the compressed file can propagate during a decompression procedure and destroy the entire granule. Such a loss is unacceptable since this data is critical for a range of applications, including weather prediction and emergency response planning. The impact of a bit error in the compressed granule is very sensitive to the error's location in the file. There is a natural hierarchy of compressed data in terms of impact on the final retrieval products. For the considered compression scheme, errors in some parts of the data yield no noticeable degradation in the final products. We formulate a priority scheme for the compressed data and present an error correction approach based on minimizing impact on the retrieval products. Forward error correction codes (e.g., turbo, LDPC) allow the tradeoff between error correction strength and file inflation (bandwidth expansion). We propose segmenting the compressed data based on its priority and applying different-strength FEC codes to different segments. In this paper we demonstrate that this approach can achieve negligible product degradation while maintaining an overall 3-to-1 compression ratio on the final file. We apply this to AIRS sounder data to demonstrate viability for the sounder on the next-generation GOES-R platform.

In this paper, which is an expository account of a lossless
compression techniques that have been developed over the course of a
sequence of papers and talks, we have sought to identify and bring
out the key features of our approach to the efficient compression of
hyperspectral satellite data. In particular we provide the
motivation for using our approach, which combines the advantages of
a clustering with linear modeling. We will also present a number of
visualizations which help clarify why our approach is particularly
effective on this dataset.
At each stage, our algorithm achieves an efficient grouping of the
data points around a relatively small number of lines in a very
large dimensional data space. The parametrization of these lines is
very efficient, which leads to efficient descriptions of data
points. Our method, which we are continuing to refine and tune, has
to date yielded compression ratios that compare favorably with what
is currently achievable by other approaches. A data sample
consisting of an entire day's worth of global AQUA-EOS AIRS Level 1A
counts (mean 12.9 bit-depth) data was used to evaluate the
compression algorithm. The algorithm was able to achieve a lossless
compression ratio on the order of 3.7 to 1.

In this paper, which is part of an ongoing sequence of papers devoted to the subject of efficient noise-tolerant lossless compression of satellite data for transmission, we describe an algorithm for this purpose which effectively addresses the above criteria. Our algorithm exhibit the potential to achieve noise-tolerant compression ratios averaging 3.2 : 1. An earlier approach, which we presented at Third GOES-R User Conference in May of 2004, was the first such method to break the 3 to 1 compression barrier for this class of data.

In this paper, we consider an application of the Empirical Mode Decomposition (EMD) introduced by Norden E. Huang in 1996 to the compression of 3D hyperspectral sounding data. The EMD is a new data analysis method which is based on expansion of the data in terms of Intrinsic Mode Functions (IMF). These IMFs are based on and derived from the data set. Since EMD adaptively represent the signal as a sum of "well behaved" amplitude/frequency modulated components, we found it very well suited for the whitening part of the compression scheme.

My Library

You currently do not have any folders to save your paper to! Create a new folder below.

Keywords/Phrases

Keywords

in

Remove

in

Remove

in

Remove

+ Add another field

Search In:

Proceedings

Volume

Journals +

Volume

Issue

Page

Journal of Applied Remote SensingJournal of Astronomical Telescopes Instruments and SystemsJournal of Biomedical OpticsJournal of Electronic ImagingJournal of Medical ImagingJournal of Micro/Nanolithography, MEMS, and MOEMSJournal of NanophotonicsJournal of Photonics for EnergyNeurophotonicsOptical EngineeringSPIE Reviews