Missing data in HelioClim-3

Last update: March 2017

The missing value is -999. When you retrieve HelioClim-3 time series, you receive missing values ONLY IF THE WHOLE DAY OF DATA IS MISSING. If at least one Meteosat image has been collected in a given day, all the 15 minute values are interpolated.

Two reasons cause a full missing day:

either an Eumetsat service failure: maintenance on the satellite, outage in the slot transmission...

or the sun elevation is below 12°, which is our threshold below which the HelioClim-3 data are not computed.

In any case, pay attention to discard the -999 values in the SoDa output CSV file retrieved via one of the HelioClim-3 services. If you forget, summing up the columns in your file will lead to incorrectly low values.

List of missing days (all Meteosat disk)

Year

days

2004

4-5-6 Oct.

2005

18 Apr., 24 to 29 Aug., 26 Nov.

2006

11 Sept.

List of missing days due to sun elevation threshold (below 12°)

More than 100 days with missing values (-999) in the high latitudes.

List of missed slots (but interpolated in HelioClim-3)

The reasons for missing a slot are numerous: local electrical or network failure, problem on the satellite or at Eumetsat during the slot transfer... In this last case, Eumetsat emits a warning issue, and the images can not be retrieved by any mean (MS = Missing Slot).

yes/no: in line/not in line with Eumetsat warnings. Numbers between brackets correspond to the announcement number indicated in the list of satellite events. You can use this link and type the announcement number in the research menu to take a look to the corresponding event.

NB: in the past, we made the distinction between a Missing Slot, and a slot received but considered as corrupted by our image quality routines, which resulted in Not Updating (NU) the HelioClim-3 database. This distinction has not been maintained.

Month (total number of slots in the month)

slot (yyyy-mm-dd-HH-MM)

Event and comment (Eumetsat announcement number)

September

2017-09-22-11-30 until 2017-09-22-12-15

yes. Due to a ground segment anomaly (3209).

2017-09-09-12-30 until 2017-09-09-13-15

Due to heavy rain falls.

August 2017

2017-08-31-18-30 and 2017-08-31-18-45

no

2017-08-28-13-15

no

2017-08-04-18-15

no

July 2017

2017-07-17-14-15 and 2017-07-17-17-15

yes. Due to a ground segment anomaly data dissemination could be delayed by several minutes (3911).

2017-07-16-12-00

no

June 2017

2017-06-22-11-45

no

2017-06-01-12-30

yes. Essential Ground Station Maintenance. There is a risk for short interruptions or degradation during the maintenance period (2906).

May 2017

2017-05-01-04-00

no

April 2017

2017-04-25-09-30 until 2017-04-25-13-00

yes. Intermittent losses can be expected until further notice. Investigation is ongoing (2824).

2017-04-04-09-30

yes. Planned maintenance. Provider maintenance: Up to 20 minutes of interruption of the service and risk of further interruptions during the maintenance period. The data delivery might be delayed and losses are expected. Receiver with low Link Margin might experience more losses during the whole maintenance period (2774).

March 2017

2017-03-28-12-15

no. Several planned maintenance planned.

2017-03-22-09-00

no

2017-03-21-10-15

yes. Planned maintenance. 2 short interruptions (of 2 seconds) are expected, leading to some losses (2729).

yes. Due to a Eumetcast problem that is currently under investigation, users are advised that data on all services may be incomplete (2517).

2017-01-11-10-45

yes. Data-unavailable. Due to a Eumetcast problem that is currently under investigation, users are advised that data on all services may be incomplete. This may occur intermittently (2517).

2017-01-10-15-45

2017-01-09-14-30

no

2017-01-07-11-45

no

October 2016

2016-10-20-09-30

no

2016-10-15-12-30 until 2016-10-15-14-15

yes. Service interrupted. MET-9 operates in Full Earth Scan mode for service continuity and MET-9 data is disseminated instead (2290 and 2291).

2016-10-14-07-45, 2016-10-14-08-15 and 2016-10-14-08-30

yes. Sun interference may affect the EUMETCast broadcast satellites. Users might experience lower signal margin or outages on EUMETCast. The exact times and durations of the signal degradation or outages depend on the reception location (2205 and 2231).

We were using the OpenLayers javascript files 2.12, and were downloaded using a URL on the OpenLayers server that has been discarded. This impacted all the web services which uses maps.

2015-12-12-12-00 to 18-00

Service down at MINES ParisTech because of a electric shortage.

2015-12-08-14-45, 15-30 and 16-00

yes. Meteosat-10 SEVIRI decontamination. During this period, Meteosat-8 image data (rectified to 0 deg) and meteorological products will replace the Meteosat-10 service. An outage of up to two repeat cycles can be expected when switching to and from Meteosat-8. For full details, see ​http://www.eumetsat.int/website/home/News/DAT_2842851.html Rectification by Eumetsat to correct the position of Met-8. Consideration of new slot by HC3 infected.

Nov. 2015

2015-11-16-08-30 and 08-45

yes. The 08.30 and 08.45 Repeat Cycles were interrupted or not available. Service has been resumed since 09.00 (2451).

Severe thunderstorm event on the whole French Riviera. Electricity failure at MINES ParisTech, service is back on Oct. 5th at approx. 15:00. Water in the server room at Transvalor, but no service failure has been reported.

2015-10-01 to 2015-10-02

A bug has been introduced in the update of the McClear clearsky model. Affected services were McClear, MACC6RAD and HelioClim-3v5. The services are back on the next day.

September 2015

2015-09-13-03-00, 03-30, 03-45, 04-45, 05-00, 05-15 and 06-00

Due to severe local storms

August 2015

2015-08-24-15-45 to 16-00

Due to heavy rain falls.

2015-08-12-06-00

2015-08-10-07-15 and 09-15

no

2015-08-03 and 2015-08-04

The HC3 database has not been updated at MINES ParisTech. The problem is being repaired (August 4th, 11:00)

July 2015

2015-07-29-10-30

no, image corrupted

2015-07-13 to 2015-07-15

The SoDa Service was disturbed at MINES ParisTech (www.soda-is.com) due to oo numerous automatic requests launched by a customer to the service hc3v4_forecast.

2015-07-10-03-00

no

2015-07-07-14-15

yes

2015-07-02-11-15

yes

June 2015

2015-06-24-09-30

yes

2015-06-14-05-15, 05-30, 05-45 and 12-00

no, certainly due to severe local storms

2015-06-09-04-30

yes

May 2015

2015-05-10-19-30

yes

Apr. 2015

2015-04-28-07-45 to 08-30 and 12-00 to 12-30

yes

2015-04-22-13-00

yes:

"METEOSAT 0 DEG SERVICE 22 April 2015 09:00 UTC Introduction of MSG MPEF (Meteorological Product Extraction Facility) Release 2.1. Main changes in this release include a new version of RTTOV (Radiative Transfer Model) and improvements to the OCA (Optimal Cloud Analysis) product. For full details and test data (for both EUMETCast and GTS), see: http://www.eumetsat.int/website/home/News/DAT_2602960.html

22 April 2015 13:00 - 13:15 UTC Essential ground segment maintenance. During this maintenance period there is a risk of interruption to the 0-Degree Service.

March 2015

2015-03-18-10-00, 16-00 and 16-30

yes

2015-03-05-11-15

no

2015-03-03-11-15, 12-00, 12-15 and 12-30

yes for the two slots 12-00 and 12-15

2015-03-02-12-00 and 12-15

yes

2015-03-01-12-00 and 12-15

yes

Jan. 2015

2015-01-24-09-45, 10-00, 10-15 and 20-00

MS, yes except for the last one

2015-01-20-09-30, 09-45 and 10-00

MS, yes

2015-01-16-20-45

MS

Dec. 2014

2014-12-27-18-30

MS

2014-12-19-08-00

MS

2014-12-08-06-45 and 07-00

MS, yes

2014-12-06-05-30

MS

2014-12-04-14-30

MS

2014-12-03-05-30

MS

2014-12-02-19-15

MS

Nov. 2014

2014-11-30-19-45

MS

2014-11-25-11-45

MS, yes

2014-11-20-15-15

MS, no

2014-11-04-11-00, 15-45, 16-30 and 18-00

MS. planned ?

Oct. 2014

2014-10-29-07-00

MS, yes

2014-10-13-10-30, 11-00 and 13-00

MS, no

2014-10-12-11-45

MS, yes

2014-10-11-11-45

MS, yes

2014-10-10-11-45

MS, yes

2014-10-09-11-45

MS, yes

Sept. 2014 Total:

2014-09-26-09-15

MS, no

Aug. 2014 Total: 9

2014-08-15-14-15, 14-45, and 15-00

MS, yes

2014-08-13-08-15 and 08-45

MS, ?

2014-08-10-15-00, 15-15, 17-15 and 17-30

MS, ?

Jul. 2014 Total: 6

2014-07-10-17-45

MS, ?

2014-07-02-10-15 to 10-30 and 11-00 to 11-15

MS, ?

2014-07-01-05-15

MS, ?

June 2014 Total: 3

2014-06-27-11-00

MS, ?

2014-06-11-05-15

MS, ?

2014-06-09-04-30

MS, ?

May 2014 Total: 0

2014-05-03 to 2014-05-04

SoDa Service failure at MINES ParisTech: The SoDa server at MINES ParisTech (Magnum) was down since May 3rd, blocking the update of the database and the synchronisation with the mirror Service. The Service was was at MINES ParisTech around 4 PM on May 5th.

Apr. 2014 Total: 1

2014-04-21-11-00

MS, no

Mar. 2014 Total: 8

2014-03-26-03-00 to 03-15

MS, yes

2014-03-04-12-00 and 12-15

MS, no

2014-03-03-12-00 and 12-15

MS, no

2014-03-02-12-00 and 12-15

MS, no

Feb. 2014 (Total: 0)

2014-02-04-10-00 to 16-00

Service down in MINES ParisTech: Migration of the SoDa website from a old to a new server. The SoDa Service was back in the afternoon at 4 PM local time (UT+1). The Customer was informed only the day after that the Service was back to avoid any problem during the update of the database at night.

Jan. 2014 (Total: 3)

2014-01-29-09-00

MS, no

2014-01-23-17-00

MS, no

2014-01-13-04-15

MS, no

Dec. 2013 (Total: 2)

2013-12-12-10-15 and 10-30

MS, yes

Nov. 2013 (Total: 1)

2013-11-26-10-00 to 16-00 legal time (UT+1)

Service down in MINES ParisTech: Migration of the SoDa website from the old to the new network. The SoDa Service was actually back at 16:30 as announced in the emails sent to the Customers.

2013-11-25-08-45

MS, no

Oct. 2013 (Total: 6)

2013-10-14-11-30 and 11-45

MS, no. Please also note that two images of this day were corrupted at MINES ParisTech: 14-30 and 14-45. The problem has been solved in due terms.

2013-10-13-11-30 and 11-45

MS, no

2013-10-12-11-45

MS, no

2013-10-11-18-00 to 22-00

Service down in MINES ParisTech: the service at MINES ParisTech has been interrupted during 4 hours due to a mishandling from the IT team. No major consequence has to be reported, except the loss of four days of monitoring logs (from Oct. 8 to 11 2013)

2013-10-11-11-45

MS, no

Sept. 2013 (Total: 8)

2013-09-29-11-30

MS, no

2013-09-26-14-30

MS, yes

2013-09-21-06-30 to 07-15 (4 slots)

MS, yes

2013-09-14-07-45

MS, yes

2013-09-10-14-00 to 2013-09-11-14-00

DNS down: an operation on the Domain Name Server had the consequence that both servers (www.soda-is.com and pro.soda-is.com) were down. The automatic requests seem to have succeeded during the night. The service was back and fully available on both sites from 14-00.

2013-09-07-04-00

MS, no

Aug. 2013 (Total: 14)

2013-08-18-08-00 to 11-00 (13 slots)

MS, yes

2013-08-14-08-15

MS, yes

July 2013 (Total: 4)

2013-07-29-13-45

MS, yes

2013-07-29-05-15

MS, no

2013-07-10-20-30

MS, no

2013-07-09-09-00

MS, no. Following the text below, in the reverse operation on July 9th 2013, the system has correctly shifted to the adequate satellite, except that the image of 9:00 GMT has been missed in the transfer.

2013-07-01

NB: On July 1st 2013, the operational Meteosat 10 (MSG3) satellite has been replaced by his backup Meteosat 8 (MSG1). This had no effect on the HelioClim3 database.

June 2013 (Total: 4)

2013-06-24-08-45, 09-00, and 09-15

MS, yes

2013-06-13-16-15

MS, yes

2013-06-04: 12:30

Down in MINES ParisTech: the SoDa service (website and wget accesses) weredown since 12:30 local hour (GMT+2). The problem has been solved at 14:15.

Apr. 2013 (Total: 2)

2013-04-23-12-15

MS, yes

2013-04-20-11-45

MS, no

March 2013 (Total: 8)

2013-03-05-04-00, 04-15 and 04-30

MS, yes

2013-03-05-12-15

MS, no

2013-03-04-12-15

MS, no

Feb. 2013 (Total: 2)

2013-02-19-08-30 and 08-45

MS, yes

Jan. 2013 (Total: 0)

Dec. 2012 (total:)

2012-12-24 to 27

Service down in MINES ParisTech: during the Christmas holidays, the machines in MINESParisTech were not accessible anymore from the exterior, even if the images acquisition and update of the HelioClim3 database were still functioning. The service was back on Dec. 27th.

2012-12-12-14-15

MS, yes

Nov. 2012 (total: )

2012-11-25

Pb in MINES ParisTech: the slot of 12:00 was missing in MINES ParisTech. The synchronisation process had to be launch again for Nov. 25th. The service was back and available on both sites on Nov. 26th 2012 around 2 PM local time.

2012-11-12-22-00 to 2012-11-13-16-00

SoDa out of order in MINES ParisTech: since Nov. 12th 2012 10 PM Legal Time, one of the 5 disks of the main SoDa server in MINES ParisTech went down (www.soda-is.com for the SoDa website and www.helioclim.org for the wget requests). During the day of Nov. 13th, the service totally out of order for maintenance reason. The service was back at 04 PM Legal Time, but to copy the images from Transvalor to MINES ParisTech and update the HelioClim3 database, an extra night was required. The service is fully back at Mines ParisTech on Nov. 14th.

Oct. 2012 (total: )

2012-10-26-16-00 and 16-15

MS, no

2012-10-13-11-45

MS, no

2012-10-12-11-45

MS, no

2012-10-11-11-45

MS, no

2012-10-10-11-45

MS, yes

2012-10-04 to 05

SoDa out of order in MINES ParisTech: Between the 4th and the 5th of Oct.: All the webservices were out of service. The origin of the problem was a error of rights in the access of the server to the temp directory where all the output files are created.

Sept. 2012 (total: 7)

2012-09-26-18-00

MS, no

2012-09-25

A Japanese company created a robot to automatically retrieve clear-sky values from a free service. To give you an order of the robot effect, more than 25 000 requests were made in a few hours, making appeared a few bugs in the administration of our Tomcat server, leading to regular stop of Tomcat. The IP from Japanese company has been blocked and bugs have been corrected. A correction is under development to block all robots to access the SoDa website.

2012-09-24-11-00 and 11-15

MS, yes

2012-09-24-09-45 and 10-00

MS, no

2012-09-24

In the morning, the service at MINES ParisTech was out of service during the morning. The origin of the problem is the service TOMCAT of managements of forms, which seemed to stop on the SoDa server. Recurrent problems concerning the different webservices of Mines ParisTech occur in Sept. 2012 due to a global re-organization of the internal network architecture.

2012-09-19

The service at Mines ParisTech was out of service between 04:00 AM to 03:30 PM due to a problem of configuration of the antivirus on the machine where Apache is running.

2012-09-15-06-00

MS, yes

2012-09-11-16-45

MS, yes

August 2012 (total: 6)

2012-08-31-03-00 to 04-00 (5 missing slots)

MS, yes

2012-08-25-19-45

MS, no

July 2012 (total: 2)

2012-07-29-18-30 and 18-45

MS, yes

2012-07-02

A few difficulties were faced due to the overload of a server. A few hours later, the problem has been solved.

June 2012 (total: 5)

2012-06-29-05-00

MS, yes

2012-06-26-06-30 and 06-45

MS, yes

2012-06-19-13-30

MS, yes

2012-06-13-03-45

MS, no

2012-06-03 to 2012-06-04

Apache was down at MINES ParisTech since June 3rd 16:00 until June 4th 10:00 local hour.