Breadcrumb trail

Limitations

For obvious reasons, we have had to put some constraints on the size of the result sets. If your query exceeds these limits, you will get an email message that your query returned too many rows and didn't execute. The solution is to break your query up into smaller sub-areas and try again. Seen notes below to take some of the guesswork out of this process.

The restrictions are:

HYDROGRAPHIC

Complete Profile

3 Million Records

SEA-SURFACE TEMPERATURE

Individual Observations

3 Million Records

OCEAN COLOUR DATABASE

Raw Data

4 million

Statistics

10 Million

either case, maximum of eight (8) polygons

OCEAN DATA INVENTORY and COASTAL TIME SERIES

No limits

Knowing whether your query will be too large can be guesswork at best. Here are a few hints.

Hydrographic: At present, our highest sampled area is on the Scotian Shelf in the vicinity of LaHave, Brown's and Georges Bank. A query area of roughly 8 or 9 degrees square (4x2 or 3x3) is the largest area where you can expect to stay within the limits. Using the TSonly or BinAvg options (see the section on the Climate Application) will reduce the number of individual observations returned, thus permitting a corresponding larger query area.

Sea Surface Temperature: This is a bit more straight forward. A fully populated 9 km grid will yield about 6400 records (observations) for each 1 degree square/year. With 20 years of data, you should be able to extract point data from an area of at least 25 square degrees. There should not be any problem running amok of the constraints if you are requesting statistics.

Ocean Colour Database: The Ocean Colour Database holds two datasets. The first has approximately 180 million records at 1.5 km, semi-monthly resolution, from 39N to 62.5N, 42W to 71W and from 16 September 1997 to 31 December 2004. The second has approximately 50 million records (to April 2008, and growing approx. 6 million per year) at 4 km, weekly resolution, from 39N to 62.5N, 42W to 71W and begins on 16 September 1997 and is updated periodically. Obviously these data cannot be retrieved in a single query, or even in a number of medium sized queries. The challenge is to design requests (both from a temporal and spatial standpoint) that target your area of interest that will return only the information needed. Sometimes it will be necessary to do a number of small queries that fall within the limitations specified in the table above. Because of the way these data were collected and processed, the data density is not completely uniform over the whole area (because of ice cover, satellite sensor variability, etc.). A close approximation of the data density is 5000 records per square degree per month. This should allow you to estimate the size of your query accurately enough so that the results are below the limitations. As with SST, if you are requesting statistics as opposed to raw data, the chances of exceeding the limits are greatly reduced.

Results sets will only be kept on the server for 5 days at which point they will be removed. Previous queries can always be re-run if you were unable to ftp them to your site.