CDM Remote Web Service (Experimental)

CDM Remote is a web service implemented in the CDM library (client) and TDS (server), providing remote access to CDM datasets, using
ncstream as the on-the-wire protocol. It provides access at the NetcdfFile and FeatureDataset levels of the CDM API, so there are
two levels of services:

CdmRemoteprovides index subsetting on remote CDM datasets, with the same functionality that NetcdfFile
provides for CDM local datasets and DODSNetcdfFile provides for remote OPeNDAP datasets. CdmRemote supports the
full CDM data model.

CdmrFeature provides coordinate subsetting on remote CDM Feature Datasets, with similar functionality to WCS and Unidata's
experimental NetCDF Subset Service (NCSS). Currently (version 4.2) it provides a remote API for Point and StationTimeSeries
Feature Datasets.

The CDM Remote services and protocol are experimental and should not be used outside of the CDM stack for now.

CdmRemote Protocol (Data Access and Coordinate Systems)

The client forms requests of the form endpoint?query. The possible query parameters are:

Capabilities response indicates which (if any) feature types the dataset can be opened as, along with the URLs of those services. TBD.

The protobuf messages are defined by thredds\cdm\src\main\java\ucar\nc2\stream\ncStream.proto.

Client implementation

ucar.nc2.stream.CdmRemote is a subclass of NetcdfFile which provides index subsetting on remote CDM datasets. NetcdfDataset.openOrAcquireFile()
looks for cdmremote:url prefix, and calls newCdmRemote(url) if found. The url must be an endpoint
for a cdmremote service.

upon opening, req=header is called and the NetcdfFile objects are read from the response

when data is requested, and the data is not already stored (immediate mode). then req=data is called

data requests return an unordered list of observation as StructureData

The protobuf messages are defined by thredds\cdm\src\main\java\ucar\nc2\ft\point\remote\pointStream.proto

Questions:

stationList ncstream vs XML message

Client implementation

ucar.nc2.stream.CdmRemoteFeatureDataset.factory() makes an HTTP request to endpoint+"?req=capabilities"
to obtain the GetCapabilities XML document. This document describes what feature type(s) the dataset supports, which determines what are the valid requests,
along with the endpoints for those feature types, which may be different than the original endpoint.

FeatureDatasetFactoryManager.open() looks for cdmremote:url prefix, and calls CdmRemoteFeatureDataset.factory(url)
if found. ThreddsDataFactory.openFeatureDataset() looks for catalog InvDatasets with
featureType.isPointFeatureType() and ServiceType.CdmRemote, and if found, calls CdmRemoteFeatureDataset.factory().

2) Temporarily, we have the configuration of the datasets in thredds\tds\src\main\webapp\WEB-INF\cdmRemote-servlet.xml: which configures the
collectionController. We will replacce this with pure catalog configuration in 4.2+. This is now being shown in serverStartup.log

CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060325_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060326_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060327_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060328_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060329_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060330_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060331_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060401_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060402_0000.nc
CompositeStationFeatureIterator open datasetQ:\station\ldm/metar/Surface_METAR_20060629_0000.nc
sent 481 features to 20V
(check to see what the strategy is for stations: coomplete scan or ??)

5) cdmremoteCatalog.xml defines datasets for the collections we have working so far.

This document is maintained by John Caron and was last
updated April 2015