GeoMesa Blob Store
==================
The GeoMesa Blob Store is found in ``geomesa-blobstore`` in the source distribution.
The ``GeoMesaIndexedBlobStore`` is an interface that describes the
function of a GeoMesa BlobStore, which is to store and retrieve files
that have spatio-temporal data associated with them. A GeoMesa Blobstore
is created by combining a geotools data store and a backing blob store
which will contain the blobs separated from the geo index. The
``BlobStore`` interface can be used to implement compatibility with
other databases or distributed file systems.
During ingest of a file into a GeoMesa BlobStore, a pluggable system of
``FileHandler``\ s examines the files (and associated maps of
parameters) and creates ``SimpleFeature``\ s which are the written to a
GeoMesa data store while the blobs are persisted using a class that
implements the ``BlobStore`` interface.
File retrieval is a two-step process. First a query is made to the
GeoMesa ``FeatureStore``. The returned ``SimpleFeature``\ s will contain
a URI field. That URI points to the desired file stored in Accumulo, and
can be retrieved by the ``GeoMesaIndexedBlobStore`` API or by the REST
interface.
FileHandler Interface
---------------------
The FileHandler interface is designed to allow file handling in the
Blobstore to be pluggable. Implementors should implement
``org.locationtech.geomesa.blob.api.handlers.FileHandler`` or extend the
abstract implementation
``org.locationtech.geomesa.blob.api.handlers.AbstractFileHandler``. The
class ``WKTFileHandler`` serves as an example of that latter approach.
``FileHandler`` implementations are loaded at runtime by the Java
ServiceLoader. As such, available implementations should have their
classname registered in
``META-INF/services/org.locationtech.geomesa.blob.api.FileHandler``.
BlobStore Interface
-------------------
The ``BlobStore`` interface is designed to allow the backing blob store
of a ``GeoMesaIndexedBlobStore`` to be pluggable. Implementors should
implement ``org.locationtech.geomesa.blob.api.BlobStore`` interface. The
``AccumuloBlobStoreImpl`` serves as an example implementation of a
``BlobStore`` .
As of this time the ``BlobStore`` interface does not yet use the Service
Provider Interface.
Provided File Handlers
----------------------
The BlobStore API contains by default the ``WKTFileHandler`` and
``ByteArrayHandler`` handlers. The ``WKTFileHandler`` allows users to
ingest any file by associating a WKT geometry to a file. Similarly the
``ByteArrayHandler`` allows users to ingest arbitrary byte arrays
accompanied with spatio-temporal attributes.
Along with these included handlers, the GeoMesa BlobStore has two sub
modules of additional handlers in the geomesa-blobstore-handlers module.
GeoMesa-BlobStore-EXIF
~~~~~~~~~~~~~~~~~~~~~~
The EXIF module contains the ``MetadataFileHandler`` which can parse
EXIF metadata to retrieve geolocation information. This enables ingest
of large collections of geotagged images into a GeoMesa BlobStore.
GeoMesa-BlobStore-GDAL
~~~~~~~~~~~~~~~~~~~~~~
The GDAL module contains the ``GDALFileHandler`` which utilizes GDAL to
extract geolocation information and time from raster/imagery files
supported by GDAL 2.0.0. For a complete list of GDAL supported file
formats please refer to: `GDAL formats list`_.
To utilize this handler the user must have compiled GDAL Java bindings
and have placed relevant JNI libraries in the LD\_LIBRARY\_PATH
environment variable. To build GDAL and the JNI libraries, follow the
instructions here: `GDAL JNI build instructions`_.
GeoMesa-BlobStore-Accumulo
--------------------------
The BlobStore-Accumulo module contains concrete implementations of the
``GeoMesaIndexedBlobStore`` interface for Accumulo as well as an
implementation of the ``BlobStore`` interface for Accumulo, the
``AccumuloBlobStoreImpl`` class. The ``GeoMesaAccumuloBlobStore``
utilizes a GeoMesa Accumulo DataStore as well as the
``AccumuloBlobStoreImpl`` for the Blob persistance layer.
File retrieval is a two-step process. First a query is made to the
GeoMesa ``FeatureStore``. The returned ``SimpleFeature``\ s will contain
a URI field. That URI points to the desired file stored in Accumulo, and
can be retrieved by the ``GeoMesaIndexedBlobStore`` API or by the REST
interface.
.. _GDAL formats list: http://www.gdal.org/formats_list.html
.. _GDAL JNI build instructions: https://trac.osgeo.org/gdal/wiki/GdalOgrInJavaBuildInstructionsUnix
REST API
--------
The GeoMesa BlobStore comes with a RESTful web interface servlet that can be used in place of more direct programmatic access via the Java and Scala classes.
As of now the the BlobStore only provides an interface to an Accumulo backed DataStore and BlobStore. Once the servlet is deployed the RESTful api for the Blobstore can be easily utilized via cURL.
DataStores are managed by assigning them to aliases, this allows users to connect to multiple blobstores.
To Register a GeoMesa Blob Store to an alias (in this case myBlobStore):
.. code-block:: bash
curl -d 'accumulo.instance.id=myCloud' -d 'accumulo.zookeepers=zoo1,zoo2,zoo3' -d 'accumulo.catalog=myBlobStore' -d 'accumulo.user=user' -d 'accumulo.password=password' http://localhost:8080/geoserver/geomesa/blobstore/ds/myBlobStore
Other Blob Store commands:
- DELETE /ds/:alias - Delete a previously registered GeoMesa data store
- GET /ds/:alias - Display a registered GeoMesa data store
- GET /ds/ - Display all registered BlobStores
To ingest a file:
.. code-block:: bash
curl -X POST -F file=@filename.whatever http://localhost:8080/geoserver/geomesa/blobstore/blob/:alias
To GET a file with the original filename preserved via id, run:
.. code-block:: bash
curl -JO http://localhost:8080/geoserver/geomesa/blobstore/blob/:alias/some-id/
The Blobstore servlet also has optional GZip support which can be used
by adding the ``--compressed`` cURL parameter.
.. code-block:: bash
curl --compressed -JO http://localhost:8080/geoserver/geomesa/blobstore/blob/:alias/some-id
To DELETE a file from the blobstore, you must do so by id:
.. code-block:: bash
curl -X "DELETE" http://localhost:8080/geoserver/geomesa/blobstore/blob/:alias/some-id
Servlet Configuration Options
-----------------------------
File upload constraints for the ``BlobstoreServlet`` can be configured with
the following system properties:
.. code-block:: bash
org.locationtech.geomesa.blob.api.maxFileSizeMB (defaults to 50MB)
.. code-block:: bash
org.locationtech.geomesa.blob.api.maxRequestSizeMB (defaults to 100MB)
The expected unit for these properties is in MB, so setting
``org.locationtech.geomesa.blob.api.maxFileSizeMB=10`` will result in a
10MB maxFileSize.