Nevertheless we will start by discussing a general characteristic of any spatial analysis, whether on land or sea. Environmental sciences are sciences that rely observation as opposed to experimental sciences. It is only in few occasions that we have the chance to test our propositions to explain different events. The most common method is to measure the different values of the selected variables and use their respective localization to explain the observed differences. Therefore, we use their geographical position as a key element to solve our problems. And this is where geographical position can intervene, and this can be done in two different ways: as location (context) or as localization (spatial).

The coastal environment consists of a diverse range of locations, spaces. Each of these locations is determined by a specific set of local environmental conditions. In the context of spatial modelling, the treatment of geographical position as “locality” is based on the variety of places that makes the value of the observed variables actually vary. We can consider as example a coastal vulnerability map: each zone (location) determines a specific set of variables used for the classification of the coastline.

In the coastal environment, the gradient (variation) of the different environmental variables are one of its most striking features. The geographical position approach as location (space) focuses on how objects relate to each other along these gradients, and on how this relative positioning can explain the observed variability.

This variability can be the result of spatial relationships and can be expressed using measurements such as distance, gradient and vicinity. A gradient is a local property of a space but derives from the relative magnitude of a variable in two adjacent areas. We can consider Distribution of a given fish species as example of a gradient. There will be areas (locations) of high population, areas (locations) where the species will be absent and a whole range of geographical positions (localizations) where one will observe a range of decreasing values.

I hope that by now you have understood the nuance between location and localization. In coastal environments, when considering locations, we must add a quite unique characteristic: the locations are not necessarily fixed (permanent) in space. The most striking example is the case of water masses. Each water mass is characterized by a whole series of parameters (salinity, temperature,…). But they move. If you consider a beach, at a given moment, you will have a temperate coastal water mass. The next day you will find a cold water mass. If you are interested in the fate of pollutants, there is no point in mixing your observations: you are in the same place, but not, at all, the same localization. The concentration of such pollutant measured the day before will be compared with today’s concentration in the same water mass, but that water mass could be miles away.

But you may wonder where I am coming from. In fact, we tend to tackle the spatial analysis by focusing on gradients and disregarding the need to highlight the locations involved in our study. The complexity and power of today’s geomatics tools make it possible to obtain quite satisfactory results, even when our intellectual approach is not rigorous. It is a trap that forces the geomatician to, always, ask himself what does, actually, do the tool he is using, about the data, and the assumptions supposed to abide by.

The most outrageous example is the application of kriging methods to coastal data that do not respect the principle seasonality of the average. The stability of the average assumes that the average is constant between samples and is independent of the location of the samples. If you want to make a bathymetric chart on a littoral zone, you will have a different average depending on whether you are near the coast or far offshore. You can ignore this and realize your kriging, you will have a result that will seem correct even if it is mathematically and conceptually wrong. The correct procedure is to start by determining the trend of the data, remove it from input values (which makes the average stable), to realize kriging and to add the tendency to the result.

The result will be different in the two approaches; the second is much acute and highlights those hidden features at a first glance. On the other hand, following the first step you will spend hours to obtain almost the same result as using an IDW interpolation method that requires a few minutes work.

While the presence of spatial autocorrelation poses an analytical challenge, there are a number of practical benefits from its integration into the coastal environments analysis:

to quantify the extent and pattern of spatial dependence in coastal areas,

to determine the presence of redundant information in the field data sets,

to quantify the nature and extent to which the basic assumptions of classical (non-spatial) statistics are not respected,

to determine the extent to which an observed model is spatial in nature or not, and

to allow the use of spatially structured model components to fill the gap of missing variables.

In the next article we will address another important aspect of spatial analysis in coastal areas: the spatial and temporal scales of the analysis.

]]>https://www.sigterritoires.fr/index.php/en/integrated-coastal-management-location-spaces-and-localization-space/feed/14885QField: the mobile device of QGis for Androidhttps://www.sigterritoires.fr/index.php/en/qfield-the-mobile-device-of-qgis-for-android/
https://www.sigterritoires.fr/index.php/en/qfield-the-mobile-device-of-qgis-for-android/#respondTue, 13 Feb 2018 07:52:23 +0000http://www.sigterritoires.fr/?p=4874Continue reading "QField: the mobile device of QGis for Android"]]>QField allows you to do fieldwork on QGis projects with Android phones or tablets.
In this article we will discuss how to install QField on a mobile device and how to install and work on a QGis project.
Let’s see what can you do with QField?
QField allows you to install a QGis project on your phone or tablet. You will have a display equivalent to your QGis display on your PC and you will be able to edit the attribute or geometric information in the field. Do not confuse QField with the QGis Android version. It opens an interface equivalent to the one you have on your PC, with all the features of QGis. QField opens a simplified interface where you are primarily limited to updating the data included in your project.

Download QField to your mobile device.

There are several ways to download QField: through the Play Store, the QGis Downloads page, or simply by searching your browser with the words “qgis android”:

Touch the option QField for QGis Experimental (not QGis Experimental)

Then touch Install

Touch Install again to allow the application access to the location data
When finished, close the Play Store window and tap the QField icon that has just been added to your desktop:

A window tells you that the different modules are being installed
Then a window shows you that the installation is completed and ready to be used:

As the message indicates, now you need to create a project on your PC version of QGis, transfer it to your mobile device and open it using the button on this page.

How to create a QGis project

You must create a project on your PC with QGis. You will have a .qgs file that you will have to transfer to your mobile. But you will also have to transfer the required data to your project. Let’s see an example:

Create a directory with all the files needed by the project

All the required data must be physically on your mobile. WFS, WMS connections, internet connections like OSM, etc., will not work.
In our example, we use OpenStreetMap as background map. In order to be able to use it on our mobile, we have to save the basemap as an image, then load it into our project instead of the OSM layer.
For WFS, WMS, etc. layers, we also have to save the layer as a shp file and replace the layers with their files in our project.
In our example, we create a directory “Campus” where we save the image from OSM and the file with the data of the buildings. We save the project in this directory.

Create the desired symbology for the fieldwork.

For each layer you can use all QGis features for their symbology and labeling. You will not be able to change these settings in QField.

Transfer the project to your mobile

Connect your mobile to your PC and copy the directory together with your project and data. Even if you can do it anywhere, you should do it in the Android / Data directory

Launching QField on your mobile

Launch QField from your mobile desktop. You will have a project selection window, tap the button to browse the files
Browse the directories, in our Android / Data / Campus example, select the project’s .qgs file and click Open.
QField opens the project window on your mobile
There are two buttons at the top left. The blue button allows you to activate the location of your mobile and display a red dot on the map indicating the geographical position of the mobile.

The menu button opens a panel:

At the top of the panel, there are two buttons. The first (menu) closes the panel. The second opens the possible options:
Besides the Open a project option to change projects, the other important option is the Mode option .

There are two working modes: Browse and Draw.
The first refers to the attributes of the entities, the second to the geometries.

Browse mode

The browse mode allows to select the entities identifiers on the right banner by touching them on the map.
If you touch the identifier of the right panel, a new window opens with the attribute table of the entity:

Drawing mode

At the top right there is a pencil to edit. Then, you can enter / modify the values of the attributes.
On the left banner you select the layer to edit. The pencil at the bottom left goes into edit mode.
The cursor is placed in the centre of the screen. Unlike the PC, you move the map and not the cursor.
• The + green button records a new point
• The X button cancels the entity and exits the edit mode
• The – button cancels the last entered point
• The yellow button completes entering the entity and saves it, leaving the edit mode.

]]>https://www.sigterritoires.fr/index.php/en/qfield-the-mobile-device-of-qgis-for-android/feed/04874How to create a SpatiaLite database with QGishttps://www.sigterritoires.fr/index.php/en/how-to-create-a-spatialite-database-with-qgis/
https://www.sigterritoires.fr/index.php/en/how-to-create-a-spatialite-database-with-qgis/#respondMon, 05 Feb 2018 08:34:40 +0000http://www.sigterritoires.fr/?p=4866Continue reading "How to create a SpatiaLite database with QGis"]]>If you start to feel tired of managing your data as shape files (this format dates back to the eighties!) and you yearn for a genuine database for your projects, most certainly, you will be looking to use PostGis. There is a big gap, and the ‘jump’ is, by no means, an easy one. In this article, we will discuss an alternative solution, the SpatiaLite database. And, believe me, it is much more easier….

Comparison between SpatiaLite and PostGis

In this article we will examine the use of SpatiaLite, an open source cross-platform software, simple, lightweight and robust. SpatiaLite, a spatial database management system (DBMS) extension of SQLite, a DBMS intrinsically simple and lightweight. SpatiaLite (and SQLite) is based on an individual architecture resulting in an, almost, inexistent management and/or installation. The defect is that it does not work properly with simultaneous access and that it does not support the client/server architecture.

For a more complete DBMS, PostGIS is an excellent option in the Open Source world. PostGis offers many more features than SpatiaLite. GeoDjango documentation shows a comparison of the spatial characteristics supported by each database https://docs.djangoproject.com/en/dev/ref/contrib/gis/db-api/#compatibility-tables.
Please note that it is not a direct comparison between the two databases. It highlights the features supported by GeoDjango. However, it offers a pretty good idea of the differences between the two applications.
The choice between PostGIS and Spatialite depends mainly on the research goals. Spatialite is a better option if you need an autonomous or embedded database to develop a mobile/desktop app. Besides, Spatialite can be used to develop a web app.
PostGIS is a better option if you need ‘writing/reading speed’, several concurrent users in the database, clustering, or network access.

How to create a spatial database

1. Open QGIS and the exploration panel. If the panel Browseis missing, click View | Panels , and click Browse. In this panel you will find the access to SpatiaLite

2.Create a new database Spatiality by clicking SpatiaLiteand selectingCreate a database …
3. Select the directory of your choice and name the new database, for example BaseTest. It will appear under the rubric SpatiaLite

Structure of a SpatiaLite database

Now that we have a new database SpatiaLite, we will analyze its primary structure and content .
For this task, we will use the Database Manager, an integrated QGIS plugin. DB Manager offers a simple graphical interface to manage the databases PostGIS and SpatiaLite We will be able to view and manage our database SpatiaLite by using the database manager. We will start by getting familiar with the interface of the database manager.
1. To open, click Database Manager in the Databasemenu. The DB management interface (as shown in the next screenshot) has four different sections: menu bar, toolbar, tree and panel information.

2. Click SpatiaLite -> BaseTest to see the tree with all the tables, views and General Info in the database we have just created.
When a new SpatiaLite database is created, it is automatically loaded with several tables and processed data. These tables and data include documents used by the DBMS to manage the structure and operation of the database. You should not change or remove these tables and/or data, unless you are completely aware of what you are doing.

Let’s see how to import a shapefile: 1. Open the database manager by clicking Database Manager in the Databasemenu. Open SpatiaLite and select BaseTest.sqlite in the tree .
2. Go to the menu Table/Import a layer or a file to open the dialog Import a vector layer, as shown in the following screenshot.
3. Highlight the shapefile you want to load.
4. Click Upgrade options to load the rest of the dialogue box. The name of the output table will be filled with the name of the input shapefile.
5. Set up the following options:
° ° Select SCR Source and enter the EPSG code for the input data..If you do not want to change the coordinates system when proceeding to the import, it is useless to define the SCR target, alternatively, you can enter the desired coordinate system for the data in the database.
° ° Select Create a spatial index.
6. Click OKto import the file.
Once the table is uploadade, you can display it directly in the cartographic window by double-clicking the table’s name.
In the following article we will discuss how to upload a non spatial table in the SpatiaLite database, how to create a table using all the different documents and how to create and manage pictures.

]]>https://www.sigterritoires.fr/index.php/en/how-to-create-a-spatialite-database-with-qgis/feed/04866How to rectify the geometry of a Postgis tablehttps://www.sigterritoires.fr/index.php/en/how-to-rectify-the-geometry-of-a-postgis-table/
https://www.sigterritoires.fr/index.php/en/how-to-rectify-the-geometry-of-a-postgis-table/#commentsFri, 26 Jan 2018 08:14:28 +0000http://www.sigterritoires.fr/?p=4800Continue reading "How to rectify the geometry of a Postgis table"]]>We have already discussed the topic of geometry validation through a series of articles (in french):

In this article we will discuss, in detail, how to detect and rectify geometric problems in a PostgreSQL / Postgis table using SQL queries.

Error checking with QGis.

You can promptly detect geometric errors using QGis. Just load the layer and launch the menu Vector – > Geometry tools – > Check the validity
You can choose one of the following methods:

GEOS validation method

QGIS validation method, and

method for interactive verification when creating a new geometry (Preferences → Scan tab).

The GEOS method is faster but only indicates the first error encountered for each object. It returns a file ‘Invalid output’ which is a layer of polygons completed by a column a column _errors:

The Qgis method can return multiple errors per object. This method returns in the ‘Invalid output’ file a layer of polygons completed by a _errors column with a less standardized message than by the GEOS method:
Personally, I only use this tool to check for errors. I execute both methods and, if they do not return any error I consider that the layer as valid.

Meanwhile, if there are errors, I go directly to pgAdmin 4 (or failing in the database manager of QGis) to work in SQL.

Detection of Geometric Errors with Postgis

The first step is the identification of errors. Note that PostGIS complies with the OGC OpenGIS Specifications, and therefore the validity is checked against GEOS.
Therefore, we will perform a query that will search the three main types of errors:

Collections of geometries, ie objects composed by several geometries of the same type or not. The most frequent examples are multipolygons that are inaccurately cataloged as FeatureCollection . In general, spatial operators do not work with input collections.

A last type of error, usually of no consequence, is the presence of duplicated nodes (two points of the entity located exactly in the same place). They do not disturb space operators, but that’s no reason not to fix them.

Here is a query that allows you to detect the first three types of errors in one pass:

SELECT ‘non valide’ AS nb, count(*) FROM table WHERE NOT ST_IsValid(the_geom)
UNION
SELECT ‘geom nulle’ AS nb, count(*) FROM table WHERE the_geom is null
UNION
SELECT ‘collection’ AS nb, count(*) FROM table WHERE not ST_IsValid(the_geom)
AND ST_GeometryType(ST_MakeValid(the_geom))=’ST_GeometryCollection’;

You can obtain details of GEOS errors with the following query:

SELECT id, ST_IsValidReason(the_geom) FROM table WHERE NOT ST_IsValid(the_geom);

Correcting errors

(We will be using the case of a polygon layer, which is the type of entity that originates most problems.)
Correct GEOS errors with St_MakeValid

UPDATE table SET the_geom = ST_MakeValid(the_geom) WHERE NOT ST_IsValid(the_geom);

Then, we rectify the collections by transforming them into Multi_polygons. Be careful not to reverse the steps. The St_MakeValid function can produce collections as an output.

This last request is a, somewhat, diverted way of doing the operation. The St_Simplify function is a generalization function, ie a function that reduces the number of points of an entity, by removing points that are at a distance smaller than the parameter of the function. By setting 0 to this distance, the function leaves only one point among all points in a zero radius (duplicates).

Once these queries have been executed, the fault finding request is executed again. If errors persist, it will be necessary to use another method to solve them.

A technique often used is the creation of a null buffer:

UPDATE table SET the_geom = ST_Multi(ST_Buffer(the_geom,0)) WHERE NOT ST_IsValid(the_geom));

At this point, if errors persist, we must give up the idea of automatically correcting them, take a breath, and do it manually.

The most suitable method to manage PostgreSQL databases is by using the pgAdmin4 GUI.

This tool is setup automatically during PostgreSQL installation. You can launch it from the programme bar:

No special management procedure is needed when loading a shapefile as a Postgis layer into your new database. But it’s easier to understand the operation by following the changes in the database.

Nevertheless, you can rest assured that, for the time being, you will not have to create anything else. When installing the Postgis, we have, already , requested the creation of a sample database. And, it is this database that we will be using for the purpose of this article.

pgAdmin4 allows you to manage a PostgreSQL / Postgis database without worrying about the corresponding SQL syntax.

When you launch the application, the first screen allows you to connect to the desired server and database.

Right-clicking on an element opens the contextual menu that proposes the different available operations for this element.

During the installation procedure, we have created, automatically, two databases :

a classic PostgreSQL database called postgres

a PostgreSQL / Postgis database called postgis_24_sample

And, it is the latter, that we will be using to load a shapefile. But before, take a look and observe which are the differences between these two databases.

If you have a look at each database extensions, you will discover that the Postgis database has a series of specific extensions that can be used to manage spatial information. It is the presence of these extensions that makes a PostgreSQL database a Postgres / Postgis database.

When you create a database, you must add these extensions to make it applicable with Postgis.

Right-clicking extensions and Create – > Extension , you will have the list of available extensions.

Add a shapefile to a Postgis database.

First, what does it mean to add a shapefile to a Postgis database? A “file” shapefile is actually a group of files with the same name and different extensions. For example:

All of these files constitute the “shapefile” ouessantCLC_L93 . One file contains the geometries, another the attribute data, and so on.

The information included in the files will be loaded as a Table into the database. Only one table will include all the information scattered in the shapefile files.

In the previous image you can examine at which level the data will be stored, and you will notice that the “Tables” level is a sub-level of a “Schema” which is a sub-level of “Database”. When loading the shapefile, you will have to specify which database, and, possibly, which schema to use.

In our example database there is a “Public” schema which, unlike the other available schemas, includes all the functions (1454). We will explain this concept later. Let’s just remember that we add functions to a schema as and when needed. In our example, all functions are already loaded, it avoids these additional operations.

We can load the shapefile with the “Postgis 2.0 Shapefile and DBF Loader” programme.

You can launch it as a stand-alone application, from the Windows programme bar:

Unlike pgAdmin III, you will not find a button in pgAdmin 4 to launch this application.

The application window opens:

You must first establish the connection with your database

Click “ Add File ” and navigate to select the shapefile to load. Click Open , the file information to be loaded will appear in the Import List

Click the Import … button
Here! It is done. And now you can observe in your pgAdmin 4 (after refreshing) the imported table with your data.

Fill in the coordinate system

Perhaps you have noticed in the Import List window that the SRID (coordinate system) is at zero. This indicates that the loader was unable to correctly interpret the shapefile’s .prj file. In our case it should be 2154 (Lambert 93 EPSG code / RGF93).
We will update the information in the geometry column of the table we have just created.

Open the SQL window of pgAdmin 4:

In the Edit SQL window type the following query:

SELECT UpdateGeometrySRID (‘ouessantclc_l93’, ‘geom’, 2154);

You only have to change the name of the table (ouessantclc_l93) ) using the name of your table, and the code of the SRID (2154) using the EPSG code of your coordinate system.

Run the query by clicking the Execute button (indicated by the blue arrow on the image).
You will receive a message stating that the request has been executed successfully:

Load the Postgis layer into QGis

To verify that everything went smoothly, open QGis and select the Postgis layer loading button

The login window opens. The first time you connect to a database you must configure your connection.

Click “ New ” and fill in the login information

You have everything you need in the console window pgAdmin 4. At the Server level if you right click on “PostgreSQL 10” – > Properties , you will have all the necessary information: Host and Port . The Name is only used to display the pull-down menu of connections. You are free to enter any name. Leave Service blank.

The database where you loaded the shapefile is “postgis_24_sample”.

You must enter the username: we will use “postgres” and the password that you defined for this user when running the installation.

Click Test Connection to make sure all information is entered, and then click OK.

The new connection is added to the list. Click “ Connect “. You will have the different Schemes available together with their specific list of spatial tables:

Select the line matching the layer loaded with the Public schema and click the Add button.

The coordinate system definition window is displayed. Select the SRC of the data (in this example Lambert 93)

The layer is loaded in QGis.

You can now work on this layer as on any other layer you were used to (display, symbology, geometric update or attribute, …)

]]>https://www.sigterritoires.fr/index.php/en/geoserver-avanzado-el-teselado-puesta-en-practica/feed/04703Advanced Geoserver : tiling (quick start)https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-quick-start/
https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-quick-start/#respondWed, 17 Jan 2018 09:04:27 +0000http://www.sigterritoires.fr/?p=4684Continue reading "Advanced Geoserver : tiling (quick start)"]]>In the previous article (Advanced Geoserver: tiling (principles) we have explored the different concepts regarding the tiling mapping.
In this article we will discuss how to implement those applications using Geoserver.
In order to run the tiling, Geoserver uses an external module called GeoWebCache.

What Is GeoWebCache?

Maps are often static. Since most mapping clients supplyWMS (Web Map Service) data every time they are queried, unnecessary processing and delays can occur. GeoWebCache optimizes this experience by saving (caching) map images, or tiles, as they are requested, in short acting as a proxy between client (such as OpenLayers or Google Maps) and server (such as GeoServer, or any WMS-compliant server). As new maps and tiles are requested, GeoWebCache intercepts these calls and returns pre-rendered tiles (if stored), or calls the server to render new tiles if they are needed. Thus, once tiles are stored, the speed of map rendering increases significantly, creating a highly improved user experience.

In the picture above, the blue box on the GeoWebCache device represents the tile storage. Ideally, most requests are answered using the data stored without consulting the WMS server(s). Hence the arrow to the clients is drawn much larger, because GeoWebCache can answer hundreds or thousands of requests per second.

Also,GeoWebCache can even be used when maps aren’t completely static, since it allows for the selective expiration of tiles, so that data remains current. In this respect GeoWebCache therefore acts as an all-purpose accelerator for map rendering.

Integration within Geoserver

The GeoWebCache module is delivered with the installation of Geoserver. It is installed and enabled by default. You can check it by going to Service Status-> Modules.

On the other hand, vector tile is not installed by default. To check, select Cached layers in the left menu. Then open a vector type layer. You have some installed by default in Geoserver (tiger:poi, sf:archsites, sf:bugsites, topp:tasmania_cities)

Go to the Tiles cache tab

If you do not see the vector formats (geojson, topojson, mapbox-vector) among the available options, the extension is not installed.

Setting up GeoWebCache

Actually everything is already set in GeoWebCache and Geoserver. The only thing left to set up is tracking the disk space used by the cache.
By default, this monitoring is disabled. This means there is no space limit for the tile cache. Depending on your configuration, this may be acceptable. But in most cases, you will face serious risks of performance drops or even server crashes.

If you go to Cached Layers , you’ll see two columns: Space limit and Quota used . Both contain N / A. This indicates that the disk quota is not enabled.

Indicate the maximum size dedicated and the type of action to be taken when this size is reached.

You have the option to request the removal from the cache the images that are called less frequently, or those that have not been called for quite a long time.

By default Geoserver configures GeoWebCache to use a H2 database located in the cache directory.

If you do not need to manage cache operation information, this option is perfect.

On the other hand, if you want to keep the operating history, the use of the tiles, etc., you can select the external database of your choice.

One final option , potentially interesting, is the Space limit column on the Cached layers page. If you look in the Geoserver documentation, you will find that this column indicates the specific cache space limit for the layer. Indeed, the independent version of GeoWebCache makes it possible to define quotas per layer. But this is not the case with the built-in Geoserver version. You do not have a way to set specific quotas. This column will remain frustatingly with N / A.

Do not try to apply these definitions directly in the geowebcache-diskquota.xml file, it will not work .

Once tracking is enabled , you will observe the space of each layer in the cache:

Configuration of a layer cache

You’ll find the Tile Cache tab on the Edit Layer page.

The first two check boxes determine whether the cache is enabled for this layer and whether tiling is enabled.

You must check the desired tile formats. Only when the server receives requests for this format (output), GeoWebCache will use the cached tiles.

At the bottom of the page you can select the zoom levels to be used by the cache.

You can only cache the most frequently used zoom levels.

The following table can be used as a guide to determine which zoom level to use:

For each zoom level (first column) you can find the corresponding geographic scale.

If you want GeoWebCache to produce the tiles, it is not enough to activate the tiling of a layer.
The tiles will be generated when a request from the server client is placed. If this is not the case, then you can request a preview of the tiles using the Hidden Layers Page. The drop-down menu allows you to choose the desired output format.

As this preview is a request made to the server, the corresponding tiles will be generated.

Last option to generate the tiles, select Generate from the Layer Actions list (last column of the table).

A page where you can define the parameters to generate the tiles will be displayed.

]]>https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-quick-start/feed/04684Advanced Geoserver: tiling (principles)https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-principles/
https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-principles/#respondTue, 09 Jan 2018 07:51:46 +0000http://www.sigterritoires.fr/?p=4653Continue reading "Advanced Geoserver: tiling (principles)"]]>One of the most interesting features of Geoserver is its ability to manage a map server. In this article we will explain how to take full advantage of this feature.

Reminder of the tiling concept

In the following image you can observe the two different operating methods for a map server.

The left image shows a classic map server:

a client application creates a query to display a map. The server receives the query

the server proceeds to retrieve the required data from a database holding the different geographical layers to generate the map

the result is properly shaped to layout the map requested

the map is sent to the client application to be posted

Everytime a query is placed, the server must query the database and formatting the data retrieved.

The right image shows how a map server with the feature « tiling » activated works:

a client application creates a query to display a map

the server receives the query according to the area of the requested map, the server selects the already calculated images from the server cache corresponding to the area and zoom level requested

the images are transmitted to the client application to be displayed

The retrieval and formatting has taken place beforehand and just once.. When using this second configuration, only a simple calculation is needed to determine which images must be retrieved to the client application. This configuration is time and resource sensitive. A map is created in accordance with a formatting symbology, finally the image is cut in equal size tiles. This task has to be done just once.

Each tile represents the formatted data and can be « served » very quickly. However, the only possible outcome is the display, it is not possible neither to access the underlying data nor to modify their format.

On the other hand, if you zoom in the tile, the pixellisation will increase very quickly. In order to solve this problem,you must create various maps, using different zoom levels. Each map is divided in tiles. At the time of receiving a query, the corresponding zoom level is calculated and the tiles pertaining to that level will aree used.

Typically you usie 4 tiles at a time. The most detailed layer is divided in tiles, generally 256×256 pixels. The underlying layer formed by 2×2 tiles is calculated (when using 4 tiles of 256×256 pixels or more, just one tile is calculated). And so forth till the level where only one tile is found.

This set of layers is called a pyramid.

Raster and Vector tiles.

Up to this moment we have been discussing a map server with images (raster). Even if the data from the database is vector data (Streets, buildings, parcs, etc) the final result is applied to the vector layers to generate a raster image, i.e., the outcome of the query.

This type of service is called WMS (Web map service).

Presently a new type of service has been developed. It allows to supply the data as tiles, the so called WMTS (Web map tile service).This service is included in gGeoserver to supply the data tiles as vector tiles.

The principle is the same used for the raster tiles :

The space is divided in squares and the vector data (entities) corresponding to each tile is retrieved.

When receiving a request, the server proceeds as with the raster tiles : the adequate level of zoom is defined, and then the pre-extracted data is sent to the client application.

Tiles creation

The tiles creation process is the same whether they are raster or vector tiles.

The first step is to retrieve the data that matches the query from the database.

Almost always this data is the vector data (point, line, polygon).

This is the data matching the entire available space, not just one tile.

The second step refers to the so-called « generalization ».

The process intends to store just the required level of detail needed for the zoom level preset, and, therefore, the unnecessary dots are eliminated.

The third step is the creation of a file to translate the grid system used by the tiling into a system of coordinates. Geoserver uses grid sets or grid subsets. GeoWebCache does not recognize the referencing system. Since GeoWebCache sends a request to WMS, it uses the information grid set and subset to transform its internal tile index into a spatial request accessible for the WMS.

The fourth step is a second type of generalization, but, instead of simplifying the geometries by reducing the number of points, it removes the features which are too small or superfluous.The step before last is the tile clipping. We haven’t yet applied the formatting. If we retrieve the exact required zone for each tile we could be missing the objects right on the joint of the tiles, as shown in the figure below.Geoserver uses a technique that reclaims a slightly larger zone than the demanded for the tile. This avoids those elements on the edges.

Finally we retrieve the formatted entitities depicting the symbology defined in one SLD file:

This file shows which entities must be taken into account, which scale range must be displayed in the tiles, and which type of symbol must be used to represent them.

Raster and vector tiles

According to the OGC standards there are three types of services

WMS-Maps supply system

WMTS-Tiles supply system

WFS-Data supply system

The raster or vector tiles are supplied by the same service, the WMTS.
Even if both types of data are supplied by similar methods, there are some differences.

In comparison to the raster tiles, the vector tiles are formatted by the client instead of the server. For example, when using the OpenLayers app, you will only have the chance to display the raster tiles with their original form, while when using the vector tiles you will have total freedom to choose the symbols.

One tiling alone can produce different maps. It must be added that, compared with the raster tiles, the vector tiles show an increased yield for the High resolution screens.

As for the disadvantages, the use of raster tiles is easier than the vector tiles. In order to work with vector tiles you must have a deeper knowledge.

You can conclude that it is more interesting to use the vector tiles service instead of the classic WFS.

Both can return vector data without formatting, but the WFS returns the underlying data UNMODIFIED, while the WMTS returns the underlying data MODIFIED : ready to be posted.

In the first case you have access to all the attributes of the entities, while in the second case you only have the attributes defined in the SLD file of the tiling.

In the next article we will discuss how to set up the raster and vector service using Geoserver.

]]>https://www.sigterritoires.fr/index.php/en/advanced-geoserver-tiling-principles/feed/04653Use of Landsat images (free) in your GIShttps://www.sigterritoires.fr/index.php/en/use-of-landsat-images-free-in-your-gis/
https://www.sigterritoires.fr/index.php/en/use-of-landsat-images-free-in-your-gis/#respondWed, 27 Dec 2017 15:07:22 +0000http://www.sigterritoires.fr/?p=4629Continue reading "Use of Landsat images (free) in your GIS"]]>When we think of accessible data to integrate into our GIS, we hardly think of satellite data. And yet, it is possible, and just a few clicks away.

Let’s see an example. The following images represent the state of the vegetation for the islands of the Gulf of Morbihan, on the left for July 2014, on the right for March 2015. If we wish, we can obtain images every 15 days, the most just one week old.

Findin, recovering and processing them in just 5 minutes. So, why deprive yourself?

Among the available applications of satellite imagery, we will focus on the use of the normalized difference vegetation index (NDVI)

What is NDVI?

The normalized difference vegetation index, also called NDVI, is calculated using the visible red (R) and the near-infrared (IR) channels.
The normalized vegetation index highlights the difference between the visible band of red and the near infrared.

NDVI = (NIR-R) / NIR + R)

This index is sensitive to the quality (health) and quantity of the vegetation.
NDVI values ​​range from -1 to +1, with negative values ​​for areas other than plant covers, such as snow, water, or clouds, where red reflectance is greater than near infrared.

In the case of little or no vegetation, and, due to similar values of reflectance in both the red and infrared channels, the NDVI yields values close to 0.

Vegetation yields NDVI values, generally between 0.1 and 0.7. The highest values ​​correspond to the densest cover.
When we observe a time series of images for a given area, we can deduce the state of stress of the vegetation by the relative variations of the index: in winter or during a drought the values ​​will be lower than in summer or during abundance of water.

How do you calculate the NDVI?

According to the formula we need an image with a red channel and an infra-red channel. In the case of aerial photos, only the red channel is present. It is not possible to calculate the NDVI with a classic aerial photo.
You need a satellite (or overhead) image with more than three channels (bands). This is the case of Landsat (7 or 8 images), which have the advantage of being made available free by the USGS.

Once downloaded, the images appear as tiff files, one per band. Landsat 7 images has 8 channels or bands, Landsat 8 images has 11 channels.

If you combine the first three Landsat 7 bands, you get a classical optical image (RGB). To calculate the NDVI it is necessary to include the near infra-red band. You will then work with images having 4 bands, with a computer screen that can handle only three (usually RGB).

Depending on the software, you will have different ways of calculating the NDVI, and then, finally calculated, you will have to apply a color gradient that allows you to visualize the vegetation and its state of stress.

We will see, in order:

how to download Landsat images

how to calculate the NDVI (with arcgis and arcgis pro, with QGis)

how to apply an appropriate symbology

How to download Landsat images

You can research the catalog, but if you want to download images you will need to login. Start by registering on the site by clicking “Register”. It’s free, easy and it only takes two minutes . You will not have to deal with ads or messages back.

Now let’s go to the catalog.
First step: zoom in on the area of ​​interest in the map window.
To enter the geographic box as a search box, click in the “Coodinates” block on the left “Use map”

Second step: define the desired period.

In the “Data Range” block, set a start date and an end date for the search.

Once these two parameters are configured, click on “DATA SETS”

The window on the left scrolls all available satellite data sources on Earth Explorer. In our case, we click on Landsat Archive-> L8 OLI / TIRS and L7 ETM + SLC off.

The different denominations correspond to periods in which satellite sensors have, or have not, operated. You do not have to be aware of all these details, because when the server detects a match between the type of image requested and the desired period, it displays a window with explanatory text and a warning of the possible problems.

What you need to know is that for Landsat 7 you will have, since 2003, on the margins of the images, black bands due to a satellite problem. If your area is in the centre band of the image, do not worry. If not, it is better to try Landsat 8.

Once ticked the right boxes, click on “Results”

At the top of the “Data Set” block you have a drop-down window with the different types of images checked previously. The thumbnails displayed only correspond to the satellite displayed in this window. To see the other images found, you must select another “data set” in the drop-down window.

By clicking on a thumbnail you can open a preview window that will allow you to see the correspondence with your work area as well as the cloud cover.

If you want to download an image, you click on the “Download options” button on the button bar of the desired image.

If you are not logged in, the window offers to connect. When connected, the product type selection window appears.

Select “Level 1 Product” to download all the bands in the image.

Once the zipped file has been downloaded, unzip it to a directory. You will get a tif image per satellite band.

That’s it! You have the satellite image on your PC. Now it will be necessary to use the data in order to display the famous NDVI of your zone of interest.

Depending on the software you are using, it’s not quite the same operations. We will see in the next article how to proceed with this work with ArcGis (ArcMap or ArcGis Pro) and in the following using QGis.

]]>https://www.sigterritoires.fr/index.php/en/use-of-landsat-images-free-in-your-gis/feed/04629How to join data from Excel to an attribute table in QGishttps://www.sigterritoires.fr/index.php/en/qgis-joins-with-excel-tables/
https://www.sigterritoires.fr/index.php/en/qgis-joins-with-excel-tables/#commentsWed, 27 Dec 2017 12:06:18 +0000http://www.sigterritoires.fr/?p=4599Continue reading "How to join data from Excel to an attribute table in QGis"]]>The present topic is meant as a brief review since the possibility of loading and attach Excel tables was introduced by QGis 1.8.
As we do not follow the general logic, for new users it is not easy to find the answer. So, step by step, how to make a join between a spatial layer (we’ll take an example here with a shapefile) and a non-spatial table like Excel (with or without XY fields). To load an Excel table in QGis it is not necessary to go through a plugin (XY Tools, MMQGIS, …). You will use XY Tools if you want to transform the Excel table with X and Y fields as a spatial layer of points.

By cons if what you want is to join the Excel table to an existing spatial layer through a key (plot number, name of the owner, …) it’s quick and easy.

1- Load the Excel table in QGis

This is the trap. To do this, you will use the Load Vector Layer button.It must be known because the Excel table is not a vector layer.

Check that “ All files ” is selected, otherwise you will not see Excel files.

The Excel table then appears in the list of layers. You can open the table in the same way as other tables in the spatial layers loaded in QGis. You can also switch to edit mode, edit the table and save it. You will have updates to your Excel table, in the same way as if you had done them with Excel.

2- Load the spatial layer

Now load the spatial layer to which you want to attach the information contained in the Excel table.
For our example we will load a shapefile with oyster farms.

We will now join the Excel table using the ZTABISSEM field of the shapefile layer and the STATION field of the Excel table.

Joining Shapefile – Excel

Double-click the shapefile layer in the Layers panel
The properties window of the layer opens. Select Joints tab and click on the green cross to add a join.

The join definition window opens

In Join Layer select the Excel table </ li>

In join field in the attached layer select the key field of your Excel table

In Join field in the target layer select the key of your shapefile

Click OK to close the join definition window

Click OK to close the Layer Properties window

If you now open the shapefile layer table, you will see that the fields in the Excel table appear after the fields that were already present.

If you save your project now, you can continue to update your Excel table without QGis, but when you open the project again you will find all the changes made to the Excel table. The join is a dynamic link and the attached table is calculated on the fly at each opening.