LiDAR Analysis Toolkit for ArcGIS

Local Relied Model of Old Sarum, Salisbury, WIlts. Based on Environment Agency LiDAR data. Produced using the LiDAR Analysis Toolkit.

Archaeogeomancy were pleased to be commissioned to produce a self contained ArcGIS Toolkit for the analysis of LiDAR data. Whilst there are a number of commonly used toolkits in existence (for example RVT and LiVT as well as implementations of functions in GIS packages such as GRASS and QGIS), the client had some specific requirements, namely that the toolkit needed to be integrated within their corporate GIS solution (ArcGIS) and could not use installable executables, dlls or anything that required Windows Installer, editing the registry or administrative rights to set up or use.

Aside from corporate ecosystem restrictions, the Toolkit needed to produce a specific range of visualisations using preset variables suitable for use by non-specialist users but with the ability for more advanced users to tweak these variables as needed.

LiDAR Visualisations

Visualisations included:

A set of four Hillshades from inter-cardinal directions

Principal Components Analysis (PCA) using 16 directions

Slope

Aspect

Local Relief Model (LRM) – after Hesse, 2010

Sky View Factor (SVF) – after Zakšek et al. 2011; Kokalj et al. 2011

Openness (positive and negative) – after Doneus 2013

Where possible, tools from within the ArcGIS System Toolboxes were used. Indeed, for the more basic visualisations, tools were built using existing system tools scripted into processing pipelines using Python and/or ModelBuilder; Default values were built into the tools as were any output specifications. Some examples of outputs can be found here.

However, after some initial testing of various system tools within the ArcGIS Spatial Analyst and 3D Analyst extensions, it became apparent a bespoke solution for the more advanced visualisations would be needed. If we think about Sky View Factor and Openness functions in particular, one possible method would be to use the Skyline Graph tool as a starting point, especially as this tool outputs the ratio of visible sky for a given location. Unfortunately, the amount of processing time required to output products such as a single Skyline graph is significant; if we then consider that one such graph would be needed for each cell in the output surface model when calculating SVF and Openness rasters for an input surface model, such an approach would simply not scale well or perform adequately.

Old Sarum, Principal Components Analysis (PCA). Based on Environment Agency LiDAR data. Produced using the LiDAR Analysis Toolkit.

LiDAR Visualisations; advanced

A solution to the more advanced visualisations was instead implemented using custom Python script tools, making use of Numpy and Cython combined with vectorised implementations of the necessary neighbourhood functions to maximise raster processing performance. The use of vectorised functions alone resulted in a massive increase in performance as the operators used to calculate output cell values can be applied across an input array as a whole without the need for cell by cell iteration; Numpy is specifically optimised for such n-dimensional array based calculations. Cython further increased performance on the Sky View Factor and Openness functions by introducing C type definitions for specific variables and allowing these more complex functions to be implemented as pre-compiled modules rather than using the usual Python interpreter.

Even so, performance is still an issue. Whilst this is by far the best performing implementation within the limitations of the specification provided, the advanced visualisation routines are significantly slower than standalone systems and also struggle with large volumes of data ie larger areas of landscape.

Toolkit

The toolkit was developed as an ArcGIS Toolbox and then packaged as an ArcGIS Add-In complete with Toolbar, all compiled for the client’s version of ArcGIS. This enables the toolkit to be deployed as needed across the organisation using the Esri Add-In Framework and avoiding the need for time consuming setup on individual machines, administrative rights to undertake software installation or extensive testing of software by corporate IT teams to enable it to be approved and installed on corporate build machines.

Full documentation was provided by means of the in built Esri help system for each Tool (authored using ArcCatalog) as well as a fuller manual deployed as an html file accessible from the Toolbar, the html file written using the structure and stylesheets used on all such Archaeogeomancy projects.

Post navigation

8 thoughts on “LiDAR Analysis Toolkit for ArcGIS”

Looks good. We would like to adopt this to the many slides of LIDAR we have here at the National Park. Wondered whether or not the ArcGIS toolkit is available through working with QGIS we have recently apopted here.. Get in touch.

The toolkit here is built to work within ArcGIS as an Add-In, as needed by the client for this project. But there are alternatives that will work with QGIS; the GRASS extensions for example or the standalone options mentioned in the post. If you would like to discuss packaging something for use in QGIS, drop me a line and we can take it further.