Ping and Done: Evolving QPS Technology for the new Era of Autonomous Hydrography

The implementation of autonomous technology in hydrography has seen steady evolution for 20+ years. The initial developments focused on deploying large diameter (0.5 to 1m) vehicles (AUV’s) at great depths. AUV’s have quickly proven to be cost-effective solutions for obtaining high resolution bathymetry, imagery and photographic images in environments where preceding technology fell short. The most recent innovation surge is focusing on autonomous surface vessels (ASV’s) where organizations have found great gains to be made by removing personnel from the vessel altogether. Since 1997, QPS has provided options for the user to minimize the interactions with the surveyor and the system by automating key functionality. The combination of QINSy’s automation and Qimera’s data processing innovations (Wilson et al., 2018) provides a tightly integrated solution for organizations to embark on this new era of Hydrography.

QINSy autopilot setup window with AutoSurvey dynamic line planning functionality with turning track (left) and the QINSy Navigation Display showing the result (right).

Remote Display Client output settings

Remote Display Controller. The RDC controller mimics what is running on the AUV/ASV and allows full remote access and configuration of the system

. QINSy Alert System for Seabed Collision Avoidance. A collision point is detected and the estimated time to contact is calculated.

For Cooperative multi-user cleaning, Qimera breaks up the project based on user-defined parameters. This example shows a simple grid of 9 subprojects.

Simplified workflow for QPS’s concurrent (real-time and/or near real-time) processing. QINSy may be utilized as a comprehensive autonomous solution. Alternatively, for non-QINSy users, Qimera Live may be utilized for near real-time data processing.

Qimera seamless ENC S57 functionality allows for features to be added regardless of view, editor (swath, slice, 3D), etc. Qimera automatically updates the wrecks, rocks, and obstructions on a corresponding S57 feature file. This is accomplished via a link between the S57 attribute value of sounding (VALSOU) and either a particular sounding or grid node.

Water-column processing above a wreck reveals a mast with a more accurate least depth to be incorporated into bathymetry, ensuring the utmost in safety of navigation for those nautical charting products later derived.

Introduction

One of the challenges posed to the autonomous hydrographic workflow are issues arising from the incorporation of a variety of software solutions with non-seamless linkages. Many autonomous operators are incorporating separate solutions for; mission planning, sensor control (navigation, sonar settings, etc.), acquisition, systems monitoring (QA/QC), data fusion, data processing and final product deliverable. For a single mission this can lead up to eight (8) different software solutions from four (4) or more vendors. A non-seamless workflow leads to increases in error due to a heavy reliance on human interaction (Wilson et al., 2018). QPS has successfully decreased error and improved efficiencies by automating the tasks best suited by computing technology. Since 1997, QINSy has had the ability to provide all eight of these requirements for surface (ASV) applications. For subsurface applications (AUV), QINSy is scheduled to be fully compliant with the incorporation of mission planning functionality by the end of 2018. This paper will outline the history of QPS’s commitment to autonomous applications and how these requirements are paving the way for this new era of hydrography.

Early Real-time Innovators

The QINSy software solution has been a market leader in flexible, stable and reliable hydrographic operations for well over two decades. Initially implemented as a software solution for complex positioning projects (rig moves, marine construction, etc), QPS implemented its hydrographic functionality beginning in 1996. By 1997, QINSy had already laid the groundwork for autonomous applications through the introduction of auto-logging; real-time data processing (including ray-tracing, height compensation — RTK, tides); multibeam filtering (depth, range, sector, quality, TPU, etc); grid generation; systems monitoring alerts (satellites, standard deviations, proximity/ collision alarms) and the Remote Display Client (low to high bandwidth remote access). By 2002, real-time sidescan and backscatter mosaicking (with or without normalization) was added, and by the end of 2006, magnetometer grid generation, autopilot interfaces, generic/user configurable output drivers and advanced survey planning features were released. In 2015, QPS introduced Qimera and shortly thereafter released Qimera Live which provides near real-time processing of multibeam data regardless of platform and data format. Beginning in 2017, QPS has responded to market demands by providing fully automated functionality specific to several key ASV manufacturers, including but not limited to; SeaRobotics, ASVGlobal, Seafloor Systems, IxBlue, Maritime Robotics. Across all of these platforms QPS provides a tightly integrated Five-Point solution which includes:

Simplifying Planning and Ensuring 100% coverage

A common challenge amongst autonomous operators is the ability to ensure 100% survey coverage in regions of varied slope. This challenge is exacerbated by lack or complete absence of historical bathymetry data essentially making survey planning a futile exercise. In response, QPS introduced “AutoSurvey” (demonstrated at OI2018), a real-time intelligent line generation tool that utilizes the most recently collected multibeam data to generate a line that accounts for variation in slope (Fig 1). Unlike prior solutions, the success of QINSy’s AutoSurvey is the result of having highly reliable and fully corrected real-time multibeam data to generate a line that will prevent gaps or “holidays” in the data by providing course corrections to follow along bathymetric contours. Another unique feature is that AutoSurvey is tightly integrated with QINSy’s Autopilot (serial or network) driver to provide accurate navigation data for vessel control. The “turning track” may also be customized to ensure proper lead-in to the survey coverage and/or to ensure proper positioning of a towed object relative to the vessel. Line planning is now reduced to a single line as a starting point to begin the autonomous survey.

While tight integration between line planning and contoured guidance is an idealized real-time scenario, several operators utilize “way point” navigation for their systems. To ensure all requirements for navigation are met, QPS provides direct integration with Control software and direct import of L84 formatted line plans. Direct integration includes driver interfaces with autopilot systems, such as the AutoNav MavLink that is utilized in manned and unmanned maritime, aerial and terrestrial operations. If issues arise due to interface complications or users wish to utilize a pre-planned mission, QINSy line plans may be exported via L84 and uploaded directly to the Control system software (e.g. ASView software by ASV Global).

Despite extensive planning, one must expect the unexpected: storms arise, sensors fail, collisions occur, power is lost. QPS is in the process of implementing “Murphy” Planning which integrates with QINSy’s on-line systems monitoring to provide system recovery options. Murphy planning is simple and straight forward but requires real-time logical reasoning by the software itself and taps into QINSy’s innovations in automation to ensure proper execution.

Innovations in Automation

A mundane truth about hydrographic surveying is that the majority of the time is spent behind a computer initiating start/stop logging sequences and configuring file management. QINSy provides several options that may be preconfigured to automate both processes. Auto-logging functionality may be initiated based on distance to start/end of line, distance from swath coverage or distance to predefined survey area. QPS implements a file structure that is automatically configured during installation. Raw data (.dB & .xtf) and processed (.qpd) file names can then be defined based on a user defined name, date (julian day, month-day-year, year-month-day, etc), time or a combination of all options in order to fit the most complex client requirements. Furthermore, eventing and/or fixing (dynamic or passive) may be automatically generated based on time, trigger, distance (sailed, along, seismic, preload). All events and fixes are stored raw data files.

Further, QPS has removed the necessity to create a predefined grid area allowing for attributed grids (SBES, MBES, Sidescan and magnetometer) and dynamic surfaces (DTM-based related to processed qpd files). Users also have the option to include ancillary data files (e.g. sound velocity) in the real-time processing such that the information contained in the resulting grids and processed point files are fully refracted and corrected. If the ASV has the ability for continuous sound velocity profiling, raytracing corrections can be implemented based on the most recently available profile in QINSy. It should be noted that Qimera has several additional raytracing options for concurrent processing including nearest in time and nearest in distance.

Systems Control

The overall purpose of unmanned versus manned surveys is to provide fully automated processes with minimal human interaction. However, it is recognized that operators need confidence in their ability to have the system self-monitor and react, be customized based on variable payloads, remotely interact with the 3rd party control systems (if needed) and provide the option to manage data from multiple sources and vessels simultaneously. QINSy currently provides over 2000 interface drivers, making it one of the most flexible and readily deployable software solutions on the market. An emerging trait of success for autonomous systems is the ability to rapidly change payloads based on mission requirements. Since these requirements may take place “on the fly”, customizations may be required to accommodate varying sensor inputs and outputs. QINSy satisfies both requirements through a generic user defined ASCII driver which is often used to send out basic ASCII string (serial, UDP, TCP/IP) to monitor ASV position or to control other devices.

Monitoring your System

In response to market demands for low graphics load monitoring options, QINSy provides two options; a TCP server based Remote Controller and the Remote Display Client. Remote controller is used to send commands over a network to start/stop logging or to shut down QINSy. This functionality also provides the operator the ability to set or redefine the file naming convention and obtain an overall status of the I/O as well as logging (storage, memory, etc.). The communication can take place internally with an AUV/ASV controller and/or externally to provide remote operator access. For those operators focused more on the later, the Remote Display Client (RDC) (Fig 2) provides the ability to receive settings (computations, current line, steered node, etc.), raw data and positioning results over a TCP/IP network from a Socket Server running under the on-board system. The RDC can set up independent displays (navigation, swath, alarms, etc.) and, if permissions are set, remotely change online settings. Furthermore, QPS provides sonar controllers for several major manufacturers including R2Sonic, Kongsberg and Edgetech. While most of these systems have fully automated sonar control functionality already incorporated, QINSy RDC provides the option for settings to be monitored and changed if necessary.

Regardless of the chosen monitoring option, the QPS alerting system has the ability to monitor all aspects of the integrated system and trigger alarms if acceptable tolerances have been exceeded. Of notable importance is seabed collision avoidance, a critical feature for manned and unmanned surveys. Collision avoidance is set up prior to survey and is based on current 3D position and speed over ground. When the seabed is at the same depth or shallower than the defined object (e.g. ASV, AUV or towfish) and within the maximum amount of time, an alarm is generated. The defined seabed may be a model, active grid or real-time observation (Fig 3).

Force Multiplier

Force multiplier technology is another significant advancement in autonomous operations. For Hydrography, running in parallel with other autonomous vehicles and/or alongside manned survey vessels, operators are covering far greater ground while minimizing logistical overhead. QPS is committed to improving the Force Multiplier concept through advances in both real-time data acquisition and processing as well as during post-processing in Qimera. For real-time, QPS is developing technology that will allow for scalable approaches for multiple systems to populate a single grid in real-time. Unique to Qimera, Cooperative Cleaning and Production Line Processing functionality provide seamless workflows highly relevant for the post-production and validation portion of the autonomous workflow.

Cooperative Cleaning — Making the unmanageable manageable

For large data sets, cooperative cleaning allows multiple users to clean and validate large survey projects by dividing into smaller, more manageable projects while maintaining data integrity. This allows multiple processors to work on the same project simultaneously, greatly increasing efficiency. Because the subprojects are entirely encapsulated (i.e. they can exist independently from the main project), further efficiency can be gained by creating the subprojects locally, which therefore reduces or completely eliminates the near constant network transactions that are normally required when point editing. Data processors work within their subprojects and, once complete, introduce their efforts back into the main project. In parallel, the main project may have ancillary data processing completed (e.g., SBET, SVP, height corrections). The edits from the subprojects may be incorporated back into the main project without affecting the overall progress of the entire project but enormously increasing efficiency (Fig 4).

Production Line Processing — Unifying separate projects for a common goal

Production-line processing allows for multiple projects to be treated as stages and encapsulated under a single, much larger effort. These stages can be done based on survey days, survey segments, survey vessels or, in this case, autonomous systems. The processing for a stage (e.g., a day, vessel/vehicle, segment) of data is handled in its own processing project. Further, if real-time processed data acquired by QINSy is present, these outputs can be aggregated into a master project for review. During review, the stage may be accepted and immediately incorporated into the master project or sent back for additional processing. This is done repeatedly and combined with other stages within the master project. The net result is the integration of multiple smaller projects processed in the exact same manner that can then be seamless integrated into the QPS hydrographic workflow.

Integrating into an already Seamless Workflow

Despite the presence of automated real-time solutions, tedious and manual processes continue to persist in many of the post-acquisition steps of many hydrographic workflows. As previously outlined, human interaction in the workflow is a highly prone to error (Wilson, et al, 2018). These errors are also quite costly, as they have been proven to dramatically increase the time required for data throughput to chart. Qimera is designed to provide a higher return on investment by minimizing the opportunities for human operators to make mistakes. This philosophy stands true whether the data is collected from a manned or unmanned system.

Real-time processing as outlined above is often referred to as “pre-processing” where all of the mathematical operations for data fusion, filtering and surface generation have been completed on-board prior to ASV/AUV retrieval. The unique integration between QINSy and Qimera provides a rapid linkage for final quality checks or to revisit the raw data to trouble-shoot, utilize Qimera’s problem solving tools or to reprocess all-together (Fig. 5). Care has been taken to simplify reprocessing by linking the proper metadata (including operations performed) to the raw (regardless of data format) and the processed data.

For non-QINSy users, QPS introduced an automated stand-alone solution called Qimera Live. With a single command, the “Live” functionality automates the guided workflow such that by the completion of every survey line, pre-flagged data files create and/or update a fully georeferenced gridded surface. The surveyor may choose to use Qimera Live to immediately identify problems in post-processing or have a completely autonomous processing solution that can deliver a final product upon hitting the quayside. Qimera Live is a part of the standard Qimera installation and is maintained through the standard release cycle. It may run on a separate computer or on the acquisition machine, allowing maximum flexibility for implementation in autonomous operations, remote vessel operations and/or multivessel surveys.

Once within Qimera, there are several QC tools available to either move towards product deliverable or fix any problems that may be present. Three of these tools are the implementation of many years of academic research to address common problems found in multibeam data; Wobble test, ENC Plus and TU Delft SV Correction. The TU Delft SV Correction tool is particularly important for autonomous surveys as relevant and reliable sound velocity profile information is often not available. This tool fixes data errors caused by sound velocity issues that are not fixable within an available set of SVP measurements. This physics-based approach takes advantage of the overlap between survey lines, harnessing the power of redundancy of the multiple observations. For a given set of pings, the algorithm simultaneously estimates sound speed corrections for the chosen pings and their neighbors by computing a best-fit solution that minimizes the mismatch in the areas of overlap between lines. This process is repeated across the entire spatial area, allowing for an adaptive solution that responds to changes in oceanographic conditions. For accountability, the algorithm also preserves the output of the inversion process for review, vetting, adjustment and reporting (Fig 6).

Artifacts associated with autonomous installations may also be an issue for the new era hydrographer. To correct these issues during post-acquisition, the Wobble Test is designed to dynamically adjust the echosounder and/or motion sensor configuration in a trial-and-error method to isolate causes of integration errors resulting in so-called “wobbles” in the sounding footprints. Finally, to comply with the deliverable requirements for Hydrographic Offices, the ENC Plus tool in Qimera features a built-in linkage between S57 features and the corresponding bathymetry (Fig 7) to ensure parity between these data submissions.

Further production can be done through QPS’s “one button” mosaic tool in FM Geocoder or utilizing Qimera’s built-in water column processing tools. Both tools have been successfully deployed for hydrographic based projects ranging from substrate identification with least depth determination to seep hunting activities. All water column detections processed utilizing Qimera are fully corrected and raytraced requiring very little interaction from the user aside from identifying what it is they wish to add to the processed data files (Fig 8).

Summary

Since its entry into the hydrographic market in 1997, QPS has strived for minimizing error due to human interaction by automating the mundane tasks which can best be achieved through computing automation. A byproduct of this early innovation is the foundation for QINSy to become a comprehensive autonomous solution suitable for both subsea and surface applications. It’s real-time processing functionality paved the way for new innovations that simplify in mission planning and provide dynamic line guidance tools that ensure 100% survey coverage. Since the data processed during survey are fully corrected and refracted, the positions on the seabed can be utilized to improve systems control and monitoring through QPS’s Collision avoidance tools. QINSy is also highly flexible tool to implement into existing systems through a wide range of systems drivers, customizable generic drivers and options to interface directly to systems control software (remote control). This flexibility is scalable allowing for users a wide range of options to fully control the system remotely (RDC) or, when something is wrong, have the system alert the user and allow them to respond (e.g. the “Murphy” plan). This scalability is now allowing for multiple vessels and/or vehicles to be incorporated into a single project. These Force Multipliers are expanding the concept of autonomous hydrography and Qimera provides the unique functionality to either break up large projects into manageable sections or to consolidate several small projects (e.g. multiple vehicles/vessels) into a common space. The seamless integration between QINSy and Qimera is providing options for users to simply QA/QC data at the end of the day or tap into the most recent innovations in post processing to identify problems, correct them, reprocess and confidently deliver.

In April 2017, CARIS Onboard™ Product Manager, Travis Hamilton, was onsite with the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) to conduct a CARIS Onboard installation and trial. JAMSTEC is an organization committed to supporting ocean research and development activities which covers a broad spectrum of marine science and technology. With several institutions located across the country, their mandate is to contribute to the creation of a holistic understanding of global concepts surrounding the ocean.

With tidal ranges up to 14m and extremely high sediment loads Bristol Port was an ideal location to test Valeport’s new SWiFTplus profiler with an integrated optical sensor, in this particular case turbidity.