Bathymetry from UAV imagery and machine learning

Accurate bathymetric mapping of shallow areas is essential for many offshore activities. Through Structure from Motion (SfM) and Multiview Stereo (MVS) techniques, images can provide a low-cost alternative compared to Lidar and sonar systems while offering important visual information. This article presents a new framework for mapping the seabed in clear and calm shallow waters for small- and large-scale surveys using aerial imagery and machine learning to correct the geometric effects of refraction on the 3D point clouds and the imagery.

SfM is a photogrammetric technique for estimating 3D structures from 2D image sequences. It involves acquiring images from several positions relative to the object of interest. To establish the spatial relationships between the camera positions, features appearing in multiple images are identified automatically. Then, a self-calibrating bundle adjustment is used to calibrate the cameras and derive a sparse 3D point cloud to represent the object. External geometric constraints can be also applied to transform the 3D data to a desired coordinate system. MVS techniques are used to generate a very high-resolution 3D point cloud. MVS algorithms can construct highly detailed depth maps and consequently 3D point clouds that exploit the stereo correspondence of more than two images. Most commercial photogrammetric software currently delivers 3D point clouds by combining these two techniques.

Image-based bathymetric mapping

Compared to onshore aerial mapping, bathymetry mapping from aerial platforms in shallow waters is considered a much more time-consuming process. This is mainly due to the necessary compensation of refraction effects, which affect the geometry and radiometry of the primary data and consequently of the results by delivering apparent depths, as aerial imagery depicting the bottom of water bodies is heavily affected by the refraction of the optical rays. Refraction acts on these images similarly to the radial distortion, differing practically at each pixel of every image, leading to unstable solutions and erroneous depths. More specifically: according to Snell’s law, the refraction of a light beam is affected by water depth and the angle of incidence of the beam in the air/water interface. The problem becomes even more complex when multiview geometry is applied, as the 3D position is derived from multiple images and thus from multiple refraction sources.

In shallower, clear water areas, and when refraction effects are successfully treated, bathymetry mapping from aerial platforms is a more efficient operation than ship-borne echo-sounding or underwater photogrammetric methods. A very important additional feature of image-based seabed mapping is that a permanent record of other features is obtained in the coastal region, such as approximate tidal levels from the waterline markings showing in the images, coastal dunes, benthic communities, marine litter, rock platforms, and beach erosion when results are compared with archival data. These benefits are especially evident in the coastal zones of up to 15–20m depth, in which most economic activities are concentrated.

The presented framework developed by our team at the National Technical University of Athens and the Cyprus University of Technology corrects the refraction effect on the 3D point clouds and the aerial images of the bottom in two complementary modules. The first module corrects the apparent depths of the 3D points generated by the SfM-MVS methods and can also be used stand-alone. In that case, only a 3D point cloud, a 3D mesh, or their derivates such as depth contours, result. The second module transfers this correction to the image space, correcting the effects of refraction on the aerial imagery. The restriction on the framework is that there should be a calm water surface, non-turbid waters, and a textured seabed. More details for each module of the framework are given below.

Correcting the apparent depths of 3D point clouds

To correct the apparent depths of 3D point clouds of the bottom of water bodies which are derived by SfM-MVS processing of low-altitude aerial imagery, a linear support vector regression model was trained. In machine learning, support vector machines are supervised learning models that analyse data used for classification and regression analysis. This model was trained on synthetic and real-world data to learn to predict the correct depths, knowing only the apparent (erroneous) depths derived from the photogrammetric process. The model can generalize over different UAV systems, cameras and flying heights, regardless of the area to be mapped. This module delivers 3D point clouds of the seabed with the corrected depths.

Correcting the refraction effects

To correct the geometric effects of refraction on the low-altitude aerial imagery, a state-of-the-art image correction method has been developed which builds on and exceeds earlier methods developed by our team. This module follows the first one and corrects the refraction effect on the original imaging dataset. This correction can be briefly described as a sophisticated reduction of the radial distance of each pixel, according to the depth and the incidence angle of the beam in the air/water interface. This module delivers refraction-free images that can be used directly with photogrammetric SfM and MVS processing pipelines, resulting in highly accurate bathymetric maps and image-based products.

Results in real-world applications

The method was tested through experimental results and validation over three synthetic and five different real-world cases in Greece and Cyprus with flying altitudes ranging from 35–210m. To evaluate the results of the framework, the initial (uncorrected) 3D point clouds of the SfM-MVS procedure and the point clouds resulting from the presented framework were compared with bathymetric Lidar point clouds and total station measurements available for the areas. An example is given where the depth differences between the Lidar data and the uncorrected image-based point clouds are presented. This comparison makes clear that the refraction effect cannot be ignored in such applications. In the presented case, the Gaussian mean of all the differences is significant, reaching 2.23m (RMSE 2.28m) in the test site. Since these values might be considered negligible in some applications, it is important to stress that more than 30% of the compared uncorrected image-based depths represent a difference of 3.00–6.07m from the Lidar points, or 20–41.1% of the real depth.

By applying the presented framework and correcting the refraction effects, a more than 97.3% reduction in the remaining vertical RMSE of the depths of all the sites was achieved. In some cases, the initial vertical RMSE of 3.34m, mean of 2.96m and standard deviation of 1.54m of the uncorrected data were reduced to 0.09m, 0.02m and 0.09m respectively after the refraction correction, while in other cases the initial vertical RMSE of 2.28m, mean of 1.71m and standard deviation of 1.18m were reduced to 0.13m, 0.04m and 0.12m respectively. In the same cases, the most popular state-of-the-art solution achieved a reduction of the vertical RMSE to 0.52m and 0.45m respectively, indicating that the new framework presented here outperforms current methods. By examining specific points instead of all the test sites, it was observed that, in some cases, the initial vertical difference from the Lidar data of 5.10m was reduced to 0.05m at the depth of 14.20m.

It is also important to note that the second module achieves a reduction in the noise of the sparse point clouds, which resulted from the SfM process and improved the accuracy and the quality of the resulting orthoimages and textures. This module also increased the resulting accuracy of the state-of-the-art method fivefold, extending the operating depths from 7.50m to 15–20m.

Conclusion

The framework developed and presented prove that through-water photogrammetry can deliver accurate bathymetric information in clear and calm waters and with textured seabeds. What is required is that refraction is addressed thoroughly. Contrary to the current state-of-the-art methods, the presented framework is designed to achieve high generalization, expanding the applicability of and boosting SfM-MVS techniques for aerial bottom mapping. They deliver highly accurate and detailed bathymetric maps and image products, satisfying the International Hydrographic Organization’s S44 Special Order TVU of 0.29m at 20m depth at the 95% confidence level. The implementation of the framework on UAV images facilitates the detailed and accurate monitoring and mapping of the sensitive coastal area. Moreover, it supports low-cost mapping strategies and enables a wide usage of UAV imagery in shallow clear and calm waters instead of more expensive Lidar and sonar systems, increasing the amount of information. 

Further reading

• Correcting Image Refraction: Towards Accurate Aerial Image-Based Bathymetry Mapping in Shallow Waters: mdpi.com/2072-4292/12/2/322
• DepthLearn: Learning to Correct the Refraction on Point Clouds Derived from Aerial Imagery for Accurate Dense Shallow Water Bathymetry Based on SVMs - Fusion with LiDAR Point Clouds. mdpi.com/2072-4292/11/19/2225
• Shallow Water Bathymetry Mapping from UAV Imagery Based on Machine Learning. https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W10/9/2019/
• A Novel Iterative Water Refraction Correction Algorithm for Use in Structure from Motion Photogrammetric Pipeline: mdpi.com/2077-1312/6/3/77