171-6 Advancing Analysis of 2D and 3D Scenes Using Deep Learning and Physics-engines

Session: Advancing Geologic Analysis with Digital Outcrops and Close-Range Remote Sensing Data

Presenting Author:

Authors:

Abstract:

Major advances have come in the last decade from the widespread access to and deployment of close-range imaging (UAV photogrammetry and lidar) which enable the construction of spectacular 3D models. A major challenge, however, lies in identifying features of interest (semantic segmentation) and analyzing their dynamics with validated, physics-based tools. 

We have applied deep learning tools to segment--at scale--imagery and point clouds to extract and characterize features of interest. Specifically, we have segmented ~105 rocks in a UAV orthomosaic to understand the geomorphic development of rocky fault scarps. 3D rock extraction has been applied to study precariously balanced rocks. Starting with the 2D extraction to cut the point cloud containing candidate rocks, we develop 3D segmentation algorithms to separate the individual rocks from their pedestal. The resulting extracted point cloud measures the 3D shape of each boulder. Their height-width ratio and the minimum angle between their center of mass and rocking points indicates their fragility (likelihood of toppling caused by earthquake ground motions). The most fragile of these boulders is then analyzed using the Virtual Shake Robot. It deploys a high performance physics engine to compute ground motion thresholds and large displacement motions of the objects due to input shaking, which helps us understand complex rock behaviors such as sliding, twisting, rotating, toppling and their relationship to earthquake ground motion. 

To advance geological analysis of 3D imagery and structure such as discussed here, segmentation and physics-based interpretations need further development. Segmentation and physics-engine-based tools are rapidly developing thanks to industry and commercial motivators. Adapting segmentation to scientific analysis is difficult because of steep learning curves, skepticism due to opaque deep learning, and how much training they need. Physics engines allow for complex geometries and process simulation. These tools also are demanding to master, and require validation (especially for complex rheologies beyond rigid body dynamics). Finally, a key tenet in science is the reduction of complex problems to their essential constituents. While deep learning and physics engines allow us to work with complex features, will our understanding advance sufficiently? 

Geological Society of America Abstracts with Program. Vol. 57, No. 6, 2025

doi: 10.1130/abs/2025AM-8950

© Copyright 2025 The Geological Society of America (GSA), all rights reserved.