Local Readjustment for High-Resolution 3D Reconstruction

Taking SfM points and camera poses as inputs, we first decompose the points into segments, and then re-optimize each individual
segment and its corresponding local cameras. This significantly improves the reconstruction quality for fine geometry details.

Abstract

Global bundle adjustment usually converges to a nonzero
residual and produces sub-optimal camera poses
for local areas, which leads to loss of details for highresolution
reconstruction. Instead of trying harder to optimize
everything globally, we argue that we should live with
the non-zero residual and adapt the camera poses to local
areas. To this end, we propose a segment-based approach
to readjust the camera poses locally and improve the reconstruction
for fine geometry details. The key idea is to partition
the globally optimized structure from motion points into
well-conditioned segments for re-optimization, reconstruct
their geometry individually, and fuse everything back into
a consistent global model. This significantly reduces severe
propagated errors and estimation biases caused by the
initial global adjustment. The results on several datasets
demonstrate that this approach can significantly improve
the reconstruction accuracy, while maintaining the consistency
of the 3D structure between segments.