Converting Depth Images and Point Clouds for Feature-based Pose Estimation

Robert Lösch1, Mark Sastuba2, Jonas Toth1, Bernhard Jung1,
1Technical University Bergakademie Freiberg, 2German Centre for Rail Traffic Research at the Federal Railway Authority

IROS 2023

Paper arXiv Video Code
Examples Supplementary Material
This is an animated gif image, but it does not move

Flexion images convert depth images into a rotation invariant, more descriptive format and enables them to be used in traditional computer vision algorithms like feature matching.

Abstract

In recent years, depth sensors became more and more affordable and found their way into a growing amount of robotic systems. However, mono- or multi-modal sensor registration, often a necessary step for further processing, faces many challenges on raw depth images or point clouds.

This paper presents a method of converting depth data into images capable of visualizing spatial details that are basically hidden in traditional depth images. After noise removal, a neighborhood of points forms two normal vectors whose difference is encoded into this new conversion. Compared to Bearing Angle images, our method yields brighter, higher-contrast images with more visible contours and more details.

We tested feature-based pose estimation of both conversions in a visual odometry task and RGB-D SLAM. For all tested features, AKAZE, ORB, SIFT, and SURF, our new Flexion images yield better results than Bearing Angle images and show great potential to bridge the gap between depth data and classical computer vision.

Video

Converting synthetic and real world depth data into Flexion images.

Synthetic Scene with distinct objects

Fine details that are basically hidden in the depth image become visible when converted to Flexion images.

“Multi-FoV” synthetic data set

The conversion to Flexion images enables a camera path in a synthetic city to be estimated solely based on depth information.

Real world RGB-D data

Depth images of the TUM RGB-D dataset can also be used to perform feature based odometry or SLAM. Depicted is a frame of the sequence fr1/desk.

Terrestrial Laser Scan

Dense point clouds, e.g., produced by terrestrial laser scanners, can easily be depicted as depth images. Therefore, they can also be converted to Flexion images and thus can be used for feature-based computer vision algorithms.

Variants

Changing the neighborhood size

By increasing the distance to the neighboring points used to calculate normal vectors, the Flexion images slightly change. The larger the neighborhood, the thicker the edges and the less noise but also less details the images contain.

Interpolate start reference image.

Flexion3x3

Loading...
Interpolation end reference image.

Flexion13x13


BibTeX


@inproceedings{Losch2023,
  author    = {L{\"o}sch, Robert and Sastuba, Mark and Toth, Jonas and Jung, Bernhard},
  title     = {Converting Depth Images and Point Clouds for Feature-based Pose Estimation},
  booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2023},
  pages     = {3422-3428},
  doi={10.1109/IROS55552.2023.10341758}
}