Lukáš Rustler presents Active Visuo-Haptic Object Shape Completion

On 2022-06-09 11:00:00 at G205, Karlovo náměstí 13, Praha 2
During the seminar, we will present our work on active visuo-haptic object
shape completion described in this publication:

Rustler, L., Lundell, J., Behrens, J. K., Kyrki, V., & Hoffmann, M. (2022).
'Active Visuo-Haptic Object Shape Completion'. IEEE Robotics and Automation
Letters 7 (2), 5254-5261.

Recent advancements in object shape completion have enabled impressive object
reconstructions using only visual input. However, due to self-occlusion, the
reconstructions have high uncertainty in the occluded object parts, which
negatively impacts the performance of downstream robotic tasks such as
In this work, we propose an active visuo-haptic shape completion method called
Act-VH that actively computes where to touch the objects based on the
reconstruction uncertainty. Act-VH reconstructs objects from point clouds and
calculates the reconstruction uncertainty using IGR, a recent state-of-the-art
implicit surface deep neural network. We experimentally evaluate the
reconstruction accuracy of Act-VH against five baselines in simulation and in
the real world. We also propose a new simulation environment for this purpose.
The results show that Act-VH outperforms all baselines and that an
uncertainty-driven haptic exploration policy leads to higher reconstruction
accuracy than a random policy and a policy driven by Gaussian Process Implicit
Surfaces. As a final experiment, we evaluate Act-VH and the best reconstruction
baseline on grasping 10 novel objects. The results show that Act-VH reaches a
significantly higher grasp success rate than the baseline on all objects.
Together, this work opens up the door for using active visuo-haptic shape
completion in more complex cluttered scenes.

complementary video:
5-min. video presentation:
code and data:

The seminar will be a short one (10-20 min).
Responsible person: Petr Pošík