Arash Amjadi presents Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task

On 2022-02-03 11:00:00 at
We address a core problem of computer vision: Detection and description of 2D
feature points for image matching. For a long time, hand-crafted designs, like
the seminal SIFT algorithm, were unsurpassed in accuracy and
efficiency. Recently, learned feature detectors emerged
that implement detection and description using neural networks. Training these
networks usually resorts to optimizing low-level matching scores, often
pre-defining sets of image patches which should or should not match, or which
should or should not contain key points. Unfortunately, increased accuracy for
these low-level matching scores does
not necessarily translate to better performance in high-level
vision tasks. We propose a new training methodology which
embeds the feature detector in a complete vision pipeline,
and where the learnable parameters are trained in an endto-end fashion. We
overcome the discrete nature of key
point selection and descriptor matching using principles
from reinforcement learning. As an example, we address
the task of relative pose estimation between a pair of images. We demonstrate
that the accuracy of a state-of-theart learning-based feature detector can be
increased when trained for the task it is supposed to solve at test time. Our
training methodology poses little restrictions on the task to
learn, and works for any architecture which predicts key
point heat maps, and descriptors for key point locations.

Published at CVPR 2020

See the page of reading groups
Responsible person: Petr Pošík