Return to Publications

Fine Virtual Manipulation with Hands of Different Sizes

Suzanne Sorli, Dan Casas, Mickeal Verschoor, Ana Tajadura Jiménez and Miguel A. Otaduy

Proc. of the International Symposium on Mixed and Augmented Reality (ISMAR), 2021 (pdf/website)

Abstract: Natural interaction with virtual objects relies on two major technology components: hand tracking and hand-object physics simulation. There are functional solutions for these two components, but their hand representations may differ in size and skeletal morphology, hence making the connection non-trivial. In this paper, we introduce a pose retargeting strategy to connect the tracked and simulated hand representations, and we have formulated and solved this hand retargeting as an optimization problem. We have also carried out a user study that demonstrates the effectiveness of our approach to enable fine manipulations that are slow and awkward with naïve approaches.