Suzanne Sorli, Dan Casas, Mickeal Verschoor, Ana Tajadura Jiménez and Miguel A. Otaduy
Proc. of the International Symposium on Mixed and Augmented Reality (ISMAR), 2021 (pdf/website)
Abstract: Natural interaction with virtual objects relies on two major technology components: hand tracking and hand-object physics simulation. There are functional solutions for these two components, but their hand representations may differ in size and skeletal morphology, hence making the connection non-trivial. In this paper, we introduce a pose retargeting strategy to connect the tracked and simulated hand representations, and we have formulated and solved this hand retargeting as an optimization problem. We have also carried out a user study that demonstrates the effectiveness of our approach to enable fine manipulations that are slow and awkward with naïve approaches.