> Academic opportunities > PHD positions

Dexterous gesture analysis methodology ? Applications in Virtual Reality, haptics and robotics

Technological challenge: Factory of the future incl. robotics and non destructive testing (learn more)

Department: Département Intelligence Ambiante et Systèmes Interactifs (LIST)

Laboratory: Laboratoire de Simulation Interactive

Start Date: 01-09-2022

Location: Saclay

CEA Code: SL-DRT-22-0687

Contact: florian.gosselin@cea.fr

Virtual Reality based digital twins have become a reference tool for the design of new products and systems. The allow an early detection and correction of errors thanks to the real time and immersive simulation of their production, assembly and use, hence reduced development time and costs. Beyond industry, their use also impacts medical training, marketing, videogames or social interactions (e.g. Metaverse). A good immersion requires however well adapted techniques and peripherals. If nowadays HMDs and 3D audio devices allow a significantly increased visual and audio immersion, dexterous manual interfaces with high quality distributed haptic feedback are still unavailable today: RGBD cameras and datagloves do not allow any haptic feedback while current haptic gloves remain heavy and bulky. To solve this issue, it is necessary to simplify their design, and therefore to focus only on the most solicited hand areas and those allowing the richer interactions. The control of the hand avatars suffers from similar limitations: efficient real time hand tracking remains difficult (e.g. occlusions, intrusive devices) and requires grasping models which are still difficult to develop due to the complexity of the underlying physical phenomena. An alternative is to use pre-registered hand movements which can be selected and adapted on the fly. They must however remain in limited number to allow for a reliable real-time recognition, and their characteristics must be chosen with care so that they still allow rich interactions. To answer these requirements, CEA LIST has developed an original methodology allowing an optimal specification of the areas of interest to be considered on the hand. This tool allows generating interaction maps and trees gathering information on the most solicited hand areas and on the diversity of the interactions rendered possible as a function of the involved hand areas. This thesis aims at enriching this methodology, with novel measurement techniques (e.g. posture and force sensors, use of physically realistic hand avatars and task models) allowing to gather information on the manipulated objects, the type of movements, their amplitude, and the force on the different hand areas. Next, AI based recognition and classification techniques will be developed to automate the analysis of the data. Finally, novel visualization techniques will be required to present the results in a way they can be easily used for the specification of haptic gloves or VR hand avatars. The results could also be used for the specification of dexterous robotic grippers.

See all positions Download the offer (.zip)

Email Bookmark and Share