> Academic opportunities > PHD positions

A Multi-Task Neural Network for Real Time Object Detection and Tracking on Embedded Systems

Technological challenge: Artificial intelligence & Data intelligence (learn more)

Department: Département Systèmes et Circuits Intégrés Numériques (LIST)

Laboratory: Laboratoire Intelligence Artificielle Embarquée

Start Date: 01-10-2022

Location: Saclay

CEA Code: SL-DRT-22-0323

Contact: martyna.poreba@cea.fr

The proposed thesis addresses the perception in the embedded environment, in particular multi-class (people, cars, cyclists...) detection and tracking in real time. Object tracking is one of the key issues due to its wide scope of applications involving autonomous vehicles, robot navigation, medical imaging or video surveillance. Based on visual information from different sensors, tracking an object requires the combination of several tasks i.e. object detection, visual appearance modelling and/or motion prediction. Tracking approaches can be divided into three types: 1) SDE (Separate Detection and Embedding) separates detection and visual features (embedding) extraction; 2) JDE (Joint Detection and Embedding) is based on end-to-end learning of neural networks for detection and appearance extraction simultaneously; 3) JDT (Joint Detection and Tracking) extends the JDE approaches by co-similarity notion between objects. This type of algorithm can also propose a network for joint object detection and motion prediction. In recent years, significant progress has been made in single-task neural network designed for object detection, visual feature extraction or motion prediction. However, little attention has been devoted to the design of neural networks that aim to jointly process a set of related tasks. With such an architecture, it is possible to improve the performance and optimize the computation time during inference. This thesis project aims at exploring multi-task networks for JDE or JDT algorithm and their integration capacity on embedded systems with limited resources (power consumption and memory footprint). The LIAE laboratory holds mobile robotic platforms and an autonomous electric vehicle, the results of this thesis will be used to feed the on-board perception bricks of these systems.

See all positions Download the offer (.zip)

Email Bookmark and Share