Home Grants
Home Grants

Action understanding

Research Area: Computer Vision
Status: In progress  
Members: Francesca Odone, Sean Ryan Fanello, Nicoletta Noceti, Matteo Santoro  

The main goal is to assess the role of different cues (e.g. visual cues, motor cues, contextual information, object semantic) in the action understanding process. The research is organized in two main streams:

  • First we study regression models to the purpose of estimating appropriate grasps for the observed object, starting from visual cues. This mapping may be useful, not only for the very purpose of learning how to grasp objects, but also to exploit multi-modality in object classification. Indeed, multi-modality is a fundamental feature that characterizes biological systems and lets them achieve high robustness in understanding skills while coping with uncertainty. Relatively recent studies showed that multi-modal learning is a potentially effective add-on to artificial systems, allowing the transfer of information from one modality to another.
  • Second, we investigate the adoption of a string-based representation of motor information for analysing and classifying action patterns (Prevete et al., 2005; Prevete et al., 2006). Kernel-based algorithms may be used to evaluate the similarity between motor acts (see Pittore et al., 2000 and Noceti et al., 2008 for previous investigations and preliminary results). The contribution of such framework is to provide a tool for assessing the role of motor representation in action observation.

Ongoing collaborations:


  • Castellini, C. et al "Using object affordances to improve object recognition", IEEE Trans. on Autonomous Mental Development, 2011
  • Barla, A et al "Learning how to grasp objects". ESANN 2010
  • Noceti, N. et al. "Towards a theoretical framework for learning multi-modal patterns for embodied agents". IEEE Proceedings of ICIAP, 2009.
  • Prevete, R., M. Santoro and F. Mariotti. "A Biologically Inspired Visuo?Motor Control Model based on a Deflationary Interpretation of Mirror Neurons". Proceedings of The 27th Annual Conference of the Cognitive Science Society (COGSCI05). Bruno G. Bara, Lawrence Barsalou, & Monica Bucci, 2005. 1779-1784.
  • Prevete, R. et al. "Towards a Biologically Inspired Semantic Segmentation of Goal Oriented Actions". Proceedings Of The 28th Annual Conference Of The Cognitive Science Society (Cogsci06), 2006.