Robot Multi-Modal Object Perception and Recognition: Synthetic Maturation of Sensorimotor Learning in Embodied Systems

Raphael Braud, Alexandros Giagkos, Patricia Shaw, Mark Lee, Qiang Shen

Research output: Contribution to journalArticlepeer-review

4 Citations (SciVal)
21 Downloads (Pure)


It is known that during early infancy, humans experience many physical and cognitive changes that shape their learning and refine their understanding of objects in the world. With the extended arm being one of the very first objects they familiarize, infants undergo a series of developmental stages that progressively facilitate physical interactions, enrich sensory information, and develop the skills to learn and recognize. Drawing inspiration from infancy, this article deals with the modeling of an open-ended learning mechanism for embodied agents that considers the cumulative and increasing complexity of physical interactions with the world. The proposed system achieves object perception and recognition as the agent (i.e., a humanoid robot) matures, experiences changes to its visual capabilities, develops sensorimotor control, and interacts with objects within its reach. The reported findings demonstrate the critical role of developing vision on the effectiveness of object learning and recognition and the importance of reaching and grasping in solving visually elicited ambiguities. Impediments caused by the interdependency of parallel components responsible for the agent's physical and cognitive functionalities are exposed, demonstrating an interesting phase transition in utilizing object perceptions for recognition.

Original languageEnglish
Article number8957396
Pages (from-to)416-428
Number of pages13
JournalIEEE Transactions on Cognitive and Developmental Systems
Issue number2
Early online date13 Jan 2020
Publication statusPublished - 10 Jun 2021


  • Developmental learning
  • iCub robot
  • longitudinal study
  • multimodal object learning
  • reaching
  • vision


Dive into the research topics of 'Robot Multi-Modal Object Perception and Recognition: Synthetic Maturation of Sensorimotor Learning in Embodied Systems'. Together they form a unique fingerprint.

Cite this