Learning by Observation through System Identification

In our previous work ((Nehmzow et al., 2007) and (Akanyeti et al., 2007b)), we present a new method to program mobile robots —“code generation by demonstration”— based on algorithmically translating human behaviours directly to robot control code using transparent mathematical functions. Our approach has three stages: i) first extracting the trajectory if the desired behaviour by observing the human, ii) making the robot follow the human trajectory blindly to log the robot’s own perception perceived along that trajectory, and finally iii) linking the robot’s perception to the desired behaviour to obtain a generalised, sensor-based model.
So far we used an external, camera based motion tracking system to log the trajectory of the human demonstrator during his initial demonstration of the desired motion. Because tracking systems are complicated to set up and expensive, we propose an alternative method to obtain trajectory information, using the robot’s own sensor perception.
To achieve this, we train a mathematical polynomial using the Narmax system identification methodology which maps the position of the “red jacket” worn by the demonstrator in the image captured by the robot’s camera, to the relative position of the demonstrator in the real world according to the robot.
We demonstrate the viability of this approach by teaching a Scitos G5 mobile robot to achieve door traversal behaviour