The broader impact of this Small Business Innovation Research (SBIR) Phase I project is to advance the capability of current robots to autonomously engage with the household environment, recognize objects, and manipulate them in a purposeful way. The proposed methodology adds this capability to a wide range of household and entertainment robots. A user can teach a robot a wide range of autonomous behaviors out of reach of the previously installed baseline software packages. The proposed technology radically lowers the threshold for teaching a robot sophisticated behaviors without complex programming nor high-quality demonstrations. This enables a class of new applications beyond the simple repetition of industrial activities. This Small Business Innovation Research (SBIR) Phase I project leverages a deep neural network-based computer vision system that creates a lower-dimensional representation of the world-view of the robot. On this latent encoding, the robot behavior is trained using a combination of optimization criteria (e.g. training loss) that ensures the generalization of the target task, smooth operation, recovery from accidental mistakes, and ability to choose between alternative solutions to the problem (e.g. avoid an obstacle to the left or to the right). The technology is novel: current learning by demonstration systems usually reproduce an identical trajectory to what has been demonstrated. This might be useful on a factory floor but would not work in a home environment. In contrast, the proposed model learns a generalizable behavior that uses demonstrations to obtain clues as to how to solve a particular manipulation problem and can both identify and learn from its own mistakes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
